A distributed flow limiting implementation method and device of a cloud-native gateway
By using Redis clusters and ZSet data types in a cloud-native gateway, combined with sliding window and token bucket algorithms, distributed rate limiting was implemented in the cloud-native gateway. This solved the problems of accuracy in traffic control and resource utilization, ensuring the safe and efficient operation of upstream services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
- Filing Date
- 2023-05-11
- Publication Date
- 2026-07-14
AI Technical Summary
In high-availability, multi-instance scenarios of cloud-native gateways, existing rate limiting solutions struggle to achieve precise traffic control in distributed architectures, leading to issues such as instantaneous traffic exceeding limits and uneven request flow after changes in traffic control policies.
A Redis cluster is used as the data storage center for CAS rate limiting, and the ZSet data type is used as the resource type for rate limiting tokens. The sliding window algorithm and token bucket algorithm are combined to set the number of token buckets, generation rate and lifespan. The expired token buckets are automatically reclaimed through Redis's resource reclamation mechanism, and the count and limit are performed when API requests are made.
It achieves precise traffic limiting, solves the traffic limiting problem in a distributed multi-instance architecture, avoids resource waste, reduces the probability of anomalies, and ensures the safe operation of upstream services and high resource utilization.
Smart Images
Figure CN116527590B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer communication technology, specifically to a distributed rate limiting method and apparatus for cloud-native gateways. Background Technology
[0002] With the continuous development of cloud-native technologies, container technologies, represented by Kubernetes, are becoming a new interface for cloud computing.
[0003] In high-availability, multi-instance business scenarios, ensuring accurate distributed rate limiting becomes a fundamental issue for cloud-native gateway services. Common rate limiting solutions include frequency-based rate limiting based on the leaky bucket algorithm and total-volume rate limiting based on the token bucket algorithm, but each of these solutions leads to different problems. Leaky bucket-based rate limiting strategies require locking on every update in a distributed architecture to ensure the algorithm's traffic control effectiveness; token bucket-based rate limiting strategies can cause the total traffic volume before and after a token bucket reset to exceed the set number of rate limits per unit time.
[0004] Based on this, a distributed rate limiting method and device for cloud-native gateways were designed and developed to ensure accurate traffic limitation within a unit of time. Summary of the Invention
[0005] This invention addresses the needs and shortcomings of current technological development by providing a distributed rate limiting method and apparatus for cloud-native gateways.
[0006] First, the distributed rate limiting implementation method for a cloud-native gateway of the present invention addresses the above-mentioned technical problems by adopting the following technical solution:
[0007] A distributed rate limiting implementation method for cloud-native gateways, comprising the following operations:
[0008] (1) Deploy a Redis cluster in a Kubernetes cluster as the data storage center for the gateway's CAS rate limiting;
[0009] (2) Select the ZSet data type in Redis as the resource type for the rate limiting token;
[0010] (3) Configure rate limiting rules and set the number of token buckets and token generation rate based on a sliding window;
[0011] (4) Set a timeout for each token bucket and use Redis’s resource reclamation mechanism to automatically reclaim expired token buckets;
[0012] (5) Each token carries a globally unique key. The globally unique key carried by the token is set as the key of the ZSet to ensure the uniqueness of the API request;
[0013] (6) Use the generation sequence number of the token in each token bucket as the Value value, and use the API built into the ZSet data structure to perform the token consumption action when consuming tokens.
[0014] (7) Set rules to count and limit the traffic of API requests.
[0015] Optionally, execute (3) to configure rate limiting rules based on the sliding window algorithm and the token bucket algorithm, and create tokens through the leader instance in the distributed gateway;
[0016] When creating a token, the token generation rate r is confirmed in advance. Based on the rate limiting rule r = n / t, the rate limiting time unit t and the number of times n are allowed to pass through per unit time are calculated. At the same time, the default setting of the number of token buckets in the sliding window is 4. Then the maximum number of tokens in each token bucket is n / 4, the effective time of each token bucket is t / 4, and the lifespan of each token bucket is t / 4.
[0017] Once the token bucket is created, it will be automatically destroyed after t / 4 time units, regardless of whether all the tokens in the bucket have been used.
[0018] Optionally, execute (4) to ensure that the lifespan of each token bucket is consistent with the expiration date of the tokens within it;
[0019] When tokens in the token bucket expire, regardless of whether all tokens have been used up, the token bucket will be automatically reclaimed by Redis's own resource reclamation mechanism.
[0020] Optionally, the rules set in (7) are executed as follows:
[0021] Each time an API request is recorded, the corresponding token bucket is calculated based on the current request time. An attempt is made to obtain a token from the bucket by invalidating the key with the smallest score in the ZSet and calculating the number of invalidated tokens. If the smallest score key is successfully invalidated, it means that the current request has successfully obtained a token. If the number of invalidated tokens is 0, it means that there are no more tokens in the current token bucket, and the request is blocked and not allowed to pass.
[0022] Optionally, after executing (1)-(7), the rate limiting process of the distributed rate limiting implementation method is as follows:
[0023] After deploying a distributed gateway, the first gateway instance to start will create a gateway leader message in Redis. The default validity period of the message is T s. Once the first gateway instance starts writing the leader message, the remaining gateway instances will see that there is already a leader and will not create a leader message.
[0024] The gateway instance that writes the leader message is called the leader instance. The leader instance maintains the creation of the token bucket and the generation of tokens. The leader instance renews the leader message in Redis every T-1 seconds. The remaining gateway services check whether the leader message exists in Redis every T seconds. If it does not exist, it writes it. The gateway instance that writes the new leader message is promoted to the new leader instance.
[0025] After configuring the rate limiting rules, the token bucket within the defined time window is initialized first, and the remaining token buckets are filled one by one according to the rate limiting rate. As the old token buckets are discarded, the time window migration will include the newly generated token buckets in the token consumption range. During this period, the leader instance will continuously create tokens and pre-set the tokens into the latest token bucket.
[0026] After traffic enters the cluster, it is transferred to one of the gateway instances through load balancing. The gateway instance determines the rate limiting rule corresponding to the API to which the request is made based on the parameters carried in the request. According to the rate limiting rule, it obtains a token from the token bucket corresponding to the current time. When a token is successfully obtained, the approved request will be directly transferred to the corresponding service. Meanwhile, requests that do not obtain a token will be directly returned to the requester to protect the upstream service.
[0027] Secondly, the distributed rate limiting implementation device for a cloud-native gateway of the present invention solves the above-mentioned technical problems with the following technical solution:
[0028] A distributed rate limiting implementation device for a cloud-native gateway, comprising:
[0029] The deployment module is used to deploy a Redis cluster in a Kubernetes cluster, serving as the CAS rate limiting data storage center for the gateway;
[0030] Select the module to choose the ZSet data type in Redis as the resource type for the rate limiting token;
[0031] The configuration module is used to configure rate limiting rules, allowing you to set the number of token buckets and the token generation rate based on a sliding window;
[0032] The configuration module is used to set the time-to-live for each token bucket and to automatically reclaim expired token buckets using Redis's resource reclamation mechanism.
[0033] The configuration module is used to set the globally unique key carried by the token as the key of the ZSet to ensure the uniqueness of API requests;
[0034] The token processing module is used to take the generation sequence number of the token in each token bucket as the value, and to perform the token consumption action using the API built into the ZSet data structure when consuming tokens.
[0035] The rules setting module is used to set rules to count and limit the traffic of API requests.
[0036] Optionally, the configuration modules involved configure rate limiting rules based on the sliding window algorithm and the token bucket algorithm;
[0037] The configuration module calls the leader instance in the gateway to create a token;
[0038] When the leader instance creates a token, the configuration module needs to confirm the token generation rate r in advance. Based on the rate limiting rule r = n / t, the rate limiting time unit t and the number of times n can pass through per unit time are calculated. At the same time, the default setting of the number of token buckets in the sliding window is 4. Then the maximum number of tokens in each token bucket is n / 4, the effective time of each token bucket is t / 4, and the lifespan of each token bucket is t / 4.
[0039] Once the token bucket is created, it will be automatically destroyed after t / 4 time units, regardless of whether all the tokens in the bucket have been used.
[0040] Optionally, the time-to-live setting for each token bucket by the relevant settings calling module should be consistent with the validity period of the tokens within it;
[0041] When a token in the token bucket expires, regardless of whether all tokens have been consumed, the setting module will invoke Redis's own resource reclamation mechanism to automatically reclaim the expired token.
[0042] Optionally, the specific rules set in the relevant rule settings module are as follows:
[0043] Each time an API request is recorded, the corresponding token bucket is calculated based on the current request time. An attempt is made to obtain a token from the bucket by invalidating the key with the smallest score in the ZSet and calculating the number of invalidated tokens. If the smallest score key is successfully invalidated, it means that the current request has successfully obtained a token. If the number of invalidated tokens is 0, it means that there are no more tokens in the current token bucket, and the request is blocked and not allowed to pass.
[0044] The distributed rate limiting method and apparatus for cloud-native gateways of the present invention have the following advantages compared with the prior art:
[0045] (1) This invention uses a Redis cluster deployed within a Kubernetes cluster as the main implementation component for the rate limiting function, ensuring the cloud-native nature of the gateway rate limiting service and its high adaptability to the cluster environment, which meets the basic requirements of the current service ecosystem. By selecting the ZSet built into Redis as the data type for storing the rate limiting pool, and by using the built-in functions of the data type as much as possible during rate limiting operations, the complexity brought by custom data storage rules is reduced, the probability of exceptions is reduced, the robustness of the function is increased, and the QPS loss is reduced. By limiting the amount of data in each token bucket, the security of upstream services is protected to the maximum extent. By configuring rate limiting rules and pre-setting token buckets, token generation and token usage are decoupled, reducing additional operations during rate limiting and reducing QPS loss. By setting the validity period of resources, the invalid occupation of resources is avoided, achieving high resource utilization and a good resource recycling mechanism. The implementation of the sliding window algorithm in the rate limiting rules ensures that the number of API accesses is limited to the rate limiting rules set by the user within the rate limiting time window, achieving precise traffic limitation and protecting the safe operation of upstream services.
[0046] (2) This invention can solve the traffic limitation problem under the distributed multi-instance architecture of the gateway, solve the problem of instantaneous traffic exceeding the limit after the change of the rate limiting strategy, solve the problem of unsmooth requests under the rate limiting strategy, and achieve precise limitation of traffic peak within any unit time period. Attached Figure Description
[0047] Appendix Figure 1 This is a flowchart of the method according to Embodiment 1 of the present invention;
[0048] Appendix Figure 2 This is a module connection block diagram of Embodiment 2 of the present invention;
[0049] Appendix Figure 3 This is a logic diagram for implementing distributed rate limiting after executing the implementation method of Embodiment 1 or using the implementation device of Embodiment 2. Detailed Implementation
[0050] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.
[0051] Example 1:
[0052] Combined with appendix Figure 1 This embodiment proposes a distributed rate limiting implementation method for cloud-native gateways, which includes the following operations:
[0053] (1) Deploy a Redis cluster in a Kubernetes cluster as the CAS rate limiting data storage center for the gateway.
[0054] Redis (Remote Dictionary Server) is an open-source, network-enabled, in-memory or persistent, log-structured key-value database written in C, offering APIs for multiple languages. Compared to conventional local rate limiting, Redis can support higher concurrency levels, minimizing the additional latency caused by rate limiting for user requests.
[0055] (2) Select the ZSet data type in Redis as the resource type for the rate limiting token.
[0056] ZSet (sorted set) is an ordered list of string type elements. It stores data members with the characteristic that duplicates are not allowed. At the same time, each member element is associated with a score (scores can be repeated). Redis uses the scores to sort the members in the set from smallest to largest.
[0057] (3) Configure rate limiting rules and set the number of token buckets and token generation rate based on the sliding window.
[0058] Rate limiting rules are configured based on the sliding window algorithm and the token bucket algorithm, and tokens are created through the leader instance in the distributed gateway.
[0059] When creating a token, the token generation rate r is confirmed in advance. Based on the rate limiting rule r = n / t, the rate limiting time unit t and the number of times n are allowed to pass through per unit time are calculated. At the same time, the default setting of the number of token buckets in the sliding window is 4. Then the maximum number of tokens in each token bucket is n / 4, the effective time of each token bucket is t / 4, and the lifespan of each token bucket is t / 4.
[0060] Once the token bucket is created, it will be automatically destroyed after t / 4 time units, regardless of whether all tokens in the bucket have been used. Compared to the traditional token bucket algorithm, this embodiment combines a sliding window algorithm, which can make full use of newly generated tokens and optimize the token expiration mode. Based on Redis's automatic recycling of expired tokens, no additional resource reclamation logic is required.
[0061] (4) Set a timeout for each token bucket and use Redis’s resource recycling mechanism to automatically reclaim expired token buckets.
[0062] The lifespan of each token bucket is consistent with the expiration date of the tokens within it;
[0063] When tokens in the token bucket expire, regardless of whether all tokens have been consumed, the token bucket will be automatically reclaimed by Redis's own resource reclamation mechanism, ensuring the survival of valid tokens and the timely cleanup of invalid tokens. At the same time, by setting a lifespan, Redis's own resource reclamation mechanism can be used to reduce additional resource reclamation operations, reduce additional operation requests to Redis, and reduce memory burden. Compared with the conventional token bucket algorithm, it reduces redundant data, avoids waste of storage resources, improves resource utilization, and reduces long-term occupation of storage resources.
[0064] (5) Each token carries a globally unique key. The globally unique key carried by the token is set as the key of ZSet to ensure the uniqueness of each request in the rate limiting resource pool and to ensure that the tokens in the token bucket will not be consumed repeatedly. Compared with conventional rate limiting, this embodiment retains the globally unique key of the token. Through the globally unique key, it can be ensured that there will be no duplicate counting and other problems, thus ensuring the accuracy of the rate limiting function.
[0065] (6) Use the generation sequence number of the token in each token bucket as the Value value. When consuming tokens, use the API built into the ZSet data structure to perform the token consumption action.
[0066] Since each token in the token bucket has a globally unique key, and the generated keys are ranked according to their generation time, with later generated keys receiving higher scores, in order to ensure that the number of tokens in each token bucket complies with the rate limiting rules, the token key uses a UUID and each key also needs to carry its token sequence number in the token bucket, in the format: UUID + token sequence number.
[0067] Using the generation sequence number of the tokens in each token bucket as the value ensures that newly generated tokens in each token bucket have a higher score. At the same time, when consuming tokens, traffic prioritizes consuming older tokens with lower scores. In addition, using the token sequence number as the score makes it easy to sort the tokens, so that when consuming tokens, the built-in API of the ZSet data structure can be used to perform token consumption actions as easily as possible.
[0068] (7) Set rules to count and limit the traffic of API requests. The specific rules are as follows:
[0069] Each time an API request is recorded, the corresponding token bucket is calculated based on the current request time. An attempt is made to obtain a token from the bucket by invalidating the key with the smallest score in the ZSet and calculating the number of invalidated tokens. If the smallest score key is successfully invalidated, it means that the current request has successfully obtained a token. If the number of invalidated tokens is 0, it means that there are no more tokens in the current token bucket, and the request is blocked and not allowed to pass.
[0070] Combined with appendix Figure 3 After executing (1)-(7), the rate limiting process of the distributed rate limiting implementation method is as follows:
[0071] After deploying a distributed gateway, the first gateway instance to start will create a gateway leader message in Redis. The default validity period of the message is T s. Once the first gateway instance starts writing the leader message, the remaining gateway instances will see that there is already a leader and will not create a leader message.
[0072] The gateway instance that writes the leader message is called the leader instance. The leader instance maintains the creation of the token bucket and the generation of tokens. The leader instance renews the leader message in Redis every T-1 seconds. The remaining gateway services check whether the leader message exists in Redis every T seconds. If it does not exist, it writes it. The gateway instance that writes the new leader message is promoted to the new leader instance.
[0073] After configuring the rate limiting rules, the token bucket within the defined time window is initialized first, and the remaining token buckets are filled one by one according to the rate limiting rate. As the old token buckets are discarded, the time window migration will include the newly generated token buckets in the token consumption range. During this period, the leader instance will continuously create tokens and pre-set the tokens into the latest token bucket.
[0074] After traffic enters the cluster, it is transferred to one of the gateway instances through load balancing. The gateway instance determines the rate limiting rule corresponding to the API to which the request is made based on the parameters carried in the request. According to the rate limiting rule, it obtains a token from the token bucket corresponding to the current time. When a token is successfully obtained, the approved request will be directly transferred to the corresponding service. Meanwhile, requests that do not obtain a token will be directly returned to the requester to protect the upstream service.
[0075] Example 2:
[0076] Combined with appendix Figure 2 This embodiment proposes a distributed rate limiting implementation device for a cloud-native gateway, which includes:
[0077] The deployment module is used to deploy a Redis cluster in a Kubernetes cluster, serving as the CAS rate limiting data storage center for the gateway;
[0078] Select the module to choose the ZSet data type in Redis as the resource type for the rate limiting token;
[0079] The configuration module is used to configure rate limiting rules, allowing you to set the number of token buckets and the token generation rate based on a sliding window;
[0080] The configuration module is used to set the time-to-live for each token bucket and to automatically reclaim expired token buckets using Redis's resource reclamation mechanism.
[0081] The configuration module is used to set the globally unique key carried by the token as the key of the ZSet to ensure the uniqueness of API requests;
[0082] The token processing module is used to take the generation sequence number of the token in each token bucket as the value, and to perform the token consumption action using the API built into the ZSet data structure when consuming tokens.
[0083] The rules setting module is used to set rules to count and limit the traffic of API requests.
[0084] In this embodiment, the configuration module configures rate limiting rules based on the sliding window algorithm and the token bucket algorithm. The configuration module calls the leader instance in the gateway to create tokens. When the leader instance creates tokens, the configuration module needs to pre-determine the token generation rate *r*. Based on the rate limiting rule *r = n / t*, it calculates the rate limiting time unit *t* and the allowed number of passes per unit time *n*. Simultaneously, the default setting for the number of token buckets in the sliding window is 4 slots. Therefore, the maximum number of tokens in each token bucket is *n / 4*, and the effective time of each token bucket is *t / 4*. The lifespan of each token bucket is *t / 4*. After the token bucket is created, regardless of whether all tokens in the bucket have been used, the token bucket will be automatically destroyed after *t / 4* time units.
[0085] In this embodiment, the time-to-live setting (TTL) for each token bucket is consistent with the expiration date of the tokens within it. When a token in a token bucket expires, regardless of whether all tokens have been consumed, the TTL setting module will invoke Redis's own resource reclamation mechanism to automatically reclaim the expired token bucket.
[0086] In this embodiment, the rules set by the rule setting module are as follows:
[0087] Each time an API request is recorded, the corresponding token bucket is calculated based on the current request time. An attempt is made to obtain a token from the bucket by invalidating the key with the smallest score in the ZSet and calculating the number of invalidated tokens. If the smallest score key is successfully invalidated, it means that the current request has successfully obtained a token. If the number of invalidated tokens is 0, it means that there are no more tokens in the current token bucket, and the request is blocked and not allowed to pass.
[0088] Combined with appendix Figure 3 The distributed rate limiting implementation device of this embodiment implements the following process for distributed rate limiting:
[0089] After deploying a distributed gateway, the first gateway instance to start will create a gateway leader message in Redis. The default validity period of the message is T s. Once the first gateway instance starts writing the leader message, the remaining gateway instances will see that there is already a leader and will not create a leader message.
[0090] The gateway instance that writes the leader message is called the leader instance. The leader instance maintains the creation of the token bucket and the generation of tokens. The leader instance renews the leader message in Redis every T-1 seconds. The remaining gateway services check whether the leader message exists in Redis every T seconds. If it does not exist, it writes it. The gateway instance that writes the new leader message is promoted to the new leader instance.
[0091] After configuring the rate limiting rules, the token bucket within the defined time window is initialized first, and the remaining token buckets are filled one by one according to the rate limiting rate. As the old token buckets are discarded, the time window migration will include the newly generated token buckets in the token consumption range. During this period, the leader instance will continuously create tokens and pre-set the tokens into the latest token bucket.
[0092] After traffic enters the cluster, it is transferred to one of the gateway instances through load balancing. The gateway instance determines the rate limiting rule corresponding to the API to which the request is made based on the parameters carried in the request. According to the rate limiting rule, it obtains a token from the token bucket corresponding to the current time. When a token is successfully obtained, the approved request will be directly transferred to the corresponding service. Meanwhile, requests that do not obtain a token will be directly returned to the requester to protect the upstream service.
[0093] In summary, the distributed rate limiting method and apparatus for cloud-native gateways of the present invention can accurately limit the peak traffic within any unit time period, solve the traffic limiting problem under the distributed multi-instance architecture of the gateway, solve the problem of instantaneous traffic exceeding the limit after the rate limiting policy is changed, and solve the problem of unsmooth requests under the rate limiting policy.
[0094] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.
Claims
1. A distributed rate limiting implementation method for a cloud-native gateway, characterized in that, This implementation method includes the following operations: (1) Deploy a Redis cluster in a Kubernetes cluster as the data storage center for the gateway's CAS rate limiting; (2) Select the ZSet data type in Redis as the resource type for the rate limiting token; (3) Configure rate limiting rules, and set the number of token buckets and the token generation rate based on the sliding window; (4) Set a timeout for each token bucket and use Redis’s resource reclamation mechanism to automatically reclaim expired token buckets; (5) Each token carries a globally unique key. The globally unique key carried by the token is set as the key of the ZSet to ensure the uniqueness of the API request; (6) Use the generation sequence number of the token in each token bucket as the Value value. When consuming tokens, use the API built into the ZSet data structure to perform the token consumption action. (7) Set rules to count and limit the traffic of API requests; After executing (1)-(7), the rate limiting process of the distributed rate limiting implementation method is as follows: After deploying the distributed gateway, the first gateway instance to start will create a gateway leader message in Redis. The default validity period of the message is T s. Once the first gateway instance starts writing the leader message, the remaining gateway instances will see that there is already a leader and will not create a leader message. The gateway instance that writes the leader message is called the leader instance. The leader instance maintains the creation of the token bucket and the generation of tokens. The leader instance renews the leader message in Redis every T-1 seconds. The remaining gateway services check whether the leader message exists in Redis every T seconds. If it does not exist, it writes it. The gateway instance that writes the new leader message is promoted to the new leader instance. After configuring the rate limiting rules, the token bucket within the defined time window is initialized first, and the remaining token buckets are filled one by one according to the rate limiting rate. As the old token buckets are discarded, the time window migration will include the newly generated token buckets in the token consumption range. During this period, the leader instance will continuously create tokens and pre-set the tokens into the latest token bucket. After traffic enters the cluster, it is transferred to one of the gateway instances through load balancing. The gateway instance determines the rate limiting rule corresponding to the API to which the request is made based on the parameters carried in the request. According to the rate limiting rule, it obtains a token from the token bucket corresponding to the current time. When a token is successfully obtained, the approved request will be directly transferred to the corresponding service. Meanwhile, requests that do not obtain a token will be directly returned to the requester to protect the upstream service.
2. The distributed rate limiting implementation method for a cloud-native gateway according to claim 1, characterized in that, Execute (3), configure rate limiting rules based on the sliding window algorithm and the token bucket algorithm, and create tokens through the leader instance in the distributed gateway; When creating a token, the token generation rate r is confirmed in advance. Based on the rate limiting rule r = n / t, the rate limiting time unit t and the number of times n can pass through per unit time are calculated. At the same time, the default setting of the number of token buckets in the sliding window is 4. Then the maximum number of tokens in each token bucket is n / 4, the effective time of each token bucket is t / 4, and the lifespan of each token bucket is t / 4. Once the token bucket is created, it will be automatically destroyed after t / 4 time units, regardless of whether all the tokens in the bucket have been used.
3. The distributed rate limiting implementation method for a cloud-native gateway according to claim 1, characterized in that, Execute (4), the lifespan of each token bucket is consistent with the validity period of the tokens within it; When tokens in the token bucket expire, regardless of whether all tokens have been used up, the token bucket will be automatically reclaimed by Redis's own resource reclamation mechanism.
4. The distributed rate limiting implementation method for a cloud-native gateway according to claim 1, characterized in that, The specific rules set in (7) are as follows: Each time an API request is recorded, the corresponding token bucket is calculated based on the current request time. An attempt is made to obtain a token from the bucket by invalidating the key with the smallest score in the ZSet and calculating the number of invalidated tokens. If the smallest score key is successfully invalidated, it means that the current request has successfully obtained a token. If the number of invalidated tokens is 0, it means that there are no more tokens in the current token bucket, and the request is blocked and not allowed to pass.
5. A distributed rate limiting implementation device for a cloud-native gateway, characterized in that, It includes: The deployment module is used to deploy a Redis cluster in a Kubernetes cluster, serving as the data storage center for the gateway's CAS rate limiting. Select the module to choose the ZSet data type in Redis as the resource type for the rate limiting token; The configuration module is used to configure rate limiting rules, allowing you to set the number of token buckets and the token generation rate based on a sliding window; The configuration module is used to set the time-to-live for each token bucket and to automatically reclaim expired token buckets using Redis's resource reclamation mechanism. The configuration module is used to set the globally unique key carried by the token as the key of the ZSet to ensure the uniqueness of API requests; The token processing module is used to take the generation sequence number of the token in each token bucket as the Value, and to perform the token consumption action using the API built into the ZSet data structure when consuming tokens. The rules setting module is used to set rules to count and limit the traffic of API requests; The process of implementing distributed rate limiting using a distributed rate limiting device is as follows: After the distributed gateway is deployed, the Redis cluster deployed by the deployment module provides data support. The first gateway instance to start will create a gateway leader message in Redis. The default validity period of the message is T s. After the first gateway instance starts writing the leader message, the remaining gateway instances will see that there is already a leader and will not create a leader message. The gateway instance that writes the leader message is called the leader instance. The leader instance maintains the creation of the token bucket and the generation of tokens. The leader instance renews the leader message in Redis every T-1 seconds. The remaining gateway services check whether the leader message exists in Redis every T seconds. If it does not exist, it writes it. The gateway instance that writes the new leader message is promoted to the new leader instance. After configuring the rate limiting rules, the configuration module completes the configuration of the rate limiting rules. Then, according to the defined time window, the token bucket within the time window is initialized first. The token processing module, together with the leader instance, completes the creation and pre-setting of tokens. The remaining token buckets are filled one by one according to the rate limiting rate. As the old token bucket is discarded, the time window migration will include the newly generated token bucket in the token consumption range. During this period, the leader instance will continuously create tokens and pre-set the tokens into the latest token bucket. After traffic enters the cluster, it is transferred to one of the gateway instances through load balancing. The gateway instance, in conjunction with the rate limiting rules configured by the rule setting module, determines the rate limiting rule corresponding to the API to which the request is made based on the parameters carried in the request. According to the rate limiting rule, the token processing module then obtains a token from the token bucket corresponding to the current time. When a token is successfully obtained, the approved request is directly transferred to the corresponding service. Meanwhile, requests that do not obtain a token are directly returned to the requester to protect the upstream service.
6. The distributed rate limiting implementation device for a cloud-native gateway according to claim 5, characterized in that, The configuration module configures rate limiting rules based on the sliding window algorithm and the token bucket algorithm. The configuration module calls the leader instance in the gateway to create a token; When the leader instance creates a token, the configuration module needs to confirm the token generation rate r in advance. Based on the rate limiting rule r = n / t, the rate limiting time unit t and the number of times n can pass through per unit time are calculated. At the same time, the default setting of the number of token buckets in the sliding window is 4, so the maximum number of tokens in each token bucket is n / 4, the effective time of each token bucket is t / 4, and the lifespan of each token bucket is t / 4. Once the token bucket is created, it will be automatically destroyed after t / 4 time units, regardless of whether all the tokens in the bucket have been used.
7. The distributed rate limiting implementation device for a cloud-native gateway according to claim 5, characterized in that, The setting module assigns a lifespan to each token bucket that is consistent with the validity period of the tokens within it. When a token in the token bucket expires, regardless of whether all tokens have been consumed, the setting module will invoke Redis's own resource reclamation mechanism to automatically reclaim the expired token.
8. A distributed rate limiting implementation device for a cloud-native gateway according to claim 5, characterized in that, The specific rules set by the rule setting module are as follows: Each time an API request is recorded, the corresponding token bucket is calculated based on the current request time. An attempt is made to obtain a token from the bucket by invalidating the key with the smallest score in the ZSet and calculating the number of invalidated tokens. If the smallest score key is successfully invalidated, it means that the current request has successfully obtained a token. If the number of invalidated tokens is 0, it means that there are no more tokens in the current token bucket, and the request is blocked and not allowed to pass.