Accurate current limiting method and device suitable for Kubernetes Ingress and electronic equipment

Abstract

The invention discloses an accurate current limiting method and device suitable for Kubernetes Ingress and electronic equipment, and the method comprises the steps: configuring a local current limiting parameter and a centralized current limiting parameter; the number of requests per second of the local Ingress Pod is counted and compared with a configured local current limiting parameter, and whether a local current limiting strategy is executed or not is judged; and when the number of requests per second of the local Ingress Pod exceeds the configured centralized flow limiting parameter, executing a centralized flow limiting strategy, collecting flow information of each Ingress Pod, and performing global flow analysis and control. According to the method, local current limiting and concentrated current limiting are combined, the advantages of the local current limiting in the aspects of performance and stability can be brought into full play, the current limiting accuracy can be improved by means of the concentrated current limiting, and the combination scheme has uniqueness in the prior art and is suitable for popularization and application. The problem of poor stability caused by inaccurate current limiting or dependence on centralized storage in the prior art can be effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of cloud computing and containerization technology, and specifically provides a precise rate limiting method, apparatus, and electronic device applicable to Kubernetes Ingress. Background Technology

[0002] With the widespread adoption of modern cloud computing and containerization technologies, Kubernetes, as a leading container orchestration platform, plays a crucial role in managing external traffic accessing services within the cluster through its Ingress gateway. The Ingress gateway, through configured rules, performs functions such as traffic routing and load balancing, ensuring that external requests are accurately distributed to the corresponding backend services.

[0003] In terms of traffic control, existing open-source Kubernetes Ingress components typically employ simple rate limiting strategies. When configuring rate limiting rules, such as setting a limit on requests per second (QPS), the Ingress Controller translates this rule into a configuration file for the underlying proxy software (such as Nginx or Traefik). For example, in Nginx, a corresponding `limit_req` directive is generated. Each Ingress Pod independently processes the traffic passing through it and controls it according to the rate limiting rules.

[0004] However, this rate limiting method has significant drawbacks. In a distributed environment, multiple Ingress Pods receive traffic through load balancing strategies, but the traffic across different Pods may not be evenly distributed. For example, suppose the rate limit is set to 100, and there are 10 Ingress Pods, each Pod independently determines whether it exceeds 10 QPS (total rate limit 100 / 10 Pods). But in reality, due to the randomness of traffic and the limitations of load balancing strategies, some Pods may receive more traffic, leading to inaccurate overall rate limiting. Some Pods may exceed the set rate limit, while other Pods do not fully utilize their processing capacity.

[0005] Furthermore, some open-source solutions attempt to address traffic imbalance by introducing centralized storage (such as Redis) as a counter and rate limiter. While this approach can improve counting accuracy, it also introduces new problems. Centralized storage becomes a single point of failure; if Redis fails, the gateway will also fail, causing the entire entry service to become unavailable and potentially leading to a serious incident. This reliance on middleware increases system complexity and maintenance costs, and reduces system stability.

[0006] In view of this, this invention patent is hereby proposed. Summary of the Invention

[0007] This invention proposes a precise rate limiting method, device, and electronic device for Kubernetes Ingress, aiming to solve the problems of inaccurate rate limiting in existing Kubernetes Ingress gateways under low-traffic scenarios and the stability risks associated with reliance on centralized storage. Specifically, the following technical solution is adopted:

[0008] In a first aspect, the present invention provides a precise rate limiting method applicable to Kubernetes Ingress, comprising:

[0009] Configure local and centralized rate limiting parameters. Local rate limiting is executed by the local rate limiting service, while centralized rate limiting is executed by the central rate limiting service.

[0010] By counting the number of requests per second in the local Ingress Pod and comparing it with the configured local rate limiting parameters, it is determined whether to execute the local rate limiting policy.

[0011] When the number of requests per second of a local Ingress Pod exceeds the configured centralized rate limiting parameter, a centralized rate limiting policy is executed, and traffic information of each Ingress Pod is collected for global traffic analysis and control.

[0012] As an optional embodiment of the present invention, in the precise rate limiting method applicable to Kubernetes Ingress described in the present invention, the configuration of local rate limiting parameters and centralized rate limiting parameters includes: configuring the centralized rate limiting parameter limit and the local maximum rate limiting parameter max;

[0013] The execution of the local rate limiting policy and the centralized rate limiting policy includes:

[0014] If the number of requests per second in the local Ingress Pod is less than the centralized rate limiting parameter limit, then local rate limiting will not be executed.

[0015] If the number of requests per second of the local Ingress Pod is greater than or equal to the local maximum rate limiting parameter max, then local rate limiting will be executed directly.

[0016] The centralized rate limiting policy is triggered when the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit, but less than the local maximum rate limiting parameter max.

[0017] As an optional embodiment of the present invention, in a precise rate limiting method for Kubernetes Ingress, the step of counting the number of requests per second of the local Ingress Pod and comparing it with the configured local rate limiting parameters to determine whether to execute the local rate limiting strategy includes:

[0018] When a traffic request arrives at the local Ingress Pod, the service identifier and the current second-level timestamp are concatenated to generate a rate limiting key;

[0019] Perform an atomic increment operation on the rate-limiting key in shared memory and return the current count current_qps;

[0020] If current_qps < limit, allow the traffic requests;

[0021] If limit ≤ cur rent_qps < max, trigger centralized rate limiting strategy;

[0022] If current_qps ≥ max, the traffic request is rejected directly.

[0023] As an optional embodiment of the present invention, in a precise rate limiting method applicable to Kubernetes Ingress, the step of triggering the centralized rate limiting strategy when the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit and less than the local maximum rate limiting parameter max includes:

[0024] The centralized rate limiting parameter limit is generated by the centralized rate limiting strategy, and the validity period of the centralized rate limiting parameter limit is also generated.

[0025] When the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit, and less than the local maximum rate limiting parameter max;

[0026] Further determine whether the current limiting parameter "limit" has expired;

[0027] If the judgment result is yes, the centralized rate limiting strategy is triggered; if the judgment result is no, the local rate limiting strategy is executed.

[0028] As an optional embodiment of the present invention, in the precise rate limiting method for Kubernetes Ingress described in the present invention, the centralized rate limiting strategy includes:

[0029] Receive rate limiting requests: Get request parameters from the Ingress Pod, including cur_limit, cur_qps, limit_key and cur_timestamp. cur_limit represents the current local rate limiting threshold, cur_qps represents the current QPS statistics per second, limit_key represents the rate limiting key, and cur_timestamp represents the precise timestamp of the current traffic request.

[0030] Aggregate global QPS: Query the current QPS of all Ingress Pods in the cluster according to the limit_key, and calculate the total number of requests for this interface;

[0031] Rate limiting decision: If global QPS + (cur_qps - the counted QPS of this Pod) ≤ GLOBAL_LIMIT, where GLOBAL_LIMIT represents the global rate limiting threshold, then allow the current request to pass; otherwise, reject the request;

[0032] Return result: Return a boolean value indicating whether rate limiting is performed to the traffic requester.

[0033] As an optional implementation manner of the present invention, a precise rate limiting method applicable to Kubernetes Ingress described in the present invention includes:

[0034] Collect historical QPS data: Obtain the global QPS sequence of the interface in the recent N time windows based on the limit_key;

[0035] Calculate the benchmark average QPS (avg_qps): The arithmetic mean of the past N windows, and the maximum fluctuation value (max_delta): The historical maximum value of the current QPS - avg_qps;

[0036] Predict future traffic: Optimistic prediction: predicted_qps = avg_qps + 0.5 * max_delta, pessimistic prediction: predicted_qps = avg_qps + max_delta;

[0037] Generate a prediction result: If predicted_qps ≤ GLOBAL_LIMIT, return allowed and set the prediction validity period;

[0038] Otherwise, return rejected and mark that data needs to be forced to be calibrated.

[0039] As an optional implementation manner of the present invention, in a precise rate limiting method applicable to Kubernetes Ingress described in the present invention, the prediction validity period is dynamically adjusted:

[0040] If max_delta / avg_qps < N1%, the validity period is extended to T1 seconds;

[0041] If max_delta / avg_qps ≥ N2%, the validity period is shortened to T2 seconds,

[0042] where N1 and N2 are preset values, N1 is less than N2, and T1 and T2 are preset values.

[0043] As an optional embodiment of the present invention, a precise rate limiting method applicable to Kubernetes Ingress according to the present invention includes:

[0044] Each Ingress Pod reports the current QPS data of all its rate-limiting interfaces to the central rate-limiting service that implements centralized rate-limiting policies at fixed time intervals;

[0045] The central rate limiting service receives the QPS data reported by each Pod, performs clustering and summation by limit_key, and obtains the total cluster-level QPS for each rate limiting interface;

[0046] The rate limiting status of each interface is updated by comparing the aggregated total QPS with the preset global rate limiting threshold.

[0047] Synchronize the updated rate limiting policy to each Ingress Pod.

[0048] In a second aspect, the present invention provides a precise rate limiting device suitable for Kubernetes Ingress, comprising:

[0049] The rate limiting parameter configuration module allows you to configure local and centralized rate limiting parameters.

[0050] The local rate limiting module counts the number of requests per second in the local Ingress Pod and compares it with the configured local rate limiting parameters to determine whether to execute the local rate limiting policy.

[0051] The centralized rate limiting module executes a centralized rate limiting strategy when the number of requests per second of a local Ingress Pod exceeds the configured centralized rate limiting parameters. It collects traffic information from each Ingress Pod and performs global traffic analysis and control.

[0052] In a third aspect, the present invention provides an electronic device including a processor and a memory, the memory being used to store a computer-executable program, wherein when the computer program is executed by the processor, the processor executes the precise rate limiting method applicable to Kubernetes Ingress.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] This invention provides a precise rate limiting method for Kubernetes Ingress, combining local and centralized rate limiting. This innovative approach fully leverages the performance and stability advantages of local rate limiting while improving accuracy through centralized rate limiting. In routine traffic processing, local rate limiting offers rapid response, reduces system latency, and enhances performance; while in scenarios with uneven traffic or requiring precise control, centralized rate limiting provides global coordination, ensuring accurate rate limiting results. This combined approach is unique in existing technologies, effectively addressing the issues of inaccurate rate limiting or poor stability caused by reliance on centralized storage in current technologies.

[0055] This invention provides a precise rate limiting method for Kubernetes Ingress, employing an asynchronous synchronization mechanism for rate limiting data. The synchronization time interval can be configured according to actual needs. More importantly, centralized rate limiting counting is only triggered in scenarios requiring precise rate limiting. This mechanism effectively reduces the frequency of access to the central rate limiting service, alleviating its burden, while also reducing network transmission and data processing overhead, thus improving the overall system efficiency and resource utilization.

[0056] This invention provides a precise rate limiting method for Kubernetes Ingress. A central rate limiting service predicts the rate limiting value by calculating parameters such as average QPS, maximum QPS fluctuation, and remaining QPS margin. This predicted value has a validity period. Within this period, the system can perform rate limiting decisions based on the predicted value without frequent access to the central rate limiting service. This prediction mechanism further reduces reliance on the central rate limiting service, improves system autonomy and response speed, and better addresses dynamic traffic changes, ensuring the stability and accuracy of the rate limiting effect.

[0057] Therefore, the precise rate limiting method for Kubernetes Ingress proposed in this invention has the following technical effects:

[0058] I. Improve performance and stability

[0059] This invention provides a precise rate limiting method for Kubernetes Ingress, combining local and centralized rate limiting. In daily traffic processing, local rate limiting is primarily relied upon. Since local rate limiting does not require frequent interaction with external centralized rate limiting services, it reduces network latency and data transmission overhead, thereby improving system performance.

[0060] The existence of local rate limiting also provides stability assurance for the system. Even if the central rate limiting service fails or there are network connection problems, local rate limiting can still continue to work, ensuring that basic traffic control functions are not affected.

[0061] II. Improve the accuracy of traffic limiting

[0062] Centralized rate limiting plays a crucial role in this invention. While local rate limiting can handle most situations, in scenarios with uneven traffic distribution, some Pods may experience unexpected traffic spikes. In such cases, the centralized rate limiting service collects traffic information from each Pod, performs global traffic analysis and control, and can precisely limit overall traffic.

[0063] The asynchronous synchronization mechanism of rate limiting data further enhances the accuracy of rate limiting. By periodically synchronizing local rate limiting data to the centralized rate limiting service, the centralized service can understand the traffic status of each Pod in real time and adjust the rate limiting strategy according to the actual situation.

[0064] III. Reduce reliance on central traffic limiting services

[0065] The asynchronous synchronization mechanism of rate limiting data ensures that centralized rate limiting counting is triggered only when precise rate limiting is required. This means that most of the time, the system primarily relies on local rate limiting for traffic control, reducing the frequency of access to the central rate limiting service. For example, when traffic is low and evenly distributed, local rate limiting can fully meet the traffic control needs without frequent interaction with the central rate limiting service, thus reducing dependence on the central rate limiting service.

[0066] The predictive mechanism of the central rate limiting service also helps reduce reliance. By calculating parameters such as average QPS, maximum QPS fluctuation, and rate limiting QPS margin, a rate limiting prediction value is generated, and this prediction value has a certain validity period. Within the validity period, the system can make rate limiting decisions based on the prediction value without frequently accessing the central rate limiting service. Attached Figure Description

[0067] Figure 1 A flowchart of a precise rate limiting method applicable to Kubernetes Ingress according to an embodiment of the present invention;

[0068] Figure 2 An overall framework diagram of a precise rate limiting device for Kubernetes Ingress according to an embodiment of the present invention;

[0069] Figure 3 A schematic diagram of the structure of the electronic device according to an embodiment of the present invention;

[0070] Figure 4A schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0072] Therefore, the following detailed description of embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely illustrates some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0073] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0074] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0075] In the description of this invention, it should be noted that the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. These terms are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0076] See Figure 1 As shown, this embodiment provides a precise rate limiting method for Kubernetes Ingress, including:

[0077] Configure local and centralized rate limiting parameters. Local rate limiting is executed by the local rate limiting service, while centralized rate limiting is executed by the central rate limiting service.

[0078] By counting the number of requests per second in the local Ingress Pod and comparing it with the configured local rate limiting parameters, it is determined whether to execute the local rate limiting policy.

[0079] When the number of requests per second of a local Ingress Pod exceeds the configured centralized rate limiting parameter, a centralized rate limiting policy is executed, and traffic information of each Ingress Pod is collected for global traffic analysis and control.

[0080] This embodiment provides a precise rate limiting method for Kubernetes Ingress, combining local and centralized rate limiting. This innovative approach fully leverages the performance and stability advantages of local rate limiting while improving accuracy through centralized rate limiting. In routine traffic processing, local rate limiting offers rapid response, reduces system latency, and enhances performance; while in scenarios with uneven traffic or requiring precise control, centralized rate limiting provides global coordination, ensuring the accuracy of rate limiting effects. This combined approach is unique in existing technologies, effectively addressing the issues of inaccurate rate limiting or poor stability caused by reliance on centralized storage in current technologies.

[0081] This embodiment provides a precise rate limiting method for Kubernetes Ingress, employing an asynchronous synchronization mechanism for rate limiting data. The synchronization time interval can be configured according to actual needs. More importantly, centralized rate limiting counting is only triggered when precise rate limiting is required. This mechanism effectively reduces the frequency of access to the central rate limiting service, alleviating its burden, while also reducing network transmission and data processing overhead, thus improving the overall system efficiency and resource utilization.

[0082] Therefore, the precise rate limiting method for Kubernetes Ingress in this embodiment has the following technical effects:

[0083] I. Improve performance and stability

[0084] This embodiment presents a precise rate limiting method for Kubernetes Ingress, combining local and centralized rate limiting. In daily traffic processing, local rate limiting is primarily relied upon. Since local rate limiting does not require frequent interaction with external centralized rate limiting services, it reduces network latency and data transmission overhead, thereby improving system performance.

[0085] For example, suppose a Kubernetes Ingress gateway deploys 10 Pods, each with local rate limiting capabilities. When traffic is evenly distributed, each Pod can quickly control traffic according to its local rate limiting rules without waiting for a response from a centralized service. This significantly reduces system response time and improves overall performance.

[0086] The existence of local rate limiting also provides stability assurance for the system. Even if the central rate limiting service fails or there are network connection problems, local rate limiting can still continue to work, ensuring that basic traffic control functions are not affected.

[0087] For example, even if the central rate limiting service goes down, each Pod can still limit traffic according to its local rate limiting configuration, preventing the entire system from crashing due to excessive traffic and thus ensuring system stability.

[0088] II. Improve the accuracy of traffic limiting

[0089] Centralized rate limiting plays a crucial role in this invention. While local rate limiting can handle most situations, in scenarios with uneven traffic distribution, some Pods may experience unexpected traffic spikes. In such cases, the centralized rate limiting service collects traffic information from each Pod, performs global traffic analysis and control, and can precisely limit overall traffic.

[0090] For example, when a Pod receives a large amount of traffic due to some reason (such as network fluctuations or unbalanced load balancing strategies), the centralized rate limiting service can detect it in time and adjust the rate limiting accordingly, ensuring that the traffic of the entire system does not exceed the set rate limit value, thereby improving the accuracy of rate limiting.

[0091] The asynchronous synchronization mechanism of rate limiting data further enhances the accuracy of rate limiting. By periodically synchronizing local rate limiting data to the centralized rate limiting service, the centralized service can understand the traffic status of each Pod in real time and adjust the rate limiting strategy according to the actual situation.

[0092] For example, suppose that the traffic to a certain Pod suddenly increases within a certain period of time. Local rate limiting may not be able to adjust the rate limiting strategy in time. However, through an asynchronous synchronous mechanism, a centralized rate limiting service can quickly obtain this information and make precise rate limiting adjustments to the Pod, avoiding system overload due to sudden traffic surges.

[0093] III. Reduce reliance on central traffic limiting services

[0094] The asynchronous synchronization mechanism of rate limiting data ensures that centralized rate limiting counting is triggered only when precise rate limiting is required. This means that most of the time, the system primarily relies on local rate limiting for traffic control, reducing the frequency of access to the central rate limiting service. For example, when traffic is low and evenly distributed, local rate limiting can fully meet the traffic control needs without frequent interaction with the central rate limiting service, thus reducing dependence on the central rate limiting service.

[0095] The predictive mechanism of the central rate limiting service also helps reduce reliance. By calculating parameters such as average QPS, maximum QPS fluctuation, and rate limiting QPS margin, a rate limiting prediction value is generated, and this prediction value has a certain validity period. Within the validity period, the system can make rate limiting decisions based on the prediction value without frequently accessing the central rate limiting service.

[0096] For example, suppose the central rate limiting service predicts that the traffic of a certain Pod will not exceed the set rate limiting value within the next 5 seconds. Then, within those 5 seconds, the Pod can perform local rate limiting based on the prediction value without having to request the central rate limiting service again, thereby further reducing its dependence on the central rate limiting service.

[0097] As an optional implementation of this embodiment, in a precise rate limiting method applicable to Kubernetes Ingress, the configuration of local rate limiting parameters and centralized rate limiting parameters includes: configuring a centralized rate limiting parameter limit and a local maximum rate limiting parameter max;

[0098] The execution of the local rate limiting policy and the centralized rate limiting policy includes:

[0099] If the number of requests per second in the local Ingress Pod is less than the centralized rate limiting parameter limit, then local rate limiting will not be executed.

[0100] If the number of requests per second of the local Ingress Pod is greater than or equal to the local maximum rate limiting parameter max, then local rate limiting will be executed directly.

[0101] The centralized rate limiting policy is triggered when the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit, but less than the local maximum rate limiting parameter max.

[0102] This embodiment configures local rate limiting with a minimum local rate limiting parameter `min` and a maximum local rate limiting parameter `max`. The minimum local rate limiting parameter `min` is equal to the centralized rate limiting parameter `limit`. Since the centralized rate limiting parameter `limit` is calculated and determined by the central rate limiting service, the minimum local rate limiting parameter `min` is dynamically updated along with the centralized rate limiting parameter `limit`. The value of the maximum local rate limiting parameter `max` can be determined through gateway traffic log statistics or manually set to twice the rate limiting value.

[0103] As an optional implementation of this embodiment, in a precise rate limiting method applicable to Kubernetes Ingress, the step of counting the number of requests per second of the local Ingress Pod and comparing it with the configured local rate limiting parameters to determine whether to execute the local rate limiting policy includes:

[0104] When a traffic request arrives at the local Ingress Pod, the service identifier and the current second-level timestamp are concatenated to generate a rate limiting key;

[0105] Perform an atomic increment operation on the rate-limiting key in shared memory and return the current count current_qps;

[0106] If current_qps < limit, allow the traffic requests;

[0107] If limit ≤ cur rent_qps < max, trigger centralized rate limiting strategy;

[0108] If current_qps ≥ max, the traffic request is rejected directly.

[0109] As an optional implementation of this embodiment, in a precise rate limiting method applicable to Kubernetes Ingress, the step of triggering the centralized rate limiting strategy when the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter `limit` and less than the local maximum rate limiting parameter `max` includes:

[0110] The centralized rate limiting parameter limit is generated by the centralized rate limiting strategy, and the validity period of the centralized rate limiting parameter limit is also generated.

[0111] When the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit, and less than the local maximum rate limiting parameter max;

[0112] Further determine whether the current limiting parameter "limit" has expired;

[0113] If the judgment result is yes, the centralized rate limiting strategy is triggered; if the judgment result is no, the local rate limiting strategy is executed.

[0114] Furthermore, in a precise rate limiting method for Kubernetes Ingress according to this embodiment, the centralized rate limiting strategy includes:

[0115] Receive rate limiting requests: Get request parameters from the Ingress Pod, including cur_limit, cur_qps, limit_key and cur_timestamp. cur_limit represents the current local rate limiting threshold, cur_qps represents the current QPS statistics per second, limit_key represents the rate limiting key, and cur_timestamp represents the precise timestamp of the current traffic request.

[0116] Aggregate global QPS: Query the current QPS of all Ingress Pods in the cluster based on limit_key, and calculate the total number of requests for this interface;

[0117] Rate limiting decision: If global QPS + (cur_qps - the Pod's counted QPS) ≤ GLOBAL_LIMIT, where GLOBAL_LIMIT represents the global rate limiting threshold, then the current request is allowed to pass; otherwise, the request is rejected.

[0118] Return result: Returns a boolean value indicating whether rate limiting is enabled to the traffic requester.

[0119] In this embodiment, the central rate limiting service predicts the rate limiting value by calculating parameters such as the average QPS, the maximum QPS fluctuation, and the remaining QPS for rate limiting. This predicted value has a validity period. Within this period, the system can perform rate limiting decisions based on the predicted value without frequent access to the central rate limiting service. This prediction mechanism further reduces reliance on the central rate limiting service, improves the system's autonomy and response speed, and better addresses dynamic traffic changes, ensuring the stability and accuracy of the rate limiting effect.

[0120] Specifically, this embodiment provides a precise rate limiting method for Kubernetes Ingress, including:

[0121] Collect historical QPS data: Obtain the global QPS sequence for the most recent N time windows of this interface based on the limit_key;

[0122] Calculate the baseline average QPS (avg_qps): the arithmetic mean of the past N windows; the maximum fluctuation (max_delta): the historical maximum value of the current QPS - avg_qps.

[0123] Predicting future traffic: Optimistic prediction: predicted_qps = avg_qps + 0.5 * max_delta, pessimistic prediction: predicted_qps = avg_qps + max_delta;

[0124] Generate prediction results: if predicted_qps ≤ GLOBAL_LIMIT, return "allow" and set the prediction validity period;

[0125] Otherwise, return a rejection and mark the data as requiring mandatory calibration.

[0126] In this embodiment, QPS (Queries Per Second): QPS, that is, the query rate per second, is an important indicator to measure the system's request processing ability. It represents the number of requests that the system can process per second and is usually used to evaluate the performance of a server or service. In the traffic limiting scenario, QPS can be used as a key parameter for traffic limiting. By setting the maximum number of requests allowed to pass per second, the entry rate of traffic can be controlled, thereby protecting the system from being overloaded.

[0127] As an optional implementation of this embodiment, in a precise traffic limiting method applicable to Kubernetes Ingress in this embodiment, the prediction validity period is dynamically adjusted:

[0128] If max_delta / avg_qps < N1%, the validity period is extended to T1 seconds;

[0129] If max_delta / avg_qps ≥ N2%, the validity period is shortened to T2 seconds,

[0130] where N1 and N2 are preset values, N1 is less than N2, and T1 and T2 are preset values.

[0131] As an optional implementation of the present invention, a precise traffic limiting method applicable to Kubernetes Ingress of the present invention includes:

[0132] Each Ingress Pod reports the current QPS data of all its traffic limiting interfaces to the central traffic limiting service that executes the centralized traffic limiting policy at a fixed time interval;

[0133] The central traffic limiting service receives the QPS data reported by each Pod, clusters and sums them according to limit_key, and obtains the cluster-level total QPS of each traffic limiting interface;

[0134] Based on the aggregated total QPS, compare it with the preset global traffic limiting threshold, and update the traffic limiting status of each interface;

[0135] Synchronize the updated traffic limiting policy to each Ingress Pod.

[0136] The present invention designs an asynchronous synchronization method for traffic limiting data, and the synchronization time interval can be configured according to actual needs. More importantly, only in the scenario where precise traffic limiting is required will the centralized traffic limiting count be triggered. Such a mechanism effectively reduces the access frequency to the central traffic limiting service, reduces the burden on the central traffic limiting service, and also reduces the network transmission and data processing overhead, improving the overall efficiency and resource utilization rate of the system.

[0137] This invention also provides a precise rate limiting device suitable for Kubernetes Ingress, comprising:

[0138] The rate limiting parameter configuration module allows you to configure local and centralized rate limiting parameters.

[0139] The local rate limiting module counts the number of requests per second in the local Ingress Pod and compares it with the configured local rate limiting parameters to determine whether to execute the local rate limiting policy.

[0140] The centralized rate limiting module executes a centralized rate limiting strategy when the number of requests per second of a local Ingress Pod exceeds the configured centralized rate limiting parameters. It collects traffic information from each Ingress Pod and performs global traffic analysis and control.

[0141] like Figure 2 The diagram shown illustrates the overall architecture of a precise rate limiting device for Kubernetes Ingress in this embodiment. This embodiment provides a precise rate limiting device for Kubernetes Ingress, employing an asynchronous synchronization method for rate limiting data. The synchronization time interval can be configured according to actual needs. More importantly, centralized rate limiting counting is only triggered in scenarios requiring precise rate limiting. This mechanism effectively reduces the frequency of access to the central rate limiting service, alleviating its burden, while also reducing network transmission and data processing overhead, thus improving the overall efficiency and resource utilization of the system.

[0142] Therefore, the precise rate limiting device for Kubernetes Ingress in this embodiment has the following technical effects:

[0143] I. Improve performance and stability

[0144] This embodiment presents a precise rate limiting device for Kubernetes Ingress, combining local and centralized rate limiting. In daily traffic processing, local rate limiting is primarily relied upon. Since local rate limiting does not require frequent interaction with external centralized rate limiting services, it reduces network latency and data transmission overhead, thereby improving system performance.

[0145] For example, suppose a Kubernetes Ingress gateway deploys 10 Pods, each with local rate limiting capabilities. When traffic is evenly distributed, each Pod can quickly control traffic according to its local rate limiting rules without waiting for a response from a centralized service. This significantly reduces system response time and improves overall performance.

[0146] The existence of local rate limiting also provides stability assurance for the system. Even if the central rate limiting service fails or there are network connection problems, local rate limiting can still continue to work, ensuring that basic traffic control functions are not affected.

[0147] For example, even if the central rate limiting service goes down, each Pod can still limit traffic according to its local rate limiting configuration, preventing the entire system from crashing due to excessive traffic and thus ensuring system stability.

[0148] II. Improve the accuracy of traffic limiting

[0149] Centralized rate limiting plays a crucial role in this invention. While local rate limiting can handle most situations, in scenarios with uneven traffic distribution, some Pods may experience unexpected traffic spikes. In such cases, the centralized rate limiting service collects traffic information from each Pod, performs global traffic analysis and control, and can precisely limit overall traffic.

[0150] For example, when a Pod receives a large amount of traffic due to some reason (such as network fluctuations or unbalanced load balancing strategies), the centralized rate limiting service can detect it in time and adjust the rate limiting accordingly, ensuring that the traffic of the entire system does not exceed the set rate limit value, thereby improving the accuracy of rate limiting.

[0151] The asynchronous synchronization mechanism of rate limiting data further enhances the accuracy of rate limiting. By periodically synchronizing local rate limiting data to the centralized rate limiting service, the centralized service can understand the traffic status of each Pod in real time and adjust the rate limiting strategy according to the actual situation.

[0152] For example, suppose that the traffic to a certain Pod suddenly increases within a certain period of time. Local rate limiting may not be able to adjust the rate limiting strategy in time. However, through an asynchronous synchronous mechanism, a centralized rate limiting service can quickly obtain this information and make precise rate limiting adjustments to the Pod, avoiding system overload due to sudden traffic surges.

[0153] III. Reduce reliance on central traffic limiting services

[0154] The asynchronous synchronization mechanism of rate limiting data ensures that centralized rate limiting counting is triggered only when precise rate limiting is required. This means that most of the time, the system primarily relies on local rate limiting for traffic control, reducing the frequency of access to the central rate limiting service. For example, when traffic is low and evenly distributed, local rate limiting can fully meet the traffic control needs without frequent interaction with the central rate limiting service, thus reducing dependence on the central rate limiting service.

[0155] The predictive mechanism of the central rate limiting service also helps reduce reliance. By calculating parameters such as average QPS, maximum QPS fluctuation, and rate limiting QPS margin, a rate limiting prediction value is generated, and this prediction value has a certain validity period. Within the validity period, the system can make rate limiting decisions based on the prediction value without frequently accessing the central rate limiting service.

[0156] For example, suppose the central rate limiting service predicts that the traffic of a certain Pod will not exceed the set rate limiting value within the next 5 seconds. Then, within those 5 seconds, the Pod can perform local rate limiting based on the prediction value without having to request the central rate limiting service again, thereby further reducing its dependence on the central rate limiting service.

[0157] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. The electronic device includes a processor and a memory. The memory is used to store a computer-executable program. When the computer program is executed by the processor, the processor executes an embodiment of a precise rate limiting method applicable to Kubernetes Ingress.

[0158] like Figure 3 As shown, the electronic device is embodied in the form of a general-purpose computing device. There can be one or more processors working collaboratively. This invention also does not preclude distributed processing, meaning that processors can be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but can also be the sum of multiple physical devices.

[0159] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0160] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0161] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data with external devices. The I / O interface can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0162] It should be understood that Figure 3The electronic device shown is merely one example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as displays, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0163] Figure 4 This is a schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. Figure 4 As shown, a computer-readable recording medium stores a computer-executable program. When executed, the computer-executable program implements a precise rate limiting method for Kubernetes Ingress according to an embodiment of the present invention. The computer-readable recording medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable recording medium may also be any readable medium other than a readable recording medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable recording medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0164] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0165] From the above description of the embodiments, those skilled in the art will readily understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. included in the system. The present invention can also be implemented by computer software executing the methods of the present invention, for example, by control software executed by a microprocessor, electronic control unit, client, server, etc. However, it should be noted that the computer software executing the methods of the present invention is not limited to execution in one or a specific set of hardware entities; it can also be implemented in a distributed manner by unspecified hardware. For computer software, the software product can be stored on a computer-readable recording medium (such as a CD-ROM, USB flash drive, portable hard disk, etc.) or distributed across a network, as long as it enables electronic devices to execute the methods according to the present invention.

[0166] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A precise rate limiting method for Kubernetes Ingress, characterized in that, include: Configure local rate limiting parameters and centralized rate limiting parameters; By counting the number of requests per second in the local Ingress Pod and comparing it with the configured local rate limiting parameters, it is determined whether to execute the local rate limiting policy. When the number of requests per second of a local Ingress Pod exceeds the configured centralized rate limiting parameter, a centralized rate limiting policy is executed, and traffic information of each Ingress Pod is collected for global traffic analysis and control.

2. The precise rate limiting method for Kubernetes Ingress according to claim 1, characterized in that, The configuration of local and centralized rate limiting parameters includes: configuring the centralized rate limiting parameter limit and the local maximum rate limiting parameter max. The execution of the local rate limiting policy and the centralized rate limiting policy includes: If the number of requests per second in the local Ingress Pod is less than the centralized rate limiting parameter limit, then local rate limiting will not be executed. If the number of requests per second of the local Ingress Pod is greater than or equal to the local maximum rate limiting parameter max, then local rate limiting will be executed directly. The centralized rate limiting policy is triggered when the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit, but less than the local maximum rate limiting parameter max.

3. The precise rate limiting method for Kubernetes Ingress according to claim 2, characterized in that, The step of counting the number of requests per second in the local Ingress Pod and comparing it with the configured local rate limiting parameters to determine whether to execute the local rate limiting policy includes: When a traffic request arrives at the local Ingress Pod, the service identifier and the current second-level timestamp are concatenated to generate a rate limiting key; Perform an atomic increment operation on the rate-limiting key in shared memory and return the current count current_qps; If current_qps < limit, allow the traffic requests; If limit ≤ cur rent_qps < max, trigger centralized rate limiting strategy; If current_qps ≥ max, the traffic request is rejected directly.

4. A precise rate limiting method for Kubernetes Ingress according to claim 2, characterized in that, The centralized rate limiting strategy is triggered when the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter `limit`, but less than the local maximum rate limiting parameter `max`. The centralized rate limiting parameter limit is generated by the centralized rate limiting strategy, and the validity period of the centralized rate limiting parameter limit is also generated. When the number of requests per second of the local Ingress Pod is greater than or equal to the centralized rate limiting parameter limit, and less than the local maximum rate limiting parameter max; Further determine whether the current limiting parameter "limit" has expired; If the judgment result is yes, the centralized rate limiting strategy is triggered; if the judgment result is no, the local rate limiting strategy is executed.

5. A precise rate limiting method for Kubernetes Ingress according to claim 4, characterized in that, The centralized rate limiting strategy includes: Receive rate limiting request: Obtain the request parameters from the Ingress Pod, including cur_limit, cur_qps, limit_key, and cur_timestamp. cur_limit represents the local current rate limiting threshold, cur_qps represents the current second-level QPS statistical value, limit_key represents the rate limiting key, and cur_timestamp represents the precise timestamp of the current traffic request; Aggregate global QPS: Query the current QPS of all Ingress Pods in the cluster according to the limit_key, and calculate the total request volume of this interface; Rate limiting decision: If the global QPS + (cur_qps - the already counted QPS of this Pod) ≤ GLOBAL_LIMIT, where GLOBAL_LIMIT represents the global rate limiting threshold, then allow the current request to pass, otherwise, reject the request; Return the result: Return a boolean value indicating whether rate limiting is performed to the traffic requestor.

6. A precise rate limiting method for Kubernetes Ingress according to claim 5, characterized in that, Include: Collect historical QPS data: Obtain the global QPS sequence of this interface in the recent N time windows based on the limit_key; Calculate the benchmark average QPS (avg_qps): The arithmetic mean of the past N windows, and the maximum fluctuation value (max_delta): The historical maximum value of the current QPS - avg_qps; Predict future traffic: Optimistic prediction: predicted_qps = avg_qps + 0.5 * max_delta, pessimistic prediction: predicted_qps = avg_qps + max_delta; Generate the prediction result: If predicted_qps ≤ GLOBAL_LIMIT, return allow to pass and set the prediction validity period; Otherwise, return reject and mark that the data needs to be forced to be calibrated.

7. A precise rate limiting method for Kubernetes Ingress according to claim 6, characterized in that, The prediction validity period is dynamically adjusted: If max_delta / avg_qps < N1%, the validity period is extended to T1 seconds; If max_delta / avg_qps ≥ N2%, the validity period is shortened to T2 seconds, where N1 and N2 are preset values, N1 is less than N2, and T1 and T2 are preset values.

8. A precise rate limiting method for Kubernetes Ingress according to claim 4, characterized in that, Include: Each Ingress Pod reports the current QPS data of all its rate limiting interfaces to the central rate limiting service that executes the centralized rate limiting policy at a fixed time interval; The central rate limiting service receives the QPS data reported by each Pod, clusters and sums them according to the limit_key, and obtains the cluster-level total QPS of each rate limiting interface; Compare the aggregated total QPS with the preset global rate limiting threshold, and update the rate limiting status of each interface; Synchronize the updated rate limiting policy to each Ingress Pod.

9. A precise rate limiting device for Kubernetes Ingress, characterized in that, Include: Rate limiting parameter configuration module, which configures local rate limiting parameters and centralized rate limiting parameters; Local rate limiting module, which counts the number of requests per second of the local Ingress Pod and compares it with the configured local rate limiting parameters to determine whether to execute the local rate limiting policy; The centralized rate limiting module executes a centralized rate limiting strategy when the number of requests per second of a local Ingress Pod exceeds the configured centralized rate limiting parameters. It collects traffic information from each Ingress Pod and performs global traffic analysis and control.

10. An electronic device, comprising a processor and a memory, the memory being used to store a computer-executable program, characterized in that, When the computer program is executed by the processor, the processor performs a precise rate limiting method for Kubernetes Ingress as described in any one of claims 1-8.

Images