A distributed current limiting control method and related apparatus

Abstract

The application provides a distributed flow limiting control method and related device, and relates to the field of flow limiting. After receiving a user request, if it is judged that flow limiting operation needs to be performed on the user request, flow limiting operation is performed on the user request based on a flow limiting strategy, which avoids the problem that concurrent resource competition is caused by the fact that a CDN cache cluster responds to high concurrent access flow, and then the bandwidth of an enterprise core application is occupied, and improves the service quality of the CDN. In addition, in the application, the flow limiting strategies of different target cache clusters are the same or different, and the flow limiting strategies of different applications configured in the target cache cluster are the same or different, so that the flow limiting demand of different clusters or applications can be adapted, and the application range of the distributed flow limiting control method is improved. The application also supports limiting the request rate, limiting the number of concurrent connections and limiting the number of requests in a time window, and can meet different flow limiting demand scenarios.

Description

Technical Field

[0001] This application relates to the field of current limiting, and more specifically, to a distributed current limiting control method and related apparatus. Background Technology

[0002] A Content Delivery Network (CDN) is a network service based on a distributed architecture. By caching and distributing content to server nodes across the country, it can improve the speed of content delivery, effectively optimize user experience, reduce network traffic, and increase network throughput and scalability.

[0003] During the use of a CDN cluster, excessively high concurrent access traffic can lead to competition for concurrent resources, which may cause the bandwidth of the enterprise's core applications to be squeezed, thereby affecting the quality of CDN services. Summary of the Invention

[0004] In view of this, this application provides a distributed rate limiting control method and related apparatus to solve the problem that excessive concurrent access traffic can lead to concurrent resource contention, which may cause the bandwidth of enterprise core applications to be squeezed, thereby affecting the quality of CDN services.

[0005] To solve the above-mentioned technical problems, this application adopts the following technical solution:

[0006] A distributed rate limiting control method is applied to any target cache cluster in a distributed content delivery network (CDN) cache cluster; the distributed rate limiting control method includes:

[0007] If a user request is received, rate limiting configuration data is obtained; the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster and / or the rate limiting policy corresponding to the application in the target cache cluster; the rate limiting policies of different target cache clusters may be the same or different; the rate limiting policies of different applications configured in the target cache cluster may be the same or different; the rate limiting policy includes limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests in a time window;

[0008] Determine whether the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster;

[0009] If included, the cluster rate limiting service is invoked to obtain the rate limiting configuration parameters of the target cache cluster, and rate limiting operation is performed on the user request based on the rate limiting configuration parameters of the target cache cluster;

[0010] If not included, obtain the target application accessed by the user request carried in the user request. If the rate limiting configuration data includes the rate limiting policy corresponding to the target application, perform rate limiting operation on the user request based on the rate limiting policy corresponding to the target application.

[0011] Optionally, the process of storing the rate limiting configuration data includes:

[0012] Monitor whether there are any content updates in the rate limiting configuration directory corresponding to the target cache cluster;

[0013] If so, obtain the rate limiting configuration data corresponding to the target cache cluster from the configuration center;

[0014] Respond to the restart command from the management terminal and perform a restart operation;

[0015] After a successful restart, the cache service process is loaded and intercepted to read the rate limiting configuration data and load it into memory.

[0016] Optionally, based on the rate limiting configuration parameters of the target cache cluster, rate limiting operations are performed on the user requests, including:

[0017] Based on the aforementioned rate limiting configuration parameters, determine whether to trigger a rate limiting operation;

[0018] If so, intercept the user request and return a preset rate-limiting fault code.

[0019] Optionally, based on the rate limiting configuration parameters, determining whether to trigger a rate limiting operation includes:

[0020] Statistics on currently approved requests;

[0021] Determine whether the allowed request data meets the rate limiting conditions corresponding to the rate limiting configuration parameters;

[0022] If the conditions are met, it is determined that a rate limiting operation is triggered.

[0023] Optionally, it also includes:

[0024] If it is determined that no rate limiting operation has been triggered, determine whether the target data corresponding to the user request is stored locally;

[0025] If stored, output the target data;

[0026] If not stored, the target data is obtained based on the source retrieval operation, and the target data is output.

[0027] Optionally, based on the rate limiting policy corresponding to the target application, rate limiting operations are performed on the user requests, including:

[0028] If the rate limiting policy corresponding to the target application is to limit the request rate, a rate limiting instance is created based on the rate limiting parameter in the rate limiting configuration data;

[0029] Define a rate limiting key to obtain the rate limiting return value during the rate limiting process;

[0030] Determine whether to trigger a rate limiting operation based on the rate limiting return value;

[0031] If a rate limiting operation is triggered, the user request will be intercepted, and a preset rate limiting fault code will be returned.

[0032] If rate limiting is not triggered, determine whether delay processing is needed based on the configured leaky bucket parameters;

[0033] If it is determined that delay processing is required, a delay-limiting operation will be performed.

[0034] Optionally, the update process of the rate limiting configuration parameters of the target cache cluster includes:

[0035] The updated rate limiting configuration parameters corresponding to the target cache cluster are obtained from the preset interface of the management terminal, and the updated rate limiting configuration parameters are used to replace the rate limiting configuration parameters of the target cache cluster stored in memory.

[0036] A distributed rate limiting control device is applied to any target cache cluster in a distributed content delivery network (CDN) cache cluster; the distributed rate limiting control device includes:

[0037] The data acquisition module is used to acquire rate limiting configuration data upon receiving a user request. The rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster and / or the rate limiting policy corresponding to the application in the target cache cluster. The rate limiting policies of different target cache clusters may be the same or different. The rate limiting policies of different applications configured in the target cache cluster may be the same or different. The rate limiting policy includes limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests in a time window.

[0038] The policy judgment module is used to determine whether the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster;

[0039] The first rate limiting module is used to, if applicable, call the cluster rate limiting service to obtain the rate limiting configuration parameters of the target cache cluster, and perform rate limiting operations on the user requests based on the rate limiting configuration parameters of the target cache cluster;

[0040] The second rate limiting module is used to, if not included, obtain the target application accessed by the user request carried in the user request, and if the rate limiting configuration data includes the rate limiting policy corresponding to the target application, perform rate limiting operation on the user request based on the rate limiting policy corresponding to the target application.

[0041] An electronic device includes at least one processor and a memory connected to the processor, wherein:

[0042] The memory is used to store computer programs;

[0043] The processor is used to execute the computer program so that the electronic device can implement the above-described distributed current limiting control method.

[0044] A computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the aforementioned distributed current limiting control method.

[0045] This application provides a distributed rate limiting control method and related apparatus. Upon receiving a user request, if it is determined that rate limiting is required, the rate limiting operation is executed based on a rate limiting policy. This avoids the problem of excessive concurrent access traffic from the CDN cache cluster leading to concurrent resource contention, which in turn causes bandwidth constraints for core enterprise applications, thus improving CDN service quality. Furthermore, in this invention, the rate limiting policies for different target cache clusters can be the same or different, and the rate limiting policies for different applications configured within the target cache clusters can also be the same or different. This adapts to the rate limiting needs of different clusters or applications, increasing the application breadth of the distributed rate limiting control method. Additionally, this invention supports limiting request rate, limiting concurrent connections, and limiting the number of requests within a time window, meeting various rate limiting scenarios. Moreover, when specifically executing rate limiting, this invention first determines whether to execute a cluster rate limiting policy, and then determines whether to execute an application rate limiting policy. This allows for rate limiting control from different perspectives based on requirements, expanding the scope of rate limiting applications. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 A flowchart of a distributed current limiting control method provided in an embodiment of the present invention;

[0048] Figure 2 This invention provides an overall architecture for distributing rate limiting configurations.

[0049] Figure 3 This is a schematic diagram of a request interception process provided in an embodiment of the present invention;

[0050] Figure 4 A flowchart illustrating a method for limiting request rate according to an embodiment of the present invention;

[0051] Figure 5 A flowchart illustrating a method for loading the latest rate limiting configuration in a distributed cache cluster, as provided in an embodiment of the present invention;

[0052] Figure 6 A flowchart of a current limiting configuration hot update process is provided for an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of a distributed current limiting control device provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] To enable those skilled in the art to better understand the present invention, the technical terms used in the present invention are explained below.

[0056] CDN (Content Delivery Network): A CDN is a network architecture designed to improve the efficiency and performance of network content delivery. It caches content near users by deploying server nodes globally, enabling users to access content from the nearest location. CDNs can effectively improve the speed and quality of content delivery, reduce network latency and congestion, and enhance the user experience.

[0057] A cache cluster is a cluster of multiple servers used to store and provide content caching services. In a CDN system, cache clusters are typically deployed across multiple nodes globally to store content caches close to users, enabling localized access and accelerated delivery.

[0058] Rate limiting: Rate limiting is a network traffic management mechanism used to control the rate and number of network requests to protect server resources and ensure service stability. In CDN systems, rate limiting can help manage and control network traffic, prevent server overload, reduce network congestion, and improve service availability and performance.

[0059] In practical applications, a Content Delivery Network (CDN) is a network service based on a distributed architecture. By caching and distributing content to server nodes across the country, it can improve the speed of content transmission, effectively optimize user experience, reduce network traffic, and increase network throughput and scalability.

[0060] Large enterprises with cross-regional business operations often face issues such as network latency, network congestion, and bandwidth bottlenecks. To ensure business stability, user experience, and data transmission security, more and more enterprises are choosing to build their own internal content delivery networks (CDNs). However, during CDN cluster construction, access traffic varies across different regions, and even within the same region, access traffic can differ significantly at different times and for different applications. Excessively high concurrent access traffic can lead to competition for resources, potentially squeezing the bandwidth of core applications and impacting the quality of CDN services. Therefore, effective rate limiting measures are needed to manage and protect CDN server resources, improve service quality, and ensure the reliability and stability of CDN acceleration.

[0061] In specific rate limiting control, the distribution of rate limiting configurations presents a challenge. Enterprise-level CDNs often have hundreds or even thousands of cache servers deployed across the country, with varying rate limiting configurations for each cache cluster. Centralized management of these configurations, requiring manual updates, management, and maintenance by cluster administrators for each cache cluster, makes consistency difficult to guarantee and complicates operations. Therefore, this invention proposes a distributed rate limiting configuration center to uniformly manage and automatically schedule rate limiting configurations across all cache clusters, ensuring data consistency and order consistency to prevent rate limiting service anomalies or unavailability due to configuration issues.

[0062] Furthermore, regarding the design of rate limiting strategies, different applications have varying rate limiting requirements due to different scenarios. How can we ensure the richness of rate limiting strategies to meet diverse needs? Additionally, many applications access enterprise-grade CDNs; how can we ensure that each application has an independent rate limiting strategy that does not interfere with others? To address this, this invention proposes three different rate limiting strategies: limiting request rate, limiting concurrent connections, and limiting the number of requests within a time window. These strategies can be flexibly configured according to application and cluster requirements. Moreover, the rate limiting configuration for each application is independently implemented through the host field in the HTTP (Hypertext Transfer Protocol) request header, achieving rate limiting isolation between different applications, ensuring that they do not affect each other, and guaranteeing system stability and reliability.

[0063] To address the issue of hot updates to rate limiting configurations, when rate limiting configurations in a cache cluster need to be updated, it is usually necessary to stop or restart the cache server, which can lead to cache service interruption. This invention addresses this problem by designing a customized API (Application Programming Interface) to enable hot updates of rate limiting configurations in the memory dictionary. This allows for instant updates of rate limiting configurations without stopping or restarting the server, improving system flexibility and maintainability, and ensuring user experience and business continuity when rate limiting configuration updates take effect.

[0064] Based on the above, one embodiment of the present invention provides a distributed rate limiting control method and related apparatus. In this application, upon receiving a user request, if it is determined that rate limiting is required, the rate limiting operation is performed on the user request based on the rate limiting policy. This avoids the problem of excessive concurrent access traffic from the CDN cache cluster leading to concurrent resource contention, which in turn causes the bandwidth of the enterprise's core applications to be squeezed, thus improving the CDN service quality. Furthermore, in this invention, during rate limiting configuration, the rate limiting policies for different target cache clusters may be the same or different, and the rate limiting policies for different applications configured in the target cache clusters may be the same or different. This adapts to the rate limiting needs of different clusters or applications, increasing the application breadth of the distributed rate limiting control method. Additionally, this invention supports limiting request rate, limiting concurrent connections, and limiting the number of requests within a time window, meeting different rate limiting scenarios. Moreover, when specifically executing rate limiting, this invention first determines whether to execute a cluster rate limiting policy, and then determines whether to execute an application rate limiting policy. This allows for rate limiting control from different perspectives based on requirements, expanding the scope of rate limiting application scenarios.

[0065] Reference Figure 1 One embodiment of the present invention provides a distributed rate limiting control method, which is applied to any target cache cluster in a distributed content delivery network (CDN) cache cluster.

[0066] In real-world scenarios, such as Figure 2 As shown, distributed cache clusters 1-n are the CDN cache clusters in this embodiment of the invention, deployed using a distributed structure. Each distributed cache cluster is configured with a cache service to provide cache services to users, such as providing data, videos, and other content. The cache service includes a proxy service, which performs rate limiting operations, including rate limiting configuration updates and rate limiting control.

[0067] In practical applications, for the aforementioned distributed cache clusters 1-n, a rate limiting configuration center is designed to uniformly manage and automatically schedule the rate limiting configurations of each layer of the enterprise-level CDN cache clusters. This ensures the consistency and order of rate limiting configurations, guaranteeing the high availability of the rate limiting service. In real-world scenarios, the rate limiting configuration data for each distributed cache cluster may be the same or different. For example, the rate limiting configuration data for distributed cache clusters in different provinces may differ, while the rate limiting configuration data for distributed cache clusters within the same province may be the same. The specific settings can be configured according to the actual situation.

[0068] In specific configuration, administrators control service nodes by calling the management terminal's node via the API service. The API service can function as a web-based configuration interface, where administrators configure rate limiting policies. Figure 2As shown, when configuring rate limiting policies, platform administrators configure cluster-level or application-level rate limiting policies on the platform management terminal to restrict the request volume of a server or a specific application. The platform management terminal generates a rate limiting policy configuration file, and the rate limiting configuration data distributed by the configuration center is stored in JSON (JavaScript Object Notation) file format. The rate limiting configuration includes fields such as cluster name, application name, rate limiting type, and specific rate limiting parameters. The configuration center synchronizes the configuration to the cache cluster through a configuration distribution mechanism to implement the actual rate limiting operation.

[0069] To elaborate, administrators configure flow control policies for cache cluster servers or specific access applications on the management console, and persistently store the specific configurations in databases and file storage services.

[0070] After receiving the user's configuration, the node configuration service on the management side generates a flow control policy configuration file and synchronizes it to the distributed cache cluster through the distribution mechanism of the configuration center. The flow control policy configuration file distributed by the configuration center can use a master-slave backup method. Figure 2 In this setup, the master is the primary node and the follower is the secondary node. The master and follower nodes contain identical configurations. The master-slave backup is configured solely to ensure the reliability of configuration distribution by executing the secondary node's configuration if the primary node fails or malfunctions. The master and follower configurations include the configuration details for each CDN node, such as node1, node2, node3, etc.

[0071] The proxy service in the distributed caching cluster monitors the configuration center in real time. When it detects changes in the configuration center's content, it updates its own rate-limiting configuration. The caching service loads the cluster's rate-limiting configuration into memory upon startup to implement rate limiting for user access. During rate limiting, if a user request is allowed but the requested content is not stored locally in the distributed caching cluster, a callback operation is executed to retrieve the corresponding content from the management end and provide it to the user.

[0072] Distributed rate limiting control methods include:

[0073] S11. If a user request is received, obtain the rate limiting configuration data.

[0074] The rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster and / or the rate limiting policy corresponding to the application in the target cache cluster.

[0075] For distributed cache clusters 1-n, different rate limiting policies can be configured for each cluster, or some clusters can be set to the same rate limiting policy, while others can be set to the same rate limiting policy. For example, the rate limiting policy for clusters within the same province is the same, while the rate limiting policy for clusters in different provinces is different.

[0076] Besides configuring rate limiting policies from a cluster perspective, rate limiting policies can also be configured from an application perspective. For example, application A can be configured with one rate limiting policy, and application B with another. Furthermore, rate limiting policies can be configured simultaneously for both clusters and applications. For instance, a rate limiting policy can be configured for application A in cluster 1, and a corresponding rate limiting policy can be configured for application B in cluster 2. The specific configuration of cluster and application rate limiting policies can be tailored to the administrator's needs. In the final configured rate limiting policy, the rate limiting policies for different target cache clusters may be the same or different, and the rate limiting policies configured for different applications within the target cache clusters may also be the same or different, to meet actual rate limiting requirements.

[0077] In addition, this invention supports a variety of rate limiting strategies, such as limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests within a time window.

[0078] Request rate limiting limits traffic based on requests per second (PS), suitable for scenarios requiring a limit on the number of requests, such as limiting the number of requests per IP address (Internet Protocol) within a certain time period to prevent malicious requests or protect resources. Concurrent connection limiting limits traffic based on the number of concurrent connections, suitable for scenarios requiring a limit on the number of concurrent connections to prevent excessive connections from exhausting system resources. Time window request limiting limits traffic based on a custom counter, allowing for flexible configuration according to specific business logic.

[0079] For both clusters and applications, you can configure the above-mentioned rate limiting strategies: limiting request rate, limiting concurrent connections, or limiting the number of requests within a time window. For example, for cluster 1, configure its rate limiting strategy to limit request rate. Similarly, for application 2, configure its rate limiting strategy to limit concurrent connections. And for application 3 within cluster 2, configure its rate limiting strategy to limit the number of requests within a time window.

[0080] S12. Determine whether the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster; if it does, proceed to step S13; if it does not, proceed to step S14.

[0081] Specifically, such as Figure 3As shown, users can make routing requests via HTTP, and the distributed caching cluster receives these requests. The distributed caching cluster first intercepts the request during the access phase, determining whether cluster flow control is configured. Specifically, this determination can be made by checking if the flow control configuration data includes the flow control policy corresponding to the target caching cluster. If it does, cluster-level flow control is applied. If not, it checks if application flow control is configured. Specifically, this determination can be made by checking if the flow control configuration data includes the flow control policy corresponding to the target application accessed by the user. If it does, application-level flow control is applied. If neither cluster flow control nor application flow control is configured, the user request is allowed, and the cluster responds to the request, executing the corresponding operation. If the requested data is not stored locally on the cluster, a back-to-origin operation is performed from the management end, the retrieved data is stored locally, and then fed back to the user.

[0082] S13. Call the cluster rate limiting service to obtain the rate limiting configuration parameters of the target cache cluster, and perform rate limiting operation on the user request based on the rate limiting configuration parameters of the target cache cluster.

[0083] Specifically, the rate limiting configuration parameters for the target cache cluster, such as Figure 3 The flow control configuration can be retrieved from memory. The cluster rate limiting service is invoked to obtain the rate limiting configuration parameters from memory. Taking request rate limiting as an example, the rate limiting configuration parameters can be specific rate limit values. Subsequently, the cluster rate limiting service performs rate limiting operations on the user requests based on the rate limiting configuration parameters of the target cache cluster.

[0084] In practice, the cluster rate limiting service determines the rate limiting type based on the rate limiting configuration parameters, such as limiting request rate, limiting concurrent connections, or limiting the number of requests within a time window. Then, it calls the rate limiting module to perform the rate limiting operation. During the specific rate limiting operation, it determines whether to trigger the rate limiting operation based on the rate limiting configuration parameters. Specifically, it can count the received requests and allowed requests, using the rate limiting strategy as the time window request count. It can determine whether the allowed request data meets the rate limiting conditions of the rate limiting configuration parameters, such as whether the traffic or number of allowed requests reaches the maximum value of the time window request count. If it does, it means the current request should not be allowed, and a rate limiting operation is triggered. At this time, the user request is intercepted, and a preset rate limiting fault code, such as a 429 status code, is fed back to the user so that the user understands that the current request is rate-limited and cannot be executed.

[0085] If rate limiting is not triggered, the request is allowed, and the cluster executes the operation corresponding to the user request. During execution, it checks whether the target data corresponding to the user request is stored locally. If stored, the target data is output to the user. If not stored, the target data is retrieved through a back-to-origin operation and then output. For example, if a user requests this month's bill data, the cluster can check whether the user's bill for this month is stored locally. If it is, the bill is sent to the user. If not, the bill can be retrieved from the management system through a back-to-origin operation, returned to the user, and stored locally.

[0086] S14. Obtain the target application accessed by the user request carried in the user request. If the rate limiting configuration data includes the rate limiting policy corresponding to the target application, perform rate limiting operation on the user request based on the rate limiting policy corresponding to the target application.

[0087] Specifically, if the rate limiting configuration data does not include the rate limiting policy corresponding to the target cache cluster, then it is necessary to continue to determine whether application flow control is configured. For specific implementation, please refer to the corresponding instructions above.

[0088] Reference Figure 4 Taking the rate limiting policy corresponding to the target application as an example to limit the request rate, the application-level rate limiting process is introduced. If application flow control is determined to exist, the target application accessed by the user request can be obtained from the user request header. Specifically, the application host information can be obtained from the user request header. Then, it can be determined whether the rate limiting configuration data includes the rate limiting policy corresponding to the target application host. If so, the specific rate limiting type can be obtained, such as limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests within a time window, as mentioned above. Then, according to the rate limiting policy corresponding to the target application, the rate limiting module can be called to perform rate limiting operations on the user request.

[0089] To elaborate, the rate limiting process is as follows:

[0090] The rate limiting parameters are retrieved from the host's memory based on the rate limiting configuration data. Based on these parameters, rate limiting parameters are defined, and a rate limiting instance is created. A rate limiting key is defined to perform rate limiting processing. During the rate limiting process, a rate limiting return value is obtained. Finally, the return value is used to determine whether rate limiting is triggered. If triggered, the request is intercepted and a preset rate limiting fault code (such as 429 status code) is returned. Otherwise, the leaky bucket parameters are checked to determine whether delay processing is needed. If delay processing is determined, a delayed rate limiting operation is performed.

[0091] The leaky bucket parameter is a parameter in the leaky bucket algorithm, which is the core algorithm for rate limiting. Core rate limiting algorithms mainly include the token bucket algorithm and the leaky bucket algorithm. Both of these algorithms are flow control algorithms based on a fixed time window, which can limit and adjust requests.

[0092] The token bucket algorithm maintains a token bucket containing a certain number of tokens. Each token represents permission for a request, and tokens are added to the bucket at a set rate. When a request arrives, if there are enough tokens in the bucket, the request is allowed to proceed, consuming one token; otherwise, the request is restricted. By controlling the rate at which tokens are generated and consumed, rate limiting can be achieved.

[0093] The leaky bucket algorithm simulates a leaky bucket, where requests are viewed as water droplets, and these droplets are injected into the bucket at a fixed rate. When a request arrives, if the bucket is not full, the request is allowed to pass through and consumes one droplet from the bucket; if the bucket is full, the request is restricted. By controlling the injection and leakage rates, rate limiting of requests can be achieved.

[0094] In rate limiting implementation, a rate limiting strategy needs to be defined in advance, including limit parameters such as request rate and concurrent connection count. These limit parameters can be configured according to system characteristics and business needs to achieve precise rate limiting control. Whenever a request arrives, a script will perform traffic statistics on the request. By counting the number of requests and the time window, it can be determined whether the rate limiting strategy's limits have been exceeded. If the limits are exceeded, the request is rejected; if the limits are not exceeded, the request is allowed to pass and the traffic statistics are updated.

[0095] The leaky bucket algorithm also includes a delayed rate limiting strategy. If the configured leaky bucket parameters indicate that delayed processing is necessary, then the corresponding delayed rate limiting operation is performed. Specifically, in the leaky bucket algorithm, when the bucket is full, newly arriving requests can be either directly blocked and rejected or delayed. In this case, a delay is made, and then the rate limiting strategy is analyzed again. If it is determined that rate limiting is still necessary, the request is blocked, and a 429 status code is returned.

[0096] In the above embodiments, rate limiting operations are performed at both the cluster and application levels. Requests are intercepted during the access phase. The presence of cluster rate limiting configuration data in shared memory determines whether cluster rate limiting is configured on the node. Similarly, the presence of application rate limiting configuration data in shared memory determines whether rate limiting should be implemented. If a rate limiting policy is configured, the rate limiting module is invoked. This module includes different rate limiting types, such as limiting request rate, limiting concurrent connections, and limiting the number of requests within a time window. Based on the rate limiting type of the request, the corresponding rate limiting method is called to determine if the limit is exceeded. If the limit is exceeded, the request is intercepted and returned; otherwise, the request is allowed to proceed.

[0097] In addition, rate limiting operations can be configured for both the cluster and the application simultaneously, such as rate limiting for application A in cluster 1. First, it is determined whether the request meets the cluster flow control requirements. If it does, rate limiting is applied according to the cluster flow control. If rate limiting determines that the request needs to be allowed, application flow control is then applied to determine whether rate limiting or allowing the request is necessary.

[0098] In this embodiment, upon receiving a user request, if it is determined that rate limiting is required, the rate limiting operation is executed based on the rate limiting policy. This avoids the problem of excessive concurrent access traffic from the CDN cache cluster leading to concurrent resource contention, which in turn causes bandwidth constraints for core enterprise applications, thus improving CDN service quality. Furthermore, in this invention, the rate limiting policies for different target cache clusters can be the same or different, and the rate limiting policies for different applications configured within the target cache clusters can also be the same or different. This adapts to the rate limiting needs of different clusters or applications, increasing the application breadth of the distributed rate limiting control method. Additionally, this invention supports limiting request rate, limiting concurrent connections, and limiting the number of requests within a time window, meeting various rate limiting scenarios. Moreover, when specifically executing rate limiting, this invention first determines whether to execute a cluster rate limiting policy, and then determines whether to execute an application rate limiting policy. This allows for rate limiting control from different perspectives based on requirements, expanding the scope of rate limiting applications.

[0099] Furthermore, current rate limiting methods are based on Nginx-based rate limiting management schemes. In practical applications, Nginx itself provides a rate limiting module to implement rate limiting, but this module is typically configured and effective on a single server, failing to achieve centralized management and dynamic deployment of rate limiting configurations. This embodiment of the invention achieves centralized management of multiple distributed cache clusters and dynamic deployment based on actual needs through a management terminal and a configuration center, improving the unified management capabilities of the cache clusters.

[0100] In addition, the design includes three rate-limiting strategies: limiting request rate, concurrent connections, and requests within a time window. Furthermore, rate-limiting configurations for each application are implemented through the host field in the HTTP request header, ensuring isolation between different application rate limits and guaranteeing system stability and reliability.

[0101] The above embodiments illustrate that rate limiting configuration data needs to be stored in memory. This invention describes the process of storing rate limiting configuration data, referring to... Figure 5 In a distributed caching cluster, the proxy service refers to the proxy service within the caching service. The configuration center typically stores rate limiting configuration data in a designated rate limiting configuration directory; the path to this directory varies depending on the cluster. The proxy service monitors the rate limiting configuration directory corresponding to the target caching cluster for updates. Specifically, the distributed caching cluster proxy service constantly monitors the directory using a configured listener class. Once a change is detected in the rate limiting configuration directory, the cluster immediately retrieves the latest rate limiting configuration data from the configuration center and replaces the local rate limiting file, ensuring the cluster always has the latest rate limiting policy and can perform corresponding rate limiting operations based on the configuration.

[0102] After the rate limiting file is updated, status information is sent back to the management terminal. If the update is complete, the management terminal confirms the cluster status. If the cluster update is confirmed, a restart command to restart the cache configuration is issued. Upon receiving this request, the proxy service performs the restart operation. After a successful restart, the cache service performs a loading operation. At this time, the rate limiting script executes a cache service process loading interception operation during the initialization phase, reading the latest rate limiting configuration data. If the data is not empty, the rate limiting configuration data is loaded into memory. The data in memory uses key-value pairs, where the key is the application's unique application host identifier or cluster identifier, and the value is the rate limiting configuration for that application or cluster. After the data is loaded into memory, the cache service continues to execute the loading service. After loading is complete, when the distributed node, i.e., the distributed cache cluster, receives a request, it can determine whether to intercept and execute the rate limiting operation based on whether the cluster or the application to which the request belongs has rate limiting configured. Specifically, when a user initiates a resource access request, the cache cluster node intercepts and judges the request according to the rate limiting policy. If the rate limiting requirements are not met, the request is allowed, and the resource is obtained from the cache node; if the rate limiting requirements are met, access is directly denied. When a cache node has no cache, it requests resources from the origin server.

[0103] If the data content is empty, the step of loading the rate limiting configuration data into memory will not be executed, and the caching service will continue to perform the loading operation.

[0104] Based on this embodiment, the update operation of the rate limiting configuration parameters of the target cache cluster is supported, and the update process includes:

[0105] The updated rate limiting configuration parameters corresponding to the target cache cluster are obtained from the preset interface of the management terminal, and the updated rate limiting configuration parameters are used to replace the rate limiting configuration parameters of the target cache cluster stored in memory.

[0106] Specifically, for Nginx-based rate limiting management schemes, updating rate limiting configurations typically requires modifying the Nginx configuration file and restarting the server, leading to business interruptions and downtime, which in turn causes caching service interruptions. This invention provides a method for updating rate limiting configurations by hot-updating the rate limiting configuration in memory. This achieves instant updates of the rate limiting configuration without stopping or restarting the server, improving system flexibility and maintainability. Specifically, a set of preset ports for the management end, such as an API interface, can be defined. If an administrator needs to update the rate limiting configuration parameters, they can configure the latest rate limiting configuration parameters on the management end's web configuration page. The management end calls the API interface to send the rate limiting configuration to be updated, i.e., it sends the latest rate limiting configuration parameters to the caching service in the cluster. The caching service retrieves the rate limiting configuration parameters to be updated from the sent data and then interacts with the rate limiting configuration dictionary maintained in memory to update the rate limiting configuration parameters. After the update is complete, the interface status is returned to the management end so that the management end is aware of the latest status. The specific interaction process is as follows: Figure 6 As shown.

[0107] This approach allows administrators to hot-update rate limiting configurations via API calls, updating rate limiting policies without restarting the caching service. This significantly reduces system maintenance downtime and improves system availability and stability.

[0108] As can be seen from the above, the embodiments of the present invention provide a distributed rate limiting design method for an enterprise-level content delivery network. The rate limiting configuration data issued by the configuration center is stored in the form of a JSON file. The rate limiting configuration includes fields such as cluster name, application name, rate limiting type, and specific rate limiting parameters, which can meet the different rate limiting scenarios with different rate limiting requirements for clusters or applications, ensuring the richness of rate limiting strategies. Furthermore, setting rate limiting separately for applications or clusters allows clusters or applications to have independent rate limiting strategies that do not affect each other.

[0109] Each cache cluster performs request traffic statistics upon receiving a request. By analyzing the number of requests and the time window, it can determine whether the rate limiting policy has been exceeded. If the limit is exceeded, the node can take appropriate rate limiting measures, such as rejecting requests or delaying processing, to ensure system stability and reliability, achieve traffic control of the cache cluster, protect CDN server resources, improve service quality, and guarantee the reliability and stability of CDN acceleration.

[0110] In this embodiment, in a distributed cluster environment, limiting the system's request traffic can effectively prevent the system from being overloaded and affected by malicious requests, improve the system's processing capacity and response speed, and protect the system's stability and reliability.

[0111] Based on the above embodiments of the distributed rate limiting control method, another embodiment of the present invention provides a distributed rate limiting control device, applied to any target cache cluster in a distributed content delivery network (CDN) cache cluster. (Refer to...) Figure 7 The distributed current limiting control device includes:

[0112] The data acquisition module 11 is used to acquire rate limiting configuration data if a user request is received; the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster and / or the rate limiting policy corresponding to the application in the target cache cluster; the rate limiting policies of different target cache clusters are the same or different; the rate limiting policies of different applications configured in the target cache cluster are the same or different; the rate limiting policy includes limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests in a time window;

[0113] The policy judgment module 12 is used to determine whether the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster;

[0114] The first rate limiting module 13 is used to, if applicable, call the cluster rate limiting service to obtain the rate limiting configuration parameters of the target cache cluster, and perform rate limiting operations on the user request based on the rate limiting configuration parameters of the target cache cluster;

[0115] The second rate limiting module 14 is used to, if not included, obtain the target application accessed by the user request carried in the user request, and if the rate limiting configuration data includes the rate limiting policy corresponding to the target application, perform rate limiting operation on the user request based on the rate limiting policy corresponding to the target application.

[0116] One implementation also includes a storage module, which comprises:

[0117] The monitoring submodule is used to monitor whether there are any content updates in the rate limiting configuration directory corresponding to the target cache cluster;

[0118] The data acquisition submodule is used to, if so, obtain the rate limiting configuration data corresponding to the target cache cluster from the configuration center;

[0119] The restart submodule is used to respond to restart commands from the management end and perform restart operations;

[0120] The data loading submodule is used to perform a cache service process loading interception operation after a successful restart, in order to read the rate limiting configuration data and load the rate limiting configuration data into memory.

[0121] In one implementation, the first current limiting module 13 includes:

[0122] The rate limiting judgment submodule is used to determine whether to trigger a rate limiting operation based on the rate limiting configuration parameters.

[0123] The request interception submodule is used to intercept the user request if the request is true and to return a preset rate limiting fault code.

[0124] In one implementation, the rate limiting judgment submodule is specifically used for:

[0125] The system counts the current allowed requests and determines whether the allowed requests meet the rate limiting conditions corresponding to the rate limiting configuration parameters. If they do, it determines that a rate limiting operation should be triggered.

[0126] In one implementation, a request execution module is further included, which is used to determine whether the target data corresponding to the user request is stored locally if it is determined that no rate limiting operation has been triggered; if it is stored, the target data is output; if it is not stored, the target data is obtained based on the origin retrieval operation and the target data is output.

[0127] In one implementation, the second current limiting module 14 includes:

[0128] The instance creation submodule is used to create a rate limiting instance based on the rate limiting parameters in the rate limiting configuration data when the rate limiting policy corresponding to the target application is to limit the request rate.

[0129] Define a submodule to define the rate limiting key so that the rate limiting return value can be obtained during the rate limiting process;

[0130] The trigger judgment submodule is used to determine whether to trigger a rate limiting operation based on the rate limiting return value.

[0131] The request processing submodule is used to intercept the user request and return a preset rate limiting fault code if a rate limiting operation is triggered.

[0132] The delay judgment submodule is used to determine whether delay processing is needed based on the configured leaky bucket parameters if the rate limiting operation is not triggered.

[0133] The delay processing submodule is used to perform delay rate limiting operations if it is determined that delay processing is required.

[0134] One implementation also includes an update module, used for:

[0135] The updated rate limiting configuration parameters corresponding to the target cache cluster are obtained from the preset interface of the management terminal, and the updated rate limiting configuration parameters are used to replace the rate limiting configuration parameters of the target cache cluster stored in memory.

[0136] In this embodiment, upon receiving a user request, if it is determined that rate limiting is required, the rate limiting operation is executed based on the rate limiting policy. This avoids the problem of excessive concurrent access traffic from the CDN cache cluster leading to concurrent resource contention, which in turn causes bandwidth constraints for core enterprise applications, thus improving CDN service quality. Furthermore, in this invention, the rate limiting policies for different target cache clusters can be the same or different, and the rate limiting policies for different applications configured within the target cache clusters can also be the same or different. This adapts to the rate limiting needs of different clusters or applications, increasing the application breadth of the distributed rate limiting control method. Additionally, this invention supports limiting request rate, limiting concurrent connections, and limiting the number of requests within a time window, meeting various rate limiting scenarios. Moreover, when specifically executing rate limiting, this invention first determines whether to execute a cluster rate limiting policy, and then determines whether to execute an application rate limiting policy. This allows for rate limiting control from different perspectives based on requirements, expanding the scope of rate limiting applications.

[0137] It should be noted that the working process of each module and sub-module in this embodiment is described in the corresponding descriptions in the above embodiments, and will not be repeated here.

[0138] This application also provides an electronic device. It includes at least one processor and a memory connected to the processor, wherein:

[0139] The memory is used to store computer programs;

[0140] The processor is used to execute the computer program so that the electronic device can implement the above-described distributed current limiting control method.

[0141] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the distributed current limiting control methods provided in this application.

[0142] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the distributed rate limiting control methods provided in this application.

[0143] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A distributed current limiting control method, characterized in that, The distributed rate limiting control method is applied to any target cache cluster in a distributed content delivery network (CDN) cache cluster; the method includes: If a user request is received, rate limiting configuration data is obtained; the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster and / or the rate limiting policy corresponding to the application in the target cache cluster; the rate limiting policies of different target cache clusters may be the same or different; the rate limiting policies of different applications configured in the target cache cluster may be the same or different; the rate limiting policy includes limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests in a time window; Determine whether the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster; If included, the cluster rate limiting service is invoked to obtain the rate limiting configuration parameters of the target cache cluster, and rate limiting operation is performed on the user request based on the rate limiting configuration parameters of the target cache cluster; If not included, obtain the target application accessed by the user request carried in the user request. If the rate limiting configuration data includes the rate limiting policy corresponding to the target application, perform rate limiting operation on the user request based on the rate limiting policy corresponding to the target application.

2. The distributed current limiting control method according to claim 1, characterized in that, The process of storing the rate limiting configuration data includes: Monitor whether there are any content updates in the rate limiting configuration directory corresponding to the target cache cluster; If so, obtain the rate limiting configuration data corresponding to the target cache cluster from the configuration center; Respond to the restart command from the management terminal and perform a restart operation; After a successful restart, the cache service process is loaded and intercepted to read the rate limiting configuration data and load it into memory.

3. The distributed current limiting control method according to claim 1, characterized in that, Based on the rate limiting configuration parameters of the target cache cluster, rate limiting operations are performed on the user requests, including: Based on the aforementioned rate limiting configuration parameters, determine whether to trigger a rate limiting operation; If so, intercept the user request and return a preset rate-limiting fault code.

4. The distributed current limiting control method according to claim 3, characterized in that, Based on the rate limiting configuration parameters, determine whether to trigger a rate limiting operation, including: Statistics on currently approved requests; Determine whether the allowed request data meets the rate limiting conditions corresponding to the rate limiting configuration parameters; If the conditions are met, it is determined that a rate limiting operation is triggered.

5. The distributed current limiting control method according to claim 3, characterized in that, Also includes: If it is determined that no rate limiting operation has been triggered, determine whether the target data corresponding to the user request is stored locally; If stored, output the target data; If not stored, the target data is obtained based on the source retrieval operation, and the target data is output.

6. The distributed current limiting control method according to claim 1, characterized in that, Based on the rate limiting policy corresponding to the target application, rate limiting operations are performed on the user requests, including: If the rate limiting policy corresponding to the target application is to limit the request rate, a rate limiting instance is created based on the rate limiting parameter in the rate limiting configuration data; Define a rate limiting key to obtain the rate limiting return value during the rate limiting process; Determine whether to trigger a rate limiting operation based on the rate limiting return value; If a rate limiting operation is triggered, the user request will be intercepted, and a preset rate limiting fault code will be returned. If rate limiting is not triggered, determine whether delay processing is needed based on the configured leaky bucket parameters; If it is determined that delay processing is required, a delay-limiting operation will be performed.

7. The distributed current limiting control method according to claim 1, characterized in that, The update process for the rate limiting configuration parameters of the target cache cluster includes: The updated rate limiting configuration parameters corresponding to the target cache cluster are obtained from the preset interface of the management terminal, and the updated rate limiting configuration parameters are used to replace the rate limiting configuration parameters of the target cache cluster stored in memory.

8. A distributed current limiting control device, characterized in that, Applied to any target cache cluster in a distributed content delivery network (CDN) cache cluster; the distributed rate limiting control device includes: The data acquisition module is used to acquire rate limiting configuration data upon receiving a user request. The rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster and / or the rate limiting policy corresponding to the application in the target cache cluster. The rate limiting policies of different target cache clusters may be the same or different. The rate limiting policies of different applications configured in the target cache cluster may be the same or different. The rate limiting policy includes limiting the request rate, limiting the number of concurrent connections, or limiting the number of requests in a time window. The policy judgment module is used to determine whether the rate limiting configuration data includes the rate limiting policy corresponding to the target cache cluster; The first rate limiting module is used to, if applicable, call the cluster rate limiting service to obtain the rate limiting configuration parameters of the target cache cluster, and perform rate limiting operations on the user requests based on the rate limiting configuration parameters of the target cache cluster; The second rate limiting module is used to, if not included, obtain the target application accessed by the user request carried in the user request, and if the rate limiting configuration data includes the rate limiting policy corresponding to the target application, perform rate limiting operation on the user request based on the rate limiting policy corresponding to the target application.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the distributed current limiting control method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the distributed current limiting control method as described in any one of claims 1 to 7.

Images