A hotspot-aware distributed adaptive traffic shaping method and apparatus

Abstract

The application discloses a kind of distributed adaptive flow limiting method and device based on hotspot perception.The method comprises: gateway node receives request and resolves parameter;Based on adaptive token bucket algorithm, according to system real-time load, dynamically adjust token generation rate and bucket capacity;Access frequency of each parameter key value is counted using sliding time window, hot parameter is identified and temporary flow limiting rule is generated;Rules are broadcast to cluster nodes through Gossip protocol.Especially, after pulling baseline rule, new nodes initiate point-to-point state synchronization request to any healthy node in the cluster, and obtain rule snapshot containing hot rule and sliding window statistics state.The application realizes millisecond-level identification and rule synchronization of hot traffic, solves the problem of new node cold start, and does not need to rely on centralized components such as Redis, which improves the stability and flexibility of distributed system.

Description

Technical Field

[0001] This invention relates to the field of computer software and distributed systems technology, and in particular to a distributed rate limiting method and apparatus that can automatically identify hotspot parameters and dynamically adjust rate limiting strategies in high-concurrency scenarios. Background Technology

[0002] With the rapid development of internet technology, distributed systems face enormous traffic surges in scenarios such as e-commerce flash sales and trending social media discussions. To prevent system crashes due to overload, rate limiting technology has become a core means of ensuring system stability.

[0003] Existing rate limiting technologies mainly include the following:

[0004] Single-machine rate limiting: algorithms such as counters, leaky buckets, and token buckets. These algorithms are simple to implement, but they can only protect a single application node and cannot handle overall traffic control in a distributed environment. For example, when the system scales up, the total QPS (queries per second) will increase linearly, which may lead to excessive pressure on backend services.

[0005] Gateway-based rate limiting: such as rate limiting at the Nginx layer. This method is not flexible enough and makes it difficult to perform fine-grained control over specific business interface parameters (such as a specific product ID).

[0006] Distributed rate limiting based on middleware (such as Redis): While global counting is achieved through Redis's atomic operations, which solves the distributed consistency problem, frequent access to Redis itself can easily become a new performance bottleneck in ultra-high concurrency scenarios, and it also increases the system's strong dependence on external components.

[0007] In addition, while existing mainstream open-source frameworks provide hotspot parameter rate limiting and system adaptive protection, they still have the following shortcomings:

[0008] Hotspot rule activation delay: The discovery and push of hotspot rules usually rely on the client to periodically poll the configuration center (such as Nacos, Apollo), which has a delay of seconds or even minutes, and cannot cope with the instantaneous burst of hotspot traffic.

[0009] Cold start problem for new nodes: Newly added nodes cannot immediately perceive the current hotspot status of the cluster and require a "warm-up" or "learning" process, during which rate limiting may fail.

[0010] Lack of state synchronization mechanism: In the existing technology, there is a lack of a lightweight inter-node state synchronization mechanism that does not rely on centralized storage, which may lead to significant differences in the rate limiting views of each node.

[0011] Therefore, there is an urgent need for a decentralized, distributed, adaptive rate limiting solution that can automatically identify hotspots, quickly synchronize rules, and is decentralized. Summary of the Invention

[0012] The present invention aims to solve the problems of poor flexibility of distributed rate limiting, delayed response to hot parameters, and performance bottlenecks caused by reliance on external middleware in the prior art.

[0013] To achieve the above objectives, the present invention provides the following technical solution:

[0014] A distributed adaptive rate limiting method based on hotspot awareness includes the following steps:

[0015] Traffic access and parsing steps: The gateway node receives external requests and parses the routing information and parameter set in the requests;

[0016] Adaptive token bucket control steps: Admission control of requests is performed based on a preset token bucket algorithm; the token generation rate R and bucket capacity C of the token bucket algorithm are dynamically adjusted according to the real-time load indicators of the system.

[0017] Hotspot parameter monitoring steps: Within the sliding time window, count the access frequency of each parameter key value. When the access frequency of a certain parameter key value meets the preset hotspot judgment condition, generate a temporary rate limiting rule for that parameter key value.

[0018] Rule synchronization and execution steps: Broadcast the temporary rate limiting rules to other nodes in the cluster via the Gossip protocol, and update the local rate limiting rule set;

[0019] The newly added node, after pulling the baseline rules from the configuration center, obtains a rule snapshot, including dynamically generated hotspot parameter rules and sliding window statistics, by initiating a point-to-point status synchronization request to any healthy node in the cluster.

[0020] The system's real-time load metrics include CPU utilization, memory utilization, average response time, and request success rate.

[0021] The load health status is calculated by the feedback controller. When the load health status is lower than the preset threshold, the token generation rate R is reduced by a preset ratio. When the load health status recovers, the token generation rate R is restored according to the linear growth strategy.

[0022] The hotspot parameter monitoring steps specifically include:

[0023] Data sampling: Extract key parameters from requests that have passed through the rate limiting component and perform hash calculations to obtain parameter keys;

[0024] Sliding window statistics: A sliding time window counter is maintained for each combination of resource and parameter key. The window is divided into multiple sub-windows, and the number of requests in each sub-window is recorded respectively.

[0025] Hotspot detection: Periodically traverse the statistical data within the sliding window. If the total number of requests for a parameter key in the main window exceeds the global threshold, and the number of requests in the most recent child window exceeds the burst threshold, then it is determined to be a hotspot parameter.

[0026] After generating the temporary rate limiting rule for this parameter key value, the method further includes:

[0027] Set the Time To Live (TTL) for this temporary rate limiting rule;

[0028] If the parameter key value is not identified as a hotspot again within the TTL validity period, the rule will be automatically removed after the TTL expires; if the parameter key value is identified as a hotspot again within the TTL validity period, the TTL will be reset.

[0029] In the rule synchronization and execution steps, the sliding window statistics obtained by the point-to-point state synchronization request are used by newly added nodes to rebuild a sliding window counter that is consistent with the current state of the cluster locally, so as to ensure the continuity of rate limiting decisions.

[0030] The method also includes visual configuration and persistence steps:

[0031] Define baseline flow control rules and circuit breaker / degradation conditions through a visualization platform, and store the rules in the configuration center database;

[0032] When the configuration center pushes an update, it carries a globally monotonically increasing version number; when the rate limiting node receives an update, it only applies the new rule if the pushed version number is higher than the local cached version number.

[0033] The method also includes a degradation processing step:

[0034] When a node loses connection with the configuration center, it enters a degraded mode and uses the latest rules cached locally to make rate limiting decisions.

[0035] Once the network is restored, the node compares its local rule version number with the latest version number in the configuration center. If the local version is outdated, the cloud-based rules will be used to overwrite the local cache.

[0036] A distributed adaptive rate limiting device based on hotspot awareness includes:

[0037] The flow control module is used to execute the adaptive token bucket algorithm and dynamically adjust the flow limiting threshold according to the real-time load.

[0038] The hotspot discovery module is used to identify frequently accessed parameter key values ​​in real time based on a sliding window statistical algorithm and generate temporary rate limiting rules.

[0039] The rule synchronization module is used to synchronize the temporary rate limiting rules among cluster nodes via the Gossip protocol and respond to the peer-to-peer status synchronization requests of new nodes.

[0040] The rule management module is used to manage the persistent storage and version control of baseline rules.

[0041] An electronic device includes a processor, a memory, and a computer program stored in the memory, wherein the processor executes the program to implement any of the above methods;

[0042] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0043] Compared with the prior art, the technical solution provided by the embodiments of the present invention has the following significant beneficial effects:

[0044] Rapid response to hotspots: Through real-time monitoring via a sliding window and rapid broadcasting via the Gossip protocol, hotspot rules take effect across the entire network within milliseconds, effectively handling sudden traffic surges.

[0045] Decentralized management: It eliminates the strong dependence on centralized components such as Redis, reducing system complexity and the risk of performance bottlenecks.

[0046] Seamless node scaling: Through a two-stage mechanism of "baseline pull + state synchronization", new nodes can instantly inherit the rate limiting state of the cluster, solving the cold start problem.

[0047] System self-protection: The adaptive token bucket algorithm can automatically adjust the traffic according to the system load, maintaining system stability without manual intervention. Attached Figure Description

[0048] Figure 1 A schematic diagram of the distributed adaptive rate limiting method based on hotspot awareness provided in the embodiments of this application;

[0049] Figure 2 A flowchart illustrating the flow control process based on the adaptive token bucket algorithm provided in this embodiment of the invention;

[0050] Figure 3 A flowchart illustrating the overall implementation of the flow control scheme based on the adaptive token bucket algorithm provided in this embodiment of the invention;

[0051] Figure 4 This is a schematic diagram of the structure of a distributed adaptive rate limiting device based on hotspot awareness, provided in an embodiment of this application. Detailed Implementation

[0052] To better understand the present invention, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Various modifications can be made to the embodiments as long as the effects of the present invention are achieved.

[0053] The present invention will now be described in detail with reference to specific embodiments.

[0054] Example 1

[0055] like Figure 1 As shown, this is a distributed adaptive rate limiting method based on hotspot awareness provided in an embodiment of the present invention, wherein,

[0056] Step 10, Traffic Access and Resolution Steps. The gateway node receives external requests and parses the routing information and parameter set in the requests.

[0057] Step 20, Adaptive Token Bucket Control Step. Admission control of requests is performed based on a preset token bucket algorithm; the token generation rate R and bucket capacity C of the token bucket algorithm are dynamically adjusted according to the real-time system load indicators.

[0058] Step 30, Hotspot Parameter Monitoring Step. Within a sliding time window, the access frequency of each parameter key value is counted. When the access frequency of a certain parameter key value meets the preset hotspot judgment condition, a temporary rate limiting rule is generated for that parameter key value.

[0059] Step 40, Rule Synchronization and Execution. Broadcast the temporary rate limiting rules to other nodes in the cluster via the Gossip protocol and update the local rate limiting rule set;

[0060] The newly added node, after pulling the baseline rules from the configuration center, obtains a rule snapshot, including dynamically generated hotspot parameter rules and sliding window statistics, by initiating a point-to-point status synchronization request to any healthy node in the cluster.

[0061] The system's real-time load metrics include CPU utilization, memory utilization, average response time, and request success rate.

[0062] The load health status is calculated by the feedback controller. When the load health status is lower than the preset threshold, the token generation rate R is reduced by a preset ratio. When the load health status recovers, the token generation rate R is restored according to the linear growth strategy.

[0063] The hotspot parameter monitoring steps specifically include:

[0064] Data sampling: Extract key parameters from requests that have passed through the rate limiting component and perform hash calculations to obtain parameter keys;

[0065] Sliding window statistics: A sliding time window counter is maintained for each combination of resource and parameter key. The window is divided into multiple sub-windows, and the number of requests in each sub-window is recorded respectively.

[0066] Hotspot detection: Periodically traverse the statistical data within the sliding window. If the total number of requests for a parameter key in the main window exceeds the global threshold, and the number of requests in the most recent child window exceeds the burst threshold, then it is determined to be a hotspot parameter.

[0067] After generating the temporary rate limiting rule for this parameter key value, the method further includes:

[0068] Set the Time To Live (TTL) for this temporary rate limiting rule;

[0069] If the parameter key value is not identified as a hotspot again within the TTL validity period, the rule will be automatically removed after the TTL expires; if the parameter key value is identified as a hotspot again within the TTL validity period, the TTL will be reset.

[0070] In the rule synchronization and execution steps, the sliding window statistics obtained by the point-to-point state synchronization request are used by newly added nodes to rebuild a sliding window counter that is consistent with the current state of the cluster locally, so as to ensure the continuity of rate limiting decisions.

[0071] The method also includes visual configuration and persistence steps:

[0072] Define baseline flow control rules and circuit breaker / degradation conditions through a visualization platform, and store the rules in the configuration center database;

[0073] When the configuration center pushes an update, it carries a globally monotonically increasing version number; when the rate limiting node receives an update, it only applies the new rule if the pushed version number is higher than the local cached version number.

[0074] The method also includes a degradation processing step:

[0075] When a node loses connection with the configuration center, it enters a degraded mode and uses the latest rules cached locally to make rate limiting decisions.

[0076] Once the network is restored, the node compares its local rule version number with the latest version number in the configuration center. If the local version is outdated, the cloud-based rules will be used to overwrite the local cache.

[0077] This embodiment provides an adaptive traffic control strategy designed based on business scenarios, which achieves high-precision traffic control and can dynamically adjust traffic thresholds and control methods according to real-time traffic data and business needs.

[0078] This embodiment provides the design of a flow control rule visualization platform, which enables convenient management of flow control rules through an intuitive graphical interface.

[0079] This embodiment provides a flow control rule persistence mechanism, which uses a configuration center and database to ensure reliable storage and dynamic updates of rules.

[0080] This invention, through real-time traffic analysis and sliding window statistics, automatically identifies frequently accessed parameter values ​​and dynamically generates rate-limiting rules for those specific parameter values. This process eliminates the need for developers to predict and code hotspot parameter indices, achieving a leap from "manual configuration of hotspot dimensions" to "system-automatic identification of hotspot values," making it particularly suitable for scenarios where hotspots are difficult to predict in advance (such as breaking news IDs or suddenly popular product IDs). This invention proposes a lightweight rule synchronization mechanism based on the Gossip protocol or a variant of the Raft consensus algorithm. When a new node joins the cluster, it first pulls baseline rules from the configuration center, and then quickly obtains the latest dynamic rules (including automatically generated hotspot rules) and the real-time traffic statistics status of each node in the cluster through a point-to-point state synchronization protocol between nodes. This approach reduces strong dependence on external middleware, lowers system complexity and potential failure points, and maintains basic consistency in the cluster's rate-limiting function even when middleware services are unstable.

[0081] Example 2

[0082] This embodiment provides a distributed rate limiting system deployed in the gateway layer or business service layer of a microservice architecture. The core of the system includes a traffic control module, a hotspot discovery module, and a rule synchronization module.

[0083] In terms of flow control, this invention employs an adaptive token bucket algorithm. The specific implementation logic is as follows:

[0084] Maintain a token bucket for each protected API resource, with the generation rate as the parameter. and capacity .

[0085] Start a background monitoring thread, every System metrics are collected every second (e.g., 1 second), including: CPU utilization. Memory usage Average response time Request success rate .

[0086] Define the load health function For example, it can be set when or At that time, it was considered that the system was unhealthy.

[0087] Adjustment based on feedback control principle value:

[0088] If the system is unhealthy, then ,in A decay coefficient (e.g., 0.1) is used to rapidly reduce the inbound flow.

[0089] If the system is healthy, then ,in To restore the step size (e.g., 10), the system's capacity limit is slowly tested. This mechanism allows the system to automatically adapt to nodes with different hardware configurations without requiring manual preset of precise QPS values.

[0090] Example 3

[0091] To address the issue that existing technologies cannot automatically respond to sudden hotspots, this invention designs a hotspot detection mechanism based on a sliding window, the process of which is as follows:

[0092] Data extraction: Intercept requests and extract predefined key parameters (such as userId, itemId).

[0093] Sliding window statistics:

[0094] Set the window size to 1 minute and divide it into 6 sub-windows (Buckets) of 10 seconds each.

[0095] Use efficient data structures (such as LongAdder) to record the count of each Resource + ParamKey combination in each sub-window.

[0096] Decision logic:

[0097] A scan is triggered every 10 seconds.

[0098] Condition A (total threshold): In the past minute, the total number of requests for a certain itemId=1001 is > 10,000.

[0099] Condition B (Burst Threshold): Within the last 10 seconds, the number of requests for itemId=1001 is > 3,000.

[0100] If both A and B are satisfied, it is determined to be a hotspot.

[0101] Rule generation:

[0102] The system automatically generates a high-priority rate limiting rule: IF resource == " / buy" AND param.itemId == "1001" THEN limit(100 QPS).

[0103] The rule is marked as a "dynamic temporary rule" and a TTL (e.g., 5 minutes) is set. This means that the rule will automatically expire if the hotspot subsides, without the need for manual cleanup.

[0104] Example 4

[0105] To address the issue of new nodes failing to immediately detect hotspot rules upon going online, this invention proposes a two-stage synchronization scheme.

[0106] Phase 1 (Baseline Synchronization):

[0107] When a node starts up, it pulls a full set of "baseline rules" from a configuration center (such as Nacos). These rules are preset by the administrator and are valid for a long time.

[0108] Phase Two (Dynamic State Synchronization):

[0109] After the node starts up, it randomly selects a healthy old node in the cluster via the Gossip protocol.

[0110] Initiate a StateSyncRequest, carrying the current set of rule version numbers on the local machine.

[0111] The old node comparison version will package and return the differentiated "dynamic temporary rules" and sliding window statistics status.

[0112] The synchronization here includes not only the rule text but also the "sliding window statistics status." For example, an older node informs a new node that "itemId=1001 is a hotspot, and its current sliding window counter value is 5000." Upon receiving this information, the new node rebuilds the counter locally, seamlessly integrating it into the cluster's rate limiting decision-making and avoiding rate limiting failures during cold starts.

[0113] Version control:

[0114] All rules are accompanied by a globally monotonically increasing version number.

[0115] The "cloud-first" principle is adopted. If the local rule version on a node is lower than that in the configuration center, it will be forcibly overwritten; if there is a network partition, the local cache will be maintained.

[0116] Example 5

[0117] To improve system availability, this invention also includes a visual configuration and degradation handling mechanism.

[0118] Visual configuration:

[0119] Administrators define baseline rules through the web console (such as limiting the rate of a certain interface to 1000 QPS by default).

[0120] The console pushes the rules to the configuration center, which then pushes them to each node via long polling or event notification mechanisms.

[0121] After receiving the rules, the node persists them to the local file system to ensure that the rules are not lost after a restart.

[0122] Downgrade mode:

[0123] When a node detects a timeout or an anomaly in its connection with the configuration center, it automatically enters "degradation mode".

[0124] In degraded mode, nodes ignore dynamically discovered hotspot rules and rely solely on locally cached baseline rules for rate limiting.

[0125] Once the network is restored, the node will fetch the latest version from the cloud again. If the local version is outdated, local modifications will be discarded and the cloud configuration will be forcibly synchronized.

[0126] Example 5

[0127] In another embodiment, rate limiting is not limited to QPS but can also be based on bandwidth. The system monitors the real-time flow rate of the network interface card (NIC). When downlink bandwidth exceeds a threshold (e.g., 100Mbps), bandwidth rate limiting is triggered. At this point, the token generation rate in the adaptive token bucket algorithm... The bandwidth utilization will be adjusted in reverse. For example, the higher the bandwidth utilization, the more... The smaller the value, the more likely it is to reject some large file download requests, thus ensuring the smooth operation of the network in the core transaction process.

[0128] Example 6

[0129] like Figure 2 The diagram shown illustrates the workflow of flow control based on the adaptive token bucket algorithm provided in this embodiment of the invention. Figure 3 The diagram shown is a flowchart illustrating the overall implementation of the flow control scheme based on the adaptive token bucket algorithm provided in this embodiment of the invention.

[0130] As described in the above embodiments, the algorithm includes two key parameters: token generation rate R (tokens / second) and token bucket capacity C (tokens). The system maintains an independent token bucket for each protected resource (such as an API interface). Each time a request arrives, it attempts to obtain a token from the bucket. If successful, the request is allowed to pass; otherwise, rate limiting is triggered. Unlike traditional token bucket algorithms, the token generation rate R and bucket capacity C of this invention are not fixed values ​​but are dynamically adjusted based on real-time monitored system metrics. Input parameters include: request success rate within the current time window, average system response time, and server CPU / memory utilization. Through a feedback controller (e.g., based on the proportional-integral (PI) control principle), when the system load increases (response time increases, success rate decreases), the controller automatically lowers the R value to tighten rate limiting; when the system load is healthy, the R value is slowly restored to achieve adaptive traffic shaping.

[0131] Set differentiated traffic thresholds for different business interfaces and requests from different sources. For example, set relatively high traffic thresholds for core business interfaces (such as insurance application and claims interfaces) to ensure the normal operation of critical businesses; set lower traffic thresholds for non-core query interfaces to allocate system resources reasonably.

[0132] The core steps are as follows:

[0133] External requests enter the system first, triggering the flow control process.

[0134] Route / API Group Matching: Identify the routing rules corresponding to a request based on the route ID or API group using filters.

[0135] Request attribute parsing: Parses the attributes of the request (such as path, parameters, etc.) to prepare for subsequent rule matching.

[0136] Flow control rules: Managed by the traffic manager, defining the traffic control policies for the API gateway layer.

[0137] Convert to hotspot parameter rules: The flow control rules will be internally converted to generate hotspot parameter rules, which refines the flow control logic at the parameter level.

[0138] Hotspot Parameter Identification and Automatic Rule Generation: The system can automatically identify frequently occurring request parameters (hotspot parameters) and dynamically generate rate limiting rules for those parameters. The specific process is as follows:

[0139] Data collection: For each request that passes through the rate limiting component, extract its key parameters (such as user ID, product ID, IP address, etc.) and perform hash calculation to obtain a parameter key (ParamKey).

[0140] Sliding window statistics: A sliding time window counter is maintained for each resource + ParamKey combination. The window size is configurable (e.g., 1 minute) and divided into multiple sub-windows (e.g., 6 10-second windows). Each sub-window records the number of times the resource + ParamKey appears.

[0141] Hotspot detection: Every statistical period (e.g., 10 seconds), the system iterates through all resources recorded in the window and adds the ParamKey.

[0142] Calculate the total number of requests (Count_total) within the entire sliding window.

[0143] Calculate the number of requests for this parameter in the most recent child window: Count_latest.

[0144] If Count_total exceeds the global threshold (e.g., 1000 times per minute) and Count_latest exceeds the burst threshold (e.g., 300 times in the last 10 seconds), then the parameter is determined to be a "hotspot parameter".

[0145] Rule Generation and Degradation: Once a parameter is identified as a hotspot, the system automatically generates a temporary "Hotspot Parameter Rate Limiting Rule," formatted as: limiting the request rate for a resource with ParamKey=a specific value. This rule's threshold is typically stricter than that of ordinary rules. Simultaneously, predefined degradation strategies can be triggered, such as returning cached content or simplifying business processes.

[0146] Hotspot fading: Set a short time-to-live (TTL) for automatically generated rules (e.g., 5 minutes). Rules automatically expire after the TTL, but the system continues monitoring. If this parameter is no longer identified as a hotspot during this period, the rule can be removed in advance.

[0147] Parameter input: Pass in the parsed request parameters.

[0148] Gateway rule check: Based on the transformed rules, verify whether the request triggers flow control and decide whether to allow or limit the flow.

[0149] The entire process, through rule definition, transformation, parameter passing and verification, enables the API gateway to achieve fine-grained control over traffic and ensure system stability.

[0150] An intuitive and visual interface is built on the platform, clearly displaying various rules such as traffic thresholds and circuit breaker / degradation conditions for different business interfaces. Administrators can create, modify, and delete rules through simple graphical operations, reducing the operational threshold, minimizing human error, and improving the efficiency and accuracy of flow control rule management.

[0151] By leveraging a configuration center to store flow control rules in a database, persistent storage and dynamic updates of these rules are achieved. Even in the event of a system failure or restart, the rules remain consistent and reliable. When business requirements change, the rules can be quickly updated, enabling the system to adapt to new traffic management requirements and ensuring stable system operation.

[0152] The system can still accurately limit traffic during expansion and contraction, and it has a mechanism for "instant" acquisition of new node rules.

[0153] When a new node (instance B) joins the cluster, its rule loading is divided into two phases to achieve rapid deployment and state synchronization:

[0154] Phase 1: Baseline Rule Retrieval. After node B starts, it first retrieves the baseline flow control rules for all current resources from the persistent storage layer (such as a database or configuration center persistent storage). These are relatively stable rules preset by the administrator. This process ensures that node B has the most basic rate limiting capabilities.

[0155] Phase Two: Dynamic State Synchronization. Next, Node B initiates a point-to-point state synchronization request to one or more existing healthy nodes in the cluster (such as Node A). Node A sends a full snapshot of its in-memory rules (including baseline rules and dynamically generated rules at runtime, such as hotspot parameter rules) and its current cluster count state (such as the current token count in the distributed token bucket) to Node B in one go. This process is transmitted via an efficient binary serialization protocol, typically completed within hundreds of milliseconds, enabling new nodes to instantly possess the same up-to-date rate limiting decision-making capabilities as older nodes.

[0156] When administrators update and publish rules through the visualization platform, the configuration center, as the rule publishing source, ensures that the order and final content of the rules received by all nodes are consistent through the following mechanisms:

[0157] Atomic versioning: Each rule or set of rules is assigned a globally monotonically increasing version number. When pushing updates, the configuration center updates the rule content and its version number in the storage center using atomic operations.

[0158] Version-based push-pull approach: Each rate-limiting node locally caches the rule version number of the current application. The configuration center proactively pushes update notifications, or nodes periodically poll, always carrying the local version number. The configuration center only returns rule change sets with version numbers higher than the node's local version, ensuring all nodes ultimately apply updates in the same order. For rapid, continuous updates of the same rule, higher versions overwrite lower versions, avoiding inconsistencies in rule states due to network latency.

[0159] Considering that the configuration center may be temporarily unavailable or the network may be partitioned, each rate limiting node maintains a complete local cache of rules in memory.

[0160] Degradation strategy: When a node loses connection with the configuration center, it automatically enters degradation mode, continuing to use the latest rules cached locally for rate limiting decisions to ensure uninterrupted service. Simultaneously, the node records the rule's "disconnected from the center" status locally.

[0161] Once the network is restored, the nodes resynchronize with the configuration center. The synchronization process follows the principle of "cloud first, version arbitration": the node compares its local rule version number with the latest version number in the configuration center. If the local version is outdated, the cloud rules are used to overwrite the local cache.

[0162] Example 7

[0163] like Figure 4 The diagram shown is a schematic representation of the distributed adaptive rate limiting device based on hotspot awareness provided in an embodiment of the present invention.

[0164] The flow control module 100 is used to execute the adaptive token bucket algorithm and dynamically adjust the flow limiting threshold according to the real-time load.

[0165] The hotspot discovery module 200 is used to identify frequently accessed parameter key values ​​in real time based on a sliding window statistical algorithm and generate temporary rate limiting rules.

[0166] The rule synchronization module 300 is used to synchronize the temporary rate limiting rules among cluster nodes via the Gossip protocol and respond to the point-to-point status synchronization request of the new node.

[0167] The rule management module 400 is used to manage the persistent storage and version control of baseline rules.

[0168] The embodiments of the present invention are more innovative, and designed more targeted adaptive strategies for complex business scenarios, which can achieve more accurate traffic control. Compared with the existing technology, they have significantly improved the accuracy and adaptability of traffic control.

[0169] The embodiments of this invention feature more advanced rule visualization and persistence capabilities. The visualization platform is simpler and more intuitive to operate, reducing management difficulty; the rule persistence mechanism ensures rule reliability while enabling faster rule updates based on business needs, whereas existing technologies may suffer from problems such as complex operation and untimely updates in these two aspects.

[0170] The embodiments of the present invention exhibit superior performance in high-concurrency scenarios. Through optimization of system architecture and algorithms, it can better maintain high performance and low latency. Compared with some existing technologies, it can more effectively cope with extreme traffic peaks, ensure stable system operation, and improve business processing efficiency and customer experience.

[0171] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0173] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0174] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0175] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0176] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0177] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0179] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0180] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0181] The accompanying drawings illustrate several block diagrams and / or flowcharts. It should be understood that some blocks, or combinations thereof, in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create means for implementing the functions / operations described in these block diagrams and / or flowcharts. The technology of this application can be implemented in hardware and / or software (including firmware, microcode, etc.). Alternatively, the technology of this application can take the form of a computer program product stored on a computer-readable storage medium, which can be used by or in conjunction with an instruction execution system.

Claims

1. A distributed adaptive rate limiting method based on hotspot awareness, characterized in that, Includes the following steps: Traffic access and parsing steps: The gateway node receives external requests and parses the routing information and parameter set in the requests; Adaptive token bucket control steps: Admission control of requests is performed based on a preset token bucket algorithm; the token generation rate R and bucket capacity C of the token bucket algorithm are dynamically adjusted according to the real-time load indicators of the system. Hotspot parameter monitoring steps: Within the sliding time window, count the access frequency of each parameter key value. When the access frequency of a certain parameter key value meets the preset hotspot judgment condition, generate a temporary rate limiting rule for that parameter key value. Rule synchronization and execution steps: Broadcast the temporary rate limiting rules to other nodes in the cluster via the Gossip protocol, and update the local rate limiting rule set; The newly added node, after pulling the baseline rules from the configuration center, obtains a rule snapshot, including dynamically generated hotspot parameter rules and sliding window statistics, by initiating a point-to-point status synchronization request to any healthy node in the cluster.

2. The method according to claim 1, characterized in that, The system's real-time load metrics include CPU utilization, memory utilization, average response time, and request success rate. The load health status is calculated by the feedback controller. When the load health status is lower than the preset threshold, the token generation rate R is reduced by a preset ratio. When the load health recovers, the token generation rate R is restored according to the linear growth strategy.

3. The method according to claim 1, characterized in that, The hotspot parameter monitoring steps specifically include: Data sampling: Extract key parameters from requests that have passed through the rate limiting component and perform hash calculations to obtain parameter keys; Sliding window statistics: A sliding time window counter is maintained for each combination of resource and parameter key. The window is divided into multiple sub-windows, and the number of requests in each sub-window is recorded respectively. Hotspot detection: Periodically traverse the statistical data within the sliding window. If the total number of requests for a parameter key in the main window exceeds the global threshold, and the number of requests in the most recent child window exceeds the burst threshold, then it is determined to be a hotspot parameter.

4. The method according to claim 3, characterized in that, After generating the temporary rate limiting rule for this parameter key value, the method further includes: Set the Time To Live (TTL) for this temporary rate limiting rule; If the parameter key value is not identified as a hotspot again within the TTL validity period, the rule will be automatically removed after the TTL expires; if the parameter key value is identified as a hotspot again within the TTL validity period, the TTL will be reset.

5. The method according to claim 1, characterized in that, In the rule synchronization and execution steps, the sliding window statistics obtained by the point-to-point state synchronization request are used by newly added nodes to rebuild a sliding window counter that is consistent with the current state of the cluster locally, so as to ensure the continuity of rate limiting decisions.

6. The method according to claim 1, characterized in that, The method also includes visual configuration and persistence steps: Define baseline flow control rules and circuit breaker / degradation conditions through a visualization platform, and store the rules in the configuration center database; When the configuration center pushes an update, it carries a globally monotonically increasing version number; when the rate limiting node receives an update, it only applies the new rule if the pushed version number is higher than the local cached version number.

7. The method according to claim 1, characterized in that, The method also includes a degradation processing step: When a node loses connection with the configuration center, it enters a degraded mode and uses the latest rules cached locally to make rate limiting decisions. Once the network is restored, the node compares its local rule version number with the latest version number in the configuration center. If the local version is outdated, the cloud-based rules will be used to overwrite the local cache.

8. A distributed adaptive current limiting device based on hotspot awareness, characterized in that, include: The flow control module is used to execute the adaptive token bucket algorithm and dynamically adjust the flow limiting threshold according to the real-time load. The hotspot discovery module is used to identify frequently accessed parameter key values ​​in real time based on a sliding window statistical algorithm and generate temporary rate limiting rules. The rule synchronization module is used to synchronize the temporary rate limiting rules among cluster nodes via the Gossip protocol and respond to the peer-to-peer status synchronization requests of new nodes. The rule management module is used to manage the persistent storage and version control of baseline rules.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images