Method and system for throttling access requests

Abstract

The application provides a method and system for access request flow limiting processing, acquires a preset multi-dimensional flow label system, constructs a flow limiting quota matrix according to the preset multi-dimensional flow label system, extracts label information of an access request in response to the access request sent by a user, acquires statistical data of a preset multi-window based on the label information, calculates a comprehensive load score corresponding to the label information based on the flow limiting quota matrix and the statistical data, compares the comprehensive load score with a preset load threshold, and performs hierarchical flow limiting processing according to a comparison result, so as to realize fine flow limiting quota control and differentiated hierarchical flow limiting processing of access requests, and guarantee the operation stability and reliability of a system under various flow scenarios.

Description

Technical Field

[0001] This invention relates to the field of flow control technology, and in particular to a method and system for rate limiting of access requests. Background Technology

[0002] As internet services become more scaled and complex, systems face increasing pressure from high-concurrency traffic. This is especially true in large-scale online education platforms and enterprise SaaS systems, where multiple products and business lines share the same system resources, leading to frequent sudden traffic spikes. Existing rate limiting solutions typically employ the following technical methods: (1) Single-dimensional fixed threshold flow limiting Classic token bucket and leaky bucket algorithms use a fixed rate or concurrency threshold as the rate limiting threshold, treating all requests equally. However, in real-world multi-product, multi-service scenarios, the business importance and user scale of different products vary significantly—the rate limiting threshold for core transaction-related services should be higher than that for auxiliary query-related services, and direct-connection services for enterprises should take precedence over public-facing open services. Fixed thresholds cannot distinguish between different dimensions of traffic characteristics, leading to important services being incorrectly limited or unimportant services consuming excessive resources.

[0003] (2) Lack of multi-dimensional fine-grained current limiting capability Existing solutions typically implement rate limiting based on a single dimension (such as IP, user ID, or interface level), failing to consider multi-dimensional combinations such as product, business unit, and user level. For example, different business units within the same product may have significantly different active user numbers and access patterns, and the traffic characteristics of the same business unit may also differ at different times. Single-dimensional rate limiting cannot achieve fine-grained traffic control.

[0004] (3) Static configuration cannot adapt to dynamic changes in traffic. Existing rate limiting solutions typically set rate limiting thresholds statically through configuration files, requiring manual adjustment by operations personnel. When sudden traffic surges occur (such as large-scale school closures leading to a surge in student access to online education systems), there is a time lag between detecting the traffic anomaly and manually adjusting the thresholds, during which time the system may already be overloaded. There is a lack of adaptive rate limiting mechanisms based on real-time traffic awareness and prediction.

[0005] (4) The traffic limiting and degradation strategy is too simplistic. Existing solutions typically return error codes or a uniform message page directly after rate limiting is triggered, without performing differentiated degradation processing based on request characteristics. For high-value users or core business requests, simply rejecting them could lead to business losses and a severe decline in user experience. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for rate limiting of access requests, which can realize fine-grained control of rate limiting quotas and differentiated graded rate limiting processing of access requests, and ensure the stability and reliability of the system under various traffic scenarios.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for rate limiting access requests, comprising: Obtain a preset multi-dimensional traffic tagging system, and construct a traffic limiting quota matrix according to the preset multi-dimensional traffic tagging system; In response to an access request sent by a user, the tag information of the access request is extracted, and statistical data of a preset multi-window is obtained based on the tag information; Calculate the comprehensive load score corresponding to the tag information based on the rate limiting quota matrix and the statistical data; The comprehensive load score is compared with a preset load threshold, and tiered flow limiting is performed based on the comparison result.

[0008] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A rate limiting system for access requests includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the rate limiting method for access requests described above.

[0009] The beneficial effects of this invention are as follows: A rate-limiting quota matrix is ​​constructed based on a preset multi-dimensional traffic tagging system, providing a refined rate-limiting benchmark for various access requests. When an access request sent by a user is received, the tag information of the access request is extracted to determine the statistical dimension corresponding to the access request. This allows for the acquisition of statistical data corresponding to that statistical dimension within a preset multi-window, achieving real-time perception and accurate quantification of the current traffic status, providing a reliable data foundation for subsequent load assessment. A comprehensive load score is calculated based on the rate-limiting quota matrix and statistical data, accurately reflecting the system load level. The comprehensive load score is compared with a preset load threshold, and graded rate-limiting processing is performed based on the comparison results. This enables differentiated traffic control actions based on load levels, ensuring the system's operational stability and reliability under various traffic scenarios. Attached Figure Description

[0010] Figure 1 This is a flowchart of a rate limiting method for access requests according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an access request rate limiting system according to an embodiment of the present invention. Detailed Implementation

[0011] Definitions:

[0012] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0013] In existing technologies, when faced with frequent sudden surges in access leading to system overload, rate limiting solutions mainly include single-dimensional fixed threshold rate limiting based on token bucket or leaky bucket algorithms, coarse-grained control strategies based on a single dimension, and setting thresholds through static configuration files supplemented by manual adjustments. Existing solutions have the following limitations: fixed thresholds cannot distinguish the importance differences between core transactions and auxiliary queries, leading to critical services being mistakenly limited in resource contention, while non-core requests may excessively consume resources; the rate limiting dimension is singular, lacking the ability to finely control multi-dimensional combinations such as "product × business unit × time period × user level," making it difficult to adapt to the traffic heterogeneity of different business units and time periods within the same product; static configuration relies on manual intervention, and when a sudden traffic event occurs, there is a lag window between anomaly detection and threshold adjustment, during which the system is prone to overload; the degradation strategy after triggering rate limiting is highly uniform, usually only returning error codes or general prompts, without differentiated processing based on user value or business priority, resulting in high-value requests being simply rejected.

[0014] To at least address the aforementioned issues, a rate-limiting quota matrix is ​​constructed based on a pre-defined multi-dimensional traffic tagging system. When a user's access request is received, the tag information of the access request is extracted, and statistical data corresponding to this tag information is obtained within a pre-defined multi-window. A comprehensive load score is then calculated based on the rate-limiting quota matrix and the statistical data. This comprehensive load score is compared with a pre-defined load threshold, and tiered rate-limiting is applied based on the comparison result. This approach enables fine-grained control of access request rate-limiting quotas and differentiated tiered rate-limiting processing, ensuring the system's operational stability and reliability under various traffic scenarios.

[0015] The following describes in detail a method for rate limiting of access requests according to the present invention. Please refer to [link / reference]. Figure 1 The method 100 includes steps 101 to 104: Step 101: Obtain a preset multi-dimensional traffic label system and construct a traffic limiting quota matrix according to the preset multi-dimensional traffic label system.

[0016] Specifically, the pre-defined multi-dimensional traffic tagging system includes product, business, user level, and time period dimensions. The product dimension identifies the product to which the request belongs, such as "Province Product A" or "Province Product B." The business dimension identifies the business type to which the request belongs, such as "Instant Messaging," "Data Query," or "File Upload." The user level dimension categorizes users into L1 (high-value), L2 (normal), and L3 (low-frequency) levels based on user attributes (e.g., whether they are active users, whether they are direct-connect users, historical access frequency, etc.). The time period dimension divides 24 hours into several time periods (e.g., morning peak, noon, evening peak, early morning), with different basic traffic limiting quotas for different time periods. Based on this pre-defined multi-dimensional traffic tagging system, a traffic limiting quota matrix Q is constructed, where Q[p][b][t][s] represents the traffic limiting quota (maximum number of requests per minute) for user level t under product p and business b during time period s. In this way, a multi-dimensional traffic profile can be constructed, providing a refined traffic limiting benchmark for various access requests.

[0017] Step 102: In response to the access request sent by the user, extract the tag information of the access request, and obtain the statistical data of the preset multi-window based on the tag information.

[0018] Specifically, a preset multi-window system is used to statistically analyze real-time traffic across various dimensions. When an access request is received from a user, the tag information of the access request is extracted to determine the statistical dimension corresponding to the access request. The statistical data corresponding to that statistical dimension is obtained from the counters in the preset multi-window system, thereby achieving real-time perception and accurate quantification of traffic status under the current dimension and providing a reliable data foundation for subsequent load assessment.

[0019] Step 103: Calculate the comprehensive load score corresponding to the tag information based on the traffic limiting quota matrix and the statistical data.

[0020] Specifically, the rate limiting quota corresponding to the tag information is obtained from the rate limiting quota matrix. Based on the rate limiting quota and the statistics of the second combination matching the tag information in each sliding window, the comprehensive load score corresponding to the tag information is calculated. It reflects the system load level under the current tag dimension, providing a clear and quantifiable basis for subsequent adaptive rate limiting threshold adjustment and differentiated request response processing, and realizing refined traffic control.

[0021] Step 104: Compare the comprehensive load score with the preset load threshold, and perform graded flow limiting processing based on the comparison result.

[0022] Specifically, the preset load thresholds include a first preset load threshold and a second preset load threshold. Based on the threshold range of the comprehensive load score, tiered rate limiting is implemented: when the comprehensive load score is less than the first preset load threshold, the system load is normal, requests are directly allowed, and access logs are recorded; when the first preset load threshold is less than or equal to the comprehensive load score but less than the second preset load threshold, the rate limiting quota for the corresponding tag dimension is dynamically reduced to appropriately shrink traffic; when the comprehensive load score is less than or equal to the second preset load threshold, it is determined whether a global circuit breaker is triggered. If the circuit breaker condition is met, differentiated allowance decisions are made based on user level and business type; if the circuit breaker condition is not met, a preset hard degradation strategy is implemented, i.e., a unified prompt page is returned for all non-core business requests, core business requests are queued, and cached data is returned after a queue timeout. In this way, differentiated traffic control actions can be performed based on load levels, ensuring the system's operational stability and reliability under various traffic scenarios.

[0023] In one embodiment of the present invention, step 101 includes step 1011: Step 1011: Combine each preset product in the product dimension with each preset business type in the business dimension to obtain a first combination set; obtain the preset total carrying capacity and the historical peak traffic of each first combination in the first combination set; set the basic quota for the corresponding first combination based on the preset total carrying capacity and the historical peak traffic; adjust each basic quota according to the weight coefficient corresponding to each preset user level in the user level dimension and the scaling factor corresponding to each preset time period in the time period dimension to obtain the traffic limiting quota for each preset business type under each preset product at each time period for each preset user level; construct a traffic limiting quota matrix based on all traffic limiting quotas.

[0024] Specifically, each preset product in the product dimension and each preset business type in the business dimension are combined one by one to obtain the first set of combinations. For example, data query under product A. The basic quota of each first set is set according to the historical peak traffic and preset total carrying capacity of each product and business.

[0025] Different user levels are assigned corresponding weighting coefficients, such as a weighting coefficient of 1.5 for L1 level, 1.0 for L2 level, and 0.5 for L3 level, allowing high-value users to receive higher access quotas. Different time periods are assigned corresponding scaling factors, such as a scaling factor of 0.8 during peak hours and 1.5 during the early morning hours, allowing more elastic access for non-core services during off-peak hours. Therefore, by adjusting the basic quotas using weighting coefficients and scaling factors, the rate limiting quotas for each preset service type under each preset product are obtained for each preset user level at each time period. A rate limiting quota matrix is ​​then constructed based on all rate limiting quotas. Furthermore, the rate limiting quota matrix supports dynamic updates; operations personnel can adjust quotas for any dimension online through the management backend, with adjustments taking effect immediately. This approach enables multi-dimensional, fine-grained rate limiting control, ensuring the stability of critical services while maintaining the system's adaptability to traffic fluctuations, providing highly flexible rate limiting settings for complex environments with multiple products and services sharing resources.

[0026] In one embodiment of the present invention, step 102 includes steps 1021 and 1022: Step 1021: Combine the preset products, preset service types and preset user levels to obtain a second combination set. Using each second combination in the second combination set as a statistical dimension, count the access requests in the first sliding window, the second sliding window and the third sliding window respectively, and store the statistical results in the counter corresponding to each sliding window.

[0027] Specifically, the preset multi-window configuration includes a first sliding window, a second sliding window, and a third sliding window: the first sliding window is a short window (e.g., 5 seconds), which captures instantaneous traffic bursts and triggers a rapid rate limiting response; the second sliding window is a medium window (e.g., 1 minute), serving as the calculation cycle for rate limiting quotas; and the third sliding window is a long window (e.g., 5 minutes), used for traffic trend analysis and load prediction. Each sliding window maintains a set of counters. Preset products, preset business types, and preset user levels are combined to obtain a second combination. Using each second combination as a statistical dimension, access requests within each sliding window are counted independently according to the "product × business × user level" dimension. The statistical results are stored in the counters corresponding to each sliding window, and the counters use atomic operations to ensure concurrency safety and automatically expire and reset through a timing wheel mechanism. This approach ensures the real-time performance and accuracy of rate limiting decisions, providing reliable data support for adaptive multi-dimensional traffic management.

[0028] Step 1022: Obtain a second combination that matches the tag information from the second combination set, and read the statistics of the second combination in the first sliding window, the second sliding window and the third sliding window respectively from the counters corresponding to each sliding window.

[0029] Specifically, a second combination matching the tag information is obtained from the second combination set. Each gateway node independently maintains a local sliding window counter. Each node reports the statistics of the second combination matching the tag information in its local counter to the aggregation node at an adjustable frequency (every 2 seconds by default) through a distributed coordination component (such as a Redis cluster or Consul). The aggregation node summarizes and sums the counts of each node to obtain the statistics of the second combination in the first sliding window, the second sliding window, and the third sliding window, respectively. This achieves efficient traffic statistics in a distributed environment and provides data support for load assessment.

[0030] Furthermore, when a gateway node is unable to report due to network partitioning or downtime, the aggregation node marks it as unavailable and removes it from the overall calculation to avoid distortion of load assessment due to missing data. Once the network is restored, the node will automatically rejoin the load assessment.

[0031] In one embodiment of the present invention, step 103 includes step 1031: Step 1031: Calculate the comprehensive load score corresponding to the tag information based on the statistics of the second combination matching the tag information in each sliding window, the preset normalization coefficient of each sliding window, the preset weighting factor of each sliding window, and the rate limiting quota matrix. L=α×(N short / Q[p][b][t][s]×W short )+β×(N mid / Q[p][b][t][s]×W mid )+γ×(N long / Q[p][b][t][s]×W long ); In the formula, L represents the comprehensive load score corresponding to the tag information, Q[p][b][t][s] represents the traffic limiting quota of service b under product p for user level t in time period s, and the tag information includes product p, service b, user level t and time period s; N short N mid N long These represent the statistics corresponding to the first, second, and third sliding windows, respectively; W short W mid W longα, β, and γ represent the preset normalization coefficients corresponding to the first, second, and third sliding windows, respectively; α, β, and γ represent the preset weighting factors corresponding to the first, second, and third sliding windows, respectively.

[0032] Specifically, when the aggregation node receives the statistics reported by each gateway node, it sums and aggregates them to obtain the statistics of the second group matching the tag information in each sliding window. Based on each statistic (N) short N mid N long (representing the statistics corresponding to the first, second, and third sliding windows respectively), and the preset normalization coefficient (W) for each sliding window. short W mid W long Let N represent the preset normalization coefficients corresponding to the first, second, and third sliding windows, respectively; the preset weight factors for each sliding window (α, β, and γ represent the preset weight factors corresponding to the first, second, and third sliding windows, respectively); and the rate limiting quota Q[p][b][t][s] corresponding to the tag information (product p, business type b, user level t, and time period s). Calculate the comprehensive load score L: L = α × (N short / Q[p][b][t][s]×W short )+β×(N mid / Q[p][b][t][s]×W mid )+γ× (N long / Q[p][b][t][s]×W long It broadcasts the overall load score L to all gateway nodes via a publish / subscribe mechanism (Pub / Sub), and each gateway node holds both the global overall load score L and its local load score L. local(Its calculation method is similar to that of the comprehensive load score). When a gateway node makes a rate-limiting decision, it uses the comprehensive load score as the standard (to prevent a single node from triggering rate limiting prematurely due to traffic imbalance). At the same time, it refers to the local load score as a quick response signal. For example, when the local load score rises sharply, even if the comprehensive load score has not yet exceeded the threshold, it can enter a soft degradation warning state in advance. The local load score of each node is stored in local memory (ConcurrentHashMap or similar concurrent data structure) and is automatically updated as the window scrolls. The global comprehensive load score is stored in a Redis cluster with the key format load_score:{product_id}:{biz_type}:{time_slot}. It uses Redis's INCRBY atomic operation to ensure concurrency safety, and sets the TTL to be consistent with the window period (1 minute for the medium window), and automatically cleans up when it expires. The historical value of the global comprehensive load score is periodically (e.g., every 5 minutes) written to a time-series database (e.g., InfluxDB or Prometheus) for trend analysis, capacity planning, and alarm rule configuration.

[0033] Furthermore, in another embodiment of the present invention, the process of setting the preset weight factors for each sliding window is as follows: First, initial values ​​are set, wherein the short window weight α is 0.5 by default, the medium window weight β is 0.35 by default, and the long window weight γ is 0.15 by default, and the three satisfy α+β+γ=1. The value of this default combination is based on the following: the short window (5 seconds) captures instantaneous bursts and is given the highest weight so that the system can respond quickly to abnormal traffic in seconds, preventing the system from being overloaded during bursts; the medium window (1 minute) is consistent with the flow limit quota calculation cycle, reflecting the real load level within the current quota cycle, and serving as the main basis for stability judgment; the long window... The 5-minute window reflects traffic trends and has a low weight. It is mainly used to help determine whether the high pressure is continuous or a short-term pulse, avoiding unnecessary rate limiting adjustments triggered by frequent instantaneous fluctuations. The overall design principle is "rapid response as the primary factor, stable judgment as a secondary factor, and trend reference as a supplement." Subsequently, the optimal weight combination is determined through offline simulation optimization. Specifically, historical request sequences are replayed from online traffic logs to simulate rate limiting behavior under different combinations of α, β, and γ. The false rate limiting rate (the proportion of normal requests that are rate-limited) and the missed rate limiting rate (the proportion of overloaded requests that are not rate-limited in time) are used as the core evaluation indicators, and the optimization objective function F is calculated by comprehensively weighting them. = w1×(1-false failure rate) + w2×(1-missed failure rate), where w1=0.4 and w2=0.6 (with a greater emphasis on preventing missed failures). Through grid search, all combinations satisfying α+β+γ=1 are traversed in the range [0.1, 0.8] with a step size of 0.05. The combination with the highest F value is selected as the optimal weight. The optimization results are solidified after online A / B verification. Subsequently, they can be re-optimized periodically (e.g., every quarter) according to changes in business traffic patterns.

[0034] The above approach enables a collaborative rate limiting mechanism that combines global load awareness and local rapid response in a distributed environment, balancing the accuracy and real-time nature of rate limiting decisions. At the same time, it enhances the system's adaptability to changes in traffic patterns through tunable multi-window weight combinations.

[0035] In one embodiment of the present invention, step 104 includes steps 1041 to 1043: Step 1041: If the overall load score is less than the first preset load threshold, then the access request is allowed and the access request is recorded in the access log.

[0036] Specifically, the preset load threshold includes a first preset load threshold, which is 0.8 by default. When the overall load score is less than the first preset load threshold (i.e., L < 0.8), it indicates that the system load is normal, and the access request is allowed. The access request is recorded in the access log, and a delay flag (X-RateLimit-Warning) is injected into the response header to prompt the client to appropriately reduce the request frequency.

[0037] Step 1042: If the overall load score is greater than or equal to the first preset load threshold and the overall load score is less than the second preset load threshold, then the rate limiting quota corresponding to the tag information in the rate limiting quota matrix is ​​reduced according to the preset quota adjustment formula.

[0038] Specifically, the preset load threshold includes a second preset load threshold, which defaults to 0.95. When 0.8 ≤ L < 0.95, it indicates that the system load is in a warning state, and the current limiting quota Q'[p][b][t][s] corresponding to the tag information in the current limiting quota matrix is ​​reduced according to the preset quota adjustment formula: Q'[p][b][t][s] = Q[p][b][t][s] × max(λ min , 1- (LL threshold )×decay_rate), where λ min The minimum retention factor is 0.3 by default, ensuring that core business operations are not completely blocked; decay_rate is the decay rate, with a default of 0.5. threshold The first preset load threshold is 0.8 by default. When the overall load score falls below the safety threshold (0.6), the quota will be gradually increased back to the original value according to the recovery rate to avoid quota oscillation. The response to access requests is as follows: if the user level in the access request's tag information is L2 or L3, the request will be queued. If the queuing times out, the corresponding response result (cache data or a downgraded page) will be returned; if the user level in the access request's tag information is L1, the request will be allowed normally.

[0039] Step 1043: If the comprehensive load score is greater than or equal to the second preset load threshold, determine whether the preset triggering condition for global circuit breaking is met. When global circuit breaking is triggered, determine whether to allow the access request based on the user level and service type in the tag information. When global circuit breaking is not triggered, execute the preset hard degradation processing strategy.

[0040] Specifically, when L ≥ 0.95, indicating a heavy system load, a check is performed to determine if a global circuit breaker has been triggered: The comprehensive load score for each of the second combinations in the second combination set is calculated, and a weighted average of all comprehensive load scores is taken to obtain the overall load score. If the overall load score is greater than the second preset load threshold, a global circuit breaker is triggered; otherwise, it is determined that no global circuit breaker has been triggered. When a global circuit breaker is triggered, the user level and business type within the tag information are used to determine whether to allow access requests: only L1 level core business requests are allowed, while L2 and L3 level requests are uniformly downgraded until the overall load score falls back to a safe level. When a global circuit breaker is not triggered, a hard degradation strategy is executed: a unified prompt page is returned for all non-core business requests, and core business requests are queued and wait; if the queue times out, cached data is returned.

[0041] Furthermore, in a global circuit breaker scenario, for access requests that are not allowed, such as those from users at level L2 or L3, the response is based on the business type: If the business type in the tag information is a query-type business, the most recent query result from the pre-cached cache is returned (the cache validity period is configured according to business characteristics, such as 5-30 seconds, to ensure that users obtain near-real-time data); if the business type in the tag information is an interactive business, a preset prompt is returned, such as a "System Busy" prompt page, and the request intent information of the access request is written to a preset waiting queue. After the load decreases, the user is asynchronously notified to resume service through a message queue; if the business type in the tag information is a transaction-type business, quota borrowing is triggered. For example, when the quota of a certain product × business is exhausted, it is supported to borrow from the idle quota of other low-load businesses of the same product. The borrowing ratio does not exceed the borrow_ratio of the current quota of the lender (default 30%), and the borrowing period is borrow_window (default 60 seconds). The quota is automatically returned after the timeout. After the quota borrowing is completed, the access request is allowed. By leveraging the quota allocation mechanism, resource utilization can be maximized without exceeding the overall load limit.

[0042] Furthermore, the process of asynchronously notifying users of service recovery via message queues is as follows: Request intent information includes user identifier, request business type and specific operation description, request enqueue timestamp, and associated resource identifier. The preset waiting queue is implemented based on message queue middleware (such as RabbitMQ or Kafka), and stored by topic according to business type to ensure reliable message delivery and persistence. The self-healing module continuously monitors the comprehensive load score of each dimension. When the load score of a certain dimension is lower than the safety threshold (0.6) for N consecutive periods (default 5 medium window periods, i.e., 5 minutes), it is determined that the load has fallen back, triggering the self-healing recovery process. After the self-healing recovery process is started, the system consumes the backlogged request intent information from the preset waiting queue, aggregates it by user dimension (the same user may have multiple degradation records), generates a recovery notification message, and sends it through the system's message push channel (such as WebSocket long connection, APP...). Push notifications or in-site messages will send recovery notifications to the corresponding users. The notification content includes a recovery status indicator and a link to the user's original request (which can directly jump to the previous operation page). If the user's original request involves a specific resource (such as a classroom entrance), the notification will include the current availability status of the resource to prevent users from clicking only to find that the resource has expired. At the same time, anti-harassment rules are set so that the same user can receive a maximum of one recovery notification (anti-harassment window) within 10 minutes. If the user has successfully accessed the resource through other means during the downgrade period (such as refreshing the page), the user's downgrade record will be automatically marked as "processed" and no further notifications will be sent. Messages in the downgrade waiting queue are set with a TTL (default 30 minutes). Messages that have not been consumed within the timeout period will be automatically discarded to avoid sending outdated recovery notifications.

[0043] In one embodiment of the present invention, in order to cope with the sudden increase in the load of the lending party in a quota borrowing scenario, the following protection mechanism is set: 1. Borrowing Security Boundary Limits: The maximum borrowing ratio for a lender is `borrow_ratio` (default 30%), meaning a maximum of 30% of the current quota can be borrowed. Even if the lender's load subsequently increases to the full quota, the remaining 70% quota can still guarantee its basic service capabilities. The 30% borrowing limit is determined through capacity assessment—for most business scenarios, 70% quota is sufficient to support 2-3 times the normal traffic surge (combined with progressive threshold adjustment), and will not cause service interruption due to borrowing.

[0044] 2. Emergency Recall Mechanism: When the lender's overall load score exceeds the first preset load threshold (0.8), the system automatically triggers an emergency recall—immediately terminating the outstanding loan quota and forcibly returning the loaned quota to the lender. The specific process is as follows: When the lender's overall load score exceeds 0.8, it sends an emergency recall request to the quota management center; the quota management center immediately deducts the loaned portion from the borrower's quota and returns it to the lender; the borrower loses its loan quota before the end of the current loan period, but it can still trigger cross-dimensional loaning or enter a downgrade process.

[0045] 3. Tiered Secondment Strategy: To reduce the frequency and impact of emergency recalls, secondments are divided into two priorities: Regular borrowing (within 50% of borrow_ratio): Emergency recall is triggered when the borrower's load exceeds 0.8; Excess borrowing (between 50% and 100% of borrow_ratio): When the borrower's load exceeds 0.6, a recall is immediately triggered to ensure that the borrower has sufficient buffer space.

[0046] 4. Resolving Quota Conflicts: When multiple borrowers borrow from the same lender at the same time, the quota management center sorts them according to borrowing priority and borrowing time, and processes all quota borrowing requests in sequence according to the sorting results, giving priority to the first borrowing request. When the lender has insufficient available borrowing quota, subsequent borrowing requests are downgraded or rejected.

[0047] During the aforementioned degradation process, degradation statistics for each dimension (degradation count, degradation level distribution, recovery time, etc.) are recorded in real time and displayed visually through the monitoring panel. When the load score is below the safety threshold for N consecutive periods (default 5 one-minute periods), a self-healing process is automatically triggered: gradually restoring the quotas for each dimension to their original values, clearing the degradation flag, restoring normal service, and notifying operations and maintenance personnel through the monitoring panel.

[0048] In one embodiment of the present invention, the following specific application scenarios are also provided: Example 1: Sudden Traffic Scenarios on Online Education Platforms An online education platform in a certain province simultaneously supports three business lines: instant messaging, data query, and file upload, all sharing the same system resources. During the period of "classes suspended but learning continues," the traffic for instant messaging surged.

[0049] Assume the basic quota for instant messaging service (biz_type=im) is 1000 times / minute, and the user level weights are L1=1.5 (active users such as teachers), L2=1.0 (ordinary students), and L3=0.5 (low-frequency visitors).

[0050] Step 1: A large number of student users request to enter; extract the tag Key=hash(edu, im, L2, peak_morning).

[0051] Step 2: The number of requests reaches 200 within a 5-second short window, and the number of requests reaches 850 within a minute. Load score L = 0.5 × (200 / 1000 × 12) + 0.35 × (850 / 1000) + 0.15 × (4200 / 5000) ≈ 1.2 + 0.298 + 0.126 = 1.0.

[0052] Step 3: L=1.0>0.95, triggering hard degradation. Checking the global load, it was found that the data query and file upload services had low loads and did not meet the global circuit breaker conditions. Therefore, hard degradation was only performed on the instant messaging service.

[0053] Step 4: Interactive requests from L2 student users return a unified "System Busy" message page, while query requests return cached data; core teaching requests from L1 teacher users are placed in a priority queue for normal processing.

[0054] Step 5: Instant messaging traffic gradually decreases after 15 minutes. If the load score is below 0.6 for 5 consecutive cycles, the self-healing process is initiated, and the quota is gradually restored to 1000 times / minute.

[0055] Results: Data query and file upload services remain unaffected, core teacher users' teaching interactions are guaranteed, and student users receive user-friendly downgrade prompts instead of error pages.

[0056] Example 2: Cross-dimensional quota borrowing scenario On an online education platform, the file upload service (biz_type=upload) experienced a peak in assignment submissions at 10:00 AM, exhausting its quota. Meanwhile, the data query service (biz_type=query) under the same product was only operating at 40% of its quota, triggering a cross-dimensional quota borrowing: 30% of the available quota (borrow_ratio=0.3) was borrowed from the data query service and allocated to the file upload service.

[0057] Results: The file upload service received a temporary supplementary quota, student assignments were submitted normally, and the quota was automatically returned after the 60-second borrowing period. The data query service remained operational despite a low load, even after 30% of the quota was borrowed.

[0058] Example 3: Global Circuit Breaker Protection Scenario During a province-wide unified online examination, an online education platform experienced a simultaneous surge in traffic across all business lines, with the overall load score reaching 0.97. This triggered global circuit breaker protection. The system only allowed core examination-related requests from L1-level users; other access requests responded according to their business type. For example, query requests returned cached data, while interactive requests returned a unified prompt page. The online examination ended after two hours, and the overall load score dropped to 0.5. After five consecutive periods below the safety threshold, the self-healing process was initiated, global quotas were gradually restored, and all services returned to normal.

[0059] Results: The core examination process was able to ensure the completion of invigilation and marking operations for high-value users such as teachers even under extreme loads, and the system did not experience a cascading failure.

[0060] In summary, this invention provides a method for rate limiting of access requests. It constructs a rate limiting quota matrix based on a preset multi-dimensional traffic tagging system, providing a refined rate limiting benchmark for various access requests. When an access request is received from a user, the tag information of the access request is extracted to determine the corresponding statistical dimension. This allows for the acquisition of statistical data corresponding to that statistical dimension within a preset multi-window, enabling real-time perception and accurate quantification of the current traffic status and providing a reliable data foundation for subsequent load assessment. A comprehensive load score is calculated based on the rate limiting quota matrix and statistical data, accurately reflecting the system load level. The comprehensive load score is compared with a preset load threshold, and graded rate limiting is performed based on the comparison results. This enables differentiated traffic control actions based on load levels, ensuring the system's operational stability and reliability under various traffic scenarios.

[0061] The present invention has the following beneficial effects: 1. Achieve multi-dimensional, refined rate limiting. By replacing single-dimensional fixed thresholds with a four-dimensional quota matrix of "product × business × user level × time period", rate limiting strategies can accurately match the importance of different businesses and the value differences of users.

[0062] 2. Achieve dynamic adaptive rate limiting based on real-time perception. Through multi-granularity sliding windows and comprehensive load scoring, the system can respond to traffic bursts within seconds, automatically adjust the rate limiting threshold, and eliminate the time delay of manual intervention.

[0063] 3. Implement cross-dimensional quota borrowing and global circuit breaker protection. While ensuring the overall system stability, maximize resource utilization efficiency through quota borrowing and prevent overall system overload through global circuit breaking.

[0064] 4. Implement differentiated degradation processing. Select different degradation strategies (cached return, queuing, asynchronous notification) based on business type and user level to minimize the impact of rate limiting on user experience and business continuity.

[0065] 5. Implement full lifecycle management of rate limiting policies. From quota setting, real-time monitoring, dynamic adjustment to degradation execution and self-healing recovery, a complete closed-loop management is formed, reducing operation and maintenance costs and improving system robustness.

[0066] Please refer to Figure 2 The present invention also provides a rate limiting system 200 for access requests, including a memory 201, a processor 202, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of a rate limiting method for access requests as described above.

[0067] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for rate limiting access requests, characterized in that, include: Obtain a preset multi-dimensional traffic tagging system, and construct a traffic limiting quota matrix according to the preset multi-dimensional traffic tagging system; In response to an access request sent by a user, the tag information of the access request is extracted, and statistical data of a preset multi-window is obtained based on the tag information; Calculate the comprehensive load score corresponding to the tag information based on the traffic limiting quota matrix and the statistical data; The comprehensive load score is compared with a preset load threshold, and tiered flow limiting is performed based on the comparison result.

2. The method for rate limiting of access requests according to claim 1, characterized in that, Construct a traffic limiting quota matrix according to the preset multi-dimensional traffic labeling system, including: The preset multi-dimensional traffic tagging system includes product dimension, business dimension, user level dimension, and time period dimension; Each preset product in the product dimension is combined with each preset business type in the business dimension to obtain a first combination set. The preset total carrying capacity and the historical peak traffic of each first combination in the first combination set are obtained. The basic quota of the corresponding first combination is set based on the preset total carrying capacity and the historical peak traffic. The basic quotas are adjusted according to the weight coefficients corresponding to each preset user level in the user level dimension and the scaling factors corresponding to each preset time period in the time period dimension. The rate limiting quotas for each preset user level under each preset product and each preset business type in each time period are obtained. A rate limiting quota matrix is ​​constructed based on all rate limiting quotas.

3. The method for rate limiting of access requests according to claim 2, characterized in that, Based on the label information, statistical data for a preset multi-window system is obtained, including: The preset multi-window includes a first sliding window, a second sliding window, and a third sliding window; The preset products, preset service types, and preset user levels are combined to obtain a second set of combinations. Using each second combination in the second set of combinations as a statistical dimension, the access requests in the first sliding window, the second sliding window, and the third sliding window are counted, and the statistical results are stored in the counter corresponding to each sliding window.

4. The method for rate limiting of access requests according to claim 3, characterized in that, Based on the label information, statistical data for a preset multi-window system is obtained, including: Obtain a second combination that matches the tag information from the second combination set, and read the statistics of the second combination in the first sliding window, the second sliding window and the third sliding window respectively from the counters corresponding to each sliding window.

5. The method for rate limiting of access requests according to claim 4, characterized in that, Based on the rate limiting quota matrix and the statistical data, the comprehensive load score corresponding to the tag information is calculated, including: The comprehensive load score corresponding to the tag information is calculated based on the statistics of the second combination matching the tag information in each sliding window, the preset normalization coefficient of each sliding window, the preset weighting factor of each sliding window, and the rate limiting quota matrix. L=α×(N short / Q[p][b][t][s]×W short )+β×(N mid / Q[p][b][t][s]×W mid )+γ× (N long / Q[p][b][t][s]×W long ); In the formula, L represents the comprehensive load score corresponding to the tag information, Q[p][b][t][s] represents the traffic limiting quota of service b under product p for user level t in time period s, and the tag information includes product p, service b, user level t and time period s; N short N mid N long These represent the statistics corresponding to the first, second, and third sliding windows, respectively; W short W mid W long α, β, and γ represent the preset normalization coefficients corresponding to the first, second, and third sliding windows, respectively; α, β, and γ represent the preset weighting factors corresponding to the first, second, and third sliding windows, respectively.

6. The method for rate limiting of access requests according to claim 3, characterized in that, The comprehensive load score is compared with a preset load threshold, and tiered rate limiting is performed based on the comparison result, including: The preset load threshold includes a first preset load threshold and a second preset load threshold; If the overall load score is less than the first preset load threshold, the access request is allowed and the access request is recorded in the access log. If the overall load score is greater than or equal to the first preset load threshold and the overall load score is less than the second preset load threshold, then the rate limiting quota corresponding to the tag information in the rate limiting quota matrix is ​​reduced according to the preset quota adjustment formula. If the overall load score is greater than or equal to the second preset load threshold, it is determined whether the preset triggering condition for global circuit breaking is met. When global circuit breaking is triggered, it is determined whether to allow the access request based on the user level and service type in the tag information. When global circuit breaking is not triggered, a preset hard degradation processing strategy is executed.

7. The method for rate limiting of access requests according to claim 6, characterized in that, Determine whether the preset trigger conditions for global circuit breaking are met, including: Calculate the comprehensive load score corresponding to each second combination in the second combination set, and perform a weighted average of all comprehensive load scores to obtain the overall load score. If the overall load score is greater than the second preset load threshold, it is determined that the global circuit breaker is triggered; otherwise, it is determined that the global circuit breaker is not triggered.

8. The method for rate limiting of access requests according to claim 6, characterized in that, When a global circuit breaker is triggered, determining whether to allow the access request based on the user level and service type within the tag information also includes: When the access request is not permitted, the access request shall be responded to according to the service type: If the business type in the tag information is a query business, then return the query result that is most recent to the access request time in the pre-cached cache; If the business type in the tag information is an interactive business, then a preset prompt will be returned, and the request intent information of the access request will be written into a preset waiting queue. If the business type in the tag information is a transaction-related business, quota borrowing will be triggered. Once the quota borrowing is completed, the access request will be allowed.

9. The method for rate limiting of access requests according to claim 8, characterized in that, Triggering quota borrowing also includes: When multiple borrowers request quota transfers from the same lender, all quota transfer requests are sorted according to the transfer ratio and transfer time, and then processed sequentially based on the sorting results.

10. A rate-limiting system for access requests, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the rate limiting method for access requests as described in any one of claims 1 to 9.

Images