Service traffic limiting method and device, server and storage medium
By acquiring traffic data of the target service, determining the rate limiting algorithm and parameters, and constructing a dynamic rate limiting strategy, the problem that traditional rate limiting methods cannot adapt to dynamic traffic patterns is solved, and the dynamic adaptability of service rate limiting and its ability to effectively cope with sudden loads are realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-30
- Publication Date
- 2026-03-31
Smart Images
Figure CN121771111A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of microservices technology, and in particular to a service rate limiting method, apparatus, server, and storage medium. Background Technology
[0002] In modern microservice architectures, service rate limiting is an important technique used to protect the system from overload and ensure high availability.
[0003] Traditional rate limiting methods are typically based on static configurations, which are ill-suited to dynamically changing traffic patterns and cannot effectively handle sudden surges in traffic or gradually increasing loads. These limitations of static rate limiting strategies can lead to service unavailability during peak hours or wasted resources during low-load periods. Summary of the Invention
[0004] This application provides a service rate limiting method, device, server, and storage medium to solve the problem that traditional rate limiting methods based on static configuration are difficult to adapt to dynamically changing traffic patterns and cannot effectively cope with sudden traffic surges or gradually increasing loads.
[0005] In a first aspect, embodiments of this application provide a service rate limiting method, including:
[0006] Obtain the initial traffic data for the target service;
[0007] The first traffic data is used to determine the rate limiting algorithm and the rate limiting parameters of the rate limiting algorithm, and the first rate limiting strategy is constructed based on the rate limiting algorithm and the rate limiting parameters; the rate limiting algorithm is an algorithm for adjusting the target service traffic, and the rate limiting parameters are key parameters contained in the rate limiting algorithm;
[0008] Based on the first rate limiting strategy, the current traffic control strategy of the target service is adjusted to obtain the target traffic control strategy, and the target traffic control strategy is run to implement rate limiting on the target service.
[0009] Secondly, embodiments of this application provide a service rate limiting device, including:
[0010] The first traffic data acquisition module is used to acquire the first traffic data of the target service.
[0011] The first rate limiting strategy determination module is used to determine a rate limiting algorithm and rate limiting parameters of the rate limiting algorithm based on the first traffic data, and to construct a first rate limiting strategy based on the rate limiting algorithm and the rate limiting parameters; the rate limiting algorithm is an algorithm for limiting the traffic of a target service, and the rate limiting parameters are key parameters contained in the rate limiting algorithm;
[0012] The first rate limiting strategy implementation module is used to adjust the current traffic control strategy of the target service based on the first rate limiting strategy to obtain the target traffic control strategy, and run the target traffic control strategy to implement rate limiting on the target service.
[0013] Thirdly, embodiments of this application provide a server, the server comprising:
[0014] At least one processor;
[0015] and a memory communicatively connected to the at least one processor;
[0016] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the service rate limiting method described in any embodiment of this application.
[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the service rate limiting method described in any embodiment of this application.
[0018] The technical solution of this application embodiment obtains first traffic data of a target service; determines a rate limiting algorithm and rate limiting parameters based on the first traffic data, and constructs a first rate limiting strategy based on the rate limiting algorithm and rate limiting parameters; the rate limiting algorithm is an algorithm for limiting the traffic of the target service, and the rate limiting parameters are key parameters included in the rate limiting algorithm; the target service's current traffic control strategy is adjusted based on the first rate limiting strategy to obtain a target traffic control strategy, and the target traffic control strategy is run to rate limit the target service. By using a dynamic rate limiting strategy based on real-time traffic data, the traditional statically configured rate limiting method is unable to adapt to dynamically changing traffic patterns and cannot effectively cope with sudden traffic spikes or gradually increasing loads. This enables the service's rate limiting strategy to adapt to dynamically changing traffic patterns, effectively cope with sudden traffic spikes or gradually increasing loads, and improve the effectiveness of microservice rate limiting.
[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 A flowchart of a service rate limiting method provided in Embodiment 1 of this application;
[0022] Figure 2 A flowchart of a service rate limiting method provided in Embodiment 2 of this application;
[0023] Figure 3 This is a schematic diagram of a service rate limiting device provided in Embodiment 3 of this application;
[0024] Figure 4 A schematic diagram of the server structure for implementing the service rate limiting method of this application embodiment. Detailed Implementation
[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0027] Example 1
[0028] Figure 1This is a flowchart illustrating a service rate limiting method provided in Embodiment 1 of this application. This embodiment is applicable to rate limiting of microservices. The method can be executed by a service rate limiting device, which can be implemented in hardware and / or software and configured in a server. Figure 1 As shown, the method includes:
[0029] S110, Obtain the first traffic data of the target service.
[0030] The target service can be understood as a microservice running on the server, which refers to a program, routine, or process that performs a specified system function to support other programs. All services running on the server at any given time are potential targets for rate limiting. The first traffic data is the real-time traffic data obtained from the target service, which can be understood as the website traffic accessed by the target service, and may include, but is not limited to, requests per second (QPS), data transfer volume, number of connections, and response time.
[0031] For example, monitoring tools or algorithms are used to collect initial traffic data for the target service and store this data in a database for subsequent analysis. This initial traffic data serves as the basis for formulating and adjusting rate limiting strategies.
[0032] S120. Determine the rate limiting algorithm and the rate limiting parameters of the rate limiting algorithm based on the first traffic data, and construct the first rate limiting strategy based on the rate limiting algorithm and the rate limiting parameters.
[0033] In this context, a rate limiting algorithm can be understood as an algorithm that restricts the traffic of a target service, while rate limiting parameters can be understood as key parameters included in the rate limiting algorithm. The rate limiting parameters in the rate limiting algorithm can be initially set to default values and can be adjusted subsequently based on initial traffic data. The rate limiting algorithm and rate limiting parameters constitute the first rate limiting strategy. The first rate limiting strategy can be understood as a rate limiting strategy formulated based on traffic data.
[0034] Optionally, rate limiting algorithms can include: fixed window counter algorithm, sliding window counter algorithm, token bucket algorithm, and leaky bucket algorithm. The fixed window counter algorithm divides time into fixed-size windows (e.g., 1 second, 1 minute), with a fixed number of requests allowed within each window. The sliding window counter algorithm divides time into finer-grained sub-windows (e.g., dividing 1 second into 10 sub-windows of 100ms each), counting the number of requests within the most recent complete window period, solving the boundary spike problem of the fixed window algorithm, resulting in smoother rate limiting. The token bucket algorithm is a rate limiting algorithm based on a token bucket, adding tokens to the bucket at a constant rate. Requests consume tokens upon arrival; when no tokens are available, requests are rejected or waited for, allowing traffic to reach the required burst level. The leaky bucket algorithm is a rate limiting algorithm based on a leaky bucket, where requests first enter the leaky bucket, and the bucket overflows at a certain rate. When the inflow rate is too high or the bucket is full, it overflows, and requests are rejected.
[0035] Specifically, the first traffic data of the target service acquired in real time is analyzed, and the rate limiting algorithm and its rate limiting parameters are determined based on the data analysis results, thereby determining the first rate limiting strategy composed of the rate limiting algorithm and the rate limiting parameters.
[0036] For example, the method of determining the rate limiting algorithm and the rate limiting parameters of the rate limiting algorithm based on the first traffic data can be based on matching a suitable rate limiting algorithm based on an algorithm mapping method or a neural network model, and determining the rate limiting parameters based on parameter calculation.
[0037] For example, data analysis methods may include: feature processing and feature analysis of traffic data. Feature processing methods may include calculating traffic data metrics on key indicators, and determining traffic data features based on feature processing models (such as neural network models). Feature analysis methods may include trend analysis, threshold monitoring, anomaly detection, response time distribution analysis, and data correlation analysis. Trend analysis methods may analyze the trend differences between real-time and historical traffic data, such as comparing the current number of requests with the historical average number of requests to identify rising or falling traffic trends. Threshold monitoring methods may set baselines or thresholds, and compare key indicators of traffic data with these baselines or thresholds. For example, if response time (or number of requests, error rate, etc.) is significantly higher than the usual average, it can be determined that the service may be overloaded. Anomaly detection methods may be based on specific rules (such as RPS exceeding a certain value or error rate exceeding a certain percentage) to identify sudden events, or use anomaly detection algorithms (such as Isolation Forest, DBSCAN) to identify complex anomaly patterns, such as the non-linear relationship between response time and number of requests. Response time distribution analysis methods may examine the distribution of response times to identify tail latency issues that may affect user experience. Data correlation analysis can be used to correlate data such as request count, CPU utilization, and error rate for multi-dimensional analysis to identify the root cause of problems. For example, high RPS accompanied by high error rate and high CPU utilization may indicate a resource bottleneck.
[0038] S130. Based on the first rate limiting strategy, adjust the current traffic control strategy of the target service to obtain the target traffic control strategy, and run the target traffic control strategy to implement rate limiting on the target service.
[0039] The current traffic control policy can be understood as the policy implemented by the target service to control traffic.
[0040] Specifically, the first rate limiting strategy generated based on traffic data adjusts the current traffic control strategy of the target service to obtain the target traffic control strategy, and then applies the adjusted target traffic control strategy to the target service to achieve rate limiting operation on the target service, so that the traffic strategy of the target service can adapt to dynamically changing traffic and effectively cope with sudden traffic or gradually increasing load.
[0041] This application provides a service rate limiting method, which involves acquiring first traffic data of a target service; determining a rate limiting algorithm and its parameters based on the first traffic data; constructing a first rate limiting strategy based on the algorithm and parameters; adjusting the current traffic strategy of the target service based on the first rate limiting strategy to obtain a target traffic control strategy; and running the adjusted target traffic control strategy to complete the rate limiting operation on the target service. This method enables the service rate limiting strategy to adapt to dynamically changing traffic patterns, effectively cope with sudden traffic surges or gradually increasing loads, and improve the effectiveness of service rate limiting.
[0042] Example 2
[0043] Figure 2 This is a flowchart of a service rate limiting method provided in Embodiment 2 of this application. This embodiment further refines the method for determining the rate limiting algorithm and rate limiting parameters of the above embodiments as follows: determining the rate limiting algorithm and its rate limiting parameters based on the first traffic data includes: calculating key indicator data of the first traffic data on preset key indicators; determining the traffic characteristics of the target service based on the key indicator data; matching a suitable rate limiting algorithm based on the traffic characteristics; and determining the rate limiting parameters corresponding to the rate limiting algorithm based on the key indicator data.
[0044] like Figure 2 As shown, the method includes:
[0045] S210, Obtain the first traffic data of the target service.
[0046] S220. Calculate the key indicator data of the first flow data on the preset key indicators.
[0047] Among them, the preset key indicators can be understood as indicators used to reflect changes in the traffic of the target service, and the key indicator data can be understood as values or results calculated or analyzed based on the preset key indicators.
[0048] Optionally, preset key metrics may include at least one of the following: Requests Per Second (RPS), standard deviation of request rate, response time, error rate, request distribution, processor utilization, memory utilization, number of concurrent connections, request queue length, and network bandwidth.
[0049] Requests per second (PS) can be understood as the number of requests a target service receives per second. It is used to identify the overall traffic level and bursts of traffic for a service, for example...
[0050] Total Requests represents the number of requests received by the target service within a period, and Time Interval represents the time period (in seconds). The standard deviation of the request rate describes the magnitude of traffic fluctuations, for example... Where, x i This represents the number of requests in each time period, μ is the average number of all requests, and N is the number of time periods. Response time can be understood as the time required for a service to respond to a request, helping to identify potential performance bottlenecks. It can be P95 response time (time to respond to 95% of requests) and / or P99 response time (time to respond to 99% of requests). Error rate can be understood as the proportion of requests that encounter errors within a certain period, identifying service anomalies and stability issues. Request distribution is used to differentiate the access frequency of various request paths and methods to identify hot paths or bottlenecks. Processor utilization and memory utilization reflect the resource usage of the service, helping to analyze whether the load is approaching server capacity; these can be obtained through monitoring tools. Concurrent connections can be understood as the number of active connections in the target service at a certain moment, reflecting the service's concurrent processing capacity; this can be obtained through application server or load balancer monitoring interfaces. Request queue length can be understood as the number of requests waiting to be processed, used to monitor load backlog; this can be tracked through server-side queue management. Network bandwidth reflects network bandwidth usage, preventing performance problems caused by insufficient network resources.
[0051] S230. Match a suitable rate limiting algorithm based on key indicator data, and determine the rate limiting parameters corresponding to the rate limiting algorithm.
[0052] Specifically, the system's (i.e., the server hosting the target service) load status and resource utilization are determined based on key performance indicator (KPI) data, traffic volatility is assessed, and a suitable rate limiting algorithm is selected based on the load status, resource utilization, and traffic volatility. Specific rate limiting parameters are then generated based on the selected algorithm and the KPI data. This process ensures that the initial rate limiting strategy can adaptively adjust according to the system's current load and traffic patterns.
[0053] In an optional embodiment, matching a suitable rate limiting algorithm based on the key indicator data includes:
[0054] The performance status of the target service is determined based on the key indicator data, and the performance status includes at least one of: load status, traffic volatility, and resource utilization.
[0055] A suitable rate limiting algorithm is matched based on the performance status and the rate limiting target. The rate limiting algorithms include: fixed window counter algorithm, sliding window counter algorithm, token bucket algorithm and leaky bucket algorithm.
[0056] The performance status of the target service can be understood as representing its performance in terms of load, traffic, and resources. Performance status can include at least one of load status, traffic volatility, and resource utilization. The rate limiting target can be understood as the objective that rate limiting needs to achieve, such as preventing service overload, optimizing user experience, and making reasonable use of resources. Load status can be determined based on at least one of RPS, processor utilization, and memory utilization, while volatility can be determined based on the difference in request rates.
[0057] For example, load status and traffic volatility are calculated based on key performance indicator (KPI) data. A fixed window counter algorithm is used for low volatility and stable traffic; a sliding window counter algorithm is used for medium volatility; and a token bucket algorithm is used for high volatility and bursts of traffic. A leaky bucket algorithm is used for high concurrency and situations requiring strict control of traffic output rates.
[0058] For example, rate limiting objectives can be: preventing service overload: limiting the number of requests entering the system when a sudden surge in traffic or high load is detected to prevent service crashes. Rate limiting objectives can also be: optimizing user experience: serving as many users as possible while ensuring system stability, avoiding excessive rate limiting that could lead to rejected user requests. Furthermore, rate limiting objectives can also be: rationally utilizing resources: appropriately relaxing rate limiting policies under low load conditions to improve resource utilization.
[0059] For example, the rate limiting parameters of a fixed window counter algorithm may include a time window and a request limit. The time window can be determined based on the number of requests and system volatility, while the request limit can be the product of the number of requests and the time window size, such as RequestLimit = RPS × Time Window. Adjustment strategies can include appropriately reducing the request limit under high load conditions to prevent system overload, and appropriately increasing the request limit under low load conditions to improve resource utilization.
[0060] The rate limiting parameters for a sliding window counter algorithm can include a time window and a request limit. The time window can be determined based on traffic volatility and system load. Adjustment strategies can include shortening the window size or reducing the request limit during periods of high volatility, and appropriately increasing the window size or imposing a request limit during periods of low volatility.
[0061] The rate-limiting parameters for the token bucket algorithm can include: token generation rate and bucket capacity. The token generation rate is the product of the number of requests and the token bucket factor, which can be determined based on the system's allowed burst traffic multiple. The bucket capacity is the product of the token generation rate and the burst duration, which can be the maximum allowed burst duration. The adjustment strategy can be to reduce the token generation rate under high load conditions to limit requests, and increase the token generation rate under low load conditions to allow more requests to pass.
[0062] The rate limiting parameters for the leaky bucket algorithm can include: the leaky bucket output rate and the bucket capacity. The leaky bucket output rate is the product of the number of requests and the leaky bucket capacity factor. The leaky bucket capacity factor can be determined based on system stability requirements. The bucket capacity is the product of the leaky bucket output rate and the capacity factor. The capacity factor can be determined based on the maximum backlog of requests the system can handle. Adjustment strategies can include appropriately reducing the bucket capacity under high load conditions to limit burst traffic, and appropriately increasing the bucket capacity under low load conditions to allow more burst traffic to pass through.
[0063] Optionally, after matching a suitable rate limiting algorithm based on the key indicator data, the method further includes:
[0064] If the key indicator data meets the preset algorithm switching conditions, the rate limiting algorithm of the target service will be switched to the rate limiting algorithm corresponding to the preset algorithm switching conditions.
[0065] The preset algorithm switching conditions can be set according to actual needs. For example, the preset algorithm switching conditions can be the conditions met by the preset key indicators corresponding to the rate limiting algorithm before the switch.
[0066] For example, (1) the switching conditions from the fixed window counter algorithm to the token bucket algorithm can be manifested as at least one of the following: high burst traffic (which can be measured by the number of requests and the standard deviation of the request rate), abnormal system performance indicators (which can be measured by P95 response time and error rate), and increased resource utilization (which can be measured by CPU utilization and memory utilization). The switching logic is that the fixed window counter algorithm is prone to causing a large number of requests to be rate-limited due to the boundary effect when dealing with high burst traffic; while the token bucket algorithm can handle burst traffic more flexibly, allowing high-concurrency requests to pass through in a short period of time, and is suitable for switching to use under the conditions of high burst traffic and resource shortage.
[0067] (2) The switching conditions for switching from the token bucket algorithm to the fixed window counter algorithm can be at least one of the following: traffic tends to stabilize or decrease, resource utilization decreases, or user experience requirements change (e.g., business requirements are adjusted, allowing for stricter rate limiting to optimize resource utilization). The switching logic is that when system traffic tends to stabilize or decrease and resource utilization decreases significantly, the algorithm can be switched to the fixed window counter algorithm to reduce algorithm complexity and improve traffic processing capacity.
[0068] (3) The switching condition for switching from the fixed window counter algorithm to the sliding window counter algorithm is at least one of the following: increased traffic volatility (which can be measured by the standard deviation of request rate), increased response time and error rate, and increased resource utilization. The switching logic is that when traffic volatility is large, the fixed window counter algorithm may cause sudden traffic surges to be throttled due to boundary effects. The sliding window counter algorithm, through more accurate request statistics, can more smoothly control the request rate, reduce system pressure, and is suitable for switching when traffic volatility increases.
[0069] (4) The switching conditions for switching from the sliding window counter algorithm to the fixed window counter algorithm are at least one of the following: traffic tends to stabilize or decrease, resource utilization decreases, and user experience requirements change (e.g., business needs allow for higher rate limiting efficiency to simplify the algorithm). The switching logic is that when system traffic tends to stabilize or decrease and resource utilization decreases significantly, the algorithm can be switched to the fixed window counter algorithm to simplify the rate limiting logic and improve traffic processing capabilities.
[0070] (5) The switching conditions from the fixed window counter algorithm to the leaky bucket algorithm are at least one of the following: continuous high concurrency and stable traffic (which can be measured by the number of requests and the standard deviation of the request rate), strict traffic output requirements (which can be expressed by the request rate), and sufficient system resources (which can be measured by CPU utilization and memory utilization). The switching logic is that under high concurrency and stable traffic conditions, the leaky bucket algorithm can strictly control the request output rate, prevent system overload, and ensure stable system operation.
[0071] (6) The switching conditions for switching from the leaky bucket algorithm to the fixed window counter algorithm are at least one of the following: traffic decreases or tends to stabilize, resource utilization decreases, and user experience requirements change (e.g., business needs allow for higher traffic bursts to improve user experience). The switching logic is that when the system no longer faces high concurrency pressure and resource utilization is low, it can switch back to the fixed window counter to simplify the rate limiting logic and improve traffic processing capabilities.
[0072] (7) The switching conditions for switching from the token bucket algorithm to the sliding window counter algorithm are at least one of the following: continuous traffic fluctuations, increased response time and error rate, and strained system resources. The switching logic is that when the system frequently processes bursts of traffic and the token bucket algorithm cannot effectively reduce response time and error rate, the sliding window counter algorithm can better control the request rate and reduce system pressure through more accurate request statistics and smooth rate limiting.
[0073] (8) The switching conditions for switching from the sliding window counter algorithm to the token bucket algorithm can be at least one of the following: traffic tends to stabilize or sudden demand decreases, resource utilization decreases, and user experience is optimized (e.g., business needs allow for a certain degree of traffic bursts to improve user experience). The switching logic is that when the system traffic volatility decreases and resource utilization decreases, it can switch to the token bucket algorithm to restore the ability to handle sudden traffic and improve user experience.
[0074] (9) The switching conditions for switching from the sliding window counter algorithm to the leaky bucket algorithm can be at least one of the following: high concurrency and continuous stable traffic, strict requirements on traffic output rate, and reasonable resource utilization. The switching logic is that the sliding window counter algorithm is suitable for scenarios with large traffic fluctuations, while the leaky bucket algorithm is suitable for scenarios that require strict control of the request output rate. When the traffic tends to be stable and strict control of the output rate is required, the leaky bucket algorithm can better smooth request processing and prevent system overload.
[0075] (10) The switching conditions for switching from the vulnerability rate limiting algorithm to the sliding window counter algorithm are at least one of increased traffic volatility, changes in system load, and abnormal response time or error rate. The switching logic is that when the system begins to face increased traffic volatility or increased load, the sliding window counter algorithm can more accurately control the request rate, reduce system pressure, and avoid the fixed output rate limitation of the leaky bucket algorithm affecting the system's responsiveness.
[0076] (11) The switching conditions from the vulnerability-based rate limiting algorithm to the token bucket algorithm can be at least one of the following: increased traffic volatility, increased resource utilization, and user experience requirements (e.g., the need for higher traffic burst capabilities to improve user experience). The switching logic is that when the system begins to face increased traffic volatility, the token bucket algorithm can handle burst traffic more flexibly, allowing high-concurrency requests to pass through in a short period of time, and adapting to dynamically changing traffic patterns.
[0077] (12) The switching conditions for switching from the token bucket algorithm to the leaky rate limiting algorithm can be at least one of the following: the traffic tends to be stable and the concurrency is high, the requirements for the traffic output rate are strict, and the resource utilization is reasonable. The switching logic is that when the system traffic tends to be stable and the concurrency is high, the leaky bucket algorithm can strictly control the request output rate, prevent the system from being overloaded, and maintain the stable operation of the system.
[0078] S240. The first rate limiting strategy is constructed based on the rate limiting algorithm and rate limiting parameters.
[0079] S250. Adjust the current traffic policy of the target service based on the first traffic limiting policy, and run the adjusted traffic policy to complete the traffic limiting operation on the target service.
[0080] The technical solution of this application embodiment involves: acquiring first traffic data of a target service; calculating key indicator data of the first traffic data on preset key indicators; matching a suitable rate limiting algorithm based on the key indicator data and determining the rate limiting parameters corresponding to the rate limiting algorithm; determining the rate limiting parameters corresponding to the rate limiting algorithm based on the key indicator data, and constructing a first rate limiting strategy based on the rate limiting algorithm and the rate limiting parameters; adjusting the current traffic control strategy of the target service based on the first rate limiting strategy to obtain a target traffic control strategy; and running the adjusted target traffic control strategy to rate limit the target service. By determining the matching rate limiting algorithm and rate limiting parameters based on traffic data to implement rate limiting on the target service, the rate limiting strategy of the service can adapt to dynamically changing traffic patterns, effectively cope with sudden traffic surges or gradually increasing loads, and improve the effect of service rate limiting.
[0081] As an optional embodiment, before obtaining the first traffic data of the target service, the method further includes:
[0082] Get server configuration information;
[0083] The server configuration information is input into the empirical model to obtain the second rate limiting strategy; the empirical model is obtained by training a machine learning model based on a sample dataset consisting of historical rate limiting data and the server configuration information.
[0084] Based on the second rate limiting strategy, the current traffic control strategy of the target service is adjusted to obtain the target traffic control strategy, and the adjusted target traffic control strategy is used to limit the rate of the target service.
[0085] The second rate limiting strategy can be understood as a rate limiting strategy based on server configuration information. Server configuration information can be considered as the configuration information related to the server running the target service.
[0086] Specifically, in the initial stage of the target service operation, a suitable second rate limiting strategy can be matched according to the server configuration information. The specific method is to input the obtained server configuration information of the target service into the trained empirical model to obtain the second rate limiting strategy that matches the server configuration information. The empirical model can be a machine learning model or a deep learning model, etc. The training method of the empirical model can be (1) collecting historical rate limiting data and server configuration information of the server. The server configuration information can include, for example, the number of CPU cores, memory, network bandwidth, etc. The historical rate limiting data can include the number of requests, response time and error rate in different time periods, different rate limiting algorithms and their parameter configurations, system performance before and after the rate limiting strategy change, etc. The collected data can also be preprocessed, such as cleaning and organizing historical data, handling missing values and outliers, and extracting important features based on feature engineering. (2) Selecting a suitable machine learning algorithm (such as linear regression, decision tree, random forest, neural network, etc.) to train the machine learning model, and cross-validation and other methods can be used to evaluate the model performance to prevent overfitting; through evaluation indicators (such as MSE, RMSE, R 2 (etc.) Evaluate the model's performance and adjust the model parameters to improve prediction accuracy.
[0087] Specifically, in the initial stage, the current traffic control policy of the target service is adjusted based on the second rate limiting strategy. The adjusted traffic control policy is then run to rate limit the target service, enabling it to quickly enter the rate-limited state. After obtaining the first traffic data of the target service and determining the matching first rate limiting strategy, the current traffic control policy of the target service is adjusted again based on the first rate limiting strategy. The adjusted traffic control strategy is then run to rate limit the target service, allowing the service's rate limiting strategy to adapt to dynamically changing traffic patterns.
[0088] As an optional embodiment, after adjusting the current traffic control policy of the target service based on the first traffic limiting policy and running the adjusted traffic control policy to limit the traffic of the target service, the method further includes:
[0089] Obtain the rate limiting feedback data for the target service;
[0090] If the rate limiting feedback data meets the preset parameter adjustment conditions, the rate limiting parameters in the current flow control strategy are adjusted according to the rate limiting feedback data.
[0091] Rate limiting feedback data can be understood as the data returned by the target service after the rate limiting policy is implemented. This feedback data can include traffic data and service performance data. Preset parameter adjustment conditions can be understood as the conditions that trigger adjustments to the rate limiting policy, such as a sudden increase in response time, an increase in error rate, CPU utilization exceeding a specified threshold, or response time exceeding a time threshold.
[0092] Specifically, after the first rate-limiting policy is implemented on the target service, the traffic and service performance of the target service are continuously monitored using monitoring tools. A decision engine is used to analyze the current rate-limiting policy; for example, the decision engine can refer to historical records, compare rate-limiting feedback data with real-time data, and evaluate the effectiveness of the current policy. If the rate-limiting feedback data meets the preset parameter adjustment conditions, the rate-limiting parameters of the traffic control policy are adjusted based on the rate-limiting feedback data. As an optional embodiment, after adjusting the current traffic control policy of the target service based on the first rate-limiting policy and running the adjusted traffic control policy to rate-limit the target service, the following steps are also included:
[0093] Obtain the second traffic data of the target service;
[0094] The second traffic data is input into the traffic prediction model to obtain the predicted traffic data;
[0095] Adjust the flow limiting parameters in the current flow control strategy based on the predicted flow data.
[0096] The second type of traffic data can be understood as traffic data used to predict future traffic trends. This can include current or historical traffic data of the target service, as well as historical peak data and periodic traffic characteristics. Predicted traffic data can be understood as traffic data predicted by a traffic prediction model. A traffic prediction model can be understood as a model used to predict the future traffic trends of the target service; it can be a time-series-based machine learning model, such as LSTM or Prophet.
[0097] Specifically, the collected second-generation traffic data is input into the traffic prediction model, which then predicts the traffic data for the target service. Based on the predicted traffic data, the optimal rate-limiting parameters are calculated, and the rate-limiting parameters in the currently implemented traffic control strategy are adjusted. For example, the token generation rate or the sliding window size can be adjusted to cope with upcoming traffic fluctuations.
[0098] For example, predicted traffic data indicates that traffic will increase or decrease, and the system automatically adjusts the rate limiting parameters. For instance, if traffic is expected to surge within the next 15 minutes, the system adjusts the rate limiting policy in advance to increase bucket capacity or reduce request frequency. The adjusted policy is applied through the service configuration management tool and takes effect immediately. After the optimized rate limiting policy is implemented, the system continuously monitors its effectiveness. If the expected results are not achieved, the system will further adjust the rate limiting parameters or policy, forming an adaptive closed loop.
[0099] Optionally, a safety margin can be set based on the flow limiting parameters determined by the predicted traffic data to prevent system overload caused by prediction errors. For example, the predicted RPS can be increased by 10% as a safety margin. The reserved margin can be dynamically adjusted according to real-time traffic data to ensure that the system still has sufficient capacity to cope with sudden traffic surges.
[0100] This implementation continuously learns traffic change patterns through machine learning algorithms. As more real-time traffic data is introduced, the traffic prediction model will be updated regularly to improve the accuracy of future predictions and the adaptability of traffic limiting strategies.
[0101] Optionally, if the first rate limiting strategy determined based on the first traffic data collected in real time differs from the first rate limiting strategy (or rate limiting parameters) determined based on the predicted data from the second traffic data, and the rate limiting strategy corresponding to historical traffic data, priorities can be set for the impact of real-time traffic data, predicted traffic data, and historical traffic data on the rate limiting parameters, and these priorities can be followed in order. For example, parameter adjustment suggestions based on real-time traffic data can override suggestions based on predicted and historical traffic data, and suggestions based on predicted traffic data can override suggestions based on historical data. For instance, if real-time traffic data shows high load but predicted traffic data does not indicate future traffic increases, the request limit can be immediately reduced or a more stringent rate limiting algorithm can be switched based on the real-time traffic data to ensure system stability. If real-time traffic data is normal but predicted traffic data indicates future traffic increases, the rate limiting parameters can be adjusted in advance based on the predicted traffic data, such as increasing the request limit or preparing to switch to a more efficient rate limiting algorithm. If real-time traffic data is normal, predicted traffic data is normal, and only historical traffic data provides adjustment suggestions, the rate limiting parameters set based on historical data can be used as the default strategy.
[0102] In addition, before officially implementing a rate limiting policy, it can be validated to ensure its effectiveness and security. Validation of the rate limiting policy can include simulation testing, canary releases, and performance monitoring. Simulation testing involves simulating real-world traffic in a test environment to verify the effect of the rate limiting policy. Canary releases involve applying the new policy to a subset of instances to observe its performance. Performance monitoring involves continuously monitoring key metrics to ensure the policy does not negatively impact normal traffic. Implementation of the rate limiting policy involves applying the validated policy to the production environment, using a configuration center or service governance framework to support hot updates of the rate limiting policy without restarting the service. In a distributed environment, it is necessary to ensure the consistency of the rate limiting policy across all service instances.
[0103] Example 3
[0104] Figure 3 This is a schematic diagram of a service rate limiting device provided in Embodiment 3 of this application. Figure 3 As shown, the device includes: a first traffic data acquisition module 310, a first rate limiting strategy determination module 320, and a first rate limiting strategy implementation module 330; wherein,
[0105] The first traffic data acquisition module 310 is used to acquire the first traffic data of the target service;
[0106] The first rate limiting strategy determination module 320 is used to determine a rate limiting algorithm and rate limiting parameters of the rate limiting algorithm based on the first traffic data, and to construct a first rate limiting strategy based on the rate limiting algorithm and the rate limiting parameters; the rate limiting algorithm is an algorithm for adjusting the target service traffic, and the rate limiting parameters are key parameters included in the rate limiting algorithm;
[0107] The first rate limiting strategy implementation module 330 is used to adjust the current traffic control strategy of the target service based on the first rate limiting strategy to obtain a target traffic control strategy, and run the adjusted target traffic control strategy to limit the rate of the target service.
[0108] This application provides a service rate limiting device that acquires first traffic data of a target service; determines a rate limiting algorithm and its parameters based on the first traffic data; and constructs a first rate limiting strategy based on the algorithm and parameters. The device then adjusts the target service's current traffic control strategy according to the first rate limiting strategy to obtain a target traffic control strategy, and applies the adjusted target traffic control strategy to limit the rate of the target service. This dynamic rate limiting strategy based on traffic data allows the service's rate limiting strategy to adapt to dynamically changing traffic patterns, effectively handling sudden traffic spikes or gradually increasing loads, and improving the effectiveness of service rate limiting.
[0109] Optionally, the rate limiting strategy determination module 320 includes:
[0110] The indicator calculation unit is used to calculate the key indicator data of the first traffic data on preset key indicators;
[0111] The rate limiting parameter determination unit is used to match a suitable rate limiting algorithm based on the key indicator data and determine the rate limiting parameters corresponding to the rate limiting algorithm.
[0112] Optionally, the current limiting parameter determination unit is specifically used for:
[0113] The performance status of the target service is determined based on the key indicator data, and the performance status includes at least one of: load status, traffic volatility, and resource utilization.
[0114] A suitable rate limiting algorithm is matched based on the performance status and the rate limiting target. The rate limiting algorithms include: fixed window counter algorithm, sliding window counter algorithm, token bucket algorithm and leaky bucket algorithm.
[0115] Optional, also includes:
[0116] The rate limiting algorithm switching module is used to switch the rate limiting algorithm of the target service to the rate limiting algorithm corresponding to the preset algorithm switching conditions after matching a suitable rate limiting algorithm based on the key indicator data and when the key indicator data meets the preset algorithm switching conditions.
[0117] Optional, also includes:
[0118] The configuration information acquisition module is used to acquire server configuration information before acquiring the first traffic data of the target service;
[0119] The second rate limiting strategy generation module is used to input the server configuration information into the empirical model to obtain the second rate limiting strategy; the empirical model is obtained by training a machine learning model based on a sample dataset consisting of historical rate limiting data and the server configuration information.
[0120] The second rate limiting strategy implementation module is used to adjust the current traffic control strategy of the target service based on the second rate limiting strategy, and run the adjusted traffic control strategy to limit the rate of the target service.
[0121] Optional, also includes:
[0122] The rate limiting feedback data acquisition module is used to obtain the rate limiting feedback data of the target service after adjusting the current traffic control policy of the target service based on the first rate limiting policy to obtain the target traffic control policy and running the adjusted target traffic control policy to limit the rate of the target service.
[0123] The first flow limiting parameter adjustment module is used to adjust the flow limiting parameters in the current flow control strategy according to the flow limiting feedback data when the flow limiting feedback data meets the preset parameter adjustment conditions.
[0124] Optional, also includes:
[0125] The second traffic data acquisition module is used to adjust the current traffic control policy of the target service based on the first traffic limiting policy to obtain the target traffic control policy, and after running the adjusted target traffic control policy to limit the traffic of the target service, acquire the second traffic data of the target service.
[0126] The traffic prediction module is used to input the second traffic data into the traffic prediction model to obtain predicted traffic data;
[0127] The second flow limiting parameter adjustment module is used to adjust the flow limiting parameters in the current flow control strategy based on the predicted flow data.
[0128] The service rate limiting device provided in this application embodiment can execute the service rate limiting method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the method execution.
[0129] Example 4
[0130] Figure 4 A schematic diagram of the structure of a server 10 that can be used to implement embodiments of this application is shown. The server is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The server can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.
[0131] like Figure 4 As shown, server 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer program stored in ROM 12 or loaded into RAM 13 from storage unit 18. RAM 13 can also store various programs and data required for the operation of server 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. Input / output (I / O) interface 15 is also connected to bus 14.
[0132] Multiple components in server 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows server 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0133] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as service rate limiting methods.
[0134] In some embodiments, the service rate limiting method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on server 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the service rate limiting method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the service rate limiting method by any other suitable means (e.g., by means of firmware).
[0135] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0136] In some embodiments, the service rate limiting method may be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements the service rate limiting method of this application. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0137] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0138] To provide interaction with the user, the systems and techniques described herein can be implemented on a server having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the server. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0139] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0140] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0141] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.
[0142] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A service throttling method, characterized by, The method comprises: obtaining first traffic data of a target service; determining a traffic limiting algorithm and a traffic limiting parameter of the traffic limiting algorithm according to the first traffic data, and constituting a first traffic limiting strategy according to the traffic limiting algorithm and the traffic limiting parameter; the traffic limiting algorithm is an algorithm for adjusting the traffic of the target service, and the traffic limiting parameter is a key parameter contained in the traffic limiting algorithm; adjusting a current traffic control strategy of the target service based on the first traffic limiting strategy to obtain a target traffic control strategy, and running the target traffic control strategy to implement traffic limiting on the target service.
2. The method of claim 1, wherein, The determination of the traffic limiting algorithm and the traffic limiting parameter of the traffic limiting algorithm according to the first traffic data comprises: calculating key indicator data of the first traffic data on a preset key indicator; wherein the preset key indicator comprises at least one of request number, request rate standard deviation, response time, error rate, request distribution, processor usage, memory usage, concurrent connection number, request queue length and network bandwidth; matching an appropriate traffic limiting algorithm according to the key indicator data, and determining the traffic limiting parameter corresponding to the traffic limiting algorithm.
3. The method of claim 2, wherein, The matching of the appropriate traffic limiting algorithm according to the key indicator data comprises: determining a performance state of the target service according to the key indicator data, the performance state comprising at least one of load state, traffic fluctuation and resource utilization; matching an appropriate traffic limiting algorithm according to the performance state and a traffic limiting target, the traffic limiting algorithm comprising a fixed window counter algorithm, a sliding window counter algorithm, a token bucket algorithm and a leaky bucket algorithm.
4. The method of claim 3, wherein, After the matching of the appropriate traffic limiting algorithm according to the key indicator data, the method further comprises: in a case where the key indicator data satisfies a preset algorithm switching condition, switching the traffic limiting algorithm of the target service to a traffic limiting algorithm corresponding to the preset algorithm switching condition.
5. The method according to any one of claims 1-4, characterized in that, Before obtaining the first traffic data of the target service, the method further comprises: obtaining server configuration information; inputting the server configuration information into an experience model to obtain a second traffic limiting strategy; the experience model is obtained by training a machine learning model according to a sample data set constituted by historical traffic limiting data and the server configuration information; adjusting the current traffic control strategy of the target service based on the second traffic limiting strategy to obtain the target traffic control strategy, and running the target traffic control strategy to implement traffic limiting on the target service.
6. The method according to any one of claims 1-4, characterized in that, After running the target traffic control strategy to implement traffic limiting on the target service, the method further comprises: obtaining traffic limiting feedback data of the target service; in a case where the traffic limiting feedback data satisfies a preset parameter adjustment condition, adjusting the traffic limiting parameter in the current traffic control strategy according to the traffic limiting feedback data.
7. The method according to any one of claims 1-4, characterized in that, After running the target traffic control strategy to implement traffic limiting on the target service, the method further comprises: obtaining second traffic data of the target service; inputting the second traffic data into a traffic prediction model to obtain predicted traffic data; adjusting the traffic limiting parameter in the current traffic control strategy according to the predicted traffic data.
8. A service throttling apparatus, characterized by, The device comprises: a first traffic data obtaining module configured to obtain first traffic data of a target service; The first traffic limiting strategy determination module is configured to determine a traffic limiting algorithm and traffic limiting parameters of the traffic limiting algorithm according to the first traffic data, and to constitute a first traffic limiting strategy according to the traffic limiting algorithm and the traffic limiting parameters; the traffic limiting algorithm is an algorithm for adjusting the target service traffic, and the traffic limiting parameters are key parameters contained in the traffic limiting algorithm; The first traffic limiting strategy implementation module is configured to adjust a current traffic control strategy of the target service based on the first traffic limiting strategy to obtain a target traffic control strategy, and to implement traffic limiting on the target service by running the target traffic control strategy.
9. A server, characterized by The server comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the service traffic limiting method in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the service traffic limiting method in any one of claims 1-7 when executed.