Api throttling method, server device, storage medium and program product

Abstract

The application provides an API flow limiting method, a server device, a storage medium and a program product, the method comprising: a feedforward control step, using a Holt-Winters model to predict future API request quantity, and obtaining a feedforward adjustment amount according to the size relationship between the future API request quantity and the rated processing API request quantity of the server; a feedback control step, performing PID control on the error between the expected average response time of the server and the actually measured average response time of the server, and obtaining a feedback adjustment amount; a control synthesis step, obtaining a synthesized adjustment amount and a candidate token generation rate; an amplitude limiting step, generating a final token generation rate based on the upper and lower limits of the token generation rate and the candidate token generation rate. By using the above technical scheme, adaptive API flow limiting is realized, the calling rate can be dynamically adjusted in the flow limiting process, the complex and changeable business scenarios can be effectively adapted, the server resource utilization efficiency is improved, and the high availability and stability in the API calling process are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of API service rate limiting, and in particular to an API rate limiting method, server-side device, storage medium, and program product. Background Technology

[0002] Currently, with the continuous expansion of the user base of the API service on the platform, the platform system faces increasingly significant pressure regarding API availability and stability, necessitating the introduction of an effective rate limiting mechanism to cope with sudden surges in user call traffic. However, traditional token bucket rate limiting schemes rely on pre-set rate and bucket capacity parameters, lacking dynamic adaptability and struggling to flexibly respond to changing business loads and sudden traffic scenarios. Their design focuses on limiting rather than adaptive adjustment, easily leading to uneven utilization of server resources. Furthermore, their ability to identify and adapt to complex business call environments is limited, impacting the overall performance and reliability of the system. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of the present invention provide an API rate limiting method, server-side device, storage medium, and program product, which achieve adaptive API rate limiting through a dual-loop control combining feedforward and feedback.

[0004] To achieve the above objectives, on the one hand, an API rate limiting method is provided, which limits API rate by periodically controlling the token generation rate, including:

[0005] The feedforward control step involves obtaining a time series containing historical API request volume and current period API request volume, using the Holt-Winters prediction model to predict future period API request volume, and further obtaining the feedforward adjustment amount of the token generation rate based on the relationship between the predicted future period API request volume and the server's rated processing API request volume.

[0006] The feedback control step involves using PID control to adjust the token generation rate by controlling the error between the expected average server response time and the actual measured average server response time.

[0007] The control synthesis step combines the feedforward adjustment amount and the feedback adjustment amount to obtain the synthesized adjustment amount, and obtains the candidate token generation rate for future periods based on the synthesized adjustment amount and the current token generation rate of the token bucket.

[0008] The limiting step generates the final token generation rate for future periods based on a predetermined lower limit for the token generation rate, a predetermined upper limit for the token generation rate, and the candidate token generation rates.

[0009] Preferably, the API rate limiting method, which uses the Holt-Winters prediction model to predict the API request volume for future periods, includes:

[0010] Obtain the estimated level L of the time series at the current period t. t ,in:

[0011] L t =α*Y t +(1-α)*(L t-1 +T t-1 )

[0012] Obtain the estimated trend T of the time series in the current period t. t ,in:

[0013] T t =β*(L t -L t-1 )+(1-β)*T t-1

[0014] Based on the estimated level L t and the estimated trend T t Predict the API request volume F for the next t+m period t+m , where m is an integer greater than or equal to 1, and:

[0015] F t+m =L t +m*T t

[0016] Among them, Y t L represents the actual number of API requests, i.e., the actual observations, for the time series over period t. t-1 The estimated level of the time series in period t-1; T t-1 α represents the estimated trend of the time series in period t-1; α is a smoothing parameter with a value range of [0,1], used to control the influence weight of the current observation on the level estimate; β is a trend smoothing parameter with a value range of [0,1], used to control the update speed of the trend estimate.

[0017] Preferably, the API rate limiting method,

[0018] When t=1:

[0019] L t =α*Y t

[0020] T t =β*L t

[0021] When m = 1:

[0022] F t+1 =L t +T t

[0023] Among them, F t+1 This represents the predicted number of API requests in the next period t+1 after the current period t.

[0024] Preferably, the API rate limiting method obtains the feedforward adjustment amount Δr of the token generation rate for the next period based on the predicted relationship between the API request volume in the next period after the current period t and the server's rated API request volume. ff ,in:

[0025] Δr ff =f(F t+1 )=k ff *(F t+1 -C nom ) / C nom *r current

[0026] Among them, C nom The server's rated capacity for processing API requests; k ff is the feedforward gain coefficient, with a value range of (0,1], used to control the strength of the feedforward effect.

[0027] Preferably, the API rate limiting method uses the following formula to perform PID control on the error between the expected average server response time and the actual measured average server response time:

[0028]

[0029] Where e(t) is the error of the current period t, e(t) = SP - PV(t); Δr fb (t) represents the feedback adjustment amount output in the current period t; SP represents the expected average server response time; and PV(t) represents the actual average server response time measured in the current period t. The cumulative sum of the errors represents the sum of all errors from the beginning to the current period t; e(t-1) is the error of the previous period t-1 before the current period t; T is the control period of the PID control; K p For proportional gain; K i K is the integral gain; d This is the differential gain.

[0030] Preferably, the API rate limiting method adds the feedforward adjustment amount and the feedback adjustment amount to obtain the synthesized adjustment amount; and adds the synthesized adjustment amount and the current token generation rate to obtain the candidate token generation rate.

[0031] Preferably, the API rate limiting method generates the final token generation rate for future periods using the following formula:

[0032] The final token generation rate = max(predetermined lower limit of token generation rate, min(candidate token generation rate, predetermined upper limit of token generation rate)).

[0033] On the other hand, a server device is provided, including a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the steps of the API rate limiting method as described in any of the above.

[0034] In another aspect, a computer-readable storage medium is provided, wherein at least one program is stored therein, the at least one program being executed by a processor to implement the steps of the API rate limiting method as described in any of the above descriptions.

[0035] In another aspect, a computer program product is provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the API rate limiting method as described above.

[0036] The above technical solution has the following technical effects:

[0037] The technical solution of this invention realizes a composite adaptive API rate limiting based on the combination of feedforward prediction and feedback control. By integrating feedforward prediction and feedback control mechanisms, it can dynamically adjust the call rate during the rate limiting process, effectively adapt to complex and ever-changing business scenarios, improve server resource utilization efficiency, and thus ensure high availability and stability during API calls. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating an API rate limiting method according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of a server device according to an embodiment of the present invention. Detailed Implementation

[0040] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0041] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0042] The API rate limiting method of this invention limits API rate by periodically controlling the token generation rate. Its core lies in designing a dual-loop control system, including: a feedforward control loop, based on time series prediction, proactively responding to foreseeable changes in traffic, i.e., changes in API request volume; and a feedback control loop, based on PID control theory, suppressing unforeseen interference and deviations in real time. The outputs of the two loops are superimposed in a control synthesizer to dynamically adjust the token generation rate r, thereby achieving high-precision and high-stability intelligent API rate limiting. Here, API rate limiting refers to API service rate limiting.

[0043] Example 1:

[0044] The API rate limiting method of this embodiment of the invention performs API rate limiting by periodically controlling the token generation rate, including:

[0045] The feedforward control step involves obtaining a time series containing historical API request volume and current period API request volume, using the Holt-Winters prediction model to predict future period API request volume, and obtaining the feedforward adjustment amount of the token generation rate based on the relationship between the predicted future period API request volume and the server's rated processing API request volume.

[0046] The feedback control step involves using PID control to adjust the token generation rate by controlling the error between the expected average server response time and the actual measured average server response time.

[0047] The control synthesis step combines the feedforward adjustment amount and the feedback adjustment amount to obtain the synthesized adjustment amount, and obtains the candidate token generation rate for future periods based on the synthesized adjustment amount and the current token generation rate of the token bucket.

[0048] The limiting step generates the final token generation rate for future periods based on a predetermined lower limit for the token generation rate, a predetermined upper limit for the token generation rate, and the candidate token generation rates.

[0049] The token bucket executor generates tokens based on the final token generation rate generated using the API rate limiting method of this invention. This allows the system to adjust the token generation rate according to the predicted future API traffic volume. This can proactively increase the token generation rate in advance when it is predicted that future traffic will exceed the system's rated processing capacity, avoiding the incorrect rejection of a large number of requests due to a sudden lack of tokens or causing system overload. Conversely, it can appropriately reduce the token generation rate when it is predicted that future traffic will be far below the system's rated processing capacity, saving unnecessary resource allocation during system idle periods and improving the overall resource utilization of the system.

[0050] Example 2:

[0051] Figure 1 This is a flowchart illustrating the API rate limiting method of an embodiment of the present invention. The API rate limiting method of this embodiment is implemented through a feedforward control path that anticipates future API request volumes, a feedback notification path that enhances system stability, a control synthesizer that combines the feedforward and feedback control quantities, and a rate limiting protector. The specific implementation of this embodiment of the present invention is described below.

[0052] I. Feedforward Control Path

[0053] 1. Prediction Model: Holt-Winters Quadratic Exponential Smoothing Model

[0054] It contains three formulas:

[0055] Level component:

[0056] Formula (1): L t =α*Y t +(1-α)*(L t-1 +T t-1 )

[0057] Trend component:

[0058] Formula (2): T t =β*(l t -L t-1 )+(1-β)*T t-1

[0059] Forecast equation:

[0060] Formula (3): F t+m =L t +m*T t

[0061] Parameter description:

[0062] Y t L represents the actual number of API requests, i.e., the actual observations, for the time series over period t. t-1 T represents the estimated level of the time series at period t-1, i.e., the smoothed value; t-1 The α parameter represents the estimated trend, i.e., the estimated increment, of the time series at period t-1; α is a smoothing parameter, ranging from [0,1], used to control the weight of the influence of the current observation on the level estimate. The larger the value, the more sensitive the model is to recent changes; β is a trend smoothing parameter, ranging from [0,1], used to control the update speed of the trend estimate.

[0063] Where, when t=1:

[0064] L t =α*Y t

[0065] T t =β*L t

[0066] When m = 1:

[0067] F t+1 =L t +T t

[0068] F t+1 This represents the predicted number of API requests in the next period t+1 after the current period t.

[0069] The Holt-Winters model is a common term used for estimating levels and trends, and its specific meaning and calculation method are known to those skilled in the art. The formula itself is not an improvement of this invention, and will not be elaborated here to avoid confusion.

[0070] 2. Feedforward Controller Formula

[0071] Based on the above prediction results, the feedforward control quantity, i.e., the feedforward adjustment quantity, is calculated. Specifically, based on the predicted API request volume in the next cycle after the current cycle t, the feedforward adjustment quantity Δr for the token generation rate in the next cycle is obtained. ff ,in:

[0072] Formula (4):

[0073] Δr ff =f(F t+1 )=k ff *(F t+1 -C nom ) / C nom *r current

[0074] Parameter description:

[0075] F t+1 The predicted number of API requests in the next period t+1 after the current period t, as predicted using the Holt-Winters model.

[0076] C nom This refers to the server's rated API request handling capacity, i.e., the system's rated processing capacity, a preset QPS value that the system can stably handle over a long period; in practice, it is usually slightly smaller than the limit value Capacity; k ff Δr is the feedforward gain coefficient, an adjustable parameter with a value range of (0,1], used to control the strength of the feedforward effect. ff =f(Ft+1 ) represents Δr ff For F t+1 The function.

[0077] Design Concept: This embodiment of the invention uses predicted flow F t+1 And further obtain the predicted flow change (F) t+1 -C nom This means that changes in API request volume are converted into adjustments to the token generation rate proportionally; the more the predicted traffic exceeds the rated capacity, the stronger the feedforward action becomes, thus enabling proactive response.

[0078] Feedforward adjustment Δr ff This is a control variable whose physical meaning is: a suggested adjustment value for the token generation rate *r* based on predictions of future traffic; its unit is the same as the token generation rate, typically "tokens / second". Specifically, it is a scalar value, and its sign and magnitude represent different control intentions, as explained below:

[0079] When Δr ff When Δr > 0, that is, when Δr ff When the value is positive, it means that the future traffic is predicted to exceed the system's rated processing capacity. The control intention at this time is to increase the token generation rate in advance. The purpose is to accumulate more tokens in the token bucket before the traffic surge arrives, thereby smoothing request processing and avoiding a large number of requests being incorrectly rejected or causing system overload due to a sudden lack of tokens.

[0080] When Δr ff When <0, that is, when Δr ff When the value is negative, it means that the future traffic is predicted to be much lower than the system's rated processing capacity. The control intention at this time is to moderately reduce the token generation rate. The purpose is to save unnecessary resource allocation during system idle periods, such as reducing the pre-allocated resources to downstream services, thereby improving the overall resource utilization while still maintaining a minimum service guarantee capability.

[0081] When Δr ff When the value is approximately 0, it means that the future flow is predicted to be within the normal range near the system's rated processing capacity. The control intention at this time is to not make any forward-looking adjustments. At this time, the system relies entirely on the feedback control loop described later to make fine-tuning adjustments to small and unpredictable flow fluctuations in order to maintain system stability.

[0082] In the specific implementation, when Δr ff When the absolute value is less than the predetermined zero threshold, it is determined to be a zero value.

[0083] II. Feedback Path

[0084] 1. Use a discrete PID controller

[0085] This embodiment employs a positional PID algorithm, whose output feedback adjustment Δr fb for:

[0086] Formula (5):

[0087]

[0088] Where e(t) is the error of the current period t, i.e., the current error, e(t) = SP - PV(t); Δr fb (t) represents the feedback adjustment amount output in the current period t; SP represents the expected average server response time, which is the target setpoint; PV(t) represents the actual average server response time measured in the current period t, which is a process variable; e(t) represents the error between the expected average server response time and the actual measured average server response time. The cumulative sum of errors represents the total sum of all errors from the beginning to the current period t; it is used to eliminate static errors; e(t-1) is the error of the previous period t-1 before the current period t; T is the control period or sampling period of the PID control, i.e., the time interval for performing one PID calculation, such as 1 second. This parameter T correlates the integral and derivative actions with real time; K p For proportional gain; K i K is the integral gain; d K represents the differential gain; e(t-1)-e(t) is the error rate of change, which is the difference between the error of the current period (the current sampling error) and the error of the previous period (the previous sampling error), representing the trend of the deviation. p The current error is amplified, which determines the system's response speed; K p An excessively large value of K may cause system oscillation; i The integral gain amplifies the accumulated error, which is used to eliminate steady-state errors in the system, such as persistent high latency caused by long-term high load; K d It amplifies the rate of change of error, which has an anticipatory adjustment effect, effectively suppresses overshoot, increases system stability, and makes the process smoother.

[0089] III. Controlled Synthesis and Limiting Protection

[0090] 1.Synthetic formula:

[0091] Formula (6): Δr total =Δr ff +Δr fb

[0092] r new =r current +Δrtotal

[0093] Δr total The adjustment amount after synthesis, Δr ff The feedforward adjustment amount, Δr fb For feedback adjustment amount; r current The current token generation rate; r new The rate at which candidate tokens are generated.

[0094] 2. Limiting protection formula:

[0095] Formula (7): r final =max(r min ,min(r new ,r max ))

[0096] Parameter description:

[0097] r final For the final token generation rate;

[0098] r min This is a lower limit for the token generation rate, used to prevent excessive idleness of system resources due to an excessively low rate.

[0099] r max This is the upper limit for the token generation rate, used to prevent the calculated rate from exceeding the system's absolute physical limits, thus ensuring system security.

[0100] IV. Process Summary

[0101] At intervals T, the server system initiates a control cycle and executes feedforward actions, feedback actions, and synthesis and protection actions; among which:

[0102] Feedforward action: Collect historical traffic data and substitute it into the above formulas (1)(2)(3) to calculate the predicted value F. t+1 Then, the feedforward adjustment amount Δr is calculated using formula (4). ff ;

[0103] Feedback action: Collect the current time response time of the system, i.e., the actual average response time PV(t) of the current period, calculate the error e(t), and substitute it into formula (5) to calculate the feedback adjustment amount Δr. fb .

[0104] Synthesis and Protection: Substitute the feedforward and feedback outputs into formula (6) for control synthesis to obtain the final adjustment command, and perform amplitude limiting protection via formula (7) to ensure safe r final Set to the new token generation rate;

[0105] Waiting for the next cycle to form a closed-loop control.

[0106] The technical solution of this invention combines prediction and feedback, achieving both foresight and stability, and realizing high-precision and high-stability adaptive API rate limiting.

[0107] Example 3:

[0108] The present invention also provides a server-side device, such as... Figure 2 As shown, the server device includes a processor 201, a memory 202, a bus 203, and a computer program stored in the memory 202 and executable on the processor 201. The processor 201 includes one or more processing cores. The memory 202 is connected to the processor 201 via the bus 203. The memory 202 is used to store program instructions. When the processor executes the computer program, it implements the steps in the above-described method embodiment of Embodiment 1 of the present invention.

[0109] Furthermore, as an executable solution, the server-side device can be a computer unit, which can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the computer unit is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.

[0110] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.

[0111] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0112] Example 4:

[0113] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0114] If the modules / units integrated in the computer unit are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0115] Example 5:

[0116] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps described above.

[0117] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. An API rate limiting method, characterized in that, API rate limiting can be implemented by periodically controlling the token generation rate, including: The feedforward control step involves obtaining a time series containing historical API request volume and current period API request volume, using the Holt-Winters prediction model to predict future period API request volume, and further obtaining the feedforward adjustment amount of the token generation rate based on the relationship between the predicted future period API request volume and the server's rated processing API request volume. The feedback control step involves using PID control to adjust the token generation rate by controlling the error between the expected average server response time and the actual measured average server response time. The control synthesis step combines the feedforward adjustment amount and the feedback adjustment amount to obtain the synthesized adjustment amount, and obtains the candidate token generation rate for future periods based on the synthesized adjustment amount and the current token generation rate of the token bucket. The limiting step generates the final token generation rate for future periods based on a predetermined lower limit for the token generation rate, a predetermined upper limit for the token generation rate, and the candidate token generation rates.

2. The API rate limiting method according to claim 1, characterized in that, The API request volume for predicting future cycles using the Holt-Winters forecasting model includes: Obtain the estimated level L of the time series at the current period t. t ,in: L t =α*Y t +(1-a)*(L t-1 +T t-1 ) Obtain the estimated trend T of the time series in the current period t. t ,in: T t =β*(L t -L t-1 )+(1-β)*T t-1 Based on the estimated level L t and the estimated trend T t Predict the API request volume F for the next t+m period t+m , where m is an integer greater than or equal to 1, and: F t+m =L t +m*T t Among them, Y t L represents the actual number of API requests, i.e., the actual observations, for the time series over period t. t-1 The estimated level of the time series in period t-1; T t-1 α represents the estimated trend of the time series in period t-1; α is a smoothing parameter with a value range of [0,1], used to control the influence weight of the current observation on the level estimate; β is a trend smoothing parameter with a value range of [0,1], used to control the update speed of the trend estimate.

3. The API rate limiting method according to claim 2, characterized in that, When t=1: L t =α*Y t T t =β*L t When m = 1: F t+1 =L t +T t Among them, F t+1 This represents the predicted number of API requests in the next period t+1 after the current period t.

4. The API rate limiting method according to claim 3, characterized in that, Based on the predicted API request volume for the next period after the current period t and the server's rated API request volume, the feedforward adjustment amount Δr for the token generation rate in the next period is obtained. ff ,in: Δr ff =f(F t+1 )=k ff *(F t+1 -C nom ) / C nom *r current Among them, C nom The server's rated capacity for processing API requests; k ff is the feedforward gain coefficient, with a value range of (0,1], used to control the strength of the feedforward effect.

5. The API rate limiting method according to claim 1, characterized in that, The following formula is used for PID control of the error between the expected average server response time and the actual measured average server response time: Where e(t) is the error of the current period t, e(t) = SP - PV(t); Δr fb (t) represents the feedback adjustment amount output in the current period t; SP represents the expected average server response time; and PV(t) represents the actual average server response time measured in the current period t. The cumulative sum of the errors represents the sum of all errors from the beginning to the current period t; e(t-1) is the error of the previous period t-1 before the current period t; T is the control period of the PID control; K p For proportional gain; K i K is the integral gain; d This is the differential gain.

6. The API rate limiting method according to claim 1, characterized in that, The feedforward adjustment and the feedback adjustment are added together to obtain the synthesized adjustment; the synthesized adjustment is added to the current token generation rate to obtain the candidate token generation rate.

7. The API rate limiting method according to claim 1, characterized in that, The final token generation rate for future cycles is generated using the following formula: The final token generation rate = max(predetermined lower limit of token generation rate, min(candidate token generation rate, predetermined upper limit of token generation rate)).

8. A server-side device, characterized in that, It includes a memory and a processor, the memory storing at least one program, which is executed by the processor to implement the steps of the API rate limiting method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one program segment, which is executed by a processor to implement the steps of the API rate limiting method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the API rate limiting method as described in any one of claims 1 to 7.

Images