Flow limiter supporting priority batch flow requests

Abstract

The invention relates to the technical field of server software resource allocation, in particular to a flow limiter supporting priority batch flow requests, comprising: a flow splitting module used for splitting a batch flow request into a plurality of batch fragments; the token management module is used for recording and managing the current token quota accumulated in the token bucket; and the flow limiting judgment and execution module is used for performing flow limiting judgment on the batch flow requests according to the flow limiter parameters and the current token quota, and executing the action of releasing or splitting the batch flow requests according to the judgment result and the flow splitting module. Through the flow splitting module, the situation that the batch flow requests are frequently limited when the quota cannot reach the expected quota of the batch flow requests is avoided; when the expected quota of the batch flow request exceeds the maximum quota of the flow limiter, the batch flow request is always limited; according to the invention, the batch traffic is split into the sub-batch traffic with proper size, so that the quota resource of the traffic limiter can be fully utilized, and the passing rate of the batch traffic request is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of server-side software resource allocation, and in particular to a flow limiter supporting priority batch flow requests. Background Art

[0002] The server software provides services to the outside world through the API. Since the request volume of the upstream caller cannot be controlled, when the instantaneous request volume increases suddenly, the server will occupy too many resources, resulting in reduced response speed, timeout, or even downtime, and even an avalanche that makes the entire system unavailable.

[0003] The rate limiter controls the flow of API requests and quickly rejects, fails, or discards requests that exceed the limit to prevent the system from crashing due to overload, thereby ensuring the stability of this service and downstream resource systems.

[0004] The implementation of the rate limiter can be based on different algorithms and strategies. The basic algorithms in the industry include fixed window algorithm, sliding window algorithm, token bucket algorithm, leaky bucket algorithm, etc. However, in some application scenarios, there are still some shortcomings in terms of rate limiting effect, functional expectations, and business development complexity. For example, in order to improve the efficiency of interaction between services, the server provides a batch read and write interface. By replacing multiple call requests with a single batch call request, the overhead of multiple network calls is saved, and the performance of the server and database for batch processing is usually better. However, the existing algorithm has a low pass rate for batch traffic requests. When traffic quota resources are tight, batch traffic requests will be frequently limited, and the pass rate is low. In extreme cases, batch traffic requests will be rejected all the time, making it difficult to process business through the rate limiter.

[0005] Therefore, a flow limiter is needed that can improve the passing rate of batch traffic requests. Summary of the invention

[0006] (I) Purpose of the invention

[0007] The object of the present invention is to provide a flow limiter supporting priority batch flow requests and capable of improving the passing rate of batch flow requests.

[0008] (II) Technical solution

[0009] To solve the above problems, the present invention provides a flow limiter supporting priority batch flow requests, comprising:

[0010] Current limiter parameter configuration module, traffic splitting module, token management module and current limit judgment and execution module;

[0011] The current limiter parameter configuration module is used to set the parameters of the current limiter;

[0012] The traffic splitting module is used to split the batch traffic request into multiple batch slices;

[0013] The token management module is used to record and manage the current accumulated token quota in the token bucket;

[0014] The current limiting judgment and execution module is used to perform current limiting judgment on batch traffic requests based on the current limiter parameters and the current token quota, and to execute the action of releasing or splitting the batch traffic requests based on the judgment result. The action of splitting the batch traffic requests is implemented by the traffic splitting module.

[0015] In another aspect of the present invention, preferably,

[0016] The parameters of the current limiter include: maximum flow quota of the bucket, time length of the current limit cycle, number of gaps per cycle, maximum allowed burst flow, maximum compensation quantity per gap, average quota per gap and time length per gap;

[0017] The average quota per gap is obtained by dividing the maximum flow quota of the bucket by the number of gaps per cycle;

[0018] The time length of each gap is obtained by dividing the time length of the current limiting cycle by the number of gaps per cycle.

[0019] In another aspect of the present invention, preferably,

[0020] The flow limiting judgment on the batch flow request includes: making a flow limiting judgment according to a first rule;

[0021] The first rule includes:

[0022] When the batch traffic request is 1;

[0023] If the number of the batch flow requests is less than or equal to the average quota per gap of the flow limiter, the flow limiter directly releases the requests;

[0024] If the number of the batch traffic requests is greater than the average quota per gap of the flow limiter, the flow limiter splits the batch traffic requests into a plurality of batch slices according to the first formula through the traffic splitting module, and the flow limiter releases them in sequence;

[0025] The first formula is expressed as:

[0026]

[0027] Where S represents the number of batch shards in the first formula, qps_batch_req represents the number of batch traffic requests, and qps_per_interval represents the average quota per interval.

[0028] In another aspect of the present invention, preferably,

[0029] The first rule also includes:

[0030] When the batch traffic request is 1, burst instantaneous traffic is allowed;

[0031] If the number of the batch traffic requests is greater than the average quota per gap and is less than or equal to the maximum allowed burst traffic, the flow limiter directly releases the requests;

[0032] If the number of the batch traffic requests is greater than the maximum allowed burst traffic, the flow limiter will split the batch traffic requests into multiple batch slices according to the second formula through the traffic splitting module, and the flow limiter will release them in sequence;

[0033] The second formula is expressed as:

[0034]

[0035] Among them, U represents the number of batch shards in the second formula, qps_batch_req represents the number of batch traffic requests, and max_bust represents the maximum allowed burst traffic.

[0036] In another aspect of the present invention, preferably,

[0037] When the number of the batch traffic requests is greater than the maximum allowed burst traffic, the current token quota in the token bucket is used;

[0038] If the current token quota is less than the maximum allowed burst flow, the token quota in the subsequent time slot is used for compensation, and the maximum compensation amount is the maximum compensation amount per slot in the flow limiter parameters.

[0039] In another aspect of the present invention, preferably,

[0040] Performing flow limiting judgment on the batch flow request further includes: performing flow limiting judgment according to the second rule;

[0041] The second rule includes:

[0042] When the batch flow request is greater than 1, the batch flow request includes a first batch flow request and a second batch flow request;

[0043] If the second batch traffic request obtains the token quota of the gap, the first batch traffic request is split according to the remaining token quota of the gap to obtain the first batch fragments and the second batch fragments;

[0044] The first batch of shards is processed by the remaining token quota of the gap;

[0045] The second batch of shards is processed according to the first rule.

[0046] In another aspect of the present invention, preferably,

[0047] The second rule also includes:

[0048] When the batch traffic request is greater than 1 and burst instantaneous traffic is allowed;

[0049] When the second batch traffic request obtains the gap token quota;

[0050] If the value of the maximum allowed burst flow minus the number of second batch flow requests is greater than or equal to the number of first batch flow requests, the flow limiter directly releases the traffic;

[0051] If the value of the maximum allowed burst flow minus the number of second batch flow requests is less than the number of first batch flow requests, the first rule is used for processing.

[0052] In another aspect of the present invention, preferably,

[0053] It also includes a priority module;

[0054] The priority module prioritizes the batch traffic requests according to a preset first priority rule or a preset second priority rule.

[0055] In another aspect of the present invention, preferably,

[0056] The preset first priority rules include:

[0057] When high-priority batch traffic needs to be split, the token quota of each time slot is all preferentially allocated to the batch fragments after the high-priority batch traffic is split until all are satisfied.

[0058] In another aspect of the present invention, preferably,

[0059] The preset second priority rules include:

[0060] When high-priority batch traffic needs to be split, part of the traffic quota of each time slot is preferentially allocated to the batch fragments after the high-priority batch traffic is split, and the remaining quota is fairly competed by the batch fragments after the high-priority batch traffic that is not satisfied and other batch traffic.

[0061] (III) Beneficial effects

[0062] The above technical solution of the present invention has the following beneficial technical effects:

[0063] The present invention splits a batch traffic request into multiple batch slices through a traffic splitting module, so as to avoid that the batch traffic request is frequently limited when the quota does not reach the expected quota of the batch traffic request; and avoid that the batch traffic request is always limited when the expected quota of the batch traffic request exceeds the maximum limit of the limiter; and splits the batch traffic into sub-batch traffic of appropriate sizes, so as to make more full use of the quota resources of the limiter, avoid the situation where the quota resources are wasted when they cannot meet the overall batch traffic, and improve the pass rate of the batch traffic request. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the overall structure of an embodiment of the present invention;

[0065] Figure 2 It is a schematic diagram of a batch flow request of 1 entering a flow limiter according to an embodiment of the present invention;

[0066] Figure 3 It is a schematic diagram of a batch flow request of one embodiment of the present invention after being split into 1 and passing through a flow limiter;

[0067] Figure 4 It is a schematic diagram of a batch flow request of an embodiment of the present invention being 1, allowing burst instantaneous flow to pass through the flow limiter;

[0068] Figure 5 is a schematic diagram of passing through a flow limiter when a batch flow request is greater than 1 according to an embodiment of the present invention;

[0069] Figure 6 is a schematic diagram of a batch flow request greater than 1 and a burst instantaneous flow passing through a flow limiter according to an embodiment of the present invention;

[0070] Figure 7 It is a preset second priority rule of one embodiment of the present invention through the current limiter;

[0071] Figure 8 This is a schematic diagram of an embodiment of the present invention in which a batch flow request is greater than 1 and a burst instantaneous flow degrades and passes through a flow limiter. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0073] Obviously, the described embodiments are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0075] The present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are represented by similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale.

[0076] Embodiment 1

[0077] A rate limiter that supports priority batch traffic requests, Figure 1 FIG. 1 shows a schematic diagram of the overall structure of an embodiment of the present invention. Figure 1 As shown, including:

[0078] Current limiter parameter configuration module, traffic splitting module, token management module and current limit judgment and execution module;

[0079] The limiter parameter configuration module is used to set the parameters of the limiter; in this embodiment, the parameters of the limiter include: maximum flow quota of the bucket, length of the flow limiting cycle, number of gaps per cycle, maximum allowed burst flow, maximum compensation quantity per gap, average quota amount per gap and length of time per gap; the average quota amount per gap is obtained by dividing the maximum flow quota of the bucket by the number of gaps per cycle; the length of time per gap is obtained by dividing the length of the flow limiting cycle by the number of gaps per cycle.

[0080] The parameters of the current limiter are represented by the following codes:

[0081] bucket_capacity: the maximum flow quota of the bucket;

[0082] bucket_unit_time: the length of the current limiting cycle;

[0083] interval_unit_count: number of intervals per cycle;

[0084] max_bust: maximum burst traffic allowed, satisfying qps_per_interval <= max_bust;

[0085] interval_compensate_max_count: maximum number of compensations per interval;

[0086] interval_compensate_max_count <= qps_per_interval;

[0087] qps_per_interval: average quota per interval;

[0088] qps_per_interval=bucket_capacity / interval_unit_count;

[0089] interval_unit_time: the length of each interval;

[0090] interval_unit_time=bucket_unit_time / interval_unit_count;

[0091] Use the following code to define the content of the batch traffic request:

[0092] qps_batch_req: number of batch traffic requests;

[0093] qps_batch_req_0...b: the split batch traffic fragments, numbered as integers from 0 to b;

[0094] split_fun: split function, which can split qps_batch_req_x traffic according to the expected traffic number;

[0095] merge_fun: merge function, which can merge the b+1 return values ​​of qps_batch_req_0...b into one return value;

[0096] token_num: To cope with instantaneous burst traffic, a token bucket is used to accumulate unused quotas. token_num indicates the current token quota.

[0097] The traffic splitting module is used to split the batch traffic request into multiple batch slices;

[0098] The token management module is used to record and manage the current accumulated token quota in the token bucket;

[0099] The current limiting judgment and execution module is used to perform current limiting judgment on batch traffic requests based on the current limiter parameters and the current token quota, and to execute the action of releasing or splitting the batch traffic requests based on the judgment result. The action of splitting the batch traffic requests is implemented by the traffic splitting module.

[0100] Further, in this embodiment, performing flow limiting judgment on the batch flow request includes: performing flow limiting judgment according to the first rule; the batch flow request may be a single one or multiple ones;

[0101] The first rule includes:

[0102] When the batch traffic request is 1; when the batch traffic request is a single one in a certain time slot;

[0103] If the number of the batch flow requests is less than or equal to the average quota per gap of the flow limiter, the flow limiter directly releases the requests; Figure 2 FIG. 4 shows a schematic diagram of a batch flow request entering a flow limiter according to an embodiment of the present invention. Figure 2 As shown, the horizontal axis is the time axis, representing each time slot; the vertical axis is the flow quantity, and the dotted line represents the average quota per gap, such as Figure 2 As shown, the average quota per gap is 5. If the number of batch flow requests is less than or equal to 5, the flow limiter directly releases them.

[0104] If the number of the batch traffic requests is greater than the average quota per gap of the flow limiter, the flow limiter splits the batch traffic requests into a plurality of batch slices according to the first formula through the traffic splitting module, and the flow limiter releases them in sequence;

[0105] The first formula is expressed as:

[0106]

[0107] Where S represents the number of batch shards in the first formula, qps_batch_req represents the number of batch traffic requests, and qps_per_interval represents the average quota per interval.

[0108] Figure 3 FIG. 1 shows a schematic diagram of a batch flow request passing through a flow limiter after being split according to an embodiment of the present invention. Figure 3 As shown, the number of batch traffic requests is 12, and time slot 1 provides 5 quotas. The split function split_fun is used to split qps_batch_req_0 with 5 batch traffic, and the remaining batch traffic is 12-5=7 (pieces). Time slot 2 can provide 5 quotas, and qps_batch_req_1 with 5 batch traffic is split, and the remaining batch traffic is 7-5=2 (pieces). Time slot 3 can provide 5 quotas, and the remaining 2 directly constitute qps_batch_req_2. The overall batch traffic requests pass through the limiter in sequence.

[0109] Furthermore, in this embodiment, the first rule also includes:

[0110] When the batch traffic request is 1 and burst instantaneous traffic is allowed; the condition for allowing burst instantaneous traffic is that there is no token quota to be compensated;

[0111] If the quantity of the batch traffic request is greater than the average quota per interval and less than or equal to the maximum allowed burst traffic, the rate limiter directly allows it to pass;

[0112] If the quantity of the batch traffic request is greater than the maximum allowed burst traffic, the rate limiter splits the batch traffic request into multiple batch shards according to the second formula through the traffic splitting module, and the rate limiter allows it to pass in sequence;

[0113] The second formula is expressed as:

[0114]

[0115] Where, U represents the number of batch shards of the second formula, qps_batch_req represents the quantity of the batch traffic request, and max_bust represents the maximum allowed burst traffic.

[0116] Furthermore, in this embodiment,

[0117] When the quantity of the batch traffic request is greater than the maximum allowed burst traffic, the current token quota in the token bucket is used;

[0118] If the number of tokens in the bucket token_num >= max_bust, then qps_batch_req_0 with a batch traffic size of max_bust is split out, and the rate limiter allows it to pass. Because the cumulative quota is used, no compensation is required in subsequent time slots.

[0119] If the current token quota is less than the maximum allowed burst traffic, the token quota in subsequent time slots is used for compensation, and the maximum compensation quantity is the maximum compensation quantity per interval in the rate limiter parameters. When the number of tokens in the bucket token_num < max_bust, the additional rate limiting quota used in this time slot, max_bust - token_num - qps_per_interval, is compensated evenly in the subsequent time slot quotas, with a maximum of interval_compensate_max_count quotas compensated in each time slot. For the traffic quota used instantaneously in bursts, the current token quota is used first. If the current token quota is not enough, for the exceeded quota, the pre-allocated quota in subsequent time slots is used. Before the additional used traffic quota is completely compensated, no new burst traffic exceeding the normal time slot quota is allowed to pass. Figure 4 Shows a schematic diagram of a batch traffic request of 1 in an embodiment of the present invention, allowing burst instantaneous traffic to pass through the rate limiter, as Figure 4As shown in the figure, time slot 1 allows 10 burst instantaneous flows to pass through, and the batch flow request is 10, which can be passed directly without splitting, but 10-0-5=5 quotas are used extra. Each subsequent time slot can compensate at most 1 quota, so 5 quotas need to be compensated in the next 5 consecutive time slots, and the quota number of these 5 consecutive time slots changes from 5 to 4. After the compensation is completed, the quota number of the time slot is restored from 4 to 5.

[0120] Further, in this embodiment, performing flow limiting judgment on batch flow requests further includes: performing flow limiting judgment according to a second rule;

[0121] The second rule includes:

[0122] When the batch traffic request is greater than 1, the batch traffic request includes a first batch traffic request and a second batch traffic request; when the batch traffic request is not a single one in a certain time slot;

[0123] If the second batch traffic request obtains the token quota of the gap, the first batch traffic request is split according to the remaining token quota of the gap to obtain the first batch fragments and the second batch fragments;

[0124] The first batch of shards is processed by the remaining token quota of the gap;

[0125] The second batch of shards is processed according to the first rule.

[0126] Figure 5 FIG. 4 shows a schematic diagram of a flow limiter when a batch flow request is greater than 1 according to an embodiment of the present invention. Figure 5 As shown, the second batch traffic request occupies 2 quotas of time slot 1, so the first batch traffic request of 12 can be split into 3 flows of qps_batch_req_0, 5 flows of qps_batch_req_1, and 4 flows of qps_batch_req_3, respectively using time slot 1, time slot 2, and time slot 3. The overall batch traffic passes through the limiter.

[0127] Further, in this embodiment,

[0128] The second rule also includes:

[0129] When the batch traffic request is greater than 1 and burst instantaneous traffic is allowed;

[0130] When the second batch traffic request obtains the gap token quota;

[0131] If the value of the maximum allowed burst flow minus the number of second batch flow requests is greater than or equal to the number of first batch flow requests, the flow limiter directly releases the traffic;

[0132] If the value of the maximum allowed burst flow minus the number of second batch flow requests is less than the number of first batch flow requests, the first rule is used for processing.

[0133] Figure 6 FIG. 4 is a schematic diagram showing a case where a batch flow request is greater than 1 and a burst instantaneous flow passes through a flow limiter according to an embodiment of the present invention. Figure 6 As shown in the figure, after the second batch of traffic in time slot 1 occupies 2 quotas, the burst traffic quota is max_bust-2=10-2=8 (quotas). When the first batch of traffic is 8, the limiter uses an additional 8+2-5=5 (quotas). Each subsequent time slot can compensate for a maximum of 1 quota, so a total of 5 quotas need to be compensated for the next 5 consecutive time slots, and the number of quotas for these 5 consecutive time slots changes from 5 to 4. After the compensation is completed, the number of time slot quotas is restored from 4 to 5.

[0134] Furthermore, the present embodiment also includes a priority module;

[0135] The priority module prioritizes the batch traffic requests according to a preset first priority rule or a preset second priority rule.

[0136] The preset first priority rules include:

[0137] When high-priority batch traffic needs to be split, the token quota of each time slot is all preferentially allocated to the batch fragments after the high-priority batch traffic is split until all are satisfied.

[0138] High priority takes absolute priority. When high-priority batch traffic does not need to be split, it will seize the limited traffic quota normally. When splitting is required, if the first high-priority batch traffic obtains the traffic quota, the traffic quota of each time slot will be allocated to the high-priority split traffic first. After the high-priority split traffic is satisfied, the remaining traffic quota will be allocated to other traffic. In this process, if a higher-priority batch traffic enters, the traffic quota will be supplied to the higher-priority county-level batch traffic first. The shortcoming of this strategy is that after the high-priority traffic seizes the traffic quota, it will lengthen the overall call return time of the low-priority batch split traffic; in extreme scenarios, if there is always high-priority traffic entering, the low-priority traffic will always be in a hungry state. In practice, for batch traffic that has not obtained the traffic quota within a certain period of time, an error message can be returned to the caller to avoid the call timeout caused by the call not returning.

[0139] The preset second priority rule includes:

[0140] When high-priority batch traffic needs to be split, part of the traffic quota of each time slot is preferentially allocated to the batch fragments after the high-priority batch traffic is split, and the remaining quota is fairly competed by the batch fragments after the high-priority batch traffic that is not satisfied and other batch traffic.

[0141] High priority is relatively preferred. When high-priority batch traffic does not need to be split, it will normally seize the flow limit quota. When splitting is required, if the first high-priority batch traffic obtains the traffic quota, the traffic quota of each time slot will be allocated to the high-priority split traffic first. In each time slot, the remaining quota will be fairly competed by the unsatisfied high-priority batch traffic and other batch traffic. The disadvantage of this strategy is that under the competition of other batch traffic, the overall call return time of the high-priority batch traffic will be prolonged. The advantage is that it can prevent the low-priority batch traffic from being in a hungry state.

[0142] When other traffic competes for quota, if there is no priority, all traffic competes for quota fairly; if there is priority, high-priority traffic has priority in slot 1, and quota competition in subsequent time slots is determined according to the priority scheduling strategy. Figure 7 FIG. 4 shows a preset second priority rule of an embodiment of the present invention through a current limiter, such as Figure 7 As shown in the figure, when high priority is used, other traffic may compete for traffic quota. Other traffic occupies 2 quotas of time slot 2, so the batch traffic of 12 can be split into 5 traffics of qps_batch_req_0, 3 traffics of qps_batch_req_1, and 4 traffics of qps_batch_req_3, using time slot 1, time slot 2, and time slot 3 respectively. The overall batch traffic passes through the limiter.

[0143] Figure 8 FIG. 4 shows a schematic diagram of a batch flow request greater than 1 and a burst instantaneous flow degradation after passing through a flow limiter according to an embodiment of the present invention, such as Figure 8 As shown in the figure, after other traffic in time slot 1 occupies 2 quotas, the burst traffic quota is max_bust-2=10-2=8 (numbers). When the batch traffic is 10, it cannot pass the limiter directly. Therefore, it degenerates to the normal split mode to pass the limiter. It is split into 3 flows of qps_batch_req_0, 5 flows of qps_batch_req_1, and 2 flows of qps_batch_req_2, and passes through the limiter.

[0144] This embodiment splits the batch traffic request into multiple batch slices through the traffic splitting module, so as to avoid the batch traffic request being frequently limited when the quota does not reach the expected quota of the batch traffic request; to avoid the batch traffic request being permanently limited when the expected quota of the batch traffic request exceeds the maximum limit of the limiter; and to split the batch traffic into sub-batch traffic of appropriate sizes, so as to make more full use of the quota resources of the limiter, avoid the situation where the quota resources are wasted when they cannot meet the overall batch traffic, and improve the pass rate of the batch traffic request.

[0145] The batch traffic request passes through the flow limiter in a controlled and orderly manner. In the prior art, if the traffic exceeds the time slot quota, it will be rejected; if the batch traffic request exceeds the periodic quota, the batch traffic request will never pass. This embodiment can split the batch traffic into batch slices suitable for the current time slot passing size and pass through the flow limiter.

[0146] At the level of the flow limiter, a flow splitting module is introduced to adjust the batch flow in a more fine-grained manner. The batch flow is split into multiple batch flows using the split function as needed, and the return values ​​of each batch are merged by the merge function and returned to the caller in a unified manner.

[0147] With a priority mechanism, high-priority traffic has a greater probability of competing for the limiter quota, and a flexible and customizable priority scheduling strategy is used to meet the needs of actual business scenarios for priority traffic competition quotas. At present, when supporting priorities in the industry, one idea is to construct multiple limiters, and different limiters have different priorities; another idea is to pre-configure the priority and predetermined quota of the service traffic source in a limiter, and compare the priority and allocate the specified quota by identifying the traffic source. The present invention does not require pre-allocation, but only uses the priority attribute of the traffic itself, and uses a customizable priority scheduling strategy to allocate the quota allocation of each time slot, which can make full use of resources. Avoid the situation where there is still a quota in the time slot, but the traffic quota cannot be used because the upper limit of the predetermined traffic quota of the business is reached, resulting in the inability to fully utilize resources. The native priority mechanism makes it easier for high-priority traffic to pass through the limiter, improves the interface response speed, and meets the needs of business scenarios.

[0148] It supports instantaneous burst traffic scenarios and introduces a quota compensation mechanism. It has both the characteristics of token buckets to cope with burst traffic and the traffic uniformity characteristics of leaky bucket algorithms. For instantaneous burst traffic, a quota compensation mechanism is introduced to prevent burst traffic from instantly exhausting the quota within the cycle, and no traffic passes through the subsequent time slots, thereby reducing the instantaneous pressure on the system. The quota compensation mechanism is introduced to reduce the number of time slot traffic quotas in the subsequent certain period of time, providing the system with time to relieve pressure.

[0149] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

[0150] The present invention has been described above with reference to the embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, a person skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

[0151] Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.

[0152] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A flow limiter supporting priority batch flow requests, characterized in that: include: Current limiter parameter configuration module, traffic splitting module, token management module and current limit judgment and execution module; The current limiter parameter configuration module is used to set the parameters of the current limiter; The traffic splitting module is used to split the batch traffic request into multiple batch slices; The token management module is used to record and manage the current token quota accumulated in the token bucket; The current limiting judgment and execution module is used to perform current limiting judgment on batch traffic requests based on the current limiter parameters and the current token quota, and to execute the action of releasing or splitting the batch traffic requests based on the judgment result. The action of splitting the batch traffic requests is implemented by the traffic splitting module.

2. The current limiter according to claim 1, characterized in that The parameters of the current limiter include: maximum flow quota of the bucket, time length of the current limit cycle, number of gaps per cycle, maximum allowed burst flow, maximum compensation quantity per gap, average quota per gap and time length per gap; The average quota per gap is obtained by dividing the maximum flow quota of the bucket by the number of gaps per cycle; The time length of each gap is obtained by dividing the time length of the current limiting cycle by the number of gaps per cycle.

3. The current limiter according to claim 2, characterized in that: The flow limiting judgment on the batch flow request includes: making a flow limiting judgment according to a first rule; The first rule includes: When the batch traffic request is 1; If the number of the batch flow requests is less than or equal to the average quota per gap of the flow limiter, the flow limiter directly releases the requests; If the number of the batch traffic requests is greater than the average quota per gap of the flow limiter, the flow limiter splits the batch traffic requests into a plurality of batch slices through the traffic splitting module according to the first formula, and the flow limiter releases them in sequence; The first formula is expressed as: Where S represents the number of batch shards in the first formula, qps_batch_req represents the number of batch traffic requests, and qps_per_interval represents the average quota per interval.

4. The current limiter according to claim 3, characterized in that: The first rule also includes: When the batch traffic request is 1, burst instantaneous traffic is allowed; If the number of the batch traffic requests is greater than the average quota per gap and is less than or equal to the maximum allowed burst traffic, the flow limiter directly releases the requests; If the number of the batch traffic requests is greater than the maximum allowed burst traffic, the flow limiter will split the batch traffic requests into multiple batch slices according to the second formula through the traffic splitting module, and the flow limiter will release them in sequence; The second formula is expressed as: Among them, U represents the number of batch shards in the second formula, qps_batch_req represents the number of batch traffic requests, and max_bust represents the maximum allowed burst traffic.

5. The current limiter according to claim 4, characterized in that When the number of the batch traffic requests is greater than the maximum allowed burst traffic, the current token quota in the token bucket is used; If the current token quota is less than the maximum allowed burst flow, the token quota in the subsequent time slot is used for compensation, and the maximum compensation amount is the maximum compensation amount per slot in the flow limiter parameters.

6. The current limiter according to claim 5, characterized in that Performing flow limiting judgment on the batch flow request further includes: performing flow limiting judgment according to the second rule; The second rule includes: When the batch flow request is greater than 1, the batch flow request includes a first batch flow request and a second batch flow request; If the second batch traffic request obtains the token quota of the gap, the first batch traffic request is split according to the remaining token quota of the gap to obtain the first batch fragments and the second batch fragments; The first batch of shards is processed by the remaining token quota of the gap; The second batch of shards is processed according to the first rule.

7. The current limiter according to claim 6, characterized in that The second rule also includes: When the batch traffic request is greater than 1 and burst instantaneous traffic is allowed; When the second batch traffic request obtains the gap token quota; If the value of the maximum allowed burst flow minus the number of second batch flow requests is greater than or equal to the number of first batch flow requests, the flow limiter directly releases the traffic; If the value of the maximum allowed burst flow minus the number of second batch flow requests is less than the number of first batch flow requests, the first rule is used for processing.

8. The current limiter according to claim 1, characterized in that It also includes a priority module; the priority module prioritizes the batch traffic requests according to a preset first priority rule or a preset second priority rule.

9. The current limiter according to claim 8, characterized in that The preset first priority rules include: When high-priority batch traffic needs to be split, the token quota of each time slot is all preferentially allocated to the batch fragments after the high-priority batch traffic is split until all are satisfied.

10. The current limiter according to claim 8, characterized in that The preset second priority rule includes: When high-priority batch traffic needs to be split, part of the traffic quota of each time slot is preferentially allocated to the batch fragments after the high-priority batch traffic is split, and the remaining quota is fairly competed by the batch fragments after the high-priority batch traffic that is not satisfied and other batch traffic.

Images