Access methods, devices, electronic equipment, and media for rate-limited interfaces

Abstract

This invention provides a method, apparatus, electronic device, and medium for accessing a rate-limiting interface, comprising: when a target node of a client cluster requests access to a rate-limiting interface of a server, obtaining the current request volume per second of the client cluster and the current token count of the target node; if the request volume per second of the client cluster is less than the access volume threshold of the rate-limiting interface of the server, and the current token count of the target node is greater than zero, then allowing the target node to access the rate-limiting interface. This invention improves the stability of client cluster requests to access the rate-limiting interface of the server.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and medium for accessing a rate-limited interface. Background Technology

[0002] With the continuous development and expansion of network technology, many network platforms and internal platforms provide various services or functions through open interfaces to meet the needs of integration with different platforms. Internet service providers offer open interfaces for third-party developers to call when developing internet applications. To ensure server stability and prevent the server from being overwhelmed by a sudden surge of access traffic, the server typically implements rate limiting for some critical interfaces. However, existing methods mostly focus on server-side rate limiting. When many clients frequently access the server-side rate-limited interfaces for data processing via APIs, problems arise where the server returns error messages due to excessive interface access frequency, leading to errors in subsequent processing logic. This makes it impossible to ensure the stability of client cluster requests accessing rate-limited interfaces. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and medium for accessing a rate-limiting interface, so as to improve the stability of client cluster requests to access the server-side rate-limiting interface.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, embodiments of the present invention provide a method for accessing a rate-limiting interface, comprising: when a target node of a client cluster requests access to a rate-limiting interface of a server, obtaining the current request volume per second of the client cluster and the current token count of the target node; if the request volume per second of the client cluster is less than the access volume threshold of the rate-limiting interface of the server, and the current token count of the target node is greater than zero, then allowing the target node to access the rate-limiting interface.

[0006] In one implementation, obtaining the number of requests per second (RPS) of the client cluster and the current number of tokens of the target node at the current moment includes: obtaining the number of RPS of each node in the client cluster at the current moment, and determining the sum of the RPS of each node as the number of RPS of the client cluster; determining the access rate of the target node based on the number of RPS of the target node, the number of RPS of the client cluster, and the access threshold of the rate limiting interface of the server, and determining the current number of tokens of the target node based on the access rate.

[0007] In one implementation, obtaining the number of requests per second for each node in the client cluster at the current moment includes: determining the number of a calculation grid based on the current timestamp in milliseconds and a pre-set statistical time interval; wherein the calculation grid is used to store the number of requests from nodes to the rate-limiting interface within the statistical time interval, and the calculation grid is divided based on the statistical time interval; updating the request count value stored in the calculation grid corresponding to the number based on the request count value at the current moment, and clearing the request count value to zero; wherein the request count value is the number of requests from nodes to the rate-limiting interface within the statistical time interval; for each node, the sum of the request count values ​​corresponding to each calculation grid of the node is used to determine the number of requests per second for the node at the current moment.

[0008] In one implementation, the access rate of the target node is determined based on the number of requests per second of the target node, the number of requests per second of the client cluster, and the access threshold of the rate limiting interface on the server side, including: calculating the access rate of the target node according to the following formula:

[0009] Vi=P*(T-Li) / ((N-1)*T)

[0010] Where Vi is the access rate of the target node, T is the number of requests per second of the client cluster, Li is the number of requests per second of the target node, N is the number of nodes in the client cluster, and P is the access threshold of the rate limiting interface on the server.

[0011] In one implementation, determining the current token count of a target node based on the access rate includes: determining the number of newly added tokens within a statistical time interval based on the access rate of the target node; determining whether the original token count of the target node exceeds a token count threshold; if the original token count exceeds the token count threshold, determining whether the sum of the original token count and the newly added token count exceeds the token count threshold; if the sum of the original token count and the newly added token count exceeds the token count threshold, determining the token count threshold as the current token count of the target node; if the sum of the original token count and the newly added token count does not exceed the token count threshold, determining the sum of the original token count and the newly added token count as the current token count of the target node.

[0012] In one implementation, the method further includes: if the number of requests per second of the client cluster is greater than or equal to the access threshold of the rate limiting interface of the server, then stop updating the token count of each node in the client cluster until the number of requests per second of the client cluster is less than the access threshold of the rate limiting interface of the server.

[0013] In one implementation, the method further includes: if the number of requests per second of the client cluster is greater than or equal to the access threshold of the rate limiting interface of the server, or the current token count of the target node is zero, then after waiting for the minimum statistical time interval, the number of requests per second of the client cluster and the current token count of the target node are re-acquired for judgment.

[0014] Secondly, embodiments of the present invention provide an access device for a rate-limiting interface, comprising: a data acquisition module, configured to acquire the number of requests per second of the client cluster and the current number of tokens of the target node at the current moment when the target node of the client cluster requests access to the rate-limiting interface of the server; and a judgment module, configured to allow the target node to access the rate-limiting interface if the number of requests per second of the client cluster is less than the access threshold of the rate-limiting interface of the server and the current number of tokens of the target node is greater than zero.

[0015] Thirdly, embodiments of the present invention provide an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the steps of any of the methods provided in the first aspect above.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the method provided in any of the first aspects above.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] The access method, apparatus, electronic device, and medium for the rate-limiting interface provided in this embodiment of the invention can obtain the current request volume per second of the client cluster and the current token count of the target node when a target node of the client cluster requests access to the rate-limiting interface of the server. If the request volume per second of the client cluster is less than the access volume threshold of the rate-limiting interface of the server, and the current token count of the target node is greater than zero, then the target node is allowed to access the rate-limiting interface. This method statistically analyzes the request volume per second of the client cluster's requests to access the server's rate-limiting interface and determines whether the target node can access the rate-limiting interface based on the request volume per second of the client cluster and the current token count of the target node. This makes the access of the rate-limiting interface by the client cluster more balanced, improves the stability of the client cluster's requests to access the server's rate-limiting interface, and avoids the problem of the entire client cluster being rate-limited due to a sudden surge in requests from a single node.

[0019] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for accessing a rate-limiting interface provided in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a client cluster node access rate limiting interface provided in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of node initialization provided in an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of a judgment logic provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of node RPS value update provided in an embodiment of the present invention;

[0027] Figure 6 This is a state diagram of node 1 before it is updated at time 1665467139500, provided in an embodiment of the present invention.

[0028] Figure 7 The state diagram of node 1 updated at time 1665467139500 is provided for an embodiment of the present invention;

[0029] Figure 8 This is a state diagram of node 1 after being updated at timestamp 1665467139600, provided in an embodiment of the present invention.

[0030] Figure 9 This is a state diagram of node i after being updated at timestamp 1665467140500, provided in an embodiment of the present invention.

[0031] Figure 10 This is a schematic diagram illustrating the update of a tokenValue according to an embodiment of the present invention;

[0032] Figure 11 A schematic diagram of the structure of an access device for a rate-limiting interface provided in an embodiment of the present invention;

[0033] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Currently, to ensure server stability and prevent overwhelmed by sudden surges in traffic, servers typically implement rate limiting for critical interfaces. However, existing research largely focuses on server-side rate limiting methods, with limited research on client-side concurrent requests to rate-limited interfaces. Many clients still frequently access rate-limited interfaces via APIs for data processing, but the high access frequency often results in server-side error messages, leading to errors in subsequent processing logic. Furthermore, client-side methods for handling concurrent request rate limiting interfaces mostly involve simply setting a total access count to limit request frequency. However, due to accuracy issues in statistics, a mismatch between the counted accesses and server requests can occur, leading to single points of failure and thus rate limiting. Therefore, this approach fails to completely resolve the issue of rate limiting for clustered access to server interfaces.

[0036] Based on this, the present invention provides a method, apparatus, electronic device and medium for accessing a rate-limiting interface, which can improve the stability of client cluster requests accessing the server-side rate-limiting interface.

[0037] To facilitate understanding of this embodiment, a method for accessing a rate-limiting interface disclosed in this invention will first be described in detail. This method can be executed by an electronic device, such as a smartphone, computer, or tablet computer. See also... Figure 1 The flowchart shown illustrates a method for accessing a rate-limited interface, indicating that the method mainly includes the following steps S101 to S102:

[0038] Step S101: When the target node of the client cluster requests access to the rate-limiting interface of the server, obtain the current request volume per second of the client cluster and the current number of tokens of the target node.

[0039] In one implementation, the client cluster includes multiple nodes, and the target node can be the i-th node requesting access to the rate-limiting interface. When the server sets an access threshold for the client cluster, meaning that the number of requests per second from each node in the client cluster to the server's rate-limiting interface exceeds the access threshold, the server's rate-limiting interface becomes unavailable, i.e., the client cluster cannot access the rate-limiting interface. Based on this, in this embodiment of the invention, when the target node of the client cluster needs to access the server's rate-limiting interface to obtain data services, the request volume per second of all nodes in the client cluster at the current moment can be counted first, and the current token count of the target node can be calculated.

[0040] In practical implementation, a minimum statistical time interval of 1 second within the current moment can be preset. The request volume of each node is counted at each statistical time interval, and the request volume within each statistical time interval is stored in the corresponding space. Finally, the sum of the request volumes within each statistical time interval is determined as the request volume of that node within 1 second at the current moment, and the sum of the request volumes of all nodes within 1 second at the current moment is determined as the request volume per second of the client cluster at the current moment. Simultaneously, the access rate of each target node is calculated based on the request volume per second of the client cluster at the current moment, and the current token count of the target node is calculated based on the access rate.

[0041] Step S102: If the number of requests per second of the client cluster is less than the access threshold of the rate limiting interface on the server, and the current token count of the target node is greater than zero, then the target node is allowed to access the rate limiting interface.

[0042] In one implementation, when a target node needs to access the rate-limiting interface, the system first verifies the number of requests per second of the client cluster and the current number of tokens of the target node. It determines whether the number of requests per second of the client cluster is less than the access threshold of the rate-limiting interface on the server side, and whether the current number of tokens of the target node is greater than zero. If both of the above conditions are met, the target node is allowed to access the rate-limiting interface, and the current number of tokens of the target node is decremented by 1.

[0043] In one implementation, if the number of requests per second (PS) of the client cluster is greater than or equal to the access threshold of the server's rate-limiting interface, or if the current token count of the target node is zero, then after a statistical time interval, the PS of the client cluster and the current token count of the target node are retrieved again for further judgment. Specifically, if both conditions—that the PS of the client cluster is less than the access threshold of the server's rate-limiting interface and the current token count of the target node is greater than zero—cannot be met simultaneously, then the process is blocked and waits for the statistical time interval to complete before re-evaluating the conditions after the tokenValue is updated.

[0044] The access method for the rate-limiting interface provided in this embodiment of the invention counts the number of requests per second from the client cluster to the server-side rate-limiting interface, and determines whether the target node can access the rate-limiting interface based on the number of requests per second from the client cluster and the current token count of the target node. This makes the access of the client cluster to the rate-limiting interface more balanced, improves the stability of the client cluster's requests to the server-side rate-limiting interface, and avoids the problem of the entire client cluster being rate-limited due to a sudden surge in requests from a single node.

[0045] To improve the accuracy of calculating the current real-time request volume of each node and the total number of requests per second for the client cluster, this embodiment of the invention also provides a method for calculating the request volume and token count of each node. Specifically, when obtaining the current request volume per second of the client cluster and the current token count of the target node, the following steps (including but not limited to steps 1 and 2) can be used:

[0046] Step 1: Obtain the number of requests per second for each node in the client cluster at the current moment, and sum the number of requests per second for each node to determine the number of requests per second for the client cluster.

[0047] First, the number of the calculation grid is determined based on the current timestamp in milliseconds and the pre-set statistical time interval.

[0048] The calculation grid is used to store the number of requests from nodes to the rate-limiting interface within a statistical time interval. The calculation grid is divided based on the statistical time interval.

[0049] In practical implementation, it is assumed that the client cluster contains N nodes (A1, A2, A3...An) that need to access the server-side rate-limiting interface to obtain data services. The total number of requests per second (RPS) of the server-side rate-limiting interface is P, which is the access threshold set for the client cluster. That is, when the total number of requests per second from the client cluster to access the server-side rate-limiting interface exceeds the value of P, the server-side rate-limiting interface becomes unavailable.

[0050] Specifically, the minimum statistical time interval (s milliseconds) for counting requests on a node can be preset. During initialization, 1000 / s calculation cells can be allocated to each node, numbered from 0 to (1000 / s-1). Each calculation cell stores the number of requests the node makes to the server's rate-limiting interface within the corresponding statistical time interval. For example, when s is 100 milliseconds, there are 10 calculation cells, numbered 0 to 9. Cell 0 stores the node's requests within 1-100 milliseconds, cell 1 stores the requests within 101-200 milliseconds, and so on, with cell 9 storing the requests within 901-1000 milliseconds. The sum of the requests in all calculation cells is the node's total requests within 1000 milliseconds at the current moment (i.e., the node's requests per second). Furthermore, the global access variable `count` (the request count value) for the minimum statistical time interval is initialized to 0. Each time a node makes a request to the server's rate-limiting interface within the statistical time interval, the `count` value is incremented by 1.

[0051] In this embodiment of the invention, the request volume of each node can be counted in real time through a scheduled task. The scheduled task execution interval is s milliseconds, that is, the request volume of each node is counted every s milliseconds. Specifically, the number of the current calculation grid is obtained by dividing the current timestamp in milliseconds by the statistical time interval s.

[0052] Then, based on the current request count value, update the request count value stored in the calculation cell corresponding to the number, and clear the request count value to zero; where the request count value is the number of requests that the node accesses the rate-limiting interface within the statistical time interval.

[0053] In practice, the current request count (i.e., the current count value) can be overwritten with the original request count value stored in the calculation cell corresponding to the current number, and the count value can be reset to 0. The request count for the rate-limiting interface in the next statistical time interval can then be recalculated. Furthermore, if the current count value is 0, it is also necessary to overwrite the original request count value stored in the calculation cell corresponding to the current number.

[0054] Finally, for each node, the sum of the request values ​​corresponding to each computation cell of the node is used to determine the request volume per second of the node at the current moment.

[0055] In practical implementation, for each node, the sum of the request values ​​stored in all computing cells is the request volume of that node in the last 1000 milliseconds, which is also the request volume Li per second of node i at the current moment. The sum of the request volumes per second of all nodes is the RPS value of the client cluster at the current moment.

[0056] Step 2: Determine the access rate of the target node based on the number of requests per second of the target node, the number of requests per second of the client cluster, and the access threshold of the rate limiting interface on the server side, and determine the current number of tokens of the target node based on the access rate.

[0057] In one implementation, after obtaining the RPS values ​​of each node and the client cluster, the access rate of each node can be calculated. Specifically, the access rate of the target node can be calculated using the following formula:

[0058] Vi=P*(T-Li) / ((N-1)*T)

[0059] In the formula, Vi is the access rate of the target node, T is the number of requests per second of the client cluster, Li is the number of requests per second of the target node, N is the number of nodes in the client cluster, and P is the access threshold of the rate limiting interface on the server.

[0060] Furthermore, when determining the current token count of a target node based on the access rate, the following methods can be used, including but not limited to: First, determine the number of newly added tokens within the statistical time interval based on the access rate of the target node; second, determine whether the original token count of the target node exceeds the token count threshold; if the original token count exceeds the token count threshold, determine whether the sum of the original token count and the newly added token count exceeds the token count threshold; if the sum of the original token count and the newly added token count exceeds the token count threshold, then the token count threshold is determined as the current token count of the target node; if the sum of the original token count and the newly added token count does not exceed the token count threshold, then the sum of the original token count and the newly added token count is determined as the current token count of the target node.

[0061] In practice, each node is initialized with a global variable of type double, `tokenValue`, and a threshold value `M` for `tokenValue` (i.e., the token count threshold). `tokenValue` is initialized to 0. When updating the token count of a node, the minimum increase in token count within the minimum statistical time interval, `addValue`, is calculated first. Then, it is determined whether the original `tokenValue` (i.e., the original token count) exceeds the set threshold. If it does, no further operations are performed. If it does not exceed the threshold, the sum of the original `tokenValue` and `addValue` is checked. If it does, the `tokenValue` is modified to the threshold. If it does not exceed the threshold, the `tokenValue` is modified to (`tokenValue` + `addValue`). The initial access rate of a node is (P / N), and the total access rate of the client cluster is (P / N*N = P).

[0062] After obtaining the access rate of the target node at the current moment, the number of newly added tokens within the statistical time interval can be calculated based on the access rate. Specifically, firstly, the total number of newly added tokens per second of the target node is calculated based on the access rate, i.e.: P*(T-Li) / ((N-1)*T). Then, the number of newly added tokens within the statistical time interval is calculated by dividing the total number of newly added tokens by the number of calculation cells: (P*(T-Li) / ((N-1)*T)) / (1000 / s). Then, it is determined whether the original number of tokens of the target node exceeds the token number threshold. If so, it is determined whether the sum of the original number of tokens and the number of newly added tokens exceeds the token number threshold. If so, the token number threshold is determined as the current number of tokens of the target node. If not, the sum of the original number of tokens and the number of newly added tokens is determined as the current number of tokens of the target node.

[0063] In one implementation, assume that the request volume of each node's rate-limiting interface at the current moment is L1, L2, L3...Ln (n is the number of client cluster nodes), and the current total RPS value of the client cluster is T, i.e., T = L1 + L2 + L3...Ln. If T is 0, the number of new tokens on each node is P / N; if T > 0, the number of new tokens on each node is P*(T-Li) / ((N-1)*T), and the current total number of new tokens is P*(T-L1+T-L2...+T-Ln) / ((N-1)*T) = P*(N*T-L1-L2...-Ln) / ((N-1)*T) = P, that is, the total number of new tokens still does not exceed the RPS rate-limiting threshold P, and at the same time, it can make the access requests among nodes more balanced and suppress nodes with faster access rates.

[0064] Furthermore, the above method also includes: if the number of requests per second (RPS) of the client cluster is greater than or equal to the access threshold of the server-side rate-limiting interface, then the update of the token count of each node in the client cluster is stopped until the number of requests per second of the client cluster is less than the access threshold of the server-side rate-limiting interface. Specifically, if the total RPS value of the client cluster exceeds the RPS threshold P, then the tokenValue of all nodes in the client cluster stops increasing until the real-time RPS value is less than the threshold P, thereby suppressing nodes with faster access rates, making the access to the rate-limiting interface of the entire client cluster more balanced, and controlling the total access rate of the client cluster at each moment to be lower than the RPS threshold P, ensuring the stability of the cluster access to the rate-limiting interface and the correctness of the business logic.

[0065] For ease of understanding, this embodiment of the invention also provides a specific example of processing access rate limiting interfaces, see [link to example]. Figure 2The diagram illustrates a client cluster node access to a rate-limiting interface. It shows that the client contains N nodes, and the business logic in each node needs to access the rate-limiting interface of the server. The following describes the access method of the aforementioned rate-limiting interface, taking the server's overall rate-limiting RPS threshold for the client cluster as P=500, N=5, and the minimum statistical time interval as 100ms.

[0066] See Figure 3 The diagram shows a node initialization process. Each node's calculation grid is initialized based on the minimum statistical time interval, which is 100ms. The number of requests within the current minimum time interval is counted every 100ms. Each node is allocated 1000 / 100 = 10 calculation grids, numbered 0-9, to store the number of requests accessing the rate-limiting interface within the corresponding statistical time interval. The initial values ​​of all of these grids are 0. The node's global variable count is also initialized to store the real-time number of requests within the current statistical time interval.

[0067] See Figure 4 The diagram illustrates a judgment logic. It obtains the real-time access request of node i to the rate-limiting interface. Before accessing the rate-limiting interface, it checks whether two conditions are met simultaneously: 1. Is the total RPS value of the current client cluster less than the RPS threshold P? 2. Is the tokenValue greater than 0? If both conditions are met, access to the rate-limiting interface is allowed, and the count value is incremented by 1, while the tokenValue is decremented by 1. Otherwise, it continues to wait for the minimum calculation time interval, and after the tokenValue is updated, it re-checks whether the conditions are met.

[0068] Furthermore, during the real-time access to the rate-limited interface by the client cluster, the node's RPS value and tokenValue can be updated via scheduled tasks. See also Figure 5 The diagram illustrates a node RPS value update process. First, the cell number is calculated based on the current timestamp in milliseconds. The current count value is then verified to be consistent with the request volume value stored in the cell. If they are inconsistent, the current count value overwrites the value in the cell. After the update, the RPS value of the current node (i.e., the sum of the request volume values ​​stored in all cells) is calculated. Then, the total RPS value of the client cluster (i.e., the sum of the RPS values ​​of all nodes) is calculated. Finally, the tokenValue generation rate (i.e., access rate) of all nodes can be updated based on the node's RPS value and the total RPS value of the client cluster.

[0069] For example, suppose node 1 requests the rate-limiting interface once at timestamp 1665467139446. The current count value is incremented by 1, changing from 0 to 1. If node 1 accesses the rate-limiting interface a total of 9 times within the time range of timestamps 1665467139446-1665467139500, the count value becomes 9. The state diagram of node 1 before the update at timestamp 1665467139500 is as follows: Figure 6 As shown, firstly, the result of dividing the current timestamp in milliseconds by the minimum statistical time interval is 500 / 100 = 5. According to the numbering rule of the calculation cells, the calculation cell number is 4. Therefore, the current count value is overwritten into the calculation cell coded as 4, and the count value is cleared to 0. The state diagram of node 1 after the update at time 1665467139500 is as follows. Figure 7 As shown.

[0070] At timestamp 1665467139600, node 1 simultaneously updates the count value of all nodes to the calculation cell coded 5. The state diagram after node 1's update at timestamp 1665467139600 is as follows. Figure 8 As shown. The sum of the values ​​in the calculation cells numbered 0-9 is the current rate-limited interface access volume Li per second for that node. If the count value of node i is 6 at timestamp 1665467140500, then this value is overwritten in calculation cell 4 of node i. The state diagram of node i after the update at timestamp 1665467140500 is as follows. Figure 9 As shown.

[0071] See Figure 10 The diagram illustrates a tokenValue update process. First, based on the node's tokenValue generation rate, the minimum statistical time interval for adding a new value (addValue) is calculated. Then, it is determined whether the original tokenValue exceeds a set threshold. If it does, no operation is performed. If it does not exceed the threshold, it is determined whether the original tokenValue plus addValue exceeds the threshold. If it does, the tokenValue is modified to the threshold. If it does not exceed the threshold, an addition operation is performed, modifying the tokenValue to (tokenValue + addValue).

[0072] For example: First, assign a token value (tokenValue) to each node, initialized to 0. Simultaneously, initialize the access rate of each node to the interface to 500 / 5 = 100, where 100 is the number of tokens added per second. Therefore, the minimum time interval for adding tokens is 100 / 10 = 10. Assuming the tokenValue threshold M is 15, then 10 tokens are added in each minimum statistical time interval. If the tokenValue is greater than or equal to the threshold 15, addition stops. Assuming the current request volume of all nodes is L1 = 5, L2 = 4, L3 = 6, L4 = 2, and L5 = 3, the total RPS of the client cluster is T = 25. Using the formula P*(T-Li) / ((N-1)*T), calculate the access rates of all nodes, which are 100, 105, 95, 115, and 110 respectively. Therefore, the number of tokens added in the minimum statistical time interval for each node is 10, 10.5, 9.5, 11.5, and 11 respectively. As can be seen, the greater the number of requests from a node, the lower the token generation rate, thus controlling the balance between nodes while keeping the total rate within the threshold range.

[0073] If the RPS of all nodes is currently 0, then the access rate of all nodes is set to 500 / 5 = 100. Assuming that at time 1665467139700, the total RPS of the cluster exceeds the RPS threshold P, then all node tokenValues ​​in the cluster will stop being added until the real-time RPS value is less than the threshold P. This suppresses nodes with faster access rates, making the overall cluster access rate limiting interface more balanced, while ensuring that the total cluster access rate at each moment is below the RPS threshold P, guaranteeing the stability of the cluster access rate limiting interface and the correctness of the business logic.

[0074] The method provided in this embodiment of the invention improves statistical accuracy by refining the calculation of the access request volume of each node in each statistical time interval, accurately calculating the RPS value of each node, and finally summing them to obtain the total RPS value of the real-time access rate limiting interface of the entire client cluster. At the same time, based on the statistical calculation of the request volume of each node, the access rate of each node is balanced and adjusted according to the request volume of each node, suppressing nodes with faster rates, making the access rate limiting interface of the entire cluster more balanced, and ensuring that the total access rate of the client cluster at each moment is lower than the target RPS threshold P, thus ensuring the stability of the cluster access rate limiting interface and solving the problem of the entire cluster being rate-limited due to a sudden request from a single node.

[0075] Regarding the aforementioned method for accessing rate-limited interfaces, this invention provides an embodiment of a device for accessing rate-limited interfaces, see [link to embodiment]. Figure 11 The schematic diagram shown illustrates the structure of a rate-limiting interface access device, indicating that the device mainly includes the following parts:

[0076] The data acquisition module 1101 is used to acquire the number of requests per second of the client cluster and the current number of tokens of the target node when the target node of the client cluster requests access to the rate limiting interface of the server.

[0077] The judgment module 1102 is used to allow the target node to access the rate-limited interface if the number of requests per second of the client cluster is less than the access threshold of the rate-limiting interface on the server and the current token count of the target node is greater than zero.

[0078] The access device for the rate-limiting interface provided in this embodiment of the invention counts the number of requests per second from the client cluster to access the server-side rate-limiting interface, and determines whether the target node can access the rate-limiting interface based on the number of requests per second from the client cluster and the current token count of the target node. This makes the access of the client cluster to the rate-limiting interface more balanced, improves the stability of the client cluster's requests to access the server-side rate-limiting interface, and avoids the problem of the entire client cluster being rate-limited due to a sudden request from a single node.

[0079] In one embodiment, the data acquisition module 1101 is further configured to: acquire the number of requests per second for each node in the client cluster at the current moment, and determine the sum of the number of requests per second for each node as the number of requests per second for the client cluster; determine the access rate of the target node based on the number of requests per second for the target node, the number of requests per second for the client cluster, and the access threshold of the rate limiting interface of the server, and determine the current number of tokens for the target node based on the access rate.

[0080] In one embodiment, the data acquisition module 1101 is further configured to: determine the number of the calculation grid based on the current timestamp in milliseconds and a pre-set statistical time interval; wherein the calculation grid is used to store the number of requests from nodes to the rate-limiting interface within the statistical time interval, and the calculation grid is divided based on the statistical time interval; update the request count value stored in the calculation grid corresponding to the number based on the request count value at the current time, and clear the request count value to zero; wherein the request count value is the number of requests from nodes to the rate-limiting interface within the statistical time interval; for each node, determine the number of requests per second of the node at the current time by summing the request count values ​​corresponding to each calculation grid of the node.

[0081] In one embodiment, the data acquisition module 1101 is further configured to: calculate the access rate of the target node according to the following formula:

[0082] Vi=P*(T-Li) / ((N-1)*T)

[0083] Where Vi is the access rate of the target node, T is the number of requests per second of the client cluster, Li is the number of requests per second of the target node, N is the number of nodes in the client cluster, and P is the access threshold of the rate limiting interface on the server.

[0084] In one embodiment, the data acquisition module 1101 is further configured to: determine the number of new tokens within a statistical time interval based on the access rate of the target node; determine whether the original number of tokens of the target node exceeds the token number threshold; if the original number of tokens exceeds the token number threshold, determine whether the sum of the original number of tokens and the number of new tokens exceeds the token number threshold; if the sum of the original number of tokens and the number of new tokens exceeds the token number threshold, determine the token number threshold as the current number of tokens of the target node; if the sum of the original number of tokens and the number of new tokens does not exceed the token number threshold, determine the sum of the original number of tokens and the number of new tokens as the current number of tokens of the target node.

[0085] In one embodiment, the above-mentioned apparatus further includes a pause update module, configured to: if the number of requests per second of the client cluster is greater than or equal to the access threshold of the rate limiting interface of the server, stop updating the number of tokens of each node in the client cluster until the number of requests per second of the client cluster is less than the access threshold of the rate limiting interface of the server.

[0086] In one embodiment, the above-mentioned device further includes a waiting module, configured to: if the number of requests per second of the client cluster is greater than or equal to the access threshold of the rate limiting interface of the server, or the current token count of the target node is equal to zero, then after waiting for the minimum statistical time interval, re-acquire the number of requests per second of the client cluster and the current token count of the target node for judgment.

[0087] It should be noted that the device provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment. The specific numerical values ​​provided in this embodiment are merely exemplary and are not intended to limit the scope of the invention.

[0088] This invention also provides an electronic device, specifically, the electronic device includes a processor and a storage device; the storage device stores a computer program, and the computer program, when run by the processor, executes the method described in any of the above embodiments.

[0089] Figure 12 The present invention provides a schematic diagram of the structure of an electronic device 100, which includes a processor 120, a memory 121, a bus 122 and a communication interface 123. The processor 120, the communication interface 123 and the memory 121 are connected through the bus 122. The processor 120 is used to execute executable modules, such as computer programs, stored in the memory 121.

[0090] The memory 121 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 123 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0091] Bus 122 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0092] The memory 121 is used to store programs. After receiving an execution instruction, the processor 120 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 120 or implemented by the processor 120.

[0093] Processor 120 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 120 or by instructions in software form. Processor 120 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 121, and processor 120 reads the information from memory 121 and, in conjunction with its hardware, completes the steps of the above method.

[0094] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0095] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for accessing a rate-limited interface, characterized in that, include: When a target node in the client cluster requests access to the rate-limiting interface of the server, the request volume per second of the client cluster and the current token count of the target node are obtained at the current moment. The minimum statistical time interval within 1 second at the current moment is preset, and the request volume of the node is counted once every statistical time interval, and the request volume within each statistical time interval is stored in the corresponding space. If the number of requests per second of the client cluster is less than the access threshold of the rate limiting interface of the server, and the current token count of the target node is greater than zero, then the target node is allowed to access the rate limiting interface. Obtaining the current request volume per second of the client cluster and the current token count of the target node includes: obtaining the current request volume per second of each node in the client cluster, and determining the sum of the request volume per second of each node as the request volume per second of the client cluster; determining the access rate of the target node based on the request volume per second of the target node, the request volume per second of the client cluster, and the access threshold of the rate limiting interface of the server, and determining the current token count of the target node based on the access rate; Determining the current token count of the target node based on the access rate includes: determining the number of newly added tokens within the statistical time interval based on the access rate of the target node; determining whether the original token count of the target node exceeds a token count threshold; if the original token count does not exceed the token count threshold, determining whether the sum of the original token count and the newly added token count exceeds the token count threshold; if the sum of the original token count and the newly added token count exceeds the token count threshold, determining the token count threshold as the current token count of the target node; if the sum of the original token count and the newly added token count does not exceed the token count threshold, determining the sum of the original token count and the newly added token count as the current token count of the target node.

2. The access method according to claim 1, characterized in that, Obtain the number of requests per second for each node in the client cluster at the current moment, including: The number of the calculation grid is determined based on the current timestamp in milliseconds and a pre-set statistical time interval; wherein, the calculation grid is used to store the number of requests from the node to the rate-limiting interface within the statistical time interval, and the calculation grid is divided based on the statistical time interval; The request count value stored in the calculation cell corresponding to the number is updated based on the request count value at the current moment, and the request count value is cleared to zero; wherein, the request count value is the number of requests made by the node to the rate limiting interface within the statistical time interval; For each node, the sum of the request values ​​corresponding to each computation cell of the node is determined as the request volume per second of the node at the current moment.

3. The access method according to claim 1, characterized in that, The access rate of the target node is determined based on the requests per second of the target node, the requests per second of the client cluster, and the access threshold of the rate limiting interface on the server, including: The access rate of the target node is calculated using the following formula: Where Vi is the access rate of the target node, T is the number of requests per second of the client cluster, Li is the number of requests per second of the target node, N is the number of nodes in the client cluster, and P is the access threshold of the rate limiting interface of the server.

4. The access method according to claim 1, characterized in that, The method further includes: If the number of requests per second in the client cluster is greater than or equal to the access threshold of the rate limiting interface on the server, then the update of the token count of each node in the client cluster will stop until the number of requests per second in the client cluster is less than the access threshold of the rate limiting interface on the server.

5. The access method according to claim 2, characterized in that, The method further includes: If the number of requests per second of the client cluster is greater than or equal to the access threshold of the rate limiting interface of the server, or if the current token count of the target node is zero, then after waiting for the statistical time interval, the number of requests per second of the client cluster and the current token count of the target node are obtained again for judgment.

6. An access device for a rate-limiting interface, characterized in that, include: The data acquisition module is used to acquire the number of requests per second of the client cluster and the current number of tokens of the target node when the target node of the client cluster requests access to the rate limiting interface of the server at the current moment; wherein, a minimum statistical time interval of 1 second is preset at the current moment, the number of requests of the node is counted once every time interval of the statistical time interval, and the number of requests in each statistical time interval is stored in the corresponding space. The judgment module is used to allow the target node to access the rate-limiting interface if the number of requests per second of the client cluster is less than the access threshold of the rate-limiting interface of the server and the current token count of the target node is greater than zero. The data acquisition module is specifically used to: acquire the number of requests per second for each node in the client cluster at the current moment, and determine the sum of the number of requests per second for each node as the number of requests per second for the client cluster; determine the access rate of the target node based on the number of requests per second for the target node, the number of requests per second for the client cluster, and the access threshold of the rate limiting interface of the server, and determine the current number of tokens for the target node based on the access rate. The data acquisition module is specifically used for: determining the number of newly added tokens within the statistical time interval based on the access rate of the target node; determining whether the original number of tokens of the target node exceeds a token number threshold; if the original number of tokens does not exceed the token number threshold, determining whether the sum of the original number of tokens and the number of newly added tokens exceeds the token number threshold; if the sum of the original number of tokens and the number of newly added tokens exceeds the token number threshold, determining the token number threshold as the current token number of the target node; if the sum of the original number of tokens and the number of newly added tokens does not exceed the token number threshold, determining the sum of the original number of tokens and the number of newly added tokens as the current token number of the target node.

7. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by a processor, performs the steps of the method described in any one of claims 1 to 5.

Images