Cluster service rate limiting management methods, devices, equipment and media

Abstract

This invention discloses a method, apparatus, device, and medium for cluster service rate limiting management. The method includes: allocating remaining tokens after pre-allocation according to at least two preset token redistribution granularities to obtain at least two token redistribution quantity queues with identical data in each queue and values ​​corresponding to the preset token redistribution granularities; upon receiving a token redistribution request from any cluster service node, reading the queue data with the largest value from the unread queue data in the token redistribution quantity queues as the target queue data; and allocating the tokens corresponding to the target queue data to the any cluster service node until all queue data in the token redistribution quantity queues have been read. This embodiment improves the accuracy of rate limiting management for service clusters and increases token utilization.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of network communication technology, and in particular to a method, apparatus, device and medium for managing rate limiting of cluster services. Background Technology

[0002] Service cluster access rate limiting is implemented through a token service center that manages the number of tokens used by service request consumer nodes within the service cluster. The token service center pre-allocates a certain number of tokens to each consumer in the service cluster, and then allocates a fixed number of new tokens to each consumer based on their token requests, until the required number of tokens has been allocated.

[0003] However, in the process of implementing the present invention, it was found that the prior art has at least the following technical problems: Since different consumers have different consumption capabilities for service requests, allocating a fixed number of new tokens to each consumer each time will result in some consumers having exhausted their tokens and no longer being able to consume new service requests, while some consumers still have tokens remaining but no new service requests, causing some tokens to be wasted and access rate limiting management is not accurate enough. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for managing rate limiting of cluster services, which can more flexibly allocate tokens to cluster service nodes and make rate limiting management of service clusters more precise.

[0005] In a first aspect, embodiments of the present invention provide a cluster service rate limiting management method, the method comprising:

[0006] Based on at least two preset token redistribution granularities, the remaining tokens to be redistributed after pre-allocation are allocated and planned to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities.

[0007] If a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data.

[0008] The number of tokens corresponding to the target queue data is allocated to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0009] Secondly, embodiments of the present invention also provide a cluster service rate limiting management device, the device comprising:

[0010] The token allocation planning module is used to allocate the remaining tokens to be allocated after pre-allocation according to at least two preset token redistribution granularities, and to obtain at least two queues of token redistribution quantity with the same data in each queue and the value corresponding to the preset token redistribution granularity.

[0011] The token allocation request acquisition module is used to read the largest value in an unread queue in the token redistribution quantity queue as the target queue data when a token redistribution request is obtained from any cluster service node.

[0012] The token allocation module is used to allocate the number of tokens corresponding to the target queue data to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0013] Thirdly, embodiments of the present invention also provide a computer device, the computer device comprising:

[0014] One or more processors;

[0015] Memory, used to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the cluster service rate limiting management method provided in any embodiment of the present invention.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cluster service rate limiting management method provided in any embodiment of the present invention.

[0018] The technical solution of this embodiment first plans the allocation of remaining tokens after pre-allocation based on at least two preset token redistribution granularities, resulting in at least two token redistribution quantity queues with identical data and values ​​corresponding to the preset token redistribution granularities. Then, when a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data. Tokens corresponding to the target queue data are allocated to any cluster service node until all queue data in the token redistribution quantity queue has been read. That is, by managing token allocation more directly through queue data, a more granular token allocation method can be used to meet the token needs of cluster service nodes, reducing the waste of excessive tokens on cluster service nodes with low service request processing volume. The technical solution of this embodiment solves the problem of low rate limiting accuracy in cluster rate limiting management, achieving more granular rate limiting management, reducing token waste, and improving the accuracy of rate limiting management to a certain extent. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a cluster service rate limiting management method provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram illustrating a cluster service rate limiting management example provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram illustrating the working process of a cluster service node according to an embodiment of the present invention;

[0022] Figure 4 A flowchart illustrating a cluster service rate limiting management method provided in an embodiment of the present invention;

[0023] Figure 5 A flowchart illustrating a cluster service rate limiting management method provided in an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of a cluster service rate limiting management device provided in an embodiment of the present invention;

[0025] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0027] Figure 1 This is a flowchart illustrating a cluster service rate limiting management method provided in an embodiment of the present invention. This embodiment is applicable to scenarios involving rate limiting management during concurrent services in a service cluster. The method can be executed by a cluster service rate limiting management device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0028] like Figure 1 As shown, the cluster service rate limiting management method in this embodiment includes the following steps:

[0029] S110. Based on at least two preset token redistribution granularities, allocate the remaining tokens to be redistributed after pre-allocation to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities.

[0030] In this context, "redistribution" refers to the token service center redistributing tokens to the corresponding cluster service nodes based on received token redistribution requests after pre-distributing tokens. This includes every token distribution after pre-distribution. The preset token redistribution granularity can be understood as the granularity at which the token service center redistributes tokens to the cluster service nodes based on received token redistribution requests; that is, the number of tokens distributed to the cluster service nodes each time after pre-distribution. At least two preset token redistribution granularities mean that the number of tokens distributed by the token service center to the cluster service nodes in a single transaction is different, including at least two different distribution quantities. This implies that during the token redistribution phase, there are different levels of rate limiting, and the size of the single distribution granularity corresponds to the degree of rate limiting.

[0031] When each cluster service node requests a new token from the token service center after using up its pre-allocated tokens, the token service center typically redistributes tokens to each cluster service node according to a pre-set fixed allocation quantity. This means there is only one token redistribution granularity, which leads to different token usage situations for cluster service nodes with different service request processing capabilities. Some nodes do not have enough tokens, while others have tokens left over. From the overall analysis of concurrent task rate limiting management, the rate limiting management is not precise enough, resulting in tokens not being fully utilized.

[0032] In this application, the tokens to be allocated are planned in advance, especially the remaining tokens to be allocated after pre-allocation are planned based on different preset token redistribution granularities. This allows the token service center to manage cluster rate limiting more granularly and improve the accuracy of management.

[0033] Specifically, the remaining tokens to be allocated after pre-allocation are planned according to at least two preset token redistribution granularities, including the following steps:

[0034] First, establish a functional relationship between the preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue, and the number of remaining tokens to be redistributed; then, parse the established functional relationship to obtain at least two token redistribution quantity queues with the same data in each queue and values ​​corresponding to the preset token redistribution granularity.

[0035] The number of remaining tokens to be allocated is determined by subtracting the number of pre-allocated tokens from the total number of pre-allocated tokens. For example, if the total number of pre-allocated tokens is 400, each cluster service node can be pre-allocated 30 tokens, and there are a total of 4 cluster service nodes, then the number of remaining tokens to be allocated is 400 - 30 * 4 = 280.

[0036] In this system, each preset token redistribution granularity corresponds to a token redistribution quantity queue. The product of each preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue is calculated to determine the number of allocatable tokens corresponding to each token redistribution quantity queue. Then, under the constraints that the sum of the product values ​​is less than or equal to the number of remaining tokens to be allocated, the number of queue data in the token redistribution quantity queue is a positive integer, and the minimum redistribution granularity in the preset token redistribution granularity is 1, the number of queue data in each token redistribution quantity queue is parsed to determine the number of queue data in each token redistribution quantity queue. This results in at least two token redistribution quantity queues with the same data in each queue and values ​​corresponding to the preset token redistribution granularity, which serve as the result of the token pre-allocation planning.

[0037] S120. If a token redistribution request is obtained from any cluster service node, read the queue data with the largest value among the unread queue data in the token redistribution quantity queue as the target queue data.

[0038] When each cluster service node is about to run out of pre-allocated tokens or tokens it has requested to be acquired, it will send a token redistribution request to the token service center to obtain more tokens. Upon receiving a token redistribution request from any cluster service node, the token service center will distribute tokens according to the token pre-allocation strategy planned in the above steps.

[0039] Specifically, during the operation of the cluster service, cluster rate limiting management through the token allocation process is a progressive process. First, queue data is read from the queue corresponding to the larger allocation granularity among the preset token redistribution granularities to determine the token redistribution quantity. Once all queue data in one queue is read, the next queue corresponding to the preset token redistribution granularity is read to determine the token redistribution quantity. That is, as the processing of concurrent services progresses, the number of tokens allocated in a single instance decreases until all remaining tokens are allocated. During this process, the number of tokens allocated in a single instance decreases, thus avoiding a situation where a large number of tokens are allocated to a single cluster service node and cannot be used, resulting in token waste. In a preferred embodiment, the smallest preset token redistribution granularity can be set to 1, ensuring that each token is allocated appropriately.

[0040] Among them, the queue with the largest value among the unread queue data in the token redistribution quantity queue is used as the target queue data, which is the maximum number of tokens that can be allocated at present.

[0041] S130. Distribute the number of tokens corresponding to the target queue data to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0042] After determining the number of tokens to be allocated in the current allocation based on the planned token allocation strategy, the corresponding number of tokens can be directly allocated to the corresponding cluster service node that issued the request.

[0043] For each token redistribution request received, the number of tokens available for redistribution can be determined from the token redistribution quantity queue until all queue data in the token redistribution quantity queue has been read, that is, all available tokens have been redistributed.

[0044] The technical solution of this embodiment first plans the allocation of remaining tokens after pre-allocation based on at least two preset token redistribution granularities, resulting in at least two token redistribution quantity queues with identical data and values ​​corresponding to the preset token redistribution granularities. Then, when a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data. Tokens corresponding to the target queue data are allocated to any cluster service node until all queue data in the token redistribution quantity queue has been read. That is, by managing token allocation more directly through queue data, a more granular token allocation method can be used to meet the token needs of cluster service nodes, reducing the waste of excessive tokens on cluster service nodes with low service request processing volume. The technical solution of this embodiment solves the problem of low rate limiting accuracy in cluster rate limiting management, achieving more granular rate limiting management, reducing token waste, and improving the accuracy of rate limiting management to a certain extent.

[0045] In a specific cluster rate limiting management scenario, two preset token redistribution granularities, 20 and 1, are established. The remaining tokens to be redistributed after pre-allocation are allocated according to a plan, resulting in two queues with identical data and values ​​corresponding to the preset token redistribution granularities. Figure 2As shown. Specifically, the token service center needs to calculate two token redistribution queues based on the number of cluster service nodes (consumers), the total rate limiting threshold (the total number of all available tokens), the number of tokens pre-allocated to each consumer, and the number of tokens pulled in a batch (i.e., the preset token redistribution granularity). The first token redistribution queue stores the larger granularity of single token allocation, which is 20. The second token redistribution queue stores the smaller granularity of single token allocation, where one token is allocated per batch. The number of queue data in the two queues is determined according to the following formula: 20x + y + 4 * 30 <= 400, 20x + 4 * 30 < 400, y >= 1. Where x represents the number of queue data in the first token redistribution queue, y represents the number of queue data in the second token redistribution queue, 400 is the total number of all available tokens, 4 represents the number of cluster service nodes, and 30 is the number of tokens pre-allocated to each cluster service node. After analyzing the functional relationship corresponding to the above formula, it was found that the number of data in the first token redistribution queue is 13, and the number of data in the second token redistribution queue is 20. That is, for the first 13 token redistribution requests, the token service center will allocate 20 new tokens to the corresponding cluster service nodes, and for the subsequent 20 token redistribution requests, it will allocate 1 new token to the corresponding cluster service node.

[0046] Assuming Figure 3 As shown, after each cluster service node uses up its pre-allocated 30 tokens cached on each node, it requests token redistribution from the token service center to obtain new tokens. Accordingly, the token service center first retrieves queue data from the first token redistribution queue, and in response to the received token redistribution request, allocates a new number of tokens to the corresponding cluster service node. If the queue data in the first token redistribution queue is consumed, in response to subsequent token redistribution requests, it retrieves queue data from the second token redistribution queue, and in response to the received token redistribution request, allocates a new number of tokens to the corresponding cluster service node. If the data in both queues is consumed, the token service center will reject the corresponding token redistribution request. If the requests are evenly distributed among cluster service nodes with the same consumption capacity, tokens can be effectively and fully utilized, thereby improving the accuracy of cluster rate limiting. Figure 3 In a distributed system, a Provider is a cluster that provides services to cluster service nodes (consumers).

[0047] Figure 4This is a flowchart illustrating a cluster service rate limiting management method provided in an embodiment of the present invention. This embodiment belongs to the same inventive concept as the cluster service rate limiting management method in the above embodiments, and further describes the process of token revocation for cluster service nodes with a small service request volume. This method can be executed by a cluster service rate limiting management device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0048] like Figure 4 As shown, the cluster service rate limiting management method in this embodiment includes the following steps:

[0049] S210. Based on at least two preset token redistribution granularities, allocate the remaining tokens to be allocated after pre-allocation to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities.

[0050] S220. If a token redistribution request is obtained from any cluster service node, read the queue data with the largest value among the unread queue data in the token redistribution quantity queue as the target queue data.

[0051] S230. Distribute the number of tokens corresponding to the target queue data to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0052] S240. Obtain the service request processing volume data of each cluster service node, and adjust the cluster service nodes according to the target token quantity based on the service request processing volume data.

[0053] During the token allocation management process, the token service center also uses a pre-defined thread to obtain service request processing volume data from each cluster service node to determine the cluster service node's ability to handle user service requests. This service request processing volume data can be the millisecond-level monitoring result of the query per second (QPS) for each cluster service node. QPS is a measure of the amount of traffic processed by a specific query server within a specified time, and can measure the performance of the system server machines. Therefore, cluster service nodes with lower QPS values ​​can be used as target nodes for adjusting the number of tokens allocated to them.

[0054] S250. When entering the token redistribution stage corresponding to the preset rate limiting redistribution granularity, obtain the remaining unused tokens of the target token quantity adjustment cluster service node, and update the queue data of the token redistribution quantity queue according to the number of remaining unused tokens.

[0055] The preset rate limiting redistribution granularity is a smaller value within the preset token redistribution granularity. The token redistribution phase corresponding to this preset rate limiting redistribution granularity can be understood as a phase where a small number of tokens are allocated when responding to token redistribution requests from cluster service nodes, such as allocating only 1, 3, or other small quantities of tokens each time. In this phase, it indicates that the rate limiting threshold is approaching, meaning all tokens will have been allocated.

[0056] Once the token redistribution phase corresponding to the preset rate limiting redistribution granularity is entered, a token reclamation mechanism can be initiated for the target token quantity adjustment cluster service node to obtain the remaining unused tokens of the target token quantity adjustment cluster service node, and update the queue data of the token redistribution quantity queue according to the number of remaining unused tokens so that other non-target token quantity adjustment cluster service nodes can use them.

[0057] Specifically, when updating the queue data of the token redistribution quantity queue based on the remaining number of tokens to be used, the remaining tokens can be allocated to the corresponding token redistribution quantity queues based on the preset token redistribution granularity value corresponding to the queues whose queue data has not been fully read, until all remaining tokens have been allocated. For example, in Figure 3 In the cluster service node 4 shown, 23 remaining tokens are available. The queues in the token redistribution queue that have not been fully read include two queues with values ​​of 3 and 1. Therefore, the queue with a value of 3 will add 7 queues, and the queue with a value of 1 will add 2 queues. That is, using the number of remaining tokens as the divisor and the relatively larger preset token redistribution granularity value in the queues with incompletely read tokens as the divisor, the quotient is the number of new queues added to the queues with relatively larger preset token redistribution granularity in the queues with incompletely read tokens. The remainder is used to update the queue data of the queues with relatively smaller preset token redistribution granularity in the queues with incompletely read tokens.

[0058] After the token redistribution quantity queue is updated, when the token service center receives a token redistribution request, it reads the corresponding queue data from the updated token redistribution quantity queue and distributes tokens based on the read queue data. Through the above steps of the remaining token redistribution mechanism, the available tokens can be fully utilized, thereby improving the accuracy of cluster rate limiting.

[0059] The technical solution of this embodiment first plans the allocation of remaining tokens after pre-allocation based on at least two preset token redistribution granularities, resulting in at least two token redistribution quantity queues with identical data and values ​​corresponding to the preset token redistribution granularities. Then, when a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data. Tokens corresponding to the target queue data are allocated to any cluster service node until all queue data in the token redistribution quantity queue has been read. During this process, unused tokens from cluster service nodes with weak service request capabilities can also be redistributed, thereby improving the accuracy of rate limiting. In other words, by managing token allocation more directly through queue data, a more granular token allocation method can be used to meet the token needs of cluster service nodes, reducing the waste of excessive tokens on cluster service nodes with low service request processing volumes. The technical solution in this embodiment addresses the problem of low accuracy in rate limiting management within a cluster, achieving more granular rate limiting management, reducing token waste, and improving the accuracy of rate limiting management to some extent.

[0060] Figure 5 This is a flowchart illustrating a cluster service rate limiting management method provided in an embodiment of the present invention. This embodiment belongs to the same inventive concept as the cluster service rate limiting management method in the above embodiments, and further describes the dynamic adjustment of the cluster service rate limiting threshold in the event of a cluster service node failure. This method can be executed by a cluster service rate limiting management device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0061] like Figure 5 As shown, the cluster service rate limiting management method in this embodiment includes the following steps:

[0062] S310. Based on at least two preset token redistribution granularities, allocate the remaining tokens to be allocated after pre-allocation to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities.

[0063] S320. If a token redistribution request is obtained from any cluster service node, read the queue data with the largest value among the unread queue data in the token redistribution quantity queue as the target queue data.

[0064] S330. Distribute the number of tokens corresponding to the target queue data to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0065] S340. Obtain network connection status information and CPU utilization information of each cluster service node, and determine the fault status of each cluster service node based on the network connection status information and / or the CPU utilization information.

[0066] During the token allocation management process, the token service center also uses a pre-defined thread to obtain the working status of each cluster service node and identify nodes that are experiencing failures or excessive CPU usage. This is because if several machines in the cluster service nodes fail, such as due to high CPU usage or service interruption, the other normally functioning cluster service nodes will bear a larger traffic load if the total rate limiting threshold (the total number of tokens that can be allocated and used) remains unchanged. This could potentially cause other machines to crash due to the inability to handle the large traffic, resulting in a cluster avalanche (collapse). In such cases, it is necessary to adjust the total rate limiting threshold to prevent the above situation from occurring.

[0067] Specifically, during the operation of the cluster service node monitoring thread, information reflecting the node's working status, such as heartbeat signals indicating the connection status of each service node and CPU utilization of the machine devices, is acquired. The corresponding token service center configuration module then determines the fault status of each cluster service node based on the acquired working status information. The fault status is either "faulty" or "non-faulty." Being in a faulty state can be due to the cluster service node's device being offline or its CPU utilization exceeding a preset threshold (such as 85%).

[0068] S350: Adjust the number of remaining tokens to be allocated based on the number of cluster service nodes in a faulty state.

[0069] During the process of adjusting the rate limiting threshold, the first step is to determine the percentage of cluster service nodes in a faulty state out of the total number of cluster service nodes. Then, based on this percentage, the number of tokens that need to be reduced from the remaining unallocated tokens is determined, i.e., the token reduction amount. The total number of allocable tokens minus the token reduction amount yields the dynamically adjusted rate limiting threshold. The token reduction amount is the product of the remaining unallocated tokens and the percentage.

[0070] Furthermore, the token service center will update the current token redistribution queue based on the adjusted rate limiting threshold. Specifically, it can reduce the number of queues with incompletely read data based on the preset token redistribution granularity value corresponding to the queues with incompletely read data in the token redistribution queue. Starting from the queues with larger preset token redistribution granularity among the queues with incompletely read data in the token redistribution queue, the number of queues to be reduced is determined by multiplying the number of queues reduced in each token redistribution queue by the corresponding queue data value, until the sum of the products of the number of queues reduced in each token redistribution queue equals the corresponding token reduction amount.

[0071] In addition, after some or all of the failed cluster service nodes have returned to normal operation, the token redistribution queue can be updated to partially or fully restore the tokens that have been reduced, thereby enabling dynamic adjustment of the rate limiting threshold.

[0072] The technical solution of this embodiment first plans the allocation of remaining tokens after pre-allocation based on at least two preset token redistribution granularities, resulting in at least two token redistribution quantity queues with identical data and values ​​corresponding to the preset token redistribution granularities. Then, when a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data. Tokens corresponding to the target queue data are allocated to any cluster service node until all queue data in the token redistribution quantity queue has been read. During this process, the rate limiting threshold can be dynamically adjusted according to the working status of the cluster service nodes to ensure the normal operation of cluster service nodes that are not in a faulty state. In other words, by managing token allocation more directly through queue data, a more granular token allocation method can be used to meet the token needs of cluster service nodes, reducing the waste of excessive tokens on cluster service nodes with low service request processing volumes. The technical solution of this embodiment solves the problem of low rate limiting accuracy in cluster rate limiting management, achieving more granular rate limiting management, reducing token waste, and improving the accuracy of rate limiting management to a certain extent.

[0073] Figure 6 This is a schematic diagram of a cluster service rate limiting management device provided in an embodiment of the present invention. This embodiment can be applied to scenarios where rate limiting management is performed during concurrent services of a service cluster. The device can be implemented by software and / or hardware and integrated into a computer terminal device with application development capabilities.

[0074] like Figure 6 As shown, the cluster service rate limiting management device includes: a token allocation planning module 410, a token allocation request acquisition module 420, and a token allocation module 430.

[0075] The token allocation planning module 410 is used to allocate the remaining tokens to be allocated after pre-allocation according to at least two preset token redistribution granularities, and obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities; the token allocation request acquisition module 420 is used to read the queue data with the largest value in one of the unread queue data in the token redistribution quantity queue as the target queue data when a token redistribution request is obtained from any cluster service node; the token allocation module 430 is used to allocate the number of tokens corresponding to the target queue data to any cluster service node until all queue data in the token redistribution quantity queue has been read.

[0076] The technical solution of this embodiment first plans the allocation of remaining tokens after pre-allocation based on at least two preset token redistribution granularities, resulting in at least two token redistribution quantity queues with identical data and values ​​corresponding to the preset token redistribution granularities. Then, when a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data. Tokens corresponding to the target queue data are allocated to any cluster service node until all queue data in the token redistribution quantity queue has been read. That is, by managing token allocation more directly through queue data, a more granular token allocation method can be used to meet the token needs of cluster service nodes, reducing the waste of excessive tokens on cluster service nodes with low service request processing volume. The technical solution of this embodiment solves the problem of low rate limiting accuracy in cluster rate limiting management, achieving more granular rate limiting management, reducing token waste, and improving the accuracy of rate limiting management to a certain extent.

[0077] Optionally, the token allocation planning module 410 is specifically used for:

[0078] Establish a functional relationship between the preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue and the number of remaining tokens to be allocated, and parse the functional relationship.

[0079] Optionally, the token allocation planning module 410 is further configured to:

[0080] Calculate the product of each preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue;

[0081] The number of queue data in each of the token redistribution queues is determined under the constraints that the sum of the product values ​​is less than or equal to the number of remaining tokens to be allocated, the number of queue data in the token redistribution quantity queue is a positive integer, and the minimum redistribution granularity in the preset token redistribution granularity is 1.

[0082] Optionally, the cluster service rate limiting management device also includes: a token allocation adjustment module, specifically used for:

[0083] Obtain service request processing volume data for each cluster service node, and adjust the cluster service nodes based on the target token quantity determined by the service request processing volume data.

[0084] When entering the token redistribution stage corresponding to the preset rate limiting redistribution granularity, the remaining unused tokens of the target token quantity adjustment cluster service node are obtained, and the queue data of the token redistribution quantity queue is updated according to the number of remaining unused tokens.

[0085] Optionally, the token allocation adjustment module is further configured to:

[0086] Based on the value of the preset token redistribution granularity corresponding to the queue whose queue data has not been fully read in the token redistribution quantity queue, the remaining tokens to be used are allocated to the corresponding token redistribution quantity queue until the remaining tokens to be used are allocated.

[0087] Optionally, the token allocation adjustment module can also be used for:

[0088] Obtain network connection status information and CPU utilization information of each cluster service node, and determine the fault status of each cluster service node based on the network connection status information and / or the CPU utilization information;

[0089] The number of remaining tokens to be allocated is adjusted based on the number of cluster service nodes that are in a faulty state.

[0090] Optionally, the token allocation adjustment module may further be used for:

[0091] Determine the percentage of cluster service nodes in a faulty state relative to the total number of cluster service nodes, and determine the token reduction amount of the currently remaining unallocated tokens based on the percentage;

[0092] Based on the value of the preset token redistribution granularity corresponding to the queues in the token reduction and redistribution queues where the queue data has not been fully read, the corresponding queue data in the queues where the queue data has not been fully read is reduced.

[0093] The cluster service rate limiting management device provided in this embodiment of the invention can execute the cluster service rate limiting management method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0094] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 7 A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention is shown. Figure 7 The computer device 12 shown is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as intelligent controllers and servers, mobile phones, and other terminal devices.

[0095] like Figure 7 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0096] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0097] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0098] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0099] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0100] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 7 As not shown, it can be used in conjunction with computer device 12 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0101] Processing unit 16 executes various functional applications and data processing by running programs stored in system memory 28, such as implementing the cluster service rate limiting management method provided in this embodiment, which includes:

[0102] Based on at least two preset token redistribution granularities, the remaining tokens to be redistributed after pre-allocation are allocated and planned to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities.

[0103] If a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data.

[0104] The number of tokens corresponding to the target queue data is allocated to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0105] This invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the cluster service rate limiting management method provided in any embodiment of this invention. The method includes:

[0106] Based on at least two preset token redistribution granularities, the remaining tokens to be redistributed after pre-allocation are allocated and planned to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities.

[0107] If a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data.

[0108] The number of tokens corresponding to the target queue data is allocated to any of the cluster service nodes until all queue data in the token redistribution queue has been read.

[0109] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0110] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0111] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0112] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0114] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for rate limiting management of cluster services, characterized in that, include: Based on at least two preset token redistribution granularities, the remaining tokens to be redistributed after pre-allocation are allocated and planned to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularities. If a token redistribution request is received from any cluster service node, the queue data with the largest value among the unread queue data in the token redistribution quantity queue is read as the target queue data. Distribute the number of tokens corresponding to the target queue data to any of the cluster service nodes until all queue data in the token redistribution queue has been read. The step of planning the allocation of remaining tokens to be allocated after pre-allocation based on at least two preset token redistribution granularities includes: establishing a functional relationship between the preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue and the number of remaining tokens to be allocated, and parsing the functional relationship; The process of establishing a functional relationship between the preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue, and the number of remaining tokens to be redistributed, and parsing the functional relationship, includes: Calculate the product of each preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue; under the constraints that the sum of the product values ​​is less than or equal to the number of remaining tokens to be allocated, the number of queue data in the token redistribution quantity queue is a positive integer, and the minimum redistribution granularity among the preset token redistribution granularities is 1, analyze the functional relationship to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularity; wherein, each preset token redistribution granularity corresponds to one token redistribution quantity queue.

2. The method according to claim 1, characterized in that, The method further includes: Obtain service request processing volume data for each cluster service node, and adjust the cluster service nodes based on the target token quantity determined by the service request processing volume data. When entering the token redistribution stage corresponding to the preset rate limiting redistribution granularity, the remaining unused tokens of the target token quantity adjustment cluster service node are obtained, and the queue data of the token redistribution quantity queue is updated according to the number of remaining unused tokens.

3. The method according to claim 2, characterized in that, Update the queue data of the token redistribution queue according to the number of remaining tokens to be used, including: Based on the value of the preset token redistribution granularity corresponding to the queue whose queue data has not been fully read in the token redistribution quantity queue, the remaining tokens to be used are allocated to the corresponding token redistribution quantity queue until the remaining tokens to be used are allocated.

4. The method according to claim 1, characterized in that, The method further includes: Obtain network connection status information and CPU utilization information of each cluster service node, and determine the fault status of each cluster service node based on the network connection status information and / or the CPU utilization information; The number of remaining tokens to be allocated is adjusted based on the number of cluster service nodes that are in a faulty state.

5. The method according to claim 4, characterized in that, The adjustment of the number of remaining tokens to be allocated based on the number of cluster service nodes in a faulty state includes: Determine the percentage of cluster service nodes in a faulty state relative to the total number of cluster service nodes, and determine the token reduction amount of the currently remaining unallocated tokens based on the percentage; Based on the value of the preset token redistribution granularity corresponding to the queues in the token reduction and redistribution queues where the queue data has not been fully read, the corresponding queue data in the queues where the queue data has not been fully read is reduced.

6. A cluster service rate limiting management device, characterized in that, include: The token allocation planning module is used to allocate the remaining tokens to be allocated after pre-allocation according to at least two preset token redistribution granularities, and to obtain at least two queues of token redistribution quantity with the same data in each queue and the value corresponding to the preset token redistribution granularity. The token allocation request acquisition module is used to read the largest value in an unread queue in the token redistribution quantity queue as the target queue data when a token redistribution request is obtained from any cluster service node. The token allocation module is used to allocate the number of tokens corresponding to the target queue data to any cluster service node until all queue data in the token redistribution queue has been read. The token allocation planning module is specifically used to establish a functional relationship between the preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue and the number of remaining tokens to be allocated, and to parse the functional relationship. The token allocation planning module is specifically used to calculate the product of each preset token redistribution granularity and the number of queue data in the corresponding token redistribution quantity queue; under the constraints that the sum of the product values ​​is less than or equal to the number of remaining tokens to be allocated, the number of queue data in the token redistribution quantity queue is a positive integer, and the minimum redistribution granularity in the preset token redistribution granularity is 1, the functional relationship is analyzed to obtain at least two token redistribution quantity queues with the same data in each queue and whose values ​​correspond to the preset token redistribution granularity; wherein, each preset token redistribution granularity corresponds to one token redistribution quantity queue.

7. A computer device, characterized in that, The computer device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the cluster service rate limiting management method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the cluster service rate limiting management method as described in any one of claims 1-5.

Images