Token acquisition method, apparatus and device of cluster and medium

Abstract

Embodiments of the present application provide a cluster token acquisition method, device, equipment and storage medium, the present application can acquire the rate of loss of flow limitation, so that the application server can determine its token request strategy based on the acceptable token loss ratio in the cluster environment, thereby providing the basis for subsequent token quantity determination; by meeting the token request condition, the number of tokens required is calculated according to the rate of loss of flow limitation, and then the calculated token quantity is requested to the token server, which reduces the network overhead and token server pressure caused by frequent requests, ensures the accuracy of flow limitation, so that the cluster can balance performance and stability in high concurrency scenarios, and realizes the balance between flow limitation accuracy and system throughput.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for obtaining tokens in a cluster. Background Technology

[0002] With the development of technology, distributed cluster architecture is widely used in the computer field. To address the shortcomings of single-machine rate limiting in adapting to cluster scenarios, cluster rate limiting solutions introduce a token server to uniformly manage token issuance in order to achieve precise rate limiting. However, the number of tokens requested from the token server each time is difficult to set reasonably, and it is impossible to guarantee a balance between performance and rate limiting accuracy. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a method, apparatus, device and medium for obtaining tokens for a cluster that overcomes or at least partially solves the above problems.

[0004] In a first aspect of the present invention, a method for obtaining a token for a cluster is provided, applied to an application server of the cluster, the method comprising: Obtain the rate limiting loss rate; the rate limiting loss rate is the proportion of the number of tokens that the application servers in the cluster are allowed to lose when they request tokens in batches from the token server in the cluster, relative to the maximum number of tokens that the token server is allowed to issue. If the token request conditions are met, the number of tokens to be requested is determined based on the rate limiting loss rate. Request tokens from the token server of the cluster based on the number of tokens.

[0005] Optionally, determining the number of tokens to be requested based on the rate limiting loss rate includes: Obtain the number of application servers deployed and the maximum number of tokens that the token server is allowed to issue; The number of tokens to be requested is determined based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the number of deployments.

[0006] Optionally, determining the number of tokens to be requested based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the deployment quantity includes: The number of tokens to be requested is determined as follows: N = LossRate * U / (M - 1) + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, U is the maximum number of tokens allowed to be issued by the token server, and M is the number of deployments.

[0007] Optionally, determining the number of tokens to be requested based on the rate limiting loss rate includes: Determine the actual number of data requests made by the application server per unit time; The number of tokens to be requested is determined based on the rate limiting loss rate and the actual number of data requests per unit time of the application server.

[0008] Optionally, determining the number of tokens to be requested based on the rate limiting loss rate and the actual number of data requests per unit time of the application server includes: The number of tokens to be requested is determined as follows: N = LossRate * q / w + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, q is the number of data requests, w is the preset time window, and q / w is the actual number of data requests made by the application server per unit time.

[0009] Optionally, the application server includes multiple interfaces, and obtaining the rate limiting loss rate includes: Obtain the rate limiting loss rate for each of the multiple interfaces; The step of determining the number of tokens to be requested based on the rate limiting loss rate, under the condition that the token request conditions are met, includes: If the token request conditions are met, the number of tokens required for each interface is determined based on the rate limiting loss rate of the multiple interfaces.

[0010] Optionally, the token request condition is that the application server has 0 tokens remaining.

[0011] In a second aspect of the invention, a token acquisition device for a cluster is also provided, applied to an application server of the cluster, the device comprising: The acquisition module is used to acquire the rate limiting loss rate, which is the ratio of the number of tokens that the application servers in the cluster are allowed to lose when they request tokens in batches from the token server in the cluster to the maximum number of tokens that the token server is allowed to issue. The determination module is used to determine the number of tokens to be requested based on the rate limiting loss rate, provided that the token request conditions are met. The request module is used to request tokens from the token server of the cluster based on the number of tokens.

[0012] Optionally, the determining module includes: The first submodule is used to obtain the number of application servers deployed and the maximum number of tokens that the token server is allowed to issue; The first determining submodule is used to determine the number of tokens to be requested based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the number of deployments.

[0013] Optionally, the number of tokens to be requested can be determined as follows: N = LossRate * U / (M - 1) + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, U is the maximum number of tokens allowed to be issued by the token server, and M is the number of deployments.

[0014] Optionally, the determining module includes: The second determination submodule is used to determine the actual number of data requests made by the application server per unit time. The third determination submodule is used to determine the number of tokens that need to be requested based on the rate limiting loss rate and the actual number of data requests per unit time of the application server.

[0015] Optionally, the number of tokens to be requested is determined based on the rate limiting loss rate and the actual number of data requests per unit time of the application server, including: The number of tokens to be requested is determined as follows: N = LossRate * q / w + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, q is the number of data requests, w is the preset time window, and q / w is the actual number of data requests made by the application server per unit time.

[0016] Optionally, the token request condition is that the number of remaining tokens on the application server is less than a preset threshold.

[0017] Optionally, the application server includes multiple interfaces, and the acquisition module includes: The acquisition submodule is used to obtain the rate limiting loss rate of multiple interfaces respectively; The determining module includes: The fourth determination submodule is used to determine the number of tokens that each interface needs to request based on the rate limiting loss rate of multiple interfaces, provided that the token request conditions are met.

[0018] Optionally, the token request condition is that the application server has 0 tokens remaining.

[0019] In another aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the steps of the above-described cluster token acquisition method.

[0020] In another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the cluster token acquisition method as described above.

[0021] This invention provides a method, apparatus, device, and storage medium for obtaining tokens in a cluster. By obtaining the rate limiting loss rate, the invention enables application servers to determine their own token request strategy based on an acceptable token loss ratio in a cluster environment, thereby providing a basis for determining the subsequent token quantity. By calculating the number of tokens to be requested based on the rate limiting loss rate when the token request conditions are met, and then sending a request to the token server according to the calculated token quantity, the network overhead and token server pressure caused by frequent requests are reduced, ensuring the accuracy of rate limiting. This allows the cluster to balance performance and stability in high-concurrency scenarios, achieving a balance between rate limiting accuracy and system throughput. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0023] Figure 1 A flowchart illustrating the steps of a cluster token acquisition method provided in this embodiment of the invention; Figure 2 This is a schematic diagram illustrating communication between a client and a distributed cluster, provided as an embodiment of the present invention. Figure 3 This is a schematic diagram of a cluster token acquisition method provided in an embodiment of the present invention; Figure 4 This is a structural block diagram of a token acquisition device for a cluster provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] In related technologies, rate limiting in distributed systems is a common application requirement. By rate limiting, we can ensure that the service is not dragged down by sudden or abnormal traffic, and also avoid mutual interference between different resources and different businesses. For example, a distributed system has multiple resources, and each resource has multiple business requests. When a certain business requests a certain resource, the request volume surges due to unexpected reasons. Without rate limiting, all resources and businesses will be affected by this abnormal traffic.

[0025] Common solutions for rate limiting in distributed systems include single-machine rate limiting, which offers the best performance because the counting operations are performed in memory and do not involve network requests for third-party dependencies. However, changes in the number of machines and uneven traffic load in a distributed system can cause inaccurate overall rate limiting. Introducing a central dependency and using a registry center to dynamically distribute the average rate limit value to each machine based on the number of registered nodes and the rate limit value can solve the problem of changing machine numbers, but it increases system complexity and cannot solve the problem of uneven load on each node.

[0026] Cluster-mode rate limiting was developed to solve the problem of single-machine rate limiting. This solution introduces a storage or middleware as a token server. This token server is responsible for accurately limiting requests based on a threshold. When each machine receives a request, it first requests the token server to obtain a token. If a token is obtained, the request is allowed; otherwise, it is blocked. However, the problem with cluster-mode rate limiting is that every request needs to access the token server, which can easily become a bottleneck for the overall load. More importantly, the latency and performance of the interface will also decrease.

[0027] If a cluster mode is used, and multiple tokens are obtained with each request to the token server, it can avoid requesting the token server for every business request. The biggest problem with this method is how many tokens to obtain from Redis for each request in a batch mode with different resources and different business request volumes. Setting the batch token number too large will reduce the accuracy of rate limiting for businesses and resources with small request volumes; setting the batch too small will cause performance loss. At the same time, from a practical point of view, relying solely on human experience to set the number of tokens cannot guarantee maximum benefits and is not suitable for enterprise-level application production environments with many resources and many businesses.

[0028] One of the core concepts of this invention is that by obtaining the rate limiting loss rate, the application server can determine its own token request strategy based on an acceptable token loss ratio in the cluster environment, thereby providing a basis for determining the subsequent token quantity. By calculating the number of tokens to be requested based on the rate limiting loss rate when the token request conditions are met, and then sending a request to the token server according to the calculated number of tokens, the network overhead and token server pressure caused by frequent requests are reduced, ensuring the accuracy of rate limiting. This allows the cluster to balance performance and stability in high-concurrency scenarios, achieving a balance between rate limiting accuracy and system throughput.

[0029] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0030] like Figure 1The diagram illustrates a flowchart of a token acquisition method for a cluster according to an embodiment of the present invention, applied to an application server in a cluster. The method may include: Step 101, obtain the rate limiting loss rate; the rate limiting loss rate is the proportion of the number of tokens that the application server in the cluster is allowed to lose when it makes a batch request for tokens from the token server in the cluster, relative to the maximum number of tokens that the token server is allowed to issue.

[0031] In this embodiment of the invention, the method can be applied to various computer application scenarios that adopt a distributed cluster architecture and require precise rate limiting, such as microservice clusters of large Internet platforms, distributed API gateways, e-commerce flash sale systems, online payment systems, high-concurrency database access portals, and cloud-native application services.

[0032] In these scenarios, due to the massive and fluctuating volume of user requests, traditional single-machine rate limiting methods cannot meet the needs of overall cluster traffic control. Meanwhile, cluster rate limiting schemes that uniformly issue tokens through a token server face the problem of difficulty in reasonably setting the number of token requests. If the number of tokens requested each time is too small, it will cause the application server to frequently interact with the token server, increasing network overhead and server pressure, and affecting system throughput. If the number of requests is too large, it may lead to token waste, reduce the accuracy of rate limiting, and even cause cluster traffic to spiral out of control. This solution introduces a rate limiting loss rate on the application server side and dynamically determines the number of tokens to be requested based on this loss rate when the token request conditions are met. This allows the application server to adaptively adjust the scale of token requests according to the current cluster traffic and the load of the token server, thereby reducing unnecessary token requests while ensuring overall rate limiting accuracy, improving system response speed and resource utilization. It is particularly suitable for deployment in critical business systems with extremely high requirements for stability, real-time performance, and rate limiting accuracy.

[0033] The cluster can include multiple application servers and token servers, and the application servers can communicate with the token server.

[0034] Rate limiting loss rate refers to the ratio of the number of tokens that the system is allowed to lose to the maximum number of tokens that the token server is allowed to issue during the process of application servers making batch requests for tokens from the token server. This ratio is preset by the user according to business needs and system performance requirements, and can be used to measure the acceptable level of token waste during cluster rate limiting. The value of the token loss rate is between 0 and 1.

[0035] By introducing a rate limiting loss rate, the application server can reasonably control the scale of token requests while ensuring the overall accuracy of rate limiting, thus providing the necessary basis for the subsequent dynamic calculation of the number of token requests. The acquisition of this rate limiting loss rate can provide objective and quantitative decision support for subsequent steps, avoiding the subjective defects of traditional solutions that rely on human experience.

[0036] Step 102: If the token request conditions are met, determine the number of tokens to be requested based on the rate limiting loss rate.

[0037] In this embodiment of the invention, after obtaining the rate limiting loss rate, the application server can determine whether the token request conditions are currently met.

[0038] Token request conditions typically include whether the number of remaining tokens on the application server is below a threshold, whether the system is under high load, and whether additional tokens are needed to cope with upcoming request peaks. When any of these conditions are met, the application server can determine the number of tokens to request from the token server based on the rate limiting loss rate it has already obtained. Specifically, the application server can combine the maximum number of tokens allowed by the token server and the rate limiting loss rate to calculate a request quantity that meets business processing needs without causing excessive token waste. In this way, the application server can avoid the problem of insufficient or excessive token requests caused by a fixed number of requests, and achieve dynamic adjustment of the token request scale.

[0039] In one example, the token request condition is when the client sends a data request to the application server, and the application server has run out of tokens. At this point, the number of tokens that need to be requested can be determined based on the rate limiting loss rate.

[0040] Step 103: Request tokens from the cluster's token server as needed.

[0041] In this embodiment of the invention, after determining the number of tokens that need to be requested, the application server can send a token request to the token server in the cluster according to the number of tokens and obtain the corresponding number of tokens.

[0042] After receiving a request, the token server can allocate a corresponding number of tokens to the application server based on its current token generation status and the overall rate limiting policy of the cluster. After obtaining the tokens, the application server can store them in a local token cache pool for rate limiting control when processing user requests later. If the tokens in the token server have been exhausted, the application server can wait for the token server to regenerate tokens at a predetermined rate before resuming the normal token acquisition process, thereby ensuring the stability of the cluster rate limiting and the availability of the overall service.

[0043] By requesting tokens according to a dynamically calculated number, the application server can reduce the frequency of interaction with the token server, reduce network overhead and the load on the token server, and ensure that the number of local tokens is always kept within a reasonable range, thereby improving system performance while ensuring the accuracy of cluster rate limiting.

[0044] This invention discloses a method for obtaining tokens in a cluster. By obtaining the rate limiting loss rate, the application server can determine its own token request strategy based on an acceptable token loss ratio in the cluster environment, thereby providing a basis for determining the number of tokens subsequently. By calculating the number of tokens to be requested based on the rate limiting loss rate when the token request conditions are met, and then sending a request to the token server according to the calculated number of tokens, the network overhead and token server pressure caused by frequent requests are reduced, ensuring the accuracy of rate limiting. This allows the cluster to balance performance and stability in high-concurrency scenarios, achieving a balance between rate limiting accuracy and system throughput.

[0045] In one embodiment of the present invention, determining the number of tokens to be requested based on the rate limiting loss rate includes: obtaining the number of application servers deployed and the maximum number of tokens allowed to be issued by the token server; and determining the number of tokens to be requested based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the number of deployments.

[0046] In this embodiment of the invention, the number of application servers deployed refers to the total number of application server nodes that are currently running in the entire cluster and can handle user requests; the maximum number of tokens that the token server is allowed to issue refers to the total number of tokens that the token server can generate and distribute to all application servers per unit time. It is the core benchmark for cluster rate limiting and determines the maximum number of requests that the cluster as a whole can bear.

[0047] In determining the number of tokens required based on the rate limiting loss rate, when the number of application servers deployed in the cluster is known, the number of application server nodes deployed in the cluster can be obtained first, and then the maximum number of tokens allowed to be issued by the token server can be determined.

[0048] Subsequently, the application server can determine the number of tokens to be requested this time by combining the already obtained rate limiting loss rate, the maximum number of tokens allowed to be issued by the token server, and the number of deployments. Specifically, the application server can calculate the total amount of tokens that can be lost in this batch request based on the maximum number of tokens allowed to be issued by the token server and the rate limiting loss rate, and then, based on the number of application servers deployed in the cluster, reasonably distribute this amount of tokens that can be lost to each application server.

[0049] In this way, each application server can calculate a number of token requests that will not lead to excessive token waste while meeting its own business processing needs, based on the cluster size and the system's allowable loss ratio. This calculation method allows the number of token requests to change dynamically with changes in cluster size and adjustments to the rate limiting loss rate, thereby reducing unnecessary token requests, lowering the load on the token server, and improving the performance and stability of the entire cluster while ensuring the accuracy of cluster rate limiting.

[0050] In one embodiment of the present invention, determining the number of tokens to be requested based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the number of deployments includes: determining the number of tokens to be requested in the following manner: N = LossRate * U / (M - 1) + 1 Formula (1) Where N is the number of tokens required, LossRate is the rate limiting loss rate, U is the maximum number of tokens allowed to be issued by the token server, and M is the number of deployments.

[0051] In this embodiment of the invention, when the number of tokens deployed in the cluster can be determined, the rate limiting loss rate can be defined as: when one application server in the cluster cannot request a token, and the remaining application servers in the cluster each request N tokens but only use 1 token, the proportion of the number of tokens wasted by the application servers in the cluster relative to the maximum number of tokens allowed to be issued by the token server.

[0052] For example, if the number of application servers deployed in the cluster is set to M, then the number of tokens wasted by M application servers is (M-1)*(N-1); then the rate limiting loss rate LossRate = (M-1)*(N-1) / U; at this time, we can deduce N = LossRate*U / (M-1)+1, and the user can set the value of the rate limiting loss rate according to the needs, that is, calculate the number of tokens to be requested based on the rate limiting loss rate that the user can accept.

[0053] In determining the number of tokens to be requested, three key parameters—rate limiting loss rate, rate limiting threshold, and number of deployments—can be incorporated into the above formula (1). The specific execution logic is as follows: The application server first obtains the specific values ​​of each parameter in the formula, where LossRate is the rate limiting loss rate set by the user, and its value ranges from 0 to 1; U represents the maximum number of request tokens allowed to be issued per unit time under this dimension, which is the core benchmark for ensuring the accuracy of rate limiting; M is the total number of application server nodes currently deployed in the cluster, which is the key basis for achieving balanced distribution of tokens among multiple nodes.

[0054] Based on this, the application server can calculate the number of tokens to be requested according to the formula (1) above.

[0055] In one example, suppose an e-commerce platform's order payment business cluster has a token rate limiting threshold of U=1000 for the payment interface, and the number of application servers deployed in the cluster is M=6. The user-defined rate limiting loss rate for each application server is LossRate=0.2. Substituting the rate limiting loss rate parameter into the formula, we get N=(0.2*1000) / (6-1)+1=41. That is, the application server should request 41 tokens from the token server this time. The number of tokens calculated by this formula ensures the basic... The basic token supply avoids waste caused by excessively large batches or frequent requests caused by excessively small batches. If the rate limiting loss rate set by the user increases to 0.5, while other parameters remain unchanged, then N = (0.5 * 1000) / 5 + 1 = 101. At this time, due to the increase in the loss rate, the number of token requests calculated by the formula increases accordingly, adapting to the increased demand of business requests. If the loss rate decreases to 0.1, then N = 0.1 * 1000 / 5 + 1 = 21, and the number of token requests decreases accordingly, effectively reducing the probability of tokens being idle and wasted.

[0056] This invention links the rate limiting loss rate, rate limiting threshold, and deployment quantity through mathematical formulas, realizing the scientific and standardized calculation of the number of token requests. It gets rid of the limitations of manual experience configuration, ensuring the adaptability of token allocation to cluster size and business needs, and effectively balancing the contradiction between rate limiting accuracy and system performance.

[0057] In one embodiment of the present invention, determining the number of tokens to be requested based on the rate limiting loss rate includes: determining the actual number of data requests made by the application server per unit time; and determining the number of tokens to be requested based on the rate limiting loss rate and the actual number of data requests made by the application server per unit time.

[0058] In this embodiment of the invention, when determining the number of tokens to be requested based on the rate limiting loss rate, in scenarios where the number of application servers deployed cannot be obtained, dynamic calculation can be performed based on the application server's own business request load as the core criterion.

[0059] The specific execution process is as follows: The application server first starts the local business request statistics module to accurately count the number of actual data requests received per unit time (usually per second). This value directly reflects the real-time business processing pressure of the current server and the actual token consumption demand. Then, it retrieves the rate limiting loss rate previously set by the user for the application server, which can quantify the matching degree between the previous token batch acquisition strategy and the actual demand. Based on this, the application server incorporates the number of actual data requests per unit time and the rate limiting loss rate into the correlation calculation model. Then, based on the rate limiting loss rate and the number of actual data requests per unit time of the application server, it determines the number of tokens that the application server is allowed to waste. Finally, by adding the minimum number of tokens that should be requested by 1 to the number of tokens that the application server is allowed to waste, the number of tokens that need to be requested can be obtained.

[0060] This invention uses the actual number of data requests per unit time from the application server instead of the number of deployments as the core reference indicator, and dynamically adjusts the number of token requests in combination with the rate limiting loss rate. Even when the number of deployments cannot be obtained, the effectiveness of the token request strategy can still be guaranteed. This avoids the problems of token waste and decreased rate limiting accuracy, while also reducing the access frequency of the token server and ensuring the stability of system performance.

[0061] In one embodiment of the present invention, determining the number of tokens to be requested based on the rate limiting loss rate and the actual number of data requests per unit time of the application server includes: determining the number of tokens to be requested in the following manner: N=LossRate*q / w+1 formula (2) Where N is the number of tokens required, LossRate is the rate limiting loss rate, q is the number of data requests, w is the preset time window, and q / w is the actual number of data requests made by the application server per unit time.

[0062] In this embodiment of the invention, in the process of determining the number of tokens to be requested based on formula (2), the real-time business request load of the application server can be used as the core basis, and a quantitative calculation model can be constructed in combination with the rate limiting loss rate.

[0063] The specific execution logic is as follows: The application server first obtains the specific values ​​of each parameter in the formula. Among them, the rate limiting loss rate LossRate is preset by the user and directly reflects the utilization efficiency of token resources; q is the total number of actual data requests received by the application server within the preset time window w, and w is the preset statistical time length, usually in seconds. The ratio of the two, q / w, is the number of actual data requests of the application server per unit time. This value represents the real-time business processing pressure of the current server and the actual token consumption demand.

[0064] Based on this, the application server can substitute the above parameters into formula (2) to obtain the number of tokens to be requested. The entire calculation process does not need to rely on the global parameter of the number of cluster deployments. It can be completed only by the real-time business statistics and historical token utilization efficiency data of the local server, which effectively avoids the problem of token request strategy failure caused by failure to obtain the number of deployments.

[0065] In one example, suppose the application server for the course on-demand service of an online education platform cannot accurately obtain the number of deployed nodes due to dynamic expansion of the cluster nodes. The server's statistical time window is set to w=10 seconds, and the total number of actual course on-demand data requests counted within this time window is q=500. The user-preset rate limiting loss rate for the application server is LossRate=0.3. Substituting the parameters into the formula, we get N=0.3*500 / 10+1=16. That is, the application server should request 16 tokens from the token server this time. In this calculation process, the actual number of data requests per second is 50. Combined with the 30% rate limiting loss rate, the formula calculates 16 tokens. The system makes reasonable adjustments based on real-time business load and ensures the supply of basic tokens by adding 1, avoiding token waste caused by excessively large batches or frequent requests caused by excessively small batches. If the user access volume of the business surges and the total number of requests q in the same time window rises to 1000, the rate limiting loss rate drops to 0.1, then N=0.1*1000 / 10+1=11. The number of token requests is adjusted to adapt to the increase in the number of requests per second and the decrease in the loss rate, which meets the needs of business growth and reduces token idleness. If the total number of requests drops to 100 and the loss rate rises to 0.5, then N=0.5*100 / 10+1=6, and the number of token requests is reduced accordingly, effectively reducing token waste.

[0066] This invention correlates the rate limiting loss rate with the actual number of data requests per second of the application server, enabling the calculation of the number of token requests in scenarios where the number of deployments cannot be obtained. This eliminates the limitations of manual experience-based configuration, ensures the adaptability of token allocation to real-time business needs, and effectively balances the contradiction between rate limiting accuracy and system performance.

[0067] In one embodiment of the present invention, the token request condition is that the application server has 0 remaining tokens.

[0068] In this embodiment of the invention, during the process of determining whether to trigger a token request operation, the token request condition can be that the application server has 0 remaining tokens.

[0069] In this embodiment of the invention, each application server in the cluster can maintain a token cache pool locally to store tokens obtained in batches from the token server, and monitor the number of remaining tokens in the cache pool in real time. The number of remaining tokens directly determines the upper limit of business requests that the current server can handle.

[0070] Another token request condition is when a user terminal sends a data request to any application server in the cluster, and the application server has run out of tokens. In this case, the application server can calculate the number of tokens that need to be requested based on the set rate limiting loss rate.

[0071] In practical distributed cluster rate limiting scenarios, when a user terminal sends a data request to an application server and the application server runs out of local tokens, a token request condition can be triggered. When the number of application servers deployed is determined, the application server can calculate the required number of tokens by combining the specific rate limiting loss rate, the maximum number of tokens allowed by the token server, and the number of application servers deployed. This ensures a balance between rate limiting accuracy and system performance. Taking the product details page access scenario of an e-commerce platform's "618" promotion as an example, assuming 11 application servers are deployed in the cluster, the maximum number of tokens allowed by the token server per unit time... The number of large tokens is 1000, and the user's pre-set rate limit loss rate is 10%. During the promotion, a large number of user terminals simultaneously send product detail page access requests to various application servers in the cluster. One application server has exhausted all the locally cached tokens due to continuous processing of requests. At this time, new user requests continue to pour in, and the application server triggers the token request condition. Subsequently, the application server begins to calculate the number of request tokens based on specific values ​​and formula (1), which is 11. While ensuring the overall rate limit accuracy of the cluster, it efficiently processes user access requests, avoids the detail page loading lag, and improves the user experience during the promotion.

[0072] It should be noted that, in addition to meeting the token request conditions, it is also necessary to ensure that the token server has not exhausted its tokens in order to obtain a token. If the token server has exhausted its tokens, then the request for tokens from the token server will be stopped.

[0073] In one embodiment of the present invention, the application server includes multiple interfaces. Obtaining the rate limiting loss rate includes: obtaining the rate limiting loss rate of each of the multiple interfaces respectively; and determining the number of tokens to be requested based on the rate limiting loss rate when the token request condition is met includes: determining the number of tokens to be requested for each interface based on the rate limiting loss rate of each of the multiple interfaces when the token request condition is met.

[0074] In this embodiment of the invention, in a scenario where the application server contains multiple interfaces, the user can first set the corresponding rate limiting loss rate for each of its own business interfaces. The rate limiting loss rate for each interface is defined as the ratio of the number of tokens allowed by that interface within the statistical period to the total number of tokens allowed to be issued by the token server.

[0075] Subsequently, the application server can monitor whether the token request conditions for each interface are met. The token request conditions here can be that the number of remaining tokens in the local cache pool of each interface is less than a preset threshold. The preset threshold needs to be set differently according to the characteristics of different interfaces, such as the business request volume and response priority. When an interface meets the token request conditions, the application server can retrieve the independent rate limiting loss rate of that interface and calculate the number of tokens that the interface needs to request separately, in conjunction with the corresponding calculation logic. Other interfaces that do not meet the request conditions will not trigger the token request process. At the same time, if multiple interfaces meet the token request conditions at the same time, the application server can initiate token requests for each interface separately, or merge the token requests of multiple interfaces and send them to the token server, thereby further reducing the frequency of interaction with the token server.

[0076] Suppose an e-commerce platform's order application server includes three interfaces: order creation, order query, and order cancellation. Technical staff set token remaining thresholds of 30, 20, and 15 for these interfaces, respectively. Statistics show that the rate limiting loss rate for the order creation interface is 0.2, for the order query interface it's 0.4, and for the order cancellation interface it's 0.1. At a certain moment, the local remaining tokens for the three interfaces are 25, 18, and 20, respectively. At this time, the remaining tokens for the order creation and order query interfaces are both below their respective thresholds, satisfying the token request condition. The order cancellation interface, however, does not meet this condition. The application server then retrieves the independent loss rates for these two interfaces and, combined with the server's actual request count per second (assuming q / w = 100 for the order creation interface and q / w = 80 for the order query interface), calculates the number of tokens required for the order creation interface as N = 0.2 * 100 + 1 = 21, and the number of tokens required for the order query interface as N = 0.4 * 80 + 1 = 33. Based on these results, a request is sent to the token server, while the order cancellation interface continues to process business using its local remaining tokens.

[0077] This invention achieves fine-grained token control at the interface level by calculating independent rate limiting loss rates for multiple interfaces of the application server and determining the number of token requests for each interface as needed. This avoids mutual crowding of token resources between different interfaces and ensures a balance between rate limiting accuracy and performance for each interface, thereby improving the adaptability of the cluster rate limiting solution to complex business scenarios with multiple interfaces.

[0078] like Figure 2This diagram illustrates a communication between a client 20 and a distributed cluster 30 according to an embodiment of the present invention. The distributed cluster 30 may include multiple application servers, namely 301, ..., 30n. The user can pre-configure the rate limiting loss rate of each application server. The distributed cluster 30 is connected to a token server 40. For example, the client 20 may send a data request to the application server 301. The application server 301 determines whether a token exists. If the application server 301 does not have a token, it can determine the number of tokens to be requested based on the pre-set rate limiting loss rate, and then request tokens from the token server 40 according to the number of tokens. In one example, if the number of tokens to be requested is 6, and the token server 40 has no tokens left, the data request can be blocked. If the token server 40 returns 6 tokens, one token can be used to allow the data request, and the remaining tokens are stored in the application server 301.

[0079] like Figure 3 The diagram illustrates a method for obtaining tokens in a cluster according to an embodiment of the present invention. A client can send a data request to an application server in the cluster. The application server determines whether the token request conditions are met. If the conditions are met, the application server determines the number of tokens based on the rate limiting loss rate and then requests tokens from the token server based on the number of tokens. If the token server has the corresponding number of tokens, it sends the corresponding number of tokens back to the application server. After receiving the tokens, the application server can consume one token to allow the data request and store the remaining tokens. If the token server has run out of tokens, the application server will not allow the data request.

[0080] This invention discloses a method for obtaining tokens in a cluster. By obtaining the rate limiting loss rate, the application server can determine its own token request strategy based on an acceptable token loss ratio in the cluster environment, thereby providing a basis for determining the number of tokens subsequently. By calculating the number of tokens to be requested based on the rate limiting loss rate when the token request conditions are met, and then sending a request to the token server according to the calculated number of tokens, the network overhead and token server pressure caused by frequent requests are reduced, ensuring the accuracy of rate limiting. This allows the cluster to balance performance and stability in high-concurrency scenarios, achieving a balance between rate limiting accuracy and system throughput.

[0081] like Figure 4 The diagram illustrates a structural block diagram of a token acquisition device for a cluster, provided by an embodiment of the present invention. This device is applied to an application server within a cluster and may include: The acquisition module 501 is used to acquire the rate limiting loss rate, which is the proportion of the number of tokens that the application servers in the cluster are allowed to lose when they request tokens in batches from the token server in the cluster, relative to the maximum number of tokens that the token server is allowed to issue. The determination module 502 is used to determine the number of tokens to be requested based on the rate limiting loss rate, provided that the token request conditions are met. Request module 503 is used to request tokens from the token server in the cluster based on the number of tokens.

[0082] This invention discloses a token acquisition device for a cluster. By acquiring the rate limiting loss rate, the application server can determine its own token request strategy based on an acceptable token loss ratio in the cluster environment, thereby providing a basis for determining the subsequent token quantity. By calculating the number of tokens to be requested based on the rate limiting loss rate when the token request conditions are met, and then sending a request to the token server according to the calculated number of tokens, the network overhead and token server pressure caused by frequent requests are reduced, ensuring the accuracy of rate limiting. This allows the cluster to balance performance and stability in high-concurrency scenarios, achieving a balance between rate limiting accuracy and system throughput.

[0083] In one embodiment of the present invention, the determining module includes: The first submodule is used to obtain the number of application servers deployed and the maximum number of tokens that the token server is allowed to issue; The first determining submodule is used to determine the number of tokens to be requested based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the number of deployments.

[0084] In one embodiment of the present invention, the number of tokens to be requested can be determined in the following manner: N = LossRate * U / (M - 1) + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, U is the maximum number of tokens allowed to be issued by the token server, and M is the number of deployments.

[0085] In one embodiment of the present invention, the determining module includes: The second determination submodule is used to determine the actual number of data requests made by the application server per unit time. The third determination submodule is used to determine the number of tokens that need to be requested based on the rate limiting loss rate and the actual number of data requests per unit time of the application server.

[0086] In one embodiment of the present invention, determining the number of tokens to be requested based on the rate limiting loss rate and the actual number of data requests per unit time of the application server includes: The number of tokens to be requested is determined as follows: N = LossRate * q / w + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, q is the number of data requests, w is the preset time window, and q / w is the actual number of data requests made by the application server per unit time.

[0087] In one embodiment of the present invention, the token request condition is that the number of remaining tokens on the application server is less than a preset threshold.

[0088] In one embodiment of the present invention, the application server includes multiple interfaces, and the acquisition module includes: The acquisition submodule is used to obtain the rate limiting loss rate of multiple interfaces respectively; The module to be determined includes: The fourth determination submodule is used to determine the number of tokens that each interface needs to request based on the rate limiting loss rate of multiple interfaces, provided that the token request conditions are met.

[0089] This invention discloses a token acquisition device for a cluster. By acquiring the rate limiting loss rate, the application server can determine its own token request strategy based on an acceptable token loss ratio in the cluster environment, thereby providing a basis for determining the subsequent token quantity. By calculating the number of tokens to be requested based on the rate limiting loss rate when the token request conditions are met, and then sending a request to the token server according to the calculated number of tokens, the network overhead and token server pressure caused by frequent requests are reduced, ensuring the accuracy of rate limiting. This allows the cluster to balance performance and stability in high-concurrency scenarios, achieving a balance between rate limiting accuracy and system throughput.

[0090] This invention also provides an electronic device, such as... Figure 5 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604. Memory 603 is used to store computer programs; When the processor 601 executes the program stored in the memory 603, it implements the above-described method steps for obtaining the token of the cluster.

[0091] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0092] The communication interface is used for communication between the aforementioned terminal and other devices.

[0093] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0094] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0095] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the steps of the token acquisition method for any of the clusters described in the above embodiments.

[0096] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0098] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for obtaining a token of a cluster, characterized in that, The method, applied to the application server of the cluster, includes: Obtain the rate limiting loss rate; the rate limiting loss rate is the proportion of the number of tokens that the application servers in the cluster are allowed to lose when they request tokens in batches from the token server in the cluster, relative to the maximum number of tokens that the token server is allowed to issue. If the token request conditions are met, the number of tokens to be requested is determined based on the rate limiting loss rate. Request tokens from the token server of the cluster according to the required number of tokens.

2. The token acquisition method of a cluster according to claim 1, wherein, Determining the number of tokens to be requested based on the rate limiting loss rate includes: Obtain the number of application servers deployed and the maximum number of tokens that the token server is allowed to issue; The number of tokens to be requested is determined based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the number of deployments.

3. The token acquisition method of a cluster according to claim 2, wherein, The step of determining the number of tokens to be requested based on the maximum number of tokens allowed to be issued by the token server, the rate limiting loss rate, and the deployment quantity includes: The number of tokens to be requested is determined as follows: N = LossRate * U / (M - 1) + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, U is the maximum number of tokens allowed to be issued by the token server, and M is the number of deployments.

4. The method for obtaining a token for a cluster according to claim 1, characterized in that, Determining the number of tokens to be requested based on the rate limiting loss rate includes: Determine the actual number of data requests made by the application server per unit time; The number of tokens to be requested is determined based on the rate limiting loss rate and the actual number of data requests per unit time of the application server.

5. The method for obtaining a token for a cluster according to claim 4, characterized in that, The step of determining the number of tokens to be requested based on the rate limiting loss rate and the actual number of data requests per unit time of the application server includes: The number of tokens to be requested is determined as follows: N = LossRate * q / w + 1; Where N is the number of tokens required, LossRate is the rate limiting loss rate, q is the number of data requests, w is the preset time window, and q / w is the actual number of data requests made by the application server per unit time.

6. The method for obtaining a token for a cluster according to claim 1, characterized in that, The application server includes multiple interfaces, and obtaining the rate limiting loss rate includes: Obtain the rate limiting loss rate for each of the multiple interfaces; The step of determining the number of tokens to be requested based on the rate limiting loss rate, under the condition that the token request conditions are met, includes: If the token request conditions are met, the number of tokens required for each interface is determined based on the rate limiting loss rate of the multiple interfaces.

7. The method for obtaining a token for a cluster according to claim 1, characterized in that, The token request condition is that the application server has 0 tokens remaining.

8. A token acquisition device for a cluster, characterized in that, The device is used in the application server of the cluster and includes: The acquisition module is used to acquire the rate limiting loss rate, which is the ratio of the number of tokens that the application servers in the cluster are allowed to lose when they request tokens in batches from the token server in the cluster to the maximum number of tokens that the token server is allowed to issue. The determination module is used to determine the number of tokens to be requested based on the rate limiting loss rate, provided that the token request conditions are met. The request module is used to request tokens from the token server of the cluster based on the number of tokens.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the token acquisition method for the cluster according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the token acquisition method for the cluster as described in any one of claims 1-7.

Images