Distributed current limiting method and system, electronic equipment and storage medium

By setting up a token bucket for each service instance and combining a load balancing algorithm to dynamically adjust the current limiting parameters, the problem of traffic control complexity and performance bottlenecks in distributed systems is solved, and efficient traffic management and system stability are achieved.

CN119697120APending Publication Date: 2025-03-25TONGCHENG NETWORK TECH CO LTD
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411685807.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-23
Publication Date
2025-03-25

AI Technical Summary

Technical Problem

In distributed systems, as business traffic increases, the system may face the problems of request response timeout and service unavailability. The existing distributed current limiting method is complex and has high performance requirements, and lacks simple and efficient solutions.

Method used

By setting up a token bucket for each service instance and determining the initial token number based on the maximum number of concurrent requests of the service instance, combining the load balancing algorithm to intelligently distribute requests, the token bucket status information in the distributed cache is dynamically adjusted.

Benefits of technology

Effectively manage the request processing capabilities of each service instance, optimize resource usage, improve system stability and throughput, and ensure that the system maintains response speed and stability in high concurrency scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119697120A_ABST
    Figure CN119697120A_ABST
Patent Text Reader

Abstract

The invention discloses a distributed flow limiting method and system, electronic equipment and a storage medium, and relates to the field of flow control. In the method, the initial token number of a token bucket is determined according to the maximum concurrent request number of service instances, the token bucket of the service instances is created according to the initial token number, and requests from a client are distributed to each currently started first target service instance according to a load balancing algorithm. Detecting whether a token can be obtained from an associated token bucket of the first target service instance; when the token can be obtained from the associated token bucket, the request is processed through the first target service instance, the number of tokens in the associated token bucket is updated after processing is completed, and the updated state information of the associated token bucket is synchronized to a distributed cache; and adjusting the current limiting parameter according to the state information of all token buckets in the distributed cache. By implementing the technical scheme provided by the invention, the intelligent control of the flow is realized, and the stability and the high efficiency of the system are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of flow control, and in particular to a distributed flow limiting method, system, electronic device and storage medium. Background Art

[0002] In a distributed system, as business traffic continues to increase, the system's service capabilities will face greater challenges, which may lead to request response timeouts, service unavailability and other problems, thereby reducing system performance and stability. In order to ensure the stability and availability of the system, it is usually necessary to adopt current limiting measures to prevent excessive requests from causing system paralysis.

[0003] In the existing technology, some distributed coordination services are used for current limiting, such as Zookeeper or etcd, but this solution is complex to implement and has high requirements on the performance of ZooKeeper. If ZooKeeper cannot handle a large number of token application and release operations, it may become a bottleneck of the system.

[0004] Therefore, a simple and efficient distributed current limiting method is urgently needed. Summary of the invention

[0005] The present application provides a distributed current limiting method, system, electronic device and storage medium, which optimizes resource utilization, can intelligently control traffic, and improve system stability.

[0006] In a first aspect of the present application, a distributed current limiting method is provided, which is applied to a distributed current limiting platform. The method includes: In response to the service instance being started, determining an initial number of tokens in a token bucket according to a maximum number of concurrent requests of the service instance, and creating a token bucket for the service instance according to the initial number of tokens; In response to receiving a request from a client, distributing the request to each currently started first target service instance according to a load balancing algorithm, and detecting whether a token can be obtained from a token bucket associated with the first target service instance; When a token can be obtained from the associated token bucket, the request is processed by the first target service instance, the number of tokens in the associated token bucket is updated after the processing is completed, and the updated state information of the associated token bucket is synchronized to the distributed cache; The current limiting parameters are adjusted according to the status information of all token buckets in the distributed cache, wherein the current limiting parameters include the maximum number of requests that can be processed per second by each service instance, the number of tokens consumed by each request, the capacity of each token bucket, and the token generation rate.

[0007] Optionally, distributing the request to each currently started first target service instance according to a load balancing algorithm includes: Parsing the request to obtain identification information of the required service, and determining all started second target service instances that can process the request according to the identification information; Evaluate the current load status of the second target service instance according to a preset load balancing algorithm, wherein the current load status includes CPU usage, memory occupancy, the number of requests currently being processed, and the number of remaining tokens in the token bucket; The weight value of the second target service instance is determined according to the current load status, the second target service instance with the highest weight value is selected as the first target service instance, and the request is distributed to the first target service instance.

[0008] Optionally, the method further includes adjusting the weight value of each service instance according to the status information of all token buckets in the distributed cache, specifically including: Calculate the average value of the target load index of each service instance within the first preset time, calculate the standard deviation of the target load index of all service instances, and determine the load imbalance index according to the average value and the standard deviation; The degree of load imbalance is determined based on the comparison between the load imbalance index and a preset threshold, and a weight allocation table corresponding to the load balancing algorithm is updated based on the load imbalance degree, wherein the weight allocation table records the weight value of each service instance.

[0009] Optionally, synchronizing the updated state information of the associated token bucket to the distributed cache includes: Calling the lock acquisition interface provided by the distributed lock service, passing in the target lock key, where the target lock key is a combination of the identifier of the first target service instance and the identifier of the associated token bucket; Calling the write interface provided by the distributed cache, passing in the serialized state information and the cache key corresponding to the target lock key to update the state information of the associated token bucket in the distributed cache; The lock release interface provided by the distributed lock service is called to release the target lock key, and other service instances are notified using a message queue or a broadcast mechanism, where the other service instances are service instances other than the first target service instance.

[0010] Optionally, adjusting the current limiting parameter according to the status information of all token buckets in the distributed cache includes: Traverse the distributed cache to obtain token bucket status information of each service instance, wherein the token bucket status information includes the current number of tokens, token bucket capacity, and token generation rate; Calculate global statistical indicators according to the token bucket status information, wherein the global statistical indicators include an average number of tokens, a maximum / minimum number of tokens, and an average value of a token generation rate; The current limiting parameters are adjusted according to the global statistical indicators.

[0011] Optionally, adjusting the current limiting parameter according to the global statistical indicator includes: When the average number of tokens is lower than a preset low threshold, and the average value of the token generation rate is lower than a preset growth threshold, increasing the capacity of each token bucket and / or increasing the token generation rate; When the average number of tokens is higher than a preset high threshold, and the average value of the token generation rate is also higher than a preset reduction threshold, reducing the capacity of each token bucket and / or reducing the token generation rate; Traffic changes within a second future preset time are predicted based on historical traffic data, and the adjustment value of the capacity of each token bucket and / or the adjustment value of the token generation rate are adjusted based on the traffic changes.

[0012] Optionally, the method further includes: When it is detected that the third target service instance fails or cannot process the request, the third target service instance is removed from the target service instance list; reallocate the request on the third target service instance to the remaining service instances according to the load status and token bucket status of the remaining service instances, the remaining service instances being service instances other than the third target service instance; The third target service instance is continuously monitored by sending a heartbeat packet, and when the third target service instance is restored, an initial weight is assigned to the third target service instance according to a current load condition and a token bucket state.

[0013] In a second aspect of the present application, a distributed current limiting system is provided, including an initial module, a distribution module, an execution module and an adjustment module, wherein: An initial module configured to determine, in response to a service instance startup, an initial number of tokens of a token bucket according to a maximum number of concurrent requests of the service instance, and create a token bucket of the service instance according to the initial number of tokens; a distribution module configured to, in response to receiving a request from a client, distribute the request to each currently started first target service instance according to a load balancing algorithm, and detect whether a token can be obtained from an associated token bucket of the first target service instance; an execution module configured to, when a token can be obtained from the associated token bucket, process the request through the first target service instance, update the number of tokens in the associated token bucket after the processing is completed, and synchronize the updated state information of the associated token bucket to the distributed cache; An adjustment module is configured to adjust the current limiting parameters according to the status information of all token buckets in the distributed cache, wherein the current limiting parameters include the maximum number of requests that each service instance can process per second, the number of tokens consumed by each request, the capacity of each token bucket and the token generation rate.

[0014] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By setting a token bucket for each service instance and determining the initial number of tokens based on the maximum number of concurrent requests of the service instance, the request processing capacity of each service instance can be effectively managed, which ensures that the service instance does not exceed its carrying capacity when processing requests, thereby avoiding resource overload and performance degradation; 2. Combined with the load balancing algorithm, it can intelligently distribute requests to each currently started service instance. This helps optimize resource usage, ensures that all service instances can evenly share the load, and improves the overall system throughput and response speed; 3. By monitoring the status information of all token buckets in the distributed cache, you can understand the request processing and token consumption of each service instance in real time. Based on this information, the system can dynamically adjust the current limiting parameters, such as the maximum number of requests that each service instance can process per second, the number of tokens consumed by each request, the capacity of each token bucket, and the token generation rate. This dynamic adjustment capability helps the system better cope with sudden traffic and traffic changes, and improves the stability and reliability of the system; 4. Since this method uses a distributed architecture and distributed cache to store the status information of the token bucket, it has high availability and scalability. Even if some service instances or cache nodes fail, the system can continue to work and process requests. In addition, as the business develops and traffic increases, new service instances and cache nodes can be easily added to expand the system's processing capacity; 5. Based on the token bucket algorithm, this is a simple and effective current limiting algorithm. At the same time, the distributed cache is used to synchronize the status information of the token bucket, making the entire system easy to implement and maintain. Administrators can easily monitor and manage the current limiting parameters and status information of the system to ensure the normal operation and performance optimization of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the distributed current limiting method disclosed in the embodiment of the present application; Figure 2 It is a schematic diagram of the implementation principle of token bucket current limiting disclosed in the embodiment of the present application; Figure 3 It is a schematic diagram of the architecture of the distributed current limiting method disclosed in the embodiment of the present application; Figure 4 It is a module schematic diagram of a distributed current limiting system disclosed in an embodiment of the present application; Figure 5 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0018] Explanation of the reference numerals: 401, initial module; 402, distribution module; 403, execution module; 404, adjustment module; 501, processor; 502, communication bus; 503, user interface; 504, network interface; 505, memory. DETAILED DESCRIPTION

[0019] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0021] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0022] This embodiment discloses a distributed current limiting method, which is applied to a distributed current limiting platform. Figure 1 is a flow chart of the distributed current limiting method disclosed in the embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: S110, in response to the service instance being started, determining an initial number of tokens of a token bucket according to a maximum number of concurrent requests of the service instance, and creating a token bucket of the service instance according to the initial number of tokens; When a service instance is started, the initial number of tokens in the token bucket is determined based on the maximum number of concurrent requests for the service instance. This maximum number of concurrent requests is usually set based on the hardware resources (such as CPU, memory, etc.) and processing power of the service instance to ensure that the service instance runs without exceeding its carrying capacity. Specifically, the maximum number of concurrent requests for the service instance is obtained from the configuration information or metadata of the service instance. This value is usually set by an administrator or an automated tool when the service instance is deployed or started. The initial number of tokens in the token bucket is calculated based on this maximum number of concurrent requests. This calculation process may take into account some additional factors, such as the token generation rate (that is, the number of tokens added to the token bucket by the system per second), the capacity of the token bucket (that is, the maximum number of tokens that the token bucket can store), etc. However, when the service instance is started, the initial number of tokens in the token bucket is usually set to be close to or equal to the maximum number of concurrent requests to ensure that the service instance can handle a sufficient number of requests at the initial startup. A token bucket is created based on the calculated initial number of tokens and associated with the service instance. This token bucket will be used to store and manage tokens for the service instance to control the flow of requests.

[0023] Figure 2 Schematic diagram of the implementation principle of token bucket current limiting disclosed in the embodiment of the present application, such as Figure 2As shown, "water droplets" represent request tokens, which are generated at a fixed rate. The "token bucket" stores all tokens. If the "token bucket" is full, the excess tokens will be discarded directly. Each request needs to take a token from the "token bucket". Only requests that have obtained tokens can pass. If the "token bucket" is empty, requests that cannot obtain tokens will be discarded.

[0024] S120, in response to receiving a request from a client, distributing the request to each currently started first target service instance according to a load balancing algorithm, and detecting whether a token can be obtained from a token bucket associated with the first target service instance; When clients send requests to the system, these requests are first received by the front-end entrance of the system (such as a load balancer or gateway). The load balancer evaluates the load of each currently started service instance according to the preset load balancing algorithm (such as polling, random, least number of connections, hash, etc.). The purpose of the load balancing algorithm is to select a relatively idle or lightly loaded service instance to handle the request to achieve load balancing of the system. Based on the results of the load balancing algorithm, the system determines one or more first target service instances, and the first target service instance is selected to handle the currently received client request. Before distributing the request to the first target service instance, the system checks whether there are available tokens in the associated token bucket of the service instance. The token bucket is a mechanism for current limiting. It stores a certain number of tokens, each token representing a request that is allowed to be processed. If there are available tokens in the token bucket, the system will take a token from it and distribute the request to the corresponding service instance. If the token is successfully obtained from the token bucket, the first target service instance will receive and process the request. If there are no available tokens in the token bucket, the system may reject the request or put it in a queue for subsequent processing (the specific behavior depends on the design and configuration of the system). After processing the request, the system may update the status of the token bucket based on the configured token generation rate and the capacity of the token bucket, such as adding new tokens to the token bucket.

[0025] Figure 3 Schematic diagram of the architecture of the distributed current limiting method disclosed in the embodiment of the present application, such as Figure 3 As shown in the figure, requests are forwarded to each service instance through load balancing, the synchronization status of the token bucket of each instance is maintained through distributed cache, and different current limiting strategies are adopted according to the configuration.

[0026] Optionally, distributing the request to each currently started first target service instance according to a load balancing algorithm includes: Parsing the request to obtain identification information of the required service, and determining all started second target service instances that can process the request according to the identification information; Evaluate the current load status of the second target service instance according to a preset load balancing algorithm, wherein the current load status includes CPU usage, memory occupancy, the number of requests currently being processed, and the number of remaining tokens in the token bucket; The weight value of the second target service instance is determined according to the current load status, the second target service instance with the highest weight value is selected as the first target service instance, and the request is distributed to the first target service instance.

[0027] When the load balancer receives a request from a client, it will parse the request. The purpose of parsing is to extract the identification information of the required service carried in the request, which is usually achieved through specific fields in the request header, request path, or request parameters. Based on the parsed service identification information, the load balancer will query the service registry or configuration information to determine all the started service instances in the current system that can handle the request, that is, the second target service instances. The load balancer will collect the current load status of each second target service instance periodically or in real time. These load status indicators include but are not limited to CPU usage, memory occupancy, the number of requests currently being processed, and the number of remaining tokens in the token bucket. These indicators together reflect the current processing capacity and potential overload risk of the service instance. According to the preset load balancing algorithm (such as weighted polling, minimum number of connections, etc.), the load balancer will calculate the weight value of each second target service instance based on its current load status. The weight value is usually proportional to the processing capacity of the service instance and inversely proportional to the load status. That is, the service instance with stronger processing capacity and lower load will obtain a higher weight value. After calculating the weight values ​​of all second target service instances, the load balancer selects the service instance with the highest weight value as the first target service instance. If there are multiple service instances with the same weight value, the load balancer may use other strategies (such as random selection, hash selection, etc.) to further determine the target. The load balancer distributes the client's request to the selected first target service instance for processing. Before distributing the request, the load balancer also checks whether there are enough tokens in the token bucket associated with the service instance. If there are insufficient tokens, the request may be processed according to the system's configured strategy (such as waiting for tokens, rejecting requests, etc.).

[0028] By parsing the request and obtaining the identification information of the required service, the system can quickly determine all the started second target service instances that can process the request. This step ensures accurate matching of the request and avoids unnecessary service search and waiting time. Subsequently, the current load status of these service instances is evaluated according to the preset load balancing algorithm, including key indicators such as CPU usage, memory occupancy, number of requests currently being processed, and number of remaining tokens in the token bucket. These indicators can fully reflect the real-time load status of the service instance, thereby providing a strong basis for subsequent request distribution. By selecting the second target service instance with the highest weight value as the first target service instance and distributing the request to the instance, the system can ensure that the request is distributed to the service instance with the strongest processing capability and the lightest load, thereby improving the response speed and efficiency of the service. The core goal of the load balancing algorithm is to evenly distribute requests on each server to avoid performance degradation or failure caused by excessive load on some servers. The above content achieves accurate distribution of requests and rational use of resources by evaluating the load status of service instances in real time and dynamically adjusting the weight value of service instances according to the load status. During this process, service instances with lighter loads will be assigned higher weights, thereby receiving more requests, while service instances with heavier loads will be assigned lower weights, reducing their chances of receiving requests. In this way, the system can ensure that all service instances can operate within a reasonable load range, avoiding single point overload and resource waste.

[0029] Optionally, the method further includes adjusting the weight value of each service instance according to the status information of all token buckets in the distributed cache, specifically including: Calculate the average value of the target load index of each service instance within the first preset time, calculate the standard deviation of the target load index of all service instances, and determine the load imbalance index according to the average value and the standard deviation; The degree of load imbalance is determined based on the comparison between the load imbalance index and a preset threshold, and a weight allocation table corresponding to the load balancing algorithm is updated based on the load imbalance degree, wherein the weight allocation table records the weight value of each service instance.

[0030] Define a series of target load indicators that can fully reflect the load of service instances, such as CPU usage, memory usage, network throughput, disk I / O, etc. Sample these indicators within the first preset time (such as the past one minute, five minutes, or ten minutes, etc.), and calculate the average of these indicators for each service instance. These averages can reflect the average load of the service instance over a period of time. After calculating the average of the target load indicators of all service instances, the system also needs to calculate the standard deviation of these averages. The standard deviation is a statistic that measures the degree of data dispersion, which can reflect the degree of difference between the loads of various service instances. Based on the average and standard deviation, the system can calculate the load imbalance index. This index is an indicator that comprehensively measures the degree of system load imbalance. It is usually obtained by comparing or calculating the load difference (i.e., standard deviation) of each service instance with the average load (i.e., average). The system will compare the calculated load imbalance index with the preset threshold to determine the current degree of load imbalance. This threshold is usually set based on the actual situation and performance requirements of the system. According to the degree of load imbalance, the system will update the weight distribution table corresponding to the load balancing algorithm accordingly. The weight distribution table is a table that records the weight value of each service instance, which determines the probability of each service instance being selected during the load balancing process. If the load imbalance is high, the system may increase the weight value of the service instance with a lighter load and reduce the weight value of the service instance with a heavier load to balance the load of each service instance. Conversely, if the load imbalance is low, the system may keep the weight distribution table unchanged or make fine adjustments.

[0031] By calculating the average value of the target load index of each service instance within the first preset time and calculating the standard deviation of the target load index of all service instances, the system can obtain the current load dispersion, that is, the load imbalance index. This index reflects the difference in load between service instances in the system. Based on this index, the system can more accurately determine the current load imbalance and dynamically adjust the weight distribution table corresponding to the load balancing algorithm. This dynamic adjustment capability enables the system to respond to load changes more flexibly, avoid single-point overload and resource waste, and thus improve the overall performance and stability of the system. Based on the comparison result of the load imbalance index with the preset threshold, the system can determine the current load imbalance and update the weight distribution table accordingly. The update of the weight distribution table means that the weight value of each service instance in the load balancing algorithm will change, which in turn affects the request distribution strategy. By reducing the weight value of the overloaded service instance, reducing its chance of receiving requests; while increasing the weight value of the lightly loaded service instance, increasing its chance of receiving requests, the system can achieve reasonable allocation of requests and optimal utilization of resources. This strategy helps to balance the load of each service instance, improve service efficiency and user satisfaction. By real-time monitoring and analysis of the status information of all token buckets in the distributed cache, the system can promptly detect potential load imbalance problems and take appropriate adjustment measures. This preventive strategy helps to avoid single point failures and overloads, thereby enhancing the stability and reliability of the system. At the same time, the dynamic update of the weight allocation table also means that the system can adapt to changes in load demand in different time periods and business scenarios, ensuring that the system can maintain efficient operation in various situations.

[0032] S130: When a token can be obtained from the associated token bucket, the request is processed by the first target service instance, the number of tokens in the associated token bucket is updated after the processing is completed, and the updated state information of the associated token bucket is synchronized to the distributed cache; After the client's request is distributed to the first target service instance by the load balancer, the first target service instance will try to obtain a token from its associated token bucket. If there is an available token in the token bucket, the service instance will successfully obtain a token and continue to process the request. When the token is successfully obtained, the first target service instance will start to process the client's request. This usually includes steps such as parsing the request data, executing the corresponding business logic, and generating response data. The specific process of processing the request depends on the business function and implementation method of the service instance. After processing the request, the first target service instance needs to update the number of tokens in its associated token bucket. This usually means removing a used token from the token bucket (if the capacity of the token bucket is fixed) or adding a new token to the token bucket according to the system's token generation strategy (if the token bucket supports dynamic token addition). Updating the token bucket status is a key step to ensure that the system's current limiting mechanism operates correctly. In order to maintain consensus on the token bucket status among all service instances in the system, the first target service instance needs to synchronize the updated status information to the distributed cache after updating the token bucket status. Distributed cache is a component for storing and sharing data, which allows multiple service instances in the system to access and update shared data in an efficient manner. By synchronizing the status information of the token bucket to the distributed cache, the system can ensure that all service instances can obtain the latest status information of the token bucket, thereby avoiding the failure of the current limiting strategy or waste of resources due to inconsistent information.

[0033] Optionally, synchronizing the updated state information of the associated token bucket to the distributed cache includes: Calling the lock acquisition interface provided by the distributed lock service, passing in the target lock key, where the target lock key is a combination of the identifier of the first target service instance and the identifier of the associated token bucket; Calling the write interface provided by the distributed cache, passing in the serialized state information and the cache key corresponding to the target lock key to update the state information of the associated token bucket in the distributed cache; The lock release interface provided by the distributed lock service is called to release the target lock key, and other service instances are notified using a message queue or a broadcast mechanism, where the other service instances are service instances other than the first target service instance.

[0034] Before updating the status information of the associated token bucket in the distributed cache, the first target service instance first needs to obtain a distributed lock by calling the acquisition lock interface provided by the distributed lock service. This lock is used to ensure that no other service instance will modify the status information of the same token bucket at the same time during the process of updating the status information, thereby avoiding data competition and inconsistency problems. The target lock key is the key parameter passed in when acquiring the lock, which is usually composed of the identifier of the first target service instance and the identifier of the associated token bucket. This design ensures that each token bucket has a unique lock key, thereby realizing concurrent update control of different token buckets. After successfully acquiring the distributed lock, the first target service instance can start to update the status information of the associated token bucket in the distributed cache. This process usually includes the following steps: serializing the updated status information so that it can be stored in the distributed cache in binary form; calling the write interface provided by the distributed cache, passing in the serialized status information and the cache key corresponding to the target lock key, which is usually associated with the target lock key and is used to locate the status information to be updated in the distributed cache; the distributed cache service will update the corresponding data item according to the passed in cache key and status information. After successfully updating the status information in the distributed cache, the first target service instance needs to call the lock release interface provided by the distributed lock service to release the previously acquired distributed lock. After releasing the lock, other service instances can obtain the lock and continue to perform update operations. In addition, in order to ensure that other service instances can perceive the update of status information in a timely manner, the first target service instance can also use a message queue or broadcast mechanism to notify them. In this way, when other service instances need to read the status information of the associated token bucket, they can obtain the latest data from the distributed cache.

[0035] By using distributed locks, it is ensured that only one service instance can perform operations when updating the token bucket status information, avoiding data inconsistency problems caused by concurrent updates. The use of distributed locks and distributed caches enables the system to tolerate the failure of some service instances. Even if a service instance crashes, other service instances can still obtain the latest token bucket status information through the distributed cache. The use of message queues or broadcast mechanisms can efficiently notify other service instances of updates to status information, reducing the direct communication overhead between service instances and improving the overall performance of the system. The design of distributed locks and distributed caches allows the system to easily add new service instances without modifying the existing state synchronization mechanism, enhancing the scalability of the system. By combining distributed locks with message queues or broadcast mechanisms, the system can efficiently handle status updates and synchronization issues in high-concurrency scenarios, ensuring the stability and response speed of the system.

[0036] S140. Adjust the current limiting parameters according to the status information of all token buckets in the distributed cache, wherein the current limiting parameters include the maximum number of requests that each service instance can process per second, the number of tokens consumed by each request, the capacity of each token bucket, and the token generation rate.

[0037] The maximum number of requests that each service instance can process per second: defines the maximum number of requests that a service instance can process per second. By adjusting this parameter, you can control the load level of the service instance and avoid overload. The number of tokens consumed by each request: This parameter determines the speed at which requests consume the token bucket. By adjusting this parameter, you can control the frequency at which requests are processed. The capacity of each token bucket: The capacity of the token bucket determines the number of tokens it can store. This parameter affects the maximum number of requests that a service instance can process in a period of time. By adjusting this parameter, you can balance the throughput and response time of the system. The token generation rate: The token generation rate determines the growth rate of tokens in the token bucket. This parameter controls the growth rate of the service instance's ability to process requests. By adjusting this parameter, you can adapt to the load requirements in different time periods and business scenarios. Real-time monitoring of the status information of all token buckets in the distributed cache, including the number of tokens in each token bucket, the request processing rate of each service instance, etc. Analyze the current system load based on the monitored status information, including whether the load of each service instance is balanced, whether there are overloaded or idle service instances, etc. Dynamically adjust the current limiting parameters based on the analysis results. For example, if the load of a service instance is too high, you can appropriately reduce the maximum number of requests it can process per second or increase the number of tokens consumed by each request; if the overall system load is low, you can increase the capacity of the token bucket or increase the token generation rate to improve the system throughput. Apply the adjusted current limiting parameters to the system to ensure that all service instances can process requests and update the token bucket according to the new parameters.

[0038] Optionally, adjusting the current limiting parameter according to the status information of all token buckets in the distributed cache includes: Traverse the distributed cache to obtain token bucket status information of each service instance, wherein the token bucket status information includes the current number of tokens, token bucket capacity, and token generation rate; Calculate global statistical indicators according to the token bucket status information, wherein the global statistical indicators include an average number of tokens, a maximum / minimum number of tokens, and an average value of a token generation rate; The current limiting parameters are adjusted according to the global statistical indicators.

[0039] The system needs to traverse the distributed cache to obtain the token bucket status information of each service instance. This information usually includes: Current number of tokens: indicates the number of tokens currently remaining in the token bucket; Token bucket capacity: indicates the maximum number of tokens that the token bucket can store; Token generation rate: Indicates the growth rate of tokens in the token bucket, usually measured in the number of tokens generated per second.

[0040] By obtaining this information, the system can have a comprehensive understanding of the token bucket status of each service instance. After obtaining the token bucket status information of each service instance, the system needs to calculate some global statistical indicators to better understand the load of the entire system and the usage of the token bucket. These global statistical indicators may include: Average number of tokens: indicates the average number of tokens in the token bucket of all service instances. This metric can help the system understand the overall token usage; Maximum / minimum number of tokens: indicates the maximum and minimum number of tokens in the token bucket of all service instances. These indicators can help the system detect potential load imbalance issues; Average token generation rate: indicates the average token generation rate in the token bucket of all service instances. This metric can help the system understand the overall token generation capacity.

[0041] By calculating these global statistical indicators, the system can have a deeper understanding of the load of the entire system and the usage of token buckets. After obtaining the global statistical indicators, the system needs to adjust the current limiting parameters based on these indicators. Current limiting parameters usually include the maximum number of requests that each service instance can process per second, the number of tokens consumed by each request, the capacity of each token bucket, and the token generation rate. The process of adjusting current limiting parameters may involve the following aspects: Increase / decrease token bucket capacity: If the average number of tokens is too low or too high, you may need to adjust the token bucket capacity to adapt to the current load situation.

[0042] Adjust the token generation rate: If the average value of the token generation rate is too low, you may need to increase the token generation rate to improve the system's processing capacity; conversely, if it is too high, you may need to reduce the token generation rate to avoid overload.

[0043] Adjust the number of tokens consumed by each request: Based on the system load and business needs, you can dynamically adjust the number of tokens consumed by each request to control the request processing frequency.

[0044] Adjust the QPS of each service instance: Based on global statistical indicators and the overall load of the system, the maximum number of requests processed per second (QPS) of each service instance can be dynamically adjusted to balance the load.

[0045] By traversing the distributed cache in real time and calculating global statistical indicators, the system can quickly perceive load changes and adjust the current limiting parameters accordingly. This helps the system respond quickly in high-concurrency scenarios and avoid overload and performance degradation. Global statistical indicators provide the system with a comprehensive load view, allowing the system to more accurately evaluate the load capacity and resource requirements of different service instances. Based on this, the system can dynamically adjust the current limiting parameters, optimize resource allocation, and avoid resource waste and bottlenecks. By dynamically adjusting the current limiting parameters, the system can adapt to load requirements in different time periods and business scenarios. This helps the system maintain stable operation in both high-load and low-load scenarios, and improves the reliability and availability of the system. Reasonable current limiting parameter adjustment can ensure that the system can still maintain response speed and stability in high-concurrency scenarios. This helps to improve user experience and enhance user trust and satisfaction with the system.

[0046] Optionally, adjusting the current limiting parameter according to the global statistical indicator includes: When the average number of tokens is lower than a preset low threshold, and the average value of the token generation rate is lower than a preset growth threshold, increasing the capacity of each token bucket and / or increasing the token generation rate; When the average number of tokens is higher than a preset high threshold, and the average value of the token generation rate is also higher than a preset reduction threshold, reducing the capacity of each token bucket and / or reducing the token generation rate; Traffic changes within a second future preset time are predicted based on historical traffic data, and the adjustment value of the capacity of each token bucket and / or the adjustment value of the token generation rate are adjusted based on the traffic changes.

[0047] When the average number of tokens is lower than the preset low threshold and the average value of the token generation rate is lower than the preset growth threshold, the system may face the problem of insufficient request processing capacity because the number of tokens in the token bucket is insufficient and the token generation rate is slow. To deal with this situation, the system can take one or a combination of the following measures: Increase the capacity of each token bucket: This enables the token bucket to store more tokens, allowing the system to handle more requests.

[0048] Increase the token generation rate: This can speed up the growth of tokens in the token bucket, allowing the system to respond to requests faster.

[0049] When the average number of tokens is higher than the preset high threshold and the average value of the token generation rate is also higher than the preset reduction threshold, the system may face the problem of resource waste because there are too many tokens in the token bucket and the token generation rate is also fast. In order to optimize resource utilization, the system can take one or a combination of the following measures: Reduce the capacity of each token bucket: This limits the number of tokens that can be stored in the token bucket, thereby reducing resource waste.

[0050] Reduce the token generation rate: This can slow down the growth of tokens in the token bucket, allowing the system to allocate resources more reasonably.

[0051] The system uses historical traffic data to predict traffic changes in the future. This helps the system prepare in advance to deal with possible traffic peaks or valleys. Based on the prediction results, the system can make further adjustments to the capacity of each token bucket and / or the token generation rate. For example, if it is predicted that future traffic will increase, the system can increase the token bucket capacity or increase the token generation rate to ensure that the system can handle more requests.

[0052] By dynamically adjusting the token bucket capacity and token generation rate, the system can make adaptive adjustments based on the load demand in different time periods, improving the flexibility and response speed of the system. The system dynamically adjusts the flow limiting parameters based on the current load situation and future traffic forecasts to ensure that resources are reasonably allocated and utilized, avoiding resource waste and bottlenecks. Through reasonable adjustment of flow limiting parameters, the system can maintain stable operation in high-concurrency scenarios, avoid overload and performance degradation, and improve the stability and reliability of the system. Reasonable adjustment of flow limiting parameters can ensure that the system can still maintain response speed and stability in high-concurrency scenarios, thereby improving user experience. The introduction of future traffic forecasts based on historical traffic data enables the system to adjust flow limiting parameters more intelligently, reducing operation and maintenance costs and improving operation and maintenance efficiency.

[0053] Optionally, the method further includes: When it is detected that the third target service instance fails or cannot process the request, the third target service instance is removed from the target service instance list; reallocate the request on the third target service instance to the remaining service instances according to the load status and token bucket status of the remaining service instances, the remaining service instances being service instances other than the third target service instance; The third target service instance is continuously monitored by sending a heartbeat packet, and when the third target service instance is restored, an initial weight is assigned to the third target service instance according to a current load condition and a token bucket state.

[0054] When the system detects that the third target service instance fails or cannot process requests, it will first remove it from the target service instance list. This step is to prevent the failed service instance from continuing to receive requests, thereby avoiding possible incorrect processing or system crashes. After removing the failed service instance, the system will reallocate the requests originally processed by the third target service instance based on the load status and token bucket status of the remaining service instances (i.e., service instances other than the third target service instance). This step is to ensure that all requests can be processed in a timely manner while avoiding allocating too many requests to service instances that are already heavily loaded. The system will continuously monitor the status of the third target service instance by sending heartbeat packets and other means. Heartbeat packets are short messages sent periodically to confirm the online status and health of service instances. Through continuous monitoring, the system can promptly discover the recovery of service instances. When the third target service instance is restored, the system will assign it an initial weight based on the current load status and token bucket status. This weight determines the priority and load distribution ratio of the service instance when receiving new requests. By assigning initial weights, the system can gradually reallocate requests to the restored service instances while avoiding excessive impact on the system.

[0055] By promptly removing failed service instances and reallocating requests, the system can ensure that it can still process requests in high-concurrency scenarios, thereby improving system availability. The system reallocates requests based on the load status and token bucket status of the remaining service instances, ensuring even load distribution and avoiding situations where some service instances are overloaded while others are idle, thereby optimizing load balancing. By continuously monitoring failed service instances through heartbeat packets, the system can promptly detect their recovery status and reassign the initial weight after recovery, allowing the system to return to normal more quickly. Timely fault handling, request reallocation, and service instance recovery ensure that the system can still maintain responsiveness and stability in high-concurrency scenarios, thereby improving user experience.

[0056] This embodiment also discloses a distributed current limiting system. Figure 4 is a module schematic diagram of a distributed current limiting system disclosed in an embodiment of the present application, such as Figure 4 As shown, the system includes an initial module 401, a distribution module 402, an execution module 403 and an adjustment module 404, wherein: Initial module 401, configured to determine the initial number of tokens of a token bucket according to the maximum number of concurrent requests of the service instance in response to the startup of the service instance, and create a token bucket of the service instance according to the initial number of tokens; The distribution module 402 is configured to distribute the request to each currently started first target service instance according to the load balancing algorithm in response to receiving the request from the client, and detect whether a token can be obtained from the associated token bucket of the first target service instance; The execution module 403 is configured to process the request through the first target service instance when a token can be obtained from the associated token bucket, update the number of tokens in the associated token bucket after the processing is completed, and synchronize the updated state information of the associated token bucket to the distributed cache; The adjustment module 404 is configured to adjust the current limiting parameters according to the status information of all token buckets in the distributed cache, and the current limiting parameters include the maximum number of requests that each service instance can process per second, the number of tokens consumed by each request, the capacity of each token bucket and the token generation rate.

[0057] Optionally, the distribution module 402 is configured to: Parsing the request to obtain identification information of the required service, and determining all started second target service instances that can process the request according to the identification information; Evaluate the current load status of the second target service instance according to a preset load balancing algorithm, wherein the current load status includes CPU usage, memory occupancy, the number of requests currently being processed, and the number of remaining tokens in the token bucket; The weight value of the second target service instance is determined according to the current load status, the second target service instance with the highest weight value is selected as the first target service instance, and the request is distributed to the first target service instance.

[0058] Optionally, the method further comprises a weight module, wherein the weight module is configured to: Calculate the average value of the target load index of each service instance within the first preset time, calculate the standard deviation of the target load index of all service instances, and determine the load imbalance index according to the average value and the standard deviation; The degree of load imbalance is determined based on the comparison between the load imbalance index and a preset threshold, and a weight allocation table corresponding to the load balancing algorithm is updated based on the load imbalance degree, wherein the weight allocation table records the weight value of each service instance.

[0059] Optionally, the execution module 403 is configured to: Calling the lock acquisition interface provided by the distributed lock service, passing in the target lock key, where the target lock key is a combination of the identifier of the first target service instance and the identifier of the associated token bucket; Calling the write interface provided by the distributed cache, passing in the serialized state information and the cache key corresponding to the target lock key to update the state information of the associated token bucket in the distributed cache; The lock release interface provided by the distributed lock service is called to release the target lock key, and other service instances are notified using a message queue or a broadcast mechanism, where the other service instances are service instances other than the first target service instance.

[0060] Optionally, the adjustment module 404 is configured to: Traverse the distributed cache to obtain token bucket status information of each service instance, wherein the token bucket status information includes the current number of tokens, token bucket capacity, and token generation rate; Calculate global statistical indicators according to the token bucket status information, wherein the global statistical indicators include an average number of tokens, a maximum / minimum number of tokens, and an average value of a token generation rate; The current limiting parameters are adjusted according to the global statistical indicators.

[0061] Optionally, the adjustment module 404 is configured to: When the average number of tokens is lower than a preset low threshold, and the average value of the token generation rate is lower than a preset growth threshold, increasing the capacity of each token bucket and / or increasing the token generation rate; When the average number of tokens is higher than a preset high threshold, and the average value of the token generation rate is also higher than a preset reduction threshold, reducing the capacity of each token bucket and / or reducing the token generation rate; Traffic changes within a second future preset time are predicted based on historical traffic data, and the adjustment value of the capacity of each token bucket and / or the adjustment value of the token generation rate are adjusted based on the traffic changes.

[0062] Optionally, the system further comprises an isolation module, wherein the isolation module is configured to: When it is detected that the third target service instance fails or cannot process the request, the third target service instance is removed from the target service instance list; reallocate the request on the third target service instance to the remaining service instances according to the load status and token bucket status of the remaining service instances, the remaining service instances being service instances other than the third target service instance; The third target service instance is continuously monitored by sending a heartbeat packet, and when the third target service instance is restored, an initial weight is assigned to the third target service instance according to a current load condition and a token bucket state.

[0063] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0064] This embodiment also discloses an electronic device, referring to Figure 5 The electronic device may include: at least one processor 501 , at least one communication bus 502 , a user interface 503 , a network interface 504 , and at least one memory 505 .

[0065] The communication bus 502 is used to realize the connection and communication between these components.

[0066] The user interface 503 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0067] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0068] Among them, the processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a chip.

[0069] Among them, the memory 505 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. As Figure 5 As shown, the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a distributed current limiting method.

[0070] exist Figure 5 In the electronic device shown, the user interface 503 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 501 can be used to call the application program of the distributed current limiting method stored in the memory 505. When executed by one or more processors 501, the electronic device executes one or more methods in the above-mentioned embodiments.

[0071] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0072] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0073] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0074] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0075] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0076] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory 505, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory 505 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0077] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A distributed current limiting method, characterized in that: Applied to a distributed current limiting platform, the method includes: In response to the service instance being started, determining an initial number of tokens in a token bucket according to a maximum number of concurrent requests of the service instance, and creating a token bucket for the service instance according to the initial number of tokens; In response to receiving a request from a client, distributing the request to each currently started first target service instance according to a load balancing algorithm, and detecting whether a token can be obtained from a token bucket associated with the first target service instance; When a token can be obtained from the associated token bucket, the request is processed by the first target service instance, the number of tokens in the associated token bucket is updated after the processing is completed, and the updated state information of the associated token bucket is synchronized to the distributed cache; The current limiting parameters are adjusted according to the status information of all token buckets in the distributed cache, wherein the current limiting parameters include the maximum number of requests that can be processed per second by each service instance, the number of tokens consumed by each request, the capacity of each token bucket, and the token generation rate.

2. The distributed current limiting method according to claim 1, characterized in that: The step of distributing the request to each currently started first target service instance according to a load balancing algorithm includes: Parsing the request to obtain identification information of the required service, and determining all started second target service instances that can process the request according to the identification information; Evaluate the current load status of the second target service instance according to a preset load balancing algorithm, wherein the current load status includes CPU usage, memory occupancy, the number of requests currently being processed, and the number of remaining tokens in the token bucket; The weight value of the second target service instance is determined according to the current load status, the second target service instance with the highest weight value is selected as the first target service instance, and the request is distributed to the first target service instance.

3. The distributed current limiting method according to claim 2, characterized in that: The method further includes adjusting the weight value of each service instance according to the status information of all token buckets in the distributed cache, specifically including: Calculate the average value of the target load index of each service instance within the first preset time, calculate the standard deviation of the target load index of all service instances, and determine the load imbalance index according to the average value and the standard deviation; The degree of load imbalance is determined based on the comparison between the load imbalance index and a preset threshold, and a weight allocation table corresponding to the load balancing algorithm is updated based on the load imbalance degree, wherein the weight allocation table records the weight value of each service instance.

4. The distributed current limiting method according to claim 1, characterized in that: The step of synchronizing the updated status information of the associated token bucket to the distributed cache includes: Calling the lock acquisition interface provided by the distributed lock service, passing in the target lock key, where the target lock key is a combination of the identifier of the first target service instance and the identifier of the associated token bucket; Calling the write interface provided by the distributed cache, passing in the serialized state information and the cache key corresponding to the target lock key to update the state information of the associated token bucket in the distributed cache; The lock release interface provided by the distributed lock service is called to release the target lock key, and other service instances are notified using a message queue or a broadcast mechanism, where the other service instances are service instances other than the first target service instance.

5. The distributed current limiting method according to claim 1, characterized in that: The adjusting the current limiting parameter according to the status information of all token buckets in the distributed cache comprises: Traverse the distributed cache to obtain token bucket status information of each service instance, wherein the token bucket status information includes the current number of tokens, token bucket capacity, and token generation rate; Calculate global statistical indicators according to the token bucket status information, wherein the global statistical indicators include an average number of tokens, a maximum / minimum number of tokens, and an average value of a token generation rate; The current limiting parameters are adjusted according to the global statistical indicators.

6. The distributed current limiting method according to claim 5, characterized in that: The adjusting of the current limiting parameters according to the global statistical indicators includes: When the average number of tokens is lower than a preset low threshold, and the average value of the token generation rate is lower than a preset growth threshold, increasing the capacity of each token bucket and / or increasing the token generation rate; When the average number of tokens is higher than a preset high threshold, and the average value of the token generation rate is also higher than a preset reduction threshold, reducing the capacity of each token bucket and / or reducing the token generation rate; Traffic changes within a second future preset time are predicted based on historical traffic data, and the adjustment value of the capacity of each token bucket and / or the adjustment value of the token generation rate are adjusted based on the traffic changes.

7. The distributed current limiting method according to claim 1, characterized in that: The method further comprises: When it is detected that the third target service instance fails or cannot process the request, the third target service instance is removed from the target service instance list; reallocate the request on the third target service instance to the remaining service instances according to the load status and token bucket status of the remaining service instances, the remaining service instances being service instances other than the third target service instance; The third target service instance is continuously monitored by sending a heartbeat packet, and when the third target service instance is restored, an initial weight is assigned to the third target service instance according to a current load condition and a token bucket state.

8. A distributed current limiting system, characterized in that: It includes the initial module, distribution module, execution module and adjustment module, among which: An initial module configured to determine, in response to a service instance startup, an initial number of tokens in a token bucket according to a maximum number of concurrent requests of the service instance, and create a token bucket for the service instance according to the initial number of tokens; a distribution module configured to, in response to receiving a request from a client, distribute the request to each currently started first target service instance according to a load balancing algorithm, and detect whether a token can be obtained from an associated token bucket of the first target service instance; an execution module configured to, when a token can be obtained from the associated token bucket, process the request through the first target service instance, update the number of tokens in the associated token bucket after the processing is completed, and synchronize the updated state information of the associated token bucket to the distributed cache; An adjustment module is configured to adjust the current limiting parameters according to the status information of all token buckets in the distributed cache, wherein the current limiting parameters include the maximum number of requests that each service instance can process per second, the number of tokens consumed by each request, the capacity of each token bucket and the token generation rate.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Citation Information

Cited By

  • Distributed current limiting method, device and system and storage medium

    CN121239636A

  • Distributed current limiting method, device, system and storage medium

    CN121239636B

  • Scheduling method, device, equipment and program product of KNX bus flow

    CN122160326A

  • Weight-based dynamic current limiting control method and system

    CN122457553A