A method, device and system for automatically managing a throttling cluster and a storage medium

Abstract

The application discloses a traffic-limiting cluster automatic management method, device and system and a storage medium. The method comprises the following steps: listening to the running states of all service instances in a cluster, and maintaining an instance list of all service instances in an online running state based on the listening result, wherein a first service instance serving as a cluster traffic control server and a second service instance serving as a cluster traffic control client are recorded in the instance list, and the number of the second service instances is greater than or equal to one; when it is found that the first service instance has an offline change, determining a second service instance from all the second service instances as a new cluster traffic control server based on a preset first election mechanism, and updating the instance list; and updating the instance list when it is found that a service instance in an offline state has an online change and / or at least one second service instance has an offline change.

Description

Technical Field

[0001] This application relates to the field of cloud service technology, and in particular to a method, apparatus, system and storage medium for automatic management of rate-limited clusters. Background Technology

[0002] In distributed systems, traffic management and control are typically required to ensure system stability and prevent service overload caused by sudden high concurrency requests. Rate limiting, as an important means of traffic control, can effectively reduce resource consumption and ensure system availability. Existing rate limiting technologies generally rely on centralized or distributed flow control components. In a distributed architecture, adopting a clustered rate limiting scheme can improve the reliability and scalability of flow control.

[0003] In cluster-based rate limiting schemes, one or more flow control servers are typically set up to manage global traffic. Other flow control clients query the flow control servers for rate limiting decisions to ensure that the flow control rules are consistent across all nodes. However, due to the dynamic nature of distributed environments, nodes in the cluster may come online or offline, causing the flow control servers to become unavailable and affecting the stability of the entire flow control mechanism.

[0004] Current cluster rate limiting management methods typically rely on manual configuration or a fixed master node mode, which has certain limitations. On the one hand, manual configuration requires the intervention of operations and maintenance personnel, which cannot meet the needs of highly dynamic distributed environments; on the other hand, the fixed master node mode may cause the entire flow control system to be temporarily unavailable when the master node fails, affecting business stability. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an automatic management method, apparatus, system, and storage medium for rate-limiting clusters.

[0006] The technical solution provided in this application is described below:

[0007] The first aspect of this application provides an automatic management method for rate-limiting clusters, the method comprising:

[0008] Monitor the running status of all service instances in the cluster, and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as a cluster flow control server and a second service instance that acts as a cluster flow control client. The number of second service instances is greater than or equal to 1.

[0009] When the offline change of the first service instance is detected, based on the preset first election mechanism, a second service instance is determined from all the current second service instances as the new cluster flow control server, and the instance list is updated.

[0010] When a change occurs in the online status of a service instance that was offline, and / or at least one of the second service instances that went offline, the instance list is updated.

[0011] Optionally, the running status of all service instances in the monitoring cluster includes:

[0012] Subscribe to change information for all service instances in the cluster through the subscription service in Nacos;

[0013] The subscription service is fed a preset listener and the service name of the current service, whereby the service name is used to identify the service instance belonging to the current service in the cluster.

[0014] The listener is used to obtain the change information in order to monitor the running status of each service instance.

[0015] Optionally, obtaining the running status of each service instance through the listener includes:

[0016] Based on Sentinel's DataSource mechanism, a DataSource is constructed to monitor the online and offline status of service instances in Nacos, and the DataSource is registered to Sentinel's cluster flow control interface.

[0017] The DataSource monitors the online / offline status of each service instance in the cluster flow control.

[0018] Optionally, the list of all service instances currently in online operation based on the monitoring results includes:

[0019] When the listener receives information about changes to service instances, it retrieves a list of all service instances currently running online using the full available instance query method.

[0020] Optionally, determining a second service instance as the new cluster flow control server from all current second service instances based on a preset first election mechanism includes:

[0021] Get all current instances of the second service;

[0022] Sort all second service instances in ascending order based on the IP address and port number of each second service instance;

[0023] The first second service instance in the sorted list is selected as the new cluster flow control server, and the remaining second service instances in the sorted list are selected as cluster flow control clients.

[0024] Optionally, when a change in the online status of an offline service instance is detected, and / or at least one of the second service instances is detected to be offline, the method further includes:

[0025] Based on the preset second election mechanism, a second service instance is determined from all currently online service instances as the new cluster flow control server, and the instance list is updated.

[0026] Optionally, the list of instances is stored in Nacos, and the method further includes:

[0027] The initialization function InitFunc of Sentinel is used to obtain a list of all currently running service instances stored in Nacos.

[0028] Based on the aforementioned list of instances, adjust the role attributes of all service instances in Sentinel's cluster flow control.

[0029] A second aspect of this application provides an automatic management device for rate limiting clusters, the device comprising:

[0030] The monitoring unit is used to monitor the running status of all service instances in the cluster and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as a cluster flow control server and a second service instance that acts as a cluster flow control client. The number of the second service instances is greater than or equal to 1.

[0031] An election unit is used to determine a second service instance as a new cluster flow control server from all the current second service instances based on a preset first election mechanism when the offline change of the first service instance is detected, and to update the instance list.

[0032] The update unit is used to update the instance list when a change in the online status of a service instance that is offline is detected, and / or at least one of the second service instances is detected to be offline.

[0033] A third aspect of this application provides an automatic rate-limiting cluster management system, the system comprising:

[0034] Processor, memory, input / output units, and bus;

[0035] The processor is connected to the memory, the input / output unit, and the bus;

[0036] The memory stores a program, which the processor invokes to execute the first aspect and any one of the optional methods in the first aspect.

[0037] A fourth aspect of this application provides a computer-readable storage medium on which a program is stored, which, when executed on a computer, performs the methods of the first aspect and any one of the first aspects.

[0038] As can be seen from the above technical solutions, this application has the following advantages:

[0039] 1. By monitoring the running status of all service instances in the cluster, the online / offline status of service instances can be detected in real time, ensuring that the flow control mechanism has dynamic adaptability. When the cluster flow control server goes offline, an election mechanism is automatically triggered to elect a new server, avoiding interruption of flow control function and improving the stability and reliability of the cluster.

[0040] 2. A preset first-order election mechanism is adopted to automatically select a new server from all running service instances, avoiding manual intervention and reducing maintenance costs. This election mechanism can ensure the stability of server instances and adapt to dynamic changes in cluster size, improving fault recovery capabilities.

[0041] 3. The instance list will be updated dynamically and automatically. Whether a new instance is added (launched) or an instance is taken offline, all nodes in the cluster will automatically detect and adjust their roles to ensure that the flow control system can adapt to changes.

[0042] 4. The cluster flow control client does not need to manually configure the server address. Instead, it automatically discovers and connects to new servers through a dynamically maintained list of instances, improving the ease of deployment and maintenance. This mechanism ensures that the client always connects to an available server, preventing rate limiting decisions from failing due to server failure and improving the fault tolerance of cluster flow control. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic flowchart of an embodiment of the automatic management method for rate limiting clusters provided in this application;

[0045] Figure 2This is a flowchart illustrating an embodiment of the automatic management method for rate-limited clusters provided in this application, based on the method subscription service instance change in Nacos.

[0046] Figure 3 This is a schematic diagram of an embodiment of the automatic management method for rate limiting clusters provided in this application, which redetermines the server based on a second election mechanism.

[0047] Figure 4 This is a schematic diagram of an embodiment of the automatic management method for rate-limiting clusters provided in this application, illustrating the execution of a rate-limiting strategy.

[0048] Figure 5 This is a schematic diagram of an embodiment of the automatic management device for rate limiting clusters provided in this application;

[0049] Figure 6 This is a schematic diagram of an embodiment of the automatic rate limiting cluster management system provided in this application. Detailed Implementation

[0050] The automatic management method for rate limiting clusters provided in this application is primarily applicable to distributed systems and microservice architectures, especially in high-concurrency, dynamically scalable scenarios, ensuring the stability and high availability of traffic control. Specific applicable scenarios include:

[0051] 1. Large-scale microservice clusters

[0052] Suitable for microservice clusters based on architectures such as Spring Cloud, Dubbo, Kubernetes, and Service Mesh, it can dynamically adapt to the online / offline status of service instances and automatically adjust the roles of the flow control server and client. Suitable for microservice systems using Nacos as the registry center, it can monitor changes to service instances, automatically discover flow control servers, and ensure that rate limiting rules are synchronized.

[0053] 2. High-concurrency internet services

[0054] Suitable for high-concurrency, high-traffic business scenarios such as e-commerce, online payment, short videos, and social media platforms, ensuring that the system will not fail due to abnormal flow control services during sudden traffic surges or peak traffic events. For example, in scenarios such as flash sales, limited-time offers, and trending events, the system needs to ensure that flow control policies are effective to avoid single points of pressure.

[0055] 3. Elastically scalable cloud-native environment

[0056] Suitable for containerized deployments based on Kubernetes, it can automatically adapt to the dynamic scaling of Pods, ensuring automatic adaptation of rate-limiting servers and clients. For example, in cloud computing platforms (such as Alibaba Cloud, AWS, and Google Cloud), it can be used to dynamically adjust flow control mechanisms, avoiding manual intervention.

[0057] 4. API Gateway and Service Governance

[0058] Suitable for traffic management based on API Gateway (such as Spring Cloud Gateway, Kong, Envoy), ensuring that rate limiting, circuit breaking, and degradation policies for all API requests are effective. Suitable for enterprise internal service governance platforms, ensuring consistency of rate limiting policies across teams and business functions, and avoiding the problem of rules being out of sync between different service instances.

[0059] The following explains the implementing entity and related technical terms of the method in this application:

[0060] 1. Service instances of the rate-limiting cluster (i.e., all application nodes)

[0061] This is responsible for running the methods described in this application, including monitoring its own state, detecting cluster changes, and participating in the election mechanism. Each service instance can become a cluster flow control server or a cluster flow control client, and will automatically adjust according to the rules.

[0062] 2. Cluster flow control server

[0063] The server is automatically elected by the election mechanism and is responsible for statistics, determining whether rate limiting is triggered, and distributing the rate limiting decision to all cluster flow control clients. When the server goes offline, other service instances will automatically elect a new server to ensure that the rate limiting function is not interrupted.

[0064] 3. Cluster flow control client

[0065] It is mainly used to report request traffic information and receive rate limiting policies from the server, and execute rate limiting according to the policies. Clients can change roles at any time, and when a new server is selected, the remaining clients will automatically adjust their connection server address.

[0066] 4. Nacos

[0067] It provides service registration and discovery capabilities, and supports monitoring the online / offline changes of all service instances.

[0068] 5. Sentinel rate limiting component

[0069] Provides a cluster rate limiting interface to manage the synchronization of flow control rules between the server and client. Dynamically integrates Nacos instance information.

[0070] The embodiments of this application are described in detail below:

[0071] Please see Figure 1 This application first provides an embodiment of an automatic management method for rate-limiting clusters, which includes:

[0072] S101. Monitor the running status of all service instances in the cluster, and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as the cluster flow control server and a second service instance that acts as the cluster flow control client. The number of second service instances is greater than or equal to 1.

[0073] In distributed systems and microservice architectures, service instances dynamically scale up and down, causing the number and status of instances in the cluster to constantly change. To ensure the stable operation of the rate limiting system, this embodiment continuously monitors the running status of all service instances and maintains a dynamic list of instances based on the monitoring results to correctly distinguish between the cluster flow control server and the cluster flow control client.

[0074] In practical applications, microservice architectures can manage instance information through service registries (such as Nacos, Eureka, Consul, and Zookeeper). This embodiment uses Nacos as the service registry, monitoring the status of all service instances of the same service and obtaining a list of all currently running service instances. This list includes all service instances still online and provides detailed information such as their IP address, port, and running status. The list also records a first service instance acting as a cluster flow control server and a second service instance acting as a cluster flow control client, with at least one second service instance.

[0075] In this application, runtime status is used to describe the current state of each service instance in the cluster. Runtime status includes the following two types:

[0076] Online (online) running status: This refers to a service instance that has started normally and joined the cluster, capable of providing services or participating in cluster flow control management. In this state, the service instance is considered active and can act as a cluster flow control client or cluster flow control server.

[0077] Offline (deactivated) status: This refers to a service instance that has been removed from the cluster due to voluntary shutdown, abnormal exit, or failure, and is temporarily not involved in cluster flow control management. Service instances in this status will not be selected as cluster flow control servers or clients, and cannot handle rate limiting requests.

[0078] S102. When the offline change of the first service instance is detected, based on the preset first election mechanism, a second service instance is determined from all the current second service instances as the new cluster flow control server, and the instance list is updated.

[0079] In a distributed rate limiting system, the first service instance, acting as the cluster flow control server, is responsible for global flow control decisions, while the second service instance, acting as the cluster flow control client, is only responsible for reporting traffic data to the server. In this embodiment, when a server goes offline, a new server needs to be elected to ensure the continuous and stable operation of the entire rate limiting mechanism.

[0080] When a server exits, all surviving service instances receive an event notification and execute a first election mechanism (such as sorting by IP address and port) to determine a new server from among all the current second-tier service instances. Specifically, in step S101, all service instances have subscribed to the cluster service status. When a server goes offline (whether due to a crash, voluntary exit, or load balancing strategy adjustment), the listener is triggered to obtain the latest list of instances still running. It can then be further determined whether the current server is still in this list. If not, it means the server has gone offline, and an election process needs to be executed.

[0081] When a server is detected to be offline, the service instances that are still running will elect a new server according to the first election mechanism.

[0082] In an optional embodiment, the first election mechanism may employ a forward sorting strategy based on IP address and port number, i.e., sorting by IP address and port number, and selecting the service instance at the top of the sorted list as the new Server. Specifically, this includes: obtaining a list of all currently running service instances; sorting the service instances forward based on their IP address and port number; selecting the first service instance in the sorted list as the new cluster flow control server, and the remaining service instances as cluster flow control clients.

[0083] After the election is completed, the information of the new server needs to be communicated to all service instances in the instance list so that the client can correctly send flow control data to the new server.

[0084] In distributed systems, election mechanisms are crucial for ensuring high availability. Besides forward ordering by IP address and port, other election strategies can be employed. This application also provides the following methods:

[0085] 1. Election based on node weight;

[0086] This approach is suitable for scenarios where nodes have varying computing power, network bandwidth, and load capacity, for example, where some nodes possess stronger computing resources. In this method, each service instance is assigned a weight value when registering with Nacos; a higher weight indicates that the node is more suitable to assume the Server role. During the election process, service instances are sorted in descending order of their weight values, and the one with the highest weight is selected as the Server.

[0087] 2. Elections based on health status;

[0088] This method is suitable for situations where some nodes are experiencing high CPU, memory, and network loads, preventing the selection of high-load instances as servers. The health status (CPU, memory, network load, etc.) of each node can be monitored using Nacos or Prometheus. Instances with poor health are filtered out. Among the healthy instances, other election strategies (such as IP + port sorting, random election, etc.) are then used to determine the server.

[0089] 3. Election based on LeaderLease;

[0090] This method is suitable for high-concurrency, low-latency distributed systems, preventing competition and conflicts during the election process. All nodes attempt to compete for a lease in the distributed storage (such as Etcd, Zookeeper, or Redis). The node that successfully acquires the lease first becomes the Server, and the other nodes become Clients. The lease expires after a certain period, and the Server needs to renew it periodically; otherwise, a new election will be triggered.

[0091] This application does not specify a particular election mechanism.

[0092] S103. When a change is detected in the online status of a service instance that is offline, and / or at least one second service instance that is offline, update the instance list.

[0093] When a change in the online status of a service instance that is not part of the clustered flow control server or client is detected, i.e., when a service instance that was offline is detected to be online, and / or at least one second service instance is detected to be offline, the instance list of all service instances needs to be updated to ensure that all nodes in the cluster can obtain the latest information on the running service instances, thereby ensuring the normal operation of the rate limiting policy.

[0094] The offline service instances in this application include those that were cluster flow control servers when they were last offline. If a service instance was previously a server but went offline due to a fault, maintenance, or other reasons and then comes back online, it should not automatically resume its server role. Instead, its role should be re-determined according to an election mechanism. Directly restoring a previously active server upon its return could result in multiple servers within the cluster, violating the principle of a single flow control decision center. A re-launched instance may no longer be suitable as a server due to changes in environment or load, and its role should still be re-confirmed through an election mechanism. By reasonably determining the server through an election mechanism, it is ensured that the new cluster flow control server can correctly take over flow control tasks, avoiding the failure or imbalance of flow control rules.

[0095] In this embodiment, since a new server has been re-elected and the instance list has been updated when the server goes offline, when the old server comes back online, it will not automatically return to being a server. Instead, it will be regarded as a new cluster node and its role will be reassigned according to a specific election mechanism. For example, it can be directly listed as a new Client or participate in a new round of election for role assignment. During this process, it may still become a Server again.

[0096] In distributed systems, service instances within a cluster may experience online / offline changes due to factors such as scaling up, scaling down, fault recovery, and network fluctuations. Failure to update the instance list in a timely manner may lead to the following problems:

[0097] Newly launched clients cannot join the flow control cluster, resulting in their traffic not being properly managed.

[0098] Clients that have been taken offline are still considered active nodes, which may cause requests to be routed to instances that have failed, affecting business availability.

[0099] The cluster flow control server may still send flow control rules to offline clients, resulting in wasted resources or incorrect flow control decisions.

[0100] Therefore, after detecting the online / offline events of non-cluster flow control servers, it is necessary to update the instance list in a timely manner to maintain consistency among all nodes in the cluster.

[0101] Step S103 ensures that each node in the cluster has the latest instance list, guaranteeing that all nodes are aware of changes in cluster size. Dynamic management of the entire rate-limited cluster is crucial for improving system availability and robustness.

[0102] In distributed systems, to achieve efficient and reliable traffic control, it is typically necessary to monitor the running status of all service instances in the cluster in real time. When an instance in the cluster goes online or offline, the system needs to be able to detect this promptly and adjust the rate limiting strategy accordingly to ensure the stability and high availability of the system.

[0103] This application also provides an embodiment of a service instance monitoring method based on Nacos. Through the service registration and discovery mechanism provided by Nacos, the running status of all service instances in the cluster can be obtained in real time, and the instance list can be dynamically maintained based on the monitoring results. This method utilizes subscription methods provided by Nacos, such as using the namingService.subscribe method to subscribe to service instance change information, thereby proactively receiving change notifications when instances go online or offline, and processing these changes through a listener to ensure the accuracy and real-time performance of cluster flow control management.

[0104] The technical solution of this embodiment can effectively reduce the impact of service instance changes on system flow control strategies and ensure that the role allocation in the cluster (such as the division between server and client) can always remain up-to-date.

[0105] See Figure 2 This embodiment includes:

[0106] S201. Subscribe to change information of all service instances in the cluster through the subscription service in Nacos;

[0107] Nacos provides a subscription service that allows applications to subscribe to instance change events of a service. When an instance of that service is added, deleted, or its state changes, Nacos will proactively push the change information to subscribers.

[0108] Specifically, first, a NamingService instance is obtained from Nacos, and then the subscribe method is called to subscribe to instance change information of the target service. When an instance in the cluster goes online or offline, Nacos will trigger a callback to notify subscribers, enabling them to be aware of the instance changes in a timely manner and make corresponding adjustments.

[0109] S202. Pass the preset listener and the service name of the current service to the subscription service method. The service name is used to identify the service instance belonging to the current service in the cluster.

[0110] In order to correctly respond to the online / offline events of service instances, you need to pass the preset listener and the service name of the current service to the subscribe method.

[0111] The service name is used to identify the specific service to be monitored, such as "flow-control-service," so that the system only receives instance change information related to that service. A listener can be a callback function that is triggered when Nacos detects instance changes. For example, by receiving change events through a listener, the system can obtain the latest instance list, filter out unhealthy or ineligible instances to ensure the accuracy of the instance list, and update the local cache so that all nodes in the cluster can make rate-limiting decisions based on the latest instance list.

[0112] S203. Obtain change information through a listener to monitor the running status of each corresponding service instance;

[0113] Once the listener is triggered, the system needs to retrieve all available service instances from Nacos and maintain the latest instance list based on the status information of these instances.

[0114] In one specific implementation, the list of instances can be obtained by first having the listener call namingService.getAllInstances(serviceName) to get the latest list of all service instances and extract key information about the service instances (such as IP, port, health status, etc.).

[0115] In one possible implementation, since the list of instances returned by Nacos may contain some instances with abnormal states (such as crashes, failed health checks, etc.), these invalid instances can be filtered out.

[0116] In distributed systems, to ensure efficient and consistent traffic control, it's necessary to dynamically sense the online / offline status of service instances and adjust flow control configurations based on cluster conditions. In one optional embodiment, an extended solution capable of monitoring changes to Nacos service instances can be built based on Sentinel's DataSource mechanism. This method allows Sentinel to monitor the running status of each service instance in Nacos in real time and automatically adjust cluster flow control configurations when instances change, thereby achieving dynamic management of the cluster flow control server and client. Specifically, in Sentinel's traffic management system, the DataSource mechanism provides dynamic data sources to support real-time rule changes. Sentinel offers various DataSource methods, such as files, Zookeeper, Redis, and Nacos. In this embodiment, Nacos is chosen as the dynamic data source. A dedicated DataSource for monitoring the online / offline status of Nacos service instances is built and registered with Sentinel's cluster flow control interface, enabling Sentinel to automatically adjust rate limiting strategies based on changes to Nacos instances.

[0117] Specifically, firstly, the ReadableDataSource interface is used to implement a Nacos data source extension class, which is used to listen for changes in the instances of the current service in Nacos. In the loadConfig() method of this class, the namingService.subscribe() method provided by Nacos is called to subscribe to the online / offline events of the target service. When change information is received, the full list of available instances is called, such as getAllInstances(), to get the latest list of available service instances.

[0118] Once the DataSource mechanism is registered with Sentinel, it can run continuously, detect changes in service instances in Nacos in real time, and feed back the latest instance status to Sentinel so that the cluster flow control policy is always adjusted based on the latest instance list.

[0119] S204. When a change in the first service instance is detected to be offline, based on the preset first election mechanism, a second service instance is determined from all the current second service instances as the new cluster flow control server, and the instance list is updated.

[0120] S205 When a change is detected in the online status of a service instance that was offline, and / or at least one second service instance that was offline, the instance list is updated.

[0121] Steps S204 and S205 in this embodiment are similar to those in the previous embodiments, and will not be described again here.

[0122] This implementation enables the rate-limiting cluster to dynamically detect changes in service instances and allocate roles reasonably based on the latest instance list, thereby optimizing the stability and reliability of flow control management.

[0123] In distributed rate-limiting cluster management, the stability and availability of the cluster flow control server directly impact the flow control strategy of the entire cluster. If the cluster flow control server goes offline, the system will select a new server based on a preset first-election mechanism and update the instance list accordingly. However, in some cases, even if the cluster flow control server remains unchanged, the online or offline of other service instances (i.e., the cluster flow control client or the previously offline cluster flow control server) may necessitate adjustments to the system architecture and flow control management. For example, a new service instance might be more suitable for the role of the cluster flow control server, or the existing cluster architecture might no longer meet the optimal configuration for the flow control strategy.

[0124] To address this issue, this embodiment provides an automatic adjustment method based on a preset second election mechanism. This method dynamically assesses the current cluster status when non-cluster flow control servers experience online / offline changes, and re-elects a cluster flow control server if necessary. This approach improves cluster stability, optimizes flow control strategies, and adapts to scenarios with dynamic instance changes, ensuring the efficiency and reliability of rate limiting management.

[0125] The following is a detailed description of this embodiment; please refer to [reference needed]. Figure 3 This embodiment includes:

[0126] S301. Monitor the running status of all service instances in the cluster, and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as the cluster flow control server and a second service instance that acts as the cluster flow control client. The number of second service instances is greater than or equal to 1.

[0127] S302. When a change in the first service instance is detected to be offline, based on the preset first election mechanism, a second service instance is determined from all the current second service instances as the new cluster flow control server, and the instance list is updated.

[0128] Steps S301 and S302 in this embodiment are implemented in a similar manner to steps S101 and S102 in the previous embodiment, and will not be described again here.

[0129] S303. When a change in the online status of a service instance that is offline is detected, and / or at least one second service instance is detected to be offline, based on the preset second election mechanism, a second service instance is determined from all currently online service instances as the new cluster flow control server, and the instance list is updated.

[0130] In this embodiment, after receiving the change information pushed by Nacos, the listener calls the namingService.getAllInstances() method to obtain a list of all currently running service instances. This list includes information such as the IP address, port number, running status, weight, and metadata of each service instance.

[0131] Since this change involves non-cluster flow control servers (i.e., cluster flow control clients or previously offline server instances), it should not directly lead to changes in the flow control server itself. However, it is still necessary to check whether there are better server options. For example, the following situations may exist:

[0132] (1) The current health status of the flow control server is abnormal (such as heartbeat timeout, excessive load, etc.).

[0133] (2) The newly added instance is more suitable as a server in terms of configuration parameters, hardware resources, and network topology.

[0134] (3) The current flow control server is approaching its load bottleneck, and the cluster needs to reassign roles.

[0135] In this embodiment, it can be determined whether the re-election conditions are met. If they are met, a second service instance is re-determined from all running service instances as the cluster flow control server according to a preset second election mechanism. The specific election mechanism can be one or more combinations mentioned in the foregoing embodiments or other possible election mechanisms.

[0136] After a new cluster flow control server is elected, the new list of instances is broadcast and synchronized to all service instances in the cluster, so that all nodes in the cluster can share the latest role information.

[0137] This implementation enhances the cluster's dynamic adaptability, automatically assessing and adjusting the flow control server when service instances go offline or online, ensuring the stability and optimal configuration of cluster flow control management. It reduces the load risk on the flow control server; if the server is under high load, it can automatically switch to a more suitable node, improving overall system performance. Furthermore, it enhances system high availability, ensuring that flow control policies continue to execute efficiently and stably even during dynamic instance changes, scaling up / down, and fault recovery scenarios.

[0138] In distributed systems, rate limiting is an important traffic management technique that prevents sudden traffic surges from overloading the system and ensures service stability. To improve the accuracy and consistency of rate limiting strategies, cluster flow control mechanisms typically employ centralized flow control decisions. This means that the cluster flow control server performs unified traffic control analysis, while the cluster flow control client is responsible for specific request interception and execution.

[0139] This embodiment provides a traffic control method based on a cluster flow control server, used to achieve efficient flow control judgment and execution in rate-limited cluster management. Specifically, when a cluster flow control client receives a rate-limiting related request, it does not execute the flow control policy independently, but instead sends a flow control judgment request to the cluster flow control server. The server then performs flow control analysis based on preset flow control rules. After the analysis result is returned, the cluster flow control client decides whether to block the request based on the result, thereby achieving a globally consistent rate-limiting policy.

[0140] This method can effectively avoid misjudgments in single-point flow control, ensure consistency in rate limiting decisions for all clients, and centralize the management of rate limiting rule updates and adjustments, thereby improving the maintainability and scalability of the system.

[0141] The following is a detailed description of this embodiment; please refer to [reference needed]. Figure 4 This embodiment includes:

[0142] S401. When the second service instance receives the rate limiting request, it sends a flow control judgment request to the first service instance.

[0143] In a distributed system, each secondary service instance, acting as a cluster flow control client, is responsible for handling the requests it receives. In this step, when a client detects a request requiring flow control, it doesn't directly execute the rate limiting policy locally. Instead, it encapsulates the request information into a flow control judgment request and sends it to the primary service instance, acting as the cluster flow control server. After encapsulating this data, the client can use remote procedure calls or message queue-based communication to send the request to the currently selected cluster flow control server and wait for its analysis results.

[0144] S402. Perform flow control analysis based on preset flow control rules through the first service instance, and send the analysis results to the second service instance;

[0145] When the flow control server receives a flow control judgment request from the client, it determines whether the current traffic exceeds a set threshold based on the system's preset flow control strategy (such as fixed time window, sliding window, token bucket, leaky bucket algorithm, etc.), and thus decides on the rate limiting strategy. For example, if the flow control threshold is not reached, a command to allow passage is returned; if the flow control threshold is exceeded, a command to trigger rate limiting is returned, informing the client to discard the request or delay processing. Finally, the calculated flow control decision result is encapsulated and sent back to the cluster flow control client that originally made the request.

[0146] S403. Based on the analysis results, perform traffic control through the second service instance.

[0147] After receiving the analysis results from the flow control server, the cluster flow control client will execute the corresponding flow control policy according to the decision. For example, if the returned result allows the request to pass, the client will process the request normally without any additional interception. If the returned result triggers flow control, the client will execute the rate limiting policy.

[0148] After executing flow control, the client can also report flow control logs or monitoring data for subsequent optimization of the flow limiting strategy. This embodiment improves the consistency, maintainability and dynamic adjustment capability of the flow control strategy, ensuring the stability and reliability of the system in high-concurrency scenarios.

[0149] Based on the foregoing method embodiments, this application also provides an embodiment of a rate-limiting cluster automatic management device, a rate-limiting cluster automatic management system, and a computer-readable storage medium, which are described below:

[0150] See Figure 5 This application provides an automatic management device for rate limiting clusters, comprising:

[0151] The monitoring unit 501 is used to monitor the running status of all service instances in the cluster and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as the cluster flow control server and a second service instance that acts as the cluster flow control client. The number of second service instances is greater than or equal to 1.

[0152] Election unit 502 is used to determine a second service instance as the new cluster flow control server from all the current second service instances based on the preset first election mechanism when the offline change of the first service instance is detected, and to update the instance list.

[0153] Update unit 503 is used to update the instance list when a service instance that was offline is detected to be brought online, and / or at least one second service instance is detected to be taken offline. Optionally, monitoring unit 501 is specifically used for:

[0154] Subscribe to change information for all service instances in the cluster through the subscription service in Nacos;

[0155] Pass the preset listener and the service name of the current service to the subscription service. The service name is used to identify the service instance belonging to the current service in the cluster.

[0156] The running status of each service instance is obtained through a listener.

[0157] Optionally, the listening unit 501 is specifically used for:

[0158] Based on Sentinel's DataSource mechanism, a DataSource is built to monitor the online and offline status of service instances in Nacos, and the DataSource is registered to Sentinel's cluster flow control interface.

[0159] Monitor the online / offline status of each service instance in the cluster flow control through DataSource.

[0160] Optionally, the listening unit 501 is specifically used for:

[0161] When a change in a service instance is received through the listener, a list of all service instances currently running online is obtained using the full available instance query method.

[0162] Optionally, election unit 502 is specifically used for:

[0163] Get all current second service instances;

[0164] Sort all second service instances in ascending order based on the IP address and port number of each second service instance;

[0165] The first second service instance in the sorted list is selected as the new cluster flow control server, and the remaining second service instances in the sorted list are selected as cluster flow control clients.

[0166] Optionally, update unit 503 is specifically used for:

[0167] Based on the preset second election mechanism, a second service instance is selected from all currently online service instances as the new cluster flow control server, and the instance list is updated.

[0168] Optionally, the list of all currently running service instances stored in Nacos also includes an adjustment unit 504, specifically used for:

[0169] The initialization function InitFunc of Sentinel is used to obtain a list of all currently running service instances stored in Nacos.

[0170] Adjust the role attributes of all service instances in Sentinel's cluster flow control based on the instance list.

[0171] Please see Figure 6 This application also provides an automatic management device for rate limiting clusters, comprising:

[0172] Processor 601, memory 602, input / output unit 603, bus 604;

[0173] The processor 601 is connected to the memory 602, the input / output unit 603, and the bus 604;

[0174] The memory 602 stores a program, and the processor 601 calls the program to execute any of the methods described above.

[0175] This application also relates to a computer-readable storage medium on which a program is stored, characterized in that, when the program is run on a computer, it causes the computer to perform any of the methods described above.

[0176] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An automatic management method for rate-limited clusters, characterized in that, The method includes: Monitor the running status of all service instances in the cluster, and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as a cluster flow control server and a second service instance that acts as a cluster flow control client. The number of second service instances is greater than or equal to 1. When the offline change of the first service instance is detected, based on the preset first election mechanism, a second service instance is determined from all the current second service instances as the new cluster flow control server, and the instance list is updated. When a change occurs in the online status of a service instance that was offline, and / or at least one of the second service instances that went offline, the instance list is updated.

2. The automatic management method for rate-limiting clusters according to claim 1, characterized in that, The running status of all service instances in the monitoring cluster includes: Subscribe to change information for all service instances in the cluster through the subscription service in Nacos; The subscription service is fed a preset listener and the service name of the current service, whereby the service name is used to identify the service instance belonging to the current service in the cluster. The listener is used to obtain the change information in order to monitor the running status of each service instance.

3. The automatic management method for rate-limiting clusters according to claim 2, characterized in that, The process of obtaining the running status of each service instance through the listener includes: Based on Sentinel's DataSource mechanism, a DataSource is constructed to monitor the online and offline status of service instances in Nacos, and the DataSource is registered to Sentinel's cluster flow control interface. The DataSource monitors the online / offline status of each service instance in the cluster flow control.

4. The automatic management method for rate-limiting clusters according to claim 2, characterized in that, The list of all service instances currently running online, maintained based on monitoring results, includes: When the listener receives information about changes to service instances, it retrieves a list of all service instances currently running online using the full available instance query method.

5. The automatic management method for rate-limiting clusters according to claim 1, characterized in that, The step of determining a second service instance as the new cluster flow control server from all current second service instances based on a preset first election mechanism includes: Get all current instances of the second service; Sort all second service instances in ascending order based on the IP address and port number of each second service instance; The first second service instance in the sorted list is selected as the new cluster flow control server, and the remaining second service instances in the sorted list are selected as cluster flow control clients.

6. The automatic management method for rate-limiting clusters according to claim 1, characterized in that, When a change to bring an offline service instance online is detected, and / or at least one of the second service instances goes offline, the method further includes: Based on the preset second election mechanism, a second service instance is determined from all currently online service instances as the new cluster flow control server, and the instance list is updated.

7. The automatic management method for rate-limited clusters according to any one of claims 1 to 6, characterized in that, The list of instances is stored in Nacos, and the method further includes: The initialization function InitFunc of Sentinel is used to obtain a list of all currently running service instances stored in Nacos. Based on the aforementioned list of instances, adjust the role attributes of all service instances in Sentinel's cluster flow control.

8. An automatic management device for rate-limiting clusters, characterized in that, The device includes: The monitoring unit is used to monitor the running status of all service instances in the cluster and maintain a list of all service instances currently running online based on the monitoring results. The list of instances records a first service instance that acts as a cluster flow control server and a second service instance that acts as a cluster flow control client. The number of the second service instances is greater than or equal to 1. An election unit is used to determine a second service instance as a new cluster flow control server from all the current second service instances based on a preset first election mechanism when the offline change of the first service instance is detected, and to update the instance list. The update unit is used to update the instance list when a change in the online status of a service instance that is offline is detected, and / or at least one of the second service instances is detected to be offline.

9. An automatic management system for rate limiting clusters, characterized in that, The system includes: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, which the processor invokes to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, the program performing the method as claimed in any one of claims 1 to 7 when executed on a computer.

Images