Distributed system hierarchical speed limiting method and device based on real-time load sensing

Abstract

The invention provides a distributed system hierarchical speed limiting method and device based on real-time load sensing, and relates to the technical field of flow control. Comprising the following steps: an initialization step: configuring system basic parameters and a load data acquisition period; receiving service requests, and identifying and distinguishing read requests, write requests and service priorities in the service requests; periodically collecting load data of distributed nodes, and calculating a standardized index threshold value; judging the load state in combination with a threshold value; according to the load state, dynamically adjusting the TPS gear of the number of transactions per second; dynamically adjusting the current maximum token number in the current token cylinder of the system according to the current gear by using a token bucket algorithm, distributing tokens as required, checking whether enough tokens exist in the current token cylinder based on the received service request, and if yes, allowing the service request to pass; if not, refusing the service request or queuing the service request; and monitoring and analyzing a read-write ratio in the service request, and executing differential speed limit control in combination with the read-write ratio and the service priority.

Description

Technical Field

[0001] The present invention relates to the technical field of flow control, and in particular to a distributed system hierarchical speed limiting method and device based on real-time load perception. Background Art

[0002] In distributed systems, rate limiting technology is an important means to ensure the stable operation of the system. Its core goal is to prevent performance degradation or service interruption caused by system overload by reasonably allocating system resources. However, with the increasing complexity and dynamics of business scenarios, existing distributed system rate limiting technology faces a series of significant challenges. These challenges not only affect the overall performance of the system, but also limit the system's ability to respond to sudden traffic.

[0003] Traditionally, the speed limiting technology of distributed systems mostly adopts statically configured speed limiting rules. Although this method is simple and easy to implement, it is powerless when facing dynamically changing load conditions. Since static rules cannot be automatically adjusted according to real-time load conditions, it often leads to over-protection of resources when the load is light, resulting in waste; while when the load is heavy, insufficient protection may cause system performance bottlenecks or even crashes. This "one-size-fits-all" speed limiting strategy obviously cannot meet the high requirements of modern distributed systems for flexibility and adaptability.

[0004] Furthermore, existing rate limiting technologies often ignore the differentiated impact of different types of requests on system resources. As two basic types of operations in distributed systems, read and write operations have significant differences in their consumption and impact on system resources. However, traditional rate limiting technologies fail to distinguish between these two operations, but instead adopt a unified rate limiting standard. This processing method not only fails to fully utilize the potential of system resources, but may also cause key read and write requests to be blocked due to uneven resource allocation, thereby affecting the efficient operation of the overall business. In addition, in actual applications, different business requests often have different priorities and read-write ratios. For example, some key business requests may require higher response speeds and more stable resource guarantees, while some non-critical business requests can be processed when resources are sufficient. However, traditional rate limiting technologies lack an in-depth understanding and flexible response to these business characteristics, and are unable to dynamically adjust the rate limiting strategy based on the priority and read-write ratio of the request, resulting in unreasonable system resource allocation and limited business processing efficiency.

[0005] In addition, the lack of real-time perception of node load status and accurate flow control is also a major flaw of existing technologies. In burst traffic scenarios, the system often finds it difficult to respond quickly and adjust the speed limit strategy, which leads to local node overload, which in turn triggers a chain reaction in the entire system, causing service interruption or a sharp drop in performance. The root cause of this problem is that the flow control algorithm is too simple and rigid, and cannot adapt to the diverse needs of complex business scenarios.

[0006] In summary, the existing distributed system speed limit technology has been unable to meet the needs of modern distributed systems for efficient, stable and adaptive operation due to its static configuration, lack of differentiated processing and insufficient real-time perception capabilities. Therefore, a new technical solution is urgently needed to overcome the above defects and achieve more intelligent, flexible and accurate speed limit control to cope with increasingly complex and changing business scenarios and load challenges. Summary of the invention

[0007] The present invention aims to provide a distributed system hierarchical speed limiting method and device based on real-time load perception to solve the problems existing in the prior art, protect the stability of the distributed system, achieve more intelligent, flexible and accurate speed limiting control, and minimize the impact on business requests.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: A distributed system hierarchical rate limiting method based on real-time load perception, comprising: Initialization step, used to configure the basic parameters of the system and the load data collection period; Receive a service request, and identify and distinguish a read request, a write request, and a service priority in the service request; Periodically collect load data of distributed nodes and calculate standardized indicator thresholds; Determine the load status in combination with the standardized indicator threshold; Dynamically adjust the number of transactions per second (TPS) according to the load status; Using a token bucket algorithm to dynamically adjust the current maximum number of tokens in the current token bucket of the system according to the current TPS gear and allocate tokens on demand, based on the received service request, check whether there are enough tokens in the current token bucket, if there are enough tokens, allow the service request to pass; otherwise, reject the service request or queue the service request; Monitor and analyze the read-write ratio in the service request, and perform differentiated rate limit control in combination with the read-write ratio and the service priority.

[0009] Furthermore, the basic system parameters include: number of CPU cores, number of transactions per second (TPS) gear, and initial speed limit parameters, etc.

[0010] Furthermore, the TPS gears include the following levels: 120, 60, 40, 30, 24, 20, 17, 15, 12, 10, 8, 6, 4, 2, 1.

[0011] Furthermore, the key parameters of the token bucket algorithm include: the maximum number of tokens in the token bucket and the number of tokens obtained each time, the maximum number of tokens in the token bucket is 120, and the number of tokens obtained each time is selected from the set {2,3,4,5,6,7,8,10,12,15,20,30,60,120}.

[0012] Furthermore, the system API or dedicated monitoring interface is periodically called to obtain load data, and standardized indicator thresholds of 1 minute, 5 minutes and 10 minutes are calculated.

[0013] Furthermore, the standardized index threshold is calculated in the following manner: 1-minute index threshold = 1-minute load / number of CPU cores; 5-minute index threshold = 5-minute load / number of CPU cores; 10-minute index threshold = 10-minute load / number of CPU cores.

[0014] Furthermore, the load status includes: a healthy state is a 5-minute index threshold <1 and a 10-minute index threshold <1; a sub-healthy state is a 5-minute index threshold ≥1 or a 10-minute index threshold ≥1; an unhealthy state is a 5-minute index threshold >1 and a 10-minute index threshold >1.

[0015] Furthermore, the dynamic adjustment of the TPS gear includes: increasing the TPS gear by 1 level when the load state is the healthy state; reducing the TPS gear by 1 level when the load state is the sub-healthy state; and reducing the TPS gear by 2 levels when the load state is the unhealthy state.

[0016] Furthermore, it includes performing stricter rate limit control on the write request.

[0017] A distributed system hierarchical speed limiting device based on real-time load perception, comprising: An initialization module, wherein the initialization module configures the basic parameters of the system and a load data collection period; A receiving module, the receiving module receives a service request, identifies and distinguishes a read request, a write request and a service priority in the service request; A load data collection module, wherein the load data collection module periodically collects load data of distributed nodes and calculates a standardized index threshold; A load state judgment module, wherein the load state judgment module judges the load state in combination with the standardized index threshold; A dynamic adjustment module, wherein the dynamic adjustment module dynamically adjusts the number of transactions per second (TPS) gear according to the load state; A speed limit execution module, which uses a token bucket algorithm to dynamically adjust the current maximum number of tokens in the current token bucket of the system according to the current TPS gear and allocates tokens on demand, and checks whether there are enough tokens in the current token bucket based on the received service request. If there are enough tokens, the service request is allowed to pass; otherwise, the service request is rejected or queued; The speed limit execution module further monitors and analyzes the read-write ratio in the service request, and performs differentiated speed limit control in combination with the read-write ratio and the service priority.

[0018] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the distributed system hierarchical speed limiting method based on real-time load perception is implemented.

[0019] The distributed hierarchical speed limiting method and device based on real-time load perception proposed in the present invention significantly improves the stability and business processing capability of the distributed system by innovatively integrating real-time data collection, intelligent state judgment and dynamic flow control strategy, and specifically exhibits the following beneficial technical effects: 1. Real-time dynamic perception and response: The key load data of each node in the distributed system is collected regularly and efficiently, including but not limited to the number of CPU cores, TPS gear, initial speed limit parameters, etc., to achieve real-time and accurate monitoring of the node operation status.

[0020] Based on the collected data, the component can quickly determine the health status of the node (healthy, sub-healthy, unhealthy), and dynamically adjust the request processing rate accordingly to ensure that the system can still maintain stable operation under high load while avoiding excessive consumption or idleness of resources.

[0021] 2. Smooth speed limit adjustment to reduce business impact: The multi-speed TPS (transactions per second) control mechanism is adopted to achieve delicate adjustment of the speed limit strategy, avoiding business interruption or performance jitter caused by sudden drop or increase of the speed limit strategy in traditional speed limit technology.

[0022] This smooth adjustment method not only protects the system from the impact of sudden traffic, but also ensures the continuity and stability of business requests, improving user experience.

[0023] 3. Read-write separation strategy to optimize resource allocation: In view of the different characteristics of resource consumption of read and write operations in distributed systems, an innovative read-write separation speed limit strategy is introduced.

[0024] Implementing stricter speed limit rules for write operations effectively prevents system resource exhaustion or performance degradation caused by too frequent write operations, while ensuring smooth read operations and improving the overall processing efficiency and response speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0026] Figure 1 Flow chart of a distributed hierarchical speed limiting method based on real-time load perception provided by an embodiment of the present invention; Figure 2 A schematic structural diagram of a distributed hierarchical speed limiting device based on real-time load perception provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and understandable, the specific embodiments of the present invention include various specific details to help understanding, but these specific details are only considered to be exemplary. Therefore, it will be appreciated by those skilled in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present invention. In addition, for the sake of clarity and brevity, the description of well-known functions and structures may be omitted.

[0028] The terms and words used in the following description and claims are not limited to the bibliographical meanings, and it is clear to those skilled in the art that the following description of various embodiments of the present invention is provided for illustrative purposes only and not for the purpose of limiting the invention as defined by the appended claims and their equivalents.

[0029] Those skilled in the art know that the blocks of the flowchart (or sequence diagram) and the combination of the flowchart can be represented and executed by computer program instructions. These computer program instructions can be loaded on a processor of a general-purpose computer, a special-purpose computer, or a programmable data processing device. When the loaded program instructions are executed by the processor, they create a means for performing the functions described in the flowchart. Because the computer program instructions can be stored in a computer-readable memory available in a special-purpose computer or a programmable data processing device, it is also possible to create an article that performs the functions described in the flowchart. Because the computer program instructions can be loaded on a computer or a programmable data processing device, when executed as a process, they can perform the operations of the functions described in the flowchart.

[0030] The blocks of the flow chart may correspond to a module, segment or code containing one or more executable instructions that implement one or more logical functions or may correspond to a portion thereof. In some cases, the functions described by the blocks may be performed in an order different from the order listed. For example, two blocks listed in sequence may be performed simultaneously or in reverse order.

[0031] In this specification, the words "unit", "module", etc. may refer to software components or hardware components, such as field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs) that can perform functions or operations. However, "units", etc. are not limited to hardware or software. Units, etc. may be configured to reside in an addressable storage medium or drive one or more processors. Units, etc. may refer to software components, object-oriented software components, class components, task components, procedures, functions, attributes, processes, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays, or variables. The functions provided by components and units may be combinations of smaller components and units, and may be combined with other components to form larger components and units. Components and units may be configured to drive devices or one or more processors in a secure multi-media card.

[0032] The specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] like Figure 1 As shown, it shows a distributed system hierarchical speed limiting method based on real-time load perception, including the following steps: Step 1, an initialization step, for configuring system basic parameters and load data collection cycle; Specifically, the basic parameters include basic information such as the number of CPU cores, the number of transactions per second (TPS) gear, and the initial speed limit parameters.

[0034] In a distributed system, different methods can be used to obtain the number of CPU cores according to different operating systems. Since the acquisition method belongs to the scope of existing technology, the method of obtaining the number of CPU cores is not introduced in detail here. After obtaining the number of CPU cores, it is used as the reference value for subsequent calculation of the threshold value of the standardized indicator.

[0035] In the distributed system speed limiting technology, TPS (transactions per second) is an important indicator to measure the system processing capacity. It represents the number of transactions that the system can process per unit time, and is of great significance for evaluating system performance and formulating speed limiting strategies.

[0036] Generally speaking, the higher the TPS level, the stronger the system's processing power is, and the more transactions it can process in a shorter time. In a distributed system, through a reasonable rate limiting strategy, the client's request rate can be limited according to the system's TPS level, thereby ensuring the stability and availability of the system.

[0037] For example, if the TPS level of a distributed system is 1000, then when designing a rate limit strategy, you can ensure that the number of requests processed per second does not exceed 1000 to avoid system overload and performance degradation. At the same time, by monitoring the system's TPS level, you can promptly discover and handle performance bottlenecks and further optimize the system's performance and rate limit strategy.

[0038] The embodiment of the present invention sets a series of TPS gears, such as 120, 60, 40, ..., 2, 1, to meet the processing requirements under different loads.

[0039] Set the load data collection cycle and use the system's timer function or configure a scheduled task to ensure that load data is automatically collected every 30 seconds.

[0040] Step 2: Receive a service request, identify and distinguish a read request, a write request, and a service priority in the service request.

[0041] Step 3: Periodically collect the load data of distributed nodes and calculate the standardized index threshold.

[0042] Periodically call the system API, such as the proc / loadavg file or a dedicated monitoring interface to obtain system load data. Taking the PostgreSQL database as an example, after installing the system_stat plug-in, execute the SQL query SELECT * FROM pg_sys_os_info() to obtain detailed load information of the database server.

[0043] After obtaining the system load data, the following standardized indicator thresholds are calculated based on the obtained load data and the number of CPU cores: 1-minute indicator threshold: threshold_1m=load_1m / number of CPU cores; 5-minute indicator threshold: threshold_5m=load_5m / number of CPU cores; 10-minute indicator threshold: threshold_10m=load_10m / number of CPU cores.

[0044] These indicator thresholds will be used for subsequent load status judgment. load_1m, load_5m, and load_10m are the server loads in the last 1 minute, 5 minutes, and 10 minutes respectively.

[0045] Step 4: determine the load status in combination with the standardized indicator threshold.

[0046] First, the load state is defined as three conditions: healthy state, sub-healthy state, and unhealthy state. The criteria for judging each condition are as follows: Healthy status: threshold_5m<1 and threshold_10m<1, indicating that the system load is low and resources are sufficient; Sub-health status: threshold_5m≥1 or threshold_10m≥1, indicating that the system load is increasing and needs close attention; Unhealthy state: threshold_5m>1 and threshold_10m>1, indicating that the system load is too high and may face resource bottlenecks.

[0047] The current state of the system is automatically determined based on the calculated load indicator threshold.

[0048] Step 5: dynamically adjust the transactions per second (TPS) level according to the load status.

[0049] Dynamically adjusting the number of transactions per second (TPS) gear according to the load status specifically includes: When the load status is healthy, the TPS gear is automatically increased by 1 level to fully utilize system resources and improve throughput.

[0050] When the load status is sub-healthy, the TPS gear is reduced by 1 level to reduce system pressure and prevent the load from further increasing.

[0051] When the load status is unhealthy, the TPS gear is urgently reduced by 2 levels to quickly reduce the system load and ensure stable operation of core functions.

[0052] Step 6, use the token bucket algorithm to dynamically adjust the current maximum number of tokens in the current token bucket of the system according to the current TPS gear and allocate tokens on demand, and check whether there are enough tokens in the current token bucket based on the received business request. If there are enough tokens, allow the business request to pass; otherwise, reject the business request or queue the business request.

[0053] The principle of the token bucket algorithm is to control the rate of data traffic through a "bucket". A certain number of "tokens" are stored in the bucket, and each token represents the permission to send a data packet or the permission to process a request. Tokens are generated at a fixed rate and added to the bucket, and the sending of data packets or the processing of requests requires the consumption of a corresponding number of tokens. If there are not enough tokens in the bucket, the data packet or request will be rejected or queued until enough tokens are available.

[0054] In traffic control, the advantages of the token bucket algorithm mainly include the following points: ‌Allow burst traffic‌: Since tokens can be accumulated in the bucket, the system can allow burst traffic to be transmitted when needed, effectively coping with traffic fluctuations in actual scenarios. This feature makes the token bucket algorithm more flexible in handling burst requests or data packets.

[0055] ‌Efficient resource utilization‌: The token bucket algorithm ensures efficient resource utilization by generating tokens at a constant rate and accumulating them in the bucket. When there are fewer requests or data packets, tokens will accumulate in the bucket, providing sufficient processing power for subsequent bursts or peak hours, thus avoiding waste of resources‌.

[0056] ‌Limiting the maximum flow rate‌: By setting the token generation rate and bucket capacity, the token bucket algorithm can effectively limit the maximum flow rate and prevent the system from crashing due to overload. This limitation ensures the stability and reliability of the system‌.

[0057] No need to release resources: Compared with some current limiting mechanisms that require acquiring and releasing resources, the token bucket algorithm does not need to release resources after request processing is completed, thus avoiding current limiting problems caused by resource release failure. This feature makes the token bucket algorithm more stable and reliable when dealing with complex business scenarios.

[0058] The token bucket algorithm can achieve precise control of network traffic and avoid network congestion and collapse by setting the capacity of the token bucket and the token generation rate.

[0059] Specifically, using the token barrel algorithm to control traffic includes: setting the maximum capacity of the token barrel to 120 tokens. According to the TPS gear, the number of tokens that can be obtained for each business request is predefined, and the token number range is selected from the set {2, 3, 4, 5, 6, 7, 8, 10, 12, 15, 20, 30, 60, 120}. According to the TPS gear adjusted in step 5, the maximum number of tokens in the token barrel is dynamically updated and tokens are allocated on demand. When a business request is received, check whether there are enough tokens in the current token barrel. If there are enough tokens, take out the corresponding number of tokens and process the business request. If there are insufficient tokens, process the business request according to a preset strategy (such as queuing or rejection).

[0060] The step 6 further includes monitoring and analyzing the read-write ratio in the service request, and performing differentiated rate limit control in combination with the read-write ratio and the service priority.

[0061] Based on the read requests and write requests in the business requests identified in step 2, the read-write ratio in the business area request is analyzed, and differentiated speed limit control is performed in combination with the read-write ratio and the business priority, specifically: a stricter speed limit control is implemented on the write request than on the read request. The speed limit control of the write request can be dynamically adjusted according to factors such as system load, business priority, read-write ratio, etc., to achieve more refined traffic control and resource allocation.

[0062] like Figure 2 As shown, it shows a distributed hierarchical speed limiting device based on real-time load perception, including: An initialization module, wherein the initialization module is configured to set the basic parameters of the system and a load data collection period.

[0063] Specifically, the basic parameters include basic information such as the number of CPU cores, the number of transactions per second (TPS) gear, and the initial speed limit parameters.

[0064] In a distributed system, different methods can be used to obtain the number of CPU cores according to different operating systems. Since the acquisition method belongs to the scope of existing technology, the method of obtaining the number of CPU cores is not introduced in detail here. After obtaining the number of CPU cores, it is used as the reference value for subsequent calculation of the threshold value of the standardized indicator.

[0065] The embodiment of the present invention sets a series of TPS gears, such as 120, 60, 40, ..., 2, 1, to meet the processing requirements under different loads.

[0066] Set the load data collection cycle and use the system's timer function or configure a scheduled task to ensure that load data is automatically collected every 30 seconds.

[0067] A receiving module receives a service request, identifies and distinguishes a read request, a write request and a service priority in the service request.

[0068] A load data collection module, wherein the load data collection module periodically collects load data of distributed nodes and calculates a standardized index threshold.

[0069] Periodically call the system API, such as the proc / loadavg file or a dedicated monitoring interface to obtain system load data. Taking the PostgreSQL database as an example, after installing the system_stat plug-in, execute the SQL query SELECT * FROM pg_sys_os_info() to obtain detailed load information of the database server.

[0070] After obtaining the system load data, the following standardized indicator thresholds are calculated based on the obtained load data and the number of CPU cores: 1-minute indicator threshold: threshold_1m=load_1m / number of CPU cores; 5-minute indicator threshold: threshold_5m=load_5m / number of CPU cores; 10-minute indicator threshold: threshold_10m=load_10m / number of CPU cores.

[0071] These indicator thresholds will be used for subsequent load status judgment. load_1m, load_5m, and load_10m are the server loads in the last 1 minute, 5 minutes, and 10 minutes respectively.

[0072] A load state judgment module is used to judge the load state in combination with the standardized index threshold.

[0073] First, the load state is defined as three conditions: healthy state, sub-healthy state, and unhealthy state. The criteria for judging each condition are as follows: Healthy status: threshold_5m<1 and threshold_10m<1, indicating that the system load is low and resources are sufficient; Sub-health status: threshold_5m≥1 or threshold_10m≥1, indicating that the system load is increasing and needs close attention; Unhealthy state: threshold_5m>1 and threshold_10m>1, indicating that the system load is too high and may face resource bottlenecks.

[0074] The current state of the system is automatically determined based on the calculated load indicator threshold.

[0075] A dynamic adjustment module is used to dynamically adjust the number of transactions per second (TPS) according to the load status.

[0076] Dynamically adjusting the number of transactions per second (TPS) gear according to the load status specifically includes: When the load status is healthy, the TPS gear is automatically increased by 1 level to fully utilize system resources and improve throughput.

[0077] When the load status is sub-healthy, the TPS gear is reduced by 1 level to reduce system pressure and prevent the load from further increasing.

[0078] When the load status is unhealthy, the TPS gear is urgently reduced by 2 levels to quickly reduce the system load and ensure stable operation of core functions.

[0079] A speed limit execution module, which uses a token bucket algorithm to dynamically adjust the current maximum number of tokens in the current token bucket of the system according to the current TPS gear and allocates tokens on demand, and checks whether there are enough tokens in the current token bucket based on the received business request. If there are enough tokens, the business request is allowed to pass; otherwise, the business request is rejected or queued.

[0080] Specifically, using the token barrel algorithm to control traffic includes: setting the maximum capacity of the token barrel to 120 tokens. According to the TPS gear, the number of tokens that can be obtained for each business request is predefined, and the token number range is selected from the set {2, 3, 4, 5, 6, 7, 8, 10, 12, 15, 20, 30, 60, 120}. According to the TPS gear adjusted in step 5, the maximum number of tokens in the token barrel is dynamically updated and tokens are allocated on demand. When a business request is received, check whether there are enough tokens in the current token barrel. If there are enough tokens, take out the corresponding number of tokens and process the business request. If there are insufficient tokens, process the business request according to a preset strategy (such as queuing or rejection).

[0081] The speed limit execution module further monitors and analyzes the read-write ratio in the service request, and performs differentiated speed limit control in combination with the read-write ratio and service priority.

[0082] The read-write ratio in the service area request is analyzed based on the read request and the write request in the service request identified in the receiving module, and differentiated speed limit control is performed in combination with the read-write ratio and the service priority, specifically: a more stringent speed limit control is implemented on the write request than on the read request. The speed limit control of the write request can be dynamically adjusted according to factors such as system load, service priority, read-write ratio, etc., to achieve more refined flow control and resource allocation.

[0083] The above-mentioned distributed hierarchical speed limit method and system based on real-time load perception can respond in real time, and the speed limit rules can be adjusted according to the real-time status of the backend resources to ensure efficient utilization and stable operation of the system. At the same time, it has strong adaptive capabilities. Under low load conditions, the speed limit strategy will automatically relax the request limit and improve throughput; under high load conditions, the system will tighten the flow control in time to avoid resource exhaustion. In addition, it can also achieve more fine-grained speed limit control, support multi-speed dynamic speed limit configuration, and combine business characteristics (such as read-write ratio) to achieve differentiated processing of different types of requests.

[0084] Through this dynamic flow control mechanism, the system not only effectively alleviates the pressure of high concurrency on backend resources, but also provides users with a more stable and reliable service experience. Not only effectively alleviates the pressure of high concurrency on backend resources, but also provides users with a more stable and reliable service experience.

[0085] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a computer program for executing the method shown in the flowchart.

[0086] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any method in the embodiment is implemented.

[0087] The computer-readable storage medium in the embodiments of the present invention can be understood by a person skilled in the art that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0088] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A distributed system hierarchical speed limiting method based on real-time load perception, characterized in that: include: Initialization step: configure the basic parameters of the distributed system and the load data collection cycle; Receive a service request, and identify and distinguish a read request, a write request, and a service priority in the service request; Periodically collect load data of distributed nodes and calculate standardized indicator thresholds; Determine the load status in combination with the standardized indicator threshold; Dynamically adjust the number of transactions per second (TPS) according to the load status; Using a token bucket algorithm to dynamically adjust the current maximum number of tokens in the current token bucket of the system according to the current TPS gear and allocate tokens on demand, based on the received service request, check whether there are enough tokens in the current token bucket, if there are enough tokens, allow the service request to pass; otherwise, reject the service request or queue the service request; Monitor and analyze the read-write ratio in the service request, and perform differentiated rate limit control in combination with the read-write ratio and the service priority.

2. The method according to claim 1, characterized in that The basic system parameters include: number of CPU cores, number of transactions per second (TPS) level, and initial speed limit parameters.

3. The method according to claim 1, characterized in that The TPS gears include the following levels: 120, 60, 40, 30, 24, 20, 17, 15, 12, 10, 8, 6, 4, 2, 1.

4. The method according to claim 1, characterized in that: The key parameters of the token bucket algorithm include: the maximum number of tokens in the token bucket and the number of tokens obtained each time, the maximum number of tokens in the token bucket is 120, and the number of tokens obtained each time is selected from the set {2, 3, 4, 5, 6, 7, 8, 10, 12, 15, 20, 30, 60, 120}.

5. The method according to claim 1, characterized in that The system API or dedicated monitoring interface is periodically called to obtain load data, and standardized indicator thresholds for 1 minute, 5 minutes, and 10 minutes are calculated.

6. The method according to claim 5, characterized in that The standardized indicator threshold is calculated in the following manner: 1-minute indicator threshold = 1-minute load / CPU core number; 5-minute indicator threshold = 5-minute load / CPU core number; 10-minute indicator threshold = 10-minute load / CPU core number.

7. The method according to claim 1, characterized in that The load status includes: a healthy state is a 5-minute index threshold <1 and a 10-minute index threshold <1; a sub-healthy state is a 5-minute index threshold ≥1 or a 10-minute index threshold ≥1; an unhealthy state is a 5-minute index threshold >1 and a 10-minute index threshold >1.

8. The method according to claim 7, characterized in that The dynamic adjustment of the transactions per second (TPS) gear includes: increasing the TPS gear by 1 level when the load state is the healthy state; decreasing the TPS gear by 1 level when the load state is the sub-healthy state; decreasing the TPS gear by 2 levels when the load state is the unhealthy state. 9 . The method according to claim 1 , further comprising performing a stricter rate limit control on the write request.

10. A distributed system hierarchical speed limiting device based on real-time load perception, characterized in that: include: An initialization module, which configures basic parameters of the distributed system and a load data collection period; A receiving module, the receiving module receives a service request, identifies and distinguishes a read request, a write request and a service priority in the service request; A load data collection module, wherein the load data collection module periodically collects load data of distributed nodes and calculates a standardized index threshold; A load state judgment module, wherein the load state judgment module judges the load state in combination with the standardized index threshold; A dynamic adjustment module, wherein the dynamic adjustment module dynamically adjusts the number of transactions per second (TPS) gear according to the load state; A speed limit execution module, which uses a token bucket algorithm to dynamically adjust the current maximum number of tokens in the current token bucket of the system according to the current TPS gear and allocates tokens on demand, and checks whether there are enough tokens in the current token bucket based on the received service request. If there are enough tokens, the service request is allowed to pass; otherwise, the service request is rejected or queued; The speed limit execution module further monitors and analyzes the read-write ratio in the service request, and performs differentiated speed limit control in combination with the read-write ratio and the service priority.

Images