Flow control method and related equipment

Abstract

The invention provides a flow control method, which is applied to a flow control system, the system comprises a token client deployed in a service server and a token server, and the method comprises the following steps: the token client asynchronously pulls tokens from the token server, and when the number of available tokens of the token client is less than or equal to a threshold value, the token client sends the tokens to the token server; and the token client receives a token deduction request triggered by the service server according to the service request, and when the state of the token client is a non-flow-limiting state and the number of available tokens of the token client is greater than or equal to the number of the token deduction, the token deduction is carried out according to the number of the token deduction. And a flow control result for indicating the service release request is returned to the service server. According to the method, the token client side pulls the tokens in batches in an asynchronous mode, the influence of the token obtaining process on a service core link is reduced, and the requirements for low-delay, large-flow and high-throughput scenes can be effectively met.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 27, 2023, with application number 202311425043.X and invention name “A flow control method and related equipment”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of flow control technology, and in particular to a flow control method, a flow control system, a computing device cluster, a computer-readable storage medium, and a computer program product. Background Art

[0003] As the scale of business continues to expand, more and more users choose to deploy their business in distributed systems. In a distributed system, there are many different nodes and services that communicate and coordinate with each other. However, when the system load is too high or the resources are insufficient, the system performance may degrade or even crash. For this reason, global flow control (referred to as flow control) is usually required in distributed systems.

[0004] Global flow control is a technology used to manage system load and resource allocation. At present, the global flow control solution used in the industry is to deploy a centralized token server. When a server node in a distributed system, such as a broker, receives a service request, it requests a token from the token server. If the flow is not limited, the service request is passed. If the flow control value has been reached, the service request is rejected.

[0005] However, the communication delay between the broker and the token server directly affects the core link of the business request, and has a great impact on the system's request processing delay and throughput. Therefore, how to optimize the global flow control solution and implement a global flow control solution that can achieve low latency and high throughput in a distributed system is an urgent problem to be solved. Summary of the invention

[0006] The present application provides a flow control method, which pulls tokens in batches in an asynchronous manner. When a business request arrives at a business server and token deduction is required, if the status of the token client is non-current-limited, the token client can directly deduct the corresponding number of tokens from the local if there are sufficient local tokens, thereby reducing the impact of the token acquisition process on the business core link, and can effectively meet the low-latency, high-traffic, and high-throughput scenarios. In addition, the method supports adaptive adjustment of the number of tokens pulled to reduce the flow control loss caused by pulling tokens in batches. The present application also provides a flow control system, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0007] In a first aspect, the present application provides a flow control method. The method can be performed by a flow control system. The flow control system is used to control the flow of a distributed system. The distributed system includes multiple business servers, and the flow control system includes a token client and a token server deployed on the business server.

[0008] Specifically, the token client pulls tokens asynchronously from the token server, and continues to pull tokens asynchronously from the token server when the number of available tokens of the token client is less than or equal to a threshold. The number of available tokens of the token client is determined based on the number of tokens pulled from the token server and the number of tokens deducted by the token client (such as the number of tokens consumed). The token client can receive a token deduction request triggered by a business server based on a business request, and the token deduction request includes the number of tokens deducted. When the status of the token client is non-current limiting, the token client deducts tokens based on the token deduction number when the number of available tokens is greater than or equal to the token deduction number. Then, the token client returns a first flow control result to the business server, and the first flow control result indicates that the business request is released.

[0009] In this method, the token client pulls tokens in batches asynchronously, thereby pulling tokens in advance and reducing the impact of the token acquisition process on the core business link, which can effectively meet the scenarios of low latency, large traffic, and high throughput.

[0010] In some possible implementations, when the number of available tokens of the token client is less than the number of tokens to be deducted, the token client may also send a token pull request to the token server. The token client receives a token pull response returned by the token server, and the token pull response includes the actual number of pulled tokens. When the actual number of pulled tokens in the token pull response is equal to 0, the token client returns a second flow control result to the business server, and the second flow control result indicates that flow control is performed on the business request.

[0011] In this method, when the available token data is less than the deducted quantity, tokens continue to be pulled from the remote end. If the actual number of pulled tokens in the token pull response is 0, it means that the remote tokens are also completely consumed. The flow of business requests can be controlled to avoid excessive flow causing impact on the business server and affecting the normal operation of the business.

[0012] In some possible implementations, the token client can also configure the state to be current limiting. In the current limiting state, the token client can perform flow control on the service request, for example, rejecting the service request, or allowing some service requests to pass and rejecting other service requests. This method configures the state to be the current limiting state, so that when a new service request arrives, there is no need to perform complex operations, and the decision to perform flow control on the service request can be made directly, reducing the pressure on the token client.

[0013] In some possible implementations, the token client can also obtain the current timestamp. If the current timestamp exceeds the current limit time, the token client sends a token pull request to the token server. This method reduces invalid pulls after the token is consumed in the same time slice by pulling the token when the current timestamp exceeds the current limit time.

[0014] In some possible implementations, when the number of tokens actually pulled in the token pull response is greater than 0, the token client continues to send a token server pull request when the updated available token number is less than the token deduction number, until the updated available token number is greater than or equal to the token deduction number. In this way, when the local token is insufficient, tokens can be pulled from the remote token server for deduction to allow business requests to pass, without being limited to local tokens, and with higher availability.

[0015] In some possible implementations, when the token client does not receive the token pull response returned by the token server, the token client can also be configured to be invalid. When the token client is in invalid state, the flow control is downgraded, and when a new service request arrives, the service request can be directly released to meet the low latency, large traffic, and high throughput scenarios.

[0016] In some possible implementations, the number of tokens requested by the token client can be determined based on the flow control value of the token server, the number of nodes in the distributed system, and the flow control loss parameter. The flow control loss parameter is related to the business operation stage. This method adjusts the token batch pull value by adaptively adjusting the flow control loss parameter to reduce the flow control loss.

[0017] In some possible implementations, the flow control loss parameter may be determined as follows:

[0018] When the service operation stage is slow start, the current flow control loss parameter is adjusted by the first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter;

[0019] When the business operation phase is in a steady state, the flow control loss parameter is determined to be a first value;

[0020] When the service operation stage is backoff, the current flow control loss parameter is adjusted by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter.

[0021] This method draws on the idea of ​​congestion control in the Transmission Control Protocol (TCP), adjusts the flow control loss parameters according to the business operation stage, and then realizes adaptive adjustment of the number of tokens pulled to reduce the flow control loss of batch token acquisition. Based on the above control method, the theoretical limit loss can be controlled within 1%.

[0022] In some possible implementations, the traffic control system includes a token server cluster, and the token servers in the token server cluster synchronize a global online view. The global online view refers to the global view of the token servers in the token server cluster that are online. The token client can also query the token server cluster for the global online view, and the token client determines the token server that performs traffic control on the distributed system based on the global online view. This method implements global flow control without third-party dependencies by deploying token servers in a cluster deployment manner. Moreover, cluster deployment can reduce system complexity, improve maintainability, and make the token server cluster highly available, supporting vertical or horizontal expansion.

[0023] In some possible implementations, when a token server fails, the token client can also determine a new token server based on the global online view to control the flow of the distributed system. This can achieve rapid switching or recovery when a token server fails, ensuring business continuity and high availability.

[0024] In a second aspect, the present application provides a flow control system. The flow control system is used to control the flow of a distributed system, the distributed system includes multiple business servers, and the flow control system includes a token client and a token server deployed on the business server;

[0025] The token client is used to asynchronously pull tokens from the token server, and continue to asynchronously pull tokens from the token server when the number of available tokens of the token client is less than or equal to a threshold value, and the number of available tokens of the token client is determined according to the number of tokens pulled from the token server and the number of tokens deducted by the token client;

[0026] The token client is also used to receive a token deduction request triggered by a business server according to a business request, and the token deduction request includes a token deduction quantity. When the state of the token client is non-current limiting, when the number of available tokens is greater than or equal to the token deduction quantity, token deduction is performed according to the token deduction quantity, and a first flow control result is returned to the business server, and the first flow control result indicates that the business request is released.

[0027] In some possible implementations, the token client is further used to:

[0028] When the number of available tokens of the token client is less than the token deduction number, sending a token pull request to the token server;

[0029] Receive a token pull response returned by the token server, wherein the token pull response includes the actual number of pulled tokens;

[0030] When the number of tokens actually pulled in the token pull response is equal to 0, a second flow control result is returned to the business server, and the second flow control result indicates that flow control is performed on the business request.

[0031] In some possible implementations, the token client is further used to:

[0032] Configure the state to be current limiting.

[0033] In some possible implementations, the token client is further used to:

[0034] Get the current timestamp, and if the current timestamp exceeds the current limit time, send a token pull request to the token server.

[0035] In some possible implementations, the token client is further used to:

[0036] When the actual number of pulled tokens in the token pull response is greater than 0, and the updated number of available tokens is less than the token deduction number, continue to send token server pull requests until the updated number of available tokens is greater than or equal to the token deduction number.

[0037] In some possible implementations, the token client is further used to:

[0038] When the token pull response returned by the token server is not received, the token client configures the state as invalid.

[0039] In some possible implementations, the number of tokens requested by the token client is determined based on the flow control value of the token server, the number of nodes in the distributed system, and a flow control loss parameter, where the flow control loss parameter is related to the business operation stage.

[0040] In some possible implementations, the flow control loss parameter is determined in the following manner:

[0041] When the service operation stage is slow start, the current flow control loss parameter is adjusted by a first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter;

[0042] When the service operation phase is in a steady state, determining the flow control loss parameter to be a first value;

[0043] When the service operation phase is backoff, the current flow control loss parameter is adjusted by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter.

[0044] In some possible implementations, the traffic control system includes a token server cluster, and token servers in the token server cluster synchronize a global online view;

[0045] The token client is further used to query the token server cluster for the global online view, and determine the token server that performs flow control on the distributed system according to the global online view.

[0046] In some possible implementations, the token client is further used to:

[0047] When the token server fails, a new token server is determined based on the global online view to perform flow control on the distributed system.

[0048] In a third aspect, the present application provides a token client. The token client is deployed in a business server of a distributed system and is used to perform flow control on the distributed system. The token client includes:

[0049] A token pulling module, used to asynchronously pull tokens from a token server, and continue to asynchronously pull tokens from the token server when the number of available tokens of the token client is less than or equal to a threshold, wherein the number of available tokens of the token client is determined according to the number of tokens pulled from the token server and the number of tokens deducted by the token client;

[0050] An interaction module, configured to receive a token deduction request triggered by a business server according to a business request, wherein the token deduction request includes a token deduction amount;

[0051] The flow control module is used to, when the state of the token client is non-current-limited and the number of available tokens is greater than or equal to the number of token deductions, perform token deductions according to the token deduction number, and return a first flow control result to the business server, wherein the first flow control result indicates that the business request is released.

[0052] In a fourth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory, so that the computing device or the computing device cluster performs the flow control method as described in the first aspect or any implementation of the first aspect.

[0053] In a fifth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and the instructions instruct a computing device or a computing device cluster to execute the flow control method described in the first aspect or any implementation of the first aspect.

[0054] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device or a computing device cluster, enables the computing device or the computing device cluster to execute the flow control method described in the first aspect or any one of the implementations of the first aspect.

[0055] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.

[0057] Figure 1 A schematic diagram of the architecture of a flow control system provided in this application;

[0058] Figure 2 A flow chart of a flow control method provided by the present application;

[0059] Figure 3 A flow chart of a flow control method provided in this application;

[0060] Figure 4 A state transition diagram of a flow control state machine provided for this application;

[0061] Figure 5 A schematic diagram of the conversion of a business operation phase provided for this application;

[0062] Figure 6 A schematic diagram of a token server cluster synchronizing a global online view provided for this application;

[0063] Figure 7 A schematic diagram of the structure of a token client provided for this application;

[0064] Figure 8 A schematic diagram of the structure of a computing device provided for this application;

[0065] Fig. 9 A schematic diagram of the structure of a computing device cluster provided in this application. DETAILED DESCRIPTION

[0066] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0067] First, some technical terms involved in the embodiments of the present application are introduced.

[0068] A distributed system is a system composed of a group of computer nodes (such as server nodes, also referred to as nodes) that communicate through a network and coordinate work to complete a common task. A distributed system can include a variety of architectures, such as peer-to-peer, master-workers, or multi-tier. Different nodes in a distributed system can provide different services. Taking an e-commerce application scenario as an example, some nodes in a distributed system running an e-commerce business can provide user management services, some nodes can provide order management services, and some nodes can provide logistics management services. Furthermore, the nodes in a distributed system can also be in cluster mode. Still taking an e-commerce application scenario as an example, considering the huge order volume, the nodes providing order management services can be multiple nodes with the same functions, and the nodes with the same functions form a cluster, so that a distributed cluster can be formed.

[0069] Flow control, also known as flow control, refers to controlling the number of requests received by each node in a distributed system to ensure that the node can process the received requests in a timely manner and avoid a large number of requests being "blocked" in the server and waiting for execution.

[0070] At present, the widely used flow control solution in the industry is to deploy a centralized token server. The token server provides tokens. Business requests with tokens will be allowed, and business requests without tokens will be rejected. Server nodes in the distributed system, such as brokers, receive business requests and request tokens from the token server. If the flow is not limited, the business request is passed. If the flow control value has been reached, the business request is rejected.

[0071] However, the communication delay between the broker and the token server directly affects the core link of the business request, and has a great impact on the system's request processing delay and throughput. In addition, the flow control framework strongly relies on third-party components (such as Redis), which increases the complexity of the system and reduces maintainability.

[0072] In view of this, the present application provides a flow control method. The method can be performed by a flow control system. The flow control system is used to control the flow of a distributed system. The distributed system includes multiple business servers (for example, brokers), and the flow control system includes a token client (token client) deployed on the business server and a token server.

[0073] Specifically, the token client pulls tokens asynchronously from the token server, and continues to pull tokens asynchronously from the token server when the number of available tokens of the token client is less than or equal to a threshold value, wherein the number of available tokens of the token client is determined according to the number of tokens pulled from the token server and the number of tokens deducted by the token client (or referred to as the number of tokens consumed), for example, it may be the difference between the number of tokens pulled and the number of tokens consumed. The token client receives a token deduction request triggered by the business server according to a business request, and the token deduction request includes the number of tokens deducted. When the status of the token client is non-current limiting, the token client deducts tokens according to the token deduction number when the number of available tokens is greater than or equal to the token deduction number. Then, the token client returns a first flow control result to the business server, and the first flow control result indicates that the business request is released.

[0074] In this method, the token client pulls tokens in batches asynchronously, thereby pulling tokens in advance and reducing the impact of the token acquisition process on the core business link, which can effectively meet the low-latency, high-traffic, and high-throughput scenarios. In addition, this method supports dynamic and adaptive pulling of tokens, reduces the flow control loss of batch token acquisition, and can control the theoretical limit loss within an acceptable range (for example, within 1%). In addition, the token server can be deployed in a cluster without third-party dependencies, reducing system complexity, improving availability and maintainability, and supporting vertical and horizontal expansion, with good scalability.

[0075] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is first introduced below with reference to the accompanying drawings.

[0076] See also Figure 1The schematic diagram of the traffic control system shown in the figure, the traffic control system 100 is used to control the traffic of a distributed system 200, the distributed system 200 includes multiple business servers 202, and the traffic control system 100 includes a token client 102 and a token server 104 deployed on the business server. The token server 104 can be a token server 104 in a token server cluster. For ease of description, Figure 1 An example is given in which a distributed system includes two business servers 202 and a token server cluster includes three token servers 104 .

[0077] Specifically, the token client 102 is used to asynchronously pull tokens from the token server 104, and when the number of available tokens of the token client is less than or equal to the threshold, continue to asynchronously pull tokens from the token server 104. The number of available tokens of the token client is determined according to the number of tokens pulled from the token server 104 and the number of tokens deducted by the token client 102.

[0078] The token client 102 is also used to receive a token deduction request triggered by a business server based on a business request. The token deduction request includes the token deduction quantity. When the state of the token client 102 (for example, the flow control state, which can be the state of the flow control state machine maintained by the token client 102) is non-flow-limited, when the number of available tokens is greater than or equal to the token deduction quantity, token deduction is performed based on the token deduction quantity, and a first flow control result is returned to the business server, and the first flow control result indicates that the business request is released.

[0079] In some possible implementations, the token client 102 is also used to send a token pull request to the token server 104 when the number of available tokens is less than the number of token deductions, and receive a token pull response returned by the token server. The token pull response includes the actual number of tokens pulled. When the actual number of tokens pulled in the token pull response is equal to 0, a second flow control result is returned to the business server, and the second flow control result indicates that flow control (referred to as flow control) is performed on the business request. Among them, flow control of the business request can be to reject the business request, or flow control of the business request can be to limit the number or proportion of business requests released. For example, flow control of the business request can include rejecting a first proportion of business requests and releasing a second proportion of business requests. The first ratio and the second ratio can be set according to empirical values, for example, can be set to 50%. Further, the token client 102 can also be configured to be limited. When in the limited flow state, regardless of whether the token of the token client 102 is sufficient, the flow control result of the token client 102 indicates that flow control is performed on the business request, for example, rejecting the business request. In other examples, the token client 102 is used to continue sending token server pull requests when the actual number of pulled tokens in the token pull response is greater than 0 and the updated number of available tokens is less than the token deduction number, until the updated number of available tokens is greater than or equal to the token deduction number.

[0080] In some possible implementations, the token client 102 is also used to configure the state to be invalid when it does not receive a token pull response returned by the token server 104. The token client 102 not receiving a token pull response returned by the token server 104 may include multiple situations, one of which is that the token client 102 fails to send a token pull request, and the other is that the token client 102 successfully sends a token pull request, but the token server 104 fails to send a token pull response. When in the invalid state, regardless of whether the token client 102 has sufficient tokens, the token client 102 instructs to release the service request.

[0081] It should be noted that the flow control system 100 can be a software system, and correspondingly, the token client 102 and the token server 104 can be client software and server software. The software system can be distributed to users in the form of a software package and deployed by the users themselves. For example, the users can deploy it in a local data center or a private cloud. In some embodiments, the software system can also be provided to users in the form of a cloud service through a public cloud. Or the software system can also be deployed in a hybrid cloud and provided to users. In some possible implementations, the flow control system 100 can be a hardware system, such as a computing device cluster with flow control functions.

[0082] Based on the aforementioned flow control system 100, the present application provides a flow control method. The flow control method of the present application is introduced below in conjunction with the accompanying drawings.

[0083] See also Figure 2 A flow chart of a flow control method is shown, the method is applied to a flow control system 100, the flow control system 100 is used to control the flow of a distributed system 200, the flow control system includes a token client 102 and a token server 104 deployed on a business server 202, the method includes the following steps:

[0084] S202 : The token client 102 asynchronously pulls tokens from the token server 104 , and continues to asynchronously pull tokens from the token server 104 when the number of available tokens of the token client 102 is less than or equal to a threshold.

[0085] A token is also called a pass. Business requests with a token will be allowed, and business requests without a token will be rejected. The token can also be set with an expiration date. When the expiration date is exceeded, the token will become invalid or unavailable. The token server 104 can write a global flow control value, which can be the number of tokens. The token server 104 can write the global flow control value to the cache in the form of a key-value pair (KV). Specifically, the token server 104 can determine the key based on the instance identifier (ID) and timestamp of the distributed system. For example, the token server 104 can concatenate the instance ID and the timestamp as the key, and write the global flow control value to the cache as the value. The token server 104 can write the global flow control value periodically. For example, the token server 104 can write the global flow control value at a time interval of 1 second (s). The token client 102 can asynchronously pull the token from the token server 104 so that when the business server where the token client 102 is located receives a business request later, it can release the business request and process the business request in time.

[0086] Among them, asynchronous is relative to synchronous. In the present application, asynchronously pulling tokens means that there is no need to pull tokens when the business server receives a business request, but the tokens can be pulled in advance. When pulling tokens, multiple tokens can be pulled in batches at a time. In the present application, the token client 102 can monitor the number of available tokens locally. The number of available tokens is determined based on the number of tokens pulled from the token server 104 and the number of tokens deducted by the token client. For example, it can be equal to the difference between the number of tokens pulled and the number of tokens deducted. When the number of available tokens locally is less than or equal to the threshold, the token 102 can continue to pull tokens asynchronously from the token server 104. When the pull is successful, the token server 104 can subtract the number of tokens pulled this time from the global flow control value corresponding to the key in the cache.

[0087] The threshold value may be set based on empirical values. For example, the user may set the threshold value based on the statistical analysis results of historical data to reduce the number of times the token client 102 obtains a token from the token server 104 in real time when a service request arrives.

[0088] S204, the token client 102 receives a token deduction request triggered by the business server according to the business request. When the state of the token client 102 is non-current limiting, execute S206 and S208. When the state of the token client is current limiting, execute S210. When the state of the token client is invalid, execute S208.

[0089] Specifically, when a business server receives a business request, it can generate a token deduction request, for example, by calling a token deduction interface of a token client to generate a token deduction request. The token deduction request includes the token deduction amount. The token deduction request is used to request the deduction of a corresponding number of tokens so that the business request can be released and then executed in a timely manner.

[0090] When the token client 102 receives a token deduction request, the status of the token client 102 can be obtained. If it is in a non-current limiting state, S206 can be executed to deduct the token. If it is in a current limiting state or an invalid state, there is no need to execute S206 to deduct the token, but the flow control result is directly returned. For example, in the current limiting state, all service requests are rejected, and the token client 102 executes S210; for another example, in the invalid state, all service requests are allowed, and the token client 102 executes S208.

[0091] S206: When the number of available tokens is greater than or equal to the number of token deductions, the token client 102 deducts tokens according to the number of token deductions.

[0092] Specifically, the token client 102 obtains the number of available tokens, and then compares the number of available tokens with the number of token deductions. When the number of available tokens is greater than or equal to the number of token deductions, the token client 102 may deduct tokens according to the number of token deductions. The token client may then execute S208 to allow the service request to pass.

[0093] When the number of available tokens is less than the number of token deductions, the token client 102 may send a token pull request to the token server. If no token pull response is received, the token client 102 may configure the state to be invalid. Further, when the state is invalid, the current timestamp may be obtained. If the current timestamp exceeds the expiration time, the token client 102 may send a token pull request to the token server 104 to pull the token. This may reduce invalid pulls within the expiration time.

[0094] If the token client 102 receives a token pull response returned by the token server 104, the token pull response includes the actual number of pulled tokens, and the token client 102 can make the next decision based on the actual pulled token data.

[0095] Specifically, when the actual number of tokens pulled in the token pull response is equal to 0, it means that the token server 104 has no available tokens, and the token client 102 can execute S210 to return a second flow control result to the business server, and the second flow control result indicates the rejection of the business request. Furthermore, the token client 102 can also be configured to be a current limiting state. In the current limiting state, all business requests are rejected. Among them, the token client 102 can also obtain the current timestamp. If the current timestamp exceeds the current limiting time, the token client 102 can send a token pull request to the token server 104 to pull the token. In this way, invalid pulls after the token is consumed in the same time slice can be reduced.

[0096] When the actual number of pulled tokens in the token pull response is greater than 0, the token client 102 can continue to send the token server pull request when the updated available token number is less than the token deduction number, until the updated available token number is greater than or equal to the token deduction number. When the number of used tokens is greater than or equal to the token deduction number through at least one token pull operation, the token client 102 can execute S206 again.

[0097] S208. The token client 102 returns the first flow control result to the service server.

[0098] S210. The token client 102 returns a second flow control result to the service server.

[0099] The first flow control result indicates that the service request is released. The second flow control result indicates that the service request is subjected to flow control. Flow control of service requests may be to reject service requests, or to allow a first proportion of service requests to pass through and reject a second proportion of service requests. When flow control of service requests means allowing some service requests to pass through, an algorithm may be used to decide whether to release or reject the current service request. The above-mentioned flow control results may be indicated by different return values. For example, the token client 102 may indicate that a service request is released by returning true, and the token client 102 may indicate that a service request is rejected by returning false.

[0100] It should be noted that the above S210 is an optional step of the embodiment of the present application, and the flow control method of the embodiment of the present application may not execute S210. For example, in a non-current limiting state or a failure state, S210 may not be executed.

[0101] Based on the above description, the present application provides a flow control method, in which the token client 102 pulls tokens in batches asynchronously. When a business request arrives at the business server and token deduction is required, if the status of the token client 102 is non-current-limited, the token client 102 can directly deduct the corresponding number of tokens from the local if there are sufficient local tokens, thereby reducing the impact of the token acquisition process on the core business link, and can effectively meet the low latency, high traffic, and high throughput scenarios.

[0102] The flow control method of the present application is described below from an interactive perspective.

[0103] See also Figure 3 The flow chart of a flow control method shown in the figure, in this example, the flow control system includes a global token server cluster and a token client deployed with a broker (business server). Among them, the token server cluster is responsible for receiving requests from each token client in the distributed system instance, calculating and issuing token allocation, and the token client is responsible for communicating with the token server and pulling tokens. The method specifically includes the following steps:

[0104] ①The token server periodically writes the global flow control value to the cache.

[0105] Specifically, the token server can write the global flow control value in KV format at the set time interval. Figure 3In the example, the token server can write timestamps and corresponding global flow control values ​​at intervals of 1 second, such as (2023-06-29 15:00:28, 2000), (2023-06-29 15:00:29, 2000), and (2023-06-29 15:00:30, 2000).

[0106] ②The token client asynchronously pulls the token from the token server.

[0107] Among them, when pulling the token, the token client can carry the instance ID in the token pull request, recorded as instance_id. The token client can trigger asynchronous token pull through the application programming interface (API) with api_key as fetch_quota. For example, the token client can call the API with api_key as fetch_quota to generate a token pull request. The token pull request includes instance_id. Then the token client sends the above token pull request to the token server.

[0108] ③The token server returns an asynchronous token pull response to the client.

[0109] The asynchronous token pull response may include the number of tokens pulled. Furthermore, the asynchronous token pull response may also include the expiration time or timeout time of the token. Figure 3 As shown in the figure, the quota field indicates the number of tokens pulled, and the expired time indicates the expiration time. Before the expiration time, the token pulled this time is valid.

[0110] ④The broker receives the business request.

[0111] Specifically, a business request is a request for executing a business, such as a data production request or a data consumption request. A business request can be generated through an API of the corresponding business type. Figure 3 In the example, the service request may be generated by an API whose api_key is PRODUCE. The parameters of the API may include the amount of data.

[0112] ⑤The broker sends a token deduction request to the token client.

[0113] The token deduction request is used to deduct the token of the token client. Specifically, the broker can generate a token deduction request through the API used to deduct or delete tokens. For example, the broker can call deleteQuota() to generate a token deduction request and send the token deduction request to the token client.

[0114] ⑥The token client returns the flow control result to the broker.

[0115] The result of flow control can be to allow the business request to pass, or to reject the business request (flow control). The token client can combine the flow control state machine to decide the returned flow control result. The flow control state machine can include non-flow-limited, flow-limited or invalid state. In the non-flow-limited state, token acquisition is normal and the business request passes normally; in the flow-limited state, token acquisition is normal, but the business request cannot pass normally; in the invalid state, token acquisition is abnormal, flow control is unavailable, and the business request can pass normally. Among them, normal token acquisition means that it can interact with the token server and obtain the token pull response returned by the token server. Token acquisition abnormality means that the token pull response returned by the token server is not obtained, or the token pull response fails to be obtained.

[0116] The global flow control process is transformed around the above three main states. The state transformation process is illustrated below with reference to the accompanying drawings.

[0117] like Figure 4 As shown in the figure, in the invalid state, if the token acquisition fails, the invalid state remains; if the token acquisition succeeds but the token is insufficient, the state is converted to the current limiting state; if the token acquisition succeeds and the token is sufficient, the state is converted to the non-current limiting state. In the current limiting state, if the token acquisition fails, the state is converted to the invalid state; if the token acquisition succeeds and the token is sufficient, the state is converted to the non-current limiting state. In the non-current limiting state, if the local or remote token is sufficient, the non-current limiting state remains; if the token acquisition fails, the invalid state is converted; if the token acquisition succeeds but the token is insufficient, the current limiting state is converted.

[0118] When the status is non-current-limited, the token client can deduct the local token, specifically the token saved by the token client. If the local token is sufficient, for example, the number of local tokens is greater than or equal to the token deduction number, the service request is allowed. If the local token is insufficient, the token client can pull the token from the token server. When pulling the token, if the token is successfully obtained and the token is sufficient (greater than 0), it will remain in the non-current-limited state, and continue to pull tokens when the number of local tokens is still less than the token deduction number, until the number of local tokens is greater than or equal to the token deduction number.

[0119] When the status is limited, the token client can reject the service request. In addition, the token client can compare the current timestamp with the limited time. When the current timestamp exceeds the limited time, the token client can try to pull the token again to reduce invalid pulls after the token is consumed in the same time slice.

[0120] When the status is invalid, the token client can release the business request. In addition, the token client can compare the current timestamp with the expiration time. When the current timestamp exceeds the expiration time, the token client can try to pull the token again to reduce invalid pulls within the expiration time.

[0121] Among them, the setting of the token batch pull value is very critical, which will affect the accuracy of flow control, the efficiency of client pull and the pressure of token server access. This application also supports adaptive adjustment of the token batch pull value. The token batch pull value can be the number of tokens requested by the token client, for example, the number of tokens requested in a token pull request.

[0122] In some possible implementations, the number of tokens requested by the token client can be determined based on the flow control value of the token server, the number of nodes in the distributed system, and the flow control loss parameter. Among them, the flow control loss parameter is related to the business operation stage. Assuming that there are n brokers in a distributed system and the flow control value is X, then X / n is the average single-machine threshold. Because the traffic of brokers in a distributed system may be unbalanced, flow control loss may occur in extreme cases. For example, all business requests are scheduled to one of the n brokers, and the tokens obtained by the remaining n-1 brokers will expire, resulting in flow control loss. In this embodiment, it is assumed that the flow control loss ratio in the extreme case is 1 / w. Among them, w is the flow control loss parameter. The token value pulled in batches each time can be set to X / n / w.

[0123] In some possible implementations, the initial value of w can be set according to the business scenario and the test data under the corresponding business scenario. For example, the value of w can be set to 20-30, and accordingly, in the extreme case, the flow control loss is about 3%-5%. Drawing on the idea of ​​the Transmission Control Protocol (TCP) congestion control algorithm, w can be adaptively adjusted according to the business operation stage.

[0124] Specifically, when the business operation stage is slow start, the token client can adjust the current flow control loss parameter by the first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter. For example, the token client can reduce the flow control loss parameter and increase the batch pull value through operations such as subtraction or division. When the business operation stage is steady state, the token client can determine that the flow control loss parameter is a first value. The first value can be a fixed value. When the business operation stage is the backoff stage, the token client can adjust the current flow control loss parameter by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter. For example, the token client can increase the flow control loss parameter and reduce the batch pull value through operations such as addition or multiplication.

[0125] The transition of business operation phase can be found in Figure 5 .like Figure 5As shown, in the slow start phase, the tokenclient continues to send token pull requests (token fetch requests), and w is greater than or equal to 20. The token client can reduce w by division operation each time it pulls the token and increase the batch pull value. For example, the token client can determine the batch pull value according to w / =1.5. When w=20, and the total flow of the previous cycle (such as the previous second) (denoted as count) is greater than the number of tokens in a fetch request (denoted as fetch unit), the slow start phase can enter the steady state. In this phase, the token client can maintain w==20. When w is less than 100 and count is less than fetch unit, the slow start phase can enter the backoff phase. The backoff phase refers to the phase in which the flow control loss parameter is increased by multiplication or addition to reduce the batch pull value. Taking the multiplication backoff phase as an example, in this phase, the token client can increase w by multiplication operation each time it pulls the token and reduce the batch pull value. For example, the token client can determine the batch pull value according to w*=2.

[0126] In steady state, when the total traffic volume in the previous second is less than the number of tokens in a fetch request, and w < 100, the steady state can enter the backoff phase, such as the multiplication backoff phase. When the total traffic volume in the previous second is greater than the number of tokens in a fetch request, and w is greater than 20, the steady state can enter the slow start phase.

[0127] In the backoff phase, the token client can increase w by multiplication each time it pulls a token, and reduce the batch pull value. For example, the token client can determine the batch pull value based on w*=2. When w increases to more than 20, and the total traffic in the previous second is greater than the number of tokens in a fetch request, it can enter the slow start phase. When w=100, it can enter the steady state from the multiplication backoff phase. When entering the steady state, w can be kept at 100.

[0128] Relevant data shows that through the above-mentioned dynamic adaptive token pulling algorithm, the theoretical limit of token loss can be controlled within 1%.

[0129] To achieve global flow control, it is usually necessary to ensure that all business servers (such as Brokers) in a distributed system (such as a distributed cluster) are connected to the same Token Server and pull tokens from the same Token Server. Based on this, this application proposes a high-availability expansion deployment solution that is simple to implement and does not rely on three parties.

[0130] Specifically, the token servers in the token server cluster synchronize the global online view. Among them, the token servers can send heartbeat messages to each other, and synchronize the online status of the token servers through the heartbeat messages, so as to achieve synchronization of the global online view between the token servers. For example, see Figure 6 , token server1 can send a heartbeat message to token server2 and token server3. The heartbeat message carries the identifier of the token server that is reachable by token sever1, such as the identifier of token server2 and token server3. When token server2 receives the message, it can determine that token server3 is online through the heartbeat message sent by token server1, even if token server2 and token server3 are unreachable. In this way, token server2 can obtain a global online view. Similarly, other token servers can also obtain a global online view in the same way.

[0131] Based on graph theory, in a connected graph, there is only one graph with more than 1 / 2+1 nodes connected. In order to ensure that business servers in a distributed system are connected to the same token server, you can select a token server whose number of token servers in the local online view is more than 1 / 2+1. The local online view can be an online view established through the reachability of itself and other token servers. When the local online view exceeds 1 / 2+1, the token server is available.

[0132] Specifically, the token client 102 queries the token server cluster for a global online view. For example, the token client 102 can send a query request to the token server cluster through the fetchOnlineServer API to query the global online view, and then the token client 102 can determine the token server that performs flow control on the distributed system based on the global online view. The token client 102 can select the token server to which the business server is connected based on the consistent hash.

[0133] Furthermore, when the token server fails, the token client 102 can determine a new token server to control the flow of the distributed system based on the global online view. It should be noted that there is no need to synchronize data between token servers. When the connected token server fails, it only needs to switch to a new token server based on the global online view. When switching to a new token server, the distributed system can be registered on the new token server to provide a token for the distributed system.

[0134] Through the above cluster deployment method, global flow control without third-party dependence can be achieved, which reduces system complexity and improves maintainability. In addition, the token server cluster has high availability and supports vertical and horizontal expansion.

[0135] Based on the aforementioned flow control method, the present application also provides a flow control system. Figure 1 As shown, the traffic control system 100 is used to perform traffic control on a distributed system 200 . The distributed system 200 includes multiple business servers 202 . The traffic control system 100 includes a token client 102 and a token server 104 deployed on the business server 202 .

[0136] The token client 102 is used to asynchronously pull tokens from the token server 104, and when the number of available tokens of the token client 102 is less than or equal to a threshold, continue to asynchronously pull tokens from the token server 104, and the number of available tokens of the token client 102 is determined according to the number of tokens pulled from the token server 104 and the number of tokens deducted by the token client 102;

[0137] The token client 102 is also used to receive a token deduction request triggered by the business server 202 according to a business request. The token deduction request includes the token deduction quantity. When the status of the token client 102 is non-current-limited and the number of available tokens is greater than or equal to the token deduction quantity, the token is deducted according to the token deduction quantity, and the first flow control result is returned to the business server 202. The first flow control result indicates that the business request is released.

[0138] Exemplarily, the token client 102 and the token server 104 may be implemented by hardware or software.

[0139] When implemented by software, the token client 102 and the token server 104 may be applications running on a computing device, such as a computing engine. When implemented by hardware, the token client 102 and the token server 104 may include at least one computing device, such as a server. Alternatively, the token client 102 and the token server 104 may also be devices implemented using application-specific integrated circuits (ASICs) or programmable logic devices (PLDs). The PLDs may be complex programmable logical devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof.

[0140] In some possible implementations, the token client 102 is further configured to:

[0141] When the number of available tokens of the token client 102 is less than the token deduction number, a token pull request is sent to the token server 104;

[0142] Receive a token pull response returned by the token server 104, where the token pull response includes the actual number of pulled tokens;

[0143] When the actual number of tokens pulled in the token pull response is equal to 0, a second flow control result is returned to the business server 202, and the second flow control result indicates that flow control is performed on the business request.

[0144] In some possible implementations, the token client 102 is also used to configure the state to be current limiting. In the current limiting state, the token client 102 can reject all business requests, or the token client 102 allows some business requests to pass and rejects other business requests, thereby achieving flow control. For example, in the current limiting state, the token client 102 can allow 50% of the business requests to pass and reject 50% of the business requests. For another example, in the current limiting state, the token client 102 can reject 100% of the business requests.

[0145] In some possible implementations, the token client 102 is further configured to:

[0146] Get the current timestamp, and if the current timestamp exceeds the current limiting time, send a token pull request to the token server 104.

[0147] In some possible implementations, the token client 102 is further configured to:

[0148] When the actual number of pulled tokens in the token pull response is greater than 0, and the updated number of available tokens is less than the token deduction number, continue to send token server pull requests until the updated number of available tokens is greater than or equal to the token deduction number.

[0149] In some possible implementations, the token client 102 is further configured to:

[0150] When the token pull response returned by the token server 104 is not received, the token client 102 configures the state as invalid.

[0151] In some possible implementations, the number of tokens requested by the token client 102 is determined based on the flow control value of the token server 104, the number of nodes in the distributed system, and a flow control loss parameter, where the flow control loss parameter is related to the business operation stage.

[0152] In some possible implementations, the flow control loss parameter is determined in the following manner:

[0153] When the service operation stage is slow start, the current flow control loss parameter is adjusted by a first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter;

[0154] When the service operation phase is in a steady state, determining the flow control loss parameter to be a first value;

[0155] When the service operation phase is backoff, the current flow control loss parameter is adjusted by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter.

[0156] In some possible implementations, the traffic control system includes a token server cluster, and token servers in the token server cluster synchronize a global online view;

[0157] The token client 102 is further used to query the global online view from the token server 104 cluster, and determine the token server that performs flow control on the distributed system according to the global online view.

[0158] In some possible implementations, the token client 102 is further configured to:

[0159] When the token server 104 fails, a new token server is determined based on the global online view to perform flow control on the distributed system.

[0160] Based on the aforementioned flow control method, the present application also provides a token client 102. The token client 102 provided by the present application is introduced below from the perspective of functional modularization.

[0161] See also Figure 7 The schematic diagram of the structure of a token client 102 is shown, and the token client 102 is deployed in the business server 202 of the distributed system 200, and is used to control the flow of the distributed system 200. The token client 102 includes:

[0162] A token pulling module 1022, configured to asynchronously pull tokens from a token server, and continue to asynchronously pull tokens from the token server 104 when the number of available tokens of the token client 102 is less than or equal to a threshold value, wherein the number of available tokens of the token client 102 is determined according to the number of tokens pulled from the token server 104 and the number of tokens deducted by the token client 102;

[0163] The interaction module 1024 is used to receive a token deduction request triggered by the business server according to the business request, wherein the token deduction request includes a token deduction amount;

[0164] The flow control module 1026 is used to, when the status of the token client 102 is non-flow-limited and the number of available tokens is greater than or equal to the number of token deductions, perform token deductions according to the token deduction number, and return a first flow control result to the business server 202, wherein the first flow control result indicates that the business request is released.

[0165] The token pulling module 1022, the interaction module 1024, and the flow control module 1026 may be implemented by software or hardware. For ease of description, the token pulling module 1022 is used as an example for description below.

[0166] When implemented by software, the token pull module 1022 may be an application running on a computing device, such as a computing engine. The application may be provided to users in the form of virtualized services. Virtualized services may include virtual machine (VM) services, bare metal server (BMS) services, and container services. Among them, the VM service may be a service that uses virtualization technology to virtualize a virtual machine (VM) resource pool on multiple physical hosts to provide users with VMs on demand for use. The BMS service is a service that virtualizes a BMS resource pool on multiple physical hosts to provide users with BMS on demand for use. The container service is a service that virtualizes a container resource pool on multiple physical hosts to provide users with containers on demand for use. VM is a simulated virtual computer, that is, a logical computer. BMS is a high-performance computing service that can be elastically scalable, and its computing performance is no different from that of a traditional physical machine, and it has the characteristics of secure physical isolation. Containers are a kernel virtualization technology that can provide lightweight virtualization to achieve the purpose of isolating user space, processes, and resources. It should be understood that the VM service, BMS service and container service in the above-mentioned virtualization services are only specific examples. In actual applications, virtualization services can also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0167] When implemented by hardware, the token pulling module 1022 may include at least one computing device, such as a server, etc. Alternatively, the token pulling module 1022 may also be a device implemented by ASIC or PLD, etc. The PLD may be implemented by CPLD, FPGA, GAL or any combination thereof.

[0168] In some possible implementations, the token pulling module 1022 is further used to:

[0169] When the number of available tokens of the token client 102 is less than the token deduction number, a token pull request is sent to the token server 104;

[0170] Receive a token pull response returned by the token server 104, where the token pull response includes the actual number of pulled tokens;

[0171] The flow control module 1026 is also used to:

[0172] When the actual number of tokens pulled in the token pull response is equal to 0, a second flow control result is returned to the business server, and the second flow control result indicates that flow control is performed on the business request.

[0173] In some possible implementations, the flow control module 1026 is further configured to:

[0174] The configuration status is current limiting.

[0175] In some possible implementations, the token pulling module 1022 is further used to:

[0176] Get the current timestamp. If the current timestamp exceeds the current limit time, send a token pull request to the token server 104.

[0177] In some possible implementations, the token pulling module 1022 is further used to:

[0178] When the actual number of pulled tokens in the token pull response is greater than 0, the token server pull request continues to be sent when the updated number of available tokens is less than the token deduction number until the updated number of available tokens is greater than or equal to the token deduction number.

[0179] In some possible implementations, the flow control module 1026 is further configured to:

[0180] When the token pull response is not received from the token server, the token client configuration status is invalid.

[0181] In some possible implementations, the number of tokens requested by the token client is determined based on the flow control value of the token server, the number of nodes in the distributed system, and the flow control loss parameter, where the flow control loss parameter is related to the business operation stage.

[0182] In some possible implementations, the flow control module 1026 is further configured to determine the flow control loss parameter in the following manner:

[0183] When the service operation stage is slow start, the current flow control loss parameter is adjusted by the first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter;

[0184] When the business operation phase is in a steady state, the flow control loss parameter is determined to be a first value;

[0185] When the service operation stage is backoff, the current flow control loss parameter is adjusted by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter.

[0186] In some possible implementations, the traffic control system 100 includes a token server cluster, the token servers in the token server cluster synchronize a global online view, and the token client 102 further includes:

[0187] A query module 1028, configured to query the token server cluster for a global online view;

[0188] The determination module 1029 is used to determine the token server that performs flow control on the distributed system according to the global online view.

[0189] Among them, the query module 1028 and the determination module 1029 can be implemented by software or by hardware. When implemented by software, the query module 1028 and the determination module 1029 can be applications running on a computing device, such as a computing engine, etc. The application can be provided to users in the form of virtualization services. Virtualization services may include VM services, BMS services, and container services. When implemented by hardware, the query module 1028 and the determination module 1029 may include at least one computing device, such as a server, etc. Alternatively, the query module 1028 and the determination module 1029 may also be devices implemented using ASIC or PLD, etc.

[0190] In some possible implementations, the determination module 1029 is further configured to:

[0191] When the token server 104 fails, a new token server 104 is determined based on the global online view to perform flow control on the distributed system 200 .

[0192] The present application also provides a computing device 800. Figure 8 As shown, the computing device 800 includes: a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate through the bus 802. The computing device 800 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 800.

[0193] The bus 802 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 802 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 802 may include a path for transmitting information between various components of the computing device 800 (eg, the memory 806, the processor 804, and the communication interface 808).

[0194] The processor 804 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0195] The memory 806 may include a volatile memory, such as a random access memory (RAM). The memory 806 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD). The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the aforementioned flow control method. Specifically, the memory 806 stores instructions for the token client 102 to execute the flow control method.

[0196] The communication interface 808 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 800 and other devices or communication networks.

[0197] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0198] like Fig. 9 As shown, the computing device cluster includes multiple computing devices, for example, at least one computing device 800 and at least one computing device 900. The computing device 900 includes a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate through the bus 902. The memory 806 of at least one computing device 800 in the computing device cluster may store the same token client 102 for executing the flow control method, and the memory 906 of at least one computing device 900 in the computing device cluster may store the same token client 102 for executing the flow control method.

[0199] In some possible implementations, one or more computing devices 800 in the computing device cluster may also be used to execute some instructions of the token client 102 for executing the flow control method, and one or more computing devices 900 in the computing device cluster may also be used to execute some instructions of the token server 104 for executing the flow control method. In other words, the combination of at least one computing device 800 and at least one computing device 900 may jointly execute instructions of the flow control system 100 for executing the flow control method.

[0200] It should be noted that the memories in different computing devices in the computing device cluster may store different instructions for executing partial functions of the flow control system 100 .

[0201] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc. The computer-readable storage medium includes instructions that instruct the computing device to execute an application to the flow control system 100 for executing the flow control method.

[0202] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-mentioned flow control method.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A flow control method, characterized in that: Applied to a flow control system, the flow control system is used to control the flow of a distributed system, the distributed system includes multiple business servers, the flow control system includes a token client and a token server deployed on the business server, and the method includes: The token client asynchronously pulls tokens from the token server, and continues to asynchronously pull tokens from the token server when the number of available tokens of the token client is less than or equal to a threshold value, wherein the number of available tokens of the token client is determined according to the number of tokens pulled from the token server and the number of tokens deducted by the token client; The token client receives a token deduction request triggered by the service server according to the service request, wherein the token deduction request includes a token deduction amount; When the state of the token client is non-current limiting, the token client performs token deduction according to the token deduction number when the number of available tokens is greater than or equal to the token deduction number; The token client returns a first flow control result to the service server, where the first flow control result indicates that the service request is allowed.

2. The method according to claim 1, characterized in that: The method further comprises: When the number of available tokens of the token client is less than the token deduction number, the token client sends a token pull request to the token server; The token client receives a token pull response returned by the token server, wherein the token pull response includes the actual number of pulled tokens; When the number of tokens actually pulled in the token pull response is equal to 0, the token client returns a second flow control result to the business server, and the second flow control result indicates that flow control is performed on the business request.

3. The method according to claim 2, characterized in that The method further comprises: The token client obtains a current timestamp, and if the current timestamp exceeds the current limiting time, the token client sends a token pull request to the token server.

4. The method according to claim 2, characterized in that: Said also includes: When the actual number of pulled tokens in the token pull response is greater than 0, the token client continues to send token server pull requests when the updated number of available tokens is less than the token deduction number until the updated number of available tokens is greater than or equal to the token deduction number.

5. The method according to any one of claims 1 to 4, characterized in that: The number of tokens requested by the token client is determined based on the flow control value of the token server, the number of nodes in the distributed system and the flow control loss parameter, and the flow control loss parameter is related to the business operation stage.

6. The method according to claim 5, characterized in that The flow control loss parameter is determined by: When the service operation stage is slow start, the current flow control loss parameter is adjusted by a first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter; When the service operation phase is in a steady state, determining the flow control loss parameter to be a first value; When the service operation phase is backoff, the current flow control loss parameter is adjusted by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter.

7. The method according to any one of claims 1 to 6, characterized in that: The traffic control system includes a token server cluster, and the token servers in the token server cluster synchronize a global online view. The method further includes: The token client queries the token server cluster for the global online view; The token client determines a token server that performs flow control on the distributed system according to the global online view.

8. The method according to claim 7, characterized in that The method further comprises: When the token server fails, the token client determines a new token server to perform flow control on the distributed system based on the global online view.

9. A flow control system, characterized in that: The flow control system is used to perform flow control on a distributed system, the distributed system includes a plurality of business servers, and the flow control system includes a token client and a token server deployed on the business servers; The token client is used to asynchronously pull tokens from the token server, and continue to asynchronously pull tokens from the token server when the number of available tokens of the token client is less than or equal to a threshold value, and the number of available tokens of the token client is determined according to the number of tokens pulled from the token server and the number of tokens deducted by the token client; The token client is also used to receive a token deduction request triggered by a business server according to a business request, and the token deduction request includes a token deduction quantity. When the state of the token client is non-current limiting, when the number of available tokens is greater than or equal to the token deduction quantity, token deduction is performed according to the token deduction quantity, and a first flow control result is returned to the business server, and the first flow control result indicates that the business request is released.

10. The system according to claim 9, characterized in that The token client is also used to: When the number of available tokens of the token client is less than the token deduction number, sending a token pull request to the token server; Receive a token pull response returned by the token server, wherein the token pull response includes the actual number of pulled tokens; When the number of tokens actually pulled in the token pull response is equal to 0, a second flow control result is returned to the business server, and the second flow control result indicates that flow control is performed on the business request.

11. The system according to claim 10, characterized in that The token client is also used to: Get the current timestamp, and if the current timestamp exceeds the current limit time, send a token pull request to the token server.

12. The system according to claim 10, characterized in that The token client is also used to: When the actual number of pulled tokens in the token pull response is greater than 0, and the updated number of available tokens is less than the token deduction number, continue to send token server pull requests until the updated number of available tokens is greater than or equal to the token deduction number.

13. The system according to any one of claims 9 to 12, characterized in that: The number of tokens requested by the token client is determined based on the flow control value of the token server, the number of nodes in the distributed system and the flow control loss parameter, and the flow control loss parameter is related to the business operation stage.

14. The system according to claim 13, characterized in that The flow control loss parameter is determined by: When the service operation stage is slow start, the current flow control loss parameter is adjusted by a first adjustment amount, and the adjusted flow control loss parameter is less than the current flow control loss parameter; When the service operation phase is in a steady state, determining the flow control loss parameter to be a first value; When the service operation phase is backoff, the current flow control loss parameter is adjusted by the second adjustment amount, and the adjusted flow control loss parameter is greater than the current flow control loss parameter.

15. The system according to any one of claims 9 to 14, characterized in that The traffic control system includes a token server cluster, and the token servers in the token server cluster synchronize the global online view; The token client is further used to query the token server cluster for the global online view, and determine the token server that performs flow control on the distributed system according to the global online view.

16. The system according to claim 15, characterized in that The token client is also used to: When the token server fails, a new token server is determined based on the global online view to perform flow control on the distributed system.

17. A token client, characterized in that: The token client is deployed on a business server of a distributed system and is used to perform flow control on the distributed system. The token client includes: A token pulling module, used to asynchronously pull tokens from a token server, and continue to asynchronously pull tokens from the token server when the number of available tokens of the token client is less than or equal to a threshold, wherein the number of available tokens of the token client is determined according to the number of tokens pulled from the token server and the number of tokens deducted by the token client; An interaction module, configured to receive a token deduction request triggered by a business server according to a business request, wherein the token deduction request includes a token deduction amount; The flow control module is used to, when the state of the token client is non-current-limited and the number of available tokens is greater than or equal to the number of token deductions, perform token deductions according to the token deduction number, and return a first flow control result to the business server, wherein the first flow control result indicates that the business request is released.

18. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster executes the flow control method as described in any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: Comprising computer-readable instructions; the computer-readable instructions are used to implement the flow control method described in any one of claims 1 to 8.

20. A computer program product, characterized in that Comprising computer-readable instructions; the computer-readable instructions are used to implement the flow control method described in any one of claims 1 to 8.

Images