Flow control method and device, communication equipment and storage medium

Abstract

The invention relates to the technical field of communication, and discloses a flow control method and device, communication equipment and a storage medium, the method is applied to the communication equipment, the communication equipment can be a server side or a client side, and the method comprises the following steps: obtaining flow undertaking upper and lower limits of the server side and / or the client side; determining an upper limit and a lower limit of a flow control window according to the flow undertaking upper limit and the flow undertaking lower limit; adjusting the size of the flow control window within the upper limit and the lower limit of the flow control window based on the actual data volume in a first preset time window; wherein the first preset time window is the current preset time window, and the actual data volume is the actual received data volume or the actual sent data volume. According to the invention, the size of the flow control window is adjusted based on the actual data transceiving condition, so that the adjustment of the flow control window is more suitable for the current data transmission requirement.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a flow control method, device, communication equipment and storage medium. Background Art

[0002] With the development of computer and Internet technology, more and more service providers are facing huge access pressure and huge data traffic requests every day, which brings severe challenges to the normal operation of the system. Among them, network traffic control (can be referred to as flow control) and scheduling algorithms will determine the rate at which each data stream in the network is transmitted, which has a very important impact on network performance and application performance. The flow control problem is of great significance to both users and service providers. If the scheduling algorithm is too strict, the network data traffic will always run at a low speed, which will cause waste of network and system resources and affect the revenue of service providers. At the same time, the user's request will not be responded to for a long time. If the user's request is not scheduled in time, it will also affect the user's service experience and indirectly affect the income of the service provider. On the other hand, if the traffic is not controlled, it will lead to excessive consumption of system resources and network resources, which will cause network system anomalies, service system crashes, or cause some users to occupy bandwidth all the time, while other users' requests are always delayed in response, lacking service fairness, and ultimately affecting the service quality.

[0003] Traffic control is also important for backup services. Backup service systems generally use backup technology to copy source data to a disaster recovery center and create backup copies. The replication strategy can be full backup, incremental backup, or differential backup. In common backup service systems, data needs to be sent from the data source (production end) through the network to the backup server, which then processes the copy data and stores it in the disaster recovery repository. In addition, in a backup system, there are generally multiple backup clients, and each client is configured with multiple backup strategies (such as scheduled, real-time backup, etc.). Therefore, data transmission occurs at all times, and the pressure on the entire network system is also changing at any time. Therefore, how to control the network traffic of multiple client servers will affect the service quality and user experience of the entire backup system. Summary of the invention

[0004] In view of this, the present invention provides a flow control method, apparatus, communication equipment and storage medium to solve the problem that flow control cannot meet the current actual situation.

[0005] In a first aspect, the present invention provides a flow control method, which is applied to a communication device, wherein the communication device is a server or a client, and the method comprises:

[0006] Obtaining upper and lower limits of the traffic acceptance of the server and / or the client;

[0007] Determine the upper and lower limits of the flow control window according to the upper and lower limits of the flow acceptance;

[0008] Based on the actual data volume within the first preset time window, the size of the flow control window is adjusted within the upper and lower limits of the flow control window; wherein the first preset time window is the current preset time window, and the actual data volume is the actual received data volume or the actual sent data volume.

[0009] In some optional specific implementations, adjusting the size of the flow control window within the upper limit and the lower limit of the flow control window based on the actual data volume in the first preset time window includes:

[0010] Counting the actual data volume in the first preset time window and the actual data volume in the second preset time window;

[0011] If the actual data volume in the first preset time window is greater than the preset threshold, and the actual data volume in the first preset time window is greater than the actual data volume in the second preset time window, adjusting the size of the flow control window;

[0012] Among them, the second preset time window is the previous preset time window of the first preset time window, the duration of the first preset time window and the second preset time window is T, and the interval between the time starting point of the first preset time window and the time starting point of the second preset time window is t, and t is less than T.

[0013] In some optional specific implementations, adjusting the size of the flow control window includes:

[0014] If the actual data volume in the first preset time window is greater than the actual data volume in the previous preset time window when the flow control window was last adjusted, and the actual data volume in the first preset time window is greater than the expected data volume, and the amount of data processed by the communication device in the first preset time window is smaller than the amount of data processed by the second preset time window, then reduce the flow control window; otherwise, increase the flow control window;

[0015] The expected data volume is determined based on the current size of the flow control window.

[0016] In some optional specific implementations, the flow control method further includes:

[0017] Obtain a target token, where the target token is generated based on the current flow control window;

[0018] If the target token is obtained, data is sent or received based on the target token;

[0019] If the target token is not obtained, wait for obtaining the newly generated target token, where the target token is generated based on the data processing speed of the other end and the current size of the flow control window.

[0020] In some optional specific implementations, obtaining the upper and lower limits of the flow acceptance of the server and / or the client includes:

[0021] Determine the upper and lower limits of the traffic acceptance of the server based on the current load information of the server;

[0022] and / or,

[0023] Based on the current load information of the client, the upper and lower limits of the traffic acceptance of the client are determined.

[0024] In some optional specific implementations, determining the upper and lower limits of the flow acceptance of the server based on the current load information of the server includes:

[0025] Acquire values ​​of multiple current performance parameters of the server, where the performance parameters indicate the load information;

[0026] Determining a weight value of the server based on the value of the performance parameter;

[0027] Based on the weight value of the server, the upper limit of the traffic acceptance of the server is determined.

[0028] In some optional specific implementations, determining the weight value of the server based on the value of the performance parameter includes:

[0029] Obtaining values ​​of the performance parameters of the server and other servers in the same server cluster;

[0030] Determining a weight value of each performance parameter based on the values ​​of the performance parameters of the server and the other servers;

[0031] Based on the weight values ​​of the respective performance parameters and the value of the performance parameter of the server, the weight value of the server is determined.

[0032] In a second aspect, the present invention provides a flow control device, which is applied to a communication device, wherein the communication device is a server or a client, and the device comprises:

[0033] An acquisition module, used to acquire the upper and lower limits of the flow acceptance of the server and / or the client;

[0034] A determination module, used to determine the upper limit and lower limit of the flow control window according to the upper and lower limits of the flow acceptance;

[0035] An adjustment module is used to adjust the size of the flow control window within the upper and lower limits of the flow control window based on the actual data volume in the first preset time window; wherein the first preset time window is the current preset time window, and the actual data volume is the actual received data volume or the actual sent data volume.

[0036] In a third aspect, the present invention provides a communication device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the flow control method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the flow control method of the first aspect or any corresponding embodiment thereof.

[0038] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the flow control method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a schematic diagram of the overall system architecture of a backup service applicable to an embodiment of the present invention;

[0041] Figure 2 is a flow chart of a flow control method according to an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of a sliding window according to an embodiment of the present invention;

[0043] Figure 4 It is a flow chart of flow control coordination between a client and a server based on a token bucket according to an embodiment of the present invention;

[0044] Figure 5It is a flow chart of backup service flow control based on dynamic weight adjustment of performance parameter CRITIC according to an embodiment of the present invention;

[0045] Figure 6 is a structural block diagram of a flow control device according to an embodiment of the present invention;

[0046] Figure 7 It is a schematic diagram of the hardware structure of the communication device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0048] Backup service: generally refers to the use of backup software to perform regular or real-time data backup and protection. The data in the production center is backed up to the disaster recovery center for storage through full backup, incremental backup, differential backup and other means.

[0049] Traffic control: in the embodiments of the present invention, it refers to network traffic control, i.e., a measure to control network data traffic by software or hardware, so as to prevent the impact of a large amount of data in the network at once on the network and ensure efficient and stable operation of the user network.

[0050] There are many mature traffic control strategies and algorithms, such as the common counter algorithm, leaky bucket algorithm, token bucket algorithm, and the rate limiting tool class RateLimiter provided by the Google open source toolkit. However, these algorithms or strategies still have certain limitations when applied to backup service scenarios:

[0051] (1) Most of the current flow control rate limits are static and cannot be adjusted dynamically according to the working status of the server cluster. For example, the flow control window can be adjusted dynamically according to the service load, rather than relying on a pre-set static flow value limit.

[0052] (2) Currently, traffic control is generally performed at the network TCP / IP layer. There are few mature and ready-made traffic control strategies at the service application layer. In particular, there is no unified traffic control solution for backup service scenarios.

[0053] (3) There have been many related research works on optimizing various classic flow control algorithms and strategies, but most algorithms are relatively complex. For example, when applied in backup systems, they also require high system resources.

[0054] (4) The current flow control concept mainly focuses on flow control on the receiver of the service. However, in the backup business scenario, the backup client needs to send and receive data (corresponding to the backup and recovery scenarios respectively). Therefore, it is necessary to coordinate flow control on the client and the server synchronously.

[0055] Figure 1 The overall system architecture of a backup service applicable to an embodiment of the present invention is shown, and the main modules include: a backup client (i.e., CLI, client), a backup server (STS), a service management (MDS), a database (i.e., database, DB) and a storage library, etc. Among them, the flow control (i.e., FC) module mainly controls the network data flow control between the backup client and the backup server STS. In the backup business system, when the client backup software is installed, the client's CPU, bandwidth, memory and other information will be registered to the database. At the same time, when the server STS is deployed and installed, the host information will be automatically obtained, and the server's own CPU, bandwidth, memory and other host performance information will be synchronously recorded in the configuration file. After the client completes the first handshake with the network connection, the server obtains the client's universally unique identifier (Universally Unique Identifier, UUID), and estimates and initializes the maximum data sending window WS based on the client information in the database and its own performance. max and data receiving window WR max At the same time, the client is notified of the flow control window information during the handshake phase. In subsequent backup or recovery tasks, the client and server will use this window as the upper limit. When the client sends or receives data and exceeds the window upper limit, it will actively block the send or receive request and actively reduce the system's network traffic.

[0056] Because traffic is a dynamically changing indicator, the data sending or receiving window needs to be dynamically and adaptively adjusted according to the actual traffic situation and the business processing speed. A flow control strategy based on a sliding window is designed in the backup business scenario. The traditional sliding window strategy is as follows: Assuming that each minute is used as a data window, the amount of data received within the statistical time range of each window, and every 1 minute, the window automatically slides back 1 grid, and at the same time, a data traffic upper limit of, for example, 5 grids (5 minutes) is set. If the upper limit is exceeded, the service will be denied. The embodiment of the present invention improves the traditional sliding window service. When the data window upper limit is exceeded, the backup does not directly deny the service, but dynamically adjusts the size of the real-time flow control window according to the business processing situation, thereby realizing adaptive window adjustment.

[0057] According to an embodiment of the present invention, a flow control method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of executable computer instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] In this embodiment, a flow control method is provided, which can be used for a communication device, where the communication device is a server or a client. Figure 2 is a flow chart of a flow control method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0059] Step S201, obtaining the upper and lower limits of the traffic acceptance of the server and / or client.

[0060] The flow acceptance upper and lower limits may be the upper and lower limits of the flow control window, that is, the upper and lower limits of the flow control window of the server and / or the upper and lower limits of the flow control window of the client are obtained.

[0061] In some optional specific implementations, step S201, obtaining the upper and lower limits of the flow acceptance of the server and / or the client, includes:

[0062] Determine the upper and lower limits of the traffic acceptance of the server based on the current load information of the server;

[0063] and / or,

[0064] Based on the current load information of the client, the upper and lower limits of the traffic acceptance of the client are determined.

[0065] In the embodiments of the present invention, the upper and lower limits of the flow acceptance of the server are determined as needed, the upper and lower limits of the flow acceptance of the client are determined, or both the upper and lower limits of the flow acceptance of the server and the upper and lower limits of the flow acceptance of the client are determined.

[0066] In other embodiments, the server may determine its own upper and lower limits of traffic acceptance and send them to the client as needed, and the client may determine its own upper and lower limits of traffic acceptance and send them to the server as needed.

[0067] In some optional specific implementations, determining the upper and lower limits of the flow acceptance of the server based on the current load information of the server includes:

[0068] Step 1: Obtain the values ​​of multiple current performance parameters of the server, where the performance parameters indicate the load information; wherein the multiple performance parameters include CPU usage, memory occupancy, IO, network real-time traffic, etc.

[0069] Step 2: determining a weight value of the server based on the value of the performance parameter;

[0070] Step 3: Based on the weight value of the server, determine the upper limit of the traffic acceptance of the server.

[0071] The flow control strategy in the related art is to set the initial flow control window size and upper and lower limits according to the configuration of the communication equipment, which only considers the network and request processing speed, and does not consider the real-time load of the communication equipment in the working state. In fact, there will be a certain deviation between the system resource situation of the communication equipment in operation and the size of the flow control window, and the upper limit of the flow control window size initially set cannot adapt to the actual load capacity of the communication equipment. Therefore, it is also extremely important to adjust the upper and lower limits of the corresponding flow control window in real time according to the load of each communication equipment.

[0072] In an embodiment of the present invention, a weight value of the server is calculated based on the values ​​of multiple current performance parameters of the server, and the weight value can be used as an adjustment factor of the upper limit of the server's traffic acceptance to adjust the upper limit of the server's traffic acceptance.

[0073] Specifically, the weight value of the server can be calculated using the following formula:

[0074]

[0075] Among them, E i is the weight value of the i-th server, W1 is the weight factor of the CPU performance parameters of the i-th server, W2 is the weight factor of the memory performance parameters of the i-th server, W3 is the weight factor of the IO performance parameters of the i-th server, and W4 is the weight factor of the network traffic performance parameters of the i-th server.

[0076] The larger the weight value of the server, the more services it can undertake, the larger the flow control window can be allocated, and the upper limit of the flow control window can be adjusted; on the contrary, when the weight value of the server is smaller, it means that the load on the server is higher, and the data transmission and reception should be reduced, the flow control window should be reduced, and the system resource consumption should be reduced. In the backup service, the load of the server and the client will be monitored in real time, so as to realize the active or passive adjustment of the flow control window for data transmission and reception.

[0077] The embodiment of the present invention mainly performs calculation and analysis on four indicators: CPU, memory, IO, and bandwidth. For other business scenarios, more performance parameters can be added for analogical calculation. The corresponding flow control window size can be adjusted according to different load conditions.

[0078] In some optional specific implementations, the above step three, i.e., determining the weight value of the server based on the value of the performance parameter, includes:

[0079] Obtaining values ​​of the performance parameters of the server and other servers in the same server cluster;

[0080] Determining a weight value of each performance parameter based on the values ​​of the performance parameters of the server and the other servers;

[0081] Based on the weight values ​​of the respective performance parameters and the value of the performance parameter of the server, the weight value of the server is determined.

[0082] by Figure 1 Taking the backup service system shown as an example, the hardware configuration of each server (i.e., server) in the backup service cluster is fixed. The load monitoring thread can be used to monitor the performance parameters of the server in real time. Each performance parameter is assigned a certain weight value. The weight value of the server is obtained after calculation. The weight value of the server can be used as an increase or decrease factor for the upper limit of the flow control window. The STS flow control FC module adjusts the upper limit of the flow control window.

[0083] There are many different calculation schemes for calculating the weight values ​​of different performance parameters. The embodiment of the present invention mainly uses the CRITIC (Criteria Importance Through Intercrieria Correlation) objective evaluation method for analysis. The CRITIC method is an objective weight method for comprehensively measuring indicators based on the comparative strength of evaluation indicators and the conflict between indicators. It considers the variability of indicators while taking into account the correlation between indicators, and fully utilizes the objective attributes of the data itself for scientific evaluation. Its main steps are as follows:

[0084] (1) Construct the original matrix

[0085] Assuming there are n samples to be evaluated and p evaluation indicators, the original indicator matrix can be formed:

[0086]

[0087] For example, in the backup service, the load matrix of the three servers is:

[0088] server CPU Memory IO network STS1 0.9 0.8 0.8 0.7 STS2 0.3 0.5 0.3 0.4 STS3 0.1 0.2 0.1 0.6

[0089] (2) Dimensionless processing

[0090] Dimensionless processing means standardizing each indicator. The CRITIC weight method generally uses forward or reverse processing. Standardization is not recommended because if standardization is used, all standard deviations will become the number 1, which will make the volatility indicator meaningless.

[0091] If the index used is as large as possible, use the positive dimensionless processing:

[0092]

[0093] If the index used is as small as possible, use negative dimensionless processing:

[0094]

[0095] (3) Extraction of variability indicators

[0096] The standard deviation is used in CRITIC to indicate the fluctuation of the internal values ​​of each indicator. The larger the standard deviation, the greater the difference in the value of the indicator, the more information it can reflect, and the more weight should be assigned:

[0097]

[0098] (4) Extracting conflict indicators

[0099] The correlation coefficient is used in CRITIC to indicate the correlation between indicators. The stronger the correlation with other indicators, the less conflict the indicator has with other indicators. The more the same information is reflected, the more repetitive the evaluation content is. The weight assigned to this indicator should be reduced:

[0100]

[0101] (5) Calculate indicator weights

[0102] C j Represents the amount of information. The larger the amount of information, the greater the role of the jth evaluation index in the entire evaluation index system, and more weight needs to be allocated. Anyway, if the amount of information is smaller, the corresponding index is smaller; the jth index is calculated to get the weight W j :

[0103]

[0104] Among them, the weight of performance parameters can be adjusted dynamically. Figure 1Taking the backup business system shown as an example, multiple backup servers in operation can update their own load conditions to the database. The cluster's service management (MDS) node collects this data regularly and executes the CRITIC weight calculation process to dynamically adjust the weights of each load factor (i.e., performance parameter). The calculated weights are notified to each server, thereby affecting the window adjustment weight coefficient of the FC module of the STS cluster service as a whole, and adaptively realizing flow control regulation. In addition, the backup client can also monitor its own load pressure, refer to the CRITIC weight calculation process, and use fixed weights or dynamic weights to actively adjust the windows for sending and receiving data, thereby realizing active flow control based on the client load conditions.

[0105] In addition, the weight value of the server calculated in the above embodiment can be used not only to adjust the upper limit of the flow acceptance, but also to adjust the lower limit of the flow acceptance. For example, the weight value of the server is used as an adjustment factor of the lower limit of the flow acceptance. The weight value of the server can also be used to adjust the size of the flow control window, that is, as an adjustment factor of the flow control window, which is similar to the increase factor λ mentioned below. inc Or reduce the factor λ dec Adjust the size of the flow control window together.

[0106] Step S202, determine the upper and lower limits of the flow control window according to the upper and lower limits of the flow acceptance. The size of the flow control window is usually in bytes. In the TCP protocol, the size of the flow control window indicates the amount of data that the receiver can receive but has not yet confirmed, that is, the maximum amount of data that the sender can send without receiving confirmation.

[0107] Specifically, the upper and lower limits of the flow control window can be determined only based on the upper and lower limits of the flow acceptance of the server. If the upper and lower limits of the flow acceptance of the server are the upper and lower limits of the flow control window, then the upper and lower limits of the flow control window can be directly obtained. The upper and lower limits of the flow control window can also be determined only based on the upper and lower limits of the flow acceptance of the client. If the upper and lower limits of the flow acceptance of the client are the upper and lower limits of the flow control window, then the upper and lower limits of the flow control window can be directly obtained. The upper and lower limits of the flow control window can also be determined based on the upper and lower limits of the flow acceptance of the server and the client. At this time, the upper limit of the flow control window can be determined based on the smaller value of the flow acceptance upper limit in the server and the client, and the lower limit of the flow control window can be determined based on the larger value of the flow acceptance upper limit in the server and the client. If the flow acceptance upper and lower limits of the server are the upper and lower limits of the flow control window, and the flow acceptance upper and lower limits of the client are the upper and lower limits of the flow control window, then the smaller value of the upper limit of the flow control window of the server and the upper limit of the flow control window of the client is determined as the upper limit of the final flow control window, and the larger value of the lower limit of the flow control window of the server and the lower limit of the flow control window of the client is determined as the lower limit of the final flow control window.

[0108] Taking the communication device as the server as an example, at the beginning, the upper and lower limits of the flow control window can be determined only based on the upper and lower limits of the flow acceptance of the server. If the upper and lower limits of the flow acceptance sent by the client are received later, the upper and lower limits of the flow control window need to be adjusted based on the upper and lower limits of the flow acceptance of the client. Specifically, the server determines the upper and lower limits of the flow acceptance according to its own load conditions (i.e., load information), and the upper and lower limits of the flow acceptance are used as the upper and lower limits of the flow control window. At the same time, the client can also determine the upper and lower limits of the flow acceptance according to its own load conditions (i.e., load information), and the upper and lower limits of the flow acceptance are the upper and lower limits of the flow control window determined by the client. The client sends the upper and lower limits of the flow control window determined by itself to the server, and the server adjusts the upper and lower limits of the flow control window determined by the server itself according to the upper and lower limits of the flow control window sent by the client, and obtains the upper and lower limits of the final flow control window. That is, when the server dynamically adjusts the size of the flow control window according to its real-time load, the client also monitors the load of the local machine in real time. When the load is too high, it can actively reduce the flow control window to reduce resource pressure. In short, the size of the flow control window is dynamically adjusted according to the real-time load of the server and the client.

[0109] In the embodiment of the present invention, the upper and lower limits of the flow control window change with the load of the server and / or the client. The initial upper and lower limits of the flow control window can be determined according to the load of the server and / or the client, or in other ways. Figure 1 Taking the backup service system shown in FIG. 1 as an example, during the backup process, the server (STS) initializes the upper and lower limits of the flow control window according to the database (DB) and the configuration file. The initialization value of the flow control window can be W min +(W max -W min ) / 2. After obtaining the initialization value of the flow control window, the server synchronizes the initialization value to the client, and starts the adaptive flow control (FC) module corresponding to the client to monitor the actual data flow of the client.

[0110] When the client is used as the data receiving end, the upper limit and the upper limit determination process of the flow control window can be found above and will not be repeated here.

[0111] Step S103, based on the actual data amount in the first preset time window, adjust the size of the flow control window within the upper and lower limits of the flow control window; wherein the first preset time window is the current preset time window, and the actual data amount is the actual received data amount or the actual sent data amount.

[0112] In some optional specific implementations, step S203, i.e., adjusting the size of the flow control window within the upper limit and the lower limit of the flow control window based on the actual data volume in the first preset time window, includes:

[0113] Step S2031, counting the actual data volume in the first preset time window and the actual data volume in the second preset time window.

[0114] Step S2032: If the actual data volume in the first preset time window is greater than the preset threshold, and the actual data volume in the first preset time window is greater than the actual data volume in the second preset time window, the size of the flow control window is adjusted.

[0115] Among them, the second preset time window is the previous preset time window of the first preset time window, the duration of the first preset time window and the second preset time window is T, and the interval between the time starting point of the first preset time window and the time starting point of the second preset time window is t, and t is less than T.

[0116] like Figure 3 As shown in the figure, the preset time window is a sliding window with a duration of T. It slides once every t time. <T。

[0117] by Figure 1 Taking the backup service system shown in the figure as an example, the flow control module in the server STS counts the actual amount of data received w every t time period. i , and then get the amount of data received within n t time periods As the actual data volume of a preset time window (n×t=T), the actual data volume of the first preset time window is recorded as The actual data volume of the second preset time window is recorded as So, if Greater than and If the value is greater than the preset threshold, it is determined that the size of the flow control window needs to be adjusted, otherwise the size of the flow control window does not need to be adjusted. The preset threshold can be a constant pre-set by the system for adjusting the size of the flow control window: SEGMENT_SIZE. In the embodiment of the present invention, the client uses a token to implement an active flow control strategy for sending or receiving.

[0118] In some optional specific implementations, step S2032, i.e., adjusting the size of the flow control window, includes:

[0119] If the actual data volume within the first preset time window is greater than the actual data volume within the previous preset time window when the flow control window was last adjusted, and the actual data volume within the first preset time window is greater than the expected data volume, and the amount of data processed by the communication device in the first preset time window is smaller than the amount of data processed in the second preset time window, then reduce the flow control window; otherwise, increase the flow control window; the increased or reduced flow control window takes effect at the next preset time window.

[0120] The expected data volume is determined based on the current size of the flow control window. Specifically, the expected data volume may be equal to the current size of the flow control window.

[0121] Specifically, the flow control window that is increased or decreased can be adjusted according to the preset increase factor λ of the flow control window. inc (also called the expansion factor, for example 2) or the reduction factor λ dec (It may also be called a reduction factor, such as 0.8.) Of course, the adjusted flow control window cannot exceed the upper and lower limits determined above.

[0122] The embodiment of the present invention uses sliding window adaptive real-time flow control, which can be applicable to backup services.

[0123] In some optional specific implementation manners, the flow control method provided by the embodiment of the present invention further includes:

[0124] Obtain a target token, where the target token is generated based on the current flow control window;

[0125] If the target token is obtained, data is sent or received based on the target token;

[0126] If the target token is not obtained, wait for obtaining the newly generated target token, where the target token is generated based on the data processing speed of the other end and the current size of the flow control window.

[0127] Specifically, if the target token cannot be obtained, it means that the target token is exhausted. At this time, the local end (which can be either the client or the server) needs to block and wait for the return of the previous request (the data processing result of the other end or the adjustment result of the flow control window), and then send or receive data based on the new target token generated by the data processing result of the other end or the adjustment result of the flow control window. In other words, the token in the embodiment of the present invention is not generated at a constant speed, nor is it issued at a constant speed by the other end, but depends on the data processing speed of the other end and the adaptive adjustment result of the flow control window, and has a strong dynamic adaptability.

[0128] In some specific implementations, the data receiving end adjusts the size of the flow control window, and then sends the adjusted flow control window size to the data sending end, so that the data sending end generates a token based on the adjusted flow control window size. If the communication device of the embodiment of the present invention is a data receiving end, then it is necessary to send the adjusted flow control window size to the data sending end, so that the data sending end generates a token based on the adjusted flow control window size. Figure 1 Take the backup service system shown in the figure as an example. Figure 4 As shown in the figure, the server performs the flow control window size adjustment, but the server does not directly and roughly reject the client's request that exceeds the window size, but synchronizes the adjusted flow control window size to the client, and the client and the server coordinate the same flow control window to send and receive data. Before sending data (packets), the client needs to get (Get) the token (TOKEN) generated based on the size of the flow control window from the token bucket to avoid data rejection or task failure. Every time the client (CLI) gets a part of the token from the token bucket, the token in the token bucket will be reduced. However, after the server processes the data received from the client, the server's flow control module (i.e., FC module) will issue (i.e., PUTTOKEN) the tokens of the corresponding data into the token bucket. In addition, the adjustment of the flow control window size (i.e., ResizeWindows) will also affect the number of tokens.

[0129] against Figure 1 The backup service system shown in Figure 5 As shown in the figure, the backup service uses a sliding window to implement a flow control strategy based on the business request volume and the processing volume. At the same time, the flow control window is synchronized on the client using a token bucket-like method to achieve two-way control between the flow control server and the client. In addition, the CRITIC method is used on the server to dynamically adjust the weight strategy of performance parameters, actively monitor performance parameters including CPU, memory, IO, network, etc., and dynamically adjust the weight of performance parameters, so as to adaptively increase or decrease the flow control window in real time, and implement a flow control strategy based on the host load. At the same time, an active flow control module is also introduced on the client. By monitoring the system load, the client's flow control window is expanded and contracted, and the sliding window flow size is adaptively changed to avoid the client's own system crash or resource waste, thereby improving the stability of the client and server programs and the reliability of the backup service as a whole.

[0130] Network traffic control is a vital link in various service systems. A service system without traffic control cannot provide stable, efficient and high-quality services. Among them, the traffic control scheduling strategy for the business scenario of the backup service can also improve the overall performance of the backup system, the stability of the system and the quality of service. The adaptive flow control scheduling scheme provided by the embodiment of the present invention is suitable for backup scenarios. First, based on the existing working architecture and implementation method of the backup service, with reference to the sliding window algorithm, a database (DB) is inserted during the early installation and deployment process to collect the performance parameters of the server (STS) and the client (CLI), calculate and estimate the upper limit of the traffic acceptance of the client and the server, and set the upper limit of the data receiving and sending window accordingly. At the same time, the STS flow control module counts the actual data transmission and reception and the back-end processing speed within the preset time range, and dynamically adjusts the real-time window size according to the data request flow. During the backup service STS service process, the load monitoring thread is responsible for monitoring the load factor of the local machine and synchronizing it to the database (DB). The control node, namely the service management (MDS), collects the service load situation, combines the CRITIC weight method to determine the weight of each parameter and synchronizes it to each STS service node. The backup service STS node adjusts the flow control window for data reception and transmission in real time according to the weight result. At the same time, the client can also actively change the flow control window dynamically according to its own load situation. In addition, after the real-time data flow control window changes, the STS service module and the client need to notify each other of the window changes, so as to coordinate the data receiving and sending window at the next moment.

[0131] Compared with the traditional network layer flow control scheme, the embodiment of the present invention proposes to perform business layer flow control based on the actual business situation, especially for the actual scenario of cloud backup, and designs a flow control scheme based on the backup service STS sliding window, which is more suitable for the cloud backup business scenario. Moreover, the flow control method provided by the embodiment of the present invention can be applied not only to the server, but also to the client, so that it can be applied not only in the backup process, but also in the backup recovery process to ensure the success of the backup and recovery tasks. Moreover, not only the performance of the server and the network request processing are considered, but also the performance of the client is paid attention to, so as to ensure the efficient and smooth progress of the backup and recovery tasks.

[0132] The embodiment of the present invention optimizes the traditional server-side unidirectional flow control strategy. In the backup scenario, bidirectional flow control is performed on the backup server and client at the same time. Bidirectional synchronous real-time flow control is performed based on a sliding window or token bucket to ensure the performance and efficiency of the backup service, while also avoiding or reducing the problem of backup task failure caused by denial of service due to flow control.

[0133] The embodiment of the present invention proposes that flow control not only needs to be adaptively adjusted based on network traffic and request processing speed, but also needs to pay attention to the host load situation for active flow control. It is proposed to monitor the load factors (CPU, memory, IO, bandwidth) of the server or client in real time, and use the CRITIC weight method to dynamically adjust the weight of the load factor. The load situation of the system is calculated by weight, so that the backup server or client can adaptively adjust the flow control window and flow limit, so as to avoid the crash of the program or system due to overload or the waste of host resources when it is idle.

[0134] In this embodiment, a flow control device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0135] This embodiment provides a flow control device, which is applied to a communication device, wherein the communication device is a server or a client, such as Figure 6 As shown, the device comprises:

[0136] An acquisition module 601 is used to acquire the upper and lower limits of the flow acceptance of the server and / or the client;

[0137] A determination module 602 is used to determine the upper limit and lower limit of the flow control window according to the upper and lower limits of the flow acceptance;

[0138] The adjustment module 603 is used to adjust the size of the flow control window within the upper and lower limits of the flow control window based on the actual data volume in the first preset time window; wherein the first preset time window is the current preset time window, and the actual data volume is the actual received data volume or the actual sent data volume.

[0139] In some optional implementations, the adjustment module 603 includes:

[0140] A statistical unit, configured to count the actual data volume in the first preset time window and the actual data volume in the second preset time window;

[0141] a flow control window adjustment unit, configured to adjust the size of the flow control window if the actual data volume in the first preset time window is greater than a preset threshold, and the actual data volume in the first preset time window is greater than the actual data volume in the second preset time window;

[0142] Among them, the second preset time window is the previous preset time window of the first preset time window, the duration of the first preset time window and the second preset time window is T, and the interval between the time starting point of the first preset time window and the time starting point of the second preset time window is t, and t is less than T.

[0143] In some optional implementations, the flow control window adjustment unit is specifically used to reduce the flow control window if the actual data volume in the first preset time window is greater than the actual data volume in the previous preset time window when the flow control window was last adjusted, and the actual data volume in the first preset time window is greater than the expected data volume, and the amount of data processed by the communication device in the first preset time window is smaller than the amount of data processed by the second preset time window; otherwise, increase the flow control window;

[0144] The expected data volume is determined based on the current size of the flow control window.

[0145] In some optional embodiments, the flow control device further comprises:

[0146] A token acquisition module, used to acquire a target token, wherein the target token is generated based on the current flow control window;

[0147] A transceiver module, configured to send or receive data based on the target token if the target token is acquired;

[0148] The waiting module is used to wait for obtaining the newly generated target token if the target token is not obtained, and the target token is generated based on the data processing speed of the other end and the current size of the flow control window.

[0149] In some optional implementations, the acquisition module 601 includes:

[0150] A first determining unit, configured to determine the upper and lower limits of the flow acceptance of the server based on the current load information of the server;

[0151] and / or,

[0152] The second determining unit is used to determine the upper and lower limits of the traffic acceptance of the client based on the current load information of the client.

[0153] In some optional implementations, the first determining unit includes:

[0154] A performance parameter value acquisition subunit, used to acquire values ​​of a plurality of current performance parameters of the server, wherein the performance parameters indicate the load information;

[0155] A server weight value determination subunit, configured to determine a weight value of the server based on a value of the performance parameter;

[0156] The traffic acceptance upper limit determination subunit is used to determine the traffic acceptance upper limit of the server based on the weight value of the server.

[0157] In some optional embodiments, the server weight value determination subunit is specifically used to obtain the values ​​of the performance parameters of the server and other servers in the same server cluster; determine the weight values ​​of each performance parameter based on the values ​​of the performance parameters of the server and other servers; determine the weight value of the server based on the weight values ​​of each performance parameter and the value of the performance parameter of the server.

[0158] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0159] The flow control device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0160] The embodiment of the present invention also provides a communication device having the above Figure 6 Flow control device shown.

[0161] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a communication device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the communication device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the communication device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple communication devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0162] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0163] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0164] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the communication device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the communication device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0166] The communication device further comprises a communication interface 30, which is used for the communication device to communicate with other devices or a communication network.

[0167] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0168] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0169] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A flow control method, characterized in that: Applied to a communication device, the communication device being a server or a client, the method comprises: Obtaining upper and lower limits of the traffic acceptance of the server and / or the client; Determine the upper and lower limits of the flow control window according to the upper and lower limits of the flow acceptance; Based on the actual data volume within the first preset time window, the size of the flow control window is adjusted within the upper and lower limits of the flow control window; wherein the first preset time window is the current preset time window, and the actual data volume is the actual received data volume or the actual sent data volume.

2. The method according to claim 1, characterized in that The adjusting the size of the flow control window within the upper limit and the lower limit of the flow control window based on the actual data volume in the first preset time window includes: Counting the actual data volume in the first preset time window and the actual data volume in the second preset time window; If the actual data volume in the first preset time window is greater than the preset threshold, and the actual data volume in the first preset time window is greater than the actual data volume in the second preset time window, adjusting the size of the flow control window; Among them, the second preset time window is the previous preset time window of the first preset time window, the duration of the first preset time window and the second preset time window is T, and the interval between the time starting point of the first preset time window and the time starting point of the second preset time window is t, and t is less than T.

3. The method according to claim 2, characterized in that The adjusting the size of the flow control window includes: If the actual data volume in the first preset time window is greater than the actual data volume in the previous preset time window when the flow control window was last adjusted, and the actual data volume in the first preset time window is greater than the expected data volume, and the amount of data processed by the communication device in the first preset time window is smaller than the amount of data processed by the second preset time window, then reduce the flow control window; otherwise, increase the flow control window; The expected data volume is determined based on the current size of the flow control window.

4. The method according to claim 1, characterized in that: Also includes: Obtain a target token, where the target token is generated based on the current flow control window; If the target token is obtained, data is sent or received based on the target token; If the target token is not obtained, wait for obtaining the newly generated target token, where the target token is generated based on the data processing speed of the other end and the current size of the flow control window.

5. The method according to claim 1, characterized in that The obtaining of the upper and lower limits of the flow acceptance of the server and / or the client includes: Determine the upper and lower limits of the traffic acceptance of the server based on the current load information of the server; and / or, Based on the current load information of the client, the upper and lower limits of the traffic acceptance of the client are determined.

6. The method according to claim 5, characterized in that The determining, based on the current load information of the server, the upper and lower limits of the flow acceptance of the server includes: Acquire values ​​of multiple current performance parameters of the server, where the performance parameters indicate the load information; Determining a weight value of the server based on the value of the performance parameter; Based on the weight value of the server, the upper limit of the traffic acceptance of the server is determined.

7. The method according to claim 6, characterized in that The determining the weight value of the server based on the value of the performance parameter includes: Obtaining values ​​of the performance parameters of the server and other servers in the same server cluster; Determining a weight value of each performance parameter based on the values ​​of the performance parameters of the server and the other servers; Based on the weight values ​​of the respective performance parameters and the value of the performance parameter of the server, the weight value of the server is determined.

8. A flow control device, characterized in that: Applied to a communication device, the communication device being a server or a client, the device comprises: An acquisition module, used to acquire the upper and lower limits of the flow acceptance of the server and / or the client; A determination module, used to determine the upper limit and lower limit of the flow control window according to the upper and lower limits of the flow acceptance; An adjustment module is used to adjust the size of the flow control window within the upper and lower limits of the flow control window based on the actual data volume in the first preset time window; wherein the first preset time window is the current preset time window, and the actual data volume is the actual received data volume or the actual sent data volume.

9. A communication device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the flow control method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the flow control method according to any one of claims 1 to 7.

Images