Flow control method and related equipment
By calculating the target concurrency degree during the scheduling cycle and the number of tasks in the task queue, dynamically control the task current limit threshold, the problem of poor flow control based on fixed QPS in the prior art is solved, and more effective task queue management is achieved and improve execution efficiency.
Patent Information
- Application Number
- CN202410732750.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-06
- Publication Date
- 2024-12-17
AI Technical Summary
The existing flow control method is based on a fixed number of response requests per second (QPS), which leads to poor current limiting effect in batch running systems, resulting in task queue accumulation, and tasks with high priority cannot be executed in a timely manner.
By calculating the target concurrency during the scheduling cycle and calculating the task current limit threshold based on the number of tasks in the task queue, the task queue is dynamically controlled to avoid the limitation of fixed QPS.
It improves the effect of traffic control, ensures the reasonable allocation and execution of tasks in the task queue, and avoids the extended data processing time and the accumulation of executor http thread pool due to the increase in the waiting time of the current limit.
Smart Images
Figure CN119155246A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a flow control method and related equipment. Background Art
[0002] The batch running system generally runs multiple tasks at the same time. Each task may depend on different downstream services, so the current limiting situations are also different. When the current limiting waiting time increases, the data processing time increases, resulting in the accumulation of the executor's http thread pool, which in turn causes the data to accumulate in the execution queue. At the same time, the task scheduler will continuously put tasks into the execution queue according to the number of calculations. Ultimately, the entire task queue is either occupied by tasks with long current limiting waiting times, or tasks with low current limiting and high priority cannot be executed in time because the execution queue is full. Therefore, it is necessary to control the flow of the task queue.
[0003] At present, a common method of flow control is to perform flow control based on QPS (number of response requests per second), but the value of QPS is a fixed value, which may result in poor flow control effect. Summary of the invention
[0004] The embodiments of the present application provide a flow control method and related equipment, which are beneficial to improving the flow control effect.
[0005] In order to solve the above technical problems, this application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides a flow control method, including:
[0007] Calculating a first target concurrency in a first scheduling period;
[0008] Calculate the task current limiting threshold according to the first target concurrency and the number of tasks in the task queue;
[0009] According to the task current limiting threshold, the number of tasks submitted to the task queue in the first scheduling period is controlled.
[0010] In a second aspect, an embodiment of the present application further provides a flow control device, comprising:
[0011] A first calculation module, used to calculate a first target concurrency in a first scheduling period;
[0012] A second calculation module is used to calculate a task current limiting threshold according to the first target concurrency and the number of tasks in the task queue;
[0013] A control module is used to control the number of tasks submitted to the task queue within the first scheduling period according to the task current limiting threshold.
[0014] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned flow control method when executing the computer program.
[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned flow control method are implemented.
[0016] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which implements the steps in the above-mentioned flow control method when executed by a processor.
[0017] It can be seen that in the flow control method of this embodiment, a fixed number of response requests per second (QPS) is no longer used for flow limiting. Instead, the target concurrency within the scheduling period is calculated, and then the target concurrency and the number of tasks in the task queue are used to calculate the task flow limiting threshold. According to the task flow limiting threshold, the number of tasks submitted to the task queue within the scheduling period is controlled to achieve flow limiting of the task queue. That is, this embodiment uses the target concurrency for flow limiting, and takes into account the number of tasks in the task queue. By determining the task flow limiting threshold based on the target concurrency and the number of tasks in the task queue, the task queue is limited to improve the flow control effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0019] Figure 1 is a flow chart of a flow control method provided by an embodiment of the present application;
[0020] Figure 2 is a structural schematic diagram of a flow control device provided in an embodiment of the present application;
[0021] Figure 3 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0023] See also Figure 1 , Figure 1 1 is a flow chart of a flow control method provided by an embodiment of the present application. The flow control method of this embodiment can be applied to electronic devices, which can be mobile devices or non-mobile devices. Figure 1 As shown, the following steps are included:
[0024] Step 101: Calculate a first target concurrency in a first scheduling period;
[0025] Concurrency can be understood as the number of tasks that the system can process simultaneously. In the embodiment of the present application, flow control can be performed in each scheduling cycle, and the corresponding target concurrency calculation is performed in each scheduling cycle. In the first scheduling cycle (for example, the current scheduling cycle), the first target concurrency in the cycle can be calculated first. The first target concurrency in the first scheduling cycle refers to the number of tasks that can be processed simultaneously in the first scheduling cycle. In addition, flow control can be understood as controlling the number of tasks placed in the task queue.
[0026] Step 102: Calculate the task current limiting threshold according to the first target concurrency and the number of tasks in the task queue;
[0027] Step 103: According to the task flow limiting threshold, the number of tasks submitted to the task queue in the first scheduling period is controlled.
[0028] The number of tasks can be understood as the number of tasks that have not been processed (i.e., to be processed) in the task queue. For example, the task flow limiting threshold of the first scheduling period can be the result of subtracting the number of tasks from the first target concurrency in the first scheduling period. After calculating the task flow limiting threshold, the task flow limiting threshold can be used to control the number of tasks submitted to the task queue in the first scheduling period, so as to realize the flow control of the task queue, so that the number of tasks in the task queue does not exceed the target concurrency.
[0029] In the flow control method of this embodiment, a fixed number of response requests per second (QPS) is no longer used for flow limiting. Instead, the target concurrency within the scheduling period is calculated, and then the target concurrency and the number of tasks in the task queue are used to calculate the task flow limiting threshold. According to the task flow limiting threshold, the number of tasks submitted to the task queue within the scheduling period is controlled to achieve flow limiting of the task queue. That is, this embodiment uses the target concurrency for flow limiting, and takes into account the number of tasks in the task queue. By determining the task flow limiting threshold based on the target concurrency and the number of tasks in the task queue, the task queue is flow limited to improve the flow control effect.
[0030] In one embodiment, calculating a first target concurrency in a first scheduling period includes:
[0031] Calculate the waiting time of tasks in the first scheduling cycle;
[0032] Obtaining a second target concurrency of a second scheduling period, where the second scheduling period is a scheduling period before the first scheduling period;
[0033] A first target concurrency in a first scheduling period is determined according to the waiting time and the second target concurrency.
[0034] Exemplarily, the waiting time may be an average waiting time, a median waiting time, etc. In addition, exemplarily, obtaining the second target concurrency of the second scheduling period may include: retrieving the second target concurrency of the second scheduling period from the cache, and the second scheduling period may be the previous scheduling period of the first scheduling period. In addition, exemplarily, after calculating the first target concurrency in the first scheduling period, it may also include: storing the first target concurrency in the first scheduling period in the cache, that is, after calculating the target concurrency of each scheduling period, it may be stored in the cache so that it can be called in the flow control process of the subsequent scheduling period (for example, the next scheduling period). For example, in the process of flow control in the next scheduling period of the first scheduling period, it is necessary to calculate the target concurrency of the next scheduling period of the first scheduling period. In the calculation process, it is necessary to schedule the target concurrency of the scheduling period before the next scheduling period, for example, the target concurrency of the previous scheduling period (that is, the first scheduling period) of the next scheduling period is scheduled, and the target concurrency of the first scheduling period (that is, the first target concurrency) has been stored in the cache, and the target concurrency of the first scheduling period can be retrieved from the cache.
[0035] That is, in the present embodiment, in the process of calculating the first target concurrency within the first scheduling cycle, not only the second target concurrency of the previous scheduling cycle is taken into account, but also the waiting time of the task within the first scheduling cycle is taken into account. The first target concurrency within the first scheduling cycle is determined by using the waiting time and the second target concurrency of the previous scheduling cycle. The concurrency of the task can be adjusted according to the second target concurrency of the previous scheduling cycle and the waiting time of the task within the first scheduling cycle. In this way, if there are many slow-executing tasks in the task queue, the concurrency can be lowered to reduce the number of slow tasks in the task queue. For fast-executing tasks, the concurrency can be increased to increase the number of fast-executing tasks in the execution queue, thereby achieving the purpose of speeding up execution.
[0036] In one embodiment, determining the first target concurrency according to the waiting time and the second target concurrency includes:
[0037] When the waiting time is less than or equal to the preset time threshold, the first target concurrency is determined according to the second target concurrency.
[0038] The preset duration threshold can be set in advance, for example, it can be set according to historical experience or actual needs, for example, the preset duration threshold can be set to 5 seconds. When the waiting time is less than or equal to the preset duration threshold, it can be indicated that the waiting time of the task in the first scheduling cycle is short, and the first target concurrency can be directly determined according to the second target concurrency, that is, in this case, the second target concurrency of the second scheduling cycle is taken into account in the process of calculating the first target concurrency, so as to improve the rationality of the first target concurrency in the first scheduling cycle, and the task queue is limited by the first target concurrency in the first scheduling cycle to improve the flow control effect.
[0039] Exemplarily, determining the first target concurrency in the first scheduling period according to the second target concurrency in the second scheduling period may include: determining the product of the second target concurrency and the second preset ratio as the first target concurrency in the first scheduling period. The second preset ratio may be preset, for example, the second preset ratio may be preset according to historical experience or actual needs, for example, the second preset ratio may be set to 1.1.
[0040] In one embodiment, determining the first target concurrency according to the waiting time and the second target concurrency includes:
[0041] When the waiting time is greater than the preset time threshold, the processing time of the task in the first scheduling period is calculated, and the interface call time is determined according to the processing time and the waiting time;
[0042] Get the number of response requests;
[0043] Determine the third concurrency of the first scheduling cycle according to the number of response requests and the duration of the interface call;
[0044] The first target concurrency is determined according to the second target concurrency and the third concurrency.
[0045] Exemplarily, the processing time may be an average processing time, a median processing time, etc., and the interface call time may be an average interface call time, a median interface call time, etc. Exemplarily, the actual number of response requests per second (QPS) may be the result of dividing the number of tasks processed in the first scheduling period by the scheduling period interval. The third concurrency may also be understood as the theoretical optimal concurrency. Exemplarily, the third concurrency of the first scheduling period is determined based on the actual number of response requests per second and the interface call time, which may include: determining the product of the actual number of response requests per second, the average interface call time and the third preset ratio as the third concurrency of the first scheduling period. The third preset ratio may be preset, for example, according to historical experience or actual needs, for example, the preset time threshold may be preset, for example, according to historical experience or actual needs, for example, the preset time threshold may be set to 1.1.
[0046] In this embodiment, when the waiting time is greater than the preset time threshold, it may indicate that the waiting time of the task in the first scheduling cycle is long. In this case, the third concurrency of the first scheduling cycle can be determined based on the actual number of response requests per second and the interface call duration, and the first target concurrency in the first scheduling cycle can be determined using the second target concurrency and the third concurrency of the second scheduling cycle. That is, in this case, in the process of calculating the first target concurrency in the first scheduling cycle, not only the second target concurrency of the second scheduling cycle is considered, but also the third concurrency calculated based on the actual number of response requests per second and the interface call duration is considered, thereby improving the rationality of the obtained target concurrency, and limiting the flow of the task queue through the first target concurrency in the first scheduling cycle to improve the flow control effect.
[0047] In one embodiment, determining the first target concurrency according to the second target concurrency and the third concurrency includes:
[0048] determining a difference between the third concurrency and the second target concurrency;
[0049] Multiplying the difference by the first preset ratio to obtain an intermediate result;
[0050] The sum of the intermediate result and the second target concurrency is determined as the first target concurrency.
[0051] It should be noted that the first preset ratio can be preset, for example, can be set according to historical experience or actual needs, for example, the first preset ratio can be set to 0.5.
[0052] In this embodiment, in the process of calculating the first target concurrency within the first scheduling cycle using the second target concurrency and the third concurrency of the second scheduling cycle, not only the second target concurrency of the second scheduling cycle is considered, but also the difference between the third concurrency and the second target concurrency of the second scheduling cycle is considered, so as to improve the rationality of the first target concurrency within the first scheduling cycle, and limit the flow of task queues through the first target concurrency within the first scheduling cycle, thereby improving the flow control effect.
[0053] The process of the flow control method is described in detail below with reference to a specific embodiment.
[0054] The batch running system generally consists of three main parts: task queue (i.e. execution queue), task executor and task scheduler.
[0055] Execution queue: stores task data to be executed or being executed; the entire batch cluster shares one execution queue; the execution queue will set an upper limit based on the computing power of the cluster.
[0056] Task executor: Each node in the batch cluster is an executor, which obtains the task data to be executed from the execution queue, calls the dependent downstream system to obtain relevant data, and then performs business calculations.
[0057] Task scheduler: During the scheduling cycle, the number of tasks that need to be put into the execution queue is controlled according to the task flow limit threshold;
[0058] The process of the method in this embodiment is as follows:
[0059] After each task is executed, the execution data of this time can be cached through redis. For example, it can be stored based on zset of redis, with the current timestamp as the key and the processing time, waiting time, etc. as the value.
[0060] The concurrency calculator periodically obtains the duration samples of each task from the redis cache to calculate the target concurrency:
[0061] The average processing time (totalCost) of each task in the calculation cycle is obtained from redis to calculate the average value of the task processing time;
[0062] The average waiting time (waitCost, also known as the average current limiting waiting time) of each task in the calculation cycle is obtained from the cached waiting time in redis to calculate the average value;
[0063] The average interface call duration (serverCost) of each task in the calculation cycle is calculated as follows: average interface call duration = average processing duration - average current limiting waiting time;
[0064] Calculate the actual QPS using the formula: Actual QPS = amount of data processed in a cycle / cycle interval;
[0065] Calculate the theoretical optimal concurrency (targetConcurrent, i.e. the third concurrency), the formula is: target optimal concurrency = actual QPS * average interface call duration * third preset ratio. The third preset ratio is to maintain a certain queue, generally set to 1.1;
[0066] Get the target concurrency (currentConcurrent) obtained in the previous cycle, that is, the second target concurrency;
[0067] Calculate the target concurrency (newConcurrent) in this cycle, that is, the first target concurrency:
[0068] If the average current limiting waiting time (waitCost) > the waiting time threshold (generally set to 5 seconds), then: newConcurrent = currentConcurrent + (targetConcurrent - currentConcurrent) * 0.5;
[0069] If the average current limiting waiting time (waitCost) ≤ the waiting time threshold (generally set to 5 seconds), then: newConcurrent = currentConcurrent*1.1.
[0070] Update the target concurrency (newConcurrent);
[0071] Save the target concurrency to redis for use in the next cycle;
[0072] Synchronize the target concurrency to the task scheduler.
[0073] Through the method of the embodiment of the present application, the system throughput can be accelerated to the maximum extent through a scheduling scheme based on target concurrency. Because the target concurrency is the result of calculation based on the actual flow limiting situation of the task, it can reflect the actual operation situation. Compared with the scheduling scheme based on QPS in the prior art, it only uses a fixed QPS value for scheduling and cannot adjust the execution of the task according to the actual execution situation. The method of the embodiment of the present application can calculate the target concurrency in each scheduling cycle, and use the target concurrency to calculate the task flow limiting threshold for flow control, thereby improving the flow control effect.
[0074] See also Figure 2 , Figure 22 is a structural diagram of a flow control device 200 provided in an embodiment of the present application, which can implement the details of the flow control method in the above embodiment and achieve the same effect. Figure 2 As shown, the flow control device 200 comprises:
[0075] A first calculation module 201 is used to calculate a first target concurrency in a first scheduling period;
[0076] A second calculation module 202 is used to calculate a task current limiting threshold according to the first target concurrency and the number of tasks in the task queue;
[0077] The control module 203 is used to control the number of tasks submitted to the task queue in the first scheduling period according to the task flow limiting threshold.
[0078] In one embodiment, the first calculation module 201 includes:
[0079] A first calculation unit calculates the waiting time of tasks in a first scheduling period;
[0080] A first acquisition unit is configured to acquire a second target concurrency of a second scheduling period, where the second scheduling period is a scheduling period before the first scheduling period;
[0081] The determination unit determines the first target concurrency according to the waiting time and the second target concurrency.
[0082] In one embodiment, determining the first target concurrency according to the waiting time and the second target concurrency includes:
[0083] When the waiting time is less than or equal to the preset time threshold, the first target concurrency is determined according to the second target concurrency.
[0084] In one embodiment, the determining unit includes:
[0085] A duration determination unit, when the waiting time is greater than a preset time threshold, calculates the processing time of the task in the first scheduling period, and determines the interface call time according to the processing time and the waiting time;
[0086] The second acquisition unit is used to obtain the actual number of response requests per second;
[0087] A first determining unit determines a third concurrency of the first scheduling period according to the number of response requests and the duration of the interface call;
[0088] The second determining unit determines the first target concurrency according to the second target concurrency and the third concurrency.
[0089] In one embodiment, the second determining unit includes:
[0090] a difference determination unit, determining a difference between the third concurrency and the second target concurrency;
[0091] A first processing unit multiplies the difference by a first preset ratio to obtain an intermediate result;
[0092] The second processing unit determines the sum of the intermediate result and the second target concurrency as the first target concurrency.
[0093] In one embodiment, determining the first target concurrency according to the second target concurrency includes:
[0094] The product of the second target concurrency and the second preset ratio is determined as the first target concurrency.
[0095] In one embodiment, determining the third concurrency of the first scheduling period according to the number of response requests and the interface call duration includes:
[0096] The product of the number of response requests, the interface call duration, and the third preset ratio is determined as the third concurrency.
[0097] The flow control device provided in the embodiment of the present application can perform each process of the flow control method in the above embodiment, and will not be described again here to avoid repetition.
[0098] like Figure 3 As shown, an embodiment of the present application also provides an electronic device 300, including a processor 310, a memory 320, and a computer program stored in the memory 20 and executable on the processor 310. When the computer program is executed by the processor 310, the various processes of the above-mentioned flow control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0099] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned flow control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0100] The embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned flow control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0101] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0103] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A flow control method, characterized in that: include: Calculating a first target concurrency in a first scheduling period; Calculate the task current limiting threshold according to the first target concurrency and the number of tasks in the task queue; According to the task current limiting threshold, the number of tasks submitted to the task queue in the first scheduling period is controlled.
2. The method according to claim 1, characterized in that The calculating the first target concurrency in the first scheduling period includes: Calculating the waiting time of the task in the first scheduling period; Obtaining a second target concurrency of a second scheduling period, where the second scheduling period is a scheduling period before the first scheduling period; The first target concurrency is determined according to the waiting time and the second target concurrency.
3. The method according to claim 2, characterized in that The determining the first target concurrency according to the waiting time and the second target concurrency includes: When the waiting time is less than or equal to a preset time threshold, the first target concurrency is determined according to the second target concurrency.
4. The method according to claim 2, characterized in that: The determining the first target concurrency according to the waiting time and the second target concurrency includes: When the waiting time is greater than a preset time threshold, calculating the processing time of the task in the first scheduling period, and determining the interface call time according to the processing time and the waiting time; Get the number of response requests; Determining a third concurrency of the first scheduling period according to the number of response requests and the duration of the interface call; The first target concurrency is determined according to the second target concurrency and the third concurrency.
5. The method according to claim 4, characterized in that The determining the first target concurrency according to the second target concurrency and the third concurrency includes: determining a difference between the third concurrency and the second target concurrency; Multiplying the difference by a first preset ratio to obtain an intermediate result; The sum of the intermediate result and the second target concurrency is determined as the first target concurrency.
6. The method according to claim 3, characterized in that: The determining the first target concurrency according to the second target concurrency includes: The product of the second target concurrency and the second preset ratio is determined as the first target concurrency.
7. The method according to claim 4, characterized in that The determining, according to the number of response requests and the duration of the interface call, a third concurrency of the first scheduling period includes: The product of the number of response requests, the interface call duration, and a third preset ratio is determined as the third concurrency.
8. A flow control device, characterized in that: include: A first calculation module, used to calculate a first target concurrency in a first scheduling period; A second calculation module is used to calculate a task current limiting threshold according to the first target concurrency and the number of tasks in the task queue; A control module is used to control the number of tasks submitted to the task queue within the first scheduling period according to the task current limiting threshold.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the flow control method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the flow control method according to any one of claims 1 to 7 are implemented.
11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.