Timing job scheduling method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses an aging job scheduling method and device, equipment and a storage medium. The method comprises the following steps: determining a full-link aging job list according to a dependency relationship among a plurality of aging jobs, and determining a target priority of the aging jobs according to a preset priority, a historical resource occupancy ratio and a historical completion time deviation of each aging job in the list; determining an aging job queue according to the target priority, and determining a first aging job in the aging job queue through a sliding time window; according to a genetic algorithm and a load balancing strategy, determining calculation node distribution information of each first aging operation, and performing distribution execution; stopping the first aging operation exceeding the index threshold and moving the first aging operation into the queue to wait again; and according to the historical operation information of the second aging jobs in the waiting state in the queue, the operation indexes of the second aging jobs are predicted, and early warning processing is performed on the second aging jobs exceeding the index threshold, so that the job efficiency and the resource utilization rate are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to computer technology, and particularly to a time-effective operation scheduling method, device, equipment and storage medium. BACKGROUND

[0002] The completion time of a time-effective operation is of high requirement, such as a time-effective operation requiring to be completed at 10 o'clock. Different time-effective operations may have different completion time requirements. Therefore, when multiple time-effective operations are processed, all the time-effective operations need to be scheduled to reasonably allocate resources and maximize resource utilization and time efficiency. Therefore, there is an urgent need for a way to intelligently schedule multiple time-effective operations. SUMMARY

[0003] Embodiments of the present application provide a time-effective operation scheduling method, device, equipment and storage medium to realize intelligent scheduling of time-effective operations and improve operation efficiency and resource utilization.

[0004] In a first aspect, embodiments of the present application provide a time-effective operation scheduling method, comprising:

[0005] According to the dependency relationship between multiple time-effective operations, a full-link time-effective operation list is determined, and according to the preset priority, historical resource occupation ratio and historical completion time deviation of each time-effective operation in the full-link time-effective operation list, a target priority of each time-effective operation is determined;

[0006] According to the target priority, a time-effective operation queue is determined, and a plurality of first time-effective operations currently to be executed in the time-effective operation queue are determined through a sliding time window;

[0007] According to a genetic algorithm and a load balancing strategy, a computing node for a time-effective operation is allocated, computing node allocation information corresponding to each first time-effective operation is determined, and each first time-effective operation is allocated to a corresponding computing node for execution according to the computing node allocation information;

[0008] It is detected whether the running index of each first time-effective operation exceeds an index threshold, the first time-effective operation exceeding the index threshold is aborted, and the first time-effective operation exceeding the index threshold is moved to the time-effective operation queue to wait for execution again;

[0009] According to the historical running information of each second time-effective operation in a waiting state in the time-effective operation queue, a running index of each second time-effective operation is predicted, and a second time-effective operation whose predicted running index exceeds an index threshold is subjected to a warning process.

[0010] In a second aspect, embodiments of the present application further provide a time-effective operation scheduling device, comprising:

[0011] a target priority determination module configured to determine a full-link time service job list according to a dependency relationship among a plurality of time service jobs, and determine a target priority of each time service job according to a preset priority, a historical resource occupation ratio and a historical completion time deviation of each time service job in the full-link time service job list;

[0012] a first time service job determination module configured to determine a time service job queue according to the target priority, and determine a plurality of first time service jobs currently to be executed in the time service job queue through a sliding time window;

[0013] a first time service job allocation module configured to perform calculation node allocation of the time service jobs according to a genetic algorithm and a load balancing strategy, determine calculation node allocation information corresponding to each first time service job, and allocate each first time service job to a corresponding calculation node for execution according to the calculation node allocation information;

[0014] a first time service job suspension module configured to detect whether a running index of each first time service job exceeds an index threshold, suspend the first time service job exceeding the index threshold, and move the first time service job exceeding the index threshold to the time service job queue to wait for execution again;

[0015] a second time service job early warning module configured to predict a running index of each second time service job in a waiting state in the time service job queue according to historical running information of the second time service job, and perform early warning processing on the second time service job whose predicted running index exceeds an index threshold.

[0016] In a third aspect, an electronic device is also provided in the embodiments of the present application, and the electronic device comprises:

[0017] at least one processor; and

[0018] a memory in communication connection with the at least one processor; wherein

[0019] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the time service job scheduling method provided in any of the embodiments of the present application.

[0020] In a fourth aspect, a computer readable storage medium is also provided in the embodiments of the present application, and the computer readable storage medium stores computer instructions for enabling a processor to execute the time service job scheduling method provided in any of the embodiments of the present application.

[0021] One of the above-mentioned embodiments has the following advantages or beneficial effects:

[0022] The target priority of each time-effect operation is dynamically determined according to the preset priority, the historical resource occupation ratio and the historical completion time deviation of each time-effect operation in the full-link time-effect operation list; the time-effect operation queue can be more accurately determined according to the target priority; and the plurality of first time-effect operations currently to be executed in the time-effect operation queue are determined through a sliding time window; the computing node allocation information corresponding to each first time-effect operation is determined according to the genetic algorithm and the load balancing strategy; and each first time-effect operation is allocated to the corresponding computing node for execution according to the computing node allocation information, so that the resources are reasonably utilized and the operation efficiency is improved. The running index of each first time-effect operation is detected to determine whether the running index exceeds an index threshold value; the first time-effect operation exceeding the index threshold value is aborted; and the first time-effect operation exceeding the index threshold value is moved to the time-effect operation queue to wait for execution again, so that the resources of the time-effect operation exceeding the threshold value are recycled, and the normal processing of other time-effect operations is ensured. The running index of each second time-effect operation in a waiting state in the time-effect operation queue is predicted according to the historical running information of each second time-effect operation; the second time-effect operation whose predicted running index exceeds the index threshold value is subjected to early warning processing, so that the resource overrun risk is predicted and intervention is made in advance, the overall operation efficiency is further ensured, and intelligent scheduling of the time-effect operation is realized.

[0023] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is a flowchart of a time-effect operation scheduling method provided by an embodiment of the present application;

[0026] Figure 2 is a flowchart of another time-effect operation scheduling method provided by an embodiment of the present application;

[0027] Figure 3 is a structural schematic diagram of a time-effect operation scheduling device provided by an embodiment of the present application;

[0028] Figure 4Fig. 1 is a structural schematic diagram of an electronic device for implementing a time-sensitive job scheduling method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the personnel in the technical field better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Figure 1 Fig. 1 is a structural schematic diagram of an electronic device for implementing a time-sensitive job scheduling method according to an embodiment of the present application. Figure 1 As shown, the method specifically includes the following steps:

[0032] S110, according to the dependency relationship between a plurality of time-sensitive jobs, determining a full-link time-sensitive job list, and according to the preset priority, historical resource occupation ratio and historical completion time deviation of each time-sensitive job in the full-link time-sensitive job list, determining the target priority of each time-sensitive job.

[0033] The job refers to a task or a procedure that needs to be executed. The time-sensitive job refers to a job that has a requirement for completion time. For example, a time-sensitive job that needs to be completed before 10 o'clock. The dependency relationship between time-sensitive jobs is used to represent the execution order of time-sensitive jobs. For example, some time-sensitive jobs need to be run first, and then other time-sensitive jobs are run. The full link is a complete chain from the beginning to the end of a certain time-sensitive job as a starting point or end point or any intermediate node. The full-link time-sensitive job list can include all time-sensitive job chains that need to be executed. The node structure of the full link can be very complex. For example, there are the same job nodes between different time-sensitive job chains, a node has multiple upstream nodes, and there can also be multiple downstream nodes. The preset priority of the time-sensitive job can be a default priority set in advance based on business requirements. The historical resource occupation ratio can be the resource ratio used by the time-sensitive job in the previous run. For example, the historical resource occupation ratio can refer to the ratio between the historical CPU and memory used. The historical completion time deviation can refer to the difference between the actual completion time of the time-sensitive job in the previous run and the required completion time. Both the historical resource occupation ratio and the historical completion time deviation can be used to represent the historical scheduling effect of the time-sensitive node. For example, if the historical resource occupation ratio and the historical completion time deviation of the time-sensitive job are larger, the priority of the time-sensitive job can be increased to execute the time-sensitive job faster. The target priority of the time-sensitive job refers to a composite priority that integrates the preset priority, the historical resource occupation ratio, and the historical completion time deviation. The higher the target priority, the more priority the time-sensitive job has.

[0034] For example, the preset priority of each time-sensitive job is determined based on the time-sensitive urgency of the time-sensitive job, wherein the time-sensitive urgency is determined according to the required completion time of the time-sensitive job. The time-sensitive urgency is used to represent the degree of urgency of the time-sensitive job. The earlier the required completion time of the time-sensitive job, the greater the time-sensitive urgency, and the greater the preset priority, so that the time-sensitive job can be executed with higher priority.

[0035] Specifically, the job dependency analysis of the directed acyclic graph is performed according to the dependency relationship between the plurality of time-sensitive jobs, so as to analyze the topological relationship between the time-sensitive jobs, obtain the full-link time-sensitive job list, and further avoid the delay of job running caused by dependency conflicts. The preset priority, the historical resource occupation ratio, and the historical completion time deviation of each time-sensitive job in the full-link time-sensitive job list are weighted and summed to obtain the target priority of each time-sensitive job, so as to integrate multiple aspects of information and obtain a more accurate job priority to ensure the rapid execution of the time-sensitive job.

[0036] S120, determining a time-sensitive job queue according to the target priority, and determining a plurality of first time-sensitive jobs currently to be executed in the time-sensitive job queue through a sliding time window.

[0037] Specifically, according to the target priority of each time-critical job in the time-critical job list, the execution order of the time-critical job is determined, and a time-critical job queue arranged in the execution order is obtained. Among the multiple time-critical jobs that can be executed simultaneously, the higher the target priority, the earlier in the queue, and thus the higher the priority of the time-critical job. It should be noted that the arrangement order of the jobs in the time-critical job queue meets the requirements of the dependency relationship between the time-critical jobs, such as the execution of the time-critical job in the queue depends on the execution of the previous job, but not on the execution of the subsequent job, thereby further avoiding the situation of job running delay due to dependency conflict. A sliding time window is used to dynamically move the window in the time-critical job queue to update the time-critical job to be executed each time, and then all time-critical jobs are executed in order according to the target priority. The scheduling and execution of each time-critical job are the same, and the following will take all time-critical jobs in the current window as the first time-critical job to be executed as an example to illustrate the job scheduling process.

[0038] S130, according to the genetic algorithm and the load balancing strategy, the computing node allocation of the time-critical job is performed, the computing node allocation information corresponding to each first time-critical job is determined, and each first time-critical job is allocated to the corresponding computing node for execution according to the computing node allocation information.

[0039] The genetic algorithm is an optimization search algorithm based on the principles of natural selection and genetic mechanism, which simulates the selection, crossover and mutation operations in the biological evolution process to gradually optimize the solution and find the optimal solution. The load balancing strategy is to dynamically adjust the allocation of computing nodes according to the current load on the computing nodes, and preferentially allocate to the computing nodes with lower load to balance the running pressure. The computing node is a service node for executing time-critical jobs. The time-critical job can run on one or more computing nodes. The computing resources (CPU, memory and network bandwidth, etc.) are allocated to multiple computing nodes, and the computing resources of each computing node can be the same. Dynamic allocation of computing resources is equivalent to dynamic allocation of computing nodes. The more computing nodes allocated, the more computing resources allocated. The computing node allocation information corresponding to the first time-critical job can include one or more specific computing nodes to which the first time-critical job is allocated.

[0040] Specifically, the dynamic allocation of the computing nodes of the time-sensitive jobs is performed by fusing the genetic algorithm and the load balancing strategy, so as to maximize the resource utilization, thereby obtaining the computing node allocation information corresponding to each first time-sensitive job in the case of maximum resource utilization. According to the computing node allocation information corresponding to each first time-sensitive job, each first time-sensitive job is allocated to the corresponding computing node for execution, so as to achieve the precise guarantee of the time-sensitive jobs and the maximization of resource efficiency. It should be noted that, when the first time-sensitive job is executed, the first time-sensitive job needs to be taken out from the queue for execution.

[0041] For example, the computing node allocation of the time-sensitive jobs according to the genetic algorithm and the load balancing strategy to determine the computing node allocation information corresponding to each first time-sensitive job in step S130 can include: according to the genetic algorithm, performing chromosome coding on the allocation between the time-sensitive jobs and the computing nodes, and evaluating the resource utilization by using the fitness function to determine the number of computing nodes allocated to each first time-sensitive job in the case of maximum resource utilization; according to the load balancing strategy, the load information of each computing node and the number of computing nodes allocated to each first time-sensitive job, determining the computing node to which each first time-sensitive job is allocated.

[0042] Specifically, the optimal node allocation scheme is found according to the selection, crossover and mutation modes in the genetic algorithm, that is, the allocation between the time-sensitive jobs and the computing nodes is performed by chromosome coding, the length of the chromosome is equal to the number of first time-sensitive jobs, each gene represents a first time-sensitive job, and the maximum value of the gene value is the number of computing nodes. For example, for the allocation of 5 first time-sensitive jobs and 3 computing nodes, a chromosome can be [1, 3, 2, 1, 2], wherein the gene value represents the number of computing nodes allocated to the time-sensitive job, that is, the first time-sensitive job is allocated to 1 computing node, the second time-sensitive job is allocated to 3 computing nodes, and so on. The population is initialized, that is, P initial chromosomes are randomly generated, and the gene values are randomly selected. The resource utilization is evaluated by using the fitness function, the gene values are operated by various genetic algorithms, such as selection, crossover and mutation operations, and the optimal solution is found according to the fitness, so as to obtain the number of computing nodes allocated to each first time-sensitive job in the case of maximum resource utilization. Then, based on the number of computing nodes allocated to each first time-sensitive job and the load information of each computing node, the computing node to which each first time-sensitive job should be allocated is determined on the basis of guaranteeing load balancing, so as to achieve the precise guarantee of the time-sensitive jobs and the maximization of resource efficiency.

[0043] S140, detecting whether the running index of each first time-sensitive job exceeds the index threshold value, aborting the first time-sensitive job exceeding the index threshold value, and moving the first time-sensitive job exceeding the index threshold value to the time-sensitive job queue to wait for execution again.

[0044] The running indicators can be indicators for characterizing the speed of the time-sensitive job running. The number of the running indicators can be one or more. The running indicators can include indicators in the time dimension and indicators in the resource dimension. For example, the running indicators include the running time and the ratio between the CPU and the memory. The indicator threshold can be an upper limit value of the running indicator of the time-sensitive job. It should be noted that the running indicators of different time-sensitive jobs can have different indicator thresholds.

[0045] Specifically, for each first time-sensitive job, at least one running indicator of the first time-sensitive job is detected in real time during the running of the first time-sensitive job. If the at least one running indicator exceeds the corresponding indicator threshold, the first time-sensitive job is aborted, the resource is released to other first time-sensitive jobs, and the first time-sensitive job is moved to the time-sensitive job queue to wait for execution again. In this way, the preemptive resource recycling is realized, and the situation that the time-sensitive job occupies the resource for a long time and cannot run other time-sensitive jobs is avoided, thereby ensuring the normal processing of other time-sensitive jobs.

[0046] For example, as an implementation manner, the indicator threshold of the running indicator of each first time-sensitive job can be a fixed value set in advance. For example, the indicator threshold of the running time of each time-sensitive job is determined according to the mean value and the standard deviation of the historical running time of each time-sensitive job. The indicator threshold of the ratio between the CPU and the memory of each time-sensitive job is determined according to the historical CPU usage and the historical memory usage of each time-sensitive job.

[0047] As another implementation manner, the indicator threshold of the running indicator of each first time-sensitive job can also be a threshold dynamically adjusted through reinforcement learning, so that a more reasonable indicator threshold can be set to ensure that the time-sensitive job can be completed within the threshold. For example, the indicator threshold of the running indicator of each first time-sensitive job is obtained through reinforcement learning based on the running state information of each first time-sensitive job and the preset threshold of the running indicator of each first time-sensitive job. The preset threshold of the running indicator of each first time-sensitive job can be a fixed value set in advance.

[0048] Specifically, for each first time-sensitive job, according to the running state information of the first time-sensitive job, such as resource utilization, task queue length and waiting time, historical threshold adjustment effect, overall load level and some abnormal indicators, etc. as the representation of the system running state, design multiple adjustment actions of the index threshold, such as increasing, decreasing, keeping or directly modifying to a new threshold on the basis of the preset threshold of the running index of the first time-sensitive job, etc.), and design reward and punishment rules according to resource utilization and running time. Through the self-adaptive mechanism, continuous reinforcement learning is carried out, and safety mechanisms (such as threshold change amplitude not more than ±10%, abnormal rollback to the last version, abnormal situation triggering manual, etc.) can be set, and the effect is monitored, so as to achieve the effect of continuous optimization while keeping the system stable.

[0049] S150, according to the historical running information of each second time-sensitive job in the time-sensitive job queue in the waiting state, predicting the running index of each second time-sensitive job, and performing early warning processing on the second time-sensitive job whose predicted running index exceeds the index threshold.

[0050] Among them, the second time-sensitive job can refer to the time-sensitive job in the time-sensitive job queue which has not been executed in the current scheduling. The historical running information of the second time-sensitive job refers to the running information of the second time-sensitive job in the historical scheduling execution. The historical running information can include but is not limited to the time in the waiting state, the historical running time, the historical number of allocated computing nodes and the historical resource usage.

[0051] Specifically, for each second time-sensitive job in the time-sensitive job queue in the waiting state, the specific value of the running index of the second time-sensitive job in the current scheduling execution can be predicted according to the historical running information of the second time-sensitive job by using a neural network model or the like, and it is detected whether the predicted running index of the second time-sensitive job exceeds the corresponding index threshold. If it exceeds, early warning processing is performed on the second time-sensitive job, such as sending running abnormal early warning information or triggering some preset processing mode (such as adjusting the execution order of the second time-sensitive job or manual handling, etc.), so as to intervene in advance, ensure the normal execution of the second time-sensitive job, and improve the overall execution efficiency of the time-sensitive job.

[0052] The technical scheme of the embodiment determines a full-link time efficiency operation list according to the dependency relationship between a plurality of time efficiency operations, dynamically determines a target priority of each time efficiency operation according to the preset priority, the historical resource occupation ratio and the historical completion time deviation of each time efficiency operation in the full-link time efficiency operation list, determines a time efficiency operation queue according to the target priority, and determines a plurality of first time efficiency operations currently to be executed in the time efficiency operation queue through a sliding time window. The computing node allocation of the time efficiency operation is performed according to the genetic algorithm and the load balancing strategy, the computing node allocation information corresponding to each first time efficiency operation is determined, and each first time efficiency operation is allocated to the corresponding computing node for execution according to the computing node allocation information, so that the resource is reasonably utilized, and the operation efficiency is improved. By detecting whether the running index of each first time efficiency operation exceeds the index threshold value, the first time efficiency operation exceeding the index threshold value is suspended, and the first time efficiency operation exceeding the index threshold value is moved into the time efficiency operation queue to wait for execution again, so that the resource of the time efficiency operation exceeding the threshold value is recycled, and the normal processing of other time efficiency operations is ensured. By predicting the running index of each second time efficiency operation in the waiting state in the time efficiency operation queue according to the historical running information of each second time efficiency operation, the second time efficiency operation whose predicted running index exceeds the index threshold value is processed, so that the resource over-limit risk is predicted and intervened in advance, the overall operation efficiency is further ensured, and the intelligent scheduling of the time efficiency operation is realized.

[0053] Figure 2 The flowchart of another time efficiency operation scheduling method provided by an embodiment of the application further includes identifying the bottleneck in the full link and further optimizing the next scheduling of the time efficiency operation on the basis of the above-mentioned embodiments. The explanation of the same or corresponding terms as those in the above-mentioned embodiments is not repeated here.

[0054] Referring to Figure 2 The another time efficiency operation scheduling method provided by the embodiment specifically includes the following steps.

[0055] S210, a full-link time efficiency operation list is determined according to the dependency relationship between a plurality of time efficiency operations, and a target priority of each time efficiency operation is determined according to the preset priority, the historical resource occupation ratio and the historical completion time deviation of each time efficiency operation in the full-link time efficiency operation list.

[0056] S220, a time efficiency operation queue is determined according to the target priority, and a plurality of first time efficiency operations currently to be executed in the time efficiency operation queue are determined through a sliding time window.

[0057] S230, performing time-efficient job computing node allocation according to the genetic algorithm and the load balancing strategy, determining the computing node allocation information corresponding to each first time-efficient job, and distributing each first time-efficient job to the corresponding computing node for execution according to the computing node allocation information.

[0058] S240, detecting whether the running index of each first time-efficient job exceeds the index threshold value, aborting the first time-efficient job exceeding the index threshold value, and moving the first time-efficient job exceeding the index threshold value to the time-efficient job queue to wait for execution again.

[0059] S250, predicting the running index of each second time-efficient job in the waiting state in the time-efficient job queue according to the historical running information of each second time-efficient job, and performing early warning processing on the second time-efficient job whose predicted running index exceeds the index threshold value.

[0060] S260, combining time-efficient jobs in the full-link time-efficient job list, and comparing the historical running conditions in the combined form to determine the time-efficient job combination causing the bottleneck in the full-link.

[0061] The time-efficient job combination can include two time-efficient jobs. Both of the two time-efficient jobs in the time-efficient job combination are running bottlenecks in the full-link, i.e., the invisible bottleneck in the full-link. The bottleneck can refer to the completion time of the time-efficient job becoming more and more late, or the running time becoming longer and longer in a period of time. The number of time-efficient job combinations can be one or more.

[0062] Specifically, the historical running information of all time-efficient jobs in a period of time is extracted for detailed comparison and analysis, all time-efficient jobs in the full-link time-efficient job list are combined two by two, such as combined in DNA form, the running conditions are compared according to the combined form, and the implicit bottleneck, i.e., the time-efficient job combination causing the bottleneck, is identified. For example, the implicit bottleneck is identified by pattern matching, i.e., the time-efficient job combination causing the delay is determined.

[0063] S270, adjusting the preset priority and / or computing node allocation strategy of each time-efficient job in the time-efficient job combination by the gradient descent optimization method, and performing next scheduling based on the adjusted preset priority and / or computing node allocation strategy.

[0064] The gradient descent optimization method can refer to step-by-step adjustment of the preset priority and / or the computing node allocation strategy to minimize the delay time and resource waste. The adjustment of the preset priority can include increasing the preset priority of one time-critical job in the time-critical job combination and decreasing the preset priority of another time-critical job, so that the two time-critical jobs are executed in sequence, avoiding the simultaneous execution of the two time-critical jobs. The adjustment of the computing node allocation strategy can include allocating the two time-critical jobs in the time-critical job combination to different computing nodes, avoiding the allocation to the same computing node, thereby optimizing the execution efficiency of the two time-critical jobs.

[0065] Specifically, by the gradient descent optimization method, the preset priority and / or the computing node allocation strategy of each time-critical job in each time-critical job combination are adjusted step by step, and the next scheduling is performed based on the adjusted preset priority and / or the computing node allocation strategy, thereby gradually eliminating the bottleneck and achieving accurate guarantee of the time-critical job and maximum resource efficiency.

[0066] For example, after adjusting the preset priority of each time-critical job in the time-critical job combination, the precedent jobs of each time-critical job in the time-critical job combination in the whole link are determined, and the preset priorities of the precedent jobs are adjusted synchronously.

[0067] The precedent job of the time-critical job in the whole link refers to other time-critical jobs that need to be executed before the execution of the time-critical job. Specifically, according to the adjustment mode of the preset priority of each time-critical job in the time-critical job combination, the preset priorities of each precedent job of the time-critical job in the whole link are adjusted synchronously, for example, the preset priority of a certain time-critical job becomes larger, and the preset priorities of its precedent jobs also become larger synchronously, thereby realizing the reverse propagation adjustment of the priority and optimizing the global scheduling.

[0068] The technical scheme of the embodiment combines the time-critical jobs in the whole link time-critical job list, compares the combined form with the historical running status, determines the time-critical job combination in the whole link that causes the bottleneck, adjusts the preset priority and / or the computing node allocation strategy of each time-critical job in the time-critical job combination by the gradient descent optimization method, and performs the next scheduling based on the adjusted preset priority and / or the computing node allocation strategy, thereby gradually eliminating the bottleneck and achieving accurate guarantee of the time-critical job and maximum resource efficiency.

[0069] The following is an embodiment of a time-critical job scheduling device provided by the embodiment of the application. The device and the time-critical job scheduling method of each embodiment described above belong to the same inventive concept. Details not described in the embodiment of the time-critical job scheduling device can be referred to the embodiment of the time-critical job scheduling method described above.

[0070] Figure 3A structural schematic diagram of a time-effective operation scheduling device provided by an embodiment of the present application can be applicable to the case of intelligent scheduling of time-effective operations. Figure 3 As shown in the figure, the device specifically includes: a target priority determination module 310, a first time-effective operation determination module 320, a first time-effective operation allocation module 330, a first time-effective operation suspension module 340, and a second time-effective operation early warning module 350.

[0071] The target priority determination module 310 is configured to determine a full-link time-effective operation list according to the dependency relationship between multiple time-effective operations, and determine a target priority of each time-effective operation according to a preset priority, a historical resource occupation ratio, and a historical completion time deviation of each time-effective operation in the full-link time-effective operation list. The first time-effective operation determination module 320 is configured to determine a time-effective operation queue according to the target priority, and determine multiple first time-effective operations currently to be executed in the time-effective operation queue through a sliding time window. The first time-effective operation allocation module 330 is configured to perform calculation node allocation of time-effective operations according to a genetic algorithm and a load balancing strategy, determine calculation node allocation information corresponding to each first time-effective operation, and allocate each first time-effective operation to a corresponding calculation node for execution according to the calculation node allocation information. The first time-effective operation suspension module 340 is configured to detect whether a running index of each first time-effective operation exceeds an index threshold value, suspend the first time-effective operation that exceeds the index threshold value, and move the first time-effective operation that exceeds the index threshold value to the time-effective operation queue to wait for execution again. The second time-effective operation early warning module 350 is configured to predict a running index of each second time-effective operation in a waiting state in the time-effective operation queue according to historical running information of the second time-effective operation, and perform early warning processing on the second time-effective operation whose predicted running index exceeds an index threshold value.

[0072] The technical scheme of the embodiment determines a full-link time efficiency operation list according to the dependency relationship between a plurality of time efficiency operations, dynamically determines a target priority of each time efficiency operation according to the preset priority, historical resource occupation ratio and historical completion time deviation of each time efficiency operation in the full-link time efficiency operation list, determines a time efficiency operation queue more accurately according to the target priority, and determines a plurality of first time efficiency operations currently to be executed in the time efficiency operation queue through a sliding time window. The computing node allocation of the time efficiency operation is performed according to a genetic algorithm and a load balancing strategy, the computing node allocation information corresponding to each first time efficiency operation is determined, and each first time efficiency operation is allocated to a corresponding computing node for execution according to the computing node allocation information, so as to realize reasonable utilization of resources and improve operation efficiency. By detecting whether the running index of each first time efficiency operation exceeds an index threshold value, the first time efficiency operation exceeding the index threshold value is suspended, and the first time efficiency operation exceeding the index threshold value is moved into the time efficiency operation queue to wait for execution again, so as to recycle resources of the time efficiency operation exceeding the threshold value and ensure normal processing of other time efficiency operations. By predicting the running index of each second time efficiency operation in a waiting state in the time efficiency operation queue according to historical running information of each second time efficiency operation, the second time efficiency operation whose predicted running index exceeds the index threshold value is processed for early warning, so as to predict resource overrun risks and intervene in advance, further ensure overall operation efficiency, and realize intelligent scheduling of time efficiency operations.

[0073] Optionally, the preset priority of each time efficiency operation is determined based on time efficiency urgency of the time efficiency operation, wherein the time efficiency urgency is determined according to a required completion time of the time efficiency operation.

[0074] Optionally, the first time efficiency operation allocation module 330 is specifically configured to:

[0075] According to the genetic algorithm, the allocation between the time efficiency operation and the computing node is chromosomally coded, and the fitness function is used to evaluate the resource utilization rate to determine the number of computing nodes corresponding to each first time efficiency operation when the resource utilization rate is maximum. According to the load balancing strategy, the load information of each computing node and the number of computing nodes corresponding to each first time efficiency operation, the computing node to which each first time efficiency operation is allocated is determined.

[0076] Optionally, the index threshold value of the running index of each first time efficiency operation is obtained based on reinforcement learning on the basis of a preset threshold value of the running index of each first time efficiency operation according to running state information of each first time efficiency operation.

[0077] Optionally, the apparatus further comprises:

[0078] A time-limiting operation combination determination module is used to combine the time-limiting operations in the full-link time-limiting operation list, compare the combined operations with historical operation status, and determine the time-limiting operation combination that causes the bottleneck in the full link;

[0079] The scheduling adjustment module is used to adjust the preset priority and / or computing node allocation strategy of each time-sensitive job in the time-sensitive job combination through a gradient descent optimization method, and perform the next scheduling based on the adjusted preset priority and / or computing node allocation strategy.

[0080] Optionally, the scheduling adjustment module is further used to determine the predecessor job of each time-limiting job in the time-limiting job combination in the entire link after adjusting the preset priority of each time-limiting job in the time-limiting job combination, and synchronously adjust the preset priority of the predecessor job.

[0081] Optionally, the operation indicators include operation time and the ratio between CPU and memory.

[0082] The time-sensitive job scheduling device provided in the embodiment of the present invention can execute the time-sensitive job scheduling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the time-sensitive job scheduling method.

[0083] It is worth noting that in the embodiment of the above-mentioned time-sensitive job scheduling device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as they can achieve the corresponding functions; in addition, the specific names of the various functional units are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention.

[0084] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0085] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected to the at least one processor 11 in communication. The memory stores a computer program executable by the at least one processor 11, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0087] The processor 11 can be various general and / or special-purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various time-based job scheduling methods and processes described above.

[0088] In some embodiments, the time-based job scheduling method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the time-based job scheduling method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the time-based job scheduling method by any other appropriate means, such as by means of firmware.

[0089] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0090] Computer programs used to implement the aging job scheduling method of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flow diagrams and / or block diagrams. The computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0091] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0092] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0093] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0094] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0095] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps recited in the present invention can be performed in parallel, in series, or in a different order, without limitation herein, as long as the desired results of the technical solutions of the present invention can be achieved.

[0096] The specific embodiments described above are not intended to be limiting, but rather to illustrate the principles of the present invention. Variations, combinations, sub-combinations, and modifications can be made to the described embodiments within the spirit and scope of the present invention. Accordingly, other implementations are within the scope of the following claims.

Claims

1. A time-sensitive job scheduling method, characterized in that: include: Determine a full-link time-limiting job list based on the dependencies between multiple time-limiting jobs, and determine the target priority of each time-limiting job based on its preset priority, historical resource utilization ratio, and historical completion time deviation. Determine a time-sensitive job queue according to the target priority, and determine a plurality of first time-sensitive jobs currently to be executed in the time-sensitive job queue through a sliding time window; Allocating computing nodes for time-sensitive jobs based on a genetic algorithm and a load balancing strategy, determining computing node allocation information corresponding to each first time-sensitive job, and allocating each first time-sensitive job to a corresponding computing node for execution based on the computing node allocation information; detecting whether the running index of each first aging job exceeds the index threshold, terminating the first aging job that exceeds the index threshold, and moving the first aging job that exceeds the index threshold into the aging job queue to wait for execution again; According to the historical operation information of each second aging operation in the waiting state in the aging operation queue, the operation index of each second aging operation is predicted, and an early warning process is performed on the second aging operation whose predicted operation index exceeds the index threshold.

2. The method according to claim 1, characterized in that The preset priority of each time-sensitive task is determined based on the time-sensitive urgency of the time-sensitive task, wherein the time-sensitive urgency is determined according to the required completion time of the time-sensitive task.

3. The method according to claim 1, characterized in that The computing nodes of the time-sensitive jobs are allocated according to the genetic algorithm and the load balancing strategy, and the computing node allocation information corresponding to each first time-sensitive job is determined, including: Based on the genetic algorithm, the allocation between time-sensitive jobs and computing nodes is coded into chromosomes, and the resource utilization is evaluated using the fitness function to determine the number of computing nodes allocated to each first time-sensitive job when the resource utilization is maximized. The computing node to which each first time-sensitive job is assigned is determined according to the load balancing strategy, the load information of each computing node, and the number of computing nodes assigned to each first time-sensitive job.

4. The method according to claim 1, wherein The indicator threshold of the operation indicator of each first aging operation is obtained by performing reinforcement learning based on the preset threshold of the operation indicator of each first aging operation according to the operation status information of each first aging operation.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Combining the time-limiting operations in the full-link time-limiting operation list, and comparing the combined operations with historical operation status to determine the time-limiting operation combination that causes the bottleneck in the full link; The preset priority and / or computing node allocation strategy of each time-sensitive job in the time-sensitive job combination is adjusted by a gradient descent optimization method, and the next scheduling is performed based on the adjusted preset priority and / or computing node allocation strategy.

6. The method according to claim 5, characterized in that After adjusting the preset priority of each aging operation in the aging operation combination, the method further includes: Determine the predecessor job of each time-sensitive job in the time-sensitive job combination in the entire link, and synchronously adjust the preset priority of the predecessor job.

7. The method according to claim 1, characterized in that The operation indicators include the operation time and the ratio between CPU and memory.

8. A time-sensitive operation scheduling device, characterized in that: include: A target priority determination module is configured to determine a full-link time-limiting job list based on the dependencies between multiple time-limiting jobs, and to determine the target priority of each time-limiting job in the full-link time-limiting job list based on the preset priority, historical resource utilization ratio, and historical completion time deviation of each time-limiting job in the full-link time-limiting job list; a first time-aging job determination module, configured to determine a time-aging job queue according to the target priority, and determine a plurality of first time-aging jobs currently to be executed in the time-aging job queue through a sliding time window; A first time-limiting job allocation module is configured to allocate computing nodes for time-limiting jobs based on a genetic algorithm and a load balancing strategy, determine computing node allocation information corresponding to each first time-limiting job, and allocate each first time-limiting job to a corresponding computing node for execution based on the computing node allocation information; A first aging job aborting module is configured to detect whether the operating index of each first aging job exceeds an index threshold, abort the first aging job that exceeds the index threshold, and move the first aging job that exceeds the index threshold into the aging job queue to wait for execution again; The second aging job warning module is configured to predict an operating index of each second aging job based on historical operating information of each second aging job in a waiting state in the aging job queue, and to perform warning processing on a second aging job whose predicted operating index exceeds an index threshold.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the time-sensitive job scheduling method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the time-sensitive job scheduling method according to any one of claims 1 to 7 when executed.

Images