Task scheduling method and device

Abstract

The invention provides a task scheduling method and device, and the method comprises the steps: carrying out the evaluation of task historical resource consumption data obtained in advance, and obtaining a resource consumption score of a corresponding task; according to a task corresponding to the task historical resource consumption data, in combination with preset task target index information and a preset weight distribution rule, determining an index score of the corresponding task; obtaining an executable capability evaluation result of each working node according to the previously obtained working node state information; and determining a target working node according to the resource consumption score, the index score and the executable capability evaluation result, and issuing a corresponding task to the target working node. According to the invention, through comprehensively evaluating the historical resource consumption of the task, the target index corresponding to the task and the state of the working node, an optimal balance point is found between the demand of the task for resources and the supply capability of the working node, the reasonable distribution of system resources is realized, and the task arrangement and scheduling efficiency and the response speed to important tasks are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a task scheduling method and device. Background Art

[0002] In the task scheduling scenario, due to the diversity of tasks, different resources and scheduling during task execution cause task accumulation, extend output time, and ultimately affect production efficiency.

[0003] At present, in the case of task accumulation, most of them are divided into queues. Tasks are assigned to different queues according to factors such as task type, priority, and required resources. Each queue is set with corresponding processing rules and resource quotas to alleviate the problem of disorderly task competition for resources and achieve preliminary task scheduling optimization.

[0004] However, since each task takes a different amount of time to execute, if a task takes a long time, the progress of the task is likely to be stuck in the corresponding queue, causing other tasks that depend on the results of the task to be unable to start, further exacerbating task backlog. Summary of the invention

[0005] The present invention provides a task scheduling method and device, which are used to solve the defect of untimely task scheduling in the prior art resulting in task execution accumulation, realize the reasonable allocation of system resources, and improve the task scheduling efficiency and the response speed to important tasks.

[0006] The present invention provides a task scheduling method, comprising: evaluating previously acquired task history resource consumption data to obtain a resource consumption score of a corresponding task; determining a target indicator corresponding to the corresponding task according to the task corresponding to the task history resource consumption data in combination with preset task target indicator information, and determining an indicator score of the corresponding task in combination with a preset weight allocation rule; wherein the preset task target indicator information includes a plurality of tasks and target indicators corresponding to each task, and the preset weight allocation rule is used to define an allocation strategy for weights corresponding to each indicator; obtaining an executable capability evaluation result of each working node according to previously acquired working node status information; wherein the working node status information is used to characterize the running state of the working node executing the corresponding task; determining a target working node according to the resource consumption score, the indicator score and the executable capability evaluation result, and issuing the corresponding task to the target working node.

[0007] According to a task scheduling method provided by the present invention, the target indicator corresponding to the corresponding task is determined according to the task corresponding to the historical resource consumption data of the task in combination with preset task target indicator information, and the indicator score of the corresponding task is determined in combination with preset weight allocation rules, including: determining the target indicator of the corresponding task according to the task corresponding to the historical resource consumption data of the task in combination with preset task target indicator information; determining the allocation weight of each target indicator based on the target indicator of the task and based on the preset weight allocation rule; and obtaining the indicator score of the corresponding task according to the allocation weight of each target indicator and the target indicator of the task.

[0008] According to a task scheduling method provided by the present invention, according to the target indicator of the task, based on the preset weight allocation rule, the allocation weight of each target indicator is determined, including: according to each target indicator of the task, respectively looking up the preset weight allocation rule to determine the allocation weight corresponding to each target indicator; wherein the preset weight allocation strategy is used to limit the allocation weight of each target indicator; or, according to the system operation status information obtained in advance, combined with the preset weight allocation strategy, weights are allocated to each target indicator of the task; wherein the preset weight allocation strategy is used to limit the allocation weight rule of each target indicator under different system operation states, and the system operation status information is used to characterize the operation status of all working nodes and the overall computing environment of each working node running tasks.

[0009] According to a task scheduling method provided by the present invention, the target indicator corresponding to the corresponding task is determined according to the task corresponding to the historical resource consumption data of the task in combination with the preset task target indicator information, and the indicator score of the corresponding task is determined in combination with the preset weight allocation rule, and also includes: determining the target indicator of the corresponding task according to the task corresponding to the historical resource consumption data of the task in combination with the preset task target indicator information; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task; according to the target indicator of the task and the system operation status information obtained in advance, using a weight prediction model, the weight of each target indicator of the task is obtained; wherein the system operation status information is used to characterize the operation status of the overall computing environment of all working nodes and each working node running tasks, and the weight prediction model is trained based on the indicator training data, the system operation status training data corresponding to the indicator training data and the corresponding indicator weight label; according to the allocation weight of each target indicator and the target indicator of the task, the indicator score of the corresponding task is obtained.

[0010] According to a task scheduling method provided by the present invention, the working node status information includes the node memory usage, the node central processing unit CPU usage and the task type currently executed by the node; based on the previously obtained working node status information, the executable capability evaluation result of each working node is obtained, including: determining the first executable task quantity of the corresponding working node based on the node memory usage and the previously obtained node memory total amount of each working node and the task predicted consumption resources; evaluating the second executable task quantity of the corresponding working node based on the node CPU usage and the previously obtained node CPU performance parameters and the task predicted consumption resources; comparing the first executable task quantity and the second executable task quantity of the corresponding working node, and selecting the smaller quantity as the third executable task quantity; wherein the third executable task quantity is used to characterize the number of executable tasks corresponding to the idle resources of the corresponding working node; adjusting the third executable task quantity based on the task and the previously obtained task dependency, and obtaining the executable capability evaluation result of the corresponding working node; wherein the task dependency is used to characterize the compatibility relationship and competition relationship between tasks.

[0011] According to a task scheduling method provided by the present invention, a target working node is determined according to a resource consumption score, an indicator score and an executable capability evaluation result, including: determining a first weight according to the resource consumption score, determining a second weight according to the indicator score, and determining a third weight according to the executable capability evaluation result; wherein the first weight and the third weight have opposite positive and negative signs, the sign of the second weight is positive, and the first weight, the second weight and the third weight are configured in advance based on the resource consumption score, the indicator score and the executable capability evaluation result; obtaining a comprehensive score of the corresponding working node according to the resource consumption score, the first weight, the indicator score, the second weight, the executable capability evaluation result and the third weight; and selecting, according to the comprehensive score of each working node, the working node corresponding to the largest comprehensive score as the target working node.

[0012] According to a task scheduling method provided by the present invention, previously acquired historical resource consumption data of tasks are evaluated to obtain a resource consumption score of the corresponding task, including: obtaining historical resource consumption data of tasks, the historical resource consumption data of tasks including multiple tasks and resource consumption data corresponding to each task; for each task, sorting the resource consumption data corresponding to the task to obtain a resource sorting result of the corresponding task; for the resource sorting result of a single task, combining the corresponding preset truncation ratio, removing extreme values ​​in the sorting result, and determining the average value of the remaining data in the sorting result to obtain the resource consumption score of the corresponding task.

[0013] The present invention also provides a task scheduling device, including: a resource evaluation module, which evaluates the previously acquired historical resource consumption data of the task to obtain the resource consumption score of the corresponding task; an indicator evaluation module, which determines the target indicator corresponding to the corresponding task according to the task corresponding to the historical resource consumption data of the task, combined with the preset task target indicator information, and determines the indicator score of the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to limit the allocation strategy of the weight corresponding to each indicator; a node evaluation module, which obtains the executable capability evaluation result of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running state of the working node executing the corresponding task; an orchestration scheduling module, which determines the target working node according to the resource consumption score, the indicator score and the executable capability evaluation result, and sends the corresponding task to the target working node.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned task scheduling methods when executing the computer program.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the task scheduling method described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned task scheduling methods.

[0017] The task scheduling method and device provided by the present invention can accurately grasp the actual consumption of various resources of each task in the past execution process by evaluating the historical resource consumption data of the task, so as to more accurately judge the resource demand of the task, which is helpful to allocate the task to the most suitable work node; by determining the corresponding target index and weight to obtain the index score, the task can be further analyzed from multiple dimensions such as the complexity, importance, and real-time requirements of the task, so as to ensure that various key characteristics of the task can play a role in the allocation decision, so that important and urgent tasks can be processed first, and complex tasks can be reasonably arranged; according to the work node status information Information is collected and the possible execution capacity of the corresponding work node is evaluated, so as to understand the current resource surplus of each work node and the load status of the tasks being executed in real time, so as to fully consider the actual bearing capacity of the work node and avoid allocating too many tasks to the already overloaded nodes. Then, the resource consumption score, indicator score and executable capacity evaluation results are integrated to find the best balance between the task's demand for resources and the work node's supply capacity, ensuring that each task can be assigned to a work node that can meet its resource needs and give full play to its own characteristics, so as to achieve reasonable allocation of system resources and improve the efficiency of task scheduling and the response speed to important tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flowcharts of the task scheduling method provided by the present invention; Figure 2 This is the second flowchart of the task scheduling method provided by the present invention; Figure 3 It is a structural schematic diagram of the task scheduling device provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Figure 1 It is a flowchart of the task scheduling method provided by the present invention, such as Figure 1 As shown, the method includes: S11, evaluating the previously acquired historical resource consumption data of the task to obtain a resource consumption score of the corresponding task; S12, according to the task corresponding to the task historical resource consumption data, combined with the preset task target indicator information, determine the target indicator corresponding to the corresponding task, and determine the indicator score of the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to define the allocation strategy of the weight corresponding to each indicator; S13, obtaining an executable capability evaluation result of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running status of the working node executing the corresponding task; S14, determining the target working node according to the resource consumption score, indicator score and executable capability evaluation results, and issuing the corresponding task to the target working node.

[0022] It should be noted that the step numbers "S1N" in this specification do not represent the order of the task scheduling method. Figure 2 The task scheduling method of the present invention is described.

[0023] Step S11, evaluating the previously acquired historical resource consumption data of the task to obtain a resource consumption score of the corresponding task.

[0024] It should be added that the historical resource consumption data of tasks includes tasks and the resource consumption data corresponding to the tasks. The resource consumption data includes the CPU usage and memory usage during the task execution. In addition, the historical resource consumption data can be collected from system logs or monitoring tools. The system is the overall computing environment for all working nodes and each working node to run tasks.

[0025] In this embodiment, the previously acquired historical resource consumption data of the task is evaluated to obtain the resource consumption score of the corresponding task, including: obtaining the historical resource consumption data of the task; for each task, sorting the resource consumption data corresponding to the task to obtain the resource sorting result of the corresponding task; for the resource sorting result of a single task, combining the corresponding preset truncation ratio, removing the extreme values ​​in the sorting result, and determining the average value of the remaining data in the sorting result to obtain the resource consumption score of the corresponding task.

[0026] It should be noted that the preset truncation ratio can be set according to the actual truncation requirements. For example, if you only need to truncate the extreme values ​​at the beginning and end of the sorting results, or you need to truncate a certain proportion of the extreme values ​​at the beginning and end of the sorting results, then the preset truncation ratio can be configured according to the truncation requirements and the corresponding data volume of the sorting results, and no further limitation is made here.

[0027] In an optional embodiment, after obtaining the resource consumption score of the corresponding task, the method includes: based on the resource consumption scores of all tasks, normalizing the resource consumption scores of each task to obtain a normalized resource consumption result of the corresponding task.

[0028] Furthermore, based on the resource consumption scores of all tasks, the resource consumption scores of each task are normalized to obtain the normalized resource consumption results of the corresponding tasks, including: determining the maximum resource consumption score and the minimum resource consumption score based on the resource consumption scores of all tasks; obtaining a first difference based on the resource consumption score and the minimum resource consumption score of the current task; obtaining a second difference based on the maximum resource consumption score and the minimum resource consumption score; and obtaining the normalized resource consumption result of the current task based on the ratio of the first difference to the second difference.

[0029] Step S12, based on the task corresponding to the task historical resource consumption data, combined with the preset task target indicator information, determine the target indicator corresponding to the corresponding task, and determine the indicator score of the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to limit the allocation strategy of the weights corresponding to each indicator.

[0030] In this embodiment, the target indicator corresponding to the corresponding task is determined based on the task corresponding to the historical resource consumption data of the task in combination with the preset task target indicator information, and the indicator score of the corresponding task is determined in combination with the preset weight allocation rule, including: determining the target indicator of the corresponding task based on the task corresponding to the historical resource consumption data of the task in combination with the preset task target indicator information; determining the allocation weight of each target indicator based on the target indicator of the task and based on the preset weight allocation rule; and obtaining the indicator score of the corresponding task based on the allocation weight of each target indicator and the target indicator of the task.

[0031] It should be noted that the allocation weight of the target indicator is specified in advance based on the indicator of the corresponding task. The target indicator can be the number of task retries, task consumption, priority and task type, etc., which can be specified according to the actual task and are not further limited here.

[0032] In an optional embodiment, the allocation weight of each target indicator is a fixed weight set in advance based on the corresponding task indicator. Accordingly, the allocation weight of each target indicator is determined based on the target indicator of the task and based on the preset weight allocation rule, including: according to each target indicator of the task, respectively searching for the preset weight allocation rule to determine the allocation weight corresponding to each target indicator; wherein the preset weight allocation strategy is used to limit the allocation weight of each target indicator.

[0033] In another optional embodiment, the allocation weight of the target indicator is dynamically adjusted according to the corresponding system operating status. Accordingly, according to the target indicator of the task, based on the preset weight allocation rule, the allocation weight of each target indicator is determined, and it also includes: according to the system operating status information obtained in advance, combined with the preset weight allocation strategy, weights are allocated to each target indicator of the task; wherein the preset weight allocation strategy is used for the allocation weight rules of each of the target indicators under different system operating states, and the system operating status information is used to characterize the operating status of all working nodes and the overall computing environment of each of the working nodes running tasks.

[0034] It should be noted that under high load conditions, complex tasks may have a greater impact on system performance, so the weight needs to be dynamically adjusted according to the system operation status. In addition, the system operation status information includes system load (such as CPU usage, memory usage, etc.) and resource availability (the number and type of idle resources).

[0035] Furthermore, the preset weight allocation strategy can also be used to limit the weight allocation rules under different load conditions. Accordingly, based on the previously acquired system operation status information and combined with the preset weight allocation strategy, weights are allocated to each target indicator of the task, including: determining the load state category corresponding to the current system operation state based on the previously acquired system operation status information and combined with the load state classification threshold; obtaining the weight allocation rules corresponding to the preset weight allocation strategy based on the load state category corresponding to the current system operation state, and allocating weights to each target indicator.

[0036] It is worth noting that the load state classification threshold can be set according to the load state category that needs to be divided. For example, if the load state classification threshold is 30% and 70%, then when the load index is lower than 30%, the load state category is low load state, when the load index is between 30% and 70%, the load state category is medium load state, and when the load index exceeds 70%, the load state category is high load state. In addition, the weight distribution rules for different load state categories can be set according to the importance of each indicator in the corresponding system operation state and prior experience, and no further limitation is made here.

[0037] In an optional embodiment, the allocation weight of the target indicator is dynamically adjusted according to the corresponding system operating status. Accordingly, the target indicator corresponding to the corresponding task is determined according to the task corresponding to the task historical resource consumption data in combination with the preset task target indicator information, and the indicator score of the corresponding task is determined in combination with the preset weight allocation rule. It also includes: determining the target indicator of the corresponding task according to the task corresponding to the task historical resource consumption data in combination with the preset task target indicator information; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task; according to the target indicator of the task and the system operating status information obtained in advance, using the weight prediction model, the weight of each target indicator of the task is obtained; wherein the system operating status information is used to characterize the operating status of the overall computing environment of all working nodes and each working node running tasks, and the weight prediction model is trained based on the indicator training data, the system operating status training data corresponding to the indicator training data and the corresponding indicator weight label; according to the allocation weight of each target indicator and the target indicator of the task, the indicator score of the corresponding task is obtained.

[0038] In an optional embodiment, after obtaining the indicator score of the corresponding task, the method includes: based on the indicator scores of all tasks, normalizing the indicator score of each task to obtain the indicator normalization result of the corresponding task.

[0039] Furthermore, based on the indicator scores of all tasks, the indicator scores of each task are normalized to obtain the indicator normalization results of the corresponding tasks, including: determining the maximum indicator score and the minimum indicator score based on the indicator scores of all tasks; obtaining a third difference based on the indicator score and the minimum indicator score of the current task; obtaining a fourth difference based on the maximum indicator score and the minimum indicator score; and obtaining the indicator normalization result of the current task based on the ratio of the third difference and the fourth difference.

[0040] Step S13, obtaining the executable capability evaluation result of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running status of the working node executing the corresponding task.

[0041] In this embodiment, the working node status information includes the node memory usage, the node central processing unit CPU usage and the type of task currently executed by the node; based on the previously obtained working node status information, the executable capability evaluation result of each working node is obtained, including: determining the first executable task quantity of the corresponding working node based on the node memory usage and the previously obtained total node memory amount of each working node and the predicted resource consumption of the task; evaluating the second executable task quantity of the corresponding working node based on the node CPU usage and the previously obtained node CPU performance parameters and the predicted resource consumption of the task; comparing the first executable task quantity and the second executable task quantity of the corresponding working node, and selecting the smaller quantity as the third executable task quantity; wherein the third executable task quantity is used to characterize the number of executable tasks corresponding to the idle resources of the corresponding working node; adjusting the third executable task quantity based on the task and the previously obtained task dependency, and obtaining the executable capability evaluation result of the corresponding working node; wherein the task dependency is used to characterize the compatibility relationship and competition relationship between tasks.

[0042] Furthermore, before determining the first number of executable tasks of the corresponding working node based on the node memory usage and the total node memory of each working node obtained in advance and the predicted resource consumption of the task, it includes: classifying the resource requirements of the task types to obtain the demand classification results; and determining the predicted resource consumption of the task based on the demand classification results.

[0043] It should be noted that the task prediction consumption resources can be configured based on the resource demand classification type in advance. The resource demand classification types include computing intensive, memory intensive and I / O intensive, which are not further limited here.

[0044] In addition, based on the node memory usage and the total node memory of each working node obtained in advance and the predicted resource consumption of the tasks, the first number of executable tasks of the corresponding working node is determined, including: based on the node memory usage and the total node memory of each working node obtained in advance, the remaining memory space of the corresponding working node is determined; based on the estimated memory occupancy of the predicted resource consumption of the tasks obtained in advance and the remaining memory space of the corresponding working node, the first number of executable tasks refers to the memory requirement that the corresponding working node can accommodate the corresponding number of tasks.

[0045] In addition, according to the node CPU usage and the previously acquired node CPU performance parameters and task predicted resource consumption, it is determined how much CPU load the corresponding working node can still bear, and the corresponding second executable task quantity is obtained.

[0046] It should be noted that due to the competition and compatibility of resources between tasks, for example, when a worker node is executing a large number of compute-intensive tasks, the remaining CPU resources and possible performance degradation need to be considered when redistributing compute-intensive tasks. Therefore, the number of executable tasks of the corresponding worker node needs to be further adjusted according to the task dependencies to avoid performance degradation caused by excessive CPU competition, and to ensure that the worker node can maintain good performance and stability when executing new tasks.

[0047] In an optional embodiment, after obtaining the corresponding executable capability evaluation result, the method includes: based on the executable capability evaluation results of all tasks, normalizing the executable capability evaluation results of each task to obtain the normalized executable capability result of the corresponding task.

[0048] Furthermore, based on the executable capability evaluation results of all tasks, the executable capability evaluation results of each task are normalized to obtain the normalized executable capability evaluation results of the corresponding tasks, including: determining the maximum executable capability evaluation result and the minimum executable capability evaluation result based on the executable capability evaluation results of all tasks; obtaining the fifth difference according to the executable capability evaluation result and the minimum executable capability evaluation result of the current task; obtaining the sixth difference according to the maximum executable capability evaluation result and the minimum executable capability evaluation result; and obtaining the normalized executable capability evaluation result of the current task according to the ratio of the fifth difference to the sixth difference.

[0049] Step S14, determining the target working node according to the resource consumption score, the indicator score and the executable capability evaluation result, and issuing the corresponding task to the target working node.

[0050] In this embodiment, the target working node is determined according to the resource consumption score, the indicator score and the executable capability evaluation result, including: determining a first weight according to the resource consumption score, determining a second weight according to the indicator score, and determining a third weight according to the executable capability evaluation result; wherein the first weight and the third weight have opposite positive and negative signs, the sign of the second weight is positive, and the first weight, the second weight and the third weight are previously configured based on the resource consumption score, the indicator score and the executable capability evaluation result; obtaining a comprehensive score of the corresponding working node according to the resource consumption score, the first weight, the indicator score, the second weight, the executable capability evaluation result and the third weight; and selecting the working node corresponding to the largest comprehensive score as the target working node according to the comprehensive score of each working node.

[0051] It should be supplemented that the sum of the absolute value of the first weight, the absolute value of the second weight and the absolute value of the third weight is 1, or the product of the absolute value of the first weight, the absolute value of the second weight and the absolute value of the third weight is 1.

[0052] In summary, the embodiments of the present invention accurately grasp the actual consumption of various resources of each task in the past execution process by evaluating the historical resource consumption data of the task, so as to more accurately judge the resource demand of the task, which is helpful to allocate the task to the most suitable work node; by determining the corresponding target indicators and weights to obtain indicator scores, the tasks are further analyzed from multiple dimensions such as the complexity, importance, and real-time requirements of the tasks, ensuring that various key characteristics of the tasks can play a role in the allocation decision, so that important and urgent tasks can be processed first, and complex tasks can be reasonably arranged; according to the work node status information, the evaluation Estimate the possible execution capacity of the corresponding work node, so as to understand the current resource surplus of each work node and the load status of the tasks being executed in real time, so as to fully consider the actual bearing capacity of the work node and avoid allocating too many tasks to the already overloaded nodes. Then, the resource consumption score, indicator score and executable capacity evaluation results are integrated to find the best balance between the task's demand for resources and the work node's supply capacity, ensuring that each task can be assigned to a work node that can meet its resource needs and give full play to its own characteristics, so as to achieve reasonable allocation of system resources, improve the efficiency of task scheduling and the response speed to important tasks.

[0053] The task scheduling device provided by the present invention is described below. The task scheduling device described below and the task scheduling method described above can be referred to each other.

[0054] Figure 3 A structural schematic diagram of a task scheduling device is shown, the device comprising: The resource evaluation module 31 evaluates the previously acquired historical resource consumption data of the task to obtain a resource consumption score of the corresponding task; The indicator evaluation module 32 determines the target indicator corresponding to the task according to the task corresponding to the task historical resource consumption data and in combination with the preset task target indicator information, and determines the indicator score of the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to define the allocation strategy of the weight corresponding to each indicator; The node evaluation module 33 obtains the executable capability evaluation result of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running status of the working node executing the corresponding task; The orchestration and scheduling module 34 determines the target working node according to the resource consumption score, the indicator score and the executable capability evaluation result, and sends the corresponding task to the target working node.

[0055] In this embodiment, the resource evaluation module 31 includes: a data acquisition unit, which acquires the historical resource consumption data of the task; a sorting unit, which sorts the resource consumption data corresponding to each task to obtain the resource sorting result of the corresponding task; a resource evaluation unit, which removes extreme values ​​in the sorting result based on the resource sorting result of a single task in combination with the corresponding preset truncation ratio, and determines the average value of the remaining data in the sorting result to obtain the resource consumption score of the corresponding task.

[0056] In an optional embodiment, the device further includes: a resource normalization module, after obtaining the resource consumption score of the corresponding task, normalizes the resource consumption score of each task based on the resource consumption scores of all tasks to obtain the resource consumption normalization result of the corresponding task.

[0057] Furthermore, the resource normalization module includes: a first score determination unit, which determines the maximum resource consumption score and the minimum resource consumption score based on the resource consumption scores of all tasks; a first difference determination unit, which obtains a first difference according to the resource consumption score and the minimum resource consumption score of the current task; a second difference determination unit, which obtains a second difference according to the maximum resource consumption score and the minimum resource consumption score; and a resource normalization unit, which obtains a resource consumption normalization processing result of the current task according to the ratio of the first difference to the second difference.

[0058] The indicator evaluation module 32 includes: an indicator determination unit, which determines the target indicator of the corresponding task according to the task corresponding to the historical resource consumption data of the task, combined with the preset task target indicator information; a weight allocation unit, which determines the allocation weight of each target indicator according to the target indicator of the task and based on the preset weight allocation rules; an indicator scoring unit, which obtains the indicator score of the corresponding task according to the allocation weight of each target indicator and the target indicator of the task.

[0059] In an optional embodiment, the weight allocation unit is used to: search for preset weight allocation rules according to each target indicator of the task, and determine the allocation weight corresponding to each target indicator; wherein the preset weight allocation strategy is used to limit the allocation weight of each target indicator.

[0060] In another optional embodiment, the weight allocation unit is also used to: allocate weights to each target indicator of the task based on the system operation status information obtained previously and in combination with a preset weight allocation strategy; wherein the preset weight allocation strategy is used for the allocation weight rules of each of the target indicators under different system operation states, and the system operation status information is used to characterize the operation status of all working nodes and the overall computing environment of each of the working nodes running tasks.

[0061] Furthermore, the preset weight allocation strategy can also be used to limit the weight allocation rules under different load conditions. Accordingly, the weight allocation unit also includes: a classification subunit, which determines the load state category corresponding to the current system operation state based on the system operation state information obtained in advance and combined with the load state classification threshold; a weight allocation subunit, which obtains the weight allocation rules corresponding to the preset weight allocation strategy based on the load state category corresponding to the current system operation state, and allocates weights to each target indicator.

[0062] In an optional embodiment, the indicator evaluation module 32 also includes: an indicator determination unit, which determines the target indicator of the corresponding task according to the task corresponding to the historical resource consumption data of the task, combined with the preset task target indicator information; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task; a weight prediction unit, which obtains the weight of each target indicator of the task according to the target indicator of the task and the system operation status information obtained in advance, using a weight prediction model; wherein the system operation status information is used to characterize the operation status of the overall computing environment of all working nodes and each working node running tasks, and the weight prediction model is obtained based on the indicator training data, the system operation status training data corresponding to the indicator training data, and the corresponding indicator weight label training; an indicator scoring unit, which obtains the indicator score of the corresponding task according to the allocated weight of each target indicator and the target indicator of the task.

[0063] In an optional embodiment, the device further includes: an indicator normalization module, which, after obtaining the indicator score of the corresponding task, normalizes the indicator score of each task based on the indicator scores of all tasks to obtain the indicator normalization result of the corresponding task.

[0064] Furthermore, the indicator normalization module includes: a second score determination unit, which determines the maximum indicator score and the minimum indicator score based on the indicator scores of all tasks; a third difference determination unit, which obtains a third difference according to the indicator score and the minimum indicator score of the current task; a fourth difference determination unit, which obtains a fourth difference according to the maximum indicator score and the minimum indicator score; and an indicator normalization unit, which obtains the indicator normalization processing result of the current task according to the ratio of the third difference and the fourth difference.

[0065] The node evaluation module 33 includes: a first quantity determination unit, which determines the first number of executable tasks of the corresponding working node according to the node memory usage and the previously obtained total node memory of each working node and the predicted resource consumption of the task; a second quantity determination unit, which evaluates the second number of executable tasks of the corresponding working node according to the node CPU usage and the previously obtained node CPU performance parameters and the predicted resource consumption of the task; a comparison unit, which compares the first number of executable tasks and the second number of executable tasks of the corresponding working node, and selects the smaller number as the third number of executable tasks; wherein the third number of executable tasks is used to characterize the number of executable tasks corresponding to the idle resources of the corresponding working node; a node evaluation unit, which adjusts the third number of executable tasks according to the task and the previously obtained task dependency, and obtains the executable capability evaluation result of the corresponding working node; wherein the task dependency is used to characterize the compatibility relationship and competition relationship between tasks.

[0066] Furthermore, the node evaluation module 33 also includes: a resource classification unit, which classifies the resource requirements of the task type to obtain a demand classification result before determining the first number of executable tasks of the corresponding working node based on the node memory usage and the total node memory amount of each working node obtained in advance and the task predicted consumption resources; a resource estimation unit, which determines the task predicted consumption resources based on the demand classification result.

[0067] In addition, the first quantity determination unit includes: a first space determination sub-unit, which determines the remaining memory space of the corresponding working node according to the node memory usage and the total node memory of each working node obtained in advance; the first quantity determination sub-unit, which determines the first executable task quantity corresponding to the corresponding working node according to the expected memory occupancy of the resources consumed by the task prediction obtained in advance and the remaining memory space of the corresponding working node, wherein the first executable task quantity refers to the memory requirement that the corresponding working node can accommodate the corresponding number of tasks.

[0068] In an optional embodiment, the device further includes: a node normalization module, which, after obtaining the corresponding executable capability evaluation result, normalizes the executable capability evaluation result of each task based on the executable capability evaluation results of all tasks to obtain the executable capability normalization result of the corresponding task.

[0069] Furthermore, the node normalization module includes: a third score determination unit, which determines the maximum executable capability evaluation result and the minimum executable capability evaluation result based on the executable capability evaluation results of all tasks; a fifth difference determination unit, which obtains the fifth difference according to the executable capability evaluation result and the minimum executable capability evaluation result of the current task; a sixth difference determination unit, which obtains the sixth difference according to the maximum executable capability evaluation result and the minimum executable capability evaluation result; and a node normalization unit, which obtains the executable capability normalization processing result of the current task according to the ratio of the fifth difference to the sixth difference.

[0070] The orchestration and scheduling module 34 includes: a weight determination unit, which determines a first weight according to a resource consumption score, determines a second weight according to an indicator score, and determines a third weight according to an executable capability assessment result; wherein the first weight and the third weight have opposite signs, the sign of the second weight is positive, and the first weight, the second weight and the third weight are configured in advance based on the resource consumption score, the indicator score and the executable capability assessment result; a comprehensive scoring unit, which obtains a comprehensive score of the corresponding working node according to the resource consumption score, the first weight, the indicator score, the second weight, the executable capability assessment result and the third weight; and a scheduling unit, which selects a working node corresponding to the largest comprehensive score as the target working node according to the comprehensive score of each working node.

[0071] In summary, the embodiment of the present invention uses a resource evaluation module to evaluate the historical resource consumption data of the task, so as to accurately grasp the actual consumption of various resources of each task in the past execution process, so as to more accurately judge the resource demand of the task, which is helpful to allocate the task to the most suitable work node; the corresponding target indicator and weight are determined by the indicator evaluation module to obtain the indicator score, so as to further analyze the task from multiple dimensions such as the complexity, importance, and real-time requirements of the task, so as to ensure that various key characteristics of the task can play a role in the allocation decision, so that important and urgent tasks can be given priority, and complex tasks can also be reasonably arranged; the node evaluation module is used to determine the corresponding target indicator and weight according to the work node Status information is used to evaluate the possible execution capabilities of the corresponding work nodes, so as to understand the current resource surplus of each work node and the load status of the tasks being executed in real time, so as to fully consider the actual bearing capacity of the work nodes and avoid allocating too many tasks to the already overloaded nodes. Then, the orchestration and scheduling module comprehensively integrates the resource consumption score, indicator score and executable capability evaluation results to find the best balance between the task's demand for resources and the work node's supply capacity, ensuring that each task can be assigned to a work node that can meet its resource needs and give full play to its own characteristics, thereby achieving reasonable allocation of system resources and improving the efficiency of task orchestration and scheduling and the response speed to important tasks.

[0072] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor (processor) 410 , a communication interface (Communications Interface) 420 , a memory (memory) 430 and a communication bus 440 , wherein the processor 410 , the communication interface 420 , and the memory 430 communicate with each other through the communication bus 440 . The processor 410 can call the logic instructions in the memory 430 to execute the task scheduling method, which includes: evaluating the previously acquired task history resource consumption data to obtain the resource consumption score of the corresponding task; determining the target indicator corresponding to the corresponding task according to the task corresponding to the task history resource consumption data, combined with the preset task target indicator information, and determining the indicator score of the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to limit the allocation strategy of the weights corresponding to each indicator; according to the previously acquired work node status information, obtaining the executable capability evaluation result of each work node; wherein the work node status information is used to characterize the running status of the work node executing the corresponding task; according to the resource consumption score, the indicator score and the executable capability evaluation result, determining the target work node, and issuing the corresponding task to the target work node.

[0073] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the task scheduling method provided by the above-mentioned methods, which method includes: evaluating the previously acquired task history resource consumption data to obtain the resource consumption score of the corresponding task; determining the target indicator corresponding to the corresponding task according to the task corresponding to the task history resource consumption data, combined with the preset task target indicator information, and determining the indicator score of the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to limit the allocation strategy of the weight corresponding to each indicator; according to the previously acquired work node status information, obtaining the executable capability evaluation result of each work node; wherein the work node status information is used to characterize the running state of the work node executing the corresponding task; according to the resource consumption score, the indicator score and the executable capability evaluation result, determining the target work node, and issuing the corresponding task to the target work node.

[0075] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the task scheduling method provided by the above-mentioned methods, the method comprising: evaluating the previously acquired historical resource consumption data of the task to obtain a resource consumption score for the corresponding task; determining the target indicator corresponding to the corresponding task according to the task corresponding to the historical resource consumption data of the task, in combination with the preset task target indicator information, and determining the indicator score for the corresponding task in combination with the preset weight allocation rule; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each task, and the preset weight allocation rule is used to limit the allocation strategy of the weights corresponding to each indicator; obtaining the executable capability evaluation result of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running status of the working node in executing the corresponding task; determining the target working node according to the resource consumption score, the indicator score and the executable capability evaluation result, and issuing the corresponding task to the target working node.

[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task scheduling method, characterized in that: include: Evaluate the previously acquired historical resource consumption data of the task to obtain the resource consumption score of the corresponding task; According to the task corresponding to the task historical resource consumption data, combined with the preset task target indicator information, the target indicator corresponding to the corresponding task is determined, and combined with the preset weight allocation rule, the indicator score of the corresponding task is determined; wherein the preset task target indicator information includes multiple tasks and the target indicators corresponding to each of the tasks, and the preset weight allocation rule is used to define the allocation strategy of the weight corresponding to each indicator; Obtaining an executable capability evaluation result of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running status of the working node in executing the corresponding task; According to the resource consumption score, the indicator score and the executable capability evaluation result, a target working node is determined, and a corresponding task is issued to the target working node.

2. The task scheduling method according to claim 1, characterized in that: According to the task corresponding to the task historical resource consumption data, combined with preset task target indicator information, the target indicator corresponding to the corresponding task is determined, and combined with the preset weight allocation rule, the indicator score of the corresponding task is determined, including: Determine the target indicator of the corresponding task according to the task corresponding to the task historical resource consumption data and in combination with the preset task target indicator information; According to the target indicators of the task, based on the preset weight allocation rules, determine the allocation weight of each target indicator; According to the allocation weight of each target indicator and the target indicator of the task, the indicator score of the corresponding task is obtained.

3. The task scheduling method according to claim 2, characterized in that: According to the target indicators of the task, based on the preset weight allocation rules, the allocation weights of the target indicators are determined, including: According to each target indicator of the task, the preset weight allocation rule is searched respectively to determine the allocation weight corresponding to each target indicator; wherein the preset weight allocation strategy is used to limit the allocation weight of each target indicator; or, According to the system operation status information obtained previously, combined with the preset weight allocation strategy, weights are allocated to each target indicator of the task; wherein the preset weight allocation strategy is used to limit the allocation weight rules of each target indicator under different system operation states, and the system operation status information is used to characterize the operation status of all working nodes and the overall computing environment of each working node running tasks.

4. The task scheduling method according to claim 1, characterized in that: According to the task corresponding to the task historical resource consumption data, combined with preset task target indicator information, the target indicator corresponding to the corresponding task is determined, and combined with a preset weight allocation rule, the indicator score of the corresponding task is determined, further comprising: According to the task corresponding to the task historical resource consumption data, combined with preset task target indicator information, determine the target indicator of the corresponding task; wherein the preset task target indicator information includes multiple tasks and the target indicator corresponding to each of the tasks; According to the target indicator of the task and the previously acquired system operation status information, the weight of each target indicator of the task is obtained by using a weight prediction model; wherein the system operation status information is used to characterize the operation status of all working nodes and the overall computing environment of each working node running the task, and the weight prediction model is obtained by training based on the indicator training data, the system operation status training data corresponding to the indicator training data, and the corresponding indicator weight label; According to the allocation weight of each target indicator and the target indicator of the task, the indicator score of the corresponding task is obtained.

5. The task scheduling method according to claim 1, characterized in that: The working node status information includes the node memory usage, the node central processing unit CPU usage and the type of task currently executed by the node; According to the previously acquired working node status information, the executable capability evaluation results of each working node are obtained, including: Determine the number of first executable tasks for the corresponding working node according to the node memory usage and the previously acquired total node memory of each working node and the predicted consumption resources of the task; According to the node CPU usage, the node CPU performance parameters obtained previously, and the predicted resource consumption of the task, the number of second executable tasks of the corresponding working node is evaluated; Compare the first number of executable tasks and the second number of executable tasks of the corresponding working node, and select the smaller number as the third number of executable tasks; wherein the third number of executable tasks is used to represent the number of executable tasks corresponding to the idle resources of the corresponding working node; According to the task and the previously acquired task dependency, the number of the third executable tasks is adjusted to obtain an executable capability evaluation result of the corresponding working node; wherein the task dependency is used to characterize the compatibility relationship and competition relationship between tasks.

6. The task scheduling method according to claim 1, characterized in that: Determining a target work node according to the resource consumption score, the indicator score, and the executable capability evaluation result includes: Determine a first weight according to the resource consumption score, determine a second weight according to the indicator score, and determine a third weight according to the executable capability evaluation result; wherein the first weight and the third weight have opposite signs, the second weight has a positive sign, and the first weight, the second weight and the third weight are configured based on the resource consumption score, the indicator score and the executable capability evaluation result; Obtaining a comprehensive score of the corresponding work node according to the resource consumption score, the first weight, the indicator score, the second weight, the executable capability evaluation result, and the third weight; According to the comprehensive scores of the working nodes, the working node corresponding to the largest comprehensive score is selected as the target working node.

7. The task scheduling method according to claim 1, characterized in that: Evaluate the previously acquired historical resource consumption data of the task to obtain the resource consumption score of the corresponding task, including: Acquire historical resource consumption data of tasks, wherein the historical resource consumption data of tasks includes a plurality of tasks and resource consumption data corresponding to each of the tasks; For each task, sort the resource consumption data corresponding to the task to obtain a resource sorting result of the corresponding task; For the resource ranking result of a single task, combined with the corresponding preset truncation ratio, the extreme values ​​in the ranking result are removed, and the average value of the remaining data in the ranking result is determined to obtain the resource consumption score of the corresponding task.

8. A task scheduling device, characterized in that: include: The resource evaluation module evaluates the previously acquired historical resource consumption data of the task and obtains the resource consumption score of the corresponding task; An indicator evaluation module determines the target indicator corresponding to the corresponding task according to the task corresponding to the task historical resource consumption data and in combination with preset task target indicator information, and determines the indicator score of the corresponding task in combination with preset weight allocation rules; wherein the preset task target indicator information includes multiple tasks and target indicators corresponding to each of the tasks, and the preset weight allocation rules are used to define the allocation strategy of the weights corresponding to each indicator; A node evaluation module obtains an evaluation result of the executable capability of each working node according to the previously acquired working node status information; wherein the working node status information is used to characterize the running status of the working node in executing the corresponding task; The orchestration and scheduling module determines the target working node according to the resource consumption score, the indicator score and the executable capability evaluation result, and sends the corresponding task to the target working node.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the task scheduling method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the task scheduling method according to any one of claims 1 to 7 is implemented.

Images