Calculation power network workflow task scheduling method and system based on task space-time attributes

Abstract

The invention relates to the technical field of workflow task scheduling, and provides a computing power network workflow task scheduling method and system based on task space-time attributes. The method comprises the following steps: respectively calculating the total priority of a plurality of workflows based on obtained user priorities, job priorities and task resource demand quotas of the plurality of workflows, and sorting according to the total priority to obtain a task queue of the workflows; constructing a multi-objective function based on the completion time, the execution cost and the resource matching degree of the workflow; and based on the multi-objective function and the computing power platform to be selected, judging the relationship between the resource quantity required by the workflow in the task queue and the resource demand quantity of the computing power platform to be selected, and adopting a multi-objective artificial rabbit optimization algorithm to select the computing power platform meeting the requirement for execution. According to the method, efficient task execution considering the task time attribute, the space attribute and the scheduling service quality in multiple dimensions can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of workflow task scheduling, and in particular to a computing power network workflow task scheduling method and system based on task spatiotemporal attributes. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Computing power has become an important resource, and a large number of artificial intelligence are applied to various industries. In daily life, artificial intelligence has become an indispensable tool for people. With the development of artificial intelligence technology, computing needs are increasing day by day, which requires more computing power resources. However, computing centers, that is, various computing platforms, are scattered all over the country and cannot coordinate to deal with a large number of tasks with diverse requirements. Therefore, it is proposed to establish a computing network through network connections between multiple computing platforms to uniformly handle task requirements and match resources globally. However, the scheduling method proposed based on the computing network cannot meet the requirements of achieving the optimal mapping between tasks and resources. In addition, current research rarely considers the connection between workflows in execution time and the connection between the geographical locations of submitted tasks. Summary of the invention

[0004] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a computing power network workflow task scheduling method and system based on the temporal and spatial attributes of tasks. The present invention studies the time and spatial attributes of workflow tasks and matches them appropriately in the resources accessed by the computing power network, so as to achieve efficient execution of tasks taking into account multiple dimensions of task time attributes, spatial attributes, and scheduling service quality.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] The first aspect of the present invention provides a computing power network workflow task scheduling method based on task spatiotemporal attributes.

[0007] A computing network workflow task scheduling method based on task spatiotemporal attributes, comprising:

[0008] Based on the obtained user priorities, job priorities and task resource requirement quotas of the multiple workflows, the total priorities of the multiple workflows are calculated respectively, and they are sorted according to the total priorities to obtain a task queue for executing the workflows;

[0009] A multi-objective function is constructed based on the completion time, execution cost and resource matching degree of the workflow. Based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the resource demand of the computing power platform to be selected is judged, and the multi-objective artificial rabbit optimization algorithm is used to select the computing power platform that meets the requirements for execution.

[0010] Furthermore, the method of constructing a multi-objective function based on the completion time, execution cost and resource matching degree of the workflow includes:

[0011] According to the workflow start execution time and task execution time, the completion time of all workflows is obtained;

[0012] Calculate user costs based on the completion time of all workflows, the time when workflows enter the workflow scheduler, the billing cycle of the computing power platform, and the unit price of each billing cycle of the computing power platform;

[0013] Calculate resource matching based on the resource requirements of all workflows executed on a certain computing platform;

[0014] Based on minimizing the completion time of all workflows, minimizing user costs and minimizing resource matching, a multi-objective function is obtained.

[0015] Furthermore, based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the amount of resources required by the computing power platform to be selected is judged, and a computing power platform that meets the requirements is selected for execution using a multi-objective artificial rabbit optimization algorithm; the method includes:

[0016] If the amount of resources required by the workflow is greater than the amount of resources required by the computing power platform to be selected, a computing power platform is obtained by using a multi-objective artificial rabbit optimization algorithm for the workflow within a time period T before its submission time; a probability is set for the computing power platform, and the computing power platform is used as an optional computing power platform with the probability within a time period T before the submission time; when the workflow is submitted, the workflow is scheduled to the pre-selected computing power platform, the probability selection method for the computing power platform is cleared, and all computing power platforms are continued to be scheduled;

[0017] If the amount of resources required by the workflow is less than or equal to the resource demand of the computing power platform to be selected, and the number of all workflow description files is much larger than the number of computing power platforms, then within the T time period before these workflows, select the computing power platform that is closest to the geographical location of the workflow submission device; for other computing power platforms whose bandwidth with the computing power platform is higher than the average, reduce workflow scheduling, wait for all workflows to be submitted, and then schedule the nearest computing power platform and the other computing power platforms to process the workflow.

[0018] Furthermore, the total priority of each workflow is the cumulative sum of the user priority, the job priority and the task resource requirement amount of each workflow.

[0019] Furthermore, the task resource requirement is calculated using the following formula:

[0020]

[0021] Among them, w represents the task resource requirement, α and β are coefficients used for normalization. Indicates the maximum amount of CPU resources used by the workflow. Indicates the maximum number of MEM resources used by the workflow.

[0022] Furthermore, the process of adopting the multi-objective artificial rabbit optimization algorithm includes:

[0023] The population is initialized using the Halton low-discrepancy sequence method. Each rabbit is a task scheduling scheme, with a total of N solutions.

[0024] Use the non-dominated sorting method to sort all solutions and calculate the congestion of all solutions to obtain a new population that is consistent with the population size; if the number of iterations is reached, output the result, otherwise continue to execute;

[0025] According to the energy contraction mechanism, if A>1, the detour foraging strategy is selected, otherwise the random hiding strategy is selected; according to the two strategies, a total of 2N solutions are obtained, and the population is updated;

[0026] The random hiding strategy adjusts the fitness changes based on the historical records during the iteration process. If the fitness changes are always within a fixed range in multiple iterations, the step size is adaptively adjusted to jump out of the local optimal solution.

[0027] The second aspect of the present invention provides a computing power network workflow task scheduling system based on task spatiotemporal attributes.

[0028] A computing network workflow task scheduling system based on task spatiotemporal attributes, comprising:

[0029] A priority sorting module is configured to: calculate the total priority of the multiple workflows based on the user priority, job priority and task resource requirement quota of the multiple workflows obtained, sort them according to the total priority, and obtain a task queue of the workflow;

[0030] The computing power platform allocation module is configured as follows: based on the completion time, execution cost and resource matching degree of the workflow, a multi-objective function is constructed; based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the resource demand of the computing power platform to be selected is judged, and a multi-objective artificial rabbit optimization algorithm is used to select a computing power platform that meets the requirements for execution.

[0031] A third aspect of the present invention provides a computer-readable storage medium.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the computing power network workflow task scheduling method based on task spatiotemporal attributes as described in the first aspect above.

[0033] A fourth aspect of the present invention provides a computer device.

[0034] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for scheduling computing power network workflow tasks based on the spatiotemporal attributes of tasks as described in the first aspect above are implemented.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention proposes a computing network workflow task scheduling method and system based on the temporal and spatial attributes of tasks, which selects a computing platform for the workflow and executes each task of the workflow within the cluster of the computing platform. In the use scenario where the time and spatial attributes of workflow scheduling are sensitive, the present invention reduces the repetitiveness of uploading data sets required for workflows, improves resource utilization efficiency, and improves user satisfaction.

[0037] The present invention provides a method for managing and using resources of multiple computing platforms distributed in different places, utilizing scheduling history data, taking into account the time and space attributes of task execution, and scheduling service requirements, to achieve optimal scheduling of global resources for each workflow in any time period, making full use of computing platforms, quickly processing workflows, and improving user satisfaction. A plug-in for workflow task scheduling system is also developed for computing clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0039] Figure 1 It is a framework diagram of computing network workflow task scheduling based on task spatiotemporal attributes shown in the present invention;

[0040] Figure 2 It is a flow chart of a computing network workflow task scheduling method based on task spatiotemporal attributes shown in the present invention;

[0041] Figure 3 It is a flow chart of the multi-objective artificial rabbit optimization algorithm shown in the present invention;

[0042] Figure 4 It is a workflow task execution diagram shown in the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0044] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0046] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0047] Embodiment 1

[0048] like Figure 1As shown, this embodiment provides a method for scheduling computing network workflow tasks based on the spatiotemporal attributes of tasks. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0049] Based on the obtained user priorities, job priorities and task resource requirement quotas of the multiple workflows, the total priorities of the multiple workflows are calculated respectively, and they are sorted according to the total priorities to obtain a task queue for executing the workflows;

[0050] A multi-objective function is constructed based on the completion time, execution cost and resource matching degree of the workflow. Based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the resource demand of the computing power platform to be selected is judged, and the multi-objective artificial rabbit optimization algorithm is used to select the computing power platform that meets the requirements for execution.

[0051] This embodiment provides a computing network workflow task scheduling method based on task spatiotemporal attributes. The method involves the execution order of different workflows and the selection of computing platforms, and includes executing workflows in a cluster of scheduled computing platforms.

[0052] Assume that the workflow is W = {w1,w2,…,w w},in Include workflow ID User Priority Workflow Priority Submission time α w , data set capacity d w . Subtask properties is the identifier of the subtask, V w = {v w,1 ,v w,2 ,…,v w,m}Includes a set of subtasks of workflow w, with a total of m tasks. Indicates v w,t With vw,m The data transmission relationship between them.

[0053] Assume that the computing power platform is P = {p1, p2, ..., p i},in Is the computing power platform logo, a i is the processing speed of a single core of the computing platform, u i is the billing cycle, is the unit price of computing power platform i in each billing cycle, b i is the internal bandwidth of computing platform i, b i,j The bandwidth between computing platforms i and j. For resource usage, They are the CPU and MEM resource usage of the computing platform, is the total amount of CPU and MEM resources of the computing platform, It is the utilization rate of CPU and MEM resources of the computing platform. A computing platform includes multiple computing clusters to process tasks. The computing cluster includes multiple nodes, which execute the subtasks of the workflow in the form of pods. The pod will be scheduled to a selected node for execution and return the execution results.

[0054] like Figure 1 , 2 As shown, the method described in this embodiment includes the following steps:

[0055] Step 1: The workflow scheduler receives multiple workflow description files, calculates the total priority according to the workflow description files, sorts the multiple workflows, and puts the workflows into the task queue in order.

[0056] Step 2: According to the multi-objective artificial rabbit optimization algorithm, the tasks in the queue are scheduled based on the completion time, execution cost, and resource matching of the three objectives. The optimal scheduling scheme is obtained based on the spatiotemporal attributes of the workflows. Then, the workflow description file and the required data set are sent to the mapped computing platform according to the scheduling scheme.

[0057] Step 3: Select a computing cluster with the least number of current task executions from multiple computing clusters on the computing power platform to execute the workflow subtasks. The task scheduling module of the computing cluster determines the execution order of the workflow subtasks according to the minimum execution time target, and obtains the task execution queue to improve the execution speed of the workflow.

[0058] Step 4: Generate pods for the currently ready tasks in the workflow according to the task execution queue.

[0059] Step 5: Send the pod to the k8s scheduling module. According to the order in which the pods are generated and the resource usage of each node in the cluster, execute the subtasks on the allocated nodes, return the execution results, and execute step 4 until all tasks in the workflow are completed.

[0060] In some embodiments, the total priority calculation in step 1 includes:

[0061] Combined with user priorities in the workflow description file Job Priority Task resource requirement w Determine the workflow execution order. User priority and job priority are defined as positive integers in the range of (1, 10), and the resource demand quota is normalized to the range of (1, 10). Based on the factors affecting the urgency and scale of workflow execution described above, the final priority is obtained. The larger the value, the higher the priority. The workflow execution queue is obtained according to the priority, and the workflow scheduling order is determined. The formula is shown in (1).

[0062]

[0063] Resource requirement quota considers CPU and MEM usage, CPU to MEM ratio and comprehensive consideration of a variable, describing the ratio of different resources used by the workflow. For workflows that use more resources, tasks with frequent communication are executed first. This is because tasks with frequent communication will occupy more memory resources relative to computing resources.

[0064] Use depth-based sorting for the workflow, calculate the CPU and MEM requirements of the tasks contained in each depth, and compare to get the maximum number of CPU resources used by the workflow and the number of MEM resources α and β are coefficients used for normalization. The calculation formula is shown in (2).

[0065]

[0066] In some embodiments, the multi-workflow scheduling of step 2 includes:

[0067] A multi-objective function is established with three objectives: total workflow completion time, total workflow execution cost, and resource matching. The multi-objective function is used to obtain the optimal scheduling scheme for multiple workflows and select a suitable computing platform for the workflow to execute.

[0068] After the workflow is submitted, it will be stored in the workflow scheduling queue and wait for the workflow scheduler to check it regularly. When using the multi-objective algorithm to obtain the scheduling plan, if there are X workflows to be scheduled, the multi-objective function is used to obtain the optimal scheduling plan for the X workflows, and the appropriate computing power platform is selected for the workflow to execute.

[0069] When establishing a multi-objective function, minimizing the total workflow completion time is taken as one goal, and the calculation formula is as follows (3):

[0070]

[0071] Using the idea of ​​heterogeneous earliest time scheduling algorithm, after determining the computing power platform for workflow scheduling, use the upward sorting algorithm, starting from the node without successor tasks in the DAG graph, and recursively calculate the priority of each task from bottom to top. The value of the earliest time required for the start of the task to the completion of all predecessor tasks is selected as the priority of each task. Tasks with high priority will be scheduled first. Calculate the earliest time that the task can be completed on this processor The task starts executing at time t start (v w,m ,p k ) and task execution time t tc (v w,m ,p k ) together. The predecessor task is the task that must be completed before the current workflow task is executed. The start execution time of the task is the end time t of the last predecessor task of the task. end (v w,m ,p n ) and data transmission time The start execution time of the task is t start (v w,t ,p k ), including the completion time t of each task's predecessor task tc (v w,m ,p k ) and data transmission time When a task without a predecessor task in a workflow starts to be scheduled, the start execution time of the task is the data transmission time t of the data set required by the workflow to be transmitted to the computing platform dt (w w ,p k ,p s ). Until all tasks are scheduled, the completion time of the last task is taken to get the completion time of the workflow. The execution time of the subtasks of the workflow is calculated by the task load l i,j Cluster processing speed k Calculated. dt (w w ,pk ) is the workflow w on the computing platform p k Data transmission time, b s,k It is a computing power platform k With p s The bandwidth between w is the capacity of the dataset required by workflow w. The transfer time between task t and task m of workflow w. is the data capacity between task t and task m of workflow w, b k It is a computing power platform k The internal bandwidth of the workflow. The completion time of the workflow is calculated as shown in formulas (4), (5), (6), (7), (8), and (9).

[0072]

[0073] When establishing a multi-objective function, minimizing user cost is taken as one of the objectives, and the calculation formula is shown in (10):

[0074]

[0075] Among them, c w The user cost of the wth workflow is obtained by dividing the billing cycle of the selected computing platform by the workflow execution time, rounding up to get the number of billing cycles, and multiplying by the unit price. The total user cost is obtained by adding up the user costs of all workflows in the workflow queue. is the time when workflow w enters the workflow scheduler, u k is the billing cycle of the k computing power platform, is the unit price of each billing cycle of the k computing power platform. The calculation formula is shown in (11)

[0076]

[0077] When establishing a multi-objective function, minimizing resource matching is taken as one of the objectives, and the calculation formula is shown in (12):

[0078]

[0079] Resource matching is used to measure the balance between the processor resources and memory resources of the computing platform where the task is scheduled. Subtract the remaining resources of computing platform k from the CPU and MEM resource requirements of workflow w to get the CPU and MEM resource utilization of workflow w, and subtract the CPU and MEM resource utilization of the original computing platform to get the CPU and MEM differences before and after applying workflow w. The resource matching degree of workflow w executed on computing platform k. If the resource matching degree is smaller, it means that the CPU and MEM resource usage of the computing platform selected by this workflow are close, and the resource matching degree is high. The calculation formula is shown in (15). is the CPU resource usage of executing workflow w on computing platform k, is the MEM resource usage rate of executing workflow w on computing platform k, and the calculation formulas are shown in (13) and (14). cpu is the CPU usage of computing platform k of unscheduled workflows, is the MEM usage rate of computing platform k that has not scheduled workflows. They are the maximum resource utilization of computing power platform k.

[0080]

[0081] like Figure 3 As shown, the multi-objective artificial rabbit optimization algorithm includes the following steps:

[0082] Step (a): Use Halton's low-discrepancy sequence method to initialize the population. Each rabbit is a task scheduling solution, with a total of N solutions.

[0083] Step (b): Use the non-dominated sorting method to sort all solutions and calculate the congestion of all solutions to obtain a new population that is consistent with the population size. If the number of iterations is reached, output the result, otherwise continue to execute.

[0084] Step (c): According to the energy contraction mechanism, if A>1, choose the detour foraging strategy, otherwise choose the random hiding strategy. According to the two strategies, a total of 2N solutions are obtained, and step (b) is executed to update the population.

[0085] Step (d): The random hiding strategy is adjusted based on the fitness change of the historical records during the iteration process. If the fitness change is always within a fixed range in multiple iterations, the step size is adaptively adjusted to jump out of the local optimal solution.

[0086] After the multi-objective optimization algorithm, a set of scheduling schemes will be obtained, and a scheduling scheme will be randomly selected for multi-workflow scheduling.

[0087] Suppose the scheduling history is SR = {sr1, sr2, ..., sr n}, Based on workflow submission time at n , the IP address of the device where the workflow is submitted n 、Number of workflow tasks n Using clustering algorithms, tasks that are classified into one category are judged as periodic workflow tasks. and Judge, set cpu large and mem large is the average value of CPU and MEM requirements of large workflows, if and For this workflow, use the multi-objective artificial rabbit optimization algorithm to obtain a computing power platform within a T time period before its submission time, and reserve resources for it. Set a probability p for the computing power platform, and use probability p as an optional computing power platform within a T time period before the submission time. When the submission time of the workflow history record arrives, schedule it to the pre-selected computing power platform, delete probability p, and continue to schedule all computing power platforms. If it is not satisfied And the number of submitted workflows >> M, for these workflows within a time period T before their submission time, select computing platforms that are close to the geographical location of the workflow submission device and other computing platforms with bandwidth higher than the average between them, reduce workflow scheduling for these computing platforms, wait for the workflow to be submitted, and schedule the above computing platforms to process the workflow. M is the number of computing platforms.

[0088] The reduction of computing power platform scheduling includes setting a probability p for some computing power platforms when executing the multi-objective optimization algorithm, and when selecting a computing power platform for a workflow, the computing power platform is selected as a resource for scheduling the workflow with probability p.

[0089] When selecting some computing platforms for the multi-workflow task with the number of submitted workflows >> M, the geographical location is calculated using the IP address, and the computing platforms with a close geographical location to the workflow submission device and the network bandwidth of other computing platforms are classified using a clustering algorithm to obtain a group of computing platforms with high network bandwidth and close geographical distance.

[0090] In some embodiments, the workflow scheduling of step 3 includes:

[0091] After the workflow definition file is sent to the specified cluster, the task scheduling module finds the task with the input file as the ready task based on the input and output files of the task defined by the workflow, and generates a pod for the ready task. The heterogeneous earliest completion time scheduling algorithm is used to obtain the workflow task execution order with the fastest execution speed as the goal.

[0092] In some embodiments, the task scheduling in step 4 includes:

[0093] For multiple ready tasks of a workflow, pods are generated according to the task execution order generated in step 3.

[0094] If there are multiple workflows that need to be scheduled, pods are generated for the ready tasks of the multiple workflows respectively. The pod generation process of the tasks between the workflows is independent of each other. Then proceed to step 4.

[0095] In some embodiments, the task execution of step 5 includes:

[0096] ETCD is a highly available distributed key-value database.

[0097] The task becomes a pod and the execution process in the k8s scheduler is as follows Figure 4 As shown in the figure, the scheduler receives the generated Pod and enters the Pod to be scheduled queue. The scheduler selects the most suitable node from the available Node queue for each Pod in the Pod to be scheduled queue in turn according to the scheduling strategy. Then the Kublet process on this node listens to the Pod binding event issued by the Kubernetes scheduler through the API Server, and then obtains the corresponding Pod resource description file from etcd, downloads the Image image, and starts the container.

[0098] Embodiment 2

[0099] This embodiment provides a computing power network workflow task scheduling system based on task temporal and spatial attributes.

[0100] A computing network workflow task scheduling system based on task spatiotemporal attributes, comprising:

[0101] A priority sorting module is configured to: calculate the total priority of the multiple workflows based on the user priority, job priority and task resource requirement quota of the multiple workflows obtained, sort them according to the total priority, and obtain a task queue of the workflow;

[0102] The computing power platform allocation module is configured as follows: based on the completion time, execution cost and resource matching degree of the workflow, a multi-objective function is constructed; based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the resource demand of the computing power platform to be selected is judged, and a multi-objective artificial rabbit optimization algorithm is used to select a computing power platform that meets the requirements for execution.

[0103] In some embodiments, the priority sorting module is configured to: obtain the completion time of all workflows based on the start execution time of the workflow and the task execution time; calculate the user cost based on the completion time of all workflows, the time when the workflow enters the workflow scheduler, the billing cycle of the computing power platform, and the unit price of each billing cycle of the computing power platform; calculate the resource matching degree based on the resource requirements of all workflows executed on a certain computing power platform; obtain a multi-objective function based on minimizing the completion time of all workflows, minimizing user costs and minimizing resource matching degree.

[0104] In some embodiments, the computing power platform allocation module is further configured to:

[0105] If the amount of resources required by the workflow is greater than the amount of resources required by the computing power platform to be selected, a computing power platform is obtained by using a multi-objective artificial rabbit optimization algorithm for the workflow within a time period T before its submission time; a probability is set for the computing power platform, and the computing power platform is used as an optional computing power platform with the probability within a time period T before the submission time; when the workflow is submitted, the workflow is scheduled to the pre-selected computing power platform, the probability selection method for the computing power platform is cleared, and all computing power platforms are continued to be scheduled;

[0106] If the amount of resources required by the workflow is less than or equal to the resource demand of the computing power platform to be selected, and the number of all workflow description files is much larger than the number of computing power platforms, then within the T time period before these workflows, select the computing power platform that is closest to the geographical location of the workflow submission device; for other computing power platforms whose bandwidth with the computing power platform is higher than the average, reduce workflow scheduling, wait for all workflows to be submitted, and then schedule the nearest computing power platform and the other computing power platforms to process the workflow.

[0107] In some embodiments, the total priority of each workflow is the cumulative sum of the user priority, job priority and task resource requirement amount of each workflow.

[0108] In some embodiments, the task resource requirement is calculated using the following formula:

[0109]

[0110] Among them, w represents the task resource requirement, α and β are coefficients used for normalization. Indicates the maximum amount of CPU resources used by the workflow. Indicates the maximum number of MEM resources used by the workflow.

[0111] In some embodiments, the computing power platform allocation module is further configured to: use the Halton low-discrepancy sequence method to initialize the population, where each rabbit is a task scheduling scheme, and there are N solutions in total; use the non-dominated sorting method for all solutions, and calculate the congestion of all solutions to obtain a new population consistent with the population size; when the number of iterations is reached, output the result, otherwise continue to execute; according to the energy contraction mechanism, if A>1, select the detour foraging strategy, otherwise select the random hiding strategy; according to the two strategies, a total of 2N solutions are obtained, and the population is updated; the random hiding strategy is adjusted based on the fitness changes in the historical records during the iteration process. If the fitness changes are always within a fixed interval in multiple iterations, the step size is adaptively adjusted to jump out of the local optimal solution.

[0112] The present invention matches appropriate dispersed resources by studying the time and space attributes of workflow tasks, and can achieve efficient execution of tasks that take into account multiple dimensions including time attributes, space attributes, and scheduling service quality.

[0113] Embodiment 3

[0114] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the computing power network workflow task scheduling method based on task spatiotemporal attributes as described in the above-mentioned embodiment 1 are implemented.

[0115] Embodiment 4

[0116] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for scheduling computing power network workflow tasks based on the spatiotemporal attributes of tasks as described in the first embodiment above are implemented.

[0117] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0118] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A computing network workflow task scheduling method based on task spatiotemporal attributes, characterized in that: include: Based on the obtained user priorities, job priorities and task resource requirement quotas of the multiple workflows, the total priorities of the multiple workflows are calculated respectively, and they are sorted according to the total priorities to obtain a task queue for executing the workflows; A multi-objective function is constructed based on the completion time, execution cost and resource matching degree of the workflow. Based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the resource demand of the computing power platform to be selected is judged, and the multi-objective artificial rabbit optimization algorithm is used to select the computing power platform that meets the requirements for execution.

2. The computing network workflow task scheduling method based on task spatiotemporal attributes according to claim 1 is characterized in that: The method of constructing a multi-objective function based on the completion time, execution cost and resource matching degree of the workflow includes: According to the workflow start execution time and task execution time, the completion time of all workflows is obtained; Calculate user costs based on the completion time of all workflows, the time when workflows enter the workflow scheduler, the billing cycle of the computing power platform, and the unit price of each billing cycle of the computing power platform; Calculate resource matching based on the resource requirements of all workflows executed on a certain computing platform; Based on minimizing the completion time of all workflows, minimizing user costs and minimizing resource matching, a multi-objective function is obtained.

3. The computing network workflow task scheduling method based on task spatiotemporal attributes according to claim 1 is characterized in that: Based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the amount of resources required by the computing power platform to be selected is judged, and a computing power platform that meets the requirements is selected for execution using a multi-objective artificial rabbit optimization algorithm; the method includes: If the amount of resources required by the workflow is greater than the resource demand of the computing power platform to be selected, a computing power platform is obtained by using a multi-objective artificial rabbit optimization algorithm within a T time period before the submission time of the workflow; a probability is set for the computing power platform, and the computing power platform is used as an optional computing power platform with the probability within a T time period before the submission time; when the workflow is submitted, the workflow is scheduled to the pre-selected computing power platform, the probability selection method for the computing power platform is cleared, and all computing power platforms continue to be scheduled.

4. The computing network workflow task scheduling method based on task spatiotemporal attributes according to claim 3 is characterized in that: Based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the amount of resources required by the computing power platform to be selected is judged, and a computing power platform that meets the requirements is selected for execution using a multi-objective artificial rabbit optimization algorithm; the method also includes: If the amount of resources required by the workflow is less than or equal to the resource demand of the computing power platform to be selected, and the number of all workflow description files is much larger than the number of computing power platforms, then within the T time period before these workflows, select the computing power platform that is closest to the geographical location of the workflow submission device; for other computing power platforms whose bandwidth with the computing power platform is higher than the average, reduce workflow scheduling, wait for all workflows to be submitted, and then schedule the nearest computing power platform and the other computing power platforms to process the workflow.

5. The computing network workflow task scheduling method based on task spatiotemporal attributes according to claim 1 is characterized in that: The total priority of each workflow is the cumulative sum of the user priority, job priority and task resource requirement amount of each workflow.

6. The computing network workflow task scheduling method based on task spatiotemporal attributes according to claim 1 or 5 is characterized in that: The task resource requirement is calculated using the following formula: Among them, w represents the task resource requirement, α and β are coefficients used for normalization. Indicates the maximum amount of CPU resources used by the workflow. Indicates the maximum number of MEM resources used by the workflow.

7. The computing network workflow task scheduling method based on task spatiotemporal attributes according to claim 1 is characterized in that: The process of adopting the multi-objective artificial rabbit optimization algorithm includes: The population is initialized using the Halton low-discrepancy sequence method. Each rabbit is a task scheduling scheme, with a total of N solutions. Use the non-dominated sorting method to sort all solutions and calculate the congestion of all solutions to obtain a new population that is consistent with the population size; if the number of iterations is reached, output the result, otherwise continue to execute; According to the energy contraction mechanism, if A>1, the detour foraging strategy is selected, otherwise the random hiding strategy is selected; according to the two strategies, a total of 2N solutions are obtained, and the population is updated; The random hiding strategy adjusts the fitness changes based on the historical records during the iteration process. If the fitness changes are always within a fixed range in multiple iterations, the step size is adaptively adjusted to jump out of the local optimal solution.

8. A computing network workflow task scheduling system based on task spatiotemporal attributes, characterized by: include: A priority sorting module is configured to: calculate the total priority of the multiple workflows based on the user priority, job priority and task resource requirement quota of the multiple workflows obtained, sort them according to the total priority, and obtain a task queue of the workflow; The computing power platform allocation module is configured as follows: based on the completion time, execution cost and resource matching degree of the workflow, a multi-objective function is constructed; based on the multi-objective function and the computing power platform to be selected, the relationship between the amount of resources required by the workflow in the task queue and the resource demand of the computing power platform to be selected is judged, and a multi-objective artificial rabbit optimization algorithm is used to select a computing power platform that meets the requirements for execution.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the computing power network workflow task scheduling method based on task spatiotemporal attributes as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the computing power network workflow task scheduling method based on the temporal and spatial attributes of tasks as described in any one of claims 1 to 7 are implemented.

Images