A method for scheduling heterogeneous computing resources

Abstract

The present invention discloses a heterogeneous computing power resource scheduling method, relates to the technical field of computing resource scheduling, and effectively improves computing resource utilization. The present invention divides an image processing task into a plurality of image processing subtasks of different task types and sorts them, allocates image processing subtasks of the same task type to the same heterogeneous computing node, sets a plurality of central task nodes and edge task nodes in the heterogeneous computing node, splits the image processing subtask into a plurality of task steps and allocates them to each edge task node for execution, sets a reward and punishment mechanism, reduces or increases computing resources of the heterogeneous computing node according to whether the image processing subtask is completed within the task constraint time, and when each heterogeneous computing node completes its allocated image processing subtask, splices the task processing results of each image processing subtask, and then outputs the image processing task result.

Description

Technical Field

[0001] The present invention relates to the technical field of computing resource scheduling, and in particular to a method for scheduling heterogeneous computing resources. Background Art

[0002] Heterogeneous computing resource scheduling refers to the process of effectively allocating and managing computing resources of different types and performance in a computing environment. The system needs to reasonably allocate tasks to different types of computing resources based on the nature and requirements of the tasks to maximize the overall performance and efficiency of the system. This scheduling needs to take into account the differences and complexity between resources, as well as factors such as task priority and time constraints, so as to achieve effective management and utilization of resources.

[0003] The existing heterogeneous computing resource scheduling has the following defects:

[0004] Resource utilization: Since heterogeneous computing resources have different performance and characteristics, there is a lot of room for improvement in resource utilization with existing technologies, making it difficult to maximize resource utilization.

[0005] Dynamic adjustment of resources: Existing technologies have certain difficulties in dynamic adjustment of resources, and are unable to quickly and effectively dynamically adjust and reconfigure resources, resulting in low resource utilization and performance loss.

[0006] Therefore, how to improve computing resource utilization while reducing performance loss is a difficulty in the existing technology. For this purpose, a heterogeneous computing resource scheduling method is provided. Summary of the invention

[0007] In order to solve the above technical problems, the purpose of the present invention is to provide a heterogeneous computing resource scheduling method.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for scheduling heterogeneous computing resources includes the following steps:

[0010] Step S1, setting up a dynamic computing resource network, and setting up a number of heterogeneous computing nodes and a shared computing resource pool in the dynamic computing resource network;

[0011] Step S2, dividing the image processing task into a number of image processing subtasks of different task types, setting a start point and an end point for each image processing subtask, matching the start point and the end point of each image processing subtask with each other, and then sorting each image processing subtask, and allocating image processing subtasks of the same task type to the same heterogeneous computing node;

[0012] Step S3: Set a number of central task nodes and edge task nodes on the heterogeneous computing nodes, set the same task constraint duration for each heterogeneous computing node, split the image processing subtask into a number of task steps and assign them to each edge task node for execution, and mark the preconditions required for each task;

[0013] Step S4, the edge task node sends the sub-processing result of the task step to the central task node, and then the central task nodes in different heterogeneous computing nodes interact with each other and distribute the interacted sub-processing results to the required edge task nodes;

[0014] Step S5: Setting a reward and punishment mechanism to reduce or increase computing resources of heterogeneous computing nodes according to whether they complete the image processing subtask within the task constraint time.

[0015] Step S6: When each heterogeneous computing node completes its assigned image processing subtask, the dynamic computing resource network splices the task processing results of each image processing subtask, and then outputs the image processing task result.

[0016] Furthermore, the dynamic computing resource network is provided with N heterogeneous computing nodes and a shared computing resource pool, where N is a natural number greater than 0;

[0017] The heterogeneous computing nodes are used to execute image processing subtasks, and each heterogeneous computing node is provided with a number;

[0018] The shared computing resource pool is communicatively connected to a plurality of physical computing devices, and integrates various computing resources of various physical computing devices to generate Num groups of computing resource packages according to the communication results, where Num is a natural number greater than 100.

[0019] Furthermore, the process of sorting each image processing subtask includes:

[0020] The user uploads the image processing task to the dynamic computing resource network, and then the dynamic computing resource network traverses the target image data according to various target operations, divides the target image data into m sub-image data according to the traversal results, and marks the corresponding target operation for each sub-image data, thereby obtaining n image processing sub-tasks, where n is a natural number greater than 0;

[0021] Setting a start point and an end point for each image processing subtask, wherein the start point includes the image data and its data format required for the execution of the image processing subtask, and the end point includes the image data and its data format output after the execution of the image processing subtask;

[0022] Match the starting point and the end point of each image processing subtask with each other. If the starting point and the end point of two image processing subtasks are the same, the two image processing subtasks are arranged in sequence. If the starting point and the end point of the two image processing subtasks are not the same, no operation is performed. Then, the image processing subtasks are sorted according to the matching results and the corresponding task execution chain is generated.

[0023] After the image processing subtasks are sorted, each image processing subtask is classified according to the target operation type, and then the image processing subtasks with the same target operation type are sorted in sequence on the same heterogeneous computing node according to the task execution chain.

[0024] Furthermore, the process of the heterogeneous computing node executing the image processing subtask includes:

[0025] A central task node and k edge task nodes are set in each heterogeneous computing node. When the heterogeneous computing node receives the image processing subtask, the dynamic computing resource network calls the shared computing resource pool to allocate the same number of computing resource packages to each receiving heterogeneous computing node and sets the same task constraint duration, where k is a natural number greater than 0.

[0026] The heterogeneous computing nodes evenly distribute the allocated computing resources to each edge task node. At the same time, the central task node splits the image processing subtask into several task steps and marks the preconditions of each task step. The preconditions are the data required for the execution of the corresponding task step.

[0027] Then, the central task node allocates each task step to each edge task node in sequence according to the splitting order of the task steps. If the number of task steps is greater than the number of edge task nodes, a task step waiting area is set up, and the unassigned task steps are arranged in the task step waiting area in sequence according to the splitting order. When any edge task node is in an idle state, the task step waiting area will allocate the current first task step to the idle edge task node.

[0028] Furthermore, the process of edge task nodes executing task steps includes:

[0029] The edge task node executes the received task steps and sends a precondition acquisition request to the central task node. Then the central task node traverses the existing sub-processing results according to the precondition acquisition request. If there is a sub-processing result required by the precondition acquisition request, the sub-processing result is sent to the corresponding edge task node.

[0030] If it does not exist, the central task node sends the precondition acquisition request to the central task nodes in other heterogeneous computing nodes. Other central task nodes traverse their own stored sub-processing results according to the precondition acquisition request. If it exists, it is sent to the central task node. If it does not exist, the task is not performed.

[0031] If the required sub-processing results do not exist in all current central task nodes, the heterogeneous computing node will temporarily stop executing the corresponding task steps and call new task steps to the corresponding edge task nodes;

[0032] When the edge task node receives the required preconditions, it executes the task steps according to the preconditions. When the edge task node completes the task steps, it sends the sub-processing results after the task steps are completed to the central task node.

[0033] Furthermore, the execution process of the reward and punishment mechanism includes:

[0034] If a heterogeneous computing node completes an image processing subtask before the task constraint duration, the dynamic computing resource network allocates a computing resource package to the corresponding heterogeneous computing node through the shared computing resource pool, and then the heterogeneous computing node evenly distributes the obtained computing resource package to each edge task node;

[0035] If a heterogeneous computing node fails to complete an image processing subtask before the task constraint duration, the dynamic computing resource network extracts a portion of computing resources from each edge task node in the corresponding heterogeneous task node through the shared computing resource pool, and integrates them into a computing resource package for recycling.

[0036] Further, when a heterogeneous computing node completes all its assigned image processing subtasks, the shared computing resource pool recycles all computing resource packages of the corresponding heterogeneous computing node, and determines whether the number of currently unassigned computing resource packages is greater than or equal to 1 / 2Num;

[0037] If the number of currently unallocated computing resource packages is less than 1 / 2Num, no operation is performed;

[0038] If the number of currently unallocated computing resource packages is greater than or equal to 1 / 2Num, a computing resource package is allocated to each of the heterogeneous computing nodes currently executing the image processing subtask.

[0039] Furthermore, when each heterogeneous computing node completes its assigned image processing subtask, the shared computing resource pool recycles the computing resource packages of each heterogeneous computing node. At the same time, the dynamic computing resource network splices the task processing results of each image processing subtask in sequence according to the segmentation order of each image processing subtask, thereby obtaining the image processing task result, and sending the image task processing result to the user.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention sets up a reward and punishment mechanism to reduce or increase computing resources according to whether heterogeneous computing nodes complete image processing subtasks within the task constraint time. When each heterogeneous computing node completes its assigned image processing subtask, the task processing results of each image processing subtask are spliced, and then the image processing task results are output, which effectively improves the computing resource utilization and reduces the performance loss of the computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention.

[0043] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0044] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0045] Embodiment 1,

[0046] like Figure 1 As shown, a heterogeneous computing resource scheduling method includes the following steps:

[0047] Step S1, setting up a dynamic computing resource network, and setting up a number of heterogeneous computing nodes and a shared computing resource pool in the dynamic computing resource network;

[0048] Step S2, dividing the image processing task into a number of image processing subtasks of different task types, setting a start point and an end point for each image processing subtask, matching the start point and the end point of each image processing subtask with each other, and then sorting each image processing subtask, and allocating image processing subtasks of the same task type to the same heterogeneous computing node;

[0049] Step S3: Set a number of central task nodes and edge task nodes on the heterogeneous computing nodes, set the same task constraint duration for each heterogeneous computing node, split the image processing subtask into a number of task steps and assign them to each edge task node for execution, and mark the preconditions required for each task;

[0050] Step S4, the edge task node sends the sub-processing result of the task step to the central task node, and then the central task nodes in different heterogeneous computing nodes interact with each other and distribute the interacted sub-processing results to the required edge task nodes;

[0051] Step S5: Setting a reward and punishment mechanism to reduce or increase computing resources for heterogeneous computing nodes based on whether they complete the image processing subtask within the task constraint time.

[0052] Step S6: When each heterogeneous computing node completes its assigned image processing subtask, the dynamic computing resource network splices the task processing results of each image processing subtask, and then outputs the image processing task result.

[0053] Embodiment 2,

[0054] This embodiment is a further limitation of Embodiment 1, and Step S1 is implemented by the following process:

[0055] The dynamic computing resource network is provided with N heterogeneous computing nodes and a shared computing resource pool, where N is a natural number greater than 0;

[0056] The heterogeneous computing nodes are used to execute image processing subtasks, and each heterogeneous computing node is provided with a number g1, g2, ..., g N ;

[0057] The shared computing resource pool is communicatively connected to a number of physical computing devices, and integrates the computing resources of each physical computing device according to the communication results to generate Num groups of computing resource packages, where the computing resources include gpu, cpu, dpu, etc., and Num is a natural number greater than 100.

[0058] Embodiment 3,

[0059] This embodiment is a further limitation of Embodiment 1, and step S2 is implemented by the following process:

[0060] The user uploads an image processing task to the dynamic computing resource network, wherein the image processing task includes various target operations such as image restoration, image content extraction, image stitching, and target image data;

[0061] Then the dynamic computing resource network traverses the target image data according to each target operation, divides the target image data into m sub-image data according to the traversal result, and marks each sub-image data with a corresponding target operation, thereby obtaining n image processing subtasks, where n is a natural number greater than 0;

[0062] It should be noted that the data size of each image processing subtask is the same;

[0063] Setting a start point and an end point for each image processing subtask, wherein the start point includes the image data and its data format required for the execution of the image processing subtask, and the end point includes the image data and its data format output after the execution of the image processing subtask;

[0064] Match the starting point and the end point of each image processing subtask with each other. If the starting point and the end point of two image processing subtasks are the same, the two image processing subtasks are arranged in sequence. If the starting point and the end point of the two image processing subtasks are not the same, no operation is performed. Then, the image processing subtasks are sorted according to the matching results and the corresponding task execution chain is generated.

[0065] After the image processing subtasks are sorted, each image processing subtask is classified according to the target operation type, and then the image processing subtasks with the same target operation type are sorted in sequence on the same heterogeneous computing node according to the task execution chain.

[0066] Embodiment 4,

[0067] This embodiment is a further limitation of Embodiment 1, and step S3 is implemented by the following process:

[0068] A central task node and k edge task nodes are set in each heterogeneous computing node. When the heterogeneous computing node receives the image processing subtask, the dynamic computing resource network calls the shared computing resource pool to allocate the same number of computing resource packages to each receiving heterogeneous computing node and sets the same task constraint duration, where k is a natural number greater than 0.

[0069] The heterogeneous computing nodes evenly distribute the allocated computing resources to each edge task node. At the same time, the central task node splits the image processing subtask into several task steps and marks the preconditions of each task step. The preconditions are the data required for the execution of the corresponding task step.

[0070] Then, the central task node allocates each task step to each edge task node in sequence according to the splitting order of the task steps. If the number of task steps is greater than the number of edge task nodes, a task step waiting area is set up, and the unassigned task steps are arranged in the task step waiting area in sequence according to the splitting order. When any edge task node is in an idle state, the task step waiting area will allocate the current first task step to the idle edge task node.

[0071] Embodiment 5,

[0072] This embodiment is a further limitation of Embodiment 1, and step S4 is implemented by the following process:

[0073] The edge task node executes the received task steps and sends a precondition acquisition request to the central task node. Then the central task node traverses the existing sub-processing results according to the precondition acquisition request. If there is a sub-processing result required by the precondition acquisition request, the sub-processing result is sent to the corresponding edge task node.

[0074] If it does not exist, the central task node sends the precondition acquisition request to the central task nodes in other heterogeneous computing nodes. Other central task nodes traverse their own stored sub-processing results according to the precondition acquisition request. If it exists, it is sent to the central task node. If it does not exist, the task is not performed.

[0075] If the required sub-processing results do not exist in all current central task nodes, the heterogeneous computing node will temporarily stop executing the corresponding task steps and call new task steps to the corresponding edge task nodes;

[0076] When the edge task node receives the required preconditions, it executes the task steps according to the preconditions, wherein the preconditions include image fragments, pixel value processing results of each pixel in the image, etc.;

[0077] When the edge task node completes the task step, it sends the sub-processing results after the task step is completed to the central task node.

[0078] Embodiment 6,

[0079] This embodiment is a further limitation of Embodiment 1, and step S5 is implemented by the following process:

[0080] The dynamic computing resource network is provided with a reward and punishment mechanism. If a heterogeneous computing node completes an image processing subtask before the task constraint duration, the dynamic computing resource network allocates a computing resource package to the corresponding heterogeneous computing node through the shared computing resource pool, and then the heterogeneous computing node evenly distributes the obtained computing resource package to each edge task node;

[0081] If a heterogeneous computing node fails to complete an image processing subtask before the task constraint duration, the dynamic computing resource network extracts a portion of computing resources from each edge task node in the corresponding heterogeneous task node through the shared computing resource pool and integrates them into a computing resource package for recycling;

[0082] When a heterogeneous computing node completes all its assigned image processing subtasks, the shared computing resource pool recycles all computing resource packages of the corresponding heterogeneous computing node and determines whether the number of currently unassigned computing resource packages is greater than or equal to 1 / 2Num;

[0083] If the number of currently unallocated computing resource packages is less than 1 / 2Num, no operation is performed;

[0084] If the number of currently unallocated computing resource packages is greater than or equal to 1 / 2Num, a computing resource package is allocated to each of the heterogeneous computing nodes currently executing the image processing subtask.

[0085] Embodiment 7,

[0086] This embodiment is a further limitation of Embodiment 1, and step S6 is implemented by the following process:

[0087] When each heterogeneous computing node completes its assigned image processing subtask, the shared computing resource pool recycles the computing resource packages of each heterogeneous computing node. At the same time, the dynamic computing resource network splices the task processing results of each image processing subtask in sequence according to the segmentation order of each image processing subtask, and then obtains the image processing task result, and sends the image task processing result to the user.

[0088] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for scheduling heterogeneous computing resources, characterized in that: The following steps are involved: Step S1, setting up a dynamic computing resource network, and setting up a number of heterogeneous computing nodes and a shared computing resource pool in the dynamic computing resource network; Step S2, dividing the image processing task into a number of image processing subtasks of different task types, setting a start point and an end point for each image processing subtask, matching the start point and the end point of each image processing subtask with each other, and then sorting each image processing subtask, and allocating image processing subtasks of the same task type to the same heterogeneous computing node; The process of sequencing the various image processing subtasks includes: The user uploads an image processing task to the dynamic computing resource network, wherein the image processing task includes image item target operations and target image data; Then the dynamic computing resource network divides the target image data into m sub-image data, and labels each sub-image data with a corresponding target operation, thereby obtaining n image processing subtasks, where n is a natural number greater than 0; Setting a start point and an end point for each image processing subtask, wherein the start point includes the image data and its data format required for the execution of the image processing subtask, and the end point includes the image data and its data format output after the execution of the image processing subtask; Match the starting point and the end point of each image processing subtask with each other. If the starting point and the end point of two image processing subtasks are the same, the two image processing subtasks are arranged in sequence. If the starting point and the end point of the two image processing subtasks are not the same, no operation is performed. Then, the image processing subtasks are sorted according to the matching results and the corresponding task execution chain is generated. After the image processing subtasks are sorted, they are classified according to the target operation type, and then the image processing subtasks with the same target operation type are sorted in sequence on the same heterogeneous computing node according to the task execution chain; Step S3: Set a number of central task nodes and edge task nodes on the heterogeneous computing nodes, set the same task constraint duration for each heterogeneous computing node, split the image processing subtask into a number of task steps and assign them to each edge task node for execution, and mark the preconditions required for each task; Step S4, the edge task node sends the sub-processing result of the task step to the central task node, and then the central task nodes in different heterogeneous computing nodes interact with each other and distribute the interacted sub-processing results to the required edge task nodes; Step S5: Setting a reward and punishment mechanism to reduce or increase computing resources of heterogeneous computing nodes according to whether they complete the image processing subtask within the task constraint time. The implementation process of the reward and punishment mechanism includes: If a heterogeneous computing node completes an image processing subtask before the task constraint duration, the dynamic computing resource network allocates a computing resource package to the corresponding heterogeneous computing node through the shared computing resource pool, and the heterogeneous computing node evenly distributes the obtained computing resource package to each edge task node; If a heterogeneous computing node fails to complete an image processing subtask before the task constraint duration, the dynamic computing resource network extracts a portion of computing resources from each edge task node in the corresponding heterogeneous task node through the shared computing resource pool and integrates them into a computing resource package for recycling; When a heterogeneous computing node completes all of its assigned image processing subtasks, the shared computing resource pool recycles all computing resource packages of the corresponding heterogeneous computing node, and determines whether the number of currently unassigned computing resource packages is greater than or equal to 1 / 2Num. Based on the comparison result, a computing resource package is allocated to each heterogeneous computing node currently executing the image processing subtask. Step S6: When each heterogeneous computing node completes its assigned image processing subtask, the dynamic computing resource network splices the task processing results of each image processing subtask, and then outputs the image processing task result.

2. A heterogeneous computing resource scheduling method according to claim 1, characterized in that: The dynamic computing resource network is provided with N heterogeneous computing nodes and a shared computing resource pool, where N is a natural number greater than 0; The heterogeneous computing nodes are used to execute image processing subtasks, and each heterogeneous computing node is set with a number. The shared computing resource pool is communicatively connected to a plurality of physical computing devices, and integrates the computing resources of each physical computing device according to the communication results to generate Num groups of computing resource packages, where Num is a natural number greater than 100.

3. A heterogeneous computing resource scheduling method according to claim 2, characterized in that: The process of heterogeneous computing nodes performing image processing subtasks includes: A central task node and k edge task nodes are respectively set in each heterogeneous computing node, and the shared computing resource pool allocates the same number of computing resource packages to each receiving heterogeneous computing node and sets the same task constraint duration, where k is a natural number greater than 0; The heterogeneous computing nodes evenly distribute the allocated computing resources to each edge task node. At the same time, the central task node splits the image processing subtask into several task steps and marks the preconditions of each task step. The preconditions are the data required for the execution of the corresponding task step. Then, the central task node allocates the task steps to each edge task node in sequence according to the splitting order of the task steps. If the number of task steps is greater than the number of edge task nodes, a task step waiting area is set up, and the unassigned task steps are arranged in the task step waiting area in sequence according to the splitting order. When any edge task node is in an idle state, the task step waiting area will allocate the current first task step to the idle edge task node.

4. A heterogeneous computing resource scheduling method according to claim 3, characterized in that: The process of edge task nodes executing task steps includes: The edge task node executes the received task steps and sends a precondition acquisition request to the central task node. Then the central task node traverses the existing sub-processing results according to the precondition acquisition request. If there is a sub-processing result required by the precondition acquisition request, the sub-processing result is sent to the corresponding edge task node. If it does not exist, the central task node sends the precondition acquisition request to the central task nodes in other heterogeneous computing nodes. Other central task nodes traverse the sub-processing results according to the precondition acquisition request. If it exists, it is sent to the central task node. If it does not exist, the task is not performed. If the required sub-processing results do not exist in all current central task nodes, the heterogeneous computing node will temporarily stop executing the corresponding task steps and call new task steps to the edge task nodes; When the edge task node receives the required preconditions, it executes the task steps according to the preconditions. When the edge task node completes the task steps, it sends the sub-processing results after the task steps are completed to the central task node.

5. A heterogeneous computing resource scheduling method according to claim 4, characterized in that: When each heterogeneous computing node completes its assigned image processing subtask, the shared computing resource pool recycles the computing resource packages of each heterogeneous computing node. At the same time, the dynamic computing resource network sequentially splices the task processing results of each image processing subtask according to the segmentation order of each image processing subtask to obtain the image processing task result, and sends the image task processing result to the user.

Images