A CPU and GPU hybrid time-sharing workflow scheduling method and system

Abstract

This invention provides a method and system for scheduling time-sharing workflows based on a hybrid CPU and GPU architecture. The method involves setting time segments for workflow usage; creating workflows and ensuring they operate within their respective time segments; allocating workflows to each node; monitoring the resource utilization of each node; and determining whether it exceeds the node's resource utilization threshold for the current time segment. If the threshold is exceeded, the utilization rates of CPU and GPU are retrieved separately. If either CPU or GPU utilization exceeds a preset threshold, the CPU and GPU utilization rates are adjusted accordingly. If, after adjustment, the CPU and GPU utilization rates still exceed the node's resource utilization threshold, an alarm is issued. This invention utilizes time segments for rational planning, fully leverages resources, and flexibly adjusts resource allocation to ultimately complete task execution and calculate the desired results.

Description

Technical Field

[0001] This invention relates to the field of cluster data processing technology, and in particular to a method and system for scheduling time-sharing workflows based on a hybrid CPU and GPU approach. Background Technology

[0002] With the advent of the artificial intelligence era, related technologies are also constantly developing. Currently, leading distributed architecture solutions deployed in multi-node clusters are widely used in deployment environments. The operation of numerous training and inference scripts requires sufficient resource support. However, when resources are limited, more rational resource optimization is necessary.

[0003] Currently, the available CPU / GPU allocation schemes are often fixed, with CPUs and GPUs bound to users or corresponding jobs based on the number of resources. When a cluster contains a large number of resources, it often leads to uneven resource utilization, causing some resources to be idle while some CPUs or GPUs remain idle. This results in ineffective utilization of system resources, affecting the system's data processing efficiency and capabilities. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention provides a time-sharing workflow scheduling method based on a hybrid CPU and GPU. The workflow scheduling method rationally plans according to time segments, makes full use of resources and flexibly adjusts them, and finally completes the task execution and calculates the desired result.

[0005] The method includes: the cluster has multiple nodes, and each node is configured with a CPU and a GPU;

[0006] Set the time segments for workflow usage; time segments include: off-peak hours, normal hours, and peak hours;

[0007] Resource quotas are allocated to nodes for different time segments;

[0008] Define the CPU and GPU utilization thresholds for each node under different time segments;

[0009] Create a workflow so that it operates within the corresponding time segment;

[0010] The workflow is assigned to each node, and the resource utilization of each node is monitored to determine whether it exceeds the node resource utilization threshold corresponding to the current time segment.

[0011] If the node resource utilization threshold is exceeded, the CPU and GPU utilization rates will be retrieved separately.

[0012] If the usage rate of one of them exceeds the preset usage rate threshold, the usage rate of CPU and GPU will be adjusted accordingly.

[0013] If the CPU and GPU utilization rates still exceed the node resource utilization threshold after the conversion and adjustment, an alarm will be issued.

[0014] It should be further noted that when the time segment is an off-peak period, a workflow is created so that the workflow operates during the off-peak period;

[0015] The workflow is assigned to each node, and the resource utilization of each node is monitored to determine whether it exceeds the node resource utilization threshold corresponding to the off-peak period.

[0016] If not, then determine whether the CPU and GPU utilization rates are within the preset utilization thresholds;

[0017] If the CPU or GPU utilization exceeds a preset utilization threshold, the CPU and GPU utilization will be converted so that the converted CPU and GPU utilization are both lower than the preset utilization threshold.

[0018] It should be further noted that if the node resource utilization threshold is exceeded, the utilization rates of CPU and GPU will be retrieved separately.

[0019] If the usage rate of one of them exceeds the preset usage rate threshold, the usage rate of CPU and GPU will be adjusted accordingly.

[0020] If the CPU and GPU utilization rates still exceed the node resource utilization threshold after the conversion and adjustment, an alarm will be issued.

[0021] Adjust the resource allocation of the node to other nodes in the cluster.

[0022] It should be further noted that the time segment transitions from off-peak hours to normal hours;

[0023] Monitor the resource utilization of each node and determine whether it exceeds the node resource utilization threshold corresponding to the normal period.

[0024] If not, then determine whether the CPU and GPU utilization rates are within the preset utilization thresholds;

[0025] If the CPU or GPU utilization exceeds a preset utilization threshold, the CPU and GPU utilization will be converted so that the converted CPU and GPU utilization are both lower than the preset utilization threshold.

[0026] It should be further noted that if the node resource utilization threshold is exceeded, the utilization rates of CPU and GPU will be retrieved separately.

[0027] If the usage rate of one of them exceeds the preset usage rate threshold, the usage rate of CPU and GPU will be adjusted accordingly.

[0028] If the CPU and GPU utilization rates still exceed the node resource utilization threshold after the conversion and adjustment, an alarm will be issued.

[0029] The cluster will automatically allocate maximum CPU and GPU utilization to continue running.

[0030] It should be further noted that when the time segment is a peak period, a workflow is created so that the workflow works during the peak period;

[0031] The workflow is assigned to each node, and the resource utilization of each node is monitored to determine whether it exceeds the node resource utilization threshold corresponding to the peak period.

[0032] If not, then determine whether the CPU and GPU utilization rates are within the preset utilization thresholds;

[0033] If the CPU or GPU utilization exceeds a preset utilization threshold, the CPU and GPU utilization will be converted so that the converted CPU and GPU utilization are both lower than the preset utilization threshold.

[0034] It should be further noted that if the node resource utilization threshold is exceeded, the utilization rates of CPU and GPU will be retrieved separately.

[0035] If the usage rate of one of them exceeds the preset usage rate threshold, the usage rate of CPU and GPU will be adjusted accordingly.

[0036] If the CPU and GPU utilization rates still exceed the node resource utilization threshold after the conversion and adjustment, an alarm will be issued.

[0037] Adjust the resource amount of the node to the cluster's resource pool;

[0038] Retrieve usage time segments and resource utilization rates of other nodes;

[0039] If a node is in an off-peak or normal period and its resource utilization rate is lower than the node resource utilization rate threshold corresponding to the current time segment, then the resources in the resource pool will be allocated to that node.

[0040] It should be further noted that the cluster server obtains the resource utilization rate and resource utilization threshold of each node in real time;

[0041] The current resource utilization of each node is summed to obtain the current resource utilization of the cluster; the idle resource processing capacity of each node is summed to obtain the idle resource capacity of the cluster.

[0042] The cluster server allocates system processing resources based on the available resources in the cluster and the available processing resources of each node.

[0043] It should be further explained that the cluster server sets the resource utilization threshold for each node and the workflow assigned to each node based on each time segment.

[0044] The present invention also provides a time-sharing workflow scheduling system based on CPU and GPU hybrid computing, the system comprising: a cluster server and multiple nodes; each node is configured with CPU and GPU.

[0045] The cluster server communicates with each node and receives preset workflow time segments. The cluster server allocates resource quotas to nodes for different time segments and defines CPU and GPU utilization thresholds for each node under different time segments.

[0046] The cluster server distributes the workflow to each node, monitors the resource utilization of each node, and determines whether it exceeds the node resource utilization threshold corresponding to the current time segment.

[0047] If the node resource utilization threshold is exceeded, the CPU and GPU utilization rates will be retrieved separately.

[0048] If the usage rate of one of them exceeds the preset usage rate threshold, the usage rate of CPU and GPU will be adjusted accordingly.

[0049] If the CPU and GPU utilization rates still exceed the node resource utilization threshold after the conversion and adjustment, an alarm will be issued.

[0050] As can be seen from the above technical solutions, the present invention has the following advantages:

[0051] The CPU and GPU hybrid time-sharing workflow scheduling system provided by this invention rationally plans resource usage, CPU utilization, and GPU utilization based on time segments, making full use of system resources. It judges the usage status of nodes based on the comparison between node and resource utilization and corresponding thresholds, and dynamically adjusts the CPU and GPU utilization within nodes to rationally operate system resources and change the original fixed CPU and GPU quota allocation, so as to maximize the utilization of system resources and obtain the required results more effectively. Attached Figure Description

[0052] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of a hybrid time-sharing workflow scheduling system based on CPU and GPU.

[0054] Figure 2 The flowchart shows the time-sharing workflow scheduling method based on CPU and GPU hybrid scheduling.

[0055] Figure 3 This is a flowchart of an embodiment of a CPU and GPU hybrid time-sharing workflow scheduling method;

[0056] Figure 4 This is a flowchart of another embodiment of the workflow scheduling method. Detailed Implementation

[0057] The CPU and GPU hybrid time-sharing workflow scheduling system architecture provided by this invention may include multiple nodes, a network, and a cluster server. The network is a medium used to provide communication links between the nodes and the cluster server. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0058] It should be understood that Figure 1 The number of nodes, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of nodes, networks, and cluster servers.

[0059] Users can use nodes to interact with the cluster server over a network to receive or send messages, etc. Nodes can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0060] A cluster server can be a server that provides various services. For example, a user sends a workflow to the cluster server using nodes. The cluster server communicates with each node and receives preset workflow time segments. The cluster server allocates resource quotas to nodes for different time segments. According to the time segments, it makes reasonable plans, makes full use of resources, and flexibly adjusts to complete the task and calculate the desired result.

[0061] The node is equipped with a CPU and a GPU, and may also include read-only memory (ROM) or random access memory (RAM). The RAM stores various programs and data required for system operation. The CPU, GPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0062] The CPU and GPU hybrid time-sharing workflow scheduling system provided by this invention includes time segments: idle time periods, normal time periods, and peak time periods. Users can divide time segments according to actual needs and set CPU and GPU resource quotas for each node. Then, time-sharing settings for tasks are enabled. When a workflow task begins its cycle, the system determines whether to adjust resource usage quotas based on the workflow's resource usage. In other words, based on system needs, real-time monitoring and statistics of all CPU and GPU resources within a node are performed, and resources are allocated accordingly.

[0063] For example, when a workflow reaches a node during an off-peak period, the system will know the actual CPU and GPU resource usage of that node and automatically release the idle CPU and GPU resources into the resource pool for other services to allocate.

[0064] The cluster server also continuously monitors whether the node's resource utilization exceeds the threshold for off-peak hours. If it does, it checks the CPU and GPU utilization rates. If either CPU or GPU utilization exceeds a preset threshold, the CPU and GPU utilization rates are adjusted accordingly. If the adjusted CPU and GPU utilization rates still exceed the node's resource utilization threshold, an alarm is triggered. In other words, the current workflow has exceeded the node's off-peak capacity, and even adjusting CPU and GPU utilization cannot meet the processing requirements.

[0065] In this situation, the processing power of other nodes can be allocated to process the data in the workflow, ensuring the rational use of cluster resources.

[0066] Of course, if the node resource utilization threshold is not exceeded, the CPU and GPU utilization rates will be retrieved separately; if one of the utilization rates is higher than the preset utilization threshold, the CPU and GPU utilization rates will be converted and adjusted; the CPU and GPU utilization rates will be balanced to a balanced state.

[0067] In the system provided by this invention, when the workflow reaches the normal period, the system will determine the CPU utilization and GPU utilization, and automatically adjust the resources according to the preset allocation adjustment set when submitting the business.

[0068] As another embodiment of the system of the present invention, if the system transitions from an idle period to a normal period, the cluster server automatically adjusts the resource utilization of the nodes, further adjusting the CPU utilization and GPU utilization to meet the preset allocation requirements.

[0069] If the system resource pool is insufficient, the cluster server checks the CPU and GPU utilization rates within each node. If either utilization rate exceeds a preset threshold, the CPU and GPU utilization rates are adjusted accordingly. If, after adjustment, the utilization rate does not exceed the node resource utilization threshold, the current state meets the usage requirements. If the threshold is still exceeded, an alarm is issued, and the system will automatically allocate CPU and GPU utilization from available nodes to ensure workflow execution.

[0070] In another embodiment of the system of the present invention, during peak hours, the system monitors the CPU and GPU utilization of the corresponding nodes in real time. When the node resource utilization threshold corresponding to the peak hour is exceeded, the system separately judges the CPU and GPU utilization. If one is higher, the CPU and GPU utilization are converted for different uses. After conversion, it is monitored whether the utilization is within the threshold. If it returns to the node resource utilization threshold, operation continues. If it still exceeds the threshold, an alarm is issued. Users can analyze the cause. If other node resources are needed for processing, the CPU and GPU utilization can be manually adjusted, and a portion of resources can be allocated from the resource pool for normal business use.

[0071] If the number of system nodes falls below the normal threshold range for a period of time, the cluster server will automatically release the CPU and GPU usage to the resource pool for use when processing other workflows.

[0072] Thus, the CPU and GPU hybrid time-sharing workflow scheduling system provided by this invention rationally plans resource usage, CPU utilization, and GPU utilization according to time segments, making full use of system resources. It judges the usage status of nodes based on the comparison between node and resource utilization and corresponding thresholds, and dynamically adjusts the CPU and GPU utilization within nodes to rationally operate system resources and change the original fixed CPU and GPU quota allocation, so as to maximize the utilization of system resources and obtain the required results more effectively.

[0073] Based on the aforementioned workflow scheduling system, this invention also provides a time-sharing workflow scheduling method based on a hybrid CPU and GPU architecture, such as... Figures 2 to 4 As shown in the diagram, the method allocates resource quotas to nodes for different time segments.

[0074] Define the CPU and GPU utilization thresholds for each node under different time segments;

[0075] Create a workflow so that it operates within the corresponding time segment;

[0076] The workflow is assigned to each node, and the resource utilization of each node is monitored to determine whether it exceeds the node resource utilization threshold corresponding to the current time segment.

[0077] If the node resource utilization threshold is not exceeded, the utilization rates of CPU and GPU are retrieved separately. If the utilization rate of one of them is higher than the preset utilization rate threshold, the utilization rates of CPU and GPU are converted and adjusted so that CPU and GPU run at the preset utilization rate.

[0078] If the node resource utilization threshold is exceeded, the CPU and GPU utilization rates will be retrieved separately. If the utilization rate of one of them is higher than the preset utilization rate threshold, the CPU and GPU utilization rates will be adjusted accordingly. If the CPU and GPU utilization rates still exceed the node resource utilization threshold after adjustment, an alarm will be issued.

[0079] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0080] Furthermore, as a refinement and extension of the above-described hybrid time-sharing workflow scheduling method based on CPU and GPU, and to fully illustrate the specific implementation process in this embodiment, another hybrid time-sharing workflow scheduling method based on CPU and GPU is provided, including:

[0081] When creating cyclical business processes such as workflows, this invention allows users to divide time segments according to actual conditions. These time segments include: off-peak hours, normal hours, and peak hours.

[0082] Configure CPU and GPU resource quotas and enable time-sharing for tasks. When a workflow task begins its cycle, the system will determine whether to adjust resource quotas based on the workflow's resource usage.

[0083] System administrators can define node resource utilization thresholds for CPU and GPU resource usage at different time intervals. Different upper and lower limits can be assigned to each time period. Based on these node resource utilization thresholds, resource allocation and adjustments can be determined.

[0084] When the workflow reaches the idle time segment, the system will determine the actual usage of CPU and GPU resources and automatically release idle CPU and GPU resources into the resource pool for other business use.

[0085] In an embodiment of the invention, if the workflow is running during an idle time segment, the cluster server determines that the underlying CPU and GPU resource usage has exceeded the idle resource utilization threshold. At this time, the system's underlying layer will separately check the CPU and GPU utilization rates. If one is higher, CPU and GPU resources will be converted for different uses. After the CPU and GPU resources are converted, if the monitored resources recover to the idle utilization threshold, the system continues to run; if they still exceed the threshold, an alarm can be issued. Users can analyze the cause and, if resources are needed, manually adjust the CPU and GPU resources, allocating a portion of resources from the resource pool for normal business use.

[0086] When the workflow reaches its normal operating period, the system will determine the actual CPU and GPU resource usage at the underlying level of the workflow and automatically adjust the resources according to the preset time-sharing settings for submitting business.

[0087] In embodiments of the present invention, if the system transitions from an idle period to a normal period, it automatically adjusts the available CPU and GPU resources from the resource pool to meet the preset allocation. If the resource pool resources are insufficient, the cluster server will determine the utilization rates of CPU and GPU respectively. If one is higher, CPU and GPU resources will be converted for use. After the conversion, if the cluster server monitors the resources and they recover to the normal time segment usage threshold, operation continues. If the threshold is still exceeded, an alarm is issued, and the system will automatically allocate the maximum available CPU and GPU resources to allow the task to continue running. If the resource pool can meet the operation requirements, normal allocation occurs, and the task runs normally.

[0088] If the system transitions from peak hours to normal hours, it will automatically release CPU and GPU resources and adjust them to meet the preset allocation.

[0089] The workflow continues running after being allocated preset resources, with the cluster server monitoring CPU and GPU usage in real time. If resources remain within the normal usage threshold during normal time segments, the workflow continues. If resources exceed the available range, the cluster server will separately assess CPU and GPU utilization; if one is higher, CPU and GPU resources will be allocated accordingly. After resource allocation, if the monitored resources return to the normal time segment usage threshold, the workflow continues; if they still exceed the threshold, an alarm will be issued. Users can analyze the cause and, if resources are needed, manually adjust CPU and GPU resources, allocating some resources from the resource pool for normal business use. If resources remain below the normal time segment threshold for a period of time (configurable by the system administrator), CPU and GPU resources will be automatically released back to the resource pool for other business use.

[0090] When the workflow reaches its peak time segment, the cluster server determines the actual CPU and GPU resource usage at the underlying level of the workflow and automatically adjusts the resources according to the original time-sharing settings when submitting the business.

[0091] When transitioning from off-peak to peak hours, the system automatically adjusts available CPU and GPU resources from the resource pool to meet preset allocations. If the resource pool is insufficient, the underlying system will assess the utilization rates of CPU and GPU separately. If one is higher, CPU and GPU resources will be allocated accordingly. After resource allocation, if the monitored resources recover to the peak usage threshold, operation continues. If the threshold is still exceeded, an alarm is triggered, and the cluster server will automatically allocate the maximum available CPU and GPU resources to allow the task to continue running. If the resource pool can meet the requirements, allocation proceeds normally, and the task runs normally.

[0092] The workflow starts during normal hours and automatically adjusts available CPU and GPU resources from the resource pool to meet preset allocations. If the resource pool resources are insufficient, the cluster servers check the CPU and GPU utilization rates separately. If one is higher, CPU and GPU resources are allocated accordingly. After resource allocation, if the monitored resources recover to the peak usage threshold, the process continues. If the threshold is still exceeded, an alarm is issued, and the system automatically allocates the maximum available CPU and GPU resources to allow the task to continue running. If the resource pool can meet the requirements, resources are allocated normally, and the task runs normally.

[0093] If the system runs tasks during peak time periods, the cluster server monitors CPU and GPU usage in real time. When the peak time period's maximum threshold is exceeded, the cluster server assesses the CPU and GPU utilization separately. If one is higher, CPU and GPU resources are allocated accordingly. After resource allocation, if the monitored resources return to the normal time period's usage threshold, operation continues. If the threshold is still exceeded, an alarm is issued, allowing users to analyze the cause. If resources are needed, CPU and GPU resources can be manually adjusted, allocating some resources from the resource pool for normal business use. If the utilization remains below the normal time period threshold for a period (the time range can be configured by the system administrator), CPU and GPU resources are automatically released back to the resource pool for other business use.

[0094] This invention can process workflows during off-peak, normal, and peak periods, and rationally plan and adjust resources flexibly to complete task execution and calculate results. This invention achieves optimal scheduling between resources and between resources and resource pools, maximizing resource utilization.

[0095] The CPU and GPU hybrid time-sharing workflow scheduling method provided by this invention comprises the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein. These units and steps can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0096] Those skilled in the art will understand that various aspects of the CPU and GPU hybrid time-sharing workflow scheduling method can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A CPU and GPU hybrid time-sharing workflow scheduling method, characterized in that, The methods include: The cluster consists of multiple nodes, each configured with a CPU and a GPU. Set the time segments for workflow usage; time segments include: off-peak hours, normal hours, and peak hours; Resource quotas are allocated to nodes for different time segments; Define the CPU and GPU utilization thresholds for each node under different time segments; Create a workflow that operates within a specified time segment; within the specified time segment, continue with the following operations: The workflow is distributed to each node, and the resource utilization of each node is monitored to determine whether it exceeds the node resource utilization threshold corresponding to the current time segment. If it does not exceed the threshold, the CPU and GPU utilization are then checked to see if they are within their respective preset utilization thresholds for the current time segment. If either the CPU or GPU utilization exceeds its respective preset threshold, the CPU and GPU utilization are converted to be lower than their respective preset thresholds for the current time segment. Furthermore, if the node resource utilization threshold for the current time segment is exceeded, the CPU and GPU utilization are retrieved separately. If one of these utilizations exceeds its preset threshold, the CPU and GPU utilization are adjusted accordingly. If, after adjustment, the CPU and GPU utilization still exceed their respective preset thresholds for the current time segment, an alarm is issued. If an alarm is issued during a normal time period, the cluster will automatically allocate the maximum utilization of CPU and GPU to continue running; If an alarm is triggered during a peak time period, the usage time periods and resource utilization rates of other nodes are retrieved. If a node is in an off-peak or normal time period and its resource utilization rate is lower than the node resource utilization rate threshold corresponding to the current time period, then the resources in the resource pool are allocated to that node. When the workflow reaches a node during off-peak hours, the system automatically releases idle CPU and GPU resources into the resource pool for use by other services.

2. The CPU and GPU hybrid time-sharing workflow scheduling method according to claim 1, characterized in that, The cluster server obtains the resource utilization rate and resource utilization threshold of each node in real time; The current resource utilization of each node is summed to obtain the current resource utilization of the cluster; the idle resource processing capacity of each node is summed to obtain the idle resource capacity of the cluster. The cluster server allocates system processing resources based on the available resources in the cluster and the available processing resources of each node. The cluster server sets resource utilization thresholds for each node and assigns workflows to each node based on various time segments.

3. A CPU and GPU hybrid time-sharing workflow scheduling system, characterized in that, The system adopts the CPU and GPU hybrid time-sharing workflow scheduling method as described in any one of claims 1 to 2; The system includes: a cluster server and multiple nodes; each node is equipped with a CPU and a GPU. The cluster server communicates with each node and receives preset workflow time segments. The cluster server allocates resource quotas to nodes for different time segments and defines CPU and GPU utilization thresholds for each node under different time segments. The cluster server distributes the workflow to each node, monitors the resource utilization of each node, and determines whether it exceeds the node resource utilization threshold corresponding to the current time segment. If the node resource utilization threshold is exceeded, the CPU and GPU utilization rates will be retrieved separately. If the usage rate of one of them exceeds the preset usage rate threshold, the usage rate of CPU and GPU will be adjusted accordingly. If the CPU and GPU utilization rates still exceed their respective preset utilization thresholds for the current time segment after the conversion and adjustment, an alarm will be issued.

Images