GPU server cluster system and GPU scheduling method

Abstract

The invention discloses a GPU server cluster system and a GPU scheduling method, and relates to the field of data processing. The system comprises a plurality of GPU service nodes, a data transmission unit, a task segmentation unit, a GPU scheduling unit and a data processing unit. The data transmission unit is used for acquiring a to-be-processed video image data set; the task segmentation unit is used for generating a video image processing task and segmenting a plurality of subtasks; the GPU scheduling unit is used for determining load performance scores according to the load performance parameters, sorting the GPU service nodes from high to low based on the load performance scores, determining a target GPU node and distributing subtasks; the target GPU node is used for executing the subtask to obtain processed partial video image data; and the data processing unit is used for merging to obtain a processed video image data set, sending the processed video image data set to the data demand end, and sending video image processing task completion information to the GPU scheduling unit. The GPU computing power can be effectively scheduled, and the performance of the GPU server cluster is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a GPU server cluster system and a GPU scheduling method. Background Art

[0002] GPU (Graphics Processing Unit), or graphics processor, is a microprocessor that specializes in image computing on personal computers, workstations, game consoles, and some mobile devices (such as tablets, smartphones, etc.). It has powerful parallel computing capabilities and was originally used mainly for graphics rendering, but with the development of technology, it now plays a vital role in many fields such as scientific computing, artificial intelligence, deep learning, and video processing.

[0003] With the rapid development of the Internet industry, the demand for video and image processing is also growing, and GPU service clusters have emerged. GPU service clusters have high parallel processing capabilities and strong scalability, and can efficiently process a variety of computing-intensive tasks such as large-scale image videos, deep learning, and scientific computing.

[0004] When GPU server clusters process tasks, due to the lack of efficient task scheduling solutions, some GPU resources may be overused while other GPU resources are idle. This uneven resource allocation will lead to a decline in overall performance and increase energy consumption and operating costs.

[0005] Therefore, how to effectively schedule GPU computing power and improve the performance of GPU server clusters is an urgent problem that needs to be solved. Summary of the invention

[0006] In view of the above technical problems and defects, the purpose of the present invention is to provide a GPU server cluster system and a GPU scheduling method, which can effectively schedule GPU computing power and improve the performance of the GPU server cluster.

[0007] To achieve the above-mentioned purpose, in a first aspect, the present invention provides a GPU server cluster system, comprising a plurality of GPU service nodes, a data transmission unit, a task splitting unit, a GPU scheduling unit and a data processing unit; the data transmission unit is used to obtain a video image data set to be processed from a data demand end; the task splitting unit is used to generate a video image processing task according to the video image data set, and split the video image processing task into a plurality of subtasks; the GPU scheduling unit is used to determine a load performance score of the GPU service node according to a load performance parameter of the GPU service node, sort the GPU service nodes from high to low based on the load performance score, determine the GPU service nodes in the top N positions as target GPU nodes, and assign the subtask to the target GPU node, wherein the load performance parameter includes a floating-point computing capability value, a video memory bandwidth utilization rate and a core clock frequency; the target GPU node is used to execute the subtask to obtain a processed portion of video image data; the data processing unit is used to merge a plurality of the processed portions of video image data into a complete processed video image data set, and send the processed video image data set to the data demand end through the data transmission unit, and at the same time send video image processing task completion information to the GPU scheduling unit.

[0008] The present invention subdivides the video image processing task into multiple subtasks through a task splitting unit, breaking the pressure limit of large tasks on a single GPU node, allowing each subtask to be processed in parallel, and fully tapping the parallel computing potential of the GPU. The GPU scheduling unit determines the load performance score of each GPU service node based on key performance parameters such as floating-point computing capability value, video memory bandwidth utilization and core clock frequency, ensuring that the subtask is assigned to the most suitable target GPU node. This intelligent scheduling mechanism avoids the situation where some nodes are overloaded and other nodes are idle due to unreasonable allocation, greatly improves the utilization efficiency of GPU computing power, and improves the performance of the GPU server cluster. In addition, multiple GPU service nodes work together to dynamically adjust the computing power allocation according to different task requirements, thereby improving the flexibility and adaptability of the system. When faced with large-scale video image data processing tasks, the system can efficiently schedule resources, quickly complete tasks, reduce processing time, and improve work efficiency. At the same time, the data processing unit merges the processed part of the video image data and interacts with the data transmission unit and the GPU scheduling unit to ensure the stable operation and efficient collaboration of the entire system, further improving the utilization efficiency of the GPU server cluster.

[0009] In combination with some embodiments of the first aspect, in some embodiments, the GPU scheduling unit is specifically used to calculate the load performance score according to a load performance scoring formula, and the load performance scoring formula includes: ; Among them, S represents the load performance score, F represents the floating-point computing capability value, M represents the memory bandwidth utilization, C represents the core clock frequency, and α, β, and γ are all experimental parameters.

[0010] By adopting the technical solution of the above embodiment, a method for quantifying the performance of GPU service nodes is provided by introducing a specific load performance scoring formula, so that the system can more accurately evaluate and compare the performance of different nodes. The formula takes into account three key parameters: floating-point computing power, video memory bandwidth utilization, and core clock frequency, and introduces experimental parameters to adapt to different scenarios, thereby enhancing the adaptability and flexibility of the system. This method helps to more reasonably allocate tasks, optimize resource utilization, and improve processing efficiency, thereby making the GPU server cluster system more efficient and stable when processing video image data sets.

[0011] In combination with some embodiments of the first aspect, in some embodiments, the GPU server cluster system also includes a task queue unit, which is used to determine the priority of each subtask according to a set priority definition rule, and sort each subtask according to the priority to obtain a subtask queue, and the GPU scheduling unit is used to call the subtask according to the subtask queue.

[0012] By adopting the technical solution of the above embodiment and adding a task queue unit, the system can sort and manage subtasks according to preset priority rules and dependencies between subtasks. This mechanism ensures that tasks are executed in the correct order, avoids execution delays caused by dependency issues, and improves the accuracy and efficiency of task processing. At the same time, it also allows the system to more intelligently schedule resources when facing a large number of concurrent tasks, reduce waiting time, and improve overall system performance.

[0013] In combination with some embodiments of the first aspect, in some embodiments, the task queue unit is further used to determine the dependency relationship between the subtasks, and sort each of the subtasks according to the priority and the dependency relationship to obtain the subtask queue.

[0014] By adopting the technical solution of the above embodiment, the function of the task queue unit is expanded, so that it can not only determine the priority of the subtask, but also identify and process the dependencies between subtasks. This improvement ensures that in a complex task processing flow, the relevant subtasks will not be executed prematurely before all dependency conditions are met, thereby avoiding resource waste and potential execution errors. This sophisticated task management capability significantly improves the stability and reliability of the system, especially when processing video image processing tasks with complex dependencies.

[0015] In combination with some embodiments of the first aspect, in some embodiments, the GPU server cluster system also includes a GPU performance monitoring unit, which is used to monitor the load performance parameters of the GPU node and send the load performance parameters to the GPU scheduling unit.

[0016] By adopting the technical solution of the above embodiment, real-time monitoring of GPU node performance parameters is achieved by introducing a GPU performance monitoring unit. This unit can dynamically collect and update the load performance parameters of the GPU and provide accurate data support for the GPU scheduling unit. This real-time performance monitoring mechanism makes scheduling decisions more accurate, helps the system respond to performance changes in a timely manner, optimizes resource allocation, and improves processing efficiency. It also provides an important basis for system fault diagnosis and performance optimization.

[0017] In combination with some embodiments of the first aspect, in some embodiments, the GPU server cluster system also includes an FPGA unit, which is used to receive the video image data set from the data transmission unit, and when the video image data set contains an encrypted video stream, decode the encrypted video stream and send the decoded video image data set to the task splitting unit.

[0018] By adopting the technical solution of the above embodiment, the system's processing capability for encrypted video streams is enhanced by integrating an FPGA unit. The introduction of the FPGA unit enables the system to quickly decrypt and decode the encrypted video stream after the data transmission unit receives the encrypted video stream, and then send the decoded data set to the task segmentation unit. This hardware-level processing capability greatly improves the efficiency of the system in processing encrypted video streams, reduces latency, and reduces the burden on GPU nodes, thereby improving the processing capability and response speed of the entire system.

[0019] In combination with some embodiments of the first aspect, in some embodiments, the GPU server cluster system also includes an edge service node, and the GPU scheduling unit is used to offload part of the subtasks to the edge service node according to the load performance parameters of the multiple GPU nodes.

[0020] By adopting the technical solution of the above embodiment, the processing capacity of the GPU server cluster system is expanded by introducing edge service nodes, allowing the system to offload some subtasks to edge service nodes. This distributed processing mechanism not only reduces the load on the central data center, but also provides faster data processing speed, especially in application scenarios with wide geographical distribution or requiring low-latency response. The introduction of edge service nodes improves the flexibility and scalability of the system, enabling the system to process large-scale distributed data more efficiently.

[0021] In combination with some embodiments of the first aspect, in some embodiments, the GPU server cluster system also includes a cloud server, and the GPU scheduling unit is used to call GPU resources of the cloud server according to the load performance parameters of the multiple GPU nodes.

[0022] By adopting the technical solution of the above embodiment and integrating the cloud server, a flexible resource expansion mechanism is provided. When the GPU server cluster system faces high load or specific computing requirements, the system can dynamically call the GPU resources on the cloud server to cope with temporary computing demand peaks. This on-demand resource expansion strategy enables the system to flexibly cope with different workloads, improve resource utilization, reduce operating costs, and enhance the scalability and adaptability of the system.

[0023] In a second aspect, the present invention also provides a GPU scheduling method, which applies the GPU server cluster system provided in the first aspect, and the method includes: determining the load performance score of the GPU service node according to the load performance parameters of the GPU service node, and the load performance parameters include floating-point computing capability value, video memory bandwidth utilization and core clock frequency; sorting the GPU service nodes from high to low based on the load performance score; determining the GPU service nodes in the top N positions as target GPU nodes; when the task segmentation unit generates a video image processing task according to the video image data set to be processed and divides the video image processing task into multiple subtasks, assigning the subtask to the target GPU node; receiving video image processing task completion information, wherein the video image processing task completion information is generated by the data processing unit after merging multiple processed partial video image data into a complete processed video image data set.

[0024] In conjunction with some embodiments of the second aspect, in some embodiments, the step of determining the load performance score of the GPU service node according to the load performance parameter of the GPU service node includes: The load performance score is calculated according to the load performance score formula, and the load performance score formula includes: ; Among them, S represents the load performance score, F represents the floating-point computing capability value, M represents the memory bandwidth utilization, C represents the core clock frequency, and α, β, and γ are all experimental parameters that can be obtained through prior experiments.

[0025] The technical effects of the method provided in the second aspect of the present invention refer to the system provided in the first aspect and will not be described in detail here.

[0026] One or more technical solutions provided by the present invention have at least the following technical effects or advantages: 1. Resource allocation is significantly optimized by accurately evaluating the load performance parameters of GPU service nodes and performing intelligent scheduling based on these parameters. The system can identify the GPU nodes with the highest performance and the lowest load, and assign video image processing tasks to these nodes, thereby reducing task completion time and improving processing efficiency. This approach ensures full utilization of GPU resources while avoiding overload conditions, improving the performance of the entire GPU server cluster system.

[0027] 2. The flexibility and scalability of the system are greatly enhanced by introducing task queue units, GPU performance monitoring units, FPGA units, edge service nodes, and cloud servers. These components enable the system to intelligently sort tasks according to their priorities and dependencies, monitor GPU performance in real time, process encrypted video streams, and use edge computing and cloud resources to expand processing capabilities when needed. This multi-level, distributed processing architecture enables the system to adapt to changing workloads and meet the needs of large-scale parallel processing.

[0028] 3. The integration of edge service nodes and cloud servers enables the geographical distribution of data processing and the dynamic expansion of computing resources, which is essential for application scenarios that require fast response. Edge service nodes can reduce the need to transmit data to central data centers, reduce latency, and increase response speed. At the same time, cloud servers provide additional computing resources to cope with temporary peaks in computing demand, ensuring that critical tasks can be completed smoothly even when resources are tight, thereby improving the real-time nature of data processing and the overall reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings herein are incorporated into and constitute a part of the specification, showing embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 It is a schematic diagram of the architecture of a GPU server cluster system according to an embodiment of the present invention; Figure 2 It is a schematic diagram of a framework of a GPU server cluster system according to an embodiment of the present invention; Figure 3 It is a flowchart of a GPU scheduling method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The terms used in the following embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to be limiting of the present invention. As used in the specification of the present invention, the singular expressions "a", "a", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present invention refers to any or all possible combinations comprising one or more of the listed items.

[0031] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0032] It should also be noted that, unless otherwise clearly specified and limited, in the embodiments of the present invention, the terms such as "setting" and "connection" should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two components; it can be a wired communication connection or a wireless communication connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. The embodiments of the present invention are described in detail below.

[0033] An embodiment of the present invention provides a GPU server cluster system (hereinafter referred to as system), such as Figure 1 As shown, it includes multiple GPU service nodes 101, a data transmission unit 102, a task division unit 103, a GPU scheduling unit 104 and a data processing unit 105. The specific functions of each unit are described in detail below.

[0034] The data transmission unit 102 is used to obtain a video image data set to be processed from a data demand end.

[0035] Among them, the data demand side can be various sources that generate a large amount of video image data, such as monitoring systems, film and television production companies, scientific research institutions, etc. The data transmission unit 102 uses efficient data transmission technology, such as high-speed network connection, data compression and decompression algorithms, etc., to ensure that the video image data set to be processed can be quickly and stably transmitted to the server cluster system. This process is crucial because only when the data is obtained in a timely and accurate manner can the entire system start the subsequent processing process. It provides the original data basis for the task segmentation unit 103, so that the system can further analyze and process these video image data to meet the needs of video image processing in different application scenarios.

[0036] The task division unit 103 is used to generate a video image processing task according to the video image data set, and divide the video image processing task into a plurality of subtasks.

[0037] First, in terms of generating video image processing tasks, the task segmentation unit 103 will conduct an in-depth analysis of the acquired video image data set. It will identify the characteristics of the data, such as resolution, frame rate, type of image content, etc. At the same time, combined with the specific requirements of the data demand side, determine the processing goals and standards. If the data comes from the security field, the tasks that may be generated are target detection, face recognition, etc.; if it comes from film and television production, it may be tasks such as special effects addition and color adjustment.

[0038] When the video image processing task is divided into multiple subtasks, the task segmentation unit 103 will adopt multiple strategies. A common method is based on spatial segmentation, where the video image is divided into multiple regions according to certain rules, and the processing task of each region is a subtask. For example, for high-resolution images, it can be divided into several small blocks, and the processing of each small block is undertaken by different GPU service nodes. Another way is to segment based on timeline or video source, for example, for video data, it can be divided into multiple segments according to the timeline, and the processing of each segment becomes an independent subtask. In addition, segmentation can also be performed according to different processing functions, such as using different functional modules such as image preprocessing, feature extraction, and target recognition as subtasks and assigning them to different nodes for processing. In this way, the parallel processing capability of the GPU server cluster can be fully utilized to improve processing efficiency.

[0039] The GPU scheduling unit 104 is used to determine the load performance score of the GPU service node 101 according to the load performance parameters of the GPU service node 101, sort the GPU service nodes 101 from high to low based on the load performance score, determine the GPU service node 101 ranked in the top N as the target GPU node, and assign the subtask to the target GPU node.

[0040] The load performance parameters include floating point computing capability, video memory bandwidth utilization, and core clock frequency. N is a positive integer greater than or equal to 1.

[0041] The floating-point computing capability value reflects the performance of the GPU in processing floating-point operations. It represents the number of floating-point operations processed per unit time, and the unit is FLOPS. The higher the value, the stronger the GPU's performance in processing floating-point intensive tasks, and the faster it can complete complex mathematical tasks.

[0042] Memory bandwidth utilization is the ratio of GPU memory bandwidth actually used to total bandwidth. Memory bandwidth determines the data transmission speed. High utilization means that data transmission is busy and close to the limit; low utilization means that there may be more available space.

[0043] The core clock frequency is the operating frequency of the GPU computing core, measured in Hz. It is similar to a metronome, the higher the frequency, the faster the calculation speed, and the more calculation steps can be performed.

[0044] In this embodiment, to determine the load performance score of the GPU service node, the GPU scheduling unit comprehensively considers the three key performance parameters of floating-point computing capability, memory bandwidth utilization, and core clock frequency, and then comprehensively evaluates the load performance score of each GPU service node. The general rule is that the higher the floating-point computing capability value, the lower the memory bandwidth utilization, and the higher the core clock frequency of the GPU service node, the higher its load performance score. Conversely, the lower the floating-point computing capability value, the higher the memory bandwidth utilization, and the lower the core clock frequency of the GPU service node, the lower its load performance score.

[0045] Next, based on the calculated load performance score, the GPU scheduling unit sorts all GPU service nodes. This sorting process ensures that the nodes with the best performance are selected first to process tasks. The sorting is from high to low, so that the GPU nodes with lower current load and more computing power are at the front of the list.

[0046] Finally, the GPU service nodes ranked in the top N positions are identified as target GPU nodes. These nodes are considered to have sufficient performance to efficiently process subtasks. Once the target GPU nodes are determined, the scheduling unit will assign subtasks to them. This allocation method can make full use of high-performance GPU resources and improve the processing efficiency and performance of the entire server cluster system.

[0047] The target GPU node is used to execute the subtask and obtain the processed portion of the video image data.

[0048] Specifically, after the GPU scheduling unit 104 assigns the subtask to the target GPU node, these GPU nodes use their powerful computing power to start processing the task. The target GPU node will perform corresponding processing operations on the assigned part of the video image data according to the specific requirements of the subtask. For example, if the subtask is image enhancement, the target GPU node will use its parallel processing architecture to adjust multiple pixels at the same time to improve the image's contrast, brightness and other parameters. For video data, the target GPU node may process frame by frame, such as removing noise, stabilizing the picture, etc. During the processing, the target GPU node gives full play to its advantages such as floating-point computing power, high video memory bandwidth and suitable core clock frequency to complete the subtask at an efficient speed. Finally, each target GPU node can output the processed part of the video image data, laying the foundation for subsequent data merging.

[0049] The data processing unit 105 is used to merge the multiple processed partial video image data into a complete processed video image data set, and send the processed video image data set to the data demand end through the data transmission unit 102, and at the same time send the video image processing task completion information to the GPU scheduling unit 104.

[0050] Specifically, the data processing unit 105 is responsible for integrating the multiple partial video image data processed by each target GPU node. These partial data may be obtained by parallel processing of different GPU nodes. The data processing unit 105 will merge these scattered data fragments into a continuous and complete processed video image data set according to the order of the original data or a specific logical relationship.

[0051] After the merger is completed, the data processing unit 105 securely sends the complete data set back to the data demand end through the data transmission unit 102 to ensure the integrity and availability of the data. In addition, the data processing unit 105 is also responsible for sending a message that a video image processing task has been completed to the GPU scheduling unit 104. This feedback information is crucial for the GPU scheduling unit to update the task status, evaluate the processing efficiency, and make subsequent scheduling decisions, thereby achieving closed-loop management of the entire data processing process.

[0052] This embodiment adopts the above system, and subdivides the video image processing task into multiple subtasks through the task segmentation unit 103, breaking the pressure limit of large tasks on a single GPU node, so that each subtask can be processed in parallel, and fully tapping the parallel computing potential of the GPU. The GPU scheduling unit determines the load performance score of each GPU service node based on key performance parameters such as floating-point computing capability value, video memory bandwidth utilization, and core clock frequency to ensure that the subtask is assigned to the most suitable target GPU node. This intelligent scheduling mechanism avoids the situation where some nodes are overloaded and other nodes are idle due to unreasonable allocation, greatly improves the utilization efficiency of GPU computing power, and improves the performance of the GPU server cluster.

[0053] In addition, multiple GPU service nodes work together to dynamically adjust the computing power allocation according to different task requirements, which improves the flexibility and adaptability of the system. When faced with large-scale video image data processing tasks, the system can efficiently dispatch resources and quickly complete tasks, reducing processing time and improving work efficiency. At the same time, the data processing unit 105 merges the processed part of the video image data and interacts with the data transmission unit 102 and the GPU scheduling unit to ensure the stable operation and efficient collaboration of the entire system, further improving the utilization efficiency of the GPU server cluster.

[0054] In some embodiments, the GPU scheduling unit 104 is specifically used to calculate the load performance score according to a load performance score formula, and the load performance score formula includes: ; Among them, S represents the load performance score, F represents the floating-point computing capability value, M represents the memory bandwidth utilization, C represents the core clock frequency, and α, β, and γ are all experimental parameters that can be obtained through prior experiments.

[0055] In this load performance scoring formula, the numerator α·F·C represents the positive performance influencing factors. The higher the floating point computing capability value F, the stronger the GPU is in processing complex mathematical computing tasks; the higher the core clock frequency C, the faster the computing speed of the GPU's internal computing core. After multiplying the two and then multiplying them by the experimental parameter α, the degree of influence of this part can be adjusted according to the actual situation.

[0056] Denominator (1-M) β +γ reflects the impact of the video memory bandwidth utilization M, where the value of M ranges from 0% to 100%. When the video memory bandwidth utilization M is low, the value of (1-M) is large. After exponential operation (1-M) βAfter that, the denominator will be relatively small, so the overall performance score S is higher. This meets the requirement of high scores for GPUs with low memory bandwidth utilization. The experimental parameter β can adjust the degree of influence of memory bandwidth utilization on the score. As a constant term, γ can further fine-tune the size of the denominator to achieve a more reasonable score effect. In addition, when γ is not 0, it can avoid the situation where the denominator is 0 when M=100%, resulting in invalid calculation of the formula.

[0057] In general, the load performance scoring formula takes into account multiple key performance parameters and aims to more accurately evaluate the load performance of GPU service nodes for efficient task allocation.

[0058] In some embodiments, Figure 2 As shown, the GPU server cluster system also includes a task queue unit 106, which is used to determine the priority of each subtask according to a set priority definition rule, and sort each subtask according to the priority to obtain a subtask queue. The GPU scheduling unit 104 is used to call the subtask according to the subtask queue.

[0059] The priority definition rules may be based on factors such as the urgency of the task, the estimated processing time, resource requirements, user-specified priority, etc. The task queue unit 106 puts the subtasks into the subtask queue according to the calculated priority order to ensure that high-priority tasks can be processed first.

[0060] The task queue unit 106 is connected to the GPU scheduling unit 104. The GPU scheduling unit 104 will retrieve subtasks from the task queue unit 106 in sequence according to the sorted subtask queue, and intelligently allocate the subtasks to the most suitable target GPU node for execution according to the performance score and current load of each GPU service node 101.

[0061] This design enables the entire system to process tasks efficiently, optimize resource allocation, reduce waiting time, and improve overall computing efficiency.

[0062] In some embodiments, the task queue unit 106 is further used to determine the dependency relationship between the subtasks, and sort the subtasks according to the priority and the dependency relationship to obtain a subtask queue.

[0063] Specifically, in the GPU server cluster system, it is responsible for finely managing the execution order of subtasks. It not only determines the priority of each subtask according to the set priority definition rules, but also is responsible for parsing the dependencies between subtasks. This means that the task queue unit 106 needs to identify which subtasks must be executed only after other specific subtasks are completed. By comprehensively considering priorities and dependencies, the task queue unit 106 can build a logically clear subtask queue to ensure that all subtasks are executed in the correct order.

[0064] For example, if subtask B depends on the result of subtask A, then even if B has a higher priority, it will be scheduled to execute after A. This sorting mechanism ensures that the execution of tasks is not blocked due to unsatisfied dependencies, and also optimizes the use of GPU resources, because high-priority subtasks that do not depend on other tasks can be scheduled for execution immediately without having to wait for low-priority or subtasks that depend on other tasks.

[0065] Finally, the task queue unit 106 outputs a subtask queue that takes priority and dependency into consideration, providing a clear execution order for the GPU scheduling unit 104, thereby ensuring efficient and coordinated operation of the entire GPU server cluster system.

[0066] In some embodiments, the GPU server cluster system further includes a GPU performance monitoring unit 107 for monitoring the load performance parameter of the GPU node and sending the load performance parameter to the GPU scheduling unit 104 .

[0067] Specifically, the GPU performance monitoring unit 107 collects these data regularly through built-in sensors and software tools, such as NVIDIA's nvidia-smi or AMD's Radeon GPU Profiler, and sends them to the GPU scheduling unit 104 in real time.

[0068] The GPU scheduling unit 104 uses this information to calculate the performance score of each GPU node and sorts the GPU nodes according to the score so as to make more reasonable decisions when scheduling subtasks.

[0069] This real-time performance monitoring and data feedback mechanism ensures that the GPU scheduling unit 104 can dynamically adjust resource allocation according to the latest performance data, optimize task scheduling, and improve the computing efficiency and response speed of the entire cluster.

[0070] In some embodiments, the GPU server cluster system also includes an FPGA unit 108, which is used to receive the video image data set from the data transmission unit 102, and when the video image data set includes an encrypted video stream, decode the encrypted video stream and send the decoded video image data set to the task splitting unit 103.

[0071] In the GPU server cluster system, the FPGA (field programmable gate array) unit plays the role of a dedicated hardware accelerator, which is directly connected to the data transmission unit 102 and is responsible for receiving and processing the video image data set entering the system. When the data transmitted by the data transmission unit 102 contains an encrypted video stream, the FPGA unit 108 uses its programmability and low latency characteristics to perform efficient decryption and decoding operations.

[0072] The FPGA unit 108 is specially configured with algorithms written in hardware description language (HDL), which are optimized for decryption and video stream decoding, and can quickly process high-bandwidth video data. The decoded video image data set is then forwarded to the task segmentation unit 103, and the FPGA unit 108 further segments the video data into multiple subtasks so that the GPU scheduling unit 104 can intelligently allocate the most suitable GPU node for subsequent image processing based on the performance score and load of each GPU node.

[0073] In this way, the FPGA unit 108 not only improves the efficiency of video stream processing, but also reduces the burden on the GPU node, allowing GPU resources to focus more on computationally intensive image processing tasks, thereby improving the processing power and response speed of the entire system.

[0074] In some embodiments, the GPU server cluster system further includes an edge service node 109 , and the GPU scheduling unit 104 is used to offload part of the subtasks to the edge service node 109 according to the load performance parameters of the multiple GPU nodes.

[0075] Among them, edge service nodes 109 refer to computing nodes that are geographically close to data sources or user terminals, and they form part of the edge computing architecture. These nodes are responsible for processing and storing data close to where it is generated to reduce the need to transmit data to a central data center. By processing data at the edge of the network, edge service nodes 109 can significantly reduce latency, improve response speed, reduce bandwidth consumption, and enhance the security and privacy of data processing. They are typically used to process applications that require fast responses, such as Internet of Things (IoT) device data, real-time video analysis, etc.

[0076] In the system of this embodiment, the introduction of edge service node 109 is to expand the processing capacity of the system and reduce the overall load of the GPU server cluster, especially in application scenarios with wide geographical distribution or requiring low-latency response. The GPU scheduling unit 104 is responsible for monitoring and evaluating the real-time load performance parameters of multiple GPU nodes in the cluster, including the floating-point computing power, video memory bandwidth utilization, and core clock frequency of each node. Based on these parameters, the GPU scheduling unit 104 can intelligently identify which subtasks can be offloaded to the edge service node 109, thereby optimizing the resource allocation and task execution efficiency of the entire system.

[0077] When the overall load of the GPU server cluster is high or the subtasks are associated with a specific geographic location, the GPU scheduling unit 104 will assign these subtasks to the edge service node 109. After the edge service node 109 completes the processing of the subtasks, it will transmit the processed part of the video image data to the data processing unit 105.

[0078] The benefit of doing this is that it can reduce data transmission delays, improve processing speed, and reduce the computing pressure on the GPU server cluster, allowing the system to respond more flexibly and efficiently to the needs of users in different regions, especially in application scenarios that require rapid processing and analysis of large amounts of data, such as video stream processing, real-time analysis, etc.

[0079] In some embodiments, the GPU server cluster system further includes a cloud server 110, and the GPU scheduling unit 104 is used to call the GPU resources of the cloud server 110 according to the load performance parameters of the multiple GPU nodes.

[0080] Among them, cloud server 110 refers to virtual server resources provided in a cloud computing environment, which provide on-demand access via the Internet. Users can dynamically allocate and release these resources as needed without directly managing the underlying hardware. Cloud server 110 provides flexibility and scalability, allowing users to quickly adjust computing resources such as GPU, memory, storage, etc. according to business needs. They are usually managed by cloud service providers such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, etc., which are responsible for maintaining the infrastructure, ensuring high availability and security, and providing various computing services and tools.

[0081] In the GPU server cluster system, the cloud server 110 acts as a supplementary resource, allowing the system to dynamically expand its computing power when facing high load or specific computing requirements. The GPU scheduling unit 104 is responsible for real-time monitoring of the load performance parameters of each GPU node in the cluster, accurately evaluating the current computing resource status of the cluster, and automatically calling the GPU resources on the cloud server 110 when it detects that the resources are tight or the requirements of a specific task exceed the existing processing capacity of the cluster.

[0082] Specifically, first, the GPU scheduling unit 104 will continuously monitor the load performance parameters of the local GPU node, including floating-point computing capability, video memory bandwidth utilization, and core clock frequency, and will also consider the number of subtasks in the local task queue, priority, and estimated required computing power. When it is found that the local GPU resources are insufficient to efficiently complete all pending subtasks, the scheduling unit will send a resource request to the cloud server. This request contains specific requirements for GPU resources, such as required computing power, video memory size, and other information.

[0083] After receiving the request, the cloud server 110 will allocate the corresponding GPU resources to the cluster system according to its own resource allocation strategy and the current available resources. The GPU scheduling unit 104 will incorporate the GPU resources of these cloud servers 110 into the overall resource pool for unified management, just like managing local GPU nodes, calculate the performance scores of these new resources according to the load performance scoring formula, and then allocate some subtasks to the GPU resources of the cloud server for processing according to the priority and required computing power in the subtask queue, so as to achieve efficient use of the cloud server GPU resources to improve the task processing capability of the entire system.

[0084] After completing the subtask processing, the GPU resource of the cloud server 110 transmits the processed portion of the video image data to the data processing unit 105 .

[0085] This on-demand calling mechanism enables the system to flexibly expand its processing capacity to cope with temporary computing peaks or perform large-scale parallel computing tasks while maintaining cost efficiency, because cloud server 110 resources are only used when additional resources are really needed. In addition, this strategy of integrating cloud services also improves the availability and reliability of the system, ensuring that critical tasks can be completed smoothly even when the resources within the cluster are limited.

[0086] The present invention provides an efficient GPU server cluster system, which aims to optimize the execution of video image data processing tasks. The system is composed of multiple GPU service nodes 101 as core computing resources, and each node has the ability to process video image data sets. The data transmission unit 102 is responsible for obtaining the video image data set to be processed from the data demand end, and sending the result data set back to the data demand end after the processing is completed. The task division unit 103 divides the large video image processing task into multiple subtasks to achieve parallel processing and improve processing efficiency.

[0087] The GPU scheduling unit 104 calculates the node load performance score according to the load performance parameters (including floating-point computing capability value, memory bandwidth utilization and core clock frequency) of each GPU service node 101, and sorts the nodes based on these scores, preferentially selecting the top N nodes with the highest scores as target GPU nodes, and assigning subtasks to them for execution. This process ensures that tasks are assigned to the GPU nodes with the best current performance and the lowest load, thereby improving overall processing efficiency and resource utilization.

[0088] The system also includes a task queue unit 106, which determines the priority of subtasks according to a preset priority definition rule, and considers the dependencies between subtasks to sort the subtasks and form a subtask queue. The GPU scheduling unit 104 retrieves subtasks according to this queue to ensure that tasks are executed in the correct order and avoid task delays caused by unsatisfied dependencies.

[0089] In order to further improve the flexibility and processing power of the system, the system also includes a GPU performance monitoring unit 107, which monitors the load performance parameters of the GPU node in real time and sends these parameters to the GPU scheduling unit 104 so that the scheduling unit can make more accurate scheduling decisions. In addition, the system also integrates an FPGA unit 108, which is used to process the video image data set containing the encrypted video stream, perform decoding processing, and send the decoded data set to the task segmentation unit 103.

[0090] The system of this embodiment is also extended to the field of edge computing, including edge service nodes 109. The GPU scheduling unit 104 can offload some subtasks to the edge service nodes 109 according to the load performance parameters of multiple GPU nodes to reduce the load of the central data center and provide faster data processing speed. At the same time, the system also includes a cloud server 110. The GPU scheduling unit 104 can call the GPU resources of the cloud server 110 as needed to cope with temporary computing demand peaks.

[0091] In summary, the GPU server cluster system of this embodiment can effectively schedule GPU computing power and improve the utilization efficiency of the GPU server cluster through various technical means such as intelligent scheduling, task segmentation, performance monitoring, FPGA-assisted processing, edge computing and cloud resource integration, and realize an efficient, flexible and scalable video image data processing solution that can meet data processing needs of different scales and complexities, especially in application scenarios that require rapid processing and analysis of large amounts of video image data, such as video surveillance, content distribution networks, real-time video analysis, etc.

[0092] The embodiment of the present invention further provides a GPU scheduling method, which is applied to the GPU server cluster system provided by the above embodiment, and can be specifically executed by a GPU scheduling unit. Figure 3 As shown, the method comprises the following steps: Step 201: Determine a load performance score of a GPU service node according to the load performance parameters of the GPU service node.

[0093] The load performance parameters include floating point computing capability, video memory bandwidth utilization and core clock frequency.

[0094] The GPU scheduling unit collects the three key performance parameters of each node: floating-point computing capability, memory bandwidth utilization, and core clock frequency, and uses these data to calculate a comprehensive load performance score. The general scoring rule is that the higher the floating-point computing capability, the lower the memory bandwidth utilization, and the higher the core clock frequency of the GPU service node, the higher its load performance score. Conversely, the lower the floating-point computing capability, the higher the memory bandwidth utilization, and the lower the core clock frequency of the GPU service node, the lower its load performance score.

[0095] This load performance score reflects the computing power and current workload of each GPU service node, providing a quantitative basis for subsequent node selection and task allocation.

[0096] In some embodiments, the load performance score may be calculated according to a load performance score formula, and the load performance score formula includes: ; Among them, S represents the load performance score, F represents the floating-point computing capability value, M represents the memory bandwidth utilization, C represents the core clock frequency, and α, β, and γ are all experimental parameters that can be obtained through prior experiments.

[0097] The load performance scoring formula comprehensively considers key GPU performance indicators such as floating-point computing capability, memory bandwidth utilization, and core clock frequency. The load performance of GPU service nodes can be evaluated more accurately to facilitate efficient task allocation.

[0098] Step 202: sort the GPU service nodes from high to low based on the load performance score.

[0099] Specifically, after obtaining the load performance scores of all GPU service nodes, the GPU scheduling unit will sort the nodes according to these scores, from high to low. This step ensures that the GPU service nodes with the strongest performance and the lowest load can be given priority, thereby improving the overall processing efficiency and response speed.

[0100] Step 203: determine the GPU service node ranked in the top N as the target GPU node.

[0101] Specifically, after the sorting is completed, the GPU scheduling unit will select the GPU service nodes ranked in the top N as the target GPU nodes. These nodes are considered to be the most suitable nodes for processing new tasks at present. They have high computing power and low workload, and can complete tasks quickly and efficiently.

[0102] Step 204 : when the task segmentation unit generates a video image processing task according to the video image data set to be processed and segments the video image processing task into a plurality of subtasks, the subtasks are allocated to the target GPU node.

[0103] Specifically, after the task segmentation unit segments the video image data set to be processed into multiple subtasks, the GPU scheduling unit will assign these subtasks to the previously determined target GPU nodes. This step ensures that tasks can be evenly and reasonably distributed to the entire GPU server cluster, optimizes resource utilization, and reduces the risk of overloading a single node.

[0104] Step 205: receiving video image processing task completion information.

[0105] The video image processing task completion information is generated by the data processing unit after merging multiple processed partial video image data into a complete processed video image data set.

[0106] Specifically, the data processing unit will merge multiple processed partial video image data into a complete processed video image data set and generate information that the video image processing task has been completed. The GPU scheduling unit receives this information, which not only marks the completion of the current task, but also provides feedback to the system for subsequent task scheduling and resource management.

[0107] The GPU scheduling method of this embodiment realizes intelligent allocation of tasks by accurately evaluating and sorting the load performance of GPU service nodes, thereby optimizing resource utilization and improving processing efficiency. It ensures that GPU nodes with high load performance scores are given priority for processing tasks, reducing waiting time and improving response speed. In addition, the method adapts to changes in workloads, balances the load between nodes, reduces resource waste, and improves the scalability and reliability of the system by real-time monitoring and dynamic adjustment of task allocation. Ultimately, this method not only speeds up the completion speed of video image processing tasks, but also extends the service life of hardware by reducing GPU overload and improving the fairness of resource allocation, bringing higher computing efficiency and better economic benefits to the GPU server cluster system.

[0108] It can be understood that the method of this embodiment applies the GPU server cluster system provided by the above embodiment, and thus can achieve the technical effect of the GPU server cluster system of the above embodiment, which will not be described in detail here.

[0109] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A GPU server cluster system, characterized in that: include: Multiple GPU service nodes; A data transmission unit, used for acquiring a video image data set to be processed from a data demand end; A task division unit, used for generating a video image processing task according to the video image data set, and dividing the video image processing task into a plurality of subtasks; A GPU scheduling unit, configured to determine a load performance score of the GPU service node according to a load performance parameter of the GPU service node, sort the GPU service nodes from high to low based on the load performance score, determine the GPU service nodes in the top N positions as target GPU nodes, and assign the subtask to the target GPU node, wherein the load performance parameter includes a floating-point computing capability value, a video memory bandwidth utilization rate, and a core clock frequency; The target GPU node is used to execute the subtask to obtain the processed portion of video image data; The data processing unit is used to merge the multiple processed partial video image data into a complete processed video image data set, and send the processed video image data set to the data demand end through the data transmission unit, and at the same time send the video image processing task completion information to the GPU scheduling unit.

2. The GPU server cluster system according to claim 1, characterized in that: The GPU scheduling unit is specifically used to calculate the load performance score according to a load performance scoring formula, and the load performance scoring formula includes: ； Among them, S represents the load performance score, F represents the floating-point computing capability value, M represents the memory bandwidth utilization, C represents the core clock frequency, and α, β, and γ are all experimental parameters.

3. The GPU server cluster system according to claim 1 or 2, characterized in that: It also includes a task queue unit, which is used to determine the priority of each subtask according to a set priority definition rule, and sort each subtask according to the priority to obtain a subtask queue. The GPU scheduling unit is used to call the subtask according to the subtask queue.

4. According to the GPU server cluster system of claim 3, the task queue unit is further used to determine the dependency relationship between the subtasks, and sort each of the subtasks according to the priority and the dependency relationship to obtain the subtask queue.

5. The GPU server cluster system according to claim 1, characterized in that: It also includes a GPU performance monitoring unit, which is used to monitor the load performance parameters of the GPU node and send the load performance parameters to the GPU scheduling unit.

6. The GPU server cluster system according to claim 1, characterized in that: It also includes an FPGA unit, which is used to receive the video image data set from the data transmission unit, decode the encrypted video stream when the video image data set contains an encrypted video stream, and send the decoded video image data set to the task segmentation unit.

7. The GPU server cluster system according to claim 1, characterized in that: It also includes an edge service node, and the GPU scheduling unit is used to offload part of the subtasks to the edge service node according to the load performance parameters of the multiple GPU nodes.

8. The GPU server cluster system according to claim 1, characterized in that: It also includes a cloud server, and the GPU scheduling unit is used to call the GPU resources of the cloud server according to the load performance parameters of the multiple GPU nodes.

9. A GPU scheduling method, characterized in that: The GPU server cluster system according to any one of claims 1 to 8 is applied, wherein the method comprises: Determine a load performance score of the GPU service node according to load performance parameters of the GPU service node, wherein the load performance parameters include a floating point computing capability value, a video memory bandwidth utilization rate, and a core clock frequency; Sorting the GPU service nodes from high to low based on the load performance scores; Determine the GPU service node ranked in the top N as the target GPU node; When the task segmentation unit generates a video image processing task according to the video image data set to be processed and segments the video image processing task into a plurality of subtasks, the subtasks are allocated to the target GPU node; The video image processing task completion information is received, wherein the video image processing task completion information is generated by a data processing unit after merging a plurality of processed partial video image data into a complete processed video image data set.

10. The method according to claim 9, characterized in that The step of determining the load performance score of the GPU service node according to the load performance parameter of the GPU service node comprises: The load performance score is calculated according to a load performance score formula, and the load performance score formula includes: ； Among them, S represents the load performance score, F represents the floating-point computing capability value, M represents the memory bandwidth utilization, C represents the core clock frequency, and α, β, and γ are all experimental parameters.

Images