Heterogeneous GPU scheduling system and method, storage medium, electronic equipment and product

Through the heterogeneous GPU scheduling system, the scheduling strategy computing platform and computing node management system are used, combined with multiple GPU type device plug-ins, the problem of heterogeneous GPU scheduling incompatibility is solved, and the compatibility scheduling and resource utilization of heterogeneous GPUs are realized. Improvement.

CN119045994APending Publication Date: 2024-11-29CHINA CONSTRUCTION BANK +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411072778.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The prior art is difficult to uniformly schedule heterogeneous GPUs in a diversified hardware environment, resulting in low overall resource utilization and incompatible heterogeneous GPU scheduling.

Method used

Provide a heterogeneous GPU scheduling system, including a scheduling policy computing platform, a computing node management system and a variety of GPU types of device plug-ins. Through the device plug-in, the resource status information is monitored, the target GPU type is determined, and the corresponding computing nodes are created and allocated. Execute tasks to achieve compatible scheduling of heterogeneous GPUs.

Benefits of technology

It realizes compatible scheduling of heterogeneous GPUs, improves resource utilization and system performance, and can effectively schedule multiple types of GPUs in different hardware environments, ensuring efficient execution of tasks on appropriate GPU devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045994A_ABST
    Figure CN119045994A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computers, and discloses a heterogeneous GPU scheduling system and method, a storage medium, electronic equipment and a product, the system comprises a scheduling strategy computing platform, a computing node management system, GPU equipment of multiple GPU types and equipment plug-ins corresponding to the GPU types respectively, the device plug-ins corresponding to the GPUs of various types are obtained by packaging based on the same packaging standard; the scheduling strategy calculation platform is used for determining a target GPU type corresponding to the to-be-processed task based on the to-be-processed task and / or the resource state information of the GPU equipment of each GPU type; and allocating GPU resources of the target GPU type to the computing node corresponding to the target GPU type based on the equipment plug-in corresponding to the target GPU type, and calling the computing node corresponding to the target GPU type to execute the to-be-processed task, thereby solving the problem that heterogeneous GPU scheduling is incompatible, and realizing compatible scheduling of the heterogeneous GPU.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a heterogeneous GPU scheduling system, method, storage medium, electronic device and product. Background Art

[0002] The Graphics Processing Unit (GPU) is one of the core components of artificial intelligence technology. GPU scheduling aims to distribute workloads to available GPU resources in an efficient manner, ensure that all processing cores are effectively utilized, and optimize the performance of the entire system.

[0003] At present, the existing technology usually dynamically predicts the throughput of each model service when it monopolizes the GPU to effectively adapt to complex and changeable workloads. Specifically, the throughput demand prediction process is periodically executed to predict the throughput demand in the new cycle for the model obtained by predicting thread allocation. The obtained predicted throughput demand is used to execute the throughput adjustment process based on the feedback control strategy at the scheduling time to optimize the actual throughput allocation of each model in the system.

[0004] However, existing technologies are highly dependent on specific hardware and software environments and may perform inconsistently in different deployment environments. In diverse hardware environments, it is difficult to uniformly schedule multiple types (i.e., heterogeneous) GPUs, resulting in poor overall resource utilization. Summary of the invention

[0005] The present invention provides a heterogeneous GPU scheduling system, method, storage medium, electronic device and product, which solves the problem of incompatible scheduling of heterogeneous GPUs and realizes compatible scheduling of heterogeneous GPUs.

[0006] According to one aspect of the present invention, a heterogeneous GPU scheduling system is provided, comprising: a scheduling strategy computing platform, a computing node management system, GPU devices of multiple GPU types, and device plug-ins corresponding to each GPU type, wherein the device plug-ins corresponding to each type of GPU are packaged based on the same packaging standard;

[0007] The device plug-in corresponding to each GPU type is used to monitor the resource status information of the GPU device of the corresponding GPU type;

[0008] The scheduling strategy computing platform receives the task to be processed, and determines the target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type;

[0009] The computing node management system is used to create computing nodes corresponding to the target GPU type;

[0010] The scheduling strategy computing platform allocates GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and calls the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

[0011] Optionally, node creation script templates corresponding to multiple GPU types are pre-stored in the computing node management system; the scheduling strategy computing platform is also used to parse the tasks to be processed and generate computing node parameters corresponding to the tasks to be processed; the computing node parameters and the target GPU type are transmitted to the computing node management system; the computing node management system calls the corresponding node creation script template based on the target GPU type, and generates a computing node creation script based on the called node creation script template and the computing node parameters; and the computing node creation script is executed to create a computing node corresponding to the target GPU type.

[0012] Optionally, the compute node parameters include the GPU usage resource amount of the target GPU type.

[0013] Optionally, the scheduling strategy computing platform is used to: determine a target GPU type that matches the business type based on the business type of the task to be processed.

[0014] Optionally, the resource status information of a GPU device of any GPU type includes resource usage data and load data; the scheduling strategy computing platform is also used to: determine the target GPU type corresponding to the task to be processed based on matching the computing node parameters corresponding to the task to be processed and the resource status information of the GPU devices of each GPU type.

[0015] Optionally, the system further includes a communication component corresponding to each GPU type, and the communication components corresponding to different GPU types provide different communication protocols.

[0016] Optionally, the scheduling strategy computing platform is also used to: after scheduling the computing node, verify the resource status information and GPU type of the computing node for the task to be processed, and reschedule the computing node if at least one of the resource status information of the computing node or the GPU type of the computing node does not match the task to be processed.

[0017] Optionally, the scheduling strategy calculation platform is configured with a display device, and the display device displays an interactive interface, and the interactive interface is used to input configuration information of the tasks to be processed.

[0018] Optionally, the tasks to be processed include one or more of a model training task and an inference task.

[0019] Optionally, the scheduling strategy computing platform is also used to call the task execution script template of the task to be processed based on the target GPU type; generate the task execution script of the task to be processed based on the configuration information of the task to be processed and the task execution script template, and send the task execution script of the task to be processed to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, so that the computing node corresponding to the target GPU type executes the task execution script of the task to be processed.

[0020] According to one aspect of the present invention, a heterogeneous GPU scheduling method is provided, which is applied to a scheduling strategy computing platform, and the method includes:

[0021] Obtain resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type;

[0022] Receive a task to be processed, and determine a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type;

[0023] Send the target GPU type to the computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type;

[0024] Allocate GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and call the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

[0025] According to one aspect of the present invention, a heterogeneous GPU scheduling device is provided, which is configured on a scheduling strategy computing platform, and the device includes:

[0026] A resource status information acquisition module is used to acquire resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type;

[0027] A target GPU type determination module is used to receive a task to be processed, and determine a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type;

[0028] A computing node creation module, used to send the target GPU type to the computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type;

[0029] The computing node calling module is used to allocate GPU resources of the target GPU type to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and call the computing node corresponding to the target GPU type to execute the task to be processed.

[0030] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0031] at least one processor; and

[0032] a memory communicatively connected to at least one processor; wherein,

[0033] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the heterogeneous GPU scheduling method of any embodiment of the present invention.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the heterogeneous GPU scheduling method of any embodiment of the present invention when executed.

[0035] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the heterogeneous GPU scheduling method of any embodiment of the present invention is implemented.

[0036] The technical solution of the embodiment of the present invention provides a heterogeneous GPU scheduling system, including: a scheduling strategy computing platform, a computing node management system, GPU devices of multiple GPU types, and device plug-ins corresponding to each GPU type; the device plug-ins corresponding to each type of GPU are packaged based on the same packaging standard; the scheduling strategy computing platform allocates GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and calls the computing nodes corresponding to the target GPU type, instead of only scheduling GPU devices of the same GPU type, thereby solving the problem of incompatible scheduling of heterogeneous GPUs and realizing compatible scheduling of heterogeneous GPUs.

[0037] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1It is a structural schematic diagram of a heterogeneous GPU scheduling system provided by Embodiment 1 of the present invention;

[0040] Figure 2 is a flow chart of a heterogeneous GPU scheduling method provided by Embodiment 3 of the present invention;

[0041] Figure 3 It is a structural schematic diagram of a heterogeneous GPU scheduling device provided by Embodiment 4 of the present invention;

[0042] Figure 4 It is a structural schematic diagram of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first characteristic data", "second characteristic data", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] The acquisition, storage and / or processing of data in the technical solutions involved in this application comply with the relevant provisions of national laws and regulations.

[0046] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0047] Embodiment 1

[0048] Figure 1This is a structural diagram of a heterogeneous GPU scheduling system provided in Example 1 of the present invention. This embodiment is applicable to the case of unified scheduling of heterogeneous GPUs. The system can be implemented in the form of hardware and / or software, and optionally, through an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0049] like Figure 1 As shown, the heterogeneous GPU scheduling system 100 may specifically include: a scheduling strategy computing platform 110, a computing node management system 120, GPU devices 130 of multiple GPU types, and device plug-ins 140 corresponding to each GPU type. The device plug-ins 140 corresponding to each type of GPU are packaged based on the same packaging standard.

[0050] The GPU device 130 is a device for providing high-performance computing, and the GPU type is the architecture type of the GPU device 130. For example, the GPU type includes but is not limited to a neural processing unit (NPU) architecture type and a deep computing unit (DCU) architecture type.

[0051] The device plug-in 140 corresponding to each GPU type is used to monitor the resource status information of the GPU device 130 of the corresponding GPU type.

[0052] In this embodiment, the device plug-in 140 may be an interface plug-in corresponding to the GPU device 130. The device plug-in 140 has a state management function and a task transmission function, and the state management function includes a function of monitoring resource state information. The resource state information is information representing the state of the GPU device 130.

[0053] It should be noted that each GPU type corresponds to a device plug-in 140. Exemplarily, the device plug-in 140 includes but is not limited to an Ascend device plug-in 140 and a Dcu device plug-in 140, wherein the Ascend device plug-in 140 corresponds to an NPU architecture type, and the Dcu device plug-in 140 corresponds to a DCU architecture type.

[0054] The packaging standards corresponding to the device plug-ins 140 corresponding to the multiple GPU types are the same packaging standard. By encapsulating the functions of the GPU devices 130 of different GPU types based on the same packaging standard, the device plug-ins 140 corresponding to the different GPU types are obtained, so that the device plug-ins 140 corresponding to the different GPU types can be uniformly scheduled. Exemplarily, the device plug-ins 140 corresponding to the different GPU types can be scheduled by the scheduling strategy calculation platform 110 based on a unified scheduling strategy.

[0055] Specifically, during the operation of the heterogeneous GPU scheduling system 100, each device plug-in 140 monitors the resource status of the GPU device 130 of the corresponding GPU type to obtain resource status information, and the device plug-in 140 transmits the resource status information to the scheduling strategy computing platform 110. Exemplarily, the Ascend device plug-in 140 monitors the resource status information of the GPU device 130 of the NPU architecture type, and the Dcu device plug-in 140 monitors the resource status information of the GPU device 130 of the DCU architecture type.

[0056] The scheduling strategy calculation platform 110 receives the task to be processed, and determines the target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of the GPU device 130 of each GPU type.

[0057] In this embodiment, the scheduling strategy computing platform 110 is a platform for automatically generating scheduling strategies for batch computing and high-performance computing scenarios, wherein the scheduling strategies include a variety of batch computing strategies and elastic workload distribution strategies, for example, the scheduling strategies include but are not limited to group scheduling, priority preemption, binning, resource reservation, and task topology relationship. Exemplarily, the scheduling strategy computing platform 110 is a Volcano platform.

[0058] The tasks to be processed are tasks executed based on GPU resources, wherein the GPU resources are computing resources provided by the GPU device 130. Optionally, the tasks to be processed include one or more of a model training task and an inference task, wherein the model training task is a task for training a deep learning model based on GPU resources, and the inference task is a task for inference using a trained deep learning model based on GPU resources. The tasks to be processed may be only a model training task, only an inference task, or a joint task including a model training task and an inference task, and this embodiment does not limit this, so that the heterogeneous GPU scheduling system 100 can be applied to the case of model training and / or inference.

[0059] Specifically, in response to the received pending tasks, the scheduling policy computing platform 110 may determine the target GPU type from multiple GPU types based on the pending tasks, based on the resource status information of the GPU devices 130 of each GPU type, or based on the pending tasks and the resource status information of the GPU devices 130 of each GPU type. The target GPU type is the GPU type of the GPU device 130 that supports the execution of the pending tasks. Furthermore, the scheduling policy computing platform 110 transmits the target GPU type to the computing node management system 120.

[0060] In some embodiments, the scheduling policy computing platform 110 determines the target GPU type based on the task to be processed. Specifically, the scheduling policy computing platform 110 matches the task to be processed with multiple GPU types respectively, and when the task to be processed successfully matches with any GPU type, the GPU type that matches successfully is determined as the target GPU type. Among them, the GPU type that successfully matches the task to be processed is the GPU type of the GPU device 130 that supports the task to be processed. The task to be processed can be successfully matched with one or more GPU types. Optionally, the scheduling policy computing platform 110 is used to: determine the target GPU type that matches the business type based on the business type of the task to be processed.

[0061] In this embodiment, the business type is used to characterize the GPU type of the GPU device 130 that can be used to execute the task to be processed. Each business type corresponds to a preset GPU type, and the preset GPU type is the GPU type characterized by the business type. The target GPU type is the GPU type of the GPU device 130 that the heterogeneous GPU scheduling system 100 needs to schedule. The scheduling strategy computing platform 110 obtains the configuration information of the task to be processed by parsing the parsed job definition (YAML) file, wherein the YAML file includes the configuration information characterizing the business type. In some embodiments, optionally, the scheduling strategy computing platform 110 is configured with a display device, and the display device displays an interactive interface, and the interactive interface is used to input the configuration information of the task to be processed.

[0062] In this embodiment, the display device is a device for presenting an interactive interface, for example, the display device includes but is not limited to a liquid crystal display screen, a projector, and an LED screen. Among them, the interactive interface is an interface for setting configuration information of the task to be processed and presenting the set configuration information.

[0063] Specifically, the scheduling strategy calculation platform 110 generates a YAML file based on the configuration information of the task to be processed input by the user in the interactive interface.

[0064] Exemplarily, the interactive interface can be an interactive document editing interface, and the scheduling strategy calculation platform 110 saves the document edited by the user in the interactive interface as a YAML file in response to the document save operation; the interactive interface can be a configuration interface for information items of the task to be processed, and the scheduling strategy calculation platform 110 responds to the configuration save operation, determines the information of each information item configured by the user in the interactive interface as a configuration information of the task to be processed, and generates a YAML file based on multiple configuration information of the task to be processed.

[0065] According to the technical solution of this embodiment, the scheduling strategy calculation platform 110 is configured with a display device, and the display device displays an interactive interface. The interactive interface is used to input configuration information of the task to be processed, and can realize interactive configuration of the configuration information and present the configuration information input by the user in real time, which helps to improve the user experience and help the user understand and adjust the configuration information.

[0066] Specifically, a task to be processed is generated based on the configuration information. The business type of the task to be processed is determined based on the configuration information characterizing the business type. If any GPU type of the GPU device 130 in the heterogeneous GPU scheduling system 100 is a preset GPU type corresponding to the business type of the task to be processed, it is considered that the GPU type matches the business type of the task to be processed, and the GPU type is determined as the target GPU type.

[0067] Exemplarily, assuming that the configuration information of the business type in the YAML file is "NPU", the business type of the task to be processed generated based on the YAML file is the NPU business type, and the target GPU type is the NPU architecture type.

[0068] The technical solution of this embodiment can quickly determine the target GPU type and save computing resources by determining the target GPU type that matches the business type based on the business type of the task to be processed.

[0069] In some embodiments, the scheduling policy computing platform 110 determines the target GPU type that matches the business type based on the resource status information of the GPU devices 130 of each GPU type. Specifically, the scheduling policy computing platform 110 determines the amount of computation that can be undertaken by the GPU devices 130 of multiple GPU types based on the resource status information of the GPU devices 130 of multiple GPU types. If the amount of computation that can be undertaken by the GPU devices 130 of any GPU type is greater than or equal to the amount of computation of the task to be processed, the GPU type is determined as the target GPU type. It should be noted that there may be one or more GPU types of GPU devices 130 that can undertake an amount of computation that is greater than or equal to the amount of computation of the task to be processed. Exemplarily, assuming that the amount of computation of the task to be processed is 16 gigabytes (GB), the amount of computation that can be undertaken by the GPU device 130 of the NPU architecture type is 14GB, and the amount of computation that can be undertaken by the GPU device 130 of the DCU architecture type is 16GB, then the target GPU type is the DCU architecture type.

[0070] In some embodiments, the scheduling policy computing platform 110 is based on the resource status information of the GPU devices 130 of the tasks to be processed and each GPU type. Specifically, the scheduling policy computing platform 110 determines each GPU type that successfully matches the tasks to be processed as the first candidate GPU type, and determines each GPU type that can bear a computing amount greater than or equal to the computing amount of the tasks to be processed as the second candidate GPU type, and determines the GPU type in the first candidate GPU type that is the same as the second candidate GPU type as the target GPU type. Exemplarily, assuming that the first candidate GPU type includes a DCU architecture type, and the first candidate GPU type includes an NPU architecture type and a DCU architecture type, then the target GPU type is a DCU architecture type.

[0071] The computing node management system 120 is used to create computing nodes corresponding to the target GPU type.

[0072] In this embodiment, the computing node management system 120 is a system for managing computing nodes, and the management of computing nodes by the computing node management system 120 includes creating computing nodes. Among them, the computing node is a virtual node for allocating GPU devices 130. Exemplarily, the computing node management system 120 is a Kubernetes-based system. It can be understood that the computing node corresponding to the target GPU type can be understood as a virtual node on the GPU device of the target GPU type, which is allocated with GPU resources of the target GPU type and has the computing power corresponding to the target GPU type.

[0073] Specifically, the scheduling strategy computing platform 110 transmits the target GPU type corresponding to the task to be processed to the computing node management system 120. The computing node management system 120 creates a computing node based on the target GPU type to obtain a computing node corresponding to the target GPU type. Exemplarily, assuming that the target GPU type is a DCU architecture type, the computing node management system 120 creates a DCU computing node, and the DCU computing node is used to schedule the GPU device 130 of the DCU architecture type to execute the task to be processed.

[0074] The scheduling strategy computing platform 110 allocates GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in 140 corresponding to the target GPU type, and calls the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

[0075] Specifically, the scheduling policy computing platform 110 responds to the computing node corresponding to the created target GPU type, and controls the device plug-in 140 corresponding to the target GPU type to allocate the GPU resources of the target GPU type to the computing node corresponding to the target GPU type. It should be noted that, in the case where the target GPU type corresponds to multiple computing nodes, the scheduling policy computing platform 110 allocates the GPU resources of the target GPU type to multiple computing nodes corresponding to the target GPU type based on the preset resource allocation strategy. Among them, the preset resource allocation strategy includes but is not limited to the load balancing strategy and the priority allocation strategy, which is not limited in this embodiment. The scheduling policy computing platform 110 routes the tasks to be processed to the computing nodes corresponding to the target GPU type by calling the computing nodes corresponding to the target GPU type allocated with GPU resources, so that the computing nodes corresponding to the target GPU type use the allocated GPU resources to execute the tasks to be processed.

[0076] Exemplarily, assuming that the target GPU type is a DCU architecture type, the scheduling policy computing platform 110 allocates GPU resources consisting of one or more GPU devices 130 of the DCU architecture type to the DCU computing node, and calls the DCU computing node to execute the task to be processed.

[0077] In some embodiments, the execution process of the pending task corresponding to each GPU type is the same. Optionally, the scheduling strategy computing platform 110 is also used to call the task execution script template of the pending task based on the target GPU type; generate the task execution script of the pending task based on the configuration information of the pending task and the task execution script template, and send the task execution script of the pending task to the computing node corresponding to the target GPU type through the device plug-in 140 corresponding to the target GPU type, so that the computing node corresponding to the target GPU type executes the task execution script of the pending task.

[0078] In this embodiment, the scheduling strategy computing platform 110 pre-stores task execution script templates corresponding to multiple GPU types. The task execution script template is a template of the task execution script for the task to be processed corresponding to each GPU type. The task execution script is a script used to control the computing node to execute the task to be processed.

[0079] Specifically, the scheduling strategy computing platform 110 sets the information to be configured corresponding to the configuration information in the task execution script template based on the configuration information of the task to be processed, and generates a task execution script for the task to be processed. The information to be configured is information in the task execution script template that can be configured based on the configuration information. The scheduling strategy computing platform 110 transmits the task execution script of the task to be processed to the device plug-in 140 corresponding to the target GPU type, and the device plug-in 140 transmits the task execution script to the computing node corresponding to the target GPU type.

[0080] Exemplarily, assuming that the task to be processed is a model training task of the DCU business type, and the target GPU type is the DCU architecture type, the task execution script template called by the scheduling strategy computing platform 110 is the DCU model training script template, and the information to be configured in the DCU model training script template includes the number of GPU devices 130, and the configuration information of the task to be processed includes configuration information representing that the number of GPU devices 130 is 5, then the number of GPU devices 130 in the task execution script of the task to be processed is 5; the scheduling strategy computing platform 110 transmits the task execution script to the computing node corresponding to the DCU architecture type through the Dcu device plug-in 140 to complete the distribution of the task to be processed; the computing node corresponding to the DCU architecture type calls the 5 allocated GPU devices 130 of the DCU architecture type to execute the task to be processed by running the received task execution script.

[0081] According to the technical solution of this embodiment, the scheduling strategy computing platform 110 calls the task execution script template of the task to be processed based on the target GPU type, and generates the task execution script of the task to be processed based on the configuration information of the task to be processed and the task execution script template, so that the computing node corresponding to the target GPU type can execute the task to be processed based on the received task execution script issued by the scheduling strategy computing platform 110, which simplifies the generation process of the task execution script corresponding to each GPU type, and can ensure that the task to be processed runs efficiently on the appropriate GPU device 130.

[0082] In some embodiments, GPU devices 130 of different GPU types in the heterogeneous GPU scheduling system 100 comply with different communication protocols. Optionally, the system also includes a communication component corresponding to each GPU type, and the communication components corresponding to different GPU types provide different communication protocols.

[0083] In this embodiment, the communication component is a component encapsulated with a communication protocol, and the communication component corresponding to each GPU type provides a communication protocol complied with by the GPU device 130 corresponding to the GPU type.

[0084] Specifically, each computing node corresponding to a GPU type corresponds to a communication component. The computing node in the heterogeneous GPU scheduling system 100 sends a scheduling request to the GPU device 130 of the GPU type through the communication component corresponding to the GPU type corresponding to the computing node, and the GPU device 130 sends the scheduling request feedback to the computing node through the communication component to implement GPU scheduling.

[0085] Exemplarily, the communication component corresponding to the NPU architecture type is the cloud computing component HCCl-Controller, and the communication protocol provided by the HCCl-Controller is the communication protocol complied with by the GPU device 130 of the NPU architecture type.

[0086] According to the technical solution of this embodiment, the heterogeneous GPU scheduling system 100 includes communication components corresponding to each GPU type. The communication components corresponding to different GPU types provide different communication protocols, thereby ensuring the reliability of communication among GPU devices 130 of various GPU types in the heterogeneous GPU scheduling system 100.

[0087] In some embodiments, the heterogeneous GPU scheduling system 100 can perform heterogeneous GPU scheduling based on multiple tasks to be processed at the same time, and there may be inaccurate scheduling of computing nodes. Optionally, the scheduling strategy computing platform 110 is also used to: after scheduling the computing node, verify the resource status information and GPU type of the computing node for the task to be processed, and reschedule the computing node when at least one of the resource status information of the computing node or the GPU type of the computing node does not match the task to be processed.

[0088] In this embodiment, the resource status information of the computing node is used to represent the overall situation of the resource status information of the GPU device 130 allocated to the computing node.

[0089] Specifically, the resource status information of the GPU device 130 assigned to the scheduled computing node is monitored based on the device plug-in 140, and the resource status information of the GPU device 130 assigned to the computing node is summed to obtain the resource status information of the computing node. Exemplarily, assuming that any computing node is assigned to three GPU devices 130, the resource status information of the computing node is the sum of the resource status information assigned to the three GPU devices 130.

[0090] The scheduling strategy calculation platform 110 makes predictions based on the resource status information of the scheduled computing nodes before the scheduled tasks and the scheduled computing nodes, and obtains the predicted resource status information of the scheduled computing nodes after the scheduling. If the difference between the predicted resource status information and the resource status information of the computing node after the scheduling is less than or equal to the preset difference range, it can be considered that the pending tasks of the resource status information of the computing node match; if the difference between the predicted resource status information and the resource status information of the computing node after the scheduling is greater than the preset difference range, it can be considered that the pending tasks of the resource status information of the computing node do not match. Exemplarily, the preset difference range is 5% of the predicted resource status information.

[0091] The scheduling strategy computing platform 110 determines the GPU type corresponding to the scheduled computing node. If the GPU type corresponding to the scheduled computing node is the target GPU type corresponding to the object to be processed, it can be considered that the GPU type of the computing node matches the task to be processed; if the GPU type corresponding to the scheduled computing node is not the target GPU type corresponding to the object to be processed, it can be considered that the GPU type of the computing node does not match the task to be processed. Exemplarily, assuming that the GPU type corresponding to the scheduled computing node is the NPU architecture type, and the target GPU type corresponding to the object to be processed is the DCU architecture type, it is determined that the GPU type of the computing node does not match the task to be processed.

[0092] If the resource status information of the computing node does not match the pending task and / or the GPU type of the computing node does not match the pending task, it can be considered that there is an error in the scheduling of the computing node by the heterogeneous GPU scheduling system 100, and the computing node is rescheduled; if the resource status information of the computing node matches the pending task and the GPU type of the computing node matches the pending task, it can be considered that the scheduling of the computing node by the heterogeneous GPU scheduling system 100 is accurate, and the pending task continues to be executed based on the computing node.

[0093] The technical solution of this embodiment verifies the resource status information and GPU type of the computing node for the task to be processed after scheduling the computing node. When at least one of the resource status information of the computing node or the GPU type of the computing node does not match the task to be processed, the computing node is rescheduled. This can promptly correct problems in the computing node scheduling process, thereby improving the fault tolerance and stability of the heterogeneous GPU scheduling system 100.

[0094] The technical solution of the embodiment of the present invention provides a heterogeneous GPU scheduling system 100, including: a scheduling strategy computing platform 110, a computing node management system 120, GPU devices 130 of multiple GPU types, and device plug-ins 140 corresponding to each GPU type; the device plug-ins 140 corresponding to each type of GPU are packaged based on the same packaging standard; the scheduling strategy computing platform 110 allocates GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in 140 corresponding to the target GPU type, and calls the computing nodes corresponding to the target GPU type, instead of only scheduling GPU devices 130 of the same GPU type, thereby solving the problem of incompatible scheduling of heterogeneous GPUs and realizing compatible scheduling of heterogeneous GPUs.

[0095] Embodiment 2

[0096] The technical solution of the embodiment of the present invention is further optimized on the basis of any of the above embodiments. For the specific implementation method, please refer to the description of this embodiment.

[0097] like Figure 1 As shown, the system may specifically include: a scheduling strategy computing platform 110, a computing node management system 120, GPU devices of multiple GPU types 130, and device plug-ins 140 corresponding to each GPU type, wherein the device plug-ins 140 corresponding to each type of GPU are packaged based on the same packaging standard.

[0098] The computing node management system 120 pre-stores node creation script templates corresponding to multiple GPU types.

[0099] In this embodiment, the node creation script template is a template of a computing node creation script for each computing node corresponding to a GPU type, and the task execution script template includes but is not limited to task execution logic and command line parameters. The computing node creation script is a script for controlling the computing node management system 120 to create a computing node. By pre-storing the node creation script templates corresponding to multiple GPU types in the computing node management system 120, it is helpful to improve the generation efficiency of the computing node creation script.

[0100] The device plug-in 140 corresponding to each GPU type is used to monitor the resource status information of the GPU device 130 of the corresponding GPU type.

[0101] The scheduling strategy calculation platform 110 receives the task to be processed, and determines the target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of the GPU device 130 of each GPU type.

[0102] The scheduling strategy computing platform 110 is also used to parse the tasks to be processed, generate computing node parameters corresponding to the tasks to be processed, and transmit the computing node parameters and the target GPU type to the computing node management system 120 .

[0103] In this embodiment, the computing node parameters are configuration parameters of the computing node during the process of creating the computing node, which are used to characterize the amount of computing that the GPU device 130 assigned to the computing node needs to undertake during the execution of the pending task. Optionally, the computing node parameters include the GPU usage resource amount of the target GPU type. Among them, the GPU usage resource amount is the amount of resources that the target GPU type GPU device 130 needs to provide when the computing node executes the pending task. It can be understood that the more complex the pending task is, the greater the GPU usage resource amount. By using the GPU usage resource amount of the target GPU type as the computing node parameter, it is helpful to improve the rationality of the computing nodes created subsequently and improve the utilization rate of GPU resources.

[0104] Specifically, the scheduling strategy computing platform 110 analyzes the pending task, determines the demand for GPU resources of the pending task, and determines the configuration parameters of the computing node for calling GPU resources based on the demand for GPU resources, and determines the configuration parameters of the computing node as the computing node parameters. Exemplarily, assuming that the pending task is a model training task, the scheduling strategy computing platform 110 can generate the computing node parameters corresponding to the pending task by analyzing one or more of the model parameter amount, training data set and training hyperparameter of the pending task. After obtaining the target GPU type and computing node parameters, the scheduling strategy computing platform 110 automatically transmits the computing node parameters and the target GPU type to the computing node management system 120. Exemplarily, the scheduling strategy computing platform 110 transmits the computing node parameters and the target GPU type to the computing node management system 120 in the form of a session, and distinguishes a set of computing node parameters and target GPU types corresponding to different pending tasks by session numbers.

[0105] GPU usage refers to the amount of resources used by tasks running on a computer's graphics processing unit (GPU). These resources include the GPU's computing power, memory bandwidth, storage capacity, and other related hardware resources. The amount of GPU usage depends on the complexity and requirements of the task, such as the task's computational intensity, memory requirements, and data transfer volume. More complex tasks generally require more GPU resources to complete, while simpler tasks require fewer resources.

[0106] In some embodiments, the scheduling strategy computing platform 110 determines the target GPU type that matches the business type based on the resource status information of the GPU devices 130 of each GPU type. Optionally, the resource status information of the GPU device 130 of any GPU type includes resource usage data and load data, wherein the resource usage data is data characterizing the amount of resources used by the GPU device 130; the load data is data characterizing the load condition of the GPU device. Accordingly, the scheduling strategy computing platform 110 is also used to: determine the target GPU type corresponding to the task to be processed based on matching the computing node parameters corresponding to the task to be processed and the resource status information of the GPU devices 130 of each GPU type.

[0107] Specifically, the scheduling strategy computing platform 110 determines the amount of computation that can be undertaken by the GPU device 130 of each GPU type based on the resource status information of the GPU device 130 of each GPU type, including resource usage data and load data. If the amount of computation that can be undertaken by the GPU device 130 of any GPU type is greater than or equal to the amount of computation that the GPU device 130 assigned to the computing node, represented by the computing node parameters corresponding to the task to be processed, needs to undertake, it can be considered that the GPU device 130 of the GPU type supports the task to be processed, and it is determined that the computing node parameters corresponding to the task to be processed match the resource status information of the GPU device 130 of the GPU type, and the GPU type is determined as the target GPU type corresponding to the task to be processed.

[0108] For example, assuming that the GPU usage resource amount of the target GPU type included in the computing node parameters corresponding to the task to be processed is 16GB, and the resource status information of the GPU device 130 of the NPU architecture type includes resource usage data of 2GB and load data of 10%, it can be known that the computing amount that the GPU device 130 of the NPU architecture type can undertake is 20GB. It can be considered that the computing node parameters corresponding to the task to be processed and the resource status information of the GPU device 130 of the NPU architecture type match, and the target GPU type is the NPU architecture type.

[0109] The technical solution of this embodiment is that the scheduling strategy calculation platform 110 is used to match the computing node parameters corresponding to the task to be processed and the resource status information of the GPU devices 130 of each GPU type, and determine the target GPU type corresponding to the task to be processed, which can ensure that the task to be processed is efficiently executed on a suitable computing node, thereby improving the scheduling performance and GPU resource utilization of the heterogeneous GPU scheduling system 100.

[0110] The computing node management system 120 calls the corresponding node creation script template based on the target GPU type, generates a computing node creation script based on the called node creation script template and computing node parameters; and executes the computing node creation script to create a computing node corresponding to the target GPU type.

[0111] Specifically, the computing node management system 120 calls the node creation script template corresponding to the pre-stored target GPU type, and sets the parameters to be configured corresponding to the computing node parameters in the node creation script template based on the computing node parameters, and generates a computing node creation script. The parameters to be configured are parameters in the computing node creation script template that can be configured based on the computing node parameters. The computing node management system 120 automatically creates a computing node corresponding to the target GPU type by running the computing node creation script.

[0112] For example, assuming that the GPU usage resource amount of the target GPU type included in the computing node parameters corresponding to the task to be processed is 16GB, the computing node management system 120 sets the to-be-configured parameter representing the GPU usage resource amount in the computing node creation script to 16GB, ensuring that the computing node can call upon 16GB of GPU resources.

[0113] The scheduling strategy computing platform 110 sets the information to be configured corresponding to the configuration information in the task execution script template based on the configuration information of the task to be processed, and generates the task execution script of the task to be processed. The information to be configured is information in the task execution script template that can be configured based on the configuration information. The scheduling strategy computing platform 110 transmits the task execution script of the task to be processed to the device plug-in 140 corresponding to the target GPU type, and the device plug-in 140 transmits the task execution script to the computing node corresponding to the target GPU type.

[0114] The scheduling strategy computing platform 110 allocates GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in 140 corresponding to the target GPU type, and calls the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

[0115] The technical solution of the embodiment of the present invention provides a heterogeneous GPU scheduling system 100, including: providing a heterogeneous GPU scheduling system 100, wherein a computing node management system 120 pre-stores node creation script templates corresponding to multiple GPU types; the scheduling strategy computing platform 110 is also used to parse the tasks to be processed and generate computing node parameters corresponding to the tasks to be processed; the computing node parameters and the target GPU type are transmitted to the computing node management system 120; the computing node management system 120 calls the corresponding node creation script template based on the target GPU type, and generates a computing node creation script based on the called node creation script template and the computing node parameters; the computing node creation script is executed to create a computing node corresponding to the target GPU type, which simplifies the generation process of the computing node creation script, improves the creation speed of the computing nodes, and helps to ensure that the tasks to be processed run efficiently on the appropriate GPU device 130.

[0116] Embodiment 3

[0117] Figure 2 This is a flow chart of a heterogeneous GPU scheduling method provided by Embodiment 3 of the present invention. The heterogeneous GPU scheduling method can be executed by a heterogeneous GPU scheduling device, which can be implemented in the form of hardware and / or software, and the heterogeneous GPU scheduling device can be configured in a scheduling strategy computing platform.

[0118] like Figure 2 As shown, the method includes:

[0119] S210 . Obtain resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type.

[0120] S220 , receiving a task to be processed, and determining a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type.

[0121] S230. Send the target GPU type to the computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type.

[0122] S240. Allocate GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and call the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

[0123] Based on the above embodiment, optionally, node creation script templates corresponding to multiple GPU types are pre-stored in the computing node management system; accordingly, the target GPU type is sent to the computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type, including: parsing the tasks to be processed and generating computing node parameters corresponding to the tasks to be processed; transmitting the computing node parameters and the target GPU type to the computing node management system; calling the corresponding node creation script template based on the target GPU type through the computing node management system, and generating a computing node creation script based on the called node creation script template and the computing node parameters; and executing the computing node creation script through the computing node management system to create a computing node corresponding to the target GPU type.

[0124] Based on the above embodiment, optionally, the computing node parameters include GPU usage resource amount of the target GPU type.

[0125] Based on the above embodiments, optionally, the target GPU type corresponding to the task to be processed is determined based on the resource status information of the task to be processed and / or GPU devices of each GPU type, including: based on the business type of the task to be processed, determining the target GPU type that matches the business type.

[0126] Based on the above embodiments, optionally, the resource status information of a GPU device of any GPU type includes resource usage data and load data; accordingly, determining the target GPU type corresponding to the task to be processed based on the resource status information of the task to be processed and / or the GPU devices of each GPU type also includes: determining the target GPU type corresponding to the task to be processed based on matching the computing node parameters corresponding to the task to be processed with the resource status information of the GPU devices of each GPU type.

[0127] Based on the above embodiment, optionally, after scheduling the computing node, the resource status information and GPU type of the computing node are verified for the task to be processed, and the computing node is rescheduled if at least one of the resource status information of the computing node or the GPU type of the computing node does not match the task to be processed.

[0128] On the basis of the above embodiment, optionally, the scheduling strategy calculation platform is configured with a display device, and the display device displays an interactive interface, and the interactive interface is used to input configuration information of the task to be processed.

[0129] Based on the above embodiment, optionally, the tasks to be processed include one or more of a model training task and a reasoning task.

[0130] On the basis of the above embodiment, optionally, a computing node corresponding to the target GPU type is called to execute the task to be processed, including: calling the task execution script template of the task to be processed based on the target GPU type; generating a task execution script for the task to be processed based on the configuration information of the task to be processed and the task execution script template, and sending the task execution script of the task to be processed to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, so that the computing node corresponding to the target GPU type executes the task execution script of the task to be processed.

[0131] The technical solution of the embodiment of the present invention obtains resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type; receives a task to be processed, and determines a target GPU type corresponding to the task to be processed based on the task to be processed and / or the resource status information of GPU devices of each GPU type; sends the target GPU type to a computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type; allocates GPU resources of the target GPU type to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and calls the computing node corresponding to the target GPU type to execute the task to be processed, instead of only scheduling GPU devices of the same GPU type, thereby solving the problem of incompatible scheduling of heterogeneous GPUs and realizing compatible scheduling of heterogeneous GPUs.

[0132] Embodiment 4

[0133] Figure 3 It is a structural diagram of a heterogeneous GPU scheduling device provided in Embodiment 4 of the present invention.

[0134] like Figure 3 As shown, the device can be configured on a scheduling strategy computing platform, including:

[0135] A resource status information acquisition module 310, configured to acquire resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type;

[0136] A target GPU type determination module 320 is used to receive a task to be processed, and determine a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type;

[0137] A computing node creation module 330, used to send the target GPU type to the computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type;

[0138] The computing node calling module 340 is used to allocate GPU resources of the target GPU type to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and call the computing node corresponding to the target GPU type to execute the task to be processed.

[0139] The technical solution of the embodiment of the present invention obtains resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type; receives a task to be processed, and determines a target GPU type corresponding to the task to be processed based on the task to be processed and / or the resource status information of GPU devices of each GPU type; sends the target GPU type to a computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type; allocates GPU resources of the target GPU type to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and calls the computing node corresponding to the target GPU type to execute the task to be processed, instead of only scheduling GPU devices of the same GPU type, thereby solving the problem of incompatible scheduling of heterogeneous GPUs and realizing compatible scheduling of heterogeneous GPUs.

[0140] Based on the above embodiment, optionally, node creation script templates corresponding to multiple GPU types are pre-stored in the computing node management system; accordingly, the computing node creation module 330 is specifically used to: parse the tasks to be processed, and generate computing node parameters corresponding to the tasks to be processed; transmit the computing node parameters and the target GPU type to the computing node management system; call the corresponding node creation script template based on the target GPU type through the computing node management system, and generate a computing node creation script based on the called node creation script template and the computing node parameters; and execute the computing node creation script through the computing node management system to create a computing node corresponding to the target GPU type.

[0141] Based on the above embodiment, optionally, the computing node parameters include GPU usage resource amount of the target GPU type.

[0142] Based on the above embodiment, optionally, the target GPU type determination module 320 is specifically used to: determine the target GPU type that matches the business type based on the business type of the task to be processed.

[0143] Based on the above embodiment, optionally, the resource status information of a GPU device of any GPU type includes resource usage data and load data; accordingly, the target GPU type determination module 320 is also used to: determine the target GPU type corresponding to the task to be processed based on matching the computing node parameters corresponding to the task to be processed and the resource status information of the GPU devices of each GPU type.

[0144] Based on the above embodiment, optionally, the heterogeneous GPU scheduling device also includes a computing node verification module; wherein the computing node verification module is used to verify the resource status information and GPU type of the computing node for the task to be processed after scheduling the computing node, and reschedule the computing node when at least one of the resource status information of the computing node or the GPU type of the computing node does not match the task to be processed.

[0145] On the basis of the above embodiment, optionally, the scheduling strategy calculation platform is configured with a display device, and the display device displays an interactive interface, and the interactive interface is used to input configuration information of the task to be processed.

[0146] Based on the above embodiment, optionally, the tasks to be processed include one or more of a model training task and a reasoning task.

[0147] Based on the above embodiment, optionally, the computing node calling module 340 is specifically used to: call the task execution script template of the task to be processed based on the target GPU type; generate the task execution script of the task to be processed based on the configuration information of the task to be processed and the task execution script template, and send the task execution script of the task to be processed to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, so that the computing node corresponding to the target GPU type executes the task execution script of the task to be processed.

[0148] The heterogeneous GPU scheduling device provided in the embodiment of the present invention can execute the heterogeneous GPU scheduling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0149] Embodiment 5

[0150] Figure 41 is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0151] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0152] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0153] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a heterogeneous GPU scheduling method.

[0154] In some embodiments, the heterogeneous GPU scheduling method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the heterogeneous GPU scheduling method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the heterogeneous GPU scheduling method in any other appropriate manner (e.g., by means of firmware).

[0155] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0157] Embodiment 6

[0158] Embodiment 6 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute a heterogeneous GPU scheduling method, and the method is applied to a scheduling strategy computing platform, including:

[0159] The method comprises the following steps: obtaining resource status information of GPU devices of multiple GPU types through device plug-ins corresponding to each GPU type; receiving a task to be processed, and determining a target GPU type corresponding to the task to be processed based on the task to be processed and / or the resource status information of GPU devices of each GPU type; sending the target GPU type to a computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type; allocating GPU resources of the target GPU type to the computing node corresponding to the target GPU type through a device plug-in corresponding to the target GPU type, and calling the computing node corresponding to the target GPU type to execute the task to be processed.

[0160] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0163] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0165] Embodiment 7

[0166] Embodiment 7 of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the heterogeneous GPU scheduling method according to any embodiment of the present invention.

[0167] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A heterogeneous GPU scheduling system, characterized in that: include: A scheduling strategy computing platform, a computing node management system, GPU devices of multiple GPU types, and device plug-ins corresponding to each GPU type, wherein the device plug-ins corresponding to each type of GPU are packaged based on the same packaging standard; The device plug-in corresponding to each GPU type is used to monitor resource status information of the GPU device of the corresponding GPU type; The scheduling strategy computing platform receives a task to be processed, and determines a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type; The computing node management system is used to create a computing node corresponding to the target GPU type; The scheduling strategy computing platform allocates GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and calls the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

2. The system according to claim 1, characterized in that The computing node management system pre-stores a plurality of node creation script templates corresponding to the GPU types respectively; The scheduling strategy computing platform is also used to parse the pending tasks and generate computing node parameters corresponding to the pending tasks; transmit the computing node parameters and the target GPU type to the computing node management system; The computing node management system calls a corresponding node creation script template based on the target GPU type, and generates a computing node creation script based on the called node creation script template and the computing node parameters; Execute the computing node creation script to create a computing node corresponding to the target GPU type.

3. The system according to claim 2, characterized in that The computing node parameters include the GPU usage resource amount of the target GPU type.

4. The system according to claim 1, characterized in that The scheduling strategy computing platform is used to: determine a target GPU type that matches the business type based on the business type of the task to be processed.

5. The system according to claim 3, characterized in that The resource status information of a GPU device of any of the GPU types includes resource usage data and load data; The scheduling strategy computing platform is also used to: determine the target GPU type corresponding to the task to be processed based on matching the computing node parameters corresponding to the task to be processed and the resource status information of the GPU devices of each GPU type.

6. The system according to claim 1, characterized in that The system further includes a communication component corresponding to each GPU type, and the communication components corresponding to different GPU types provide different communication protocols.

7. The system according to claim 1, characterized in that The scheduling strategy computing platform is also used to: after scheduling the computing node, verify the resource status information and GPU type of the computing node against the task to be processed, and reschedule the computing node if at least one of the resource status information of the computing node or the GPU type of the computing node does not match the task to be processed.

8. The system according to claim 1, characterized in that The scheduling strategy calculation platform is configured with a display device, and the display device displays an interactive interface, and the interactive interface is used to input configuration information of the task to be processed.

9. The system according to claim 1, characterized in that The tasks to be processed include one or more of a model training task and an inference task.

10. The system according to claim 9, characterized in that The scheduling strategy computing platform is also used to call the task execution script template of the task to be processed based on the target GPU type; A task execution script for the task to be processed is generated based on the configuration information of the task to be processed and the task execution script template, and the task execution script for the task to be processed is sent to the computing node corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, so that the computing node corresponding to the target GPU type executes the task execution script for the task to be processed.

11. A heterogeneous GPU scheduling method, characterized in that: Applied to a scheduling strategy computing platform, the method includes: Obtain resource status information of multiple GPU devices of the GPU type through a device plug-in corresponding to each GPU type; Receive a task to be processed, and determine a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type; Sending the target GPU type to a computing node management system so that the computing node management system creates a computing node corresponding to the target GPU type; Allocate GPU resources of the target GPU type to the computing nodes corresponding to the target GPU type through the device plug-in corresponding to the target GPU type, and call the computing nodes corresponding to the target GPU type to execute the tasks to be processed.

12. A heterogeneous GPU scheduling device, characterized in that: Configured on a scheduling strategy computing platform, the device includes: A resource status information acquisition module, used to acquire resource status information of multiple GPU devices of the GPU type through device plug-ins corresponding to each GPU type; A target GPU type determination module is used to receive a task to be processed, and determine a target GPU type corresponding to the task to be processed based on the task to be processed and / or resource status information of GPU devices of each GPU type; A computing node creation module, used to send the target GPU type to a computing node management system, so that the computing node management system creates a computing node corresponding to the target GPU type; A computing node calling module is used to allocate GPU resources of the target GPU type to the computing node corresponding to the target GPU type through a device plug-in corresponding to the target GPU type, and to call the computing node corresponding to the target GPU type to execute the task to be processed.

13. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the heterogeneous GPU scheduling method described in claim 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the heterogeneous GPU scheduling method described in claim 11 when executed.

15. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the heterogeneous GPU scheduling method according to claim 11.

Citation Information

Cited By

  • Heterogeneous GPU scheduling system, method and equipment and readable storage medium

    CN120144297A

  • Kubernetes cluster and resource management method thereof, electronic equipment, storage medium and computer program product

    CN121455676A