Method and apparatus for managing GPU resources

Abstract

Embodiments of the present specification provide a GPU resource management method and device, the method is applied to a cloud side node, comprising: receiving a processing request sent by a user, analyzing the processing request, determining an object identifier and GPU resource demand information corresponding to the processing request; in the case of determining that the container responds to the processing request according to the object identifier, determining a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group according to the GPU resource demand information, and in the case of determining that the virtual machine responds to the processing request according to the object identifier, determining a target edge node from the edge node group based on the idle virtual machine GPU resource information of each edge node according to the GPU resource demand information, wherein the edge node group contains at least two edge nodes, and each edge node is configured with at least two GPUs; sending the object identifier and the GPU resource demand information to the target edge node.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a GPU resource management method. Background Technology

[0002] With the popularization and promotion of cloud-native technologies, and considering the high cost of using CPUs at edge nodes, edge node GPUs are designed to provide a large amount of computing power at a relatively low cost. Furthermore, to further reduce computing costs, the need for edge node GPU virtualization has emerged. However, in practical applications, the software programs configured in edge nodes are quite complex, making it impossible for them to utilize GPU resources for virtual machine or container virtualization, thus increasing the difficulty of popularizing and promoting cloud-native technologies. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a GPU resource management method. One or more embodiments of this specification also relate to another GPU resource management method, a GPU resource management device, another GPU resource management device, a GPU resource management system, two GPU resource management nodes, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a GPU resource management method is provided, applied to a cloud-side node, comprising:

[0005] Receive a processing request sent by a user, analyze the processing request, and determine the object identifier and GPU resource requirement information corresponding to the processing request;

[0006] If it is determined that a container is responding to the processing request based on the object identifier, the target edge node is determined from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information.

[0007] If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs;

[0008] The object identifier and the GPU resource requirement information are sent to the target edge node.

[0009] According to a second aspect of the embodiments of this specification, a GPU resource management device is provided, applied to a cloud-side node, comprising:

[0010] The request receiving module is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests.

[0011] The scheduling module is configured to, upon determining, based on the object identifier that a container will respond to the processing request, determine a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group, and according to the GPU resource requirement information.

[0012] If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs;

[0013] The information sending module is configured to send the object identifier and the GPU resource requirement information to the target edge node.

[0014] According to a third aspect of the embodiments of this specification, a GPU resource management node is provided, comprising:

[0015] The request receiving module is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests.

[0016] An object scheduling module is configured to, upon determining that a container is responding to the processing request based on the object identifier, determine a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group, and according to the GPU resource requirement information.

[0017] If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs;

[0018] The information sending module is configured to send the object identifier and the GPU resource requirement information to the target edge node.

[0019] According to a fourth aspect of the embodiments of this specification, another GPU resource management method is provided, applied to edge nodes, including:

[0020] The cloud-side node receives object identifiers and GPU resource requirement information, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request.

[0021] If it is determined from the object identifier that a container is responding to the processing request, container GPU resources corresponding to the GPU resource requirement information are allocated from the container GPU group, and a container responding to the processing request is created based on the container GPU resources.

[0022] If it is determined from the object identifier that a virtual machine is responding to the processing request, virtual machine GPU resources corresponding to the GPU resource requirement information are allocated from the virtual machine GPU group, and a virtual machine responding to the processing request is created based on the virtual machine GPU resources.

[0023] According to a fifth aspect of the embodiments of this specification, another GPU resource management apparatus is provided for use at an edge node, comprising:

[0024] The information receiving module is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request;

[0025] The container management module is configured to, when it is determined from the object identifier that a container is responding to the processing request, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources, wherein the container runs in the container management unit;

[0026] The virtual machine management module is configured to, when it is determined from the object identifier that a virtual machine is responding to the processing request, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in the container management unit.

[0027] According to a sixth aspect of the embodiments of this specification, another GPU resource management node is provided, comprising:

[0028] The information receiving module is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request;

[0029] The container management module is configured to, when it is determined from the object identifier that a container is responding to the processing request, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources, wherein the container runs in the container management unit;

[0030] The virtual machine management module is configured to, when it is determined from the object identifier that a virtual machine is responding to the processing request, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in the container management unit.

[0031] According to a seventh aspect of the embodiments of this specification, a GPU resource management system is provided, the system comprising a cloud-side node and at least two edge nodes, each edge node being configured with at least two GPUs, wherein...

[0032] The cloud-side node is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing request. If the object identifier indicates that a container is responding to the processing request, the node determines a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information. Similarly, if the object identifier indicates that a virtual machine is responding to the processing request, the node determines a target edge node from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information. The edge node group contains at least two edge nodes, and each edge node is configured with at least two GPUs. The object identifier and the GPU resource requirement information are then sent to the target edge node.

[0033] The edge node is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of processing requests; if it is determined that a container will respond to the processing request based on the object identifier, the edge node allocates container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and creates a container to respond to the processing request based on the container GPU resources, wherein the container runs in a container management unit; and if it is determined that a virtual machine will respond to the processing request based on the object identifier, the edge node allocates virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and creates a virtual machine to respond to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in a container management unit.

[0034] According to an eighth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0035] Memory and processor;

[0036] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the GPU resource management method described above.

[0037] According to a ninth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the GPU resource management method described above.

[0038] According to a tenth aspect of an embodiment of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the GPU resource management method described above.

[0039] The GPU resource management method provided in this specification is applied to cloud-side nodes and includes: receiving a processing request sent by a user; analyzing the processing request to determine the object identifier and GPU resource requirement information corresponding to the processing request; if it is determined that a container will respond to the processing request based on the object identifier, determining a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information; and if it is determined that a virtual machine will respond to the processing request based on the object identifier, determining a target edge node from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes, and each edge node is configured with at least two GPUs; and sending the object identifier and the GPU resource requirement information to the target edge node.

[0040] The GPU resource management method provided in one embodiment of this specification is applied to cloud-side nodes. Upon receiving a processing request from a user, it can select a target edge node from the edge node group for creating a container based on the idle container GPU resource information provided by each edge node; and select a target edge node from the edge node group for creating a virtual machine based on the idle virtual machine GPU resource information provided by each edge node. The target edge node for creating the container and the target edge node for creating the virtual machine can be the same node. By sending the object identifier and GPU resource information—information used to create virtual machines and containers—to the target edge node, the target edge node can complete the creation of the virtual machine and container. This achieves the creation of virtual machines and containers on the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies. Attached Figure Description

[0041] Figure 1 This is a schematic diagram illustrating an application scenario of a GPU resource management method provided in one embodiment of this specification;

[0042] Figure 2 This is a flowchart illustrating a GPU resource management method provided in one embodiment of this specification;

[0043] Figure 3 This is a flowchart illustrating another GPU resource management method provided in one embodiment of this specification;

[0044] Figure 4 This is a schematic diagram illustrating an application scenario of a GPU resource management system provided in one embodiment of this specification;

[0045] Figure 5 This is a schematic diagram of the structure of a GPU resource management system provided in one embodiment of this specification;

[0046] Figure 6 This is a schematic diagram of cloud-native virtualization of a GPU resource management system provided in one embodiment of this specification;

[0047] Figure 7 This is a schematic diagram of a virtual machine directly connected to a GPU architecture for a GPU resource management system provided in one embodiment of this specification;

[0048] Figure 8 This is a schematic diagram of a GPU resource management system with memory isolation provided in one embodiment of this specification;

[0049] Figure 9 This is a schematic diagram illustrating the interaction between a module and hardware in a GPU resource management system according to one embodiment of this specification.

[0050] Figure 10 This is a schematic diagram of a GPU resource management system with GPU device dual-plugin technology and container virtual machine co-pool scheduling process provided in one embodiment of this specification;

[0051] Figure 11 This is a flowchart illustrating the processing procedure of a GPU resource management system according to one embodiment of this specification;

[0052] Figure 12 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0053] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0054] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0056] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0057] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0058] GPU: is an abbreviation for Graphics Processing Unit.

[0059] I0T is an abbreviation for Internet of Things.

[0060] Kubernetes (K8s) is an open-source application used to manage containerized applications across multiple hosts in a cloud platform.

[0061] Device plugin: Device plugins enable device manufacturers to allow kubelet to use the resources of their manufactured devices (Kubernetes-managed resources include CPU, memory, and storage resources) without modifying the Kubernetes core code.

[0062] KubeVirt is a Kubernetes plugin that provides Kubernetes with the ability to provision, manage, and control virtual machines on the same infrastructure as containers. KubeVirt enables Kubernetes to schedule, deploy, and manage virtual machines using the same tools as containerized workloads, eliminating the need for separate environments with different monitoring and management tools. It makes it possible for virtual machines and Kubernetes to work together. With KubeVirt, you can declare: create virtual machines, schedule virtual machines on a Kubernetes cluster, start virtual machines, stop virtual machines, and delete virtual machines. Virtual machines run in Kubernetes pods and utilize standard Kubernetes networking and storage.

[0063] A Pod is the smallest deployable unit that can be created and managed in Kubernetes.

[0064] eGPU: External graphics card.

[0065] CUDA (Compute Unified Device Architecture) is a parallel computing framework.

[0066] Runtime: The container runtime is the place and environment in which the container runs. The runtime needs to work closely with the operating system kernel to provide a runtime environment for the container.

[0067] NVML library: A C-based program interface used to monitor and manage various states in a GPU.

[0068] YAML: A file format.

[0069] Annotation: A note or annotation.

[0070] PCI devices: PCI devices refer to devices that are plugged into PCI slots. A PCI slot is an expansion slot based on the PCI (Peripheral Component Interconnection) bus. Devices such as graphics cards, sound cards, and network cards that can be plugged into PCI slots are all PCI devices.

[0071] Linux: An operating system

[0072] IOMMU is an abbreviation for Input / Output Memory Management Unit, which refers to a memory management unit.

[0073] vendorID: Vendor identification code.

[0074] Device ID: Device identification code.

[0075] KVM stands for Kernel-based Virtual Machine, which is a full virtualization technology based on the Linux kernel and employing hardware-assisted virtualization.

[0076] With the continuous development of computer technology, compared to the highly platform-based and centralized digital economy (such as the internet industry) or non-physical economy (such as the financial industry), the energy and power industry, the foundation of the real economy and industrial production, inevitably establishes a large number of geographically dispersed physical branches. For example, whether it is a wind farm, oilfield geophysical exploration, or coal mine, the work site generates a large amount of data and conducts on-site monitoring and operation management based on edge IoT devices. The complexity of on-site operations and the importance of safe production require edge nodes to have sufficient autonomous computing power to ensure autonomous management capabilities. At the same time, the enterprise's cloud-side nodes need to closely collaborate with the edge to ensure the implementation of enterprise control while enabling site autonomy. Based on this, a cloud-edge collaboration solution has been proposed and designed. This solution is a distributed open platform that integrates wide area networks, massive computing power, data storage, and application services. Compared to the global, long-cycle, high-latency, and big data computing characteristics of central sites (i.e., cloud-side nodes), the short-cycle, low-latency local data, and rapid response and decision-making characteristics of edge-side sites can better support local computing tasks. The edge side and the cloud side are not simply substitutes, but rather complementary and collaborative partners.

[0077] In practical applications, image processing requires significant computing power, while traditional CPUs (Central Processing Units) are not designed for intensive computation and are therefore costly. GPUs were designed to provide a large amount of computing power at a relatively low cost, assisting CPUs in computation. With the rise of artificial intelligence and large-scale models, GPUs are typically used in servers for AI model training. In cloud-edge collaborative solutions, models are usually trained at a central site (cloud-side node) using massive computing power, and then deployed at edge sites (edge ​​nodes) for on-site analysis and decision-making based on local computing power and data. To further reduce computing costs, the need for edge site GPU virtualization has emerged. By virtualizing edge site GPUs and reusing them across application services as much as possible, the role of GPUs can be maximized, resulting in cost reduction and efficiency improvement.

[0078] While edge site GPU virtualization solutions offer some support for GPU virtualization reuse, the increasing prevalence of cloud-native technologies presents challenges. Given the heterogeneous cloud environment and the complexity of software configurations on edge nodes, GPU virtualization on a single physical machine (edge ​​node) cannot simultaneously support virtual machines and containers. For example, on a physical server with multiple GPUs, if virtual machines manage the GPUs, all GPUs must be managed by virtual machines; similarly, if containers manage the GPUs, all GPUs must be managed by containers. For instance, in one solution provided in this specification, the GPU scheduling is node-based (server-level), requiring pre-planning of whether a GPU on a particular node will be used for containers or virtual machines. On the other hand, GPU vendors offer a vGPU solution for shared GPU scheduling. However, the vGPU solution only addresses GPU virtualization; on a given server, it can only be used by either virtual machines or containers, thus failing to solve the problem of shared GPU scheduling between containers and virtual machines on a single physical machine. Therefore, the inability to simultaneously support virtual machine and container virtualization on the same machine hinders the ability to meet the latest customer needs, increasing the difficulty of popularizing and promoting cloud-native technologies.

[0079] Based on this, this specification provides a GPU resource management method, and also relates to another GPU resource management method, a GPU resource management device, another GPU resource management device, a GPU resource management system, two GPU resource management nodes, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0080] Figure 1 This diagram illustrates an application scenario of a GPU resource management method according to an embodiment of this specification, applied to a cloud-side node. The cloud-side node 102 in this GPU resource management method is capable of managing at least two edge nodes 104. Furthermore, each edge node 104 is configured with at least two GPUs. See also... Figure 1As can be seen, the server cluster includes one cloud-side node 102 (i.e., the cloud or central node) and multiple edge nodes 104 (i.e., the edge side), each edge node 104 being configured with multiple GPUs. Furthermore, the multiple GPU devices configured on the edge node 104 are divided into container GPU groups and virtual machine GPU groups. The container GPU group can be understood as a set of CPUs within the edge node designated for use by containers. This container GPU group can contain one GPU device or at least two GPU devices. The virtual machine GPU group can be understood as a set of CPUs within the edge node designated for use by virtual machines. This virtual machine GPU group can contain one GPU device or at least two GPU devices. It should be noted that the GPU resource management method provided in this specification can be applied to various scenarios. Taking the power plant scenario as an example, the server cluster architecture in this method can be applied to the power plant scenario. The cloud-side node 102 can manage multiple edge nodes 104. Edge node 104 can be a server deployed in hundreds of power generation stations in various locations, and multiple edge nodes 104 can be managed uniformly through a cloud-side node 102.

[0081] Based on this, see Figure 1 When a cloud-side node receives a processing request from a user, it analyzes the request to determine the object identifier and GPU resource requirements needed to respond. The response can be achieved by creating a virtual machine or a container. If the object identifier indicates a container will respond, the cloud-side node determines a target edge node from the edge node cluster for container creation based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirements. If the object identifier indicates a virtual machine will respond, the cloud-side node determines a target edge node from the edge node group for virtual machine creation based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirements. It should be noted that the target edge node for container creation and the target edge node for virtual machine creation can be the same node. The object identifier and GPU resource requirements are then sent to the target edge node. This enables the edge node 104 to create a virtual machine responding to the processing request using virtual machine GPU groups and a container responding to the processing request using container GPU groups. This enables the creation of virtual machines and containers on the same edge node, avoiding the problem that software programs cannot be virtualized into virtual machines or containers, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0082] Figure 2A flowchart of a GPU resource management method according to an embodiment of this specification is shown. This GPU resource management method is applied to a cloud-side node and specifically includes the following steps.

[0083] Step 202: Receive the processing request sent by the user, analyze the processing request, and determine the object identifier and GPU resource requirement information corresponding to the processing request.

[0084] It should be noted that, in one or more embodiments of this specification, the cloud-side node used in the GPU resource management method can be a cloud-side node in the GPU resource management system. The GPU resource management system also includes at least two edge nodes, each configured with at least two GPUs.

[0085] In this context, the cloud-side node can be understood as the node that manages and controls the edge node, and this cloud-side node can be one or more servers. The edge node can be understood as the node managed and controlled by the cloud-side node, and this edge node can be a server, a physical machine, or a 10T device, without specific limitations. For example, in application scenarios such as wind farms, oilfield geophysical exploration, and coal mines, this edge node can be a server or an edge IoT device deployed at the work site.

[0086] A processing request can be understood as a request that requires cloud-side nodes to control edge nodes to generate containers or virtual machines for computation. For example, a user sends a processing request to a cloud-side node, instructing it to control the edge node to create a container or virtual machine that responds to the request and uses it to perform specific computational tasks. During container or virtual machine creation, the edge node needs to allocate corresponding GPU resources. An object identifier can be understood as an identifier for the container or virtual machine, which the cloud-side node or edge node uses to determine which container or virtual machine should respond to the processing request. For example, the object identifier can be type information, tag information, number, ID, name, etc., without specific limitations. The GPU resource requirement information can be understood as information indicating the GPU resources needed to create the container or virtual machine. Using this GPU resource requirement information, the edge node can allocate corresponding GPU resources to the container and virtual machine.

[0087] Specifically, the GPU resource management method provided in this manual can receive processing requests sent by users. These requests can be sent by the user to the cloud-side node via a terminal, or by the user sending a processing request to the cloud-side node through an interactive page provided by the cloud-side node.

[0088] Step 204: If it is determined that the processing request is responded to by a container based on the object identifier, a target edge node is determined from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information. If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information. The edge node group contains at least two edge nodes, and each edge node is configured with at least two GPUs.

[0089] Specifically, idle container GPU resource information can be understood as information indicating the current GPU resource usage of container GPU groups within that edge node. For example, available GPU resources for a container can be identified through tags, parameters, and numerical values. Idle virtual machine GPU resource information can be understood as information indicating the current GPU resource usage of that virtual machine GPU group. For example, available GPU resources for a virtual machine can be identified through tags, parameters, and numerical values.

[0090] An edge node group can be understood as a group consisting of all edge nodes managed by cloud-side nodes. Each edge node in the edge node group reports data such as its CPU, memory, GPU quantity, and GPU resource usage to the cloud-side nodes. A target edge node can be understood as an edge node in the edge node group used to create virtual machines; or a target edge node can be understood as an edge node in the edge node group used to create containers.

[0091] In one or more embodiments provided in this specification, determining the target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information includes:

[0092] Determine the available container GPU resource information for each edge node in the edge node group;

[0093] Based on the idle container GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group;

[0094] Each idle edge node is evaluated to obtain a first evaluation result for each idle edge node;

[0095] Based on the first evaluation result, the target edge node is determined from the at least two idle edge nodes.

[0096] The first evaluation result can be understood as the evaluation result of whether the cloud-side node can create containers on the edge node. This first evaluation result can be any value in the range [0, 1] or [0, 100]. Alternatively, the first evaluation result can be a label used to represent the performance of the edge node, such as "very idle" or "idle".

[0097] For example, when a cloud-side node determines that a container will respond to the processing request, the scheduler uses information such as the GPU resource requirements for creating the container, the edge node's CPU status, memory status, the number of GPUs used for the container, and the GPU usage for the container to sequentially filter each edge node to see if it meets the container creation conditions. This process identifies idle edge nodes that can create the container; in other words, only the remaining edge nodes are schedulable. Once multiple schedulable edge nodes are identified in the edge node group, an algorithm calculates their scores, and the edge node with the highest score is selected as the one capable of scheduling the container. This allows for the selection of the best-performing edge node for container scheduling based on the actual operating status of different edge nodes, thus facilitating successful container creation. The algorithm for calculating edge node scores is one that can calculate edge node performance based on GPU resource requirements, the edge node's uploaded CPU status, memory status, the number of GPUs used for the container, and the GPU usage for the container. This algorithm can be configured according to the actual application scenario, and no specific restrictions are imposed here.

[0098] It should be noted that, when an idle edge node is determined from the edge node group based on the idle container GPU resource information, and there is only one idle edge node, the idle edge node is determined as the target edge node.

[0099] In one or more embodiments provided in this specification, determining the target edge node from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information includes:

[0100] Determine the idle virtual machine GPU resource information for each edge node in the edge node group;

[0101] Based on the idle virtual machine GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group;

[0102] Each idle edge node is evaluated to obtain a second evaluation result for each idle edge node;

[0103] Based on the second evaluation result, the target edge node is determined from the at least two idle edge nodes.

[0104] The second evaluation result can be understood as the evaluation result of whether the cloud-side node can create virtual machines on the edge node. This second evaluation result can be any value in the range [0, 1] or [0, 100]. Alternatively, the second evaluation result can be a label used to represent the performance of the edge node, such as "very idle" or "idle".

[0105] For example, when a cloud-side node determines that a virtual machine should respond to a processing request, it uses the scheduler to filter each edge node based on the GPU resource requirements for creating the virtual machine, as well as information such as the CPU status, memory status, number of GPUs used for the virtual machine, and GPU usage uploaded by the edge node. This filters whether each edge node meets the conditions for creating a virtual machine, thus obtaining idle edge nodes capable of creating the virtual machine. In other words, only the remaining edge nodes are schedulable. Once it is determined that there are multiple schedulable edge nodes in the edge node group, an algorithm calculates the scores of these nodes and selects the edge node with the highest score as the one capable of scheduling the virtual machine. This allows for the selection of the best-performing edge node for virtual machine scheduling based on the actual operating status of different edge nodes, thus facilitating successful virtual machine creation. The algorithm for calculating the edge node score is one that can calculate the edge node performance based on information such as GPU resource requirements, CPU status, memory status, number of GPUs used for the virtual machine, and GPU usage. This algorithm can be configured according to the actual application scenario, and no specific restrictions are imposed here.

[0106] It should be noted that, when an idle edge node is determined from the edge node group based on the idle virtual machine GPU resource information, and there is only one idle edge node, the idle edge node is determined as the target edge node.

[0107] Step 206: Send the object identifier and the GPU resource requirement information to the target edge node.

[0108] In one or more embodiments provided in this specification, the object identifier and the GPU resource requirement information are sent to the target edge node, including:

[0109] An object configuration file is generated based on the object identifier and the GPU resource requirement information, and the object configuration file is sent to the target edge node.

[0110] The object configuration file can be understood as a configuration information file used to configure virtual machines or containers. When a virtual machine responds to the processing request, the object configuration file may contain configuration parameters such as the virtual machine image name, virtual machine interface information, and number of virtual machines required to configure the virtual machine. Alternatively, when a container responds to the processing request, the object configuration file may contain configuration parameters such as the container image name, container interface information, and number of containers required to configure the container. In one or more embodiments provided in this specification, the object configuration file can be a pod configuration file, which can be used to create pods on edge nodes and configure virtual machines or containers within the pods. In one embodiment provided in this specification, the pod configuration file can refer to a pod template. A pod template in Kubernetes can be a document for creating pods, containing information such as configurations used to create pods, pod container images, pod resource requirements, and pod port information. Based on this, in the embodiments of this specification, the pod configuration required to execute the processing request can be obtained by analyzing the processing request sent by the user. The configuration in the pod template can be adjusted using this pod configuration, for example, the pod resource requirements, to obtain a pod template capable of executing the processing request. This pod template prepares the necessary environment resources for creating pods on edge nodes later.

[0111] Specifically, the cloud-side node generates an object configuration file based on the object identifier and GPU resource requirement information, and sends the object configuration file to the target edge node, thereby scheduling virtual machines and containers to the target edge node, and facilitating the creation of virtual machines and containers by the target edge node through the object configuration file.

[0112] In one or more embodiments provided in this specification, generating an object configuration file based on the object identifier and the GPU resource requirement information includes:

[0113] If it is determined from the object identifier that the container is responding to the processing request, a container configuration file is generated based on the object identifier and the GPU resource requirement information; and

[0114] If it is determined that the processing request is to be responded to by a virtual machine based on the object identifier, a virtual machine configuration file is generated based on the object identifier and the GPU resource requirement information, and the virtual machine configuration file is converted into a containerized virtual machine configuration file that is recognized by the cloud-side node and the edge node.

[0115] The container configuration file is used to configure the configuration information of the container; for example, it can be a pod template. The virtual machine configuration file is used to configure the configuration information of the virtual machine; however, it cannot be recognized by cloud-side nodes and edge nodes, therefore a virtual machine cannot be generated based on it. The containerized virtual machine configuration file can be understood as a configuration information file for the virtual machine, but it can be recognized by cloud-side nodes and edge nodes, so a virtual machine can be generated based on it; for example, it can be a transformed pod template that can be recognized by the Kubernetes cluster.

[0116] Continuing with the previous example, the processing request can be a computation instruction sent by a user and received by the cloud-side node. In one or more embodiments provided in this specification, after receiving the computation instruction, it is formatted and encapsulated by a converter, thereby transforming the computation instruction into a processing request that an API server in the cloud-side node can recognize. By analyzing the processing request, the pod template required to execute the processing request is determined.

[0117] In one or more embodiments provided in this specification, after determining the pod template, the scheduler determines the container scheduling strategy based on the pod configuration in the pod template, the resources required by the pod, and the resource information uploaded by each edge node. Simultaneously, it is necessary to determine how many pod replicas need to be created for the pod running in the container. A pod replica is a copy of a pod; Kubernetes can schedule multiple pod replicas of the same pod on multiple edge nodes, thereby deploying the pod to the edge nodes.

[0118] Since the scheduling policy predefines the scheduling path for each pod replica, specifying which edge node each pod replica will be scheduled to, the cloud-side nodes record the scheduling information for each pod replica to determine which edge node each pod replica will be scheduled to. For example, pod replica 1 will be scheduled to edge node 1.

[0119] After recording the scheduling information, the edge nodes that need to be scheduled for the pod replica are determined based on this information. The pod template is then sent to the edge nodes, which are then responsible for starting the containers based on the pod template. Subsequently, the edge node finds the corresponding container image from the container image repository or local cache based on the container name recorded in the pod template, and starts the corresponding container through the container engine.

[0120] It should be noted that in one or more embodiments of this specification, the container engine's role can be: 1. Creating container images. A container image can be understood as a template for creating containers. Multiple container images need to be created before running a container, and these images are stored in a container image repository or a local cache. Before each container is launched from a pod, the corresponding container image is retrieved from the container image repository based on the container image name specified in the pod. In some cases, the container image may be stored in a local cache, so the corresponding container image can be retrieved from the cache. 2. The container engine is a runtime environment for containers. During container operation, the container engine provides the runtime environment; that is, the container engine sets how to run the container. 3. The container engine can manage containers. The container engine does not require Kubernetes and can manage containers independently.

[0121] In one or more embodiments provided in this specification, after receiving a computation instruction, its format is converted and encapsulated, thereby transforming the computation instruction into a processing request that an API server in a cloud-side node can recognize. By analyzing the processing request, the virtual machine template required to execute the request is determined. In another embodiment provided in this specification, when a user creates and sends a processing request to the cloud-side node, a virtual machine template for creating a virtual machine can be simultaneously issued. This virtual machine template may contain the configuration and specifications of the virtual machine. The virtual machine template may include settings such as the virtual machine name, virtual machine image name, resource requirements (e.g., CPU, memory, storage requirements), and network settings.

[0122] In one or more embodiments provided in this specification, after the cloud-side node determines the virtual machine template, it converts the virtual machine template into a Pod template recognized by the Kubernetes cluster, also known as a virtual machine pod template, through the kubevirt component in the cloud-side node. The GPU resource requirements specified in the Pod template are converted into the resource requirements of the virtual machine Pod template. It should be noted that the virtual machine pod template contains a virtual machine image, which is used to create virtual machines. Therefore, virtual machines in a pod can be managed through the pod, thus managing virtual machines like containers. The virtual machine image is stored in an image repository in the form of a container image. By creating image files from the virtual machine image and then pushing them to different image repositories, the virtual machine image can be pulled from the image repository when creating a virtual machine, thereby quickly creating the virtual machine.

[0123] After determining the virtual machine pod template, the cloud-side node uses the scheduler to determine the scheduling strategy based on the pod configuration in the virtual machine pod template, the resources required by the pod, and the resource information reported by each edge node. Simultaneously, it determines how many pod replicas need to be created for this pod.

[0124] Since the scheduling policy has already set the scheduling path for each pod replica, the cloud-side node will record the scheduling information of each pod replica. Therefore, based on the scheduling information, the edge node that the pod replica needs to be scheduled is determined, and the virtual machine pod template is sent to the edge node to control the edge node to start the virtual machine based on the virtual machine pod template.

[0125] In one embodiment provided in this specification, the cloud-side node adds GPU resource requirements to the issued Pod template, using the identifier corresponding to the container's GPU resource. It also adds GPU resource requirements to the issued virtual machine Pod template, using the identifier corresponding to the virtual machine's GPU resource. Therefore, although both container and virtual machine GPU resource requests are ultimately converted into scheduling for GPU extension resources of the Pod recognized by Kubernetes, the resource identifiers corresponding to container and virtual machine GPU resource requests are different, and the scheduling is ultimately based on unified scheduling by the Kubernetes scheduler. Because containers and virtual machines have different resource identifiers, the GPU allocation between containers and virtual machines at the physical machine level will not be confused at the scheduling level.

[0126] In one or more embodiments provided in this specification, before determining the target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information, the method further includes:

[0127] Receive idle container GPU resource information and idle virtual machine GPU resource information sent by each edge node;

[0128] The historical container GPU resource information is updated based on the idle container GPU resource information, and the historical virtual machine GPU resource information is updated based on the idle virtual machine GPU resource information.

[0129] Historical container GPU resource information can be understood as the idle container GPU resource information that was historically uploaded by the edge node and stored locally on the cloud-side node. Historical virtual machine GPU resource information can be understood as the idle virtual machine GPU resource information that was historically uploaded by the edge node and stored locally on the cloud-side node.

[0130] Specifically, in the GPU resource management method provided in this specification, Kubernetes deployed on the cloud-side nodes is responsible for container management and scheduling. Kubernetes enables the management of edge nodes in the server cluster. The edge nodes report idle container GPU resource information, such as the number of GPUs used for containers and their usage, as well as idle virtual machine GPU resource information, such as the number of GPUs used for virtual machines and their usage, facilitating the scheduling of virtual machines and containers by the cloud-side nodes.

[0131] Continuing with the previous example, the kubelet component in the edge node identifies the idle container GPU resources and idle virtual machine GPU resources of the edge node according to the device-plugin mechanism, and reports this information to the cloud (cloud-side node). At this point, the cloud-side node can see the available GPU resources for virtual machines and containers in each edge node in the reported node resources. The edge node's GPUs can then be used by the cloud for resource scheduling of containers and virtual machines.

[0132] In one or more embodiments provided in this specification, before determining the target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information, the method further includes:

[0133] Receive and record the GPU device group information sent by each edge node, wherein the GPU device group information includes GPU device information corresponding to container GPU groups and GPU device information corresponding to virtual machine GPU groups.

[0134] In this context, a GPU device group can be understood as a group of GPU devices. GPU device information can be understood as the device identifier, device performance, and / or device model of that GPU device.

[0135] Specifically, edge nodes can group their configured GPU devices based on the received GPU grouping information. After grouping, they report the GPU device group information to the cloud-side node. The cloud-side node can receive and record the GPU device group information reported by each edge node, thus enabling the cloud-side node to know the device status of the GPU devices in each edge node.

[0136] This specification provides a GPU resource management method applied to cloud-side nodes. Upon receiving a processing request from a user, the method selects a target edge node from the edge node group for creating a container based on the idle container GPU resource information provided by each edge node; and selects a target edge node for creating a virtual machine based on the idle virtual machine GPU resource information provided by each edge node. The target edge node for creating the container and the target edge node for creating the virtual machine can be the same node. By sending object identifiers and GPU resource requirement information—information used to create virtual machines and containers—to the target edge node, the target edge node can complete the creation of the virtual machine and container. This enables the creation of virtual machines and containers on the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0137] Figure 3 A flowchart of another GPU resource management method according to an embodiment of this specification is shown. This other GPU resource management method is applied to an edge node and specifically includes the following steps.

[0138] Step 302: Receive object identifier and GPU resource requirement information sent by the cloud-side node, wherein the object identifier and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request.

[0139] In one or more embodiments provided in this specification, receiving the object identifier and GPU resource requirement information sent by the cloud-side node includes:

[0140] The cloud-side node receives an object configuration file, wherein the object configuration file is generated by the cloud-side node through analysis of the processing request sent by the user and the object identifier and GPU resource requirement information.

[0141] For explanations regarding the object configuration file, request handling, and other related content, please refer to the corresponding or relevant content in the aforementioned GPU resource management method; further details will not be elaborated upon here.

[0142] For example, after receiving an object configuration file from the cloud node, the edge node will determine the object identifier and GPU resource requirement information from the configuration file. The object identifier is used to determine whether the processing request is being handled by a container or a virtual machine, and the GPU resource requirement information is used to allocate GPU resources to that container or virtual machine. Furthermore, the edge node can quickly create virtual machines or containers based on this object configuration file, improving the efficiency of container or virtual machine creation.

[0143] Step 304: If it is determined that the processing request is responded to by a container based on the object identifier, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container that responds to the processing request based on the container GPU resources; and if it is determined that the processing request is responded to by a virtual machine based on the object identifier, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine that responds to the processing request based on the virtual machine GPU resources.

[0144] In this context, a container GPU group can be understood as a set of CPUs within an edge node designated for use by containers. This container GPU group can contain one GPU device or at least two GPU devices. A virtual machine GPU group can be understood as a set of CPUs within an edge node designated for use by virtual machines. This container GPU group can contain one GPU device or at least two GPU devices. Container GPU resources can be understood as the computing resources or storage resources of the GPU devices within a container GPU group. Virtual machine GPU resources can be understood as the computing resources or storage resources of the GPU devices within a virtual machine GPU group, or simply the GPU devices within the virtual machine GPU group.

[0145] In one embodiment provided in this specification, the need for edge site GPU virtualization is proposed to further reduce computing costs. Meanwhile, the GPU resource management system provided in this specification, in order to enable GPU virtualization on a single physical machine to simultaneously support virtual machines and containers, divides the GPU on that physical machine into two parts, one for use by virtual machines and the other by containers. For GPUs on edge nodes, management can be achieved through virtual machine GPU driver modules and container GPU driver modules deployed on the edge nodes.

[0146] In one or more embodiments provided in this specification, the edge node is able to allocate GPU resources from a shared container GPU, thereby supporting the normal operation of the container. Specifically, allocating container GPU resources corresponding to the GPU resource requirement information from the container GPU group includes:

[0147] Idle container GPU resources are determined from the container shared GPUs contained in the container GPU group;

[0148] Allocate container GPU resources from the idle container GPU resources that correspond to the GPU resource requirement information.

[0149] Following the previous example, the cloud-side node detects the idle container GPU resources of the container shared GPUs contained in the container GPU group, and allocates idle GPU resources from the currently idle container GPU resources that are consistent with the GPU resources required by the user to create the container, thereby completing the operation of allocating container GPU resources for the container. Because the GPU resource management method provided in this specification divides the GPUs in the edge node into container GPU groups and virtual machine GPU groups, and uses them to create containers and virtual machines respectively, it ensures that both virtual machines and containers can be created on the same edge node by isolating the local GPUs. In one or more embodiments provided in this specification, the allocation results of the container GPUs are subsequently recorded on the pod during the allocation phase, so that the container device plugin can request GPU memory for the container according to the allocation results.

[0150] In one or more embodiments provided in this specification, the edge node selects an idle GPU from the virtual machine GPU group as the virtual machine GPU resource to support the normal operation of the virtual machine. The allocation of virtual machine GPU resources from the virtual machine GPU group corresponding to the GPU resource requirement information includes:

[0151] Identify the idle virtual machine GPUs in the virtual machine GPU group;

[0152] The idle virtual machine GPU is identified as the virtual machine GPU resource corresponding to the GPU resource requirement information.

[0153] Following the previous example, after determining that the object to be processed is a virtual machine, the edge node will select an idle GPU device from the virtual machine GPU group and use the idle GPU device as the virtual machine GPU resource corresponding to the GPU resource requirement information, so as to facilitate the subsequent creation of virtual machines based on the virtual machine GPU resource.

[0154] In one or more embodiments provided in this specification, the object configuration file is a container configuration file;

[0155] Accordingly, creating a container to respond to the processing request based on the container GPU resources includes:

[0156] Obtain the container image identifier from the container configuration file, and obtain the container image corresponding to the container image identifier from the container image repository;

[0157] Run the container image to obtain a container that responds to the processing request, and bind the container's GPU resources to the container.

[0158] The container image identifier can be understood as information that uniquely identifies a container image, such as its name and ID. The container image repository can be understood as a storage unit used to store the container image. This repository can be a region within storage devices such as cache, memory, or external storage used to store the container image.

[0159] Continuing with the previous example, after the cloud-side node sends the pod template to the edge node, the edge node will find the corresponding container image from the container image repository or local cache based on the container name recorded in the pod template, and then use the container engine to start the corresponding container, thus quickly completing the creation of the container responding to the processing request. This container resides in the pod on the cloud side. This container can be one or at least two.

[0160] In one or more embodiments provided in this specification, the object configuration file is a containerized virtual machine configuration file;

[0161] Accordingly, creating a virtual machine to respond to the processing request based on the virtual machine GPU resources includes:

[0162] Obtain the virtual machine image identifier from the containerized virtual machine configuration file, and obtain the virtual machine image corresponding to the virtual machine image identifier from the container image repository:

[0163] Run the virtual machine image according to the virtual machine configuration parameters to obtain a containerized virtual machine that responds to the processing request, and bind the virtual machine's GPU resources to the containerized virtual machine.

[0164] The virtual machine image identifier can be understood as information that uniquely identifies a virtual machine image, such as the name and number of the virtual machine image.

[0165] The virtual machine configuration parameters can be understood as the hardware configuration parameters required for the virtual machine to run, such as hard disk parameters, CPU parameters, memory parameters, port parameters, etc. These hardware configuration parameters are generated through various virtual machine emulation components. It should be noted that because the virtual machine needs to simulate various hardware parameters during operation using virtual machine emulation components, the edge node needs to configure the corresponding components to simulate the virtual machine's running environment during the creation of the virtual machine. These virtual machine emulation components can be KVM and / or QEMU.

[0166] Continuing with the previous example, after the cloud-side node sends the virtual machine pod template to the edge node, the edge node will find the corresponding virtual machine image from the container image repository or local cache based on the virtual machine image name recorded in the virtual machine pod template. Then, it will simulate the corresponding hardware configuration for the virtual machine using KVM and / or QEMU, thereby running the containerized virtual machine. It should be noted that there can be one or at least two containerized virtual machines. These containerized virtual machines can run within a pod and be managed by the pod. This allows for managing virtual machines like containers, further enabling the simultaneous configuration of containers and virtual machines on a single physical machine.

[0167] It should be noted that when you need to deploy an application in a virtual machine, you need to log in to the virtual machine from the cloud-side node and then use the cloud-side node to deploy the application to the virtual machine.

[0168] In one or more embodiments provided in this specification, the GPU resource management method provided in this specification, in order to improve management efficiency, allows a single cloud-side node to manage virtual machines and containers on multiple edge nodes. This architecture, which unifies the management of computing resources across multiple edge nodes through a cloud-side node, can be termed a "cloud-edge virtualization hyperconverged architecture."

[0169] Furthermore, running virtual machines and containers on edge nodes is because computing tasks on edge nodes may have time-sensitive requirements. If virtual machines and containers were run on cloud-side nodes, it would result in higher latency and make it impossible to process computing tasks on edge nodes in a timely manner.

[0170] In one or more embodiments provided in this specification, since the container GPU grouping adopts an eGPU shared scheduling scheme, multiple containers can share GPU scheduling. However, in order to ensure the normal execution of multiple containers sharing GPU scheduling and avoid failures caused by call errors, it is necessary to manage the container's use of GPU resources. Specifically, after creating a container to respond to the processing request based on the container GPU resources, the method further includes:

[0171] When the container performs a call operation on the container's GPU resources, determine the current usage status of the container's GPU resources;

[0172] Based on the current usage status, the container's calls to the container's GPU resources are managed.

[0173] Continuing with the previous example, the edge node monitors the container's GPU resource scheduling operations. If it determines that a container is making a call to its GPU resources, it determines the current GPU resource usage of the container. Then, it queries whether the container's available GPU resources are sufficient for the call, or whether the container's call to the shared GPU resource exceeds the amount of GPU resources allocated to it. Based on the query results, it decides whether the container resource allocation-related call was successful, thereby managing the container's GPU resource usage.

[0174] In one or more embodiments provided in this specification, in order to simultaneously create containers and virtual machines on the same edge node, the GPUs in the edge node need to be divided into container GPU groups and virtual machine GPU groups, and used for creating containers and virtual machines respectively. This ensures that both virtual machines and containers can be created on the same edge node by isolating the local GPUs. Specifically, before receiving the object identifier and GPU resource requirement information sent by the cloud-side node, the process further includes:

[0175] Receive GPU grouping information for the local GPU, wherein the GPU grouping information includes container GPU grouping information and virtual machine GPU grouping information;

[0176] Based on the container GPU grouping information and the virtual machine GPU grouping information, the container GPUs to be allocated and the virtual machine GPUs to be allocated are identified from the local GPUs.

[0177] Configure the virtual machine GPUs to be allocated as virtual machine GPU groups, and configure the container GPUs to be allocated as container GPU groups;

[0178] The GPU device information corresponding to the virtual machine GPU group and the GPU device information corresponding to the container GPU group are sent to the cloud-side node as GPU device group information.

[0179] Specifically, after receiving container GPU grouping information and virtual machine GPU grouping information sent to the local GPU, the edge node identifies the container GPU to be allocated from the local GPU based on the container GPU grouping information, and identifies the virtual machine GPU to be allocated from the local GPU based on the virtual machine GPU grouping information. Then, it configures the virtual machine GPU to be allocated as a virtual machine GPU group and the container GPU to be allocated as a container GPU group. It should be noted that, to ensure isolation between the container GPU group and the virtual machine GPU group, a driver for the container GPU can be used to manage the container GPU group, and a driver for the virtual machine GPU can be used to manage the virtual machine GPU group.

[0180] In one or more embodiments provided in this specification, the GPU grouping information for the local GPU may be sent from the node manager corresponding to the edge node to the edge node. For example, the edge node manager may be the device or program that manages the edge node. Obtaining the GPU grouping information for the local GPU may be sent from the cloud-side node to the edge node, or it may be sent from the user to the edge node.

[0181] In one or more embodiments provided in this specification, after configuring the virtual machine GPU to be allocated as a virtual machine GPU group and configuring the container GPU to be allocated as a container GPU group, the method further includes:

[0182] Determine the idle container GPU resource information corresponding to the container GPU group, and determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group;

[0183] The idle container GPU resource information and the idle virtual machine GPU resource information are sent to the cloud-side node.

[0184] Using the previous example, the edge node can identify the idle container GPU resource information corresponding to the container GPU group, and the idle virtual machine GPU resource information corresponding to the virtual machine GPU group; and send the idle container GPU resource information and the idle virtual machine GPU resource information to the cloud-side node, so that the cloud-side node can know the GPU resources available for virtual machines and containers, which is convenient for subsequent resource scheduling of containers and virtual machines.

[0185] In one or more embodiments provided in this specification, the edge node can identify the current usage of container GPU resources, thereby facilitating subsequent management of GPU devices based on these container GPU resources. Specifically, determining the idle container GPU resource information corresponding to the container GPU group includes:

[0186] Identify the runtime status information of the container-shared GPUs contained in the container GPU group;

[0187] Based on the running status information, the idle container GPU resource information of the container GPU group is determined.

[0188] Continuing with the previous example, edge nodes can use a driver that manages container GPUs to query container GPU memory usage. This driver is configured with a query component for querying container GPU usage. When it is necessary to query the usage of container GPU resources, this query component can be called to query the current container's GPU usage through the NVML library, thereby obtaining information on the available container GPU resources for that container GPU group.

[0189] In one or more embodiments provided in this specification, the edge node can identify the current usage of virtual machine GPU resources, thereby facilitating subsequent management of the GPU device based on those virtual machine GPU resources. Specifically, determining the idle virtual machine GPU resource information corresponding to the virtual machine GPU group includes:

[0190] Identify the idle virtual machine GPUs in the virtual machine GPU group;

[0191] Based on the idle virtual machine GPUs, determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group.

[0192] Using the previous example, edge nodes can identify the number of idle virtual machine GPUs in a virtual machine GPU group through the virtual machine GPU driver module that manages virtual machine GPUs, and use this number of idle virtual machine GPUs as the idle virtual machine GPU resource information corresponding to the virtual machine GPU group.

[0193] This specification provides an alternative GPU resource management method, applied to edge nodes. When a cloud-side node sends object identifiers and GPU resource requirement information to the edge node, the edge node can allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group and create a container responding to the processing request based on the container GPU resources; or allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group and create a virtual machine responding to the processing request based on the virtual machine GPU resources. This enables the creation of virtual machines and containers within the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0194] Figure 4 This diagram illustrates an application scenario of a GPU resource management system according to one embodiment of this specification. Figure 4This specification provides an overall architecture for edge-side container virtual machine GPU co-pooling scheduling in a cloud-edge scenario. Under the cloud-edge virtualization hyperconverged architecture, this system is divided into a cloud-side management and control side and an edge side, implementing a strategy of cloud-side management and moderate edge autonomy. Specifically, cloud-side node 402 manages and operates virtual machines based on cloud-native virtualization. Containers and virtual machines share Kubernetes cluster computing, network, and storage resources, and global unified scheduling of container and virtualization resources is achieved through Kubernetes. Cloud-side node 402 deploys management and control components, including Kubernetes management components, cloud-edge collaborative control plane components, and cloud-native virtualization control plane components. The GPU resource management system provided in this specification supports one-click addition of edge node 404 to the edge cluster and unified management of edge node 404 through the edge node 404 pool. Given the limited number and resource scarcity of edge nodes 404 in cloud-edge scenarios, each edge node 404 (i.e., server node) can be configured with multiple GPU cards to support the simultaneous GPU usage needs of AI applications from virtual machines and containers scheduled to this node.

[0195] In the cloud-edge virtualization hyperconverged architecture, edge node 404 groups the local GPUs based on instructions from cloud node 402. Figure 4 For example, if an edge node 404 (bare metal server) has four GPU cards, two of them, GPU-1 and GPU-2, are directly connected to virtual machines and serve as the computing power supply for AI (artificial intelligence) applications deployed on virtual machines; the other two GPU cards, GPU-3 and GPU-4, are provided to all containers on this edge node 404 for shared use through GPU sharing scheduling.

[0196] Furthermore, it should be noted that the GPU resource management system provided in this manual adopts the core technology of container virtual machine GPU co-pool scheduling. This technology relies on four core technological foundations to achieve container and virtual machine GPU co-pool scheduling: 1. Cloud-native virtualization technology: Enables unified management of containers and virtual machines. 2. GPU driver isolation technology: Allocates GPU resources to containers and virtual machines at the kernel level, providing finer granularity. 3. Container GPU shared scheduling technology: Through the provided container GPU shared scheduling scheme, it achieves container GPU shared scheduling and high-density deployment, increasing the practical value of this solution. 4. GPU device dual-plugin technology: Virtual machines and containers can use two plug-ins to manage their corresponding GPU resources, each plug-in performing its specific function, resulting in a very concise system.

[0197] The GPU resource management process provided in this manual is as follows: First, after pre-grouping GPU resources at edge node 404, cloud-side node 402 configures container GPU drivers for containers and virtual machine GPU drivers for virtual machines as needed. Second, cloud-side node 402 configures container device plugins for containers and virtual machine device plugins for virtual machines at edge node 404. These device plugins can identify their corresponding drivers; the container device plugin for containers can identify the container GPU driver, and the virtual machine device plugin for virtual machines can identify the virtual machine GPU driver. Furthermore, the device plugins use the corresponding GPU drivers to manage GPU resources in edge node 404 and report GPU resource usage to cloud-side node 402. Finally, when cloud-side node 402 receives a processing request from a user, it sends an object configuration file to edge node 404. The container device plugin or virtual machine device plugin in edge node 404, based on the object configuration file sent by cloud-side node 402, calls the container GPU driver or virtual machine GPU driver to allocate corresponding GPU resources for the virtual machine or container and create the virtual machine or container. This effectively balances virtual machine and container virtualization within the same edge node 404, avoiding the problem that software programs cannot be modified for virtual machine or container virtualization.

[0198] Book Figure 5 This diagram illustrates a structural schematic of a GPU resource management system according to an embodiment of this specification. The system includes a cloud-side node and at least two edge nodes, each edge node 504 being configured with at least two GPUs.

[0199] The cloud-side node is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing request. If the object identifier indicates that a container is responding to the processing request, the node determines a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information. Similarly, if the object identifier indicates that a virtual machine is responding to the processing request, the node determines a target edge node from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information. The edge node group contains at least two edge nodes, and each edge node is configured with at least two GPUs. The object identifier and the GPU resource requirement information are then sent to the target edge node.

[0200] The edge node is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of processing requests; if it is determined based on the object identifier that a container will respond to the processing request, the edge node allocates container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and creates a container responding to the processing request based on the container GPU resources, wherein the container runs in a container management unit; and if it is determined based on the object identifier that a virtual machine will respond to the processing request, the edge node allocates virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and creates a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in a container management unit.

[0201] In one embodiment provided in this specification, the GPU resource management system adopts cloud-native virtualization technology. This cloud-native virtualization technology uses Kubernetes and kubevirt to achieve co-pool management of containers and virtual machines, that is, to simultaneously orchestrate and schedule containers and virtual machines at a single layer on edge nodes (i.e., physical machines and servers, which contain corresponding operating systems). See details. Figure 6 , Figure 6 This diagram illustrates a cloud-native virtualization GPU resource management system according to one embodiment of this specification. This enables unified resource scheduling for containers and virtual machines, rather than the traditional separate management of virtual machines and containers. Cloud-native virtualization is the cornerstone of container-virtual machine GPU co-pooling scheduling, making co-pooling of container and virtual machine GPUs possible.

[0202] In this context, the cloud-side node can be understood as a node in the GPU resource management system that manages and controls edge nodes. This cloud-side node can be one or more servers. The edge node can be understood as a node in the GPU resource management system that is managed and controlled by the cloud-side node. This edge node can be a server, a physical machine, or an IoT device; no specific restrictions are placed here. For example, in application scenarios such as wind farms, oilfield geophysical exploration, and coal mines, this edge node can be a server or edge IoT device deployed at the work site. It should be noted that this GPU resource management system contains multiple edge nodes. A processing request can be understood as a request instructing the edge node to create a virtual machine or container to execute a specific computing task.

[0203] The container management module can be understood as a device plugin deployed in the edge node for managing containers; the virtual machine management module can be understood as a device plugin deployed in the edge node for managing virtual machines. In one or more embodiments provided in this specification, to utilize the GPU resources of the same physical machine (edge ​​node) to simultaneously create virtual machines and containers, the GPU resource processing method provided in this specification manages the container GPU driver module through the container management module. This allows the container management module to access the edge node's GPU resources through the container GPU driver module, thereby achieving the goal of creating containers using the edge node's GPU resources. Similarly, the GPU resource processing method provided in this specification manages the virtual machine GPU driver module through the virtual machine management module. This allows the virtual machine management module to access the edge node's GPU resources through the virtual machine GPU driver module, thereby achieving the goal of creating virtual machines using the edge node's GPU resources. In one embodiment provided in this specification, the GPU resource management system, in order to utilize the GPU resources in the edge node to simultaneously create virtual machines and containers on the same edge node, employs Kubernetes and kubevirt to achieve the creation of containers and virtual machines on the same edge node. However, Kubernetes and kubevirt cannot utilize the GPU resources on edge nodes during operation. Therefore, a container management module and a container GPU driver module are configured on the edge node. Kubernetes can then identify GPU resources through these modules and use them to create containers. Similarly, a virtual machine management module and a virtual machine GPU driver module are configured on the edge node. Kubernetes and kubevirt can then identify GPU resources through these modules and use them to create virtual machines. In other words, the GPU resource management system provided in this specification employs a dual-plugin technology for GPU devices. This technology addresses the issue that each edge node only installs one device plug-in for each device of the same type. To achieve shared pool scheduling of containers and virtual machines, a dual-plugin mechanism is used on top of GPU driver isolation technology, thus solving the problem of resource allocation for containers and virtual machines. Specifically, for virtual machines, a scheme is adopted where the virtual machine device plug-in (which can be understood as the aforementioned virtual machine management module) directly connects to the GPU. This allows the virtual machine device plug-in to perceive the GPU bound to the virtual machine GPU driver module and enables the virtual machine to bind to that GPU through direct connection. This virtual machine GPU driver module can be understood as a driver that manages the GPU devices in the virtual machine GPU group. Figure 7This is a schematic diagram of a virtual machine directly connected to a GPU architecture for a GPU resource management system provided in one embodiment of this specification. Based on Figure 7 As can be seen, in edge nodes, the virtual machine device plugin can identify the GPU bound to the virtual machine GPU driver and manage the GPUs in the virtual machine GPU group through Kubernetes, thereby creating virtual machines using the GPUs in that virtual machine GPU group. It should be noted that the virtual machine provided in the embodiments of this specification can be a containerized virtual machine, based on... Figure 7 It can be seen that the pod running this virtual machine contains KVM and Qemu. KVM and Qemu can simulate the corresponding hardware configuration parameters for the virtual machine to run.

[0204] The container management unit can be understood as a unit in the edge node that manages containers and containerized virtual machines. For example, the container management unit can be a pod.

[0205] The container creation process utilizes a container device plugin (which can be understood as the aforementioned container management module), enabling shared GPU scheduling for containers in multi-GPU scenarios on a single node. This container device plugin is aware of the GPU bound to the container's GPU driver module, allowing the container to bind to that GPU. The container GPU driver module can be understood as a driver program that manages the GPU devices within a container GPU group. It's important to note that the GPU resource management system provided in this specification proposes a container GPU shared scheduling technology in the process of implementing shared GPU scheduling for containers using the container device plugin. The container identification plugin in this technology primarily utilizes the device plugin mechanism. Called on the node via kubelet (a component deployed on edge nodes), it is responsible for implementing shared GPU scheduling for Pods, adapting to mainstream graphics cards by being compatible with multiple GPU driver versions. Furthermore, it provides computing power and memory isolation capabilities through eGPU (CUDA hijacking).

[0206] The GPU resource management system described in this manual employs an application container engine. This engine can package applications and their dependencies into a portable image, which can then be deployed to machines running the operating system (e.g., edge nodes), and can also achieve virtualization. This application container engine can be modified by altering the runtime. When the application container runs, it modifies the container's startup parameters by default, thus achieving runtime modification. The application container engine includes an information collection component that collects and processes information from the container, and also invokes a container mounting component. This container mounting component is used to mount the GPU driver and some dependent components and library files into the container. This component primarily exists in the form of host mounting. For example, the information collection component determines whether to allocate a GPU device and the device ID to be mounted based on environment variables. If it is unspecified or invalid (void), it is considered a non-GPU container and no action is taken. Otherwise, the container mounting component is invoked, passing the GPU device as a parameter. The container mounting component maps the driver library's .so file and GPU device information into the container through file mounting, allowing the container to access the GPU. For details on the implementation process, please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram illustrating memory isolation in a GPU resource management system according to one embodiment of this specification. The eGPU of the container device plugin provided in this specification can achieve GPU shared scheduling by hijacking CUDA. See also... Figure 8 eGPU encapsulates a memory control component on top of the CUDA API call library to manage GPU memory. It also implements a memory usage query component to check GPU memory usage. When a user program calls the CUDA API, it first enters the memory control component, which triggers a query action and communicates with the memory usage query component. The memory usage query component uses the NVML library to query the current container's GPU storage usage. The memory control component determines the success or failure of memory allocation-related API calls based on the returned query results. Simultaneously, a compute allocation component handles compute allocation. After GPU resources are requested in the Pod's YAML file, the compute allocation component determines during the scheduling phase whether a single GPU card on the node can provide sufficient GPU storage resources. During the allocation phase, it records the GPU allocation results on the pod using annotations, so that the plugin device can allocate GPU memory to the container according to the allocation results.

[0207] In this context, a container GPU group can be understood as a set of CPUs within an edge node designated for use by containers. This container GPU group can contain one GPU device or at least two GPU devices. For example... Figure 4 GPU-3 and GPU-4 are located on the edge nodes. A virtual machine GPU group can be understood as a set of CPUs within an edge node designated for use by virtual machines. This container GPU group can contain one GPU device or at least two GPU devices. For example... Figure 4 GPU-1 and GPU-2 at the middle edge nodes.

[0208] In one embodiment provided in this specification, the need for edge site GPU virtualization is proposed to further reduce computing costs. Meanwhile, the GPU resource management system provided in this specification, in order to enable GPU virtualization on a single physical machine to simultaneously support virtual machines and containers, divides the GPU on that physical machine into two parts, one for use by a virtual machine and the other by a container. Both the virtual machine GPU driver module and the container GPU driver module deployed on the edge node support binding and unbinding PCI devices, which lays the foundation for this solution to segment different GPU resources at the underlying level. By enabling the corresponding modules and calling the virtual machine GPU driver module and the container GPU driver module to bind different PCI devices to different drivers, different modules can manage different GPUs. Specifically, the edge node includes a container GPU driver module and a virtual machine GPU driver module;

[0209] The edge node is also configured to receive GPU packet information sent to the local GPU, wherein the GPU packet information includes container GPU packet information and virtual machine GPU packet information;

[0210] Based on the container GPU grouping information and the virtual machine GPU grouping information, the container GPU to be allocated and the virtual machine GPU to be allocated are identified from the local GPU, wherein the local GPU is bound to the virtual machine GPU driver module;

[0211] The virtual machine GPU to be allocated is identified as a virtual machine GPU group, and the binding between the container GPU to be allocated and the virtual machine GPU driver module is removed;

[0212] The container GPU to be assigned is bound to the container GPU driver module to obtain a container GPU group.

[0213] The GPU grouping information can be understood as information indicating whether the GPUs of the edge nodes are assigned to container GPU groups and virtual machine GPU groups, respectively. For example, this GPU grouping information includes the identifiers of the GPU devices assigned to the container GPU group and the identifiers of the GPU devices assigned to the virtual machine GPU group. In one embodiment provided in this specification, the GPU grouping information can be a GPU grouping strategy. For example, randomly dividing the GPUs in the edge nodes into two groups, or dividing the GPUs into two groups according to their name order, etc.

[0214] Container GPU grouping information can be understood as the identifiers of GPUs assigned to container GPU groups within an edge node. Alternatively, it can be understood as the number of GPUs, with the edge node randomly assigning a specified number of GPU devices to container GPU groups based on this number. Virtual machine GPU grouping information can also be understood as the identifiers of GPUs assigned to virtual machine GPU groups within an edge node. Alternatively, it can be understood as the number of GPUs, with the edge node randomly assigning a specified number of GPU devices to virtual machine GPU groups based on this number.

[0215] The following section uses the GPU resource management module provided in this manual as an example to illustrate the grouping of GPU devices based on edge node grouping information. Specifically, after enabling the virtual machine GPU driver (which can be understood as the aforementioned virtual machine GPU driver module), and writing the corresponding configuration file based on the vendorID and deviceID of the PCI device, the virtual machine GPU driver will register the corresponding PCI device as a device of the virtual machine GPU driver based on the PCI device ID. During the process of grouping GPUs based on GPU grouping information, it is necessary to identify which GPUs in the local GPU are assigned to the container GPU group and which are assigned to the virtual machine GPU group based on the container GPU grouping information and the virtual machine GPU grouping information. After the container GPU driver module is installed and running, it will detect PCI devices and load the corresponding components to manage the PCI devices assigned to the container GPU group. If a device is bound to the virtual machine GPU driver, then that device cannot be managed by the container GPU driver module at the same time. In this case, if it is necessary to specify that a specific device (i.e., the GPU device recorded in the container GPU grouping information that needs to be assigned to the container GPU group) be bound to the container GPU driver module, the PCI device can be unbound through the user-space function provided by the virtual machine GPU driver. At this time, the device is in an unclaimed state. A function can be used to bind the container GPU driver (which can be understood as the container GPU driver module mentioned above) to the corresponding PCI device based on the PCI device ID. Through this dynamic binding and unbinding configuration of PCI devices, it is possible to have PCI devices managed by both the virtual machine GPU driver and the container GPU driver simultaneously within a single system.

[0216] For the specific principles of the virtual machine GPU driver module and the container GPU driver module, please refer to [link / reference needed]. Figure 9 , Figure 9This diagram illustrates the interaction between a GPU resource management system module and hardware, as provided in one embodiment of this specification. The virtual machine GPU driver module employs a technique that directly allocates physical devices to the virtual machine. With virtualization capabilities enabled on the hardware, the virtual machine GPU driver module can expose device I / O, DMA, and other capabilities to user space, allowing users to directly access their bound devices within the virtual machine. The virtual machine GPU driver module provides a unified access interface (the interface of the virtual machine GPU driver module) to user space through device files, and its interaction with the physical device is shown in the diagram. The interface of the virtual machine GPU driver module encapsulates the interaction between the memory management unit driver and the memory management unit component, and interacts through the PCI bus driver and the PCI bus component. The former exposes DMA and other operations to user space, while the latter provides device configuration simulation and other functions to user space. Users can bind the corresponding PCI device, in this case, the GPU, by calling the interface of the virtual machine GPU driver. The virtual machine GPU driver will map the bound device to device identifier files such as dev0 and dev1. Before binding other devices, the virtual machine GPU driver needs to unbind the devices from other drivers. The device management of the virtual machine GPU driver module involves the concepts of Container, Group, and Device. One container corresponds to multiple groups, and one group corresponds to multiple devices (GPU devices). The entire container can be viewed as a physical device domain, the partitioning of which depends on the hardware's IOMMU topology.

[0217] Following the container GPU driver module, it will automatically bind to the PCI device (GPU). The CUDA driver can be invoked in user space, and through the CUDA driver, the container GPU driver module can be called to ultimately utilize the GPU's capabilities.

[0218] In one embodiment provided in this specification, the edge node is further configured to receive a container management module to be installed and a virtual machine management module to be installed sent by the cloud-side node.

[0219] Install the container management module and the virtual machine management module to be installed;

[0220] Identify the container GPU driver module corresponding to the container management module, and bind the container management module to the container GPU driver module;

[0221] Identify the virtual machine GPU driver module corresponding to the virtual machine management module, and bind the virtual machine management module to the virtual machine GPU driver module.

[0222] Following the previous example, the co-pooling scheduling scheme for container and cloud-native virtual machine GPUs is implemented based on the Kubernetes device plugin mechanism. Unified scheduling of GPU resources for virtual machines and containers is achieved using Kubernetes, with the allocation and release of node GPU resources handled through the kubeletdevice plugin mechanism on edge nodes. Based on this, edge nodes can receive virtual machine device plugins and container device plugins sent by cloud-side nodes and install them locally. These two types of device plugins can only recognize the corresponding GPU drivers. The virtual machine device plugin corresponds to the PCI bus driver in the virtual machine GPU driver module and manages virtual machine GPU resources based on the PCI bus driver; the container device plugin corresponds to the container GPU driver module and manages container GPU resources based on the container GPU driver module. This facilitates the subsequent creation and management of virtual machines and containers on the same edge node using the two device plugins.

[0223] In one embodiment provided in this specification, to facilitate cloud-side nodes in monitoring the usage of GPU resources in edge nodes, the GPU resource management module in the edge node reports the GPU resource usage to the cloud-side node, enabling the cloud-side node to manage the edge node based on its GPU resource usage. Specifically, the edge node is further configured to determine idle container GPU resource information through the container management module and the container GPU driver module.

[0224] The virtual machine management module and the virtual machine GPU driver module are used to determine the information on idle virtual machine GPU resources;

[0225] The GPU resource management module sends the idle container GPU resource information and the idle virtual machine GPU resource information to the cloud-side node.

[0226] The GPU resource management module can be understood as a module deployed on the edge node that monitors the operational status of the edge node. For example, this GPU resource management module could be a Kubelet, a component that periodically receives new or modified Pod specifications from the cloud-side node and ensures that the Pod and its containers run under the expected specifications. This component also reports the operational status of the edge node to the cloud-side node.

[0227] Idle container GPU resource information can be understood as information indicating the current GPU resource usage of the container's GPU group, such as parameters and values ​​identifying the available GPU resources for the container. Idle virtual machine GPU resource information can be understood as information indicating the current GPU resource usage of the virtual machine's GPU group, such as parameters and values ​​identifying the available GPU resources for the virtual machine.

[0228] Continuing with the previous example, kubelet identifies the GPU resources of edge nodes based on the device plugin mechanism. Kubelet reports these GPU resources to the cloud (cloud-side nodes), where the cloud-side nodes can then see the available GPU resources for virtual machines and containers in the reported node resources. This can be used for resource scheduling of cloud containers and virtual machines.

[0229] In one embodiment provided in this specification, the edge node can invoke the virtual machine GPU driver through the virtual machine device plugin to identify the current usage of virtual machine GPU resources, thereby facilitating subsequent management of the GPU device based on these virtual machine GPU resources. Specifically, the virtual machine GPU driver module is also configured to receive virtual machine GPU resource query requests sent by the virtual machine management module.

[0230] Based on the virtual machine GPU resource query request, identify the idle virtual machine GPUs in the virtual machine GPU group and obtain the idle virtual machine GPU resource information corresponding to the virtual machine GPU group;

[0231] The idle virtual machine GPU resource information is sent to the virtual machine management module;

[0232] The virtual machine management module is configured to provide the idle virtual machine GPU resource information to the GPU resource management module.

[0233] Specifically, the virtual machine management module sends a virtual machine GPU resource query request to the virtual machine GPU driver module, which then queries the usage status of the virtual machine GPUs. Based on the query request, the virtual machine GPU driver module identifies the number of idle virtual machine GPUs in the virtual machine GPU group and uses this number as the idle virtual machine GPU resource information corresponding to the virtual machine GPU group. Then, it sends this idle virtual machine GPU resource information to the virtual machine management module. After obtaining the idle virtual machine GPU resource information, the virtual machine management module provides it to the GPU resource management module, which then reports it to the cloud-side node.

[0234] In one embodiment provided in this specification, the edge node can invoke the container GPU driver through the container device plugin to identify the current usage of container GPU resources, thereby facilitating subsequent management of the GPU device based on these container GPU resources. Specifically, the container GPU driver module is further configured to receive container GPU resource query requests sent by the container management module;

[0235] Based on the container GPU resource query request, identify the running status information of the container shared GPUs contained in the container GPU group;

[0236] Based on the running status information, determine the idle container GPU resource information of the container GPU group, and send the idle container GPU resource information to the container management module;

[0237] The container management module is configured to provide the idle container GPU resource information to the GPU resource management module.

[0238] Continuing with the previous example, the container device plugin's eGPU encapsulates a memory control component on top of the CUDA call library to achieve memory control. It also implements a memory usage query component to check GPU memory usage. When the container device plugin needs to query the container's GPU memory usage, it calls this memory usage query component to query the current container's GPU resource usage through the NVML library. This information is then reported to the kubelet component.

[0239] In one embodiment provided in this specification, to ensure the successful creation of containers or virtual machines, the cloud-side node selects a superior edge node (i.e., an edge node with sufficient idle GPU resources to support the container or virtual machine to be created) based on the usage of the container GPU and virtual machine GPU on the edge node, and sends a container or virtual machine creation request to that edge node. Specifically, the cloud-side node is also configured to receive processing requests sent by users;

[0240] If it is determined from the object identifier that the container is responding to the processing request, a container configuration file is generated based on the object identifier and the GPU resource requirement information; and

[0241] If it is determined that the processing request is to be responded to by a virtual machine based on the object identifier, a virtual machine configuration file is generated based on the object identifier and the GPU resource requirement information, and the virtual machine configuration file is converted into a containerized virtual machine configuration file that is recognized by the cloud-side node and the edge node.

[0242] It should be noted that the descriptions of generating the container configuration file and the containerized virtual machine configuration file can be found in the corresponding explanations in the above embodiments, and will not be elaborated further here.

[0243] After determining the container configuration file and the containerized virtual machine configuration file, and in the case that the container is to respond to the processing request based on the object identifier, the target edge node is determined from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information, and the container configuration file is sent to the target edge node.

[0244] If it is determined that the processing request is to be responded to by a virtual machine based on the object identifier, the target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, and the containerized virtual machine configuration file is sent to the target edge node.

[0245] Following the previous example, traditional solutions use only one device plugin for each device to achieve resource awareness, management, and allocation. The GPU resource management system provided in this specification implements shared pool management of GPU resources through dual-plugin technology. As described in the above embodiments, driver isolation technology is used to partition the GPU resources of a single edge node for virtual machines and containers. The two plugins manage their corresponding GPU resources through dual drivers, and use different resource identifiers to ensure that the use of GPU resources by virtual machines and containers does not conflict.

[0246] Based on this Figure 10 This is a schematic diagram illustrating a GPU device dual-plugin technology and container virtual machine co-pool scheduling process in a GPU resource management system according to an embodiment of this specification. The process involves containers and virtual machines sharing a scheduling mechanism, as detailed below. Figure 10 As shown. This manual addresses virtual machine management based on cloud-native virtualization. When a user creates a virtual machine, a virtual machine template is issued. The kubevirt component in the cloud node converts the virtual machine template into a virtual machine Pod template that is recognized by the Kubernetes cluster. The GPU resource requirements specified in the virtual machine template are converted into the resource requirements of the virtual machine Pod template.

[0247] When a user creates a container, the GPU resource requirement is added to the distributed Pod template by default. This means that GPU resource requests from containers and virtual machines are ultimately translated into scheduling for Pod GPU extension resources recognized by Kubernetes. These resources have different identifiers, and the scheduling is ultimately based on the Kubernetes scheduler. Because containers and virtual machines have different resource identifiers, the GPU allocation between containers and virtual machines at the physical machine level is not confused at the scheduling level. It's worth noting that the scheduler adds a shared GPU scheduling extension to the native Kubernetes scheduler for container GPU sharing, enabling container GPU sharing scheduling. Virtual machine scheduling, however, is based on the native scheduler and is not limited by the shared GPU scheduling extension; the two coexist peacefully.

[0248] Among them, for the dual-plugin technology of GPU devices, based on Figure 10 As can be seen, the virtual machine device plugin manages and allocates GPU resources to the virtual machine through the virtual machine GPU driver, reports the GPU resources to be used by the virtual machine to the node information, and can view the total amount of GPU resources and the number of GPU resources that can be allocated to the virtual machine.

[0249] The container device plugin manages and allocates container GPU resources through the container GPU driver, and reports the GPU resources to be used by the container to the node information. Users can view the total amount of GPU resources and the number of allocated GPU resources for the container. This approach achieves unified management of GPU resources for both containers and virtual machines.

[0250] In one embodiment provided in this specification, the edge node can invoke the container GPU driver through the container device plugin to allocate GPU resources from the shared container GPU, thereby supporting the normal operation of the container. Specifically, the container management module is further configured to send a container GPU allocation instruction to the container GPU driver module, wherein the container GPU allocation instruction includes the GPU resource requirement information;

[0251] The container GPU driver module is configured to determine free container GPU resources from the container shared GPUs included in the container GPU group in response to the container GPU allocation instruction.

[0252] Allocate container GPU resources from the idle container GPU resources that correspond to the GPU resource requirement information.

[0253] Continuing with the previous example, the container device plugin allocates computing power through a computing power allocation component. After requesting GPU resources in the Pod's YAML file, the computing power allocation component, during the scheduling phase, determines whether a single GPU card in the container GPU group on the edge node can provide sufficient GPU memory, and allocates idle GPU resources from the currently available GPU resources to match the GPU resources required by the user to create the container. Finally, during the allocation phase, the GPU allocation result is recorded on the pod so that the container device plugin can subsequently request GPU memory for the container according to the allocation result.

[0254] In the embodiments provided in this specification, the edge node invokes the virtual machine GPU driver through the virtual machine device plugin to select an idle GPU as the virtual machine GPU resource, thereby supporting the normal operation of the virtual machine. Specifically, the virtual machine management module is further configured to send a virtual machine GPU allocation instruction to the virtual machine GPU driver module, wherein the virtual machine GPU allocation instruction includes the GPU resource requirement information;

[0255] The virtual GPU driver module is configured to determine the idle virtual machine GPUs in the virtual machine GPU group in response to the virtual machine GPU allocation instruction.

[0256] The idle virtual machine GPU is used as the virtual machine GPU resource corresponding to the GPU resource requirement information.

[0257] Continuing with the previous example, the virtual machine device plugin sends a virtual machine GPU allocation instruction to the virtual machine GPU driver. In response to the virtual machine GPU allocation instruction, the virtual machine GPU driver selects an idle virtual machine GPU device from the current virtual machine GPU group as the virtual machine GPU resource corresponding to the GPU resource requirement information.

[0258] In one embodiment provided in this specification, the edge node binds containers and virtual machines to corresponding GPU resources, thereby solving the problem of containers and virtual machines sharing GPU scheduling on a single physical machine. Specifically, the edge node is further configured to create a container based on an object configuration file and bind the container's GPU resources to the container when the container or virtual machine is determined to be a container based on the object identifier.

[0259] Specifically, creating a container based on an object configuration file can be understood as obtaining a container image identifier from the container configuration file and obtaining a container image corresponding to the container image identifier from the container image library; running the container image to obtain a container that responds to the processing request.

[0260] If the container or virtual machine is determined to be a virtual machine based on the object identifier, a containerized virtual machine is created based on the object configuration file, and the virtual machine's GPU resources are bound to the containerized virtual machine.

[0261] Specifically, creating a containerized virtual machine based on an object configuration file can be understood as obtaining a virtual machine image identifier from the containerized virtual machine configuration file and retrieving the virtual machine image corresponding to the virtual machine image identifier from a container image repository; running the virtual machine image according to the virtual machine configuration parameters to obtain a containerized virtual machine that responds to the processing request.

[0262] The process of creating the container and containerized virtual machine can be found in the corresponding or relevant content of the above embodiments. Further details will not be elaborated upon here.

[0263] Continuing with the previous example, once the edge node recognizes that a virtual machine has been scheduled to the current node, it starts the virtual machine using the virtual machine pod template. Then, it allocates the corresponding virtual machine GPU resources through the virtual machine device plugin and mounts the CPU device (i.e., the virtual machine GPU resource) to the virtual machine.

[0264] When an edge node detects that a container has been scheduled to the current node, it starts the container using a pod template and allocates the corresponding container GPU resources through the container device plugin. The GPU device (i.e., the GPU device corresponding to the container's GPU resources) is then mounted to the container, and the container's use of the container's GPU resources is restricted based on GPU resource requests.

[0265] In one embodiment provided in this specification, the solution employs an eGPU shared scheduling scheme, which enables multiple containers to share GPU scheduling based on the virtual machine GPU passthrough scheme. However, to ensure the normal execution of shared GPU scheduling by multiple containers and avoid failures caused by call errors, it is necessary to manage the containers' use of GPU resources. Specifically, the edge node is further configured to determine the current usage status of the container's GPU resources based on the container management module when the container performs a call operation on the container's GPU resources.

[0266] Based on the current usage status, the container's calls to the container's GPU resources are managed.

[0267] Continuing with the previous example, when a user program calls the CUDA API, it first enters the memory control component. When needed, the memory control component triggers a query action to communicate with the memory usage query component. The memory usage query component uses the NVML library to query the current container's GPU resource usage. The memory control component then determines whether the relevant call operation was successful based on the returned query results.

[0268] As can be seen from the above embodiments, the GPU resource management system provided in one embodiment of this specification, by deploying a container management module and a virtual machine management module at the edge node, enables the edge node to allocate container GPU resources corresponding to the GPU resource requirements from the container GPU group based on the container management module when the cloud-side node sends object identifiers and GPU resource requirement information to the edge node, and create a container responding to the processing request based on the container GPU resources; or to allocate virtual machine GPU resources corresponding to the GPU resource requirements from the virtual machine GPU group based on the virtual machine GPU module, and create a virtual machine responding to the processing request based on the virtual machine GPU resources. This achieves the creation of virtual machines and container virtualization on the same edge node, avoiding the problem that software programs cannot be modified for virtual machine or container virtualization, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0269] The following is in conjunction with the appendix Figure 11 Taking the GPU resource management system provided in this specification as an example in a GPU resource scheduling scenario, the GPU resource management system will be further explained. Figure 11 This document illustrates a flowchart of a GPU resource management system based on an embodiment of this specification. The GPU resource management system employs a shared-pool scheduling scheme for container and cloud-native virtual machine GPUs. It utilizes the Kubernetes device plugin mechanism to achieve unified scheduling of GPU resources for virtual machines and containers, and uses an edge node device plugin mechanism to allocate and release GPU resources on nodes. Specifically, it includes the following steps.

[0270] Step 1102: At the edge node, according to the GPU grouping requirements issued by the cloud control plane (cloud node), the physical machine GPU resources are pre-grouped and drivers are configured as needed. For example, GPU-1 and GPU-2 are used for virtual machines, and virtual machine GPU drivers are configured; GPU-3 and GPU-4 are used for containers, and container GPU drivers are configured.

[0271] Step 1104: On the edge node, install the virtual machine device plugin and manage the virtual machine GPU resources based on the virtual machine driver. Also, install the container device plugin and manage the container GPU resources based on the container GPU driver.

[0272] Step 1106: Edge nodes, identify edge node GPU resources using kubelet based on the d device plugin.

[0273] Step 1108: Edge nodes report GPU resources to the cloud control plane via kubelet. At this point, cloud-side nodes can obtain the GPU resources available for virtual machines and containers, which can be used for resource scheduling of cloud containers and virtual machines.

[0274] Step 1110: The cloud control plane receives the user's processing request, analyzes the processing request, and obtains the GPU resource requirements for creating containers or virtual machines.

[0275] The cloud control plane can be understood as a cloud-side node; users can declare GPU requirements when processing requests.

[0276] It should be noted that the cloud control plane also deploys Kubernetes management components and cloud-native virtualization control plane components. These components are used for unified management of the edge node pool.

[0277] Step 1112: Cloud Control Plane. Use the cloud control plane scheduler to schedule containers or virtual machines to edge nodes that meet the GPU resources declared by the user.

[0278] Step 1114: The edge node recognizes that the virtual machine has been scheduled to the current node, starts the virtual machine, allocates the corresponding GPU resources through the virtual machine device plugin, and mounts the device to the virtual machine.

[0279] Step 1116: The edge node recognizes that the container has been scheduled to the current node, starts the container, allocates the corresponding GPU resources through the container device plugin, mounts the GPU to the container, and restricts the container's use of GPU resources according to the GPU resource request.

[0280] Based on this, the cloud-edge collaborative solution for GPU resource management systems in this specification, which supports co-pooling scheduling of virtual machines and container GPUs, is fundamentally based on this. Specifically, on a physical server equipped with multiple GPU cards, some physical GPU cards are managed by virtual machines, while others are managed by containers. Its main features are: 1. Unlike traditional virtualization solutions, the solution provided in this specification is based on cloud-native virtualization, enabling containers and virtual machines to share Kubernetes scheduling, network, storage, and computing resources. The Kubernetes device plugin mechanism enables unified reporting, scheduling, and management of GPU resources on physical server nodes, which is the core foundation for co-pooling scheduling of container and virtual machine GPUs. 2. This solution isolates the GPUs used by containers and virtual machines through kernel drivers. Container GPU device plugins can detect container GPU drivers, while virtual machine device plugins can only detect virtual machine GPU drivers, thus achieving isolation between the GPU cards used by containers and virtual machines. Based on this, containers and virtual machines can schedule, allocate, and manage GPU cards based on their respective device plugins. This solution employs an eGPU shared scheduling scheme, which, based on the virtual machine GPU passthrough scheme, enables multiple containers to share GPU scheduling, increasing container deployment density and thus improving container GPU utilization. This further enhances the practical significance of GPU co-pool scheduling. It overcomes the limitations in GPU virtualization reuse support, contributing to the further popularization and promotion of cloud-native technologies. Facing the heterogeneous cloud environment brought about by the digital transformation and upgrading of the energy industry, it effectively balances virtual machine and container virtualization, meeting the latest customer needs.

[0281] Corresponding to the above method embodiments, this specification also provides an embodiment of a GPU resource management device applied to a cloud-side node, including:

[0282] The request receiving module is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests.

[0283] The scheduling module is configured to, upon determining, based on the object identifier that a container will respond to the processing request, determine a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group, and according to the GPU resource requirement information.

[0284] If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs;

[0285] The information sending module is configured to send the object identifier and the GPU resource requirement information to the target edge node.

[0286] Optionally, the scheduling module is further configured to:

[0287] Determine the available container GPU resource information for each edge node in the edge node group;

[0288] Based on the idle container GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group;

[0289] Each idle edge node is evaluated to obtain a first evaluation result for each idle edge node;

[0290] Based on the first evaluation result, the target edge node is determined from the at least two idle edge nodes.

[0291] Optionally, the scheduling module is further configured to:

[0292] Determine the idle virtual machine GPU resource information for each edge node in the edge node group;

[0293] Based on the idle virtual machine GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group;

[0294] Each idle edge node is evaluated to obtain a second evaluation result for each idle edge node;

[0295] Based on the second evaluation result, the target edge node is determined from the at least two idle edge nodes.

[0296] Optionally, the GPU resource management device further includes an information update module, configured to:

[0297] Receive idle container GPU resource information and idle virtual machine GPU resource information sent by each edge node;

[0298] The historical container GPU resource information is updated based on the idle container GPU resource information, and the historical virtual machine GPU resource information is updated based on the idle virtual machine GPU resource information.

[0299] Optionally, the information sending module is further configured to:

[0300] An object configuration file is generated based on the object identifier and the GPU resource requirement information, and the object configuration file is sent to the target edge node.

[0301] Optionally, the information sending module is further configured to:

[0302] If it is determined from the object identifier that the container is responding to the processing request, a container configuration file is generated based on the object identifier and the GPU resource requirement information; and

[0303] If it is determined that the processing request is to be responded to by a virtual machine based on the object identifier, a virtual machine configuration file is generated based on the object identifier and the GPU resource requirement information, and the virtual machine configuration file is converted into a containerized virtual machine configuration file that is recognized by the cloud-side node and the edge node.

[0304] Optionally, the GPU resource management device further includes an information receiving module, configured to:

[0305] Receive and record the GPU device group information sent by each edge node, wherein the GPU device group information includes GPU device information corresponding to container GPU groups and GPU device information corresponding to virtual machine GPU groups.

[0306] This specification provides an embodiment of a GPU resource management device applied to a cloud-side node. Upon receiving a processing request from a user, it can select a target edge node from the edge node group for creating a container based on the idle container GPU resource information provided by each edge node; and select a target edge node from the edge node group for creating a virtual machine based on the idle virtual machine GPU resource information provided by each edge node. The target edge node for creating the container and the target edge node for creating the virtual machine can be the same node. By sending the object identifier and GPU resource requirement information—information used to create the virtual machine and container—to the target edge node, the target edge node can complete the creation of the virtual machine and container. This enables the creation of virtual machines and containers on the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0307] The above is an illustrative scheme of a GPU resource management device according to this embodiment. It should be noted that the technical solution of this GPU resource management device and the technical solution of the GPU resource management method described above belong to the same concept. For details not described in detail in the technical solution of the GPU resource management device, please refer to the description of the technical solution of the GPU resource management method described above.

[0308] Corresponding to the above method embodiments, this specification also provides a GPU resource management node, including:

[0309] The request receiving module is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests.

[0310] The scheduling module is configured to, upon determining, based on the object identifier that a container will respond to the processing request, determine a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group, and according to the GPU resource requirement information.

[0311] If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs;

[0312] The information sending module is configured to send the object identifier and the GPU resource requirement information to the target edge node.

[0313] Optionally, the scheduling module is further configured to:

[0314] Determine the available container GPU resource information for each edge node in the edge node group;

[0315] Based on the idle container GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group;

[0316] Each idle edge node is evaluated to obtain a first evaluation result for each idle edge node;

[0317] Based on the first evaluation result, the target edge node is determined from the at least two idle edge nodes.

[0318] Optionally, the scheduling module is further configured to:

[0319] Determine the idle virtual machine GPU resource information for each edge node in the edge node group;

[0320] Based on the idle virtual machine GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group;

[0321] Each idle edge node is evaluated to obtain a second evaluation result for each idle edge node;

[0322] Based on the second evaluation result, the target edge node is determined from the at least two idle edge nodes.

[0323] Optionally, the GPU resource management node further includes an information update module, configured to:

[0324] Receive idle container GPU resource information and idle virtual machine GPU resource information sent by each edge node;

[0325] The historical container GPU resource information is updated based on the idle container GPU resource information, and the historical virtual machine GPU resource information is updated based on the idle virtual machine GPU resource information.

[0326] Optionally, the information sending module is further configured to:

[0327] An object configuration file is generated based on the object identifier and the GPU resource requirement information, and the object configuration file is sent to the target edge node.

[0328] Optionally, the information sending module is further configured to:

[0329] If it is determined from the object identifier that the container is responding to the processing request, a container configuration file is generated based on the object identifier and the GPU resource requirement information; and

[0330] If it is determined that the processing request is to be responded to by a virtual machine based on the object identifier, a virtual machine configuration file is generated based on the object identifier and the GPU resource requirement information, and the virtual machine configuration file is converted into a containerized virtual machine configuration file that is recognized by the cloud-side node and the edge node.

[0331] Optionally, the GPU resource management node further includes an information receiving module, configured to:

[0332] Receive and record the GPU device group information sent by each edge node, wherein the GPU device group information includes GPU device information corresponding to container GPU groups and GPU device information corresponding to virtual machine GPU groups.

[0333] This specification provides an embodiment of a GPU resource management node that, upon receiving a processing request from a user, can select a target edge node from the edge node group for creating a container based on the idle container GPU resource information provided by each edge node; and select a target edge node for creating a virtual machine based on the idle virtual machine GPU resource information provided by each edge node. The target edge node for creating the container and the target edge node for creating the virtual machine can be the same node. By sending the object identifier and GPU resource requirement information—information used to create the virtual machine and container—to the target edge node, the target edge node can complete the creation of the virtual machine and container. This achieves the creation of virtual machines and containers on the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0334] The above is an illustrative scheme of a GPU resource management node according to this embodiment. It should be noted that the technical solution of this GPU resource management node and the technical solution of the GPU resource management method described above belong to the same concept. For details not described in detail in the technical solution of the GPU resource management node, please refer to the description of the technical solution of the GPU resource management method described above.

[0335] Corresponding to the above method embodiments, this specification also provides another embodiment of a GPU resource management device applied to an edge node, including:

[0336] The information receiving module is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request;

[0337] The container management module is configured to, when it is determined from the object identifier that a container is responding to the processing request, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources, wherein the container runs in the container management unit;

[0338] The virtual machine management module is configured to, when it is determined from the object identifier that a virtual machine is responding to the processing request, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in the container management unit.

[0339] Optionally, the other GPU resource management device further includes a GPU grouping module, configured as follows:

[0340] Receive GPU grouping information for the local GPU, wherein the GPU grouping information includes container GPU grouping information and virtual machine GPU grouping information;

[0341] Based on the container GPU grouping information and the virtual machine GPU grouping information, the container GPUs to be allocated and the virtual machine GPUs to be allocated are identified from the local GPUs.

[0342] Configure the virtual machine GPUs to be allocated as virtual machine GPU groups, and configure the container GPUs to be allocated as container GPU groups;

[0343] The GPU device information corresponding to the virtual machine GPU group and the GPU device information corresponding to the container GPU group are sent to the cloud-side node as GPU device group information.

[0344] Optionally, the other GPU resource management device further includes an information sending module, configured to:

[0345] Determine the idle container GPU resource information corresponding to the container GPU group, and determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group;

[0346] The idle container GPU resource information and the idle virtual machine GPU resource information are sent to the cloud-side node.

[0347] Optionally, the information sending module is further configured to:

[0348] Identify the runtime status information of the container-shared GPUs contained in the container GPU group;

[0349] Based on the running status information, the idle container GPU resource information of the container GPU group is determined.

[0350] Optionally, the information sending module is further configured to:

[0351] Identify the idle virtual machine GPUs in the virtual machine GPU group;

[0352] Based on the idle virtual machine GPUs, determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group.

[0353] Optionally, the information receiving module is further configured to:

[0354] The cloud-side node receives an object configuration file, wherein the object configuration file is generated by the cloud-side node through analysis of the processing request sent by the user and the object identifier and GPU resource requirement information.

[0355] Optionally, the object configuration file is a container configuration file;

[0356] Accordingly, the container management module is also configured as follows:

[0357] Obtain the container image identifier from the container configuration file, and obtain the container image corresponding to the container image identifier from the container image repository;

[0358] Run the container image to obtain a container that responds to the processing request, and bind the container's GPU resources to the container.

[0359] Optionally, the object configuration file is a containerized virtual machine configuration file;

[0360] Accordingly, the virtual machine management module is also configured as follows:

[0361] Obtain the virtual machine image identifier from the containerized virtual machine configuration file, and obtain the virtual machine image corresponding to the virtual machine image identifier from the container image repository;

[0362] Run the virtual machine image according to the virtual machine configuration parameters to obtain a containerized virtual machine that responds to the processing request, and bind the virtual machine's GPU resources to the containerized virtual machine.

[0363] Optionally, the container management module is further configured to:

[0364] Idle container GPU resources are determined from the container shared GPUs contained in the container GPU group;

[0365] Allocate container GPU resources from the idle container GPU resources that correspond to the GPU resource requirement information.

[0366] Optionally, the virtual machine management module is further configured to:

[0367] Identify the idle virtual machine GPUs in the virtual machine GPU group;

[0368] The idle virtual machine GPU is identified as the virtual machine GPU resource corresponding to the GPU resource requirement information.

[0369] Optionally, the other GPU resource management device further includes a scheduling control module, configured to:

[0370] When the container performs a call operation on the container's GPU resources, determine the current usage status of the container's GPU resources;

[0371] Based on the current usage status, the container's calls to the container's GPU resources are managed.

[0372] Another GPU resource management device provided in one embodiment of this specification is applied to an edge node. When the cloud-side node sends object identifiers and GPU resource requirement information to the edge node, the edge node can allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources; or allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources. This enables the creation of virtual machines and containers within the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0373] The above is an illustrative scheme of another GPU resource management device in this embodiment. It should be noted that the technical solution of this other GPU resource management device and the technical solution of the other GPU resource management method described above belong to the same concept. For details not described in detail in the technical solution of the other GPU resource management device, please refer to the description of the technical solution of the other GPU resource management method described above.

[0374] Corresponding to the above method embodiments, this specification also provides another GPU resource management node, including:

[0375] The information receiving module is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request;

[0376] The container management module is configured to, when it is determined from the object identifier that a container is responding to the processing request, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources, wherein the container runs in the container management unit;

[0377] The virtual machine management module is configured to, when it is determined from the object identifier that a virtual machine is responding to the processing request, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in the container management unit.

[0378] Optionally, the other GPU resource management node further includes a GPU grouping module, configured as follows:

[0379] Receive GPU grouping information for the local GPU, wherein the GPU grouping information includes container GPU grouping information and virtual machine GPU grouping information;

[0380] Based on the container GPU grouping information and the virtual machine GPU grouping information, the container GPUs to be allocated and the virtual machine GPUs to be allocated are identified from the local GPUs.

[0381] Configure the virtual machine GPUs to be allocated as virtual machine GPU groups, and configure the container GPUs to be allocated as container GPU groups;

[0382] The GPU device information corresponding to the virtual machine GPU group and the GPU device information corresponding to the container GPU group are sent to the cloud-side node as GPU device group information.

[0383] Optionally, the other GPU resource management node further includes an information sending module, configured to:

[0384] Determine the idle container GPU resource information corresponding to the container GPU group, and determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group;

[0385] The idle container GPU resource information and the idle virtual machine GPU resource information are sent to the cloud-side node.

[0386] Optionally, the information sending module is further configured to:

[0387] Identify the runtime status information of the container-shared GPUs contained in the container GPU group;

[0388] Based on the running status information, the idle container GPU resource information of the container GPU group is determined.

[0389] Optionally, the information sending module is further configured to:

[0390] Identify the idle virtual machine GPUs in the virtual machine GPU group;

[0391] Based on the idle virtual machine GPUs, determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group.

[0392] Optionally, the information receiving module is further configured to:

[0393] The cloud-side node receives an object configuration file, wherein the object configuration file is generated by the cloud-side node through analysis of the processing request sent by the user and the object identifier and GPU resource requirement information.

[0394] Optionally, the object configuration file is a container configuration file;

[0395] Accordingly, the container management module is also configured as follows:

[0396] Obtain the container image identifier from the container configuration file, and obtain the container image corresponding to the container image identifier from the container image repository;

[0397] Run the container image to obtain a container that responds to the processing request, and bind the container's GPU resources to the container.

[0398] Optionally, the object configuration file is a containerized virtual machine configuration file;

[0399] Accordingly, the virtual machine management module is also configured as follows:

[0400] Obtain the virtual machine image identifier from the containerized virtual machine configuration file, and obtain the virtual machine image corresponding to the virtual machine image identifier from the container image repository;

[0401] Run the virtual machine image according to the virtual machine configuration parameters to obtain a containerized virtual machine that responds to the processing request, and bind the virtual machine's GPU resources to the containerized virtual machine.

[0402] Optionally, the container management module is further configured to:

[0403] Idle container GPU resources are determined from the container shared GPUs contained in the container GPU group;

[0404] Allocate container GPU resources from the idle container GPU resources that correspond to the GPU resource requirement information.

[0405] Optionally, the virtual machine management module is further configured to:

[0406] Identify the idle virtual machine GPUs in the virtual machine GPU group;

[0407] The idle virtual machine GPU is identified as the virtual machine GPU resource corresponding to the GPU resource requirement information.

[0408] Optionally, the other GPU resource management node further includes a scheduling control module, configured to:

[0409] When the container performs a call operation on the container's GPU resources, determine the current usage status of the container's GPU resources;

[0410] Based on the current usage status, the container's calls to the container's GPU resources are managed.

[0411] This specification provides an alternative GPU resource management node according to one embodiment. When a cloud-side node sends object identifiers and GPU resource requirement information to an edge node, the edge node can allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources; or allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources. This enables the creation of virtual machines and containers within the same edge node, avoiding the problem of software programs being unable to perform virtual machine or container virtualization modifications, meeting customer needs, and contributing to the further popularization and promotion of cloud-native technologies.

[0412] The above is an illustrative scheme of another GPU resource management node in this embodiment. It should be noted that the technical solution of this other GPU resource management node belongs to the same concept as the technical solution of the other GPU resource management method described above. For details not described in detail in the technical solution of the other GPU resource management node, please refer to the description of the technical solution of the other GPU resource management method described above.

[0413] Figure 12 A structural block diagram of a computing device 1200 according to an embodiment of this specification is shown. The components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and a database 1250 is used to store data.

[0414] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0415] In one embodiment of this specification, the aforementioned components of the computing device 1200 and Figure 12 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 12 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0416] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1200 can also be a mobile or stationary server.

[0417] The processor 1220 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the GPU resource management method described above.

[0418] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the GPU resource management method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the GPU resource management method described above.

[0419] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the GPU resource management method described above.

[0420] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the GPU resource management method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the GPU resource management method described above.

[0421] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the GPU resource management method described above.

[0422] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the aforementioned GPU resource management method belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the aforementioned GPU resource management method's technical solution.

[0423] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0424] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0425] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0426] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0427] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A GPU resource management method, applied to cloud-side nodes, comprising: Receive a processing request sent by a user, analyze the processing request, and determine the object identifier and GPU resource requirement information corresponding to the processing request; If it is determined that a container is responding to the processing request based on the object identifier, the target edge node is determined from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information. If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs; The object identifier and the GPU resource requirement information are sent to the target edge node.

2. The GPU resource management method according to claim 1, wherein determining the target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource demand information includes: Determine the available container GPU resource information for each edge node in the edge node group; Based on the idle container GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group; Each idle edge node is evaluated to obtain a first evaluation result for each idle edge node; Based on the first evaluation result, the target edge node is determined from the at least two idle edge nodes.

3. The GPU resource management method according to claim 1, wherein determining the target edge node from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource demand information includes: Determine the idle virtual machine GPU resource information for each edge node in the edge node group; Based on the idle virtual machine GPU resource information, at least two idle edge nodes that meet the GPU resource requirement information are determined from the edge node group; Each idle edge node is evaluated to obtain a second evaluation result for each idle edge node; Based on the second evaluation result, the target edge node is determined from the at least two idle edge nodes.

4. The GPU resource management method according to claim 1, before determining the target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource demand information, further includes: Receive idle container GPU resource information and idle virtual machine GPU resource information sent by each edge node; The historical container GPU resource information is updated based on the idle container GPU resource information, and the historical virtual machine GPU resource information is updated based on the idle virtual machine GPU resource information.

5. The GPU resource management method according to claim 1, wherein sending the object identifier and the GPU resource requirement information to the target edge node comprises: An object configuration file is generated based on the object identifier and the GPU resource requirement information, and the object configuration file is sent to the target edge node.

6. The GPU resource management method according to claim 5, wherein generating an object configuration file based on the object identifier and the GPU resource requirement information includes: If it is determined that the processing request is to be responded to by a container based on the object identifier, a container configuration file is generated based on the object identifier and the GPU resource requirement information. as well as If it is determined that the processing request is to be responded to by a virtual machine based on the object identifier, a virtual machine configuration file is generated based on the object identifier and the GPU resource requirement information, and the virtual machine configuration file is converted into a containerized virtual machine configuration file that is recognized by the cloud-side node and the edge node.

7. The GPU resource management method according to claim 1, before determining the target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource demand information, further includes: Receive and record the GPU device group information sent by each edge node, wherein the GPU device group information includes GPU device information corresponding to container GPU groups and GPU device information corresponding to virtual machine GPU groups.

8. A GPU resource management method, applied to edge nodes, comprising: The cloud-side node receives object identifiers and GPU resource requirement information, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request. If it is determined from the object identifier that a container is responding to the processing request, container GPU resources corresponding to the GPU resource requirement information are allocated from the container GPU group, and a container responding to the processing request is created based on the container GPU resources. If it is determined from the object identifier that a virtual machine is responding to the processing request, virtual machine GPU resources corresponding to the GPU resource requirement information are allocated from the virtual machine GPU group, and a virtual machine responding to the processing request is created based on the virtual machine GPU resources.

9. The GPU resource management method according to claim 8, further comprising, before receiving the object identifier and GPU resource requirement information sent by the cloud-side node: Receive GPU grouping information for the local GPU, wherein the GPU grouping information includes container GPU grouping information and virtual machine GPU grouping information; Based on the container GPU grouping information and the virtual machine GPU grouping information, the container GPUs to be allocated and the virtual machine GPUs to be allocated are identified from the local GPUs. Configure the virtual machine GPUs to be allocated as virtual machine GPU groups, and configure the container GPUs to be allocated as container GPU groups; The GPU device information corresponding to the virtual machine GPU group and the GPU device information corresponding to the container GPU group are sent to the cloud-side node as GPU device group information.

10. The GPU resource management method according to claim 9, further comprising, after configuring the virtual machine GPU to be allocated as a virtual machine GPU group and configuring the container GPU to be allocated as a container GPU group: Determine the idle container GPU resource information corresponding to the container GPU group, and determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group; The idle container GPU resource information and the idle virtual machine GPU resource information are sent to the cloud-side node.

11. The GPU resource management method according to claim 10, wherein determining the idle container GPU resource information corresponding to the container GPU group includes: Identify the runtime status information of the container-shared GPUs contained in the container GPU group; Based on the running status information, the idle container GPU resource information of the container GPU group is determined.

12. The GPU resource management method according to claim 10, wherein determining the idle virtual machine GPU resource information corresponding to the virtual machine GPU group includes: Identify the idle virtual machine GPUs in the virtual machine GPU group; Based on the idle virtual machine GPUs, determine the idle virtual machine GPU resource information corresponding to the virtual machine GPU group.

13. The GPU resource management method according to claim 8, wherein receiving the object identifier and GPU resource requirement information sent by the cloud-side node includes: The cloud-side node receives an object configuration file, wherein the object configuration file is generated by the cloud-side node through analysis of the processing request sent by the user and the object identifier and GPU resource requirement information.

14. The GPU resource management method according to claim 13, wherein the object configuration file is a container configuration file; Accordingly, creating a container to respond to the processing request based on the container GPU resources includes: Obtain the container image identifier from the container configuration file, and obtain the container image corresponding to the container image identifier from the container image repository; Run the container image to obtain a container that responds to the processing request, and bind the container's GPU resources to the container.

15. The GPU resource management method according to claim 13, wherein the object configuration file is a containerized virtual machine configuration file; Accordingly, creating a virtual machine to respond to the processing request based on the virtual machine GPU resources includes: Obtain the virtual machine image identifier from the containerized virtual machine configuration file, and obtain the virtual machine image corresponding to the virtual machine image identifier from the container image repository; Run the virtual machine image according to the virtual machine configuration parameters to obtain a containerized virtual machine that responds to the processing request, and bind the virtual machine's GPU resources to the containerized virtual machine.

16. The GPU resource management method according to claim 8, wherein allocating container GPU resources corresponding to the GPU resource requirement information from the container GPU group comprises: Idle container GPU resources are determined from the container shared GPUs contained in the container GPU group; Allocate container GPU resources from the idle container GPU resources that correspond to the GPU resource requirement information.

17. The GPU resource management method according to claim 8, wherein allocating virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group comprises: Identify the idle virtual machine GPUs in the virtual machine GPU group; The idle virtual machine GPU is identified as the virtual machine GPU resource corresponding to the GPU resource requirement information.

18. The GPU resource management method according to claim 8, further comprising, after creating a container responding to the processing request based on the container GPU resource: When the container performs a call operation on the container's GPU resources, determine the current usage status of the container's GPU resources; Based on the current usage status, the container's calls to the container's GPU resources are managed.

19. A GPU resource management node, comprising: The request receiving module is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests. The scheduling module is configured to, upon determining, based on the object identifier that a container will respond to the processing request, determine a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group, and according to the GPU resource requirement information. If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs; The information sending module is configured to send the object identifier and the GPU resource requirement information to the target edge node.

20. A GPU resource management node, comprising: The information receiving module is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request; The container management module is configured to, when it is determined from the object identifier that a container is responding to the processing request, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources, wherein the container runs in the container management unit; The virtual machine management module is configured to, when determining that a virtual machine is responding to the processing request based on the object identifier, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in the container management unit.

21. A GPU resource management system, the system comprising a cloud-side node and at least two edge nodes, each edge node being configured with at least two GPUs, wherein, The cloud-side node is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests. If, based on the object identifier, it is determined that the processing request is being responded to by a container, a target edge node is determined from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group and the GPU resource requirement information; and if, based on the object identifier, it is determined that the processing request is being responded to by a virtual machine, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes, and each edge node is configured with at least two GPUs; the object identifier and the GPU resource requirement information are sent to the target edge node; The edge node is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of processing requests; if it is determined based on the object identifier that a container will respond to the processing request, the edge node allocates container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and creates a container responding to the processing request based on the container GPU resources, wherein the container runs in a container management unit; and if it is determined based on the object identifier that a virtual machine will respond to the processing request, the edge node allocates virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and creates a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in a container management unit.

22. A GPU resource management device, applied to a cloud-side node, comprising: The request receiving module is configured to receive processing requests sent by users, analyze the processing requests, and determine the object identifier and GPU resource requirement information corresponding to the processing requests. The scheduling module is configured to, upon determining, based on the object identifier that a container will respond to the processing request, determine a target edge node from the edge node cluster based on the idle container GPU resource information of each edge node in the edge node group, and according to the GPU resource requirement information. If it is determined that the processing request is responded to by a virtual machine based on the object identifier, a target edge node is determined from the edge node group based on the idle virtual machine GPU resource information of each edge node and the GPU resource requirement information, wherein the edge node group contains at least two edge nodes and each edge node is configured with at least two GPUs; The information sending module is configured to send the object identifier and the GPU resource requirement information to the target edge node.

23. A GPU resource management device, applied to an edge node, comprising: The information receiving module is configured to receive object identifiers and GPU resource requirement information sent by the cloud-side node, wherein the object identifiers and GPU resource requirement information are obtained by the cloud-side node through analysis of the processing request; The container management module is configured to, when it is determined from the object identifier that a container is responding to the processing request, allocate container GPU resources corresponding to the GPU resource requirement information from the container GPU group, and create a container responding to the processing request based on the container GPU resources, wherein the container runs in the container management unit; The virtual machine management module is configured to, when determining that a virtual machine is responding to the processing request based on the object identifier, allocate virtual machine GPU resources corresponding to the GPU resource requirement information from the virtual machine GPU group, and create a virtual machine responding to the processing request based on the virtual machine GPU resources, wherein the virtual machine runs in the container management unit.

24. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the GPU resource management method according to any one of claims 1 to 7 and the GPU resource management method according to any one of claims 8 to 18.

25. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the GPU resource management method of any one of claims 1 to 7 and the steps of the GPU resource management method of any one of claims 8 to 18.

Images