GPU operation method, device, electronic device and medium of edge node

Abstract

The present application provides a GPU operation method, device, electronic device and medium for edge nodes, the method comprising: obtaining task processing information of a GPU task, the task processing information comprising a task identifier, the task identifier being used to indicate the task type of the GPU task; creating a Pod corresponding to the task type indicated by the task identifier; searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier; scheduling the created Pod to the target edge node so that the target edge node executes the GPU task. Through the present application, corresponding scheduling methods can be adopted for GPU tasks of different task types, effectively improving the GPU utilization efficiency of edge nodes.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to a GPU operation method, device, electronic device and medium for an edge node. Background Art

[0002] Graphics Processing Unit (GPU) has powerful computing capabilities and is widely used in the fields of deep learning and high-performance computing.

[0003] Kubernetes is an open source system for managing cross-host container applications, providing basic mechanisms for application deployment, maintenance, and expansion and reduction. When scheduling tasks for various types of GPUs under Kubernetes, the scheduling granularity is allocated to tasks on demand based on the number of GPU cards. Summary of the invention

[0004] In view of this, the embodiments of the present application at least provide a GPU operation method, device, electronic device and medium for an edge node, which can adopt corresponding scheduling methods for GPU tasks of different task types, effectively improving the GPU utilization efficiency of the edge node.

[0005] In a first aspect, an exemplary embodiment of the present application provides a GPU operation method for an edge node, comprising: obtaining task processing information of a GPU task, the task processing information comprising a task identifier, the task identifier being used to indicate a task type of the GPU task; creating a Pod corresponding to the task type indicated by the task identifier; searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier; and scheduling the created Pod to the target edge node so that the target edge node executes the GPU task.

[0006] In a possible implementation manner, the task type includes a GPU exclusive task, and the edge node search strategy corresponding to the GPU exclusive task is to search for edge nodes based on the device requirement of the GPU exclusive task for the GPU.

[0007] In one possible implementation, the step of searching for the target edge node according to the edge node search strategy indicated by the task requirement information includes: determining the operating status of each edge node in the node pool, wherein the operating status includes a working status and an idle status; and determining the edge node that meets the device requirements of the GPU exclusive task for the GPU searched from the edge nodes in the idle status as the target edge node.

[0008] In a possible implementation manner, the task type includes a GPU sharing task group, and the edge node search strategy corresponding to the GPU sharing task group is to search for edge nodes based on an occupancy identifier preset for the GPU sharing task group.

[0009] In one possible implementation, the task processing information also includes an occupancy identifier pre-set for a GPU shared task group, the GPU shared task group including multiple shared subtasks, wherein a target edge node is determined for each shared subtask in the following manner: searching from each edge node in a node pool to see whether there is an edge node with an occupancy identifier corresponding to the GPU shared task group; if so, determining the searched edge node as the target edge node for executing the shared subtask; if not, selecting an edge node from edge nodes in the node pool that do not execute GPU exclusive tasks, and determining the selected edge node as the target edge node for executing the shared subtask.

[0010] In a possible implementation, the method further includes: adding an occupancy flag to an edge node selected from edge nodes that do not execute GPU exclusive tasks; and clearing the occupancy flag added to the target edge node after the target edge node completes all shared subtasks in the GPU shared task group.

[0011] In one possible implementation, the step of scheduling the created Pod to the target edge node so that the target edge node executes the GPU task includes: when starting the Pod, calling the GPU device plug-in to obtain device environment variables, and the device environment variables are obtained by the GPU device plug-in in the following manner: obtaining the device requirements of the GPU task for the GPU, selecting a GPU from the target edge node according to the device requirements, and generating device environment variables based on the device information of the selected GPU; injecting the obtained device environment variables into the Pod, and starting the container to execute the GPU task.

[0012] In a possible implementation, the method further includes: when starting the container, according to the device information declared in the device environment variable, mapping the GPU indicated by the device information into the container, and also mapping the driver library of the GPU indicated by the device information into the container.

[0013] In one possible implementation, it also includes: performing node registration on each edge node in the node pool, wherein node registration is performed for each edge node in the following manner: building an information viewing tool on the edge node; calling the information viewing tool through an edge component on the edge node to obtain GPU status information according to a preset information specification; calling the edge component through a GPU device plug-in on the edge node to complete node registration based on the obtained GPU status information.

[0014] In one possible implementation, an information viewing tool is constructed on each edge node in the following manner: an initial information viewing tool is adapted according to a node environment of the edge node and a GPU type of a GPU arranged on the edge node, so that the adapted initial information viewing tool can obtain GPU status information from a GPU of the GPU type under the node environment; and the adapted initial information viewing tool is determined as the information viewing tool.

[0015] In a second aspect, an exemplary embodiment of the present application provides a GPU operating device for an edge node, comprising: an acquisition module for acquiring task processing information of a GPU task, the task processing information comprising a task identifier, and the task identifier is used to indicate the task type of the GPU task; a creation module for creating a Pod corresponding to the task type indicated by the task identifier; a search module for searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier; and a scheduling module for scheduling the created Pod to the target edge node so that the target edge node executes the GPU task.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the GPU operation method of the edge node in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the GPU operation method of the edge node in the above-mentioned first aspect or any possible implementation manner of the first aspect are executed.

[0018] The GPU operation method, device, electronic device and storage medium of the edge node provided in the embodiments of the present application can formulate different edge node search strategies for GPU tasks of different task types to implement the scheduling of different types of GPU tasks, effectively improving resource utilization and GPU resource scheduling efficiency.

[0019] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart showing a GPU operation method of an edge node provided by an exemplary embodiment of the present application is shown;

[0022] Figure 2 A flowchart showing the steps of determining a target edge node for a GPU exclusive task provided by an exemplary embodiment of the present application;

[0023] Figure 3 A flowchart showing the steps of executing a GPU task by a target edge node provided by an exemplary embodiment of the present application;

[0024] Figure 4 A flowchart showing the steps of determining a target edge node for a GPU sharing task group provided by an exemplary embodiment of the present application;

[0025] Figure 5 A flowchart showing the steps of performing GPU node registration provided by an exemplary embodiment of the present application is shown;

[0026] Figure 6 A schematic diagram showing the structure of a GPU operating device of an edge node provided by an exemplary embodiment of the present application is shown;

[0027] Figure 7 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0029] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0030] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0031] It should be understood that in the embodiments of the present application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0032] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0033] Currently, when Kubernetes performs GPU node registration and / or GPU task scheduling, it only supports node registration and task scheduling for a single type of GPU. For example, only Nvidia GPUs are supported under Kubernetes, and the scheduling granularity is allocated to tasks on demand based on the number of GPU cards.

[0034] The following example shows how Kubernetes schedules Nvidia GPUs:

[0035] GPU-Device-Plugin is deployed on the GPU node and reports the GPU information and corresponding DeviceID of the registered node.

[0036] When a GPU Pod with the declaration nvidia.com / gpu is created, the scheduler will consider the number of GPU cards and schedule the Pod to a node with sufficient GPU devices.

[0037] When the kubelet on the node starts the Pod, it calls the Allocate interface of each GPU-Device-Plugin according to the declaration in the request. Since the container declares the GPU, the kubelet selects the appropriate device based on the Device information received by the ListAndWatch interface before, uses the DeviceID as a parameter, and calls the Allocate interface of the GPU-Device-Plugin.

[0038] GPU-Device-Plugin receives the call, converts the DeviceID to the NVIDIA_VISIBLE_DEVICES environment variable, and returns it to kubelet.

[0039] The kubelet injects the environment variables into the Pod and starts the container.

[0040] The above solution only supports Nvidia GPUs under Kubernetes. The scheduling granularity is allocated to tasks on demand according to the number of GPU cards. That is, Kubernetes can only schedule GPUs under a fixed architecture. However, in edge computing scenarios, the node environment / GPU types are very rich. The above solution does not further abstract the task characteristics in edge computing scenarios and cannot support the scheduling of different node environments and multiple types of GPUs.

[0041] In response to the above problems, an embodiment of the present application provides a GPU operation method, device, electronic device and storage medium for an edge node to achieve at least one of the following aspects: scheduling of GPU tasks for different task types and registration of GPU nodes for different node environments / GPU types.

[0042] To facilitate understanding of the present application, the GPU operation method, device, electronic device and storage medium of the edge node provided in the embodiments of the present application are introduced in detail below.

[0043] See also Figure 1 , which is a flow chart of a GPU operation method of an edge node provided by an exemplary embodiment of the present application, specifically comprising:

[0044] S110: Obtain task processing information of GPU tasks;

[0045] S120: Create a Pod corresponding to the task type indicated by the task identifier;

[0046] S130: searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier;

[0047] S140: Schedule the created Pod to the target edge node so that the target edge node executes the GPU task.

[0048] First, the names involved in the embodiments of the present application are briefly introduced.

[0049] Container technology is based on namespace, cgroup, and UnionFS underlying technologies to encapsulate and isolate processes. It is a virtualization technology at the operating system level. Container technology is lighter and faster than virtual machine technology. Docker is the most mainstream container engine. Combined with the design of Dockerfile, the container fully demonstrates standardized cross-platform capabilities.

[0050] Kubernetes is an open source distributed container orchestration management platform that aims to reduce the complexity of computing, network and storage resource orchestration, allowing operators and developers to focus on container-based applications. Kubernetes has comprehensive cluster management capabilities, including rapid deployment, maintenance and rapid expansion and contraction of applications, cross-host scheduling, security protection and access mechanisms, multi-tenant resource isolation, transparent service registration and service discovery mechanisms, load balancing, fault detection and repair capabilities, and multi-dimensional resource quota management capabilities.

[0051] Pod is the smallest deployable unit that can create and manage Kubernetes computing. It consists of one or more containers and is also the basic granularity of Kubernetes scheduling.

[0052] Node, the working node of the Kubernetes cluster, can be a physical host or a virtual machine (VM) that hosts the operation of the Pod instance.

[0053] KubeEdge is an open source system that extends containerized application orchestration capabilities to Edge nodes. It is an open intelligent edge computing platform based on Kubernetes extensions that provides cloud-edge collaboration capabilities. It is the first official CNCF project in the field of intelligent edge. KubeEdge implements cloud-edge collaboration, computing sinking, massive edge device management, edge autonomy and other capabilities on top of Kubernetes' native container orchestration and scheduling capabilities. With KubeEdge, complex machine learning, image recognition, event processing and other advanced applications can be deployed to the Edge.

[0054] Jetson TX2 is an embedded module released by Nvidia for edge device AI applications. Jetson TX2 adopts NVIDIA PascalNV architecture and is equipped with two Denver 2 64-bit cores and four ARM A57 core processors. The Pascal architecture GPU with 256 CUDA cores can bring 1.33TFLOPs of computing power, which is suitable for intelligent cutting-edge devices.

[0055] The following takes the application of the above method to an edge node as an example to illustrate the above exemplary steps provided in the embodiment of the present application.

[0056] In an exemplary embodiment of the present application, the above-mentioned GPU operation method can be executed in an edge node. Here, the edge node may refer to a node with computing capability, data storage capability, and GPU computing capability in an edge scenario.

[0057] Exemplarily, edge nodes may include, but are not limited to, nodes in the following application scenarios: robots (such as robots suitable for home use, security, and public services), drones, smart cameras, and portable medical devices.

[0058] Here, the edge node is different from the non-edge scenario. Usually, the edge node has only one graphics card. The GPU exclusive task and multi-task sharing GPU under the single-card edge node are the core issues. However, it should be understood that the GPU operation method of this application is not limited to single-card edge nodes, but also applicable to multi-card edge nodes.

[0059] In step S110 , the task processing information includes a task identifier, which is used to indicate the task type of the GPU task. Exemplarily, the task type may include but is not limited to: an exclusive task and a shared task group.

[0060] For example, when an edge node has a GPU task processing requirement, the GPU-Task-Manager on the edge node will create a GPU task and add a task identifier to the created GPU task, and the task identifier is written into the task processing information of the created GPU task.

[0061] In one case, the GPU task is a GPU exclusive task.

[0062] In this case, the GPU-Task-Manager creates a GPU exclusive task and adds a task identifier indicating that the task type of the GPU task is an exclusive task, for example, gpu-type=exclusive is added to the label.

[0063] In another case, the GPU task is a GPU shared task group.

[0064] In this case, the GPU-Task-Manager creates a GPU shared task group and adds a task identifier indicating that the task type of the GPU task is a shared task group, such as adding gpu-type=share to the label.

[0065] In step S120, a corresponding Pod is created based on the task type of the GPU task.

[0066] In step S130, the edge node search strategies adopted for GPU tasks of different task types are also different.

[0067] In the embodiment of the present application, by adopting different edge node search strategies for GPU tasks of each task type, the GPU utilization efficiency of the edge nodes can be effectively improved.

[0068] In case that the GPU task is a GPU exclusive task, the edge node search strategy corresponding to the GPU exclusive task is to search for edge nodes based on the device requirements of the GPU exclusive task for the GPU.

[0069] Exemplarily, the device requirement of the GPU exclusive task for the GPU may include the number of GPUs, but in addition, may also include the size of the video memory, the GPU usage rate, and / or the video memory usage rate.

[0070] For example, when a Pod with gpu-type=exclusive is created, the scheduler of the edge node will comprehensively consider the idle status of the GPUs of each edge node in the node pool and schedule the Pod to an edge node with sufficient GPUs.

[0071] Refer to the following Figure 2 Here we introduce the process of determining the target edge node according to the edge node search strategy corresponding to the GPU exclusive task.

[0072] Figure 2 A flowchart showing the steps of determining a target edge node for a GPU-exclusive task provided by an exemplary embodiment of the present application.

[0073] Reference Figure 2 In step S131, the operating state of each edge node in the node pool is determined. Here, the operating state includes a working state and an idle state.

[0074] As an example, the node pool may include only edge nodes with GPU task processing requirements, or may include edge nodes with GPU task processing requirements and other edge nodes connected to the edge nodes.

[0075] In step S132, an edge node that meets the device requirement of the GPU exclusive task for the GPU, which is searched from the edge nodes in the idle state, is determined as a target edge node.

[0076] For example, all edge nodes in the node pool are traversed, and the edge nodes in the idle state are first selected, and then the GPU device information of each edge node in the idle state is obtained, such as the number of GPUs, the size of the video memory, the GPU usage rate, and / or the video memory usage rate. Based on the obtained GPU device information, an edge node that meets the device requirements of the GPU exclusive task for the GPU is selected from each edge node in the idle state and determined as the target edge node. If the edge node that meets the device requirements is not unique, an edge node can be randomly selected to be determined as the target edge node.

[0077] Through the above task allocation method, the GPU exclusive task can be completed independently by an edge node, avoiding interference with the execution of the GPU exclusive task.

[0078] In the case where the GPU task is a GPU shared task group, the edge node search strategy corresponding to the GPU shared task group is to search for edge nodes based on an occupancy identifier preset for the GPU shared task group.

[0079] In this case, the task processing information of the GPU task may include, in addition to the task identifier, an occupancy identifier pre-set for the GPU shared task group. Here, the GPU shared task group may include multiple shared subtasks, and the occupancy identifier is set for the GPU shared task group, that is, all shared subtasks under the GPU shared task group have the same occupancy identifier.

[0080] Exemplarily, GPU-Task-Manager creates a GPU shared task group, and adds a task identifier and an occupation identifier indicating that the task type of the GPU task is a shared task group, such as adding gpu-type=share and gpu-share-id={id} to the label.

[0081] For example, when a Pod with gpu-type=share is created, the scheduler will ignore the gpu request information and search for the target edge node based on the gpu-share-id specified in the Pod. That is, for GPU shared task groups, the scheduler determines the target edge node based on the occupancy identifier, while for GPU exclusive tasks, the scheduler determines the target edge node based on the device requirements indicated by the gpu request.

[0082] Refer to the following Figure 4Let's introduce the process of determining the target edge node for each shared subtask in the GPU shared task group.

[0083] Figure 4 A flowchart showing the steps of determining a target edge node for a GPU sharing task group provided by an exemplary embodiment of the present application.

[0084] Reference Figure 4 In step S133, it is searched from the edge nodes in the node pool whether there is an edge node with an occupation identifier corresponding to the GPU sharing task group.

[0085] As described above, the node pool may include only edge nodes with GPU task processing requirements, or may include edge nodes with GPU task processing requirements and other edge nodes connected to the edge nodes.

[0086] Here, when an edge node is assigned a shared subtask in a GPU shared task group, an occupancy flag is added to the edge node to indicate that there is a shared subtask under the GPU shared task group indicated by the occupancy flag in the edge node, and the occupancy flags corresponding to different GPU shared task groups are different.

[0087] If there is an edge node with an occupation identifier corresponding to the GPU shared task group in the node pool, step S134 is executed: the searched edge node is determined as a target edge node for executing the shared subtask.

[0088] Here, if there is an edge node with gpu-share-id, it indicates that the shared task group already exists in the cluster, and the scheduler selects the edge node with gpu-share-id for scheduling.

[0089] Through the above process, the shared task group will only be scheduled to one edge node to share the GPU capability of the edge node.

[0090] Here, the multiple shared subtasks deployed on an edge node have at least one of the following characteristics: the tasks are transparent, there is interaction between tasks, they are developed by the same person, and there is little interference with each other. Based on the above characteristics of the shared subtasks, in this application, the shared task group is scheduled to an edge node, that is, all shared subtasks in the same shared task group are scheduled to the same edge node for execution, so as to improve the GPU utilization efficiency of the edge node.

[0091] If there is no edge node with an occupancy identifier corresponding to the GPU shared task group in the node pool, execute step S135: select an edge node from the edge nodes in the node pool that do not execute GPU exclusive tasks, and determine the selected edge node as the target edge node for executing the shared subtask.

[0092] Through the above process, it is possible to prevent the exclusive task and the shared task group from coexisting, that is, neither the exclusive task nor the shared task group will be executed on one edge node.

[0093] In addition, after step S135, an occupation flag is added to the edge node selected from the edge nodes that do not execute the GPU exclusive task.

[0094] For example, if there is no edge node with gpu-share-id, it means that the shared task group does not exist in the cluster currently. The scheduler selects edge nodes from nodes without exclusive tasks for scheduling, and then GPU-Task-Manager adds gpu-share-id to the node label.

[0095] In addition, the GPU operation method of the present application further includes: after the target edge node completes executing all shared subtasks in the GPU shared task group, clearing the occupancy flag added for the target edge node.

[0096] For example, if the edge node executing the GPU sharing task group does not contain a GPU task, the GPU-Task-Manager will clear the gpu-share-id of the edge node.

[0097] In step S140, if the GPU task is a GPU exclusive task, the target edge node can use Figure 3 The GPU task is executed in the manner shown.

[0098] Figure 3 A flowchart showing the steps for a target edge node to execute a GPU task provided by an exemplary embodiment of the present application.

[0099] Reference Figure 3 , in step S141, when starting the Pod, the GPU device plug-in is called to obtain the device environment variables.

[0100] As an example, the device environment variables can be obtained by the GPU device plug-in in the following way: obtain the device requirements of the GPU for the GPU exclusive task, select a GPU from the target edge node according to the device requirements, generate the device environment variables based on the device information of the selected GPU, and convert the device environment variables into the corresponding library file to execute the subsequent container startup process based on the library file.

[0101] For example, when edged on the target edge node starts a Pod, it calls the GPU-Device-Plugin (GPU device plug-in), selects the appropriate GPU based on the gpu request value, and converts it to the NVIDIA_VISIBLE_DEVICES device environment variable and returns it.

[0102] Here, the gpu request value includes the device requirements of the GPU exclusive task for the GPU. At this time, the corresponding GPU is selected from the target edge node according to the device requirements. For example, if the device requirements indicate that 3 GPUs are required for the GPU exclusive task, the corresponding number of GPUs can be selected from the target edge node. It should be understood that the above is only an example, and the present application is not limited thereto.

[0103] In step S142, the obtained device environment variables are injected into the Pod, and the container is started to execute the GPU task.

[0104] For example, edged of the target edge node injects the obtained device environment variables into the Pod and starts the container.

[0105] Exemplarily, when starting the container, according to the device information declared in the device environment variable, the GPU indicated by the device information is mapped into the container, and the driver library of the GPU indicated by the device information is also mapped into the container.

[0106] For example, when the container is started, gpu-container-runtime maps the device to the container based on the GPU device information declared in the container's NVIDIA_VISIBLE_DEVICES, and also maps the corresponding Nvidia Driver Lib (assuming the GPU is Nvidia GPU) to the container.

[0107] In the case where the GPU task is a GPU shared task group, the target edge node can execute the GPU task in the following manner.

[0108] Specifically, when starting the Pod, the GPU device plug-in is called to obtain the device environment variables, the obtained device environment variables are injected into the Pod, and the container is started. When starting the container, according to the device information declared in the device environment variables, the GPU indicated by the device information is mapped to the container, and the driver library of the GPU indicated by the device information is also mapped to the container.

[0109] Here, the device environment variables can be obtained by the GPU device plug-in in the following manner: obtaining the device requirements of the GPU sharing task for the GPU, selecting a GPU from the target edge node according to the device requirements, and generating the device environment variables based on the device information of the selected GPU.

[0110] For example, edged of the edge node receives the Pod, injects the device environment variable NVIDIA_VISIBLE_DEVICES=all into the Pod, and starts the container.

[0111] When the container is started, gpu-container-runtime maps the device to the container based on the GPU device information declared in the container's NVIDIA_VISIBLE_DEVICES, and also maps the corresponding Nvidia Driver Lib to the container.

[0112] In the GPU operation method of the edge node of the exemplary embodiment of the present application, in addition to scheduling different types of GPU tasks, it also includes: performing node registration on each edge node in the node pool.

[0113] Refer to the following Figure 5 Let's introduce the process of node registration for each edge node.

[0114] Figure 5 A flowchart showing the steps of performing GPU node registration provided by an exemplary embodiment of the present application is shown.

[0115] Reference Figure 5 , in step S150, an information viewing tool is constructed on the edge node.

[0116] For example, an information viewing tool can be constructed on each edge node in the following manner: according to the node environment of the edge node and the GPU type of the GPU arranged on the edge node, the initial information viewing tool is adapted so that the adapted initial information viewing tool can obtain GPU status information from the GPU of the GPU type in the node environment; and the adapted initial information viewing tool is determined as the information viewing tool.

[0117] Here, the node environment may refer to the operating environment of the GPU on the edge node, for example, the operating system loaded on the edge node. The GPU type may include but is not limited to: the GPU manufacturer, and / or the GPU access method (independent GPU, integrated GPU).

[0118] Through the above construction process, the information viewing tool is adapted to the loaded operating system and to the basic tools provided by different GPU manufacturers. That is, the modeling is standardized through the adaptation of the development tool so that the information viewing tool can be applicable to different node environments and different GPU types.

[0119] In an embodiment of the present application, the GPU status information of the edge node is pre-standardized. Exemplarily, the GPU status information under the preset information specification may include but is not limited to at least one of the following items: GPU type, number of GPUs, video memory size, GPU usage, and video memory usage.

[0120] Exemplarily, the above-mentioned information viewing tool may refer to the viewing tool gpu-stat under Kubernetes, and each edge node constructs a viewing tool gpu-stat that conforms to the above-mentioned specification for GPU status information.

[0121] In step S160, an information viewing tool is called through an edge component on an edge node to obtain GPU status information according to a preset information specification.

[0122] Here, each edge node runs GPU-Device-Plugin (GPU device plugin for Kubernetes) and edged (a component of Kubernetes), and edged obtains GPU status information by calling gpu-stat. Among them, GPU-Device-Plugin is adapted to the framework to obtain GPU status information defined by the preset information specification. Exemplarily, the above preset information specification is a pre-defined various information required to be obtained to characterize the GPU status.

[0123] In step S170 , the edge component is called through the GPU device plug-in on the edge node to complete the node registration based on the acquired GPU status information.

[0124] For example, GPU-Device-Plugin (Kubernetes' device plugin mechanism, compatible with KubeEdge) calls edged to obtain GPU status information, and then completes GPU node registration with KubeEdge.

[0125] The GPU node registration step in the above steps standardizes the edge node registration process. You only need to implement the GPU status information viewing tool gpu-stat of the corresponding edge node to complete the registration of the KubeEdge edge GPU node, avoiding the access problem of different node environments / GPU types. However, the existing technology requires a large number of Jetson nodes to connect to KubeEdge separately, which is very time-consuming.

[0126] According to the exemplary embodiments of the present application, the GPU task scheduling framework under the KubeEdge framework in the edge computing scenario is mainly solved, which can support GPU Pod scheduling of different edge nodes and different GPU types, and solve the scheduling capabilities such as GPU exclusive tasks and multi-task shared GPU under the same edge node.

[0127] The actual business scenarios of edge nodes (AI edge nodes, edge nodes with GPU processing capabilities) often have only one graphics card (such as Jetson nodes). A single node may run GPU-exclusive tasks or GPU-shared tasks. This is because the tasks running on the same edge node are often related and do not have the high requirements for GPU isolation in non-edge scenarios. Based on this, the task scheduling steps in the above GPU operation method abstract GPU-exclusive tasks and GPU-shared task groups. Based on this task model, the utilization efficiency of edge GPU nodes can be effectively improved.

[0128] Based on the same application concept, the embodiments of the present application also provide a GPU operating device for an edge node corresponding to the method provided in the above embodiments. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the GPU operating method for an edge node in the above embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0129] Figure 6 The schematic diagram of the structure of the GPU operation device of the edge node provided by the exemplary embodiment of the present application. Figure 6 As shown, the GPU operating device 600 includes:

[0130] The acquisition module 610 is used to acquire task processing information of the GPU task. Here, the task processing information includes a task identifier, and the task identifier is used to indicate the task type of the GPU task.

[0131] The creation module 620 is used to create a Pod corresponding to the task type indicated by the task identifier.

[0132] The search module 630 is configured to search for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier.

[0133] The scheduling module 640 is used to schedule the created Pod to the target edge node so that the target edge node executes the GPU task.

[0134] In one case, the GPU task is a GPU exclusive task.

[0135] In this case, the task type includes a GPU exclusive task, and the edge node search strategy corresponding to the GPU exclusive task is to search for edge nodes based on the device requirement of the GPU exclusive task for the GPU.

[0136] In a possible implementation manner of the present application, the search module 630 may determine the target edge node according to the edge node search strategy corresponding to the GPU exclusive task in the following manner.

[0137] For example, the search module 630 determines the operating status of each edge node in the node pool, where the operating status includes a working status and an idle status; and determines the edge node that meets the device requirements of the GPU exclusive task for the GPU from the edge nodes in the idle status as the target edge node.

[0138] The scheduling module 640 executes GPU tasks in the following manner.

[0139] For example, when starting a Pod, the scheduling module 640 calls the GPU device component to obtain device environment variables. The device environment variables are obtained by the GPU device component in the following ways: obtaining the device requirements of the GPU exclusive task for the GPU, selecting a GPU from the target edge node according to the device requirements, and generating device environment variables based on the device information of the selected GPU; injecting the obtained device environment variables into the Pod, and starting the container to execute the GPU task.

[0140] In one possible implementation of the present application, when starting the container, the scheduling module 640 maps the GPU indicated by the device information to the container according to the device information declared in the device environment variable, and also maps the driver library of the GPU indicated by the device information to the container.

[0141] In another case, the GPU task is a GPU shared task group.

[0142] In this case, the task type includes a GPU sharing task group, and the edge node search strategy corresponding to the GPU sharing task group is to search for edge nodes based on an occupation identifier preset for the GPU sharing task group.

[0143] At this time, the task processing information may also include an occupancy identifier preset for the GPU shared task group, and the GPU shared task group includes a plurality of shared subtasks.

[0144] In a possible implementation manner of the present application, the search module 630 determines the target edge node for each shared subtask in the following manner.

[0145] For example, the search module 630 searches from the edge nodes of the node pool whether there is an edge node with an occupancy identifier corresponding to the GPU shared task group; if so, the searched edge node is determined as the target edge node for executing the shared subtask; if not, an edge node is selected from the edge nodes in the node pool that do not execute GPU exclusive tasks, and the selected edge node is determined as the target edge node for executing the shared subtask.

[0146] In one possible implementation of the present application, the scheduling module 640 may also be used to add an occupancy flag to an edge node selected from edge nodes that do not execute GPU exclusive tasks. In addition, after the target edge node completes all shared subtasks in the GPU shared task group, the occupancy flag added for the target edge node is cleared.

[0147] In a possible implementation manner of the present application, the GPU operating device 600 may further include a registration module (not shown in the figure) for performing node registration on each edge node in the node pool.

[0148] Exemplarily, the registration module can register nodes for each edge node in the following manner: construct an information viewing tool on the edge node; call the information viewing tool through the edge component on the edge node to obtain GPU status information according to a preset information specification; call the edge component through the GPU device component on the edge node to complete the node registration based on the obtained GPU status information.

[0149] In one possible implementation of the present application, the registration module can construct an information viewing tool on each edge node in the following manner: adapt the initial information viewing tool according to the node environment of the edge node and the GPU type of the GPU arranged on the edge node, so that the adapted initial information viewing tool can obtain GPU status information from the GPU type of the GPU in the node environment; determine the adapted initial information viewing tool as the information viewing tool.

[0150] According to the GPU operating device of the present application, fast access to GPUs of different node environments / GPU types can be achieved. In addition, the scheduling problem of GPU exclusive tasks and multi-task shared GPU under the same edge node is solved, effectively improving the utilization efficiency of edge GPU nodes.

[0151] See also Figure 7 , Figure 7 The schematic diagram of the structure of the electronic device provided by the exemplary embodiment of the present application. Figure 7 As shown, the electronic device 700 includes a processor 710 , a memory 720 and a bus 730 .

[0152] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 communicates with the memory 720 via the bus 730. When the machine-readable instructions are executed by the processor 710, the steps in the operation of the GPU of the edge node described above can be executed, which are as follows:

[0153] Obtaining task processing information of a GPU task, where the task processing information includes a task identifier, and the task identifier is used to indicate a task type of the GPU task;

[0154] Create a Pod corresponding to the task type indicated by the task identifier;

[0155] Searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier;

[0156] Schedule the created Pod to the target edge node so that the target edge node can execute the GPU task.

[0157] In one case, the GPU task is a GPU exclusive task.

[0158] In this case, the task type includes a GPU exclusive task, and the edge node search strategy corresponding to the GPU exclusive task is to search for edge nodes based on the device requirement of the GPU exclusive task for the GPU.

[0159] In a possible implementation of the present application, the processor 710 may perform the following processing to determine the target edge node according to the edge node search strategy corresponding to the GPU exclusive task.

[0160] For example, the operating status of each edge node in the node pool is determined, where the operating status includes a working status and an idle status; an edge node that meets the device requirements of the GPU exclusive task for the GPU, which is searched from the edge nodes in the idle status, is determined as the target edge node.

[0161] In a possible implementation of the present application, the processor 710 may perform the following processing to execute GPU tasks.

[0162] For example, when starting a Pod, the GPU device component is called to obtain device environment variables. The device environment variables are obtained by the GPU device component in the following ways: obtaining the device requirements of the GPU exclusive task for the GPU, selecting a GPU from the target edge node according to the device requirements, and generating device environment variables based on the device information of the selected GPU; injecting the obtained device environment variables into the Pod, and starting the container to execute the GPU task.

[0163] In a possible implementation of the present application, the processor 710 may also perform the following processing: according to the device information declared in the device environment variable, the GPU indicated by the device information is mapped into the container, and the driver library of the GPU indicated by the device information is also mapped into the container.

[0164] In another case, the GPU task is a GPU shared task group.

[0165] In this case, the task type includes a GPU sharing task group, and the edge node search strategy corresponding to the GPU sharing task group is to search for edge nodes based on an occupation identifier preset for the GPU sharing task group.

[0166] At this time, the task processing information may also include an occupancy identifier preset for the GPU shared task group, and the GPU shared task group includes a plurality of shared subtasks.

[0167] In a possible implementation manner of the present application, the processor 710 may further perform the following processing to determine a target edge node for each shared subtask.

[0168] For example, a search is performed from each edge node in the node pool to determine whether there is an edge node with an occupancy identifier corresponding to the GPU shared task group; if so, the searched edge node is determined as the target edge node for executing the shared subtask; if not, an edge node is selected from the edge nodes in the node pool that do not execute GPU exclusive tasks, and the selected edge node is determined as the target edge node for executing the shared subtask.

[0169] In one possible implementation of the present application, the processor 710 may also perform the following processing: adding an occupancy flag to an edge node selected from edge nodes that do not execute GPU exclusive tasks, and in addition, after the target edge node completes all shared subtasks in the GPU shared task group, clearing the occupancy flag added for the target edge node.

[0170] In a possible implementation manner of the present application, the processor 710 may further perform the following processing: registering each edge node in the node pool.

[0171] Exemplarily, node registration can be performed for each edge node in the following manner: building an information viewing tool on the edge node; calling the information viewing tool through the edge component on the edge node to obtain GPU status information according to a preset information specification; calling the edge component through the GPU device component on the edge node to complete node registration based on the obtained GPU status information.

[0172] In one possible implementation of the present application, the processor 710 may also perform the following processing to build an information viewing tool on each edge node: adapt the initial information viewing tool according to the node environment of the edge node and the GPU type of the GPU arranged on the edge node, so that the adapted initial information viewing tool can obtain GPU status information from the GPU type of the GPU in the node environment; determine the adapted initial information viewing tool as the information viewing tool.

[0173] According to the electronic device of the exemplary embodiment of the present application, it is possible to achieve rapid access to GPUs of different node environments / GPU types. In addition, it also solves the scheduling problem of GPU exclusive tasks and multi-task shared GPU under the same edge node, effectively improving the utilization efficiency of edge GPU nodes.

[0174] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the GPU operation method of the edge node can be executed, which are as follows:

[0175] Obtaining task processing information of a GPU task, where the task processing information includes a task identifier, and the task identifier is used to indicate a task type of the GPU task;

[0176] Create a Pod corresponding to the task type indicated by the task identifier;

[0177] Searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier;

[0178] Schedule the created Pod to the target edge node so that the target edge node can execute the GPU task.

[0179] In one case, the GPU task is a GPU exclusive task.

[0180] In this case, the task type includes a GPU exclusive task, and the edge node search strategy corresponding to the GPU exclusive task is to search for edge nodes based on the device requirement of the GPU exclusive task for the GPU.

[0181] In a possible implementation of the present application, the processor may perform the following processing to determine the step of the target edge node according to the edge node search strategy corresponding to the GPU exclusive task.

[0182] For example, the operating status of each edge node in the node pool is determined, where the operating status includes a working status and an idle status; an edge node that meets the device requirements of the GPU exclusive task for the GPU, which is searched from the edge nodes in the idle status, is determined as the target edge node.

[0183] In a possible implementation of the present application, the processor may perform the following processing to execute the GPU task.

[0184] For example, when starting a Pod, the GPU device component is called to obtain device environment variables. The device environment variables are obtained by the GPU device component in the following ways: obtaining the device requirements of the GPU exclusive task for the GPU, selecting a GPU from the target edge node according to the device requirements, and generating device environment variables based on the device information of the selected GPU; injecting the obtained device environment variables into the Pod, and starting the container to execute the GPU task.

[0185] In a possible implementation of the present application, the processor may also perform the following processing: according to the device information declared in the device environment variable, mapping the GPU indicated by the device information into the container, and also mapping the driver library of the GPU indicated by the device information into the container.

[0186] In another case, the GPU task is a GPU shared task group.

[0187] In this case, the task type includes a GPU sharing task group, and the edge node search strategy corresponding to the GPU sharing task group is to search for edge nodes based on an occupation identifier preset for the GPU sharing task group.

[0188] At this time, the task processing information may also include an occupancy identifier preset for the GPU shared task group, and the GPU shared task group includes a plurality of shared subtasks.

[0189] In a possible implementation manner of the present application, the processor may further perform the following processing to determine a target edge node for each shared subtask.

[0190] For example, a search is performed from each edge node in the node pool to determine whether there is an edge node with an occupancy identifier corresponding to the GPU shared task group; if so, the searched edge node is determined as the target edge node for executing the shared subtask; if not, an edge node is selected from the edge nodes in the node pool that do not execute GPU exclusive tasks, and the selected edge node is determined as the target edge node for executing the shared subtask.

[0191] In one possible implementation of the present application, the processor may also perform the following processing: adding an occupancy flag to an edge node selected from edge nodes that do not execute GPU exclusive tasks, and in addition, after the target edge node completes all shared subtasks in the GPU shared task group, clearing the occupancy flag added for the target edge node.

[0192] In a possible implementation manner of the present application, the processor may further perform the following processing: registering each edge node in the node pool.

[0193] Exemplarily, node registration can be performed for each edge node in the following manner: building an information viewing tool on the edge node; calling the information viewing tool through the edge component on the edge node to obtain GPU status information according to a preset information specification; calling the edge component through the GPU device component on the edge node to complete node registration based on the obtained GPU status information.

[0194] In one possible implementation of the present application, the processor may also perform the following processing to build an information viewing tool on each edge node: adapt the initial information viewing tool according to the node environment of the edge node and the GPU type of the GPU arranged on the edge node, so that the adapted initial information viewing tool can obtain GPU status information from the GPU type of the GPU in the node environment; determine the adapted initial information viewing tool as the information viewing tool.

[0195] According to the computer-readable storage medium of the exemplary embodiment of the present application, rapid access to GPUs of different node environments / GPU types can be achieved. In addition, the scheduling problem of GPU-exclusive tasks and multi-task shared GPUs under the same edge node is solved, effectively improving the utilization efficiency of edge GPU nodes.

[0196] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0197] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0199] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0200] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A GPU operation method of an edge node, characterized in that: include: Acquire task processing information of a GPU task, wherein the task processing information includes a task identifier and an occupation identifier preset for a GPU shared task group, wherein the task identifier is used to indicate a task type of the GPU task, wherein the task type includes a GPU exclusive task and a GPU shared task group, wherein the GPU shared task group includes a plurality of shared subtasks; Create a Pod corresponding to the task type indicated by the task identifier; Searching for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier, wherein the edge node search strategy corresponding to a GPU exclusive task is to search for edge nodes based on a device requirement of the GPU exclusive task for the GPU, and the edge node search strategy corresponding to a GPU shared task group is to search for edge nodes based on an occupancy identifier pre-set for the GPU shared task group; Schedule the created Pod to the target edge node so that the target edge node executes the GPU task. The target edge node is determined for each shared subtask in the following way: Searching from each edge node in the node pool whether there is an edge node having an occupancy identifier corresponding to the GPU shared task group; If so, the searched edge node is determined as the target edge node for executing the shared subtask; If not, an edge node is selected from the edge nodes in the node pool that do not execute GPU exclusive tasks, and the selected edge node is determined as the target edge node for executing the shared subtask.

2. The method according to claim 1, characterized in that The step of searching for a target edge node according to the edge node search strategy indicated by the task processing information includes: Determine the operating state of each edge node in the node pool, where the operating state includes a working state and an idle state; An edge node that meets the device requirement of the GPU exclusive task for the GPU and is searched from the edge nodes in the idle state is determined as the target edge node.

3. The method according to claim 1, characterized in that Also includes: Adding an occupation flag to the edge nodes selected from the edge nodes that do not execute GPU exclusive tasks; After the target edge node completes all shared subtasks in the GPU shared task group, the occupancy flag added for the target edge node is cleared.

4. The method according to claim 1, characterized in that: The step of scheduling the created Pod to the target edge node so that the target edge node executes the GPU task includes: When starting the Pod, the GPU device plug-in is called to obtain the device environment variables, and the device environment variables are obtained by the GPU device plug-in in the following manner: obtaining the device requirements of the GPU task for the GPU, selecting a GPU from the target edge node according to the device requirements, and generating the device environment variables based on the device information of the selected GPU; The obtained device environment variables are injected into the Pod, and the container is started to execute the GPU task.

5. The method according to claim 4, characterized in that Also includes: When starting the container, according to the device information declared in the device environment variable, the GPU indicated by the device information is mapped into the container, and the driver library of the GPU indicated by the device information is also mapped into the container.

6. The method according to claim 1, characterized in that Also includes: Register each edge node in the node pool. Among them, node registration is performed for each edge node in the following way: Building an information viewing tool on the edge node; Calling an information viewing tool through the edge component on the edge node to obtain GPU status information according to a preset information specification; The edge component is called through the GPU device plug-in on the edge node to complete the node registration based on the acquired GPU status information.

7. The method according to claim 6, characterized in that Build information viewing tools on each edge node in the following ways: Adapting the initial information viewing tool according to the node environment of the edge node and the GPU type of the GPU arranged on the edge node, so that the adapted initial information viewing tool can obtain GPU status information from the GPU type in the node environment; The adapted initial information viewing tool is determined as the information viewing tool.

8. A GPU operating device for an edge node, characterized in that: include: an acquisition module, configured to acquire task processing information of a GPU task, wherein the task processing information includes a task identifier and an occupation identifier preset for a GPU shared task group, wherein the task identifier is used to indicate a task type of the GPU task, wherein the task type includes a GPU exclusive task and a GPU shared task group, wherein the GPU shared task group includes a plurality of shared subtasks; A creation module, used to create a Pod corresponding to the task type indicated by the task identifier; A search module is used to search for a target edge node according to an edge node search strategy corresponding to the task type indicated by the task identifier, wherein the edge node search strategy corresponding to the GPU exclusive task is to search for edge nodes based on the device requirements of the GPU for the GPU exclusive task, and the edge node search strategy corresponding to the GPU shared task group is to search for edge nodes based on an occupancy identifier pre-set for the GPU shared task group; A scheduling module, used to schedule the created Pod to the target edge node so that the target edge node executes the GPU task, Wherein, the search module is also used for: Searching from each edge node in the node pool whether there is an edge node having an occupancy identifier corresponding to the GPU shared task group; If so, the searched edge node is determined as the target edge node for executing the shared subtask; If not, an edge node is selected from the edge nodes in the node pool that do not execute GPU exclusive tasks, and the selected edge node is determined as the target edge node for executing the shared subtask.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the GPU operation method of the edge node as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the GPU operation method of the edge node according to any one of claims 1 to 7 are executed.

Images