Non-intrusive resource allocation method, device and storage medium based on Kubernetes cluster

Abstract

The present application relates to a non-intrusive resource allocation method, device, and storage medium based on a Kubernetes cluster. The non-intrusive resource allocation method based on a Kubernetes cluster includes: upon receiving a registration request from a GPU resource device plug-in in a K8s cluster node, creating a proxy server that acts as a proxy for the GPU resource device plug-in based on preset configuration information; and partitioning the GPU resources on the node through the proxy server, wherein the GPU resources on the node are obtained through a deployed proxy service. Through this application, refined management of GPU resources on the node is achieved, improving resource utilization.

Description

Technical Field

[0001] The present application relates to the field of cloud-native scheduling, and in particular to a non-intrusive resource allocation method, device, and storage medium based on a Kubernetes cluster. Background Art

[0002] A Kubernetes cluster (often referred to as Kubernetes) is an open-source platform for automating the deployment, scaling, and management of containerized applications. With the rapid development of cloud-native technologies, exemplified by Kubernetes, a growing number of deep learning and high-performance computing tasks are being orchestrated and managed uniformly through the Kubernetes platform. Furthermore, with advancements in computing hardware, GPUs (GPUs), as core computing resources, are widely used in modern deep learning and high-performance computing scenarios. However, due to their high cost and the wide variety of vendors, many Kubernetes clusters adopt a heterogeneous GPU architecture, where GPUs from multiple models or vendors coexist in the cluster to maximize resource utilization and reduce costs. In actual deployments, Kubernetes allows users to dynamically deploy, manage, and scale applications within the cluster. GPU vendors provide their own device plugins, adhering to the Kubernetes plugin extension mechanism standard. These plugins register GPU resources with the Kubernetes cluster, enabling scheduling and allocation. In a Kubernetes (K8s) cluster, a node is the basic unit of the cluster, used to run and manage containerized applications (Pods). K8s manages and schedules GPU resources through a plug-in extension mechanism. It usually deploys the same type of GPU on a single node, that is, only GPU devices of the same model or manufacturer are deployed on a node. Each node is considered the basic unit of the resource pool, and resources are called directly on each node through the plug-in extension mechanism.

[0003] However, within nodes, the types and performance differences of cloud computing devices are increasing. Especially with the prevalence of heterogeneous devices such as GPUs, TPUs, and FPGAs, different types of devices (CPUs, GPUs, TPUs, FPGAs, etc.) have different performance and uses. Traditional node-based resource management cannot effectively distinguish these differences, resulting in inefficient resource allocation. Secondly, applications and tasks have varying resource requirements. Some tasks may rely more on the computing power of the CPU, while others may require the parallel processing capabilities of the GPU. Traditional node-based resource management cannot meet the needs of different task types. Furthermore, in a multi-tenant environment, resource isolation between different tenants is necessary to avoid resource contention and performance degradation. Traditional node-based resource management cannot avoid resource contention, resulting in low GPU resource utilization.

[0004] There is currently no effective solution to the problem of low resource utilization in related technologies. Summary of the Invention

[0005] In this embodiment, a resource allocation method, device, electronic device, and storage medium are provided to solve the problem of low resource utilization in related technologies.

[0006] First, in this embodiment, a non-intrusive resource allocation method based on a Kubernetes cluster is provided, including:

[0007] In the K8s cluster node, after receiving the registration request of the GPU resource device plug-in, a proxy server for the GPU resource device plug-in is created according to the preset configuration information;

[0008] The GPU resources on the node are divided by the proxy server, wherein the GPU resource status on the node is obtained through the deployed proxy service.

[0009] In some embodiments, before receiving the registration request of the GPU resource device plug-in, the method further includes:

[0010] Initialize the configuration information of the GPU resources on the node.

[0011] In some embodiments, after the GPU resources on the node are divided by the proxy server, the method further includes:

[0012] The corresponding configuration strategy is executed according to the preset configuration information through the interface extension point mechanism, wherein each configuration strategy corresponds to a configuration item function in the preset configuration information.

[0013] In some embodiments, the configuration item function includes grouping information of the GPU resources and hidden switch configuration of the GPU resources.

[0014] In some embodiments, creating a proxy server for the GPU resource device plug-in according to preset configuration information includes:

[0015] Reading the preset configuration information, and determining the number of groups of the GPU resources according to the configuration information;

[0016] Create proxy servers for the GPU resource device plug-ins, the same number of which is the same as the number of groups of the GPU resources.

[0017] In some embodiments, after creating the same number of proxy servers as the number of groups of GPU resources to act as proxies for the GPU resource device plug-ins, the method further includes:

[0018] The proxy server registers the resource specification name with the main node agent on the node to obtain a new resource specification name, so that the k8s cluster can retrieve GPU resources according to the new resource specification name.

[0019] In some embodiments, when the GPU resource device plug-in on the node is offline, the proxy server corresponding to the GPU resource device plug-in is destroyed.

[0020] In the second aspect, a non-intrusive resource allocation device based on a Kubernetes cluster is provided in this embodiment, including: a creation module and a segmentation module, wherein:

[0021] The creation module is used to create a proxy server that acts as a proxy for the GPU resource device plug-in in the K8s cluster node after receiving a registration request from the GPU resource device plug-in according to preset configuration information;

[0022] The segmentation module is configured to segment the GPU resources on the node through the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service.

[0023] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the non-invasive resource allocation method based on the Kubernetes cluster described in the first aspect above is implemented.

[0024] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the non-intrusive resource allocation method based on the Kubernetes cluster described in the first aspect above is implemented.

[0025] Compared with related technologies, the non-intrusive resource allocation method based on the Kubernetes cluster provided in this embodiment creates a proxy server for the GPU resource device plug-in in the K8s cluster node according to preset configuration information after receiving a registration request for the GPU resource device plug-in; the GPU resources on the node are divided by the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service, thereby realizing refined management of GPU resources on the node and improving resource utilization.

[0026] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0028] Figure 1 This is a hardware structure block diagram of a terminal of the non-intrusive resource allocation method based on a Kubernetes cluster in this embodiment.

[0029] Figure 2 This is a flowchart of the non-intrusive resource allocation method based on the Kubernetes cluster of this embodiment.

[0030] Figure 3 Schematic diagram of node resource allocation of the non-invasive resource allocation method based on Kubernetes cluster in this embodiment.

[0031] Figure 4 This is a schematic diagram of the proxy server destruction process based on non-intrusive resource allocation of the Kubernetes cluster in this embodiment.

[0032] Figure 5 This is a flowchart of a GPUA resource allocation process based on non-invasive resource allocation of a Kubernetes cluster in this embodiment.

[0033] Figure 6 This is a flowchart of the proxy server operation of the non-intrusive resource allocation based on the Kubernetes cluster in this embodiment.

[0034] Figure 7 This is a flowchart of another preferred non-intrusive resource allocation method based on a Kubernetes cluster according to this embodiment.

[0035] Figure 8 This is a structural block diagram of the non-intrusive resource allocation device based on the Kubernetes cluster of this embodiment. DETAILED DESCRIPTION

[0036] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0037] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0038] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 This is a hardware structure diagram of the terminal of the non-intrusive resource allocation method based on the Kubernetes cluster of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 The processor 102 (only one is shown) and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input / output device 108. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0039] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the non-invasive resource allocation method based on the Kubernetes cluster in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] Transmission device 106 is used to receive or transmit data via a network. This network may include a wireless network provided by the terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] In this embodiment, a resource allocation method is provided. Figure 2 This is a flow chart of the non-intrusive resource allocation method based on the Kubernetes cluster of this embodiment. Figure 2 As shown, the process includes the following steps:

[0042] Step S201: In the K8s cluster node, after receiving the registration request of the GPU resource device plug-in, a proxy server for the GPU resource device plug-in is created according to preset configuration information.

[0043] Specifically, Kubernetes (often referred to as k8s) is an open source container orchestration platform for automating the deployment, scaling, and management of containerized applications. The k8s cluster consists of multiple nodes, divided into master nodes (Master Node) and worker nodes (WorkerNode), where nodes can be physical servers or virtual machines, which are machines running containerized applications. The master node is responsible for cluster management and includes components such as API Server, Scheduler, Controller Manager, and etcd. The worker node is responsible for running application containers. Each worker node runs the k8s main node agent (kubelet), which is responsible for managing the life cycle of the container. Pod is the smallest deployment unit in k8s and can contain one or more containers. The containers within the Pod share the same network namespace and storage volumes, so they can communicate with each other as if they were on the same machine. The controller is a component in k8s used to manage the life cycle and status of the Pod. Common controllers include:

[0044] Deployment: used to manage stateless applications;

[0045] StatefulSet: used to manage stateful applications;

[0046] DaemonSet: Ensures that a Pod copy runs on each node;

[0047] Job: used to run one-time tasks.

[0048] In a Kubernetes cluster, GPUs (Graphics Processing Units) are a specialized hardware resource typically used to accelerate compute-intensive tasks such as machine learning, deep learning, scientific computing, and graphics rendering. Due to the unique nature of GPU resources, Kubernetes requires specialized plugins or extensions to manage and schedule GPU resources. Kubernetes itself does not directly manage GPU resources. The GPU resource device plugin exposes GPU devices as manageable resources to the Kubernetes cluster. By interacting with the Kubernetes Device Plugin API, the plugin registers GPU devices with the Kubernetes resource management system. Once registered, Kubernetes can manage and schedule GPU resources similarly to CPU and memory resources. The Kubernetes scheduler determines which node to assign a pod to based on its resource requests, including GPU resources. For example, if a pod requires GPU resources, the scheduler will search for nodes with available GPUs and ensure that the GPU resources meet the pod's requirements. The GPU resource device plugin provides GPU status information (such as availability and model) to assist the scheduler in making decisions.

[0049] Currently, when the scheduler finds available GPU nodes, it searches based on the node. All GPU resources within the node are visible to Kubernetes. Kubernetes randomly selects GPUs from the available GPU nodes to allocate Pods. When there are many tasks, resource contention often occurs. Therefore, before allocating Pods, this embodiment deploys an agent service (Agent) and a proxy service manager (ProxyManager) on each node in the Kubernetes cluster. The Agent on each node obtains GPU resources in the node and reports them to the ProxyManager via a GPU resource device plug-in. After the GPU resource device plug-in initiates a registration request to the ProxyManager program, the ProxyManager program creates a proxy server (Proxy) based on preset configuration information (such as the GPU resource grouping strategy and hidden switch configuration). For example, a Kubernetes cluster contains two types of GPU cards, GPUA and GPUB, with the same type of GPU card deployed on each node. Two types of GPU resource device plug-ins, DeviceA and DeviceB, are installed in the cluster to report different types of GPU resources. The resource names registered with Kubernetes are ResourceA and ResourceB, respectively. Figure 3 : is a schematic diagram of node resource allocation of the non-invasive resource allocation method for the Kubernetes cluster of this embodiment, as shown in FIG. Figure 3As shown, there are four nodes, two of which are equipped with GPUB, denoted as ResourceB:5 and ResourceB:2. ResourceB:5 represents five GPUBs in the node, and ResourceB:2 represents two GPUBs in the node. The other two nodes are equipped with GPUA, denoted as ResourceA:8 and ResourceA:8. Similarly, ResourceA:8 represents eight GPUA cards in the node. Taking the two nodes where GPUA resides as an example, the left diagram shows the GPUA card in node 1, and the right diagram shows the GPUA card in node 2. Each node has eight GPU cards, and the configuration information requires that four cards on node 1 be configured in group 1, and the other four cards on node 1 be configured in group 2. Two cards on node 2 are configured in group 1, with one card's switch set to off. The other six cards on node 2 are configured in group 2. According to the configuration information, before allocation, there are 8 GPUA cards in Node 1, and the resource name registered in k8s is ResourceA:8; there are 8 GPUA cards in Node 2, and the resource name registered in k8s is ResourceA:8; two proxy servers Proxy1 and Proxy2 are created on Node 1, and two proxy servers Proxy3 and Proxy4 are also created on Node 2. The created proxy servers are used to manage the GPU resources on each node respectively. After the proxy service, the reported resource specification names become Resource_Group1A and Resource_Group2A. The number of visible resources on the two nodes has also changed. Since a card is hidden on Node 2, the number of GPU cards visible on Node 2 has become 7.

[0050] During this process, a monitoring channel is established to continuously monitor changes in GPU resources on the node. Once a change in GPU resources is detected, the GPU information cache module built into the Proxy is updated, and a resource information change notification is reported to the main node agent (Kubelet). The Proxy saves the service address of the corresponding GPU resource device plug-in and forwards Kubelet's resource request to the GPU resource device plug-in. The Proxy receives the response result of the GPU resource device plug-in, customizes the response data according to the configuration of the extension point, and returns it to Kubelet.

[0051] Specifically, when a user modifies the configuration of a GPU resource, the ProxyManager uses the Agent to determine whether any tasks are currently running on the GPU resource. If so, the configuration change fails. If not, the GPU resource's configuration information is changed, notifying the proxies that previously grouped the GPU resource and the proxies that will be assigned to it after the change. The affected proxies then re-report the GPU resource information to the Kubelet, the primary node agent. This enables dynamic updates of GPU resource configuration information.

[0052] Step S202: Segment the GPU resources on the node through the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service.

[0053] Specifically, the GPU resources in the nodes are managed separately by deploying proxy servers in the nodes. For example, in step S201 above, two proxies are set on node 1 and node 2 respectively. Figure 3 As shown, Proxy1 is used to manage four GPU cards in Node 1 ( Figure 3 The 4 GPUs in black text on a white background in the lower left image are named GPUA). The resource name of the GPU card managed by Proxy1 is named Resource_Group1A and reported to the k8s cluster. The remaining 4 GPU cards in Node 1 are managed by Proxy2 ( Figure 3 The GPU card resource managed by Proxy2 is named Resource_Group2A and reported to the k8s cluster to establish a mapping relationship between the proxy server Proxy and the GPU resources. Similarly, the two GPU cards in Node 2 are managed by Proxy3 in Node 2 ( Figure 3 The two GPUs (black text on a white background) in the lower right middle image are shown as follows: Figure 3 The GPU card managed by Proxy3 is named Resource_Group1A and reported to the k8s cluster; the remaining 6 GPU cards in Node 2 are managed by Proxy4 ( Figure 3The six GPUAs (white text on a black background) in the lower-right image above are displayed. The GPU resources managed by Proxy4 are named Resource_Group2A and reported to the Kubernetes cluster. Four proxies are created on Node 1 and Node 2 to separate the GPU resources. When a user configures GPUA resources through the Kubernetes cluster, based on user permissions (for example, if the user can only view GPU resources in Group 1), Node 1 is displayed to the Kubernetes cluster as: Resource_Group1A: 4, Resource_Group2A: 4; Node 2 is displayed to the Kubernetes cluster as: Resource_Group2A: 6, Resource_Group1A: 1. The other GPUA is disabled and invisible. At this time, the user can only view the four GPU cards Resource_Group1A:4 in Group1 in node 1 and the one GPU card Resource_Group1A:1 in Group1 in node 2 through the k8s cluster. For the user, Group2 in node 1 and node 2 is invisible at this time and can be viewed and called by users with other permissions. When the user's Pod requests GPUA resources, the resource identifier requested by the pod is switched from the original GPUA resource specification name to the Resource_Group1A resource specification name. The k8s scheduler will search for suitable GPUA cards in Group1 in node 1 and node 2 when scheduling the Pod, thereby achieving a finer division of resources in the node, achieving more reasonable resource allocation, avoiding resource competition, and thus improving resource utilization.

[0054] Through the above steps S201 to S202, in the K8s cluster node, after receiving the registration request of the GPU resource device plug-in, a proxy server for the GPU resource device plug-in is created according to the preset configuration information; the GPU resources on the node are divided by the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service. Compared with the resource management method in the prior art and the node-based resource management method, this embodiment deploys a proxy service and a proxy service management program in the node, and creates a proxy server in each node according to the preset configuration information through the proxy service management program to separately manage the GPU resources in each node, thereby achieving the division of resources in the node, realizing further resource segmentation management, avoiding resource competition, achieving more rational resource allocation, and improving resource utilization.

[0055] In some embodiments, before receiving the registration request of the GPU resource device plug-in, the process further includes: initializing the configuration information of the GPU resource on the node.

[0056] Specifically, the GPU resource information in the node is obtained through the agent service Agent pre-deployed in the node to form an original configuration file. At this time, the configuration information of the GPU resources on all nodes in the original configuration file is in the default state. The grouping information and hidden switch configuration of the GPU resources on all nodes are initialized, and the original default state is cleared to facilitate subsequent user customized configuration.

[0057] In another embodiment, after the GPU resources on the node are divided through the proxy server, it also includes: executing corresponding configuration strategies according to preset configuration information through the interface extension point mechanism, wherein each configuration strategy corresponds to a configuration item function in the preset configuration information.

[0058] Specifically, after segmenting the GPU resources on a node, this embodiment uses an interface extension point mechanism to implement different configuration strategies based on pre-set GPU resource configuration information. Extension points are a system design mechanism that allows developers or users to customize system behavior through extension strategies, adding new functionality or modifying existing functionality without changing the core system logic. Each configuration extension strategy typically corresponds to a configuration item function in the system configuration list. Extension strategies can be implemented as plug-in mechanisms or configuration drivers. By developing plug-ins or extension modules, the system can execute these plug-ins at runtime. Plugins can process specific interface requests based on the functional requirements of the configuration items. Using configuration items in configuration files or databases, the system can dynamically adjust its behavior based on these configurations. For example, configuration items can specify which algorithm to use, enable or disable certain functions, and control the powering on and off of GPU cards. Different extension strategies can be set using the extension point mechanism. Based on the set extension strategies, corresponding configuration information is obtained. Operations and maintenance personnel can assemble their own operations and maintenance instructions by manipulating the configuration items. The proxy service manager, ProxyManager, creates and manages the required proxy servers based on the configuration information to proxy the GPU resources on each node. Through the extension point mechanism, the system can quickly adapt to new business needs through simple configuration or plug-in development without modifying the core code. By separating the core code from the extension logic, system maintenance is easier and the development and testing of extension modules are more independent. In addition, the system can dynamically load or unload extension modules as needed, supporting unlimited expansion. By properly designing extension points and expansion strategies, the system's scalability and user experience can be significantly improved.

[0059] In some embodiments, the configuration item function includes grouping information of GPU resources and hidden switch configuration of GPU resources.

[0060] In the above embodiment, when the corresponding configuration strategy is executed through the extension point mechanism, each configuration strategy corresponds to a configuration item function in the preset configuration information, wherein the configuration item function mainly configures the grouping information of the GPU resources and the hidden switch of the GPU resources, that is, sets the number of groups of GPU resources in the node, sets the opening and closing of the GPU resources in the node, and controls the available GPU resources.

[0061] In another embodiment, creating a proxy server for proxy GPU resource device plug-ins based on preset configuration information includes: reading the preset configuration information, determining the number of groups of GPU resources based on the configuration information; and creating proxy servers for proxy GPU resource device plug-ins in the same number as the number of groups of GPU resources.

[0062] Specifically, the proxy service management program (ProxyManager) reads the preset configuration information, where the preset configuration information includes the number of groups and switch status of GPU resources, creates the same number of proxy servers Proxy according to the number of groups of GPU resources in the configuration information, analyzes the GPU resource grouping on the node, creates multiple proxy servers Proxy if there are multiple groups, and establishes a mapping relationship between Proxy and GPU resources.

[0063] In some of the embodiments, after creating a number of proxy servers for proxy GPU resource device plug-ins that is the same as the number of groups of GPU resources, it also includes: registering the resource specification name with the main node agent on the node through the proxy server to obtain a new resource specification name, so that k8s can call GPU resources according to the new resource specification name.

[0064] After the proxy servers Proxy are created based on the number of groups of GPU resources in the configuration information, each proxy server Proxy registers the new resource specification name after the proxy with the main node agent (Kubelet) on the node. For example, Resource_Group1A and Resource_Group2A described in step S202 are the resource specification names of the new proxy GPU resources registered by each proxy server Proxy to the k8s cluster. Among them, Kubelet is a core component in k8s, running on each node in the cluster. It is the main node agent of the node, managing the containers and Pods on the node, and ensuring that they run correctly according to the instructions of the k8s control plane. Kubelet is the bridge between the node and the control plane in the k8s cluster, and is responsible for performing node-related operations. By reporting the new resource specification name of each proxy server Proxy, k8s can call new GPU resources according to user needs.

[0065] In another embodiment, when the GPU resource device plug-in on the node goes offline, the proxy server corresponding to the GPU resource device plug-in is destroyed.

[0066] Based on the full lifecycle management of the proxy server, the ProxyManager creates a Proxy when it receives the GPU resource device plug-in registration. When the GPU resource device plug-in is detected to be offline, uninstalled, or crashed, the connection between the Proxy and the GPU resource device plug-in is severed and the ProxyManager automatically destroys the Proxy. This avoids affecting the normal operation of the Kubernetes cluster and saves storage resources in the Kubernetes cluster. Figure 4 FIG. 1 is a flow chart of a proxy server destruction process based on non-invasive resource allocation of a Kubernetes cluster in this embodiment. Figure 4 As shown in the figure, during the process of the GPU resource device plug-in service providing proxy service through the proxy server (Proxy), a monitoring channel is established to continuously monitor the GPU resource changes on the node. When the GPU resource device plug-in channel is detected to be disconnected, the connection between the Proxy and the GPU resource device plug-in is cut off, and the proxy server (Proxy) is automatically destroyed through the proxy service management program (ProxyManager).

[0067] For example, Figure 5 This is a flowchart of a GPUA resource allocation process based on non-invasive resource allocation of a Kubernetes cluster in this embodiment. Figure 5 As shown in the figure, first, in the k8s centralized node, the GPU resource device plug-in (DeviceA) initiates a registration request to the proxy service manager (ProxyManager); second, the proxy service manager (ProxyManager) loads the configuration list and creates two proxy servers (ProxyDeviceAGroup1 and ProxyDeviceAGroup2) according to the configuration list. The GPU resources in the node are divided into two groups through the proxy server (Proxy); finally, the proxy server (Proxy) registers the divided GPU resources with the main node agent (Kubelet) on the node with new resource specification names (Resource_Group1A and Resource_Group2A), completing the allocation of GPUA resources in the node.

[0068] For example, Figure 6 This is a flowchart of the proxy server operation based on the non-intrusive resource allocation of the Kubernetes cluster in this embodiment. Figure 6 As shown, the process includes:

[0069] Step 1: Register the service. The GPU device plug-in first registers the service with the proxy service manager (ProxyManager), indicating that it can manage GPU device resources so as to provide remote procedure call framework services (grpc services).

[0070] Step 2: Generate a Proxy component. After receiving the registration request, the ProxyManager generates one or more Proxy components based on the configuration file for communication between the GPU device plug-in and the kubelet.

[0071] Step 3: The proxy registers the plug-in (device) resource with the primary node agent (Kubelet) and provides a Unix domain socket (sock) address for subsequent communication;

[0072] Step 4: Initiate a request. When a request is made to access the GPU device, the kubelet initiates a request to the proxy to obtain GPU device information or perform related operations.

[0073] Step 5: Forward the request. The proxy agent forwards the kubelet request to the GPU device plug-in service.

[0074] Step 6: Receive the plug-in response information. The plug-in service receives and processes the request from the proxy service, obtains the relevant information of the GPU device, and the proxy service receives the plug-in response information.

[0075] Step 7: Intercept the response body. During the request transmission process, the interceptor will intercept and process the plug-in response information;

[0076] Step 8: Return the response result. The Proxy proxy service returns the plug-in's response result to the kubelet.

[0077] In Kubernetes nodes, the Proxy service enables the kubelet to dynamically acquire and manage GPU device resources. This further refines GPU resource management. The Proxy acts as a bridge, forwarding requests and delivering responses. This makes GPU resource management more flexible and efficient, while also facilitating scalability and maintenance.

[0078] This embodiment also provides a resource allocation method. Figure 7 This is a flowchart of another preferred non-intrusive resource allocation method based on Kubernetes clusters in this embodiment. Figure 7 As shown, the process includes the following steps:

[0079] Step S701, deploying a proxy service and a proxy service management program in each node of the K8s cluster;

[0080] Step S702: Obtain the GPU resources on the node through the proxy service, and initialize the configuration information of the GPU resources on the node;

[0081] Step S703: deploying a GPU resource device plug-in on the node, and the GPU resource device plug-in initiates a registration request to the proxy service management program;

[0082] Step S704: The proxy service management program determines the number of groups of GPU resources based on preset configuration information, and creates proxy servers for proxy GPU resource device plug-ins of the same number as the number of groups of GPU resources, and divides the GPU resources on the node through the proxy servers.

[0083] Step S705: register the resource specification name with the primary node agent on the node through the proxy server to obtain a new resource specification name, so that k8s can call GPU resources according to the new resource specification name;

[0084] Step S706: Execute the corresponding configuration strategy according to the retrieved GPU resource configuration information through the interface extension point mechanism.

[0085] Through the above steps S701 to S706, compared with the resource management method in the prior art and the node-based method, this embodiment obtains all GPU resource information on the node through the proxy service deployed on the node. The GPU resource initiates a registration request to the proxy service manager. The proxy service manager creates a proxy server agent corresponding to the GPU resource device plug-in based on the preset configuration information, thereby realizing the segmentation of GPU resources on the node, obtaining the segmented GPU resources, and then scheduling the GPU resources according to the native scheduling mechanism of k8s. Among them, by establishing different proxy servers on the node to proxy different GPU resources, the resource segmentation and allocation are realized on the node, the resources are subdivided on the node, and the resource utilization is improved.

[0086] This embodiment also provides a resource allocation device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0087] Figure 8This is a structural block diagram of the non-intrusive resource allocation device based on the Kubernetes cluster of this embodiment. Figure 8 As shown, the device 80 includes: a creation module 81 and a segmentation module 82, wherein,

[0088] A creation module 81 is configured to create a proxy server for the GPU resource device plug-in in a K8s cluster node after receiving a registration request from the GPU resource device plug-in according to preset configuration information;

[0089] The segmentation module 82 is configured to segment the GPU resources on the node through a proxy server, wherein the GPU resources on the node are obtained through a deployed proxy service.

[0090] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0091] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0092] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0093] S1, in the K8s cluster node, after receiving the registration request of the GPU resource device plug-in, create a proxy server that acts as a proxy for the GPU resource device plug-in according to the preset configuration information;

[0094] S2: Segment the GPU resources on the node through the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service.

[0095] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0096] In addition, in conjunction with the resource allocation method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any of the non-intrusive resource allocation methods based on a Kubernetes cluster in the above embodiments is implemented.

[0097] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0098] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0099] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0100] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processors (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0101] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A non-intrusive resource allocation method based on Kubernetes cluster, characterized in that: include: In the k8s cluster node, after receiving the registration request of the GPU resource device plug-in, read the preset configuration information, and determine the number of groups of the GPU resources according to the configuration information; Creating a number of proxy servers for the GPU resource device plug-ins that is the same as the number of groups of the GPU resources; wherein the configuration information includes a grouping strategy and a hidden switch configuration of the GPU resources; The GPU resources on the node are divided by the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service.

2. The non-intrusive resource allocation method based on Kubernetes cluster according to claim 1, characterized in that Before receiving the registration request of the GPU resource device plug-in, the method further includes: Initialize the configuration information of the GPU resources on the node.

3. The non-intrusive resource allocation method based on Kubernetes cluster according to claim 1, characterized in that After the GPU resources on the node are divided by the proxy server, the method further includes: The corresponding configuration strategy is executed according to the preset configuration information through the interface extension point mechanism, wherein each configuration strategy corresponds to a configuration item function in the preset configuration information.

4. The non-intrusive resource allocation method based on Kubernetes cluster according to claim 3 is characterized in that The configuration item functions include grouping information of the GPU resources and hidden switch configuration of the GPU resources.

5. The non-intrusive resource allocation method based on Kubernetes cluster according to claim 1, characterized in that After creating the same number of proxy servers as the number of groups of GPU resources that act as proxies for the GPU resource device plug-ins, the method further includes: The proxy server registers the resource specification name with the main node agent on the node to obtain a new resource specification name, so that the k8s cluster can retrieve GPU resources according to the new resource specification name.

6. The non-intrusive resource allocation method based on Kubernetes cluster according to claim 1, characterized in that The method further comprises: When the GPU resource device plug-in on the node goes offline, the proxy server corresponding to the GPU resource device plug-in is destroyed.

7. A non-intrusive resource allocation device based on Kubernetes cluster, characterized in that: include: Create modules and split modules, where The creation module is configured to, in a k8s cluster node, read preset configuration information after receiving a registration request from a GPU resource device plug-in, determine the number of groups of the GPU resources based on the configuration information, and create a number of proxy servers that act as proxies for the GPU resource device plug-ins, the same number as the number of groups of the GPU resources; wherein the configuration information includes a grouping strategy and a hidden switch configuration for the GPU resources; The segmentation module is configured to segment the GPU resources on the node through the proxy server, wherein the GPU resources on the node are obtained through the deployed proxy service.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the non-intrusive resource allocation method based on a Kubernetes cluster according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the non-intrusive resource allocation method based on a Kubernetes cluster are implemented as described in any one of claims 1 to 6.

Images