Dynamic regulation and control method for GPU (Graphic Processing Unit) resources of development machine, computer device and medium

Abstract

The invention discloses a development machine GPU resource dynamic regulation and control method, a computer device and a medium, and the method comprises the steps: responding to a development machine creation request, creating a user-defined resource object containing development machine configuration information, creating a target Pod in a Kubernetes cluster, determining a to-be-scheduled target GPU node, obtaining the GPU application number of the to-be-scheduled target GPU node, and then determining a target physical GPU, creating a virtual GPU device file on the host, and establishing a soft link between the virtual GPU device file and the target physical GPU; after the creation is completed, starting a development machine container and mounting the virtual GPU equipment file into the container; when it is monitored that the container GPU resource idle information meets the preset idle condition, the soft link is released, and the real GPU state is updated to be idle. According to the method, container mirror images and original K8s deployment logic do not need to be modified, sharing and multiplexing of GPU resources among multi-container development machines are supported, and the GPU resource utilization rate is remarkably increased.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to a GPU resource dynamic regulation method for development machine, computer device and medium. BACKGROUND

[0002] In today's technology field, artificial intelligence (AI) development and deep learning training are booming at an unprecedented speed, and these advanced technology applications require computing resources. With the rise of container technology, tools such as Kubernetes and Docker are widely used in the deployment of development machine systems. However, there are serious limitations in the existing container deployment solutions in terms of GPU resource allocation.

[0003] Currently, in the development machine system based on Kubernetes or Docker, the allocation of GPU resources usually adopts a static allocation method. When the container starts, the system will allocate a certain amount of GPU resources to the container according to the pre-set configuration. This method exposes many problems in practical application. On the one hand, during the development process, due to the complexity and uncertainty of development tasks, the use of GPU resources is often dynamically changing. Many times, GPU resources will be in a long idle state, but since they have been statically allocated to specific containers, these idle resources cannot be utilized by other tasks that need them, resulting in serious resource waste. On the other hand, this static allocation method lacks flexibility in scheduling and cannot adjust the allocation strategy in real time according to the actual resource demand. When there is a new high-priority task that needs to use GPU resources, due to the fixed occupation of resources, it may not be able to allocate enough resources in time, resulting in task delay or failure to execute, and thus affecting the overall development efficiency.

[0004] Existing solutions mostly focus on optimizing the task scheduling layer. For example, by optimizing the task scheduling algorithm, it tries to allocate GPU resources more reasonably to improve resource utilization. However, these methods do not work well in development scenarios with long-term keep-alive characteristics. Development machines usually need to be kept running for a long time to meet the needs of developers to develop and debug at any time. In this case, the optimization of the task scheduling layer cannot solve the fundamental problem caused by static allocation because the resource demand of the development machine has uncertainty and dynamics, and traditional scheduling algorithms cannot accurately predict and respond to these changes in real time. SUMMARY

[0005] Therefore, the embodiments of the present disclosure provide a GPU resource dynamic regulation method for development machine, computer device and medium, which can support dynamic linking / disabling of real GPU resources during container operation, maximizing the reuse efficiency of GPU devices in the cluster.

[0006] In a first aspect, the embodiments of the present disclosure provide a GPU resource dynamic regulation method for a development machine, comprising: In response to a user-initiated development machine creation request, a custom resource object containing development machine configuration information is created according to a custom resource specification of the development machine; According to the custom resource object, a corresponding Pod is created in a Kubernetes cluster, denoted as a target Pod; A target GPU node to be scheduled for the target Pod is determined; According to the target GPU node, the number of GPUs applied for by the target Pod is obtained; According to the number of GPUs applied for, a plurality of target physical GPUs are determined; According to the plurality of target physical GPUs, a plurality of virtual GPU device files are created on a host; A soft link is established between each virtual GPU device file and the corresponding target physical GPU; In response to a creation completion instruction of the virtual GPU device file, a development machine container is started, and the virtual GPU device file is mounted into the container; When the obtained container GPU resource idle information meets a preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is released, and the use state of the corresponding real GPU is updated to idle.

[0007] In a second aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution: The computer device comprises: At least one processor; and A memory in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the GPU resource dynamic regulation method for a development machine described in any of the above.

[0008] In a third aspect, the embodiments of the present disclosure further provide a computer readable storage medium storing computer instructions for causing a computer to execute the GPU resource dynamic regulation method for a development machine described in any of the above.

[0009] In a fourth aspect, the embodiments of the present disclosure further provide a computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the method described in any of the above.

[0010] The GPU resource dynamic regulation method disclosed in the application first responds to a development machine creation request initiated by a user, creates a custom resource object containing development machine configuration information according to a custom resource specification of the development machine, creates a corresponding Pod in a Kubernetes cluster according to the custom resource object, and records the Pod as a target Pod, which provides a basis for subsequent dynamic allocation of GPU resources. Through the custom resource object, various resources required by the development machine can be flexibly configured, including relevant information of the GPU resource, instead of fixed configuration as in traditional static allocation. Then, the target GPU node to be scheduled by the target Pod is determined, the number of GPU applications corresponding to the target Pod is obtained according to the target GPU node, and a plurality of target physical GPUs are determined according to the number of GPU applications. This way makes the allocation of GPU resources closely related to the specific Pod demand, and can dynamically select appropriate GPU resources according to the actual development task demand, instead of fixed static allocation when the container starts. Then, a plurality of virtual GPU device files are created on the host according to the plurality of target physical GPUs, a soft link between each virtual GPU device file and the corresponding target physical GPU is established, the development machine container is started in response to a creation completion instruction of the virtual GPU device file, and the virtual GPU device file is mounted into the container. Through the virtual GPU device file, dynamic mounting of the GPU resource is realized, and the container can dynamically obtain and use the GPU resource according to the actual demand, instead of being allocated with a fixed number of GPU resources at the beginning, thereby improving the flexibility of resource allocation. Finally, when the container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle. This mechanism can monitor the use of the GPU resource in real time, and when the GPU resource in a container is idle, it is released in time, so that these idle resources can be used by other tasks that need them, avoiding waste of resources. At the same time, this dynamic resource release method also enhances the elastic scheduling capability of the system. When a new high-priority task needs to use the GPU resource, the system can allocate the idle GPU resource in time to avoid task delay or failure, thereby improving the overall development efficiency. The method supports dynamic linking / disabling of real GPU resources during container operation, which maximizes the reuse efficiency of GPU devices in the cluster.

[0011] The above description is only a summary of the technical solutions of the present disclosure. In order to more clearly understand the technical means of the present disclosure, the content of the specification can be implemented, and in order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and all other drawings obtained by those of ordinary skill in the art based on the drawings without creative labor should also be considered as the scope of the present disclosure.

[0013] Figure 1 The flowchart of the method for dynamically regulating GPU resources provided by the embodiments of the present disclosure is shown.

[0014] Figure 2 The corresponding architecture diagram is shown. Figure 1 The corresponding architecture diagram is shown.

[0015] Figure 3 The flowchart of the method for triggering the execution of the GPU resource dynamic regulation strategy provided by the embodiments of the present disclosure is shown.

[0016] Figure 4 The corresponding architecture diagram is shown. Figure 3 The corresponding architecture diagram is shown.

[0017] Figure 5 The structure diagram of a computer device provided by the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0019] It should be clear that the embodiments of the present disclosure are described below through specific and concrete examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, not all. The present disclosure can also be implemented or applied by other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor should be considered as the scope of the present disclosure.

[0020] It should be apparent that the aspects described herein can be embodied in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that an aspect described herein can be implemented both as any claim and as an combination of claims. For example, an apparatus can be implemented using any number of the aspects described herein. Additionally, an apparatus can be implemented using other structure and / or functionality not expressly described herein. Additionally, it should be understood that any logic or information described herein can be implemented in hardware, software, firmware, or any combination thereof.

[0021] It should also be noted that the figures provided in the following embodiments are only to schematically illustrate the basic concept of the present disclosure, and only the components related to the present disclosure are shown in the figures, not drawn according to the number, shape and size of the components when actually implemented, and the shape, number and ratio of each component when actually implemented can be a random change, and the layout pattern of the components can also be more complex.

[0022] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, one skilled in the art will understand that the aspects described can be practiced without these specific details.

[0023] Referring to Figure 1 The first aspect of the present application discloses a GPU resource dynamic regulation method, comprising: S100, in response to a user-initiated development machine creation request, creating a custom resource object containing development machine configuration information according to the custom resource specification of the development machine.

[0024] Through the custom resource object, the user's development machine creation requirements can be standardized and encapsulated, facilitating subsequent system unified processing and management of these information; at the same time, the custom resource specification can be flexibly adjusted according to different business requirements, improving the scalability of the system.

[0025] Specifically, the user can initiate a development machine creation request to the system through a graphical interface or a command line tool, and at the same time specify the required resources of the development machine in the request, such as CPU core number, memory size, GPU number, etc.; after the system receives the request, according to the pre-defined custom resource specification, the configuration information is sorted into a custom resource object, for example, a JSON or YAML format file, containing the name of the development machine, the required resource specification, the type of operating system and other detailed information.

[0026] S200, create a corresponding Pod in the Kubernetes cluster according to the custom resource object, denoted as the target Pod.

[0027] Using Kubernetes to create and manage Pods can take full advantage of its features such as automated deployment, elastic scaling, self-repairing, etc., improving the reliability and availability of development machines.

[0028] S300, determine the target GPU node to be scheduled for the target Pod.

[0029] By reasonably scheduling the target Pod to the appropriate GPU node, the GPU resources in the cluster can be fully utilized, avoiding waste and uneven allocation of resources. At the same time, scheduling the Pod to a node with low load can improve the performance of the development machine.

[0030] Specifically, the Kubernetes scheduler will filter out nodes with available GPU resources according to the resource requirements of the target Pod (especially GPU requirements); the scheduler will consider factors such as CPU load, memory usage, GPU quantity and model, etc., to select the most suitable node as the target GPU node. For example, if the target Pod requires 2 NVIDIA Tesla V100 GPUs, the scheduler will find nodes that have at least 2 GPUs of that model and are low in load.

[0031] S400, obtain the GPU application quantity corresponding to the target Pod according to the target GPU node.

[0032] Determining the GPU application quantity of the target Pod is the basis for subsequent GPU resource allocation. Only by accurately obtaining this information can we ensure that the target Pod is allocated sufficient GPU resources.

[0033] Specifically, the required GPU quantity can be extracted from the configuration information of the target Pod.

[0034] S500, determine a number of target physical GPUs according to the GPU application quantity.

[0035] By reasonably selecting the target physical GPU, we can ensure that the target Pod obtains the most optimal GPU resources, improving the computing efficiency of the development machine.

[0036] Specifically, on the target GPU node, we can query the list of physical GPUs on the node, and select appropriate physical GPUs according to the GPU application quantity. Specifically, we can select GPUs according to factors such as usage status, performance indicators, etc., for example, we can preferentially select idle and high-performance GPUs.

[0037] S600, create a plurality of virtual GPU device files on the host according to the plurality of target physical GPUs.

[0038] The virtual GPU device file can finely divide the physical GPU resources, so that multiple Pods can share the same physical GPU, improving the utilization of GPU resources.

[0039] Specifically, a GPU virtualization technology can be used to create a corresponding virtual GPU device file on the host of the target GPU node for each target physical GPU. For example, using the NVIDIA vGPU technology, one or more virtual GPU instances are created for each target physical GPU by calling the relevant management tool, and the corresponding device file is generated.

[0040] S700, establish a soft link between each virtual GPU device file and the corresponding target physical GPU.

[0041] The soft link establishes a mapping relationship between the virtual GPU device file and the physical GPU, so that the development machine container can access the corresponding physical GPU resources through the virtual GPU device file.

[0042] Specifically, on the host, a soft link pointing to the corresponding target physical GPU can be created for each virtual GPU device file through file system operations. For example, the ln-s command is used to create a soft link in the Linux system, and the virtual GPU device file is associated with the device file of the physical GPU.

[0043] S800, in response to the creation completion instruction of the virtual GPU device file, start the development machine container and mount the virtual GPU device file into the container.

[0044] Mounting the virtual GPU device file into the container allows the development machine container to directly access and use the physical GPU resources, enabling dynamic mounting of GPU resources.

[0045] Specifically, when the system receives a notification that the creation of the virtual GPU device file is complete, it will start the development machine container in the target Pod; during the container startup process, the mounting function of the container runtime (such as Docker) is used to mount the virtual GPU device file on the host to a specified path inside the container. For example, the / dev / nvidia-vgpu0 device file on the host is mounted to the / dev / nvidia0 path inside the container.

[0046] S900, when the obtained container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle.

[0047] By dynamically releasing the idle GPU resources, the utilization rate of the GPU resources can be improved, and the waste of resources can be avoided.

[0048] The GPU resource dynamic regulation method disclosed in the application can realize fine-grained division and sharing of GPU resources through GPU dynamic mounting and resource dynamic regulation, so that multiple development machines can share the same physical GPU, and the utilization rate of GPU resources can be improved. Users can dynamically adjust the GPU resource configuration of the development machine according to actual needs, and the system can quickly respond and perform resource allocation and release, thereby improving the flexibility and adaptability of the system. Since the utilization rate of GPU resources is improved, the demand for physical GPU devices is reduced, thereby reducing the hardware procurement and maintenance costs. The development machine can quickly obtain the required GPU resources and release them in time when the resources are idle, thereby avoiding resource waiting and waste, significantly improving the utilization rate of development machine GPU resources, reducing idle costs, improving development efficiency, and effectively applicable to development scenes with long-term keep-alive characteristics.

[0049] The scheme does not rely on traditional task scheduling algorithms to solve the GPU resource allocation problem, but directly starts from the bottom mechanism of resource allocation through the way of GPU dynamic mounting. The resource demand of the development machine has uncertainty and dynamics, and the traditional scheduling algorithm is difficult to accurately predict and respond to these changes in real time. The scheme can allocate and release GPU resources according to the actual demand of the development machine in real time by dynamically creating and mounting virtual GPU device files, and better adapt to the scene of long-term keep-alive and dynamic change of resource demand of the development machine. The development machine GPU resource dynamic regulation method disclosed in the application effectively solves the problems of resource waste, lack of flexible scheduling ability and poor effect of traditional scheduling algorithms in the development scene caused by the static allocation of GPU resources in the prior art through a series of specific technical means.

[0050] For the method of S200, "creating a corresponding Pod in the Kubernetes cluster according to the custom resource object", specifically comprising: S210, sending the custom resource object to the Kubernetes cluster; S220, verifying the custom resource object according to the preset custom resource definition, and when the verification is passed, creating a corresponding Pod in the Kubernetes cluster according to the configuration information in the custom resource object, which is recorded as a target Pod.

[0051] If the custom resource object is stored in the form of a YAML file, developers can use the Kubernetes command-line tool `kubectl` to send the object. For example, enter kubectl apply -f custom_resource.yaml in the terminal, where custom_resource.yaml is the file containing the custom resource object. kubectl will send the file contents to the API server of the Kubernetes cluster. If using a programmatic approach, you can use the client library provided by Kubernetes (such as Python's kubernetes-client) to construct HTTP requests through code and send the contents of the custom resource object to the corresponding endpoint of the API server to achieve the connection between user needs and the Kubernetes cluster. After the custom resource object is sent to the cluster, the cluster can perform subsequent operations based on this information, laying the foundation for creating a pod that meets user needs; using Kubernetes' existing tools and client libraries, operations are simple and standardized, reducing the complexity of interacting with the cluster.

[0052] Among them, the preset custom resource definition (CRD) specifies the structure and rules of custom resource objects. The API server of the Kubernetes cluster will verify the received custom resource objects based on these rules. For example, CRD stipulates that the custom resource object must contain the gpu_count field, and the value of this field must be a positive integer; when receiving the custom resource object, the API server will check whether the field exists and whether its value meets the requirements. If the verification passes, the API server will call the Kubernetes scheduler and controller based on the configuration information in the custom resource object, such as container images, environment variables, resource requests, etc., to create the corresponding target Pod in the cluster. The scheduler will select the appropriate node based on resource requirements, and the controller will ensure that the Pod runs as expected.

[0053] In this embodiment, the verification mechanism ensures the legitimacy and integrity of custom resource objects. Only objects that meet pre-set rules pass verification, preventing Pod creation failures or abnormal operations caused by incorrect or incomplete configuration information. Creating Pods based on verified configuration information ensures that the target Pods accurately meet user requirements, improving the success rate and reliability of Pod creation.

[0054] The method disclosed in the embodiment ensures that the created target Pod fully meets the configuration requirements of the user by sending the custom resource object to the Kubernetes cluster and verifying, reduces problems caused by configuration errors, and improves the accuracy of Pod creation; the verification mechanism can discover and prevent configuration information that does not meet the rules from entering the system in advance, avoids the impact of incorrect configuration on the stability of the cluster, and enhances the stability of the system; users can flexibly configure various parameters of the Pod through the custom resource object, and the system can accurately understand and execute these configurations, providing more convenient and personalized services for users and improving user experience; using standardized custom resource objects and verification mechanisms makes the creation and management of Pods more standardized, facilitating unified management and maintenance by system administrators.

[0055] For the method of S900 "when the obtained container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle", specifically comprising: when the obtained container GPU resource idle information is 0 in the preset duration period, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle; the released real GPU reenters the Kubernetes resource scheduling pool.

[0056] Setting the preset duration period and requiring the video memory and core usage rate to be 0 in the period can effectively avoid the false release of GPU resources due to temporary resource idleness. In actual operation, containers may have momentary resource idleness due to some temporary waiting or data transmission gaps. If there is no duration period limit, resource release and reallocation may be performed frequently, which not only increases system overhead but also may affect the normal operation of the container; only when the video memory and core usage rate are both 0 and last for a certain time, it is considered that the GPU resource is truly idle, which ensures that the container does not need to use the GPU before releasing the resource, thereby avoiding abnormal operation of the container due to premature release of the resource. This resource release mechanism based on explicit conditions makes the management of GPU resources more stable and reliable, can accurately determine when to release resources according to the preset rules, reduces human intervention and uncertainty, and improves the stability of the entire system.

[0057] After the released real GPU is put back into the Kubernetes resource scheduling pool, these idle GPUs can be used again by other needed Pods, which realizes the dynamic allocation and recycling of GPU resources, improves the overall utilization of GPU resources, and avoids the waste of idle resources. The Kubernetes resource scheduling pool can dynamically allocate GPU resources according to the resource needs and priorities of each Pod. When there is a new Pod that needs GPU resources, the scheduler can select appropriate GPUs from the resource scheduling pool for allocation, so that the system can better adapt to different workloads and business needs, enhancing the flexibility and flexibility of the system; at the same time, by improving the utilization of GPU resources, the need for additional GPU hardware is reduced, thereby reducing hardware procurement and maintenance costs, and enterprises can support more businesses with limited hardware resources, improving the return on investment. This dynamic release and rescheduling mechanism based on the idle state of GPU resources can significantly improve the utilization of GPU resources, reduce costs, and enhance the stability, flexibility and flexibility of the system, enabling the Kubernetes cluster to run more efficiently and provide strong support for the business development of enterprises.

[0058] Further, with reference to Figure 1 and with reference to Figure 2 In a specific embodiment, when a user creates a development machine, a CR (Custom Resource) for creating a DevMachine (development machine) is submitted to K8s, and the DevMachine Operator generates a Pod corresponding to the development machine. K8s will schedule the Pod to a certain GPU node.

[0059] The Kubelet component on the GPU node will List / Watch the creation event of the development machine Pod, and call the Device Plugin to apply for the GPU according to the number of GPUs applied by the development machine Pod.

[0060] The Scheduler component in the Device Plugin searches for the corresponding number of physical GPUs according to the idle state of the physical GPUs on the node, and calls the Proxy component for initialization; The Proxy component creates a virtual GPU device file on the host and establishes a soft link with the corresponding physical GPU; The Device Plugin returns the virtual GPU device file to the Kubelet, and the Kubelet component starts the development machine container and mounts the virtual GPU device file to the container. At this time, the GPU device file mounted to the container points to the real physical GPU device, so the physical GPU device can be accessed and used through the virtual GPU device file.

[0061] Device Plugin periodically monitors the GPU usage (memory usage, core utilization, etc.) in the container in the background; and reports the monitoring data to Promethues. Device Plugin is another container orchestration technology that allows virtualization of physical devices (such as GPUs, FPGAs, or storage devices) as resources available to containers. Device Plugin is usually integrated with container orchestration engines (such as Kubernetes) to provide access and management of device resources. This allows containers to use device resources without direct access to device hardware, thereby improving the flexibility and portability of container deployment.

[0062] The DevMachine Controller component in the DevMachine Operator periodically obtains the GPU usage monitoring data of the corresponding development machine container from Promethues, and detects when the container GPU resources are idle. It notifies the Proxy component, which removes the soft link between the virtual GPU device file and the real GPU. It also sets the real GPU usage status to idle. The released real GPU resources re-enter the pending pool for use by other development machines or training tasks.

[0063] This scheme proposes a dynamic and flexible GPU resource allocation and management mechanism to address the problems of existing static GPU resource allocation. By periodically monitoring the GPU usage in the container through Device Plugin, when the container GPU resources are detected to be idle, the soft link between the virtual GPU device file and the real GPU is removed in a timely manner, and the real GPU resources are put back into the pending pool. This effectively avoids the long-term idle of GPU resources caused by static allocation, so that the idle GPU resources can be used by other development machines or training tasks that have needs, greatly improving the overall utilization of GPU resources.

[0064] During development, the resource requirements of development tasks are dynamically changing, and static allocation methods are prone to resource waste. This scheme can dynamically adjust according to the actual GPU resource usage, avoiding excessive allocation and idling of resources, and reducing unnecessary resource waste.

[0065] At the task execution level, the scheme can respond to the resource demand changes of the development machine in real time. When there are new high-priority tasks that need to use GPU resources, since the idle GPU resources will enter the pending scheduling pool in time, the system can quickly allocate the required resources for these tasks, avoiding task delays or inability to execute due to fixed resource occupation, and improving the efficiency of task execution. The development machine usually needs to remain in the running state for a long time to meet the needs of the developers to develop and debug at any time. The present scheme can dynamically allocate GPU resources according to the actual resource needs of the development machine, ensuring that the developers can obtain sufficient resource support in time when needed, thereby improving the development efficiency.

[0066] At the system architecture level, the physical GPU device is virtualized into a resource available to the container by using the Device Plugin, so that the container can use the GPU resources without directly accessing the device hardware, which improves the flexibility and portability of container deployment, and makes the development machine system better adapt to different hardware environments and application scenarios. The development machine corresponding to the Pod is generated by the DevMachine Operator, and the GPU resources are allocated and managed by the Device Plugin, realizing fine-grained control of the GPU resources. The system can accurately allocate the required GPU resources according to the specific needs of the development machine, avoiding unreasonable allocation of resources.

[0067] At the monitoring and management level, the Device Plugin periodically monitors the GPU usage in the container in the background and reports the monitoring data to Prometheus. The DevMachine Controller component can obtain the monitoring data from Prometheus at regular intervals, realizing real-time monitoring and feedback of the GPU resource usage, which helps the system administrator to understand the resource usage of the system in time, so as to make reasonable decisions.

[0068] The whole scheme realizes the automatic allocation and management of GPU resources. From the resource application when the development machine is created, to the resource monitoring and idle resource release during the running process, to the resource rescheduling, all are automatically completed by the system, reducing manual intervention and improving management efficiency.

[0069] The present application realizes the elastic management and dynamic scheduling of GPU resources by constructing a virtual GPU mounting and Device Plugin dynamic link control mechanism, effectively solves the problem of low utilization caused by long-term occupation of GPU resources by the development machine, has the advantages of simple implementation, strong deployment compatibility and good expansibility, is suitable for large-scale containerized GPU development platform, and has significant industrial value and promotion prospect.

[0070] Further reference Figure 3 andFigure 4 When the development machine needs to use the GPU again, the specific processing logic includes: in response to the GPU call request, triggering the execution of the GPU resource dynamic regulation strategy.

[0071] The method of triggering the execution of the GPU resource dynamic regulation strategy specifically includes: S10, generating an API call request for CUDA according to the GPU running task.

[0072] Specifically, the deep learning framework or other GPU application can automatically generate a CUDA API call request according to the task demand, without the need for the developer to manually write a large amount of underlying CUDA code, thereby improving the development efficiency; the scheme disclosed in this step can dynamically generate appropriate API call requests according to different tasks, fully utilize the functions of CUDA, and improve the computing performance of the GPU.

[0073] S20, in response to the generation message of the API call request, triggering the interception of the API call of CUDA.

[0074] In this step, the API call of CUDA can be intercepted and processed without modifying the application code, facilitating the dynamic regulation of GPU resources; the CUDA API call can be monitored to obtain relevant information such as the function name and parameters of the call, thereby providing a basis for subsequent resource regulation.

[0075] S30, judging whether the virtual GPU device file in the current development machine points to a physical GPU, and if so, forwarding the API call request to CUDA.

[0076] Specifically, if the virtual GPU device file has correctly pointed to the physical GPU, directly forwarding the API call request can reduce unnecessary processing steps, improve the use efficiency of the GPU, and ensure that the application program can normally use the CUDA library for GPU computing, and will not affect the normal operation of the application program due to the intervention of the resource regulation mechanism.

[0077] S40, if the virtual GPU device file in the current development machine does not point to a physical GPU, judging whether there is a physical GPU on the node initiating the GPU call request that meets the required number of physical GPUs of the development machine; if so, determining the corresponding physical GPU according to the required number of physical GPUs, and recording it as the target physical GPU.

[0078] In this step, the physical GPU resources can be dynamically allocated according to the actual demand of the development machine, avoiding waste and shortage of resources; when the virtual GPU device file does not correctly point to the physical GPU, the resource allocation is timely, ensuring that the development machine can normally use the GPU for computing.

[0079] S50, link the virtual GPU device file to the target physical GPU; S60, in response to the soft link completion instruction, forward the API call request to CUDA.

[0080] By soft linking, the application can transparently use physical GPU resources without worrying about the actual physical GPU device file path. The soft link can be changed at any time to achieve dynamic adjustment of GPU resources. After the soft link is created, the system will issue a soft link completion instruction. When the instruction is intercepted, the previously intercepted API call request will be forwarded to the CUDA library for processing. This ensures that the API call request is forwarded to CUDA after the virtual GPU device file correctly points to the physical GPU, avoiding errors caused by unprepared resources and ensuring that GPU computing of the application can proceed smoothly without interruption due to resource regulation.

[0081] Further, if the node that initiates the GPU call request does not have the required number of physical GPUs, a restart development machine strategy is triggered.

[0082] Specifically, triggering the restart development machine strategy includes deleting the corresponding development machine Pod and creating a new development machine Pod, and selecting a target node from the Kubernetes resource scheduling pool that meets the resource requirements of the development machine Pod to start the development machine Pod.

[0083] In this embodiment, it refers to in the Kubernetes cluster environment, when the physical GPU resources required by the development machine are insufficient on a certain node, the system will notify the DevMachine Controller component, which is a custom controller component responsible for managing the life cycle of the development machine and monitoring and maintaining the state of the development machine.

[0084] After receiving the resource shortage notification, the DevMachine Controller component will restart the development machine. The specific method is to delete the corresponding development machine Pod, which means terminating the current running development machine instance in Kubernetes. After deleting the old development machine Pod, the DevMachine Controller creates a new development machine Pod, which contains the containers and configuration information required for the development machine to run, and also declares resource requirements for a specific number of physical GPUs.

[0085] After the new development machine Pod is created, it will be submitted to the Scheduler component of Kubernetes for scheduling. The main responsibility of the Scheduler is to find a suitable node for the Pod to run on. At this time, the Scheduler will traverse all the nodes in the cluster and evaluate the resource availability of each node to select a node with sufficient physical GPU resources to start the new development machine Pod, thereby ensuring that the development machine can obtain the required resources to run normally.

[0086] Suppose a Kubernetes cluster has three nodes, Node A, Node B, and Node C, each with 2, 4, and 6 physical GPUs respectively. Now there is a development machine Pod that requires 5 physical GPUs to run. It is initially scheduled to Node A, but since Node A only has 2 physical GPUs, it cannot meet the development machine's requirements. The system will notify the DevMachineController, which will delete the development machine Pod and then recreate a new Pod. The Kubernetes Scheduler will evaluate the nodes in the cluster and ultimately select Node C to start the new development machine Pod, as Node C has 6 physical GPUs, which can meet the resource requirements of the development machine.

[0087] For the method of S20 "generating a message in response to an API call request, triggering interception of CUDA's API call", specifically comprising: S21, creating a custom dynamic link library, which contains functions with the same name as CUDA API; S22, intercepting CUDA's API call through the custom dynamic link library.

[0088] Specifically, a custom dynamic link library (.so file) can usually be created on a Linux system using C or C++ language, and the LD_PRELOAD environment variable can be used to load the custom dynamic link library. When the application calls the CUDA API, the system will first look for the same function in the custom dynamic link library and execute the code of the custom function.

[0089] This embodiment can enable or disable the interception function at any time without modifying the system configuration or application installation, only by setting or canceling the LD_PRELOAD environment variable. This method is suitable for various Linux-based systems and can be compatible with different versions of CUDA and applications, with wide applicability.

[0090] By intercepting CUDA's API calls, resource control logic can be added at the critical nodes of the calls, providing a foundation for subsequent GPU resource dynamic allocation and management. This allows for easy monitoring and debugging of CUDA API calls, and by adding logging or performance analysis code in custom functions, detailed call information can be obtained, helping developers better understand the GPU usage of the application and identify potential issues.

[0091] From a business perspective, the traditional GPU resource sales model is to sell the resources corresponding to several physical GPUs directly to customers. This model does not take into account the actual usage of customers. Some customers may not need to use all GPU resources during certain time periods, or even not use them at all, but the resources are still locked and cannot be utilized by other customers, resulting in low resource utilization and economic benefits.

[0092] The GPU resource dynamic control method disclosed in this application can effectively solve the drawbacks of the traditional GPU resource sales model, improve resource utilization, and significantly enhance economic benefits. Specifically, customers can flexibly customize the resource specifications of the development machine according to their actual needs and only apply for the required GPU resources. For example, during the initial stage of a project or during low-load periods, customers can reduce the number of GPUs applied for, avoiding the payment for idle resources, thereby reducing the cost of use. For small businesses or start-ups, this on-demand usage model allows them to obtain GPU resource support at a lower cost, reducing the entry threshold and promoting innovation and business development.

[0093] Customers can dynamically adjust the use of GPU resources according to different stages of the project and business needs. When the project enters a high-load stage, GPU resources can be increased; during low-load or idle periods, resource occupation can be reduced, without worrying about resource waste. For temporary computing tasks, customers can quickly create and destroy development machines to flexibly respond to various business scenarios, improving the response speed and flexibility of the business.

[0094] By dynamically allocating and recycling GPU resources, service providers can make full use of idle GPU resources. When the GPU resources of a certain customer are idle, the system will automatically remove the soft link and mark the corresponding real GPU as idle so that other customers can use it, avoiding waste and locking of resources. For example, in the traditional mode, a node with 10 physical GPUs may only have 3-4 fully utilized, while after adopting this scheme, the resource utilization rate can be increased to 7-8 or even higher, greatly improving the efficiency of resource use. Higher resource utilization means that service providers can serve more customers on the same hardware resources, thereby increasing revenue. For example, GPU resources that can only serve 10 customers can now serve 15-20 customers, directly improving economic efficiency. Due to efficient use of resources, service providers can reduce hardware procurement costs, and in the case of meeting the same business needs, they do not need to purchase too many GPU devices, reducing operating costs and improving profit margins.

[0095] At the same time, customers can obtain more flexible resource use experience at a lower cost, which will significantly improve customer satisfaction and loyalty. Satisfied customers are more likely to use the service provider's products for a long time and recommend them to other potential customers, bringing more business opportunities to service providers; service providers can provide more personalized resource solutions and services to customers based on their usage data, further enhancing customer stickiness and competitiveness.

[0096] The method provided by the application can effectively reduce the threshold for enterprises and developers to use GPU resources, enabling more innovative projects to obtain resource support. For example, some emerging artificial intelligence and machine learning projects can more easily conduct research and development, promoting technological innovation and development in the industry. At the same time, flexible resource use patterns encourage more start-ups and individual developers to participate in the GPU computing field, promoting industry diversification and competition and driving progress in the entire industry. The scheme realizes dynamic allocation and optimal configuration of GPU resources, enabling resources to flow to where they are most needed. From a macro perspective, this helps improve the resource utilization efficiency of the entire industry, reduces resource waste, and promotes sustainable development of the industry.

[0097] In a second aspect, the application discloses a development machine GPU resource dynamic regulation system for executing the development machine GPU resource dynamic regulation method disclosed in the first aspect of the application, and the system comprises: A self-defined resource object creation module is configured to, in response to a development machine creation request initiated by a user, create a self-defined resource object containing development machine configuration information according to self-defined resource specifications of the development machine. A pod creation module is configured to create a corresponding pod in a Kubernetes cluster according to a custom resource object, and the pod is referred to as a target pod; A target GPU node determination module is configured to determine a target GPU node to which the target pod is to be scheduled; A GPU application quantity acquisition module is configured to acquire a GPU application quantity corresponding to the target pod according to the target GPU node; A target physical GPU determination module is configured to determine a plurality of target physical GPUs according to the GPU application quantity; A virtual GPU device file creation module is configured to create a plurality of virtual GPU device files on a host according to the plurality of target physical GPUs; A linking module is configured to establish a soft link between each virtual GPU device file and a corresponding target physical GPU; A mounting module is configured to, in response to a creation completion instruction of the virtual GPU device file, start a development machine container and mount the virtual GPU device file into the container; A dynamic regulation module is configured to, when the acquired container GPU resource idle information satisfies a preset idle condition, cancel a soft link between a virtual GPU device file corresponding to the container GPU resource idle information and a real GPU, and update a use state of the corresponding real GPU to idle.

[0098] A computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is configured to store non-transitory computer readable instructions. Specifically, the memory can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0099] The processor can be a central processing unit (CPU) or other forms of processing units with data processing and / or instruction execution capabilities, and can control other components in the computer device to perform desired functions. In an embodiment of the present disclosure, the processor is configured to run the computer readable instructions stored in the memory, so that the computer device performs all or part of the steps of the development machine GPU resource dynamic regulation method of the embodiments of the present disclosure.

[0100] Those skilled in the art should understand that, in order to solve the technical problem of how to obtain a good user experience effect, the present embodiment can also include well-known structures such as a communication bus, an interface, etc., which should also be included in the protection scope of the present disclosure.

[0101] As Figure 5 A structural schematic diagram of a computer device is provided according to an embodiment of the present disclosure. It shows a structural schematic diagram of a computer device suitable for implementing the computer device in the embodiment of the present disclosure. Figure 5 The computer device shown is merely an example and should not impose any limitation on the functions and use range of the embodiment of the present disclosure.

[0102] As Figure 5 As shown, the computer device can include a processor (such as a central processor, a graphics processor, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) or loaded into a random access memory (RAM) from a storage device. In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0103] Generally, the following devices can be connected to the I / O interface: input devices including, for example, sensors or visual information collection devices, etc.; output devices including, for example, display screens, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. The communication devices can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Although Figure 5 The computer device with various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices can be implemented or provided instead.

[0104] In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device, or installed from the ROM. When the computer program is executed by the processor, all or part of the steps of the development machine GPU resource dynamic regulation method of the embodiments of the present disclosure are performed.

[0105] The detailed description of the present embodiment can refer to the corresponding description in the foregoing embodiments, which will not be repeated here.

[0106] The computer readable storage medium according to the embodiments of the present disclosure has non-transitory computer readable instructions stored thereon. When the non-transitory computer readable instructions are run by the processor, all or part of the steps of the development machine GPU resource dynamic regulation method of the embodiments of the present disclosure are performed.

[0107] The computer-readable storage medium described above includes, but is not limited to, an optical storage medium (for example, a CD-ROM and a DVD), a magneto-optical storage medium (for example, an MO), a magnetic storage medium (for example, a magnetic tape or a moving hard disk), a medium having a built-in rewritable nonvolatile memory (for example, a memory card), and a medium having a built-in ROM (for example, a ROM cartridge).

[0108] For detailed description of the present embodiment, reference can be made to the corresponding description in the foregoing embodiments, which will not be repeated here.

[0109] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present disclosure are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details of the above disclosure are only for the purpose of example and for the purpose of understanding, and are not limiting, and the above details do not limit the present disclosure to be necessarily implemented with the above specific details.

[0110] In the present disclosure, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. The block diagrams of devices, apparatuses, equipment, systems involved in the present disclosure are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0111] In addition, as used herein, "or" used in the list of items preceded by "at least one of" means a disjunctive list, such that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "example" does not mean that the described example is preferred or better than other examples.

[0112] It should also be noted that in the systems and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalents of the present disclosure.

[0113] Various changes, modifications, and alterations to the techniques described herein can be made without departing from the teachings of the attached claims. Moreover, the scope of the claims of this disclosure is not limited to the particular aspects described above. In addition, where a process, machine, manufacture, composition of matter, means, method, or result containing procedural, business, and other steps is described, it is understood that the description is meant to encompass the specific implementation of the steps described, as well as the substitution of equivalent steps, or equivalent steps in the performance order. Accordingly, the attached claims are to be interpreted as embracing the specific aspects and embodiments described herein, as well as future modifications, changes, and alterations of the aspects and embodiments.

[0114] The above description of the disclosed aspects is given for illustrative purposes only and is not intended to limit the scope of the disclosure. The aspects are described in terms of "preferred" embodiments and various modifications, alterations, and permutations of these preferred embodiments. These descriptions are not exhaustive and are intended to provide further examples. Accordingly, other alternatives, modifications, and variations should be apparent to those skilled in the art.

[0115] The above description has been given for illustrative purposes only and is not intended to limit the embodiments of the disclosure. Although various example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations of the aspects and embodiments.

Claims

1. A method for dynamically controlling GPU resources of a development machine, characterized in that: include: In response to a development machine creation request initiated by a user, a custom resource object containing configuration information of the development machine is created according to the custom resource specification of the development machine; According to the custom resource object, a corresponding Pod is created in the Kubernetes cluster and recorded as the target Pod; Determine the target GPU node to be scheduled for the target Pod; Obtain the GPU application quantity corresponding to the target Pod according to the target GPU node; Determine a number of target physical GPUs according to the GPU application quantity; Creating a plurality of virtual GPU device files on the host according to the plurality of target physical GPUs; Establishing a soft link between each virtual GPU device file and the corresponding target physical GPU; In response to the instruction to complete the creation of the virtual GPU device file, start the development machine container and mount the virtual GPU device file into the container; When the obtained container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is released, and the usage status of the corresponding real GPU is updated to idle.

2. The method for dynamically controlling the GPU resources of a development machine according to claim 1, wherein: Creating a corresponding Pod in the Kubernetes cluster based on the custom resource object includes: Send the custom resource object to the Kubernetes cluster; The custom resource object is verified according to the preset custom resource definition, and when the verification passes, a corresponding Pod is created in the Kubernetes cluster according to the configuration information in the custom resource object, which is recorded as the target Pod.

3. The method for dynamically controlling GPU resources of a development machine according to claim 1, wherein: When the obtained container GPU resource idle information meets a preset idle condition, releasing the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU, and updating the usage status of the corresponding real GPU to idle, including: When the memory and core usage rates of the obtained container GPU resource idle information are zero within a preset duration period, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is released, and the usage status of the corresponding real GPU is updated to idle; The released real GPU re-enters the Kubernetes resource scheduling pool.

4. The method for dynamically controlling the GPU resources of a development machine according to claim 3, wherein: Also includes: In response to GPU call requests, trigger the execution of GPU resource dynamic control strategy; The triggering and executing of the GPU resource dynamic control strategy includes: Generate API call requests to CUDA based on GPU running tasks; In response to a generation message of the API call request, triggering interception of a CUDA API call; Determine whether the virtual GPU device file in the current development machine points to the physical GPU. If so, forward the API call request to CUDA; If the virtual GPU device file in the current development machine does not point to a physical GPU, determine whether the node that initiates the GPU call request has the number of physical GPUs required by the development machine; If yes, determine the corresponding physical GPU according to the required number of physical GPUs and record it as the target physical GPU; Point the virtual GPU device file soft link to the target physical GPU; In response to the soft link completion instruction, the API call request is forwarded to CUDA.

5. The method for dynamically controlling the GPU resources of a development machine according to claim 4, wherein: If the node that initiates the GPU call request does not have the number of physical GPUs required by the development machine, the development machine restart strategy is triggered.

6. The method for dynamically controlling GPU resources of a development machine according to claim 5, characterized in that: The triggering execution of the restart development machine policy includes: deleting the Pod of the corresponding development machine, creating a new development machine Pod, and selecting a target node that meets the resource requirements of the development machine Pod from the Kubernetes resource scheduling pool to start the development machine Pod.

7. The method for dynamically controlling GPU resources of a development machine according to claim 4, wherein: The generating message in response to the API call request triggers intercepting the CUDA API call, including: Creating a custom dynamic link library, wherein the custom dynamic link library includes functions with the same names as the CUDA API; The CUDA API call is intercepted by the custom dynamic link library.

8. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for dynamically controlling GPU resources of a development machine as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the method for dynamically controlling GPU resources of a development machine as described in any one of claims 1-7.

10. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images