Method for dynamically regulating GPU resources, computer device and medium

Abstract

The application discloses a kind of development machine GPU resource dynamic regulation and control method, computer device, medium, wherein, method includes: response development machine creation request, create the custom resource object containing development machine configuration information, create target Pod in Kubernetes cluster, determine the target GPU node to be scheduled, obtain its GPU application quantity, then determine target physical GPU, after creating virtual GPU device file on host, establish the soft link between it and target physical GPU;After creation, start development machine container and mount virtual GPU device file into container;When monitoring container GPU resource idle information meets preset idle condition, remove soft link, update real GPU state to idle.The method does not need to modify container image and original K8s deployment logic, supports GPU resource sharing and reuse between multiple container development machines, significantly improves GPU resource utilization.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to a GPU resource dynamic regulation method for development machine, computer device and medium. BACKGROUND

[0002] In today's technology field, artificial intelligence (AI) development and deep learning training are booming at an unprecedented speed, and these advanced technology applications require computing resources. With the rise of container technology, tools such as Kubernetes and Docker are widely used in the deployment of development machine systems. However, there are serious limitations in the existing container deployment solutions in terms of GPU resource allocation.

[0003] Currently, in the development machine system based on Kubernetes or Docker, the allocation of GPU resources usually adopts a static allocation method. When the container starts, the system will allocate a certain amount of GPU resources to the container according to the pre-set configuration. This method exposes many problems in practical application. On the one hand, during the development process, due to the complexity and uncertainty of development tasks, the use of GPU resources is often dynamically changing. Many times, GPU resources will be in a long idle state, but since they have been statically allocated to specific containers, these idle resources cannot be utilized by other tasks that need them, resulting in serious resource waste. On the other hand, this static allocation method lacks flexibility in scheduling and cannot adjust the allocation strategy in real time according to the actual resource demand. When there is a new high-priority task that needs to use GPU resources, due to the fixed occupation of resources, it may not be able to allocate enough resources in time, resulting in task delay or failure to execute, and thus affecting the overall development efficiency.

[0004] Existing solutions mostly focus on optimizing the task scheduling layer. For example, by optimizing the task scheduling algorithm, it tries to allocate GPU resources more reasonably to improve resource utilization. However, these methods do not work well in development scenarios with long-term keep-alive characteristics. Development machines usually need to be kept running for a long time to meet the needs of developers to develop and debug at any time. In this case, the optimization of the task scheduling layer cannot solve the fundamental problem caused by static allocation because the resource demand of the development machine has uncertainty and dynamics, and traditional scheduling algorithms cannot accurately predict and respond to these changes in real time. SUMMARY

[0005] Therefore, the embodiments of the present disclosure provide a GPU resource dynamic regulation method for development machine, computer device and medium, which can support dynamic linking / disabling of real GPU resources during container operation, maximizing the reuse efficiency of GPU devices in the cluster.

[0006] In a first aspect, the embodiments of the present disclosure provide a GPU resource dynamic regulation method for a development machine, comprising:

[0007] In response to a user-initiated development machine creation request, a custom resource object containing development machine configuration information is created according to a custom resource specification of the development machine;

[0008] According to the custom resource object, a corresponding Pod is created in a Kubernetes cluster, denoted as a target Pod;

[0009] A target GPU node to be scheduled for the target Pod is determined;

[0010] According to the target GPU node, the number of GPUs applied by the target Pod is obtained;

[0011] According to the number of GPUs applied, a plurality of target physical GPUs are determined;

[0012] According to the plurality of target physical GPUs, a plurality of virtual GPU device files are created on a host;

[0013] A soft link is established between each virtual GPU device file and the corresponding target physical GPU;

[0014] In response to a creation completion instruction of the virtual GPU device file, a development machine container is started, and the virtual GPU device file is mounted into the container;

[0015] When the obtained container GPU resource idle information meets a preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is released, and the use state of the corresponding real GPU is updated to idle.

[0016] In a second aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical scheme:

[0017] The computer device comprises:

[0018] at least one processor; and

[0019] a memory in communication connection with the at least one processor; wherein

[0020] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the GPU resource dynamic regulation method for the development machine as described above.

[0021] In a third aspect, the embodiments of the present disclosure further provide a computer readable storage medium storing computer instructions for causing a computer to execute the GPU resource dynamic regulation method described above.

[0022] In a fourth aspect, the embodiments of the present disclosure further provide a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the method described above.

[0023] The GPU resource dynamic regulation method disclosed in the application first responds to a development machine creation request initiated by a user, creates a custom resource object containing development machine configuration information according to a custom resource specification of the development machine, creates a corresponding Pod in a Kubernetes cluster according to the custom resource object, and records the Pod as a target Pod, which provides a basis for subsequent dynamic allocation of GPU resources. Through the custom resource object, various resources required by the development machine can be flexibly configured, including relevant information of the GPU resource, instead of fixed configuration as in traditional static allocation. Then, the target GPU node to be scheduled by the target Pod is determined, the number of GPU applications corresponding to the target Pod is obtained according to the target GPU node, and a plurality of target physical GPUs are determined according to the number of GPU applications. This way makes the allocation of GPU resources closely related to the specific Pod demand, and can dynamically select appropriate GPU resources according to the actual development task demand, instead of fixed static allocation when the container starts. Then, a plurality of virtual GPU device files are created on the host according to the plurality of target physical GPUs, a soft link between each virtual GPU device file and the corresponding target physical GPU is established, the development machine container is started in response to a creation completion instruction of the virtual GPU device file, and the virtual GPU device file is mounted into the container. Through the virtual GPU device file, dynamic mounting of the GPU resource is realized, and the container can dynamically obtain and use the GPU resource according to the actual demand, instead of being allocated with a fixed number of GPU resources at the beginning, thereby improving the flexibility of resource allocation. Finally, when the container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle. This mechanism can monitor the use of the GPU resource in real time, and when the GPU resource in a container is idle, it is released in time, so that these idle resources can be used by other tasks that need them, avoiding waste of resources. At the same time, this dynamic resource release method also enhances the elastic scheduling capability of the system. When a new high-priority task needs to use the GPU resource, the system can allocate the idle GPU resource in time to avoid task delay or failure, thereby improving the overall development efficiency. The method supports dynamic linking / disabling of real GPU resources during container operation, which maximizes the reuse efficiency of GPU devices in the cluster.

[0024] The above description is only a summary of the technical solutions of the present disclosure. In order to more clearly understand the technical means of the present disclosure, the content of the specification can be implemented, and in order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and all other drawings obtained by those of ordinary skill in the art based on the drawings without creative labor should also be considered as the scope of the present disclosure.

[0026] Figure 1 The flowchart of the method for dynamically regulating GPU resources provided by the embodiments of the present disclosure is shown.

[0027] Figure 2 The corresponding architecture diagram is shown. Figure 1 The corresponding architecture diagram is shown.

[0028] Figure 3 The flowchart of the method for triggering the execution of the GPU resource dynamic regulation strategy provided by the embodiments of the present disclosure is shown.

[0029] Figure 4 The corresponding architecture diagram is shown. Figure 3 The corresponding architecture diagram is shown.

[0030] Figure 5 The structure diagram of a computer device provided by the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0031] The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0032] It should be clear that the following describes the embodiments of the present disclosure through specific and concrete examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, not all. The present disclosure can also be implemented or applied by other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor should be considered as the scope of the present disclosure.

[0033] It should be apparent that the aspects described herein can be embodied in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that an aspect described herein can be implemented both as any claim and as an combination of claims. For example, an apparatus can be implemented using any number of the aspects described herein. Additionally, an apparatus can be implemented using other structure and / or functionality not expressly described herein. Additionally, it should be understood that any logic or information described herein can be implemented in software and / or firmware and / or hardware, and that one skilled in the art could implement such software and / or firmware and / or hardware using the disclosure provided herein as a starting point.

[0034] It should also be noted that the figures provided in the following embodiments are only to schematically illustrate the basic concepts of the present disclosure, and only the components related to the present disclosure are shown in the figures, not drawn according to the number, shape and size of the components in actual implementation, and the shape, number and ratio of each component in actual implementation can be a random change, and the layout of the components can be more complex.

[0035] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, one skilled in the art will understand that the aspects described can be practiced without these specific details.

[0036] Referring to Figure 1 The first aspect of the present application discloses a GPU resource dynamic regulation method, comprising:

[0037] S100, in response to a user-initiated development machine creation request, creating a custom resource object containing development machine configuration information according to the custom resource specification of the development machine.

[0038] Through the custom resource object, the user's development machine creation requirements can be standardized and encapsulated, facilitating subsequent system unified processing and management of these information; at the same time, the custom resource specification can be flexibly adjusted according to different business requirements, improving the scalability of the system.

[0039] Specifically, the user can initiate a development machine creation request to the system through a graphical interface or a command line tool, and at the same time specify the required resources of the development machine in the request, such as CPU core number, memory size, GPU number, etc.; after the system receives the request, according to the pre-defined custom resource specification, the configuration information is sorted into a custom resource object, for example, a JSON or YAML format file, containing the name of the development machine, the required resource specification, the type of operating system and other detailed information.

[0040] S200, create a corresponding Pod in the Kubernetes cluster according to the custom resource object, denoted as the target Pod.

[0041] Using Kubernetes to create and manage Pods can take full advantage of its features such as automated deployment, elastic scaling, self-repairing, etc., improving the reliability and availability of development machines.

[0042] S300, determine the target GPU node to be scheduled for the target Pod.

[0043] By reasonably scheduling the target Pod to the appropriate GPU node, the GPU resources in the cluster can be fully utilized, avoiding waste and uneven allocation of resources. At the same time, scheduling the Pod to a node with low load can improve the performance of the development machine.

[0044] Specifically, the Kubernetes scheduler will filter out nodes with available GPU resources according to the resource requirements of the target Pod (especially GPU requirements) in the cluster; the scheduler will consider factors such as CPU load, memory usage, GPU quantity and model, etc., to select the most suitable node as the target GPU node. For example, if the target Pod requires 2 NVIDIA Tesla V100 GPUs, the scheduler will find nodes that have at least 2 GPUs of that model and are low in load.

[0045] S400, obtain the GPU application quantity corresponding to the target Pod according to the target GPU node.

[0046] Determining the GPU application quantity of the target Pod is the basis for subsequent GPU resource allocation. Only by accurately obtaining this information can we ensure that the target Pod is allocated sufficient GPU resources.

[0047] Specifically, the required GPU quantity can be extracted from the configuration information of the target Pod.

[0048] S500, determine a number of target physical GPUs according to the GPU application quantity.

[0049] By reasonably selecting the target physical GPU, we can ensure that the target Pod obtains the most optimal GPU resources, improving the computing efficiency of the development machine.

[0050] Specifically, on the target GPU node, we can query the list of physical GPUs on the node, and select appropriate physical GPUs according to the GPU application quantity. Specifically, we can select GPUs according to factors such as usage status, performance indicators, etc., for example, we can preferentially select idle and high-performance GPUs.

[0051] S600, create a plurality of virtual GPU device files on the host according to the plurality of target physical GPUs.

[0052] The virtual GPU device file can finely divide the physical GPU resources, so that multiple Pods can share the same physical GPU, improving the utilization of GPU resources.

[0053] Specifically, a GPU virtualization technology can be used to create a corresponding virtual GPU device file on the host of the target GPU node for each target physical GPU. For example, using the NVIDIA vGPU technology, one or more virtual GPU instances are created for each target physical GPU by calling the relevant management tool, and the corresponding device file is generated.

[0054] S700, establish a soft link between each virtual GPU device file and the corresponding target physical GPU.

[0055] The soft link establishes a mapping relationship between the virtual GPU device file and the physical GPU, so that the development machine container can access the corresponding physical GPU resources through the virtual GPU device file.

[0056] Specifically, on the host, a soft link pointing to the corresponding target physical GPU can be created for each virtual GPU device file through file system operations. For example, the ln-s command is used to create a soft link in the Linux system, and the virtual GPU device file is associated with the device file of the physical GPU.

[0057] S800, in response to the creation completion instruction of the virtual GPU device file, start the development machine container, and mount the virtual GPU device file into the container.

[0058] Mounting the virtual GPU device file into the container allows the development machine container to directly access and use the physical GPU resources, enabling dynamic mounting of GPU resources.

[0059] Specifically, when the system receives the notification that the creation of the virtual GPU device file is completed, the development machine container in the target Pod is started; during the container startup process, the mounting function of the container runtime (such as Docker) is used to mount the virtual GPU device file on the host to a specified path inside the container. For example, the / dev / nvidia-vgpu0 device file on the host is mounted to the / dev / nvidia0 path inside the container.

[0060] S900, when the obtained container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle.

[0061] By dynamically releasing the idle GPU resources, the utilization rate of the GPU resources can be improved, and the waste of resources can be avoided.

[0062] The GPU resource dynamic regulation method disclosed in the application can realize fine-grained division and sharing of GPU resources through GPU dynamic mounting and resource dynamic regulation, so that multiple development machines can share the same physical GPU, and the utilization rate of GPU resources can be improved. Users can dynamically adjust the GPU resource configuration of the development machine according to actual needs, and the system can quickly respond and perform resource allocation and release, thereby improving the flexibility and adaptability of the system. Since the utilization rate of GPU resources is improved, the demand for physical GPU devices is reduced, thereby reducing the hardware procurement and maintenance costs. The development machine can quickly obtain the required GPU resources and release them in time when the resources are idle, thereby avoiding resource waiting and waste, significantly improving the utilization rate of development machine GPU resources, reducing idle costs, improving development efficiency, and effectively applicable to development scenarios with long-term keep-alive characteristics.

[0063] The scheme does not rely on traditional task scheduling algorithms to solve the GPU resource allocation problem, but directly starts from the bottom mechanism of resource allocation through the way of GPU dynamic mounting. The resource demand of the development machine has uncertainty and dynamics, and the traditional scheduling algorithm is difficult to accurately predict and respond to these changes in real time. The scheme can allocate and release GPU resources according to the actual needs of the development machine in real time by dynamically creating and mounting virtual GPU device files, and better adapt to the scene of long-term keep-alive and dynamic changes of resource demand of the development machine. The development machine GPU resource dynamic regulation method disclosed in the application effectively solves the problems of resource waste, lack of flexible scheduling ability and poor effect of traditional scheduling algorithms in the development scene caused by the static allocation of GPU resources in the prior art through a series of specific technical means.

[0064] For the method of S200, "creating a corresponding Pod in the Kubernetes cluster according to the custom resource object", specifically includes:

[0065] S210, sending the custom resource object to the Kubernetes cluster;

[0066] S220, verifying the custom resource object according to the preset custom resource definition, and when the verification is passed, creating a corresponding Pod, denoted as a target Pod, in the Kubernetes cluster according to the configuration information in the custom resource object.

[0067] If the custom resource object is stored in a YAML file, the developer can use the command-line tool of Kubernetes, kubectl, to send the object. For example, enter kubectl apply -f custom_resource.yaml in the terminal, where custom_resource.yaml is the file containing the custom resource object. kubectl will send the file content to the API server of the Kubernetes cluster. If using a programming approach, you can use the client library provided by Kubernetes, such as kubernetes-client for Python, to build an HTTP request through code and send the content of the custom resource object to the corresponding endpoint of the API server to connect the user's demand with the Kubernetes cluster. After sending the custom resource object to the cluster, the cluster can perform subsequent operations based on this information to lay the foundation for creating a Pod that meets the user's needs. Using the existing tools and client libraries of Kubernetes, the operation is simple and standardized, reducing the complexity of interacting with the cluster.

[0068] The predefined custom resource definition (CRD) specifies the structure and rules of the custom resource object. The API server of the Kubernetes cluster will verify the received custom resource object based on these rules. For example, the CRD specifies that the custom resource object must contain the gpu_count field, and the value of this field must be a positive integer. When receiving the custom resource object, the API server will check whether this field exists and whether its value meets the requirements. If the verification is passed, the API server will create the corresponding target Pod in the cluster based on the configuration information in the custom resource object, such as container image, environment variable, resource request, etc. The scheduler will select the appropriate node based on resource requirements, and the controller will ensure that the Pod runs as expected.

[0069] In this embodiment, the verification mechanism ensures the legality and integrity of the custom resource object. Only objects that meet the preset rules can pass the verification, avoiding Pod creation failure or abnormal operation caused by incorrect or incomplete configuration information; creating a Pod based on the verified configuration information can ensure that the target Pod accurately meets the user's needs, improving the success rate and reliability of Pod creation.

[0070] The method disclosed in the embodiment ensures that the created target Pod fully meets the configuration requirements of the user by sending the custom resource object to the Kubernetes cluster and verifying, reduces problems caused by configuration errors, and improves the accuracy of Pod creation; the verification mechanism can discover and prevent configuration information that does not meet the rules from entering the system in advance, avoids the impact of incorrect configuration on the stability of the cluster, and enhances the stability of the system; users can flexibly configure various parameters of the Pod through the custom resource object, and the system can accurately understand and execute these configurations, providing more convenient and personalized services for users and improving user experience; using standardized custom resource objects and verification mechanisms makes the creation and management of Pods more standardized, facilitating unified management and maintenance by system administrators.

[0071] For the method of S900 "when the obtained container GPU resource idle information meets the preset idle condition, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle", specifically comprising: when the obtained container GPU resource idle information is 0 in the preset duration period, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the use state of the corresponding real GPU is updated to idle; the released real GPU reenters the Kubernetes resource scheduling pool.

[0072] Setting a preset duration period and requiring the video memory and core usage rate to be 0 in the period can effectively avoid the misrelease of GPU resources due to temporary resource idleness. In actual operation, containers may have momentary resource idleness due to some temporary waiting or data transmission gaps. If there is no duration period limit, resource release and reallocation may be performed frequently, which not only increases system overhead but also may affect the normal operation of the container; only when the video memory and core usage rate are both 0 and last for a certain time, it is considered that the GPU resource is truly idle, which ensures that the container does not need to use the GPU before releasing the resource, thereby avoiding abnormal operation of the container due to premature release of the resource. This resource release mechanism based on explicit conditions makes the management of GPU resources more stable and reliable, can accurately determine when to release resources according to the preset rules, reduces human intervention and uncertainty, and improves the stability of the entire system.

[0073] After the released real GPU is put back into the Kubernetes resource scheduling pool, these idle GPUs can be used again by other needed Pods, which realizes the dynamic allocation and recycling of GPU resources, improves the overall utilization of GPU resources, and avoids the waste of idle resources. The Kubernetes resource scheduling pool can dynamically allocate GPU resources according to the resource needs and priorities of each Pod. When there is a new Pod that needs GPU resources, the scheduler can select appropriate GPUs from the resource scheduling pool for allocation, so that the system can better adapt to different workloads and business needs, enhancing the flexibility and flexibility of the system; at the same time, by improving the utilization of GPU resources, the need for additional GPU hardware is reduced, thereby reducing hardware procurement and maintenance costs, and enterprises can support more businesses with limited hardware resources, improving the return on investment. This dynamic release and rescheduling mechanism based on the idle state of GPU resources can significantly improve the utilization of GPU resources, reduce costs, and at the same time enhance the stability, flexibility and flexibility of the system, so that the Kubernetes cluster can run more efficiently, providing strong support for the business development of enterprises.

[0074] Further, with reference to Figure 1 With reference to Figure 2 In a specific embodiment, when a user creates a development machine, a CR (Custom Resource) for creating a DevMachine (development machine) is submitted to K8s, and the DevMachine Operator generates a Pod corresponding to the development machine. K8s will schedule the Pod to a certain GPU node.

[0075] The Kubelet component on the GPU node will List / Watch the creation event of the development machine Pod, and call the Device Plugin to apply for the GPU according to the number of GPUs applied by the development machine Pod.

[0076] The Scheduler component in the Device Plugin finds the corresponding number of physical GPUs according to the idle state of the physical GPUs on the node, and calls the Proxy component for initialization;

[0077] The Proxy component creates a virtual GPU device file on the host and establishes a soft link with the corresponding physical GPU;

[0078] The Device Plugin returns the virtual GPU device file to the Kubelet, and the Kubelet component starts the development machine container and mounts the virtual GPU device file into the container. At this time, the GPU device file mounted into the container points to the real physical GPU device, so the physical GPU device can be accessed and used through the virtual GPU device file.

[0079] The Device Plugin periodically monitors the GPU usage (memory usage, core utilization, etc.) in the container in the background; and reports the monitoring data to Promethues. Device Plugin is another container orchestration technology that allows virtualization of physical devices (such as GPUs, FPGAs, or storage devices) as resources available to containers. Device Plugin is usually integrated with container orchestration engines (such as Kubernetes) to provide access and management of device resources. This allows containers to use device resources without directly accessing device hardware, thereby improving the flexibility and portability of container deployment.

[0080] The DevMachine Controller component in the DevMachine Operator periodically obtains the GPU usage monitoring data of the corresponding development machine container from Promethues, and detects when the container GPU resources are idle. When the container GPU resources are idle, the Proxy component is notified to remove the soft link between the virtual GPU device file and the real GPU; and the use status of the real GPU is set to idle; the released real GPU resources re-enter the pending pool for use by other development machines or training tasks.

[0081] This scheme proposes a dynamic and flexible GPU resource allocation and management mechanism to address the problems of existing static GPU resource allocation. Through periodic monitoring of GPU usage in the container by the Device Plugin background, when the container GPU resources are detected to be idle, the soft link between the virtual GPU device file and the real GPU is removed in a timely manner, and the real GPU resources are put back into the pending pool, effectively avoiding the long-term idle of GPU resources caused by static allocation, so that the idle GPU resources can be used by other development machines or training tasks that have needs, greatly improving the overall utilization of GPU resources.

[0082] During development, the resource requirements of development tasks are dynamically changing, and static allocation methods are prone to resource waste. This scheme can dynamically adjust according to the actual GPU resource usage, avoiding excessive allocation and idling of resources, and reducing unnecessary resource waste.

[0083] At the task execution level, the scheme can respond to the resource demand changes of the development machine in real time. When there are new high-priority tasks that need to use GPU resources, since the idle GPU resources will enter the pending scheduling pool in time, the system can quickly allocate the required resources for these tasks, avoiding task delays or inability to execute due to fixed resource occupation, and improving the efficiency of task execution. The development machine usually needs to remain in the running state for a long time to meet the needs of the developers to develop and debug at any time. The present scheme can dynamically allocate GPU resources according to the actual resource needs of the development machine, ensuring that the developers can obtain sufficient resource support in time when needed, thereby improving the development efficiency.

[0084] At the system architecture level, the physical GPU device is virtualized into a resource available to the container by using the Device Plugin, so that the container can use the GPU resources without directly accessing the device hardware, which improves the flexibility and portability of container deployment, and makes the development machine system better adapt to different hardware environments and application scenarios. The development machine corresponding to the Pod is generated by the DevMachine Operator, and the GPU resources are allocated and managed by the Device Plugin, realizing fine-grained control of the GPU resources. The system can accurately allocate the required GPU resources according to the specific needs of the development machine, avoiding unreasonable allocation of resources.

[0085] At the monitoring and management level, the Device Plugin periodically monitors the GPU usage in the container in the background and reports the monitoring data to Prometheus. The DevMachine Controller component can obtain the monitoring data from Prometheus at regular intervals, realizing real-time monitoring and feedback of the GPU resource usage, which helps the system administrator to understand the resource usage of the system in time, so as to make reasonable decisions.

[0086] The whole scheme realizes the automatic allocation and management of GPU resources. From the resource application when the development machine is created, to the resource monitoring and idle resource release during the running process, to the resource rescheduling, all are automatically completed by the system, reducing manual intervention and improving management efficiency.

[0087] The present application realizes the elastic management and dynamic scheduling of GPU resources by constructing a virtual GPU mounting and Device Plugin dynamic link control mechanism, effectively solves the problem of low utilization caused by long-term occupation of GPU resources by the development machine, has the advantages of simple implementation, strong deployment compatibility and good expansibility, is suitable for large-scale containerized GPU development platform, and has significant industrial value and promotion prospect.

[0088] Further reference Figure 3 andFigure 4 When the development machine needs to use the GPU again, the specific processing logic includes: in response to the GPU calling request, triggering the execution of the GPU resource dynamic regulation strategy.

[0089] The method of triggering the execution of the GPU resource dynamic regulation strategy specifically includes:

[0090] S10, generating an API calling request of CUDA according to the GPU running task.

[0091] Specifically, the deep learning framework or other GPU application program can be enabled to automatically generate the CUDA API calling request according to the task demand, without the need for the developer to manually write a large amount of underlying CUDA code, thereby improving the development efficiency; the scheme disclosed in this step can dynamically generate appropriate API calling requests according to different tasks, fully utilize the functions of CUDA, and improve the computing performance of the GPU.

[0092] S20, in response to the generation message of the API calling request, triggering the interception of the API calling of CUDA.

[0093] In this step, the API calling of CUDA can be intercepted and processed without modifying the application program code, which facilitates the dynamic regulation of the GPU resource; the CUDA API calling can be monitored to obtain relevant information of the calling, such as the function name, parameters, etc., thereby providing a basis for subsequent resource regulation.

[0094] S30, judging whether the virtual GPU device file in the current development machine points to the physical GPU, if yes, forwarding the API calling request to CUDA.

[0095] Specifically, if the virtual GPU device file has correctly pointed to the physical GPU, directly forwarding the API calling request can reduce unnecessary processing steps, improve the use efficiency of the GPU, and ensure that the application program can normally use the CUDA library for GPU computing, and the normal operation of the application program will not be affected due to the intervention of the resource regulation mechanism.

[0096] S40, if the virtual GPU device file in the current development machine does not point to the physical GPU, judging whether the node initiating the GPU calling request has the physical GPU quantity meeting the needs of the development machine; if yes, determining the corresponding physical GPU according to the required physical GPU quantity, and recording it as the target physical GPU.

[0097] In this step, the physical GPU resource can be dynamically allocated according to the actual needs of the development machine, thereby avoiding the waste and deficiency of the resource; when the virtual GPU device file does not correctly point to the physical GPU, the resource allocation is timely performed, thereby ensuring that the development machine can normally use the GPU for computing.

[0098] S50, link the virtual GPU device file to the target physical GPU;

[0099] S60, in response to the soft link completion instruction, forward the API call request to CUDA.

[0100] Through the method of soft link, the application program can transparently use the physical GPU resource, without worrying about the actual physical GPU device file path, and the pointing of the soft link can be changed at any time to realize the dynamic adjustment of the GPU resource. After the creation of the soft link is completed, the system will issue a soft link completion instruction. When the instruction is intercepted, the previously intercepted API call request is forwarded to the CUDA library for processing. After the virtual GPU device file correctly points to the physical GPU, the API call request is forwarded to CUDA to avoid errors caused by unprepared resources and ensure that the GPU calculation of the application program can proceed smoothly without interruption due to resource regulation process.

[0101] Further, if the node that initiates the GPU call request does not exist on the node that satisfies the required number of physical GPUs of the development machine, a restart development machine strategy is triggered.

[0102] Specifically, triggering the restart development machine strategy includes deleting the Pod corresponding to the development machine, and newly creating a new development machine Pod, and selecting a target node that satisfies the resource requirements of the development machine Pod from the Kubernetes resource scheduling pool to start the development machine Pod.

[0103] In this embodiment, it refers to in the Kubernetes cluster environment, when the physical GPU resource required by the development machine is insufficient on a certain node, the system will notify the DevMachine Controller component. The DevMachine Controller is a custom controller component that is responsible for managing the life cycle of the development machine and monitoring and maintaining the state of the development machine.

[0104] After the DevMachine Controller component receives the notification of insufficient resources, it will take the operation of restarting the development machine. The specific method is to delete the Pod corresponding to the development machine. In Kubernetes, the Pod is the smallest deployable unit, and deleting the Pod means terminating the currently running development machine instance. After deleting the old development machine Pod, the DevMachine Controller creates a new development machine Pod. This new Pod contains the containers and configuration information required for the development machine to run, and also declares the resource requirement of a certain number of physical GPUs.

[0105] After the new development machine Pod is created, it will be submitted to the Scheduler component of Kubernetes for scheduling. The main responsibility of the Scheduler is to find a suitable node for the Pod to run on. At this time, the Scheduler will traverse all the nodes in the cluster and evaluate the resource availability of each node to select a node with sufficient physical GPU resources to start the new development machine Pod, thereby ensuring that the development machine can obtain the required resources to run normally.

[0106] Suppose a Kubernetes cluster has three nodes, Node A, Node B, and Node C, each with 2, 4, and 6 physical GPUs respectively. Now there is a development machine Pod that requires 5 physical GPUs to run. It is initially scheduled to Node A, but since Node A only has 2 physical GPUs, it cannot meet the development machine's requirements. The system will notify the DevMachineController, which will delete the development machine Pod and then recreate a new Pod. Kubernetes' Scheduler will evaluate the nodes in the cluster and ultimately select Node C to start the new development machine Pod, as Node C has 6 physical GPUs, which can meet the development machine's resource requirements.

[0107] For the method of S20 "generating a message in response to an API call request, triggering interception of CUDA's API call", specifically comprising:

[0108] S21, creating a custom dynamic link library, the custom dynamic link library containing functions with the same name as CUDA API;

[0109] S22, intercepting CUDA's API call through the custom dynamic link library.

[0110] Specifically, a custom dynamic link library (.so file) can usually be created on a Linux system using C or C++ language, and the LD_PRELOAD environment variable can be used to load the custom dynamic link library. When the application calls the CUDA API, the system will first look for the same function in the custom dynamic link library and execute the code of the custom function.

[0111] This embodiment can enable or disable the interception function at any time without modifying the system configuration or application installation, only by setting or canceling the LD_PRELOAD environment variable. This method is suitable for various Linux-based systems and can be compatible with different versions of CUDA and applications, and has wide applicability.

[0112] By intercepting CUDA's API calls, resource control logic can be added at the critical nodes of the calls, providing a foundation for subsequent GPU resource dynamic allocation and management. This makes it easy to monitor and debug CUDA API calls. By adding logging or performance analysis code in custom functions, detailed call information can be obtained, helping developers better understand the GPU usage of the application and identify potential problems.

[0113] From a business perspective, the traditional GPU resource sales model is to sell the resources corresponding to several physical GPUs directly to customers. This model does not take into account the actual usage of customers. Some customers may not need to use all GPU resources during certain time periods, or not use them at all. However, the resources are still locked and cannot be utilized by other customers, resulting in low resource utilization and economic benefits.

[0114] The GPU resource dynamic control method disclosed in this application can effectively solve the drawbacks of the traditional GPU resource sales model, improve resource utilization, and significantly enhance economic benefits. Specifically, customers can flexibly customize the resource specifications of the development machine according to their actual needs and only apply for the required GPU resources. For example, during the initial stage of a project or during low-load periods, customers can reduce the number of GPUs applied for, avoiding the payment for idle resources, thereby reducing the cost of use. For small businesses or start-ups, this on-demand usage model allows them to obtain GPU resource support at a lower cost, reducing the entry threshold and promoting innovation and business development.

[0115] Customers can dynamically adjust the use of GPU resources according to different stages of the project and business needs. When the project enters a high-load stage, GPU resources can be increased; during low-load or idle periods, resource occupation can be reduced, without worrying about resource waste. For temporary computing tasks, customers can quickly create and destroy development machines to flexibly respond to various business scenarios, improving the response speed and flexibility of the business.

[0116] By dynamically allocating and recycling GPU resources, service providers can make full use of idle GPU resources. When the GPU resources of a certain customer are idle, the system will automatically remove the soft link and mark the corresponding real GPU as idle so that other customers can use it, avoiding waste and locking of resources. For example, in the traditional mode, a node with 10 physical GPUs may only have 3-4 fully utilized, while after adopting this scheme, the resource utilization rate can be increased to 7-8 or even higher, greatly improving the efficiency of resource use. Higher resource utilization means that service providers can serve more customers on the same hardware resources, thereby increasing revenue. For example, GPU resources that can only serve 10 customers can now serve 15-20 customers, directly improving economic efficiency. Due to efficient use of resources, service providers can reduce hardware procurement costs and, under the same business demand, do not need to purchase too many GPU devices, reducing operating costs and improving profit margins.

[0117] At the same time, customers can obtain more flexible resource use experience at a lower cost, which will significantly improve customer satisfaction and loyalty. Satisfied customers are more likely to use the service provider's products for a long time and recommend them to other potential customers, bringing more business opportunities to service providers; service providers can provide more personalized resource solutions and services to customers based on their use data, further enhancing customer stickiness and competitiveness.

[0118] The method provided by the application can effectively reduce the threshold for enterprises and developers to use GPU resources, enabling more innovative projects to obtain resource support. For example, some emerging artificial intelligence and machine learning projects can more easily conduct research and development, promoting technological innovation and development in the industry. At the same time, flexible resource use patterns encourage more start-ups and individual developers to participate in the GPU computing field, promoting industry diversification and competition and driving progress in the entire industry. The scheme realizes dynamic allocation and optimal configuration of GPU resources, enabling resources to flow to where they are most needed. From a macro perspective, this helps improve the resource utilization efficiency of the entire industry, reduces resource waste, and promotes sustainable development of the industry.

[0119] In a second aspect, the application discloses a development machine GPU resource dynamic regulation system for executing the development machine GPU resource dynamic regulation method disclosed in the first aspect of the application, and the system comprises:

[0120] The self-defined resource object creation module is configured to, in response to a development machine creation request initiated by a user, create a self-defined resource object containing development machine configuration information according to a self-defined resource specification of the development machine.

[0121] a pod creation module, configured to create a corresponding pod in the Kubernetes cluster according to the custom resource object, and the pod is referred to as a target pod;

[0122] a target GPU node determination module, configured to determine a target GPU node to which the target pod is to be scheduled;

[0123] a GPU application quantity acquisition module, configured to acquire a GPU application quantity corresponding to the target pod according to the target GPU node;

[0124] a target physical GPU determination module, configured to determine a plurality of target physical GPUs according to the GPU application quantity;

[0125] a virtual GPU device file creation module, configured to create a plurality of virtual GPU device files on a host according to the plurality of target physical GPUs;

[0126] a linking module, configured to establish a soft link between each virtual GPU device file and a corresponding target physical GPU;

[0127] a mounting module, configured to, in response to a creation completion instruction of the virtual GPU device file, start a development machine container and mount the virtual GPU device file into the container;

[0128] a dynamic regulation module, configured to, when the acquired container GPU resource idle information satisfies a preset idle condition, cancel a soft link between a virtual GPU device file corresponding to the container GPU resource idle information and a real GPU, and update a use state of the corresponding real GPU to idle.

[0129] A computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is configured to store non-transitory computer readable instructions. Specifically, the memory can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0130] The processor can be a central processing unit (CPU) or other forms of processing units with data processing and / or instruction execution capabilities, and can control other components in the computer device to perform desired functions. In an embodiment of the present disclosure, the processor is configured to run the computer readable instructions stored in the memory, so that the computer device performs all or part of the steps of the development machine GPU resource dynamic regulation method of the embodiments of the present disclosure.

[0131] Those skilled in the art shall understand that, in order to solve the technical problem of how to obtain a good user experience effect, the embodiment can also include well-known structures such as a communication bus, an interface, and the like, which shall also be included in the protection scope of the present disclosure.

[0132] As Figure 5 A structural schematic diagram of a computer device provided by the embodiment of the present disclosure is shown. The structural schematic diagram is suitable for implementing the computer device in the embodiment of the present disclosure. Figure 5 The computer device shown is only an example and shall not bring any limitation to the functions and use range of the embodiment of the present disclosure.

[0133] As Figure 5 As shown, the computer device can include a processor (for example, a central processing unit, a graphics processing unit, or the like) which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0134] Generally, the following devices can be connected to the I / O interface: an input device including, for example, a sensor or a visual information collection device; an output device including, for example, a display screen; a storage device including, for example, a magnetic tape, a hard disk, or the like; and a communication device. The communication device can allow the computer device to perform wireless or wired communication with other devices (such as an edge computing device) to exchange data. Although Figure 5 The computer device with various devices is shown, but it shall be understood that it is not required to implement or have all the devices shown. More or fewer devices can be alternatively implemented or provided.

[0135] In particular, according to the embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiment of the present disclosure includes a computer program product including a computer program carried on a non-transitory computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device, or installed from the ROM. When the computer program is executed by the processor, all or part of the steps of the development GPU resource dynamic regulation method of the embodiment of the present disclosure are performed.

[0136] The detailed description of the embodiment can refer to the corresponding description in the foregoing embodiments, which will not be described here again.

[0137] A computer readable storage medium according to embodiments of the present disclosure has non-transitory computer readable instructions stored thereon. When the non-transitory computer readable instructions are run by a processor, all or part of the steps of the method for dynamically regulating GPU resources according to the embodiments of the present disclosure are performed.

[0138] The computer readable storage medium described above includes, but is not limited to, an optical storage medium (for example, a CD-ROM and a DVD), a magneto-optical storage medium (for example, an MO), a magnetic storage medium (for example, a magnetic tape or a mobile hard disk), a medium with a built-in rewritable nonvolatile memory (for example, a memory card), and a medium with a built-in ROM (for example, a ROM cartridge).

[0139] Detailed descriptions of the embodiments are described above with reference to the corresponding descriptions in the foregoing embodiments, and will not be repeated here.

[0140] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be noted that the advantages, advantages, effects and the like mentioned in the present disclosure are only examples and are not limiting, and these advantages, advantages, effects and the like cannot be considered as the must-have of each embodiment of the present disclosure. In addition, the specific details of the above disclosure are only for the purpose of example and for the purpose of understanding, and are not limited to the above specific details, and the present disclosure is not limited to the above specific details.

[0141] In the present disclosure, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. The block diagrams of devices, apparatuses, equipment, systems involved in the present disclosure are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0142] In addition, as used herein, "or" used in the list of items preceded by "at least one of" means a separate list, so that, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the phrase "exemplary" does not mean that the described example is preferred or better than other examples.

[0143] It is also important to note that the systems and methods of the present disclosure can be embodied in a variety of contexts. For example, the systems and methods of the present disclosure can be implemented in the context of a computer system, a mobile device, a server, a network, a distributed computing environment, etc. It is also important to note that the various components and steps of the systems and methods of the present disclosure can be decomposed and / or recombined. Such decompositions and / or recombinations should be considered equivalents of the present disclosure.

[0144] Various changes, modifications, and alterations to the techniques described herein can be made without departing from the teachings of the technology defined by the appended claims. In addition, the scope of the claims of the present disclosure is not limited to the specific aspects described above. Rather, the scope of the claims of the present disclosure is intended to encompass all techniques, apparatuses, manufactures, compositions of matter, means, methods, and actions that would fall within the scope of a person of ordinary skill in the art, having the benefit of the benefit of this disclosure, and the equivalent thereof.

[0145] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0146] The above description has been presented for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although various example aspects and embodiments have been discussed above, those of skill in the art will recognize certain modifications, permutations, alterations, additions and sub-combinations thereof.

Claims

1. A method for dynamically adjusting GPU resources on a development machine, characterized in that, include: In response to a user-initiated development machine creation request, a custom resource object containing the development machine configuration information is created according to the development machine's custom resource specifications. Based on the custom resource object, create the corresponding Pod in the Kubernetes cluster, denoted as the target Pod; Determine the target GPU node to be scheduled for the target Pod; The number of GPU requests corresponding to the target Pod is obtained based on the target GPU node; A number of target physical GPUs are determined based on the number of GPU requests; Based on the target physical GPUs, several virtual GPU device files are created on the host; the virtual GPU device files can finely divide the physical GPU resources, allowing multiple Pods to share the same physical GPU; Establish a symbolic link between each virtual GPU device file and the corresponding target physical GPU; In response to the creation completion instruction of the virtual GPU device file, the development machine container is started, and the virtual GPU device file is mounted into the container so that the development machine container can directly access and use physical GPU resources; When the obtained container GPU resource idle information meets the preset idle conditions, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the usage status of the corresponding real GPU is updated to idle. It also includes: responding to GPU call requests and triggering the execution of dynamic GPU resource control strategies; The triggering of the GPU resource dynamic adjustment strategy includes: Generate API call requests to CUDA based on the GPU's running tasks; In response to the generated message of the API call request, the interception of the CUDA API call is triggered; Determine whether the virtual GPU device file in the current development machine points to the physical GPU. If so, forward the API call request to CUDA. If the virtual GPU device file in the current development machine does not point to a physical GPU, determine whether there is a number of physical GPUs required by the development machine on the node that initiated the GPU call request; If so, determine the corresponding physical GPUs based on the required number of physical GPUs, and record them as the target physical GPUs; Create a symbolic link to the virtual GPU device file pointing to the target physical GPU; In response to the symbolic link completion instruction, the API call request is forwarded to CUDA.

2. The method for dynamic control of GPU resources on a development machine according to claim 1, characterized in that, The step of creating a corresponding Pod in the Kubernetes cluster based on the custom resource object includes: Send the custom resource object to the Kubernetes cluster; The custom resource object is verified according to the preset custom resource definition. When the verification is successful, a corresponding Pod is created in the Kubernetes cluster according to the configuration information in the custom resource object, and it is denoted as the target Pod.

3. The method for dynamic control of GPU resources on a development machine according to claim 1, characterized in that, When the acquired container GPU resource idle information meets the preset idle condition, the symbolic link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the usage status of the corresponding real GPU is updated to idle, including: When the memory and core usage of the acquired container GPU resource idle information is 0 within a preset duration, the soft link between the virtual GPU device file corresponding to the container GPU resource idle information and the real GPU is removed, and the usage status of the corresponding real GPU is updated to idle. The released real GPUs re-enter the Kubernetes resource scheduling pool.

4. The method for dynamic control of GPU resources on a development machine according to claim 3, characterized in that, If the node that initiated the GPU call request does not have enough physical GPUs to meet the requirements of the development machine, the development machine restart strategy will be triggered.

5. The method for dynamic control of GPU resources on a development machine according to claim 4, characterized in that, The triggering and execution of the development machine restart strategy includes: deleting the corresponding development machine Pod, creating a new development machine Pod, and selecting a target node from the Kubernetes resource scheduling pool that meets the resource requirements of the development machine Pod to start the development machine Pod.

6. The method for dynamic control of GPU resources on a development machine according to claim 3, characterized in that, The generation message in response to the API call request triggers the interception of CUDA API calls, including: Create a custom dynamic link library containing functions with the same names as the CUDA API; Intercept CUDA API calls using the custom dynamic link library.

7. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the dynamic GPU resource control method for the development machine as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the development machine GPU resource dynamic control method as described in any one of claims 1-6.

9. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1-6.

Images