GPU access method and device in cloud platform, electronic equipment and storage medium
By using the command line interface in the Kubernetes platform to obtain GPU information, perform computing power evaluation and standard verification, automatic identification and access of GPUs are achieved, and the problem of low efficiency of single card scheduling and multi-card identification is solved, and GPU resources are improved. Utilization efficiency and computing efficiency.
Patent Information
- Application Number
- CN202411364824.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2024-12-03
AI Technical Summary
The existing Kubernetes platform has problems of resource exclusiveness and performance in GPU resource management and scheduling caused by single card scheduling, and it is difficult to automatically identify and configure multi-card GPUs, resulting in inefficiency.
Determine the target GPU through detection instructions, obtain basic information and performance information using the command line interface, convert it into computing power evaluation index data, use the preset computing power evaluation model to process, and perform standard verification of the information to realize automatic GPU identification and access, register to the cloud platform, and divide the virtual GPU according to the computing power evaluation results, and perform task allocation and resource scheduling.
It realizes the automatic identification and access of GPU devices by the cloud platform, improves the efficiency of GPU resources utilization, avoids the waste of resources for single card scheduling and uneven allocation of multiple card sharing, and improves the overall computing efficiency.
Smart Images

Figure CN119065855A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of cloud platforms, and in particular to a GPU access method in a cloud platform, and also to a GPU access device, electronic device, and computer-readable storage medium in a cloud platform. Background Art
[0002] In the field of artificial intelligence (AI), the combination of containerization technology and the Kubernetes (K8s) platform has brought revolutionary changes to the management and scheduling of GPU (Graphics Processing Unit) resources. As a leader in cloud computing resource orchestration, K8s has achieved effective management and exposure of GPU resources through a plug-in mechanism, allowing GPUs of different manufacturers and models to be seamlessly integrated into the K8s cluster, greatly simplifying the scheduling process of GPU resources. However, the current K8s GPU scheduling method still has limitations, such as resource exclusivity and uneven performance caused by single-card scheduling, and inefficiency caused by the difficulty of automatic identification of multiple cards and the need for manual batch configuration of GPUs.
[0003] Therefore, how to realize the automatic identification and automatic access of GPU devices by the cloud platform and further ensure the application efficiency of GPU is an urgent problem to be solved by technical personnel in this field. Summary of the invention
[0004] The purpose of this application is to provide a GPU access method in a cloud platform, which can realize the cloud platform's automatic identification and automatic access to GPU devices, further ensuring the application efficiency of the GPU; another purpose of this application is to provide a GPU access device, electronic device, computer-readable storage medium and computer program product in a cloud platform, all of which have the above-mentioned beneficial effects.
[0005] In a first aspect, the present application provides a GPU access method in a cloud platform, comprising:
[0006] Determine the target GPU according to the detection instruction, and obtain basic information and performance information of the target GPU through a command line interface;
[0007] Convert each of the performance information into computing power evaluation index data, and process each of the computing power evaluation index data using a preset computing power evaluation model to obtain a computing power evaluation result of the target GPU;
[0008] Performing a standard check on the basic information and computing power evaluation result of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform;
[0009] When the target GPU meets the GPU access standard of the cloud platform, the target GPU is registered to the cloud platform.
[0010] Optionally, when the target GPU meets the GPU access standard of the cloud platform, after registering the target GPU to the cloud platform, the method further includes:
[0011] Determine the GPU computing power value according to the computing power evaluation result of the target GPU;
[0012] Divide the computing power of the target GPU according to a preset division threshold to obtain multiple virtual GPUs; wherein the GPU computing power value of each virtual GPU is the preset division threshold;
[0013] Accordingly, the method further comprises:
[0014] When receiving a task request sent by a client, determining the task priority and computing power requirement value of the task request;
[0015] Adding the task request to a task request queue according to the priority of the task request; wherein each task request in the task request queue is executed in order of priority from high to low;
[0016] For each task request in the task request queue, determining an adapted GPU and a target virtual GPU on the adapted GPU according to a computing power requirement value of the task request;
[0017] The task request is distributed to the adapted GPU, so that the adapted GPU executes the task request using the target virtual GPU.
[0018] Optionally, after distributing the task request to the adapted GPU so that the adapted GPU executes the task request using the target virtual GPU, the method further includes:
[0019] Obtaining computing power allocation information of each GPU in the cloud platform;
[0020] Generate a GPU computing power resource allocation table in the cloud platform according to the basic information, performance information, and computing power allocation information of each GPU;
[0021] The basic information of the GPU includes one or more combinations of the model information, configuration information, and supported task types of the GPU;
[0022] The performance information of the GPU includes one or more combinations of floating-point operation speed, memory information, bandwidth information, and power consumption information of the GPU.
[0023] Optionally, the GPU access method in the cloud platform further includes:
[0024] When the completion information of the task request is obtained, the computing power resources of the target virtual GPU on the adapted GPU are released, and the GPU computing power resource allocation table in the cloud platform is updated.
[0025] Optionally, obtaining basic information and performance information of the target GPU through a command line interface includes:
[0026] Sending an information acquisition command to the target GPU through the command line interface; wherein the information acquisition command is an NVIDIA-SMI command or a ROCm command;
[0027] The basic information and the performance information fed back by the target GPU according to the information acquisition command are obtained.
[0028] Optionally, determining a target GPU according to the detection instruction includes:
[0029] When access information is sensed through a preset interface, responding to the detection instruction;
[0030] The target GPU is determined according to the detection instruction.
[0031] Optionally, the GPU access method in the cloud platform further includes:
[0032] When the target GPU does not meet the GPU access standard of the cloud platform, an access exception prompt is output;
[0033] The access abnormality prompt includes basic information and performance information of the target GPU, verification information of the basic information, and verification information of the performance information.
[0034] In a second aspect, the present application also discloses a GPU access device in a cloud platform, including:
[0035] An acquisition module, used to determine a target GPU according to a detection instruction, and obtain basic information and performance information of the target GPU through a command line interface;
[0036] A processing module, used to convert each of the performance information into computing power evaluation index data, and use a preset computing power evaluation model to process each of the computing power evaluation index data to obtain a computing power evaluation result of the target GPU;
[0037] A verification module, used to perform a standard verification on the basic information and computing power evaluation result of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform;
[0038] A registration module is used to register the target GPU to the cloud platform when the target GPU meets the GPU access standard of the cloud platform.
[0039] In a third aspect, the present application further discloses an electronic device, comprising:
[0040] Memory for storing computer programs;
[0041] A processor is used to implement the steps of any one of the above-mentioned methods for accessing a GPU in a cloud platform when executing the computer program.
[0042] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the GPU access method in any one of the cloud platforms described above are implemented.
[0043] In a fifth aspect, the present invention further discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the GPU access method in any one of the cloud platforms described above.
[0044] The present application provides a GPU access method in a cloud platform, comprising: determining a target GPU according to a detection instruction, and obtaining basic information and performance information of the target GPU through a command line interface; converting each of the performance information into computing power evaluation index data, and using a preset computing power evaluation model to process each of the computing power evaluation index data to obtain a computing power evaluation result of the target GPU; performing a standard check on the basic information and computing power evaluation result of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform; when the target GPU meets the GPU access standard of the cloud platform, registering the target GPU to the cloud platform.
[0045] By applying the technical solution provided by this application, the target GPU applying for access to the cloud platform is determined in response to the detection instruction, and its basic information and performance information are automatically obtained through the command line interface, wherein the computing power resources of the target GPU can be evaluated through the performance information of the target GPU, so as to realize the cloud platform to verify the validity of the basic information and computing power of the target GPU, and then realize the automatic access of the target GPU that meets the cloud platform access standard to the cloud platform. It can be seen that the technical solution provided by this application can realize the automatic identification and automatic access of the cloud platform to the GPU device, and further ensure the application efficiency of the GPU.
[0046] In one embodiment of the present application, for a target GPU applying for access to a cloud platform, an information acquisition command can be sent to it through a command line interface to obtain basic information and performance information of the target GPU, thereby realizing automated acquisition of information related to the target GPU in an efficient and accurate manner; and, this implementation method does not require any improvements to the cloud platform, and can be achieved by simply entering a corresponding command line through a command line interface, which is simple and easy to implement, and effectively controls development costs.
[0047] The GPU access device, electronic device, computer-readable storage medium and computer program product in the cloud platform provided in this application also have the above-mentioned technical effects, and this application will not repeat them here. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.
[0049] Figure 1 A flowchart of a GPU access method in a cloud platform provided by this application;
[0050] Figure 2 A schematic diagram of the structure of a cloud platform provided for this application;
[0051] Figure 3 A schematic diagram of the structure of a GPU access device in a cloud platform provided by this application;
[0052] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION
[0053] The core of this application is to provide a GPU access method in a cloud platform, which can realize the cloud platform's automatic identification and automatic access to GPU devices, further ensuring the application efficiency of the GPU; another core of this application is to provide a GPU access device, electronic device, computer-readable storage medium and computer program product in a cloud platform, all of which have the above-mentioned beneficial effects.
[0054] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0055] An embodiment of the present application provides a method for accessing a GPU in a cloud platform.
[0056] Please refer to Figure 1 , Figure 1 This is a flow chart of a GPU access method in a cloud platform provided in the present application. The GPU access method in the cloud platform may include the following S101 to S104.
[0057] S101: Determine a target GPU according to a detection instruction, and obtain basic information and performance information of the target GPU through a command line interface.
[0058] This step aims to determine the target GPU and obtain relevant information. The target GPU is the GPU currently applying to access the cloud platform, and needs to be identified for access; the relevant information of the GPU obtained may include its basic information and performance information. For example, the basic information of the GPU may include but is not limited to the model information, configuration information, supported task types, etc. of the GPU, and the performance information of the GPU may include but is not limited to the floating-point operation speed, memory information, bandwidth information, power consumption information, etc. of the GPU.
[0059] Furthermore, the determination of the target GPU is in response to a received detection instruction, which may be an instruction directly initiated by an administrator through a command interaction interface, or an instruction automatically triggered based on a preset condition (such as a timing condition), which is not limited in this application. The acquisition of the basic information and performance information of the target GPU can be achieved through a command line interface, that is, the automatic acquisition of the target GPU related information can be achieved directly through the command line interface, which is efficient and accurate.
[0060] In one embodiment of the present application, determining the target GPU according to the detection instruction may include:
[0061] When access information is sensed through a preset interface, respond to a detection instruction;
[0062] Determine the target GPU based on the detection instruction.
[0063] The embodiment of the present application provides a method for determining the target GPU according to the detection instruction, that is, an automated detection method based on automatic sensing conditions. Specifically, real-time information monitoring can be performed through a preset interface, and when access information is sensed, the detection instruction can be automatically responded to, thereby determining the target GPU. It can be understood that the preset interface is an interface reserved in the cloud platform for accessing GPU devices.
[0064] In one embodiment of the present application, obtaining basic information and performance information of a target GPU through a command line interface may include:
[0065] Sending an information acquisition command to a target GPU through a command line interface; wherein the information acquisition command is an NVIDIA-SMI command (an information acquisition command of NVIDIA) or a ROCm command (an information acquisition command of AMD);
[0066] Get the basic information and performance information of the target GPU according to the information acquisition command feedback.
[0067] The embodiment of the present application provides a method for obtaining target GPU related information through a command line interface, that is, an information acquisition command can be directly sent to the target GPU through a command line interface to obtain its basic information and performance information. The information acquisition command can specifically be a code command line in a related tool.
[0068] It can be imagined that in the embodiment of the present application, for the target GPU applying for access to the cloud platform, an information acquisition command can be sent to it through the command line interface to obtain basic information and performance information of the target GPU, thereby realizing the automatic acquisition of information related to the target GPU, which is efficient and accurate; and this implementation method does not require any improvement to the cloud platform, but only needs to enter the corresponding command line through the command line interface, which is simple and easy to implement, and effectively controls the development cost.
[0069] S102: Convert each performance information into computing power evaluation index data, and use a preset computing power evaluation model to process each computing power evaluation index data to obtain a computing power evaluation result of the target GPU.
[0070] This step aims to achieve GPU computing power evaluation based on the target GPU performance information, that is, to determine the computing power resources that the target GPU can provide. In the implementation process, each performance information is first converted into computing power evaluation index data that can be used for model processing, for example, the floating-point operation speed is converted into floating-point operations per second (FLOPs) or the processing speed under a specific task; further, each computing power evaluation index data is input into a preset computing power evaluation model for processing, and the output of the model is the computing power evaluation result of the target GPU, which is essentially the computing power resources that the target GPU can provide. Among them, the preset computing power evaluation model is a network model obtained by training based on historical big data, which can be pre-stored in the corresponding storage space and directly called when used. In addition, in order to ensure the accuracy of the evaluation results, historical data from multiple time periods can also be used to update it regularly to obtain a preset computing power evaluation model with higher accuracy.
[0071] S103: Perform a standard check on the basic information and computing power evaluation result of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform.
[0072] This step is intended to achieve standard verification of the target GPU related information to ensure that the GPU connected to the cloud platform meets the GPU access standards of the cloud platform and ensure the stability of the cloud platform system. Specifically, for each GPU device, the cloud platform pre-sets a set of evaluation standards for its basic information and computing power evaluation results. After obtaining the basic information and computing power evaluation results of the target GPU, the evaluation standards can be used to perform standard verification on the basic information and computing power evaluation results of the target GPU respectively to determine whether the target GPU meets the GPU access standards of the cloud platform. It can be understood that when the standard verification of the basic information and computing power evaluation results of the target GPU passes, it can be determined that the target GPU meets the GPU access standards of the cloud platform and the target GPU is allowed to access the cloud platform; otherwise, it is considered that the target GPU does not meet the GPU access standards of the cloud platform, so the target GPU is not allowed to access the cloud platform.
[0073] S104: When the target GPU meets the GPU access standard of the cloud platform, register the target GPU to the cloud platform.
[0074] This step aims to achieve effective registration of the target GPU so as to connect the target GPU to the cloud platform. In other words, when the target GPU meets the GPU access standard of the cloud platform, it can be registered with the cloud platform, which means that the target GPU is connected to the cloud platform. Furthermore, the cloud platform can record the registration information of each GPU connected to the cloud platform to facilitate the effective management of each GPU device in the cloud platform.
[0075] In one embodiment of the present application, the GPU access method in the cloud platform may further include:
[0076] When the target GPU does not meet the GPU access standards of the cloud platform, an access exception prompt is output;
[0077] The access exception prompt includes basic information and performance information of the target GPU, as well as verification information of the basic information and verification information of the performance information.
[0078] As described above, when the standard verification of the basic information and computing power evaluation results of the target GPU fails to pass all, it can be considered that the target GPU does not meet the GPU access standard of the cloud platform, so the target GPU is not allowed to access the cloud platform. On this basis, an access exception prompt corresponding to the target GPU can also be output, and the access exception prompt may include but is not limited to the basic information and performance information of the target GPU, as well as the verification information of the basic information and the verification information of the performance information, etc., to provide a reference for the technicians so that the technicians can clearly understand the detailed information of the target GPU.
[0079] It can be seen that the GPU access method in the cloud platform provided by the embodiment of the present application determines the target GPU that applies for access to the cloud platform in response to the detection instruction, and automatically obtains its basic information and performance information through the command line interface, wherein the computing power resources of the target GPU can be evaluated through the performance information of the target GPU, so as to realize the cloud platform to verify the validity of the basic information and computing power of the target GPU, and then realize the automatic access of the target GPU that meets the cloud platform access standard to the cloud platform. It can be seen that the technical solution provided by the present application can realize the automatic identification and automatic access of the cloud platform to the GPU device, and further ensure the application efficiency of the GPU.
[0080] Based on the above embodiments:
[0081] In one embodiment of the present application, when the target GPU meets the GPU access standard of the cloud platform, after registering the target GPU to the cloud platform, the following steps may also be included:
[0082] Determine the GPU computing power value based on the computing power evaluation result of the target GPU;
[0083] The target GPU is divided into computing power according to a preset division threshold to obtain multiple virtual GPUs; wherein the GPU computing power value of each virtual GPU is the preset division threshold;
[0084] Accordingly, the GPU access method in the cloud platform also includes:
[0085] When receiving a task request sent by a client, determine the task priority and computing power requirement value of the task request;
[0086] Adding the task request to the task request queue according to the priority of the task request; wherein each task request in the task request queue is executed in the order of priority from high to low;
[0087] For each task request in the task request queue, determine the adapted GPU and the target virtual GPU on the adapted GPU according to the computing power requirement value of the task request;
[0088] The task request is distributed to the adapted GPU, so that the adapted GPU executes the task request using the target virtual GPU.
[0089] The embodiment of the present application provides a method for implementing task allocation to each GPU in a cloud platform, aiming to effectively ensure the effective utilization of the computing power resources of each GPU and the load balancing of the entire cloud platform. Specifically, after registering the target GPU that meets the cloud platform access standard to the cloud platform, the computing power resources can be divided to obtain multiple virtual GPUs (vGPUs) with independent computing power resources, and the computing power resources that each virtual GPU can provide are equal, that is, the GPU computing power value is the preset division threshold; on this basis, when receiving a task request and assigning it a task, it can be implemented with reference to its task priority and the required computing power resources (that is, the above-mentioned computing power requirement value), wherein the task priority is used to determine the processing order of the task request, that is, high-priority task requests are processed first, and low-priority task requests are processed later; the computing power requirement value is used to implement the determination of the adapted GPU and its adapted virtual GPU (that is, the above-mentioned target virtual GPU, so as to distribute the task request to its corresponding adapted GPU, and use the computing power resources of the target virtual GPU therein to execute the task request.
[0090] It is understandable that the specific value of the above-mentioned preset division threshold does not affect the implementation of the present technical solution, and can be set by technical personnel according to actual needs, and this application does not limit this.
[0091] In one embodiment of the present application, after distributing the task request to the adapted GPU so that the adapted GPU executes the task request using the target virtual GPU, the following method may also be included:
[0092] Get the computing power allocation information of each GPU in the cloud platform;
[0093] Generate a GPU computing resource allocation table in the cloud platform based on the basic information, performance information, and computing power allocation information of each GPU;
[0094] The basic information of the GPU includes one or more combinations of the GPU model information, configuration information, and supported task types;
[0095] The performance information of the GPU includes one or more combinations of the floating-point operation speed, memory information, bandwidth information, and power consumption information of the GPU.
[0096] The GPU access method in the cloud platform provided in the embodiment of the present application can also record the task request after completing the task request distribution, so that the cloud platform can effectively manage the GPU resources in the cloud. Here, the relevant information can be recorded in the form of a table, that is, the above-mentioned GPU computing power resource allocation table, and its recorded content can include but is not limited to the basic information, performance information, and computing power allocation information of each GPU, where the computing power allocation information is the allocation of each virtual GPU on the corresponding GPU.
[0097] In one embodiment of the present application, the GPU access method in the cloud platform may further include:
[0098] When the completion information of the task request is obtained, the computing power resources of the target virtual GPU on the adapter GPU are released, and the GPU computing power resource allocation table in the cloud platform is updated.
[0099] In the embodiment of the present application, for the GPU device that has completed the task request, the virtual GPU corresponding to the currently completed task request can be released and recycled to effectively avoid unnecessary waste of resources. At the same time, the GPU computing resource allocation table in the cloud platform can be further updated to effectively ensure the real-time and accuracy of the GPU computing resource allocation table.
[0100] In one embodiment of the present application, the GPU access method in the cloud platform may further include:
[0101] For each GPU in the cloud platform, each virtual GPU in the GPU is monitored in real time to obtain the computing resource utilization rate of each virtual GPU;
[0102] When the computing resource usage of the virtual GPU meets the preset threshold condition, the computing quota of the virtual GPU is adjusted.
[0103] The GPU access method in the cloud platform provided by the embodiment of the present application can further realize the adaptive adjustment of the computing power quota of each virtual GPU in the GPU to effectively ensure the normal execution of the assigned business request. Specifically, the computing power resource usage of each virtual GPU on each GPU in the cloud platform can be monitored in real time to obtain the computing power resource usage rate of each virtual GPU, so as to adjust the computing power quota according to the computing power resource usage of each virtual GPU. Obviously, when the computing power resource usage rate reaches the first threshold, the computing power resource capacity of the corresponding virtual GPU can be expanded; when the computing power resource usage rate is lower than the second threshold, the computing power resource capacity of the corresponding virtual GPU can be reduced. Of course, the computing power quota adjustment here should be implemented with reference to the actual computing power capacity of the GPU to which the virtual GPU belongs. At the same time, the computing power quota adjustment information of each virtual GPU can be synchronized to the GPU computing power resource allocation table.
[0104] Similarly, the specific value range of the above-mentioned preset threshold conditions (first threshold and second threshold) does not affect the implementation of the present technical solution, and can be set by technical personnel according to actual needs, and this application does not limit this.
[0105] Based on the above embodiments, the embodiment of the present application provides another GPU access method in a cloud platform.
[0106] First, please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of a cloud platform provided by the present application. The cloud platform may include a hardware collection and evaluation device, a hardware registration device, and a scheduling and allocation device:
[0107] 1. Hardware acquisition and evaluation device:
[0108] The hardware acquisition and evaluation device is the core component, which is mainly responsible for real-time collection and evaluation of the actual computing power of GPU resources. The device can accurately measure the performance parameters (performance information) of the GPU under different loads through the built-in algorithm (preset computing power evaluation model), including but not limited to floating-point operation speed, memory, bandwidth, power consumption, etc. At the same time, the device also has powerful data processing capabilities, which can convert these raw data into quantitative indicators that can be used for computing power evaluation, such as the number of floating-point operations per second (FLOPs) or the processing speed under specific tasks. In addition, during the computing power evaluation process, the hardware acquisition and evaluation device will also consider multiple factors (basic information) such as the GPU model, configuration, working status, and the type of task currently running to ensure the accuracy and reliability of the evaluation results. In addition, the device also has self-learning and optimization capabilities, and can continuously adjust the evaluation algorithm based on historical data and operating experience to adapt to the changing application needs and GPU hardware development trends.
[0109] 2. Hardware registration device:
[0110] The hardware registration device is responsible for registering the evaluated GPU resources into the container cloud platform for subsequent unified management and scheduling. The device works closely with the hardware acquisition and evaluation device to ensure that only GPU resources that meet certain standards can be registered into the platform. During the registration process, the hardware registration device will record detailed information about the GPU, including model, computing power evaluation results, current status, availability, etc., and store this information in the platform's database.
[0111] In order to support the dynamic expansion and update of GPU resources, the hardware registration device also provides a flexible registration mechanism. When new GPU resources are added to the platform, the device can automatically identify and start the evaluation process; when existing GPU resources change or fail, the device can also update the relevant information in the database in a timely manner to ensure that the GPU resources in the cloud platform are always up to date and accurate.
[0112] 3. Dispatching and distribution device:
[0113] The scheduling and allocation device is the key link to realize the actual value of GPU computing power cutting and allocation. Based on the data provided by the hardware acquisition and evaluation device and the hardware registration device, the device uses innovative algorithms and strategies to efficiently and accurately schedule and allocate GPU resources.
[0114] During the scheduling process, the scheduling allocation device will comprehensively consider multiple factors such as the computing power requirements of the upper-level applications, the actual computing power of the GPU resources, the current load conditions, and the priority of the tasks. Through intelligent matching and dynamic adjustment, the device can allocate the appropriate GPU computing power to the corresponding application (task request), ensuring that each task can obtain sufficient computing resources and run efficiently.
[0115] Furthermore, in order to realize the split allocation of GPU computing power, the scheduling and allocation device adopts a virtualization technology based on the minimum unit of computing power. This technology divides the GPU computing power into multiple minimum units that can be independently allocated, and flexibly combines and allocates them according to the needs of upper-level applications. In this way, the scheduling and allocation device can effectively avoid the waste of computing power in single-card scheduling and the uneven allocation problem in multi-card shared scheduling, thereby maximizing the use of GPU resources and improving overall computing efficiency.
[0116] It can be seen that the cloud platform has realized an efficient, accurate and flexible GPU computing power cutting and allocation method through the organic combination of hardware collection and evaluation devices, hardware registration devices and scheduling and allocation devices. This method can not only significantly improve the management efficiency and utilization rate of GPU resources in the container cloud platform, but also provide strong support for the sustainable development of AI technology.
[0117] Secondly, the specific implementation contents of each device in the cloud platform are as follows:
[0118] 1. Specific implementation of the hardware acquisition and evaluation device:
[0119] (1) Hardware interface: Use NVIDIA's NVIDIA-SMI tool or AMD's ROCm tool to obtain real-time GPU performance data through the command line interface.
[0120] (2) Evaluation algorithm: Design an evaluation model based on machine learning. The model takes into account factors such as GPU model, number of cores, memory size, current load, temperature, etc. to predict the actual GPU computing power. The model is trained using historical data to continuously optimize the evaluation accuracy.
[0121] (3) Data processing: The data collected in real time is preprocessed and aggregated through the streaming processing framework and then input into the evaluation model.
[0122] 2. Specific implementation of the hardware registration device:
[0123] (1) Registration process: When a new GPU resource is added, the hardware acquisition and evaluation device automatically starts the evaluation and sends the evaluation results to the hardware registration device. The hardware registration device verifies the validity of the evaluation data and stores the GPU information (including model, computing power value, status, etc.) in the custom resource (CRD) of the Kubernetes (k8s) cluster.
[0124] (2) Dynamic update: Monitor the status changes of GPU resources, such as failures and performance degradation, and update the information in CRD in real time.
[0125] 3. Specific implementation of the scheduling and allocation device:
[0126] (1) Scheduling strategy: A priority-based scheduling algorithm is used to schedule based on factors such as GPU computing power requirements, task priority, and GPU current load. GPU computing power is virtualized, and the computing power of a single GPU is divided into multiple virtual GPUs (vGPUs), each with an independent computing power quota.
[0127] (2) Allocation mechanism: When a user submits a task, the scheduling allocation device searches for suitable GPU resources from the CRD of the Kubernetes cluster based on the computing power requested. The Kubernetes scheduler schedules the task to a GPU with sufficient computing power and configures the corresponding vGPU resources.
[0128] (3) Dynamic adjustment: Dynamically adjust the computing power quota of the vGPU based on the actual computing power requirements and GPU load during task execution.
[0129] Finally, based on the above cloud platform, the operation process of the GPU access method in the cloud platform provided in the embodiment of the present application is as follows:
[0130] 1. Hardware access and evaluation: Connect the GPU hardware to the cloud platform and conduct real-time evaluation through the hardware acquisition and evaluation device. The evaluation results include key indicators such as GPU computing power, memory bandwidth, and power consumption.
[0131] 2. Register GPU resources: After the evaluation is completed, the hardware registration device registers the GPU information to the CRD of the Kubernetes cluster. The registration information includes GPU model, computing power value, current status, etc.
[0132] 3. User submits task: Users submit tasks through the Kubernetes API and specify the required GPU computing power.
[0133] 4. Task scheduling and allocation: The scheduling and allocation device performs intelligent scheduling and allocation according to task requirements and GPU resource conditions, schedules tasks to nodes with sufficient computing power, and configures corresponding vGPU resources.
[0134] 5. Task execution and monitoring: Tasks are executed on the assigned vGPU. The scheduling and allocation device continuously monitors the task execution status and GPU load, and dynamically adjusts the computing power quota of the vGPU as needed to ensure efficient operation of the task.
[0135] 6. Resource release and recycling: After the task is completed, the occupied vGPU resources are released and the CRD information in the Kubernetes cluster is updated. In addition, the hardware registration device can regularly clean up invalid or faulty GPU resource information.
[0136] It can be seen that the GPU access method in the cloud platform provided by the embodiment of the present application determines the target GPU that applies for access to the cloud platform in response to the detection instruction, and automatically obtains its basic information and performance information through the command line interface, wherein the computing power resources of the target GPU can be evaluated through the performance information of the target GPU, so as to realize the cloud platform to verify the validity of the basic information and computing power of the target GPU, and then realize the automatic access of the target GPU that meets the cloud platform access standard to the cloud platform. It can be seen that the technical solution provided by the present application can realize the automatic identification and automatic access of the cloud platform to the GPU device, and further ensure the application efficiency of the GPU.
[0137] An embodiment of the present application provides a GPU access device in a cloud platform.
[0138] Please refer to Figure 3 , Figure 3This is a structural schematic diagram of a GPU access device in a cloud platform provided by the present application. The GPU access device in the cloud platform may include:
[0139] Acquisition module 1, used to determine the target GPU according to the detection instruction, and obtain basic information and performance information of the target GPU through the command line interface;
[0140] Processing module 2, used to convert each performance information into computing power evaluation index data, and use a preset computing power evaluation model to process each computing power evaluation index data to obtain a computing power evaluation result of the target GPU;
[0141] Verification module 3, used to perform standard verification on the basic information and computing power evaluation results of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform;
[0142] The registration module 4 is used to register the target GPU to the cloud platform when the target GPU meets the GPU access standard of the cloud platform.
[0143] It can be seen that the GPU access device in the cloud platform provided by the embodiment of the present application determines the target GPU that applies for access to the cloud platform in response to the detection instruction, and automatically obtains its basic information and performance information through the command line interface, wherein the computing power resources of the target GPU can be evaluated through the performance information of the target GPU, so as to realize the cloud platform to verify the validity of the basic information and computing power of the target GPU, and then realize the automatic access of the target GPU that meets the cloud platform access standard to the cloud platform. It can be seen that the technical solution provided by the present application can realize the automatic identification and automatic access of the cloud platform to the GPU device, and further ensure the application efficiency of the GPU.
[0144] In one embodiment of the present application, the GPU access device in the cloud platform may also include:
[0145] A partitioning module is used to determine the GPU computing power value according to the computing power evaluation result of the target GPU after registering the target GPU to the cloud platform when the target GPU meets the GPU access standard of the cloud platform; perform computing power partitioning on the target GPU according to a preset partitioning threshold to obtain multiple virtual GPUs; wherein the GPU computing power value of each virtual GPU is the preset partitioning threshold;
[0146] An allocation module is used to determine the task priority and computing power requirement value of the task request when receiving a task request sent by a client; add the task request to a task request queue according to the priority of the task request; wherein each task request in the task request queue is executed in order from high to low priority; for each task request in the task request queue, determine the adapted GPU and the target virtual GPU on the adapted GPU according to the computing power requirement value of the task request; distribute the task request to the adapted GPU so that the adapted GPU uses the target virtual GPU to execute the task request.
[0147] In one embodiment of the present application, the GPU access device in the cloud platform may also include:
[0148] A generation module is used to obtain the computing power allocation information of each GPU in the cloud platform after distributing the task request to the adapted GPU so that the adapted GPU uses the target virtual GPU to execute the task request; generate a GPU computing power resource allocation table in the cloud platform according to the basic information, performance information, and computing power allocation information of each GPU; wherein the basic information of the GPU includes one or more combinations of the model information, configuration information, and supported task types of the GPU; the performance information of the GPU includes one or more combinations of the floating-point operation speed, memory information, bandwidth information, and power consumption information of the GPU.
[0149] In one embodiment of the present application, the GPU access device in the cloud platform may also include:
[0150] The release module is used to release the computing power resources of the target virtual GPU on the adapter GPU when the completion information of the task request is obtained, and to update the GPU computing power resource allocation table in the cloud platform.
[0151] In one embodiment of the present application, the above-mentioned acquisition module 1 can be specifically used to send an information acquisition command to the target GPU through a command line interface; wherein the information acquisition command is an NVIDIA-SMI command or a ROCm command; and obtain basic information and performance information fed back by the target GPU according to the information acquisition command.
[0152] In one embodiment of the present application, the acquisition module 1 may be specifically configured to respond to a detection instruction when access information is sensed through a preset interface; and determine a target GPU according to the detection instruction.
[0153] In one embodiment of the present application, the GPU access device in the cloud platform may also include:
[0154] The prompt module is used to output an access exception prompt when the target GPU does not meet the GPU access standard of the cloud platform; wherein the access exception prompt includes basic information and performance information of the target GPU and verification information of the basic information and verification information of the performance information.
[0155] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.
[0156] An embodiment of the present application provides an electronic device.
[0157] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application, and the electronic device may include:
[0158] A memory 11, used for storing computer programs;
[0159] The processor 10 can implement the steps of any of the above-mentioned methods for accessing a GPU in a cloud platform when executing a computer program.
[0160] like Figure 4 , which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.
[0161] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.
[0162] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of a method for accessing a GPU in a cloud platform.
[0163] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:
[0164] Determine the target GPU according to the detection instruction, and obtain the basic information and performance information of the target GPU through the command line interface;
[0165] Convert each performance information into computing power evaluation index data, and use the preset computing power evaluation model to process each computing power evaluation index data to obtain the computing power evaluation result of the target GPU;
[0166] Perform standard verification on the target GPU's basic information and computing power evaluation results to determine whether the target GPU meets the cloud platform's GPU access standards;
[0167] When the target GPU meets the GPU access standard of the cloud platform, register the target GPU to the cloud platform.
[0168] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.
[0169] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.
[0170] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.
[0171] Of course, it should be noted that Figure 4 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 4 More or fewer components than shown, or combinations of certain components.
[0172] An embodiment of the present application provides a computer-readable storage medium.
[0173] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of the GPU access method in any of the above-mentioned cloud platforms can be implemented.
[0174] Among them, the computer-readable storage medium can be any available medium that can be stored in a computer or a data storage device such as a server or data center that includes one or more available media integrated. For example, it can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape, etc.), an optical medium (such as a DVD) or a semiconductor medium (such as a solid-state hard disk) and other media that can store computer program codes.
[0175] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.
[0176] An embodiment of the present application provides a computer program product.
[0177] The computer program product provided in the embodiments of the present application includes a computer program / instruction, which, when executed by a processor, can implement the steps of any of the above-mentioned methods for accessing a GPU in a cloud platform.
[0178] Specifically, in the above embodiments, all or part of them can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of them can be implemented in the form of a computer program product.
[0179] Among them, the computer program product may include one or more computer programs / instructions, and when the computer program / instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application may be generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0180] For an introduction to the computer program product provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.
[0181] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0182] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0183] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0184] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.
Claims
1. A GPU access method in a cloud platform, characterized in that: include: Determine the target GPU according to the detection instruction, and obtain basic information and performance information of the target GPU through a command line interface; Convert each of the performance information into computing power evaluation index data, and process each of the computing power evaluation index data using a preset computing power evaluation model to obtain a computing power evaluation result of the target GPU; Performing a standard check on the basic information and computing power evaluation result of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform; When the target GPU meets the GPU access standard of the cloud platform, the target GPU is registered to the cloud platform.
2. The GPU access method in the cloud platform according to claim 1, characterized in that: When the target GPU meets the GPU access standard of the cloud platform, after registering the target GPU to the cloud platform, the method further includes: Determine the GPU computing power value according to the computing power evaluation result of the target GPU; Divide the computing power of the target GPU according to a preset division threshold to obtain multiple virtual GPUs; wherein the GPU computing power value of each virtual GPU is the preset division threshold; Accordingly, the method further comprises: When receiving a task request sent by a client, determining the task priority and computing power requirement value of the task request; Adding the task request to a task request queue according to the priority of the task request; wherein each task request in the task request queue is executed in order of priority from high to low; For each task request in the task request queue, determining an adapted GPU and a target virtual GPU on the adapted GPU according to a computing power requirement value of the task request; The task request is distributed to the adapted GPU, so that the adapted GPU executes the task request using the target virtual GPU.
3. The GPU access method in the cloud platform according to claim 2, characterized in that: After distributing the task request to the adapted GPU so that the adapted GPU executes the task request using the target virtual GPU, the method further includes: Obtaining computing power allocation information of each GPU in the cloud platform; Generate a GPU computing power resource allocation table in the cloud platform according to the basic information, performance information, and computing power allocation information of each GPU; The basic information of the GPU includes one or more combinations of the model information, configuration information, and supported task types of the GPU; The performance information of the GPU includes one or more combinations of floating-point operation speed, memory information, bandwidth information, and power consumption information of the GPU.
4. The GPU access method in the cloud platform according to claim 3, characterized in that: Also includes: When the completion information of the task request is obtained, the computing power resources of the target virtual GPU on the adapted GPU are released, and the GPU computing power resource allocation table in the cloud platform is updated.
5. The GPU access method in the cloud platform according to claim 1, characterized in that: Obtain basic information and performance information of the target GPU through the command line interface, including: Sending an information acquisition command to the target GPU through the command line interface; wherein the information acquisition command is an NVIDIA-SMI command or a ROCm command; The basic information and the performance information fed back by the target GPU according to the information acquisition command are obtained.
6. The GPU access method in the cloud platform according to claim 1, characterized in that: Determine the target GPU based on the detection instructions, including: When access information is sensed through a preset interface, responding to the detection instruction; The target GPU is determined according to the detection instruction.
7. The GPU access method in the cloud platform according to claim 1, characterized in that: Also includes: When the target GPU does not meet the GPU access standard of the cloud platform, an access exception prompt is output; The access abnormality prompt includes basic information and performance information of the target GPU, verification information of the basic information, and verification information of the performance information.
8. A GPU access device in a cloud platform, characterized in that: include: An acquisition module, used to determine a target GPU according to a detection instruction, and obtain basic information and performance information of the target GPU through a command line interface; A processing module, used to convert each of the performance information into computing power evaluation index data, and use a preset computing power evaluation model to process each of the computing power evaluation index data to obtain a computing power evaluation result of the target GPU; A verification module, used to perform a standard verification on the basic information and computing power evaluation result of the target GPU to determine whether the target GPU meets the GPU access standard of the cloud platform; A registration module is used to register the target GPU to the cloud platform when the target GPU meets the GPU access standard of the cloud platform.
9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the GPU access method in the cloud platform as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the GPU access method in the cloud platform according to any one of claims 1 to 7 are implemented.
