Cluster GPU utilization rate management method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of software monitoring and finance, and relates to a cluster GPU utilization rate management method, which comprises the steps of dividing a cluster into namespaces, setting a business GPU resource quota and a public GPU resource quota of a public namespace for each business namespace, and allocating GPU resources for each business namespace, resource monitoring data of service GPU resources of all service namespaces are collected in real time, when idle service namespaces exist, the resources of the idle service namespaces are recycled to the public namespace, and the resources of the idle service namespaces are allocated according to the GPU resource application, the current resource use condition and a preset resource allocation priority rule. And allocating the GPU resources in the public namespace to the target business namespace. The invention further provides a cluster GPU utilization rate management device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the resource monitoring data can be stored in a block chain. According to the method, the GPU resources can be dynamically scheduled, and the utilization rate of the GPU resources is maximized.

Description

Technical Field

[0001] The present application relates to the fields of software monitoring and financial technology, and in particular to a method, device, computer equipment and storage medium for managing cluster GPU utilization. Background Art

[0002] In the financial field, with the rapid development of artificial intelligence and big data technologies, GPUs (Graphics Processing Units) have become a key computing resource for processing complex financial models, big data analysis, and real-time trading. However, traditional GPU resource management methods face many challenges in financial applications.

[0003] First, the financial sector has extremely high requirements for the stability and efficiency of computing resources, but traditional methods often fail to maximize the use of GPU resources when managing them, resulting in waste and inefficiency of GPU resources. This not only increases the operating costs of financial institutions, but also limits the processing speed and accuracy of financial services.

[0004] In view of the above problems, the prior art has proposed some solutions, but there are still deficiencies in the application of the financial field. For example, the multi-type GPU cluster resource management and scheduling method and system for deep learning proposed by Chinese patent CN201910430474.7 can divide the GPU cluster into several GPU groups according to the GPU model, obtain the highest priority pending job in the job queue, and schedule the GPU cluster resources according to the pending job. Although this method improves the flexibility of resource scheduling to a certain extent, it still cannot achieve complete isolation and dynamic scheduling of GPU resources, resulting in potential security risks when processing sensitive financial data, and further improvement of resource utilization is limited. On the other hand, patent CN202211442603.4 proposes a multi-tenant-based Kubernetes cluster resource management method and system, which creates a tenant's ID and namespace (nameSpace) through resource quota information and pre-configured quota policies. Although this method has made certain progress in resource management and multi-tenant support, it fails to make full use of Kubernetes technology for dynamic scheduling between different namespaces, and does not realize comprehensive management of GPU resources by combining multiple monitoring indicators, thus failing to meet the needs of the financial field for refined resource management and efficient utilization. In addition, patent CN202110224312.0 proposes a cluster resource management and scheduling method that can divide cluster resources into multiple resource pools to achieve task isolation. However, this method does not use namespaces to manage virtualized resources in groups, resulting in a complex and cumbersome process for scheduling group resources and adjusting the number of resources in each group, which cannot adapt to the rapidly changing business needs in the financial field.

[0005] In general, existing technologies cannot effectively improve cluster GPU utilization while reducing costs. While achieving resource grouping isolation, they can implement dynamic calls through a public namespace and combine multiple monitoring indicators to achieve resource management of the entire cluster, thereby reducing costs while improving GPU resource utilization. Summary of the invention

[0006] The purpose of the embodiments of the present application is to propose a cluster GPU utilization management method, device, computer equipment and storage medium to solve the technical problem that the prior art cannot simultaneously take into account cost reduction, resource grouping isolation and resource dynamic scheduling in GPU cluster resource management.

[0007] In order to solve the above technical problems, the present application embodiment provides a method for managing cluster GPU utilization, which adopts the following technical solution:

[0008] Determine a namespace division method according to a preset division condition, and divide the cluster into namespaces based on the namespace division method, wherein the namespaces include multiple business namespaces and a public namespace;

[0009] Setting a service GPU resource quota for each service namespace, and obtaining a public GPU resource quota for the public namespace according to the service GPU resource quota;

[0010] Allocate GPU resources to each of the service namespaces according to the service GPU resource quota and the public GPU resource quota to obtain service GPU resources;

[0011] Collect resource monitoring data of service GPU resources of each service namespace in real time;

[0012] According to the resource monitoring data and the preset scheduling policy, it is determined whether there is an idle service namespace, and if so, a resource recycling process is triggered to recycle the service GPU resources of the idle service namespace into the public namespace;

[0013] A GPU resource application request of a target business namespace is received, and whether an allocation condition is met according to application information in the GPU resource application request, current GPU resource usage data, and a preset resource allocation priority rule is determined; if the allocation condition is met, the GPU resources in the public namespace are allocated to the target business namespace.

[0014] In order to solve the above technical problems, the embodiment of the present application also provides a cluster GPU utilization management device, which adopts the following technical solution:

[0015] A partitioning module, used to determine a namespace partitioning method according to a preset partitioning condition, and partition a cluster into namespaces based on the namespace partitioning method, wherein the namespaces include a plurality of business namespaces and a public namespace;

[0016] A setting module, used to set a business GPU resource quota for each business namespace, and obtain a public GPU resource quota for the public namespace according to the business GPU resource quota;

[0017] A resource allocation module, configured to allocate GPU resources to each of the service namespaces according to the service GPU resource quota and the public GPU resource quota to obtain service GPU resources;

[0018] A collection module, used for collecting resource monitoring data of service GPU resources of each service namespace in real time;

[0019] A resource recovery module, used to determine whether there is an idle service namespace according to the resource monitoring data and a preset scheduling policy, and if so, trigger a resource recovery process to recover the service GPU resources of the idle service namespace into the public namespace;

[0020] The resource scheduling module is used to receive a GPU resource application request of a target business namespace, and determine whether an allocation condition is met based on the application information in the GPU resource application request, current GPU resource usage data, and a preset resource allocation priority rule; if the allocation condition is met, the GPU resources in the public namespace are allocated to the target business namespace.

[0021] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0022] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the cluster GPU utilization management method as described above when executing the computer-readable instructions.

[0023] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0024] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the cluster GPU utilization management method as described above are implemented.

[0025] Compared with the prior art, this application has the following beneficial effects:

[0026] The present application provides a method for managing cluster GPU utilization, which realizes mutual isolation of resources by dividing a cluster into namespaces according to preset division conditions; sets a business GPU resource quota for each business namespace, obtains a public GPU resource quota of a public namespace according to the business GPU resource quota, and allocates GPU resources to each business namespace according to the business GPU resource quota and the public GPU resource quota, so as to reasonably allocate resources and improve the utilization of GPU resources; collects resource monitoring data of business GPU resources of each business namespace in real time, triggers a resource recovery process when there is an idle business namespace according to the resource monitoring data and a preset scheduling strategy, and recovers the business GPU resources of the idle business namespace into the public namespace, so as to dynamically adjust the resource quota, avoid each namespace from occupying resources, improve the utilization of GPU resources, and reduce costs; receives a GPU resource application request from a business namespace, judges whether the allocation condition is met according to the application information in the GPU resource application request, the current resource usage status, and the preset resource allocation priority rule, and allocates the GPU resources in the public namespace to the target business namespace of the application when the allocation condition is met, so as to dynamically schedule GPU resources and maximize the utilization of GPU resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0029] Figure 2 is a flow chart of an embodiment of a method for managing cluster GPU utilization according to the present application;

[0030] Figure 3 is a structural diagram of an embodiment of a cluster GPU utilization management device according to the present application;

[0031] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0036] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0037] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0038] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0039] It should be noted that the cluster GPU utilization management method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the cluster GPU utilization management device is generally set in the server / terminal device.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0041] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for managing cluster GPU utilization according to the present application, comprising the following steps:

[0042] Step S201 , determining a namespace division method according to a preset division condition, and dividing the cluster into namespaces based on the namespace division method, wherein the namespaces include a plurality of business namespaces and a public namespace.

[0043] In this embodiment, the cluster is a k8s cluster, which consists of a group of physical or virtual machines that are organized into a single computing resource pool and run the Kubernetes platform on it. The main purpose of the K8s cluster is to provide high availability, scalability, and fault tolerance to ensure that the application can always run normally.

[0044] The preset division conditions include one or more combinations of division according to organizational structure, business classification, function and purpose. The namespace division method is determined according to the preset division conditions, including namespace name, resource type, resource restriction and access rights. Based on the namespace division method, the K8s cluster is divided into n business namespace groups and one public namespace group, where n business namespaces include nameSpace1, nameSpace2, nameSpace3... nameSpace n , where n is a positive integer; the public namespace is public nameSpace. The identification information and division conditions of each namespace (including multiple business namespaces and public namespaces) are recorded in the resource management database, and the resource quota mechanism is adopted to set the business GPU resource quota for each business namespace, and the quota information is stored in the resource management database.

[0045] Resource Quotas is a mechanism used in Kubernetes to limit resource usage within a namespace. It can help the cluster management module (K8s Cluster) control resource usage and prevent a single namespace or user from occupying too many resources and affecting the performance of other users or applications.

[0046] In this embodiment, the management module K8s Cluster is used to uniformly manage n different namespace groups and publicnamespace groups to achieve resource isolation. For example, according to functional division, the k8s cluster can be divided into different namespace groups such as R&D, testing, and production. For example, nameSpace1 is used for R&D environment, nameSpace2 is used for testing environment, nameSpace3 is used for production environment, etc. The public nameSpace group is used for resource scheduling and recycling.

[0047] For example, GPU k8s clusters are divided into training clusters and low-priority training, debugging and other functional clusters according to their uses. The clusters grouped by public nameSpace are low-level training and debugging clusters, and other nameSpaces are clusters for training purposes. Clusters grouped by public nameSpace can be further divided into low-level training clusters and debugging and other functional clusters according to their uses, or distinguished by namespaces, such as public nameSpace_train for training clusters and public nameSpace_debug for debugging and other functional clusters. Of course, other methods can also be used to distinguish clusters of different uses in public, such as adding the prefix train or debug to the name, and the distinction method is not limited here.

[0048] Step S202: Set a service GPU resource quota for each service namespace, and obtain a public GPU resource quota for a public namespace according to the service GPU resource quota.

[0049] Specifically, the historical resource usage data and business priority information of each business namespace are obtained, and the weight coefficient of each business namespace is obtained according to the business priority information; based on the historical resource usage data and the weight coefficient, the resource usage growth rate of each business namespace in a preset time period is predicted; according to the resource usage growth rate and the weight coefficient, the maximum GPU quota of each business namespace is calculated using an optimization algorithm; based on the maximum GPU quota, the minimum GPU quota of each business namespace is determined; the maximum GPU quota and the minimum GPU quota are used as the business GPU resource quota of the corresponding business namespace; and the public GPU resource quota is calculated according to the maximum GPU quota and the minimum GPU quota.

[0050] Among them, historical resource usage data includes time series data such as GPU vacancy rate, CPU usage rate, memory usage rate, etc. Different weight coefficients are assigned according to the business priority information of each business namespace. The business namespace with high priority has a large weight coefficient. For example, the business namespace is divided into three categories of high, medium and low priorities according to the business priority, and the weight coefficients assigned are 5, 3, and 1 respectively.

[0051] In this embodiment, a time series prediction algorithm is used to predict the resource usage growth rate of each business namespace in the future (i.e., a preset time period, such as one week, one month, or half a year, etc.) based on the historical resource usage data and weight coefficient of each business namespace. The time series prediction algorithm can use a trained LSTM time series prediction model for prediction.

[0052] In this embodiment, the step of using the optimization algorithm to calculate the maximum GPU quota of each service namespace includes:

[0053] Through the clustering algorithm, clustering is performed based on the resource usage data of each business namespace, and business namespaces with similar resource usage patterns are divided into the same category. In each cluster, based on the predicted resource usage growth rate and weight coefficient, the objective function is to minimize the total GPU quota, and the constraints are the resource usage growth rate and weight coefficient of each business namespace. A MILP (mixed integer linear programming) model is constructed to solve the optimal GPU quota allocation of the business namespace, that is, the maximum GPU quota, then the minimum GPU quota = maximum GPU quota * preset ratio, where the preset ratio can be set according to the actual situation, for example, the minimum GPU quota is set to 70% of the maximum GPU quota.

[0054] In this embodiment, the quota information is stored in a preset resource quota database according to a predefined data structure, wherein the quota information includes fields such as namespace identifier, maximum GPU quota, minimum GPU quota, and current GPU quota. It should be understood that the GPU quota is the number of GPUs that a namespace can apply for or use.

[0055] The maximum GPU quota and the minimum GPU quota of each business namespace are calculated, and the public GPU resource quota of the public namespace can be inferred based on the maximum GPU quota and the minimum GPU quota.

[0056] For example, assuming there are 3 namespaces, then n = 3, G1_high = 16, G2_high = 14, G3_high = 10, G1_low = 16*70% = 11, G2_low = 14*70% = 9, G3_low = 10*70% = 7, then the public GPU resource quota of the public namespace is G p =G 总 -G_low 总 =(16+14+10)-(11+9+7)=13.

[0057] If the calculated number of GPUs has decimals, it can be rounded down. It should be noted that this application does not limit the processing method for rounding off decimals in the number of GPUs, and other methods can be used.

[0058] By comprehensively considering the historical resource usage data of the business namespace, business priority information, and resource usage growth rate to calculate the business GPU resource quota and public GPU resource quota, it is possible to achieve reasonable allocation of GPU resources, minimize idle waste of GPU resources while meeting the business needs of each business namespace, and achieve efficient use of GPU resources in the entire cluster.

[0059] Step S203: allocate GPU resources to each service namespace according to the service GPU resource quota and the public GPU resource quota to obtain service GPU resources.

[0060] Specifically, an application request for GPU resources from a business namespace is received, and the number of GPU applications in the business namespace is obtained according to the application request; if the number of GPU applications is greater than the maximum GPU quota, the application request is rejected, and GPU resources are allocated according to the maximum GPU quota; if the number of GPU applications is less than the minimum GPU quota, it is determined whether the idle public GPU resource quota in the public namespace is greater than the minimum GPU quota; when the public GPU resource quota is greater than the minimum GPU quota, the idle public GPU resources are scheduled from the public namespace to be allocated to the corresponding business namespace.

[0061] The application request is an application request for GPU resources by each business namespace before the task starts.

[0062] In this embodiment, after receiving an application request for GPU resources from a business namespace, the quota controller of the k8s cluster obtains the GPU quota information of the business namespace, and performs quota control and allocation on the GPU resources applied for by the business namespace according to the GPU quota information. The application request is parsed to obtain the number of GPU applications for the business namespace; if the number of GPU applications exceeds the maximum GPU quota, the quota controller rejects the application request of the business namespace; if the number of GPU applications is lower than the minimum GPU quota, idle public GPU resources are scheduled from the public namespace to be allocated to the business namespace until the GPU resources of the business namespace reach its minimum GPU quota.

[0063] Before scheduling idle public GPU resources in the public namespace to be allocated to the business namespace, determine whether the available idle public GPU resources in the public namespace are sufficient for scheduling. If the public namespace has insufficient GPU resources, the resource expansion module automatically purchases GPU resources based on usage demand and adds the newly added GPU resources to the resource pool of the public namespace. The resource quota management module can regularly compare resource allocation with quota settings, and dynamically adjust the GPU resource quota of each business namespace according to actual usage and changes in business needs, update the resource quota database, realize elastic scaling and dynamic balance of GPU resources, and improve GPU resource utilization.

[0064] In some optional implementations, if the GPU resources in the public namespace are insufficient, before purchasing GPU resources, a review is initiated through the k8s Cluster management module to coordinate the withdrawal of GPU resources from the public namespace. The withdrawal order can be sorted according to the priority of the tasks running in the public namespace, and low-priority task resources are allocated to each business namespace first. If the GPU resource utilization rate is very high, resulting in a small number of GPUs in the public namespace group, additional GPU resources are purchased.

[0065] By monitoring the GPU resource application requests of the business namespace, resource allocation can be optimized in real time according to the changes in the GPU resource usage of each business namespace, thereby improving GPU resource utilization and avoiding GPU resource waste.

[0066] Step S204: collecting resource monitoring data of the service GPU resources of each service namespace in real time.

[0067] In this embodiment, the GPU resource usage of each business namespace is collected in real time through the resource monitoring system, including GPU utilization, CPU utilization, memory utilization, disk utilization and other resource usage indicators as monitoring indicators, and the resource usage data corresponding to the monitoring indicators are resource monitoring data. The collected resource monitoring data is stored in the monitoring database.

[0068] Real-time collection of resource monitoring data for each namespace can enable real-time monitoring of resource usage in each namespace, helping to dynamically optimize resource allocation and improve resource utilization.

[0069] In this embodiment, clusters with different purposes are managed using different monitoring indicators. For example, indicator 1 is used for clusters for training purposes, such as GPU idle rate r1, CPU usage rate r2, etc. Clusters with debugging and other functions use indicator 2, such as memory usage rate r3, disk usage rate r4, and off-duty time t1.

[0070] It should be emphasized that in order to further ensure the privacy and security of resource monitoring data, the above resource monitoring data can also be stored in a blockchain node.

[0071] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0072] Step S205 , judging whether there is an idle service namespace according to the resource monitoring data and the preset scheduling policy, and if so, triggering the resource recycling process to recycle the service GPU resources of the idle service namespace into the public namespace.

[0073] Specifically, the resource monitoring data is compared with the GPU resource quota information of the business namespace to obtain the GPU resource utilization rate and GPU resource idle rate of the business namespace; when there is a GPU resource utilization rate less than a preset utilization rate threshold, and the GPU resource idle rate is higher than the preset idle rate threshold, the corresponding business namespace is an idle business namespace; the priority information of the tasks running in the idle business namespace is obtained, and the tasks are sorted from low to high according to the priority information to obtain the task sorting result; the task sorting result is traversed from the task with the lowest priority to determine whether scheduling the business GPU resources occupied by the currently traversed task to the public namespace will exceed the upper limit of the public GPU resource quota of the public namespace; if it will not exceed the upper limit, the business GPU resources occupied by the currently traversed task are scheduled to the public namespace, and the GPU resource usage data of the corresponding business namespace is updated; until all the tasks of the idle business namespaces are checked, or the GPU resources of the public namespace have reached the quota upper limit, the resource recovery process is completed.

[0074] By using the preset utilization rate threshold and the preset idle rate threshold, it is determined whether the GPU resources of each business namespace are idle. When the GPU resource utilization rate is less than the preset utilization rate threshold, or the GPU resource idle rate is higher than the preset idle rate threshold, the resource recovery process is triggered. The GPU resource information of the idle business namespace is obtained, including the number and model of GPUs, and these idle GPU resources are temporarily recovered to the resource pool of the public namespace. The dynamic scheduling system allocates idle GPU resources from the resource pool of the public namespace to the task based on the priority of the task, the type and quantity of required GPU resources, etc., to improve the utilization of GPU resources.

[0075] In this embodiment, all idle business namespaces are determined, and the idle business namespaces are sorted from low to high according to the GPU resource utilization rate, and the priority information of the tasks running in the idle business namespace is obtained, and the tasks are sorted from low to high according to the priority. Starting from the idle business namespace with the lowest GPU resource utilization rate, all tasks are traversed in order from low to high priority, and it is determined whether scheduling the GPU resources occupied by the current task to the public namespace will exceed the GPU quota upper limit of the public namespace. If it does not exceed the upper limit, the GPU resources occupied by the task are scheduled to the public namespace, and the GPU resource usage data of the business namespace is updated. At the same time, the scheduling record is written into the log database for subsequent analysis and optimization of resource utilization.

[0076] By recycling the GPU resources of idle business namespaces into the resource pool of the public namespace, GPU resource utilization can be maximized, avoiding idle resources in each business namespace, improving the GPU resource utilization of each business namespace and the overall GPU resource utilization, and saving costs.

[0077] Step S206, receiving a GPU resource application request of a target business namespace, and judging whether an allocation condition is met based on the application information in the GPU resource application request, current GPU resource usage data, and a preset resource allocation priority rule; if the allocation condition is met, the GPU resources in the public namespace are allocated to the target business namespace.

[0078] Specifically, the current GPU resource usage data of the target business namespace and the application information in the GPU resource application request are obtained, and the application information includes the currently applied GPU quantity, usage time and priority parameters; the priority score of the target business namespace is calculated according to the preset resource allocation priority rule and the current GPU resource usage data; it is determined whether the currently applied GPU quantity exceeds the maximum GPU quota of the target business namespace, and if so, the application is rejected and a prompt message is returned; it is determined whether the number of available GPUs in the public namespace meets the application requirements of the target business namespace; when the application requirements of the target business namespace are not met, GPU resources are scheduled from the idle resources of other business namespaces to the public namespace according to the order of priority scores from low to high; when the GPU resources in the public namespace still cannot meet the application requirements, the application is rejected and a prompt message of insufficient resources is returned; when the number of available GPUs in the public namespace meets the application requirements, a corresponding number of GPU resources are allocated from the public namespace to the target business namespace.

[0079] The target business namespace is the business namespace currently applying for GPU resources; the GPU resource application request is a GPU resource application during task execution based on actual usage and changes in business requirements. Among them, the preset resource allocation priority rule is a priority score calculation method. In this embodiment, a weighted average algorithm is used to calculate the priority score based on the current GPU resource usage data and priority parameters of the target business namespace.

[0080] The number of available GPUs in the public namespace meets the application requirements of the target business namespace, indicating that the number of available GPUs in the public namespace is greater than or equal to the number of GPUs applied for by the target business namespace.

[0081] In some optional implementations, a corresponding number of GPU resources are allocated from the resource pool of the public namespace to the requested target business namespace, and resource usage data is updated at the same time, that is, information such as the number of GPUs allocated this time and the duration of use is recorded in the statistical indicators of the corresponding business namespace and the public namespace, and information on successful GPU resource allocation is returned, including parameters such as the number of allocated GPUs, duration of use, and expiration time.

[0082] By dynamically scheduling GPU resources according to resource allocation priority rules and resource usage, GPU resource utilization can be maximized.

[0083] The present application achieves mutual isolation of resources by dividing the cluster into namespaces; allocates GPU resources to each business namespace according to the business GPU resource quota and the public GPU resource quota, which can reasonably allocate resources and improve the utilization rate of GPU resources; collects resource monitoring data of the business GPU resources of each business namespace in real time, and triggers the resource recovery process when there is an idle business namespace according to the resource monitoring data and the preset scheduling strategy, and recovers the business GPU resources of the idle business namespace to the public namespace, which can dynamically adjust the resource quota to avoid each namespace from occupying resources, improve GPU resource utilization and reduce costs; determines whether the allocation conditions are met according to the application information in the GPU resource application request, the current GPU resource usage data and the preset resource allocation priority rules, and when the allocation conditions are met, allocates the GPU resources in the public namespace to the target business namespace, which can dynamically schedule GPU resources and maximize the utilization rate of GPU resources.

[0084] In some optional implementations, after the step of allocating the GPU resources in the public namespace to the target service namespace, the method further includes:

[0085] Regularly obtain resource usage, quota usage, and business priority data for each business namespace;

[0086] Calculate the resource balance of each business namespace based on resource utilization, quota utilization, and business priority data;

[0087] Determine whether there is an imbalance in resource allocation based on the resource balance;

[0088] If there is an imbalance in resource allocation, the number of GPU resources migrated between different business namespaces is calculated based on the resource balancing algorithm;

[0089] Migrate the number of migrated GPU resources to the corresponding business namespace in an incremental migration manner;

[0090] After the migration is completed, the resource usage and quota usage of each business namespace after the migration are obtained as updated resource data;

[0091] Based on the updated resource data, determine whether the resources of each business namespace are balanced. If not, continue migration until resources are balanced.

[0092] The resource utilization rate is the GPU utilization rate. In this embodiment, a weighted average algorithm is used to calculate the resource balance of each business namespace. For example, the weights of resource utilization rate, quota utilization rate and business priority data are 4, 3 and 3 respectively, then resource balance rate = resource utilization rate * 4 + quota utilization rate * 3 + business priority * 3. If the calculated resource balance rate is greater than or equal to the preset balance threshold, it means that there is an imbalance in resource allocation in the business namespace, and the rebalancing process is triggered.

[0093] According to the resource balancing algorithm, the number of GPU resources to be migrated between different business namespaces is calculated. For example, 10% of GPU resources are selected from business namespaces with resource utilization lower than 50%, quota utilization lower than 60%, and business priority of medium or low, and migrated to business namespaces with resource utilization higher than 80%, quota utilization higher than 90%, and business priority of high.

[0094] In this embodiment, an incremental migration method is adopted, that is, a small portion of GPU resources is migrated each time until the target migration quantity is reached, to avoid business interruption caused by a large number of migrations at one time. After achieving resource balance, the migration results are synchronized to the K8s cluster management system, the GPU resource quota of the business namespace is updated, the GPU resource rebalancing across business namespaces is completed, and the balancing process and results are recorded to provide data support for subsequent optimization of the resource balancing algorithm.

[0095] In some optional implementations, a resource monitoring component is used to monitor the GPU resource usage of each business namespace in real time, obtain the current GPU usage of each business namespace, and if the GPU usage of a business namespace exceeds its maximum GPU quota, an alarm is triggered and the over-limit event is recorded in the scheduling decision log.

[0096] This application implements resource balancing calculation for business namespaces, thereby realizing resource migration between business namespaces, achieving dynamic resource balancing, ensuring the consistency of cluster GPU resource status, and improving the resource utilization of the entire cluster.

[0097] In some optional implementations, after the step of allocating the GPU resources in the public namespace to the target service namespace, the method further includes:

[0098] According to the preset idle resource monitoring cycle, GPU resource usage monitoring indicator data of each business namespace is regularly collected;

[0099] Compare the GPU resource usage monitoring indicator data with the preset resource recycling trigger conditions to determine whether the resource recycling trigger conditions are met;

[0100] When the resource recycling trigger conditions are met, the GPU resource dynamic expansion and contraction mechanism is started to temporarily recycle the idle GPU resources to the public namespace;

[0101] While reclaiming GPU resources, update the resource management policy of the public namespace and adjust the resource allocation policy based on the resource demand priority of the business namespace.

[0102] The preset idle resource monitoring period may be set according to actual conditions, for example, one week, half a month, one month or half a year, etc., and is not limited here.

[0103] Specifically, the GPU resource usage monitoring indicator data of each business namespace in the k8s cluster is obtained, including GPU usage, CPU usage, memory usage, disk usage, etc.; according to the obtained GPU resource usage monitoring indicator data, the GPU resource utilization of each business namespace is judged to determine whether there is a situation where GPU resources are idle or underutilized; if it is judged that the GPU utilization rate of a certain business namespace is lower than the preset utilization rate threshold, the resource recycling trigger condition is met, and the business namespace is marked as a low utilization group, and the number of GPU resources currently used by the business namespace is obtained; by comparing the number of GPU resources of the business namespace marked as low utilization with its preset minimum GPU quota, the number of idle GPU resources that can be recycled is obtained, and the idle GPU resources are temporarily recycled to the public namespace.

[0104] While reclaiming GPU resources, update the resource management policy of the public namespace, increase the number of allocatable GPU resources, and adjust the resource allocation policy based on the resource demand priority of the business namespace. When the actual GPU resource usage of a business namespace is close to its minimum GPU quota, dynamically dispatch idle GPU resources from the public namespace to the business namespace. When scheduling GPU resources, prioritize business namespaces with high resource demand priorities. If the public namespace has insufficient GPU resources, reclaim GPU resources from low-priority business namespaces to the public namespace based on usage priority. If a large number of GPU resources in the public namespace are idle, reallocate some GPU resources to each business namespace based on the GPU vacancy rate threshold to improve GPU resource utilization.

[0105] By regularly monitoring the actual resource usage and resource demand changes of each business namespace, dynamic resource scheduling is achieved. At the same time, resource management strategies are optimized based on monitoring data to continuously improve GPU resource utilization.

[0106] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0107] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0109] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0110] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a cluster GPU utilization management device, and the device embodiment is similar to Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0111] like Figure 3 As shown, the cluster GPU utilization management device 300 described in this embodiment includes: a division module 301, a setting module 302, a resource allocation module 303, a collection module 304, a resource recovery module 305 and a resource scheduling module 306. Among them:

[0112] The partitioning module 301 is used to determine a namespace partitioning method according to a preset partitioning condition, and partition a cluster into namespaces based on the namespace partitioning method, wherein the namespaces include multiple business namespaces and a public namespace;

[0113] The setting module 302 is used to set a service GPU resource quota for each of the service namespaces, and obtain a public GPU resource quota for the public namespace according to the service GPU resource quota;

[0114] The resource allocation module 303 is used to allocate GPU resources to each of the business namespaces according to the business GPU resource quota and the public GPU resource quota to obtain business GPU resources;

[0115] The collection module 304 is used to collect resource monitoring data of the service GPU resources of each service namespace in real time;

[0116] The resource recovery module 305 is used to determine whether there is an idle service namespace according to the resource monitoring data and the preset scheduling policy, and if so, trigger the resource recovery process to recover the service GPU resources of the idle service namespace into the public namespace;

[0117] The resource scheduling module 306 is used to receive a GPU resource application request from a target business namespace, and determine whether an allocation condition is met based on the application information in the GPU resource application request, current GPU resource usage data, and a preset resource allocation priority rule; if the allocation condition is met, the GPU resources in the public namespace are allocated to the target business namespace.

[0118] It should be emphasized that in order to further ensure the privacy and security of resource monitoring data, the above resource monitoring data can also be stored in a blockchain node.

[0119] The management device 300 based on the above cluster GPU utilization realizes mutual isolation of resources by dividing the cluster into namespaces; allocates GPU resources to each business namespace according to the business GPU resource quota and the public GPU resource quota, which can reasonably allocate resources and improve the utilization of GPU resources; collects resource monitoring data of the business GPU resources of each business namespace in real time, and triggers the resource recovery process when there is an idle business namespace according to the resource monitoring data and the preset scheduling strategy, and recovers the business GPU resources of the idle business namespace to the public namespace, which can dynamically adjust the resource quota to avoid each namespace occupying resources, improve GPU resource utilization and reduce costs; according to the application information in the GPU resource application request, the current GPU resource usage data and the preset resource allocation priority rules, when the allocation conditions are met, the GPU resources in the public namespace are allocated to the target business namespace, which can dynamically schedule GPU resources and maximize the utilization of GPU resources.

[0120] In some optional implementations, the setting module 302 includes:

[0121] An acquisition submodule, used to acquire historical resource usage data and service priority information of each of the service namespaces, and obtain a weight coefficient of each of the service namespaces according to the service priority information;

[0122] A prediction submodule, used to predict the resource usage growth rate of each of the business namespaces in a preset time period based on the historical resource usage data and the weight coefficient;

[0123] An optimization calculation submodule, configured to calculate the maximum GPU quota of each of the service namespaces using an optimization algorithm according to the resource usage growth rate and the weight coefficient;

[0124] A determination submodule, configured to determine a minimum GPU quota for each of the service namespaces based on the maximum GPU quota;

[0125] A quota submodule, configured to use the maximum GPU quota and the minimum GPU quota as the service GPU resource quota of the corresponding service namespace;

[0126] The public resource calculation submodule is used to calculate the public GPU resource quota according to the maximum GPU quota and the minimum GPU quota.

[0127] By comprehensively considering the historical resource usage data of the business namespace, business priority information, and resource usage growth rate to calculate the business GPU resource quota and public GPU resource quota, it is possible to achieve reasonable allocation of GPU resources, minimize idle waste of GPU resources while meeting the business needs of each business namespace, and achieve efficient use of GPU resources in the entire cluster.

[0128] In some optional implementations of this embodiment, the resource allocation module 303 includes:

[0129] A receiving submodule, configured to receive an application request for GPU resources from the business namespace, and obtain a GPU application quantity for the business namespace according to the application request;

[0130] A judgment submodule, configured to reject the application request and allocate GPU resources according to the maximum GPU quota if the GPU application quantity is greater than the maximum GPU quota;

[0131] A comparison submodule, configured to determine whether the idle public GPU resource quota in the public namespace is greater than the minimum GPU quota if the GPU application quantity is less than the minimum GPU quota;

[0132] The allocation submodule is used to schedule idle public GPU resources from the public namespace to be allocated to the corresponding business namespace when the public GPU resource quota is greater than the minimum GPU quota.

[0133] By comprehensively considering the historical resource usage data of the business namespace, business priority information, and resource usage growth rate to calculate the business GPU resource quota and public GPU resource quota, it is possible to achieve reasonable allocation of GPU resources, minimize idle waste of GPU resources while meeting the business needs of each business namespace, and achieve efficient use of GPU resources in the entire cluster.

[0134] In some optional implementations of this embodiment, the resource recovery module 305 includes:

[0135] A comparison submodule, used to compare the resource monitoring data with the GPU resource quota information of the business namespace to obtain the GPU resource utilization rate and GPU resource idle rate of the business namespace;

[0136] An idle resource judgment submodule, used for setting the corresponding service namespace as an idle service namespace when the GPU resource usage rate is less than a preset usage rate threshold, or the GPU resource idle rate is higher than a preset idle rate threshold;

[0137] A sorting submodule, used to obtain priority information of tasks running in the idle service namespace, sort the tasks from low to high according to the priority information, and obtain a task sorting result;

[0138] A traversal submodule, used to traverse the task sorting results starting from the task with the lowest priority, and determine whether scheduling the service GPU resources occupied by the currently traversed task to the public namespace will exceed the upper limit of the public GPU resource quota of the public namespace;

[0139] A scheduling submodule, configured to schedule the service GPU resources occupied by the currently traversed task to the public namespace if the upper limit is not exceeded, and update the GPU resource usage data of the corresponding service namespace;

[0140] The end submodule is used to complete the resource recovery process until all the tasks of the idle business namespace are checked or the GPU resources of the public namespace have reached the quota upper limit.

[0141] By recycling the GPU resources of idle business namespaces into the resource pool of the public namespace, GPU resource utilization can be maximized, avoiding idle resources in each business namespace, improving the GPU resource utilization of each business namespace and the overall GPU resource utilization, and saving costs.

[0142] In some optional implementations of this embodiment, the resource scheduling module 306 includes:

[0143] An information acquisition submodule, used to obtain the current GPU resource usage data of the target service namespace and the application information in the GPU resource application request, wherein the application information includes the number of GPUs currently applied for, the usage duration and the priority parameter;

[0144] A priority calculation submodule, used to calculate the priority score of the target service namespace according to the preset resource allocation priority rule and the current GPU resource usage data;

[0145] The first judgment submodule is used to judge whether the number of GPUs currently applied for exceeds the maximum GPU quota of the target business namespace, and if so, reject the application and return a prompt message;

[0146] The second judgment submodule is used to judge whether the number of available GPUs in the public namespace meets the application requirements of the target business namespace;

[0147] A scheduling submodule, configured to schedule GPU resources from idle resources of other business namespaces to the public namespace according to the order of the priority scores from low to high when the application requirements of the target business namespace are not met;

[0148] A rejection submodule, used for rejecting the application and returning a prompt message of insufficient resources when the GPU resources in the public namespace still cannot meet the application requirements;

[0149] The allocation submodule is used to allocate a corresponding number of GPU resources from the public namespace to the target business namespace when the number of available GPUs in the public namespace meets the application requirements.

[0150] By dynamically scheduling GPU resources according to resource allocation priority rules and resource usage, GPU resource utilization can be maximized.

[0151] In some optional implementations, the cluster GPU utilization management device 300 further includes a resource balancing module, which is used to:

[0152] Regularly obtain resource usage, quota usage and service priority data of each of the service namespaces;

[0153] Calculating the resource balance of each of the service namespaces according to the resource utilization rate, the quota utilization rate and the service priority data;

[0154] Determining whether there is an imbalance in resource allocation according to the resource balance degree;

[0155] If there is an imbalance in resource allocation, the number of GPU resources migrated between different business namespaces is calculated based on the resource balancing algorithm;

[0156] Migrate the number of migrated GPU resources to the corresponding business namespace in an incremental migration manner;

[0157] After the migration is completed, the resource usage rate and quota usage rate of each of the migrated business namespaces are obtained as updated resource data;

[0158] According to the updated resource data, it is determined whether the resources of each business namespace have reached a balanced state. If not, the migration is continued until the resources are balanced.

[0159] By performing resource balancing calculations on business namespaces, resource migration between business namespaces is achieved, dynamic resource balancing is achieved, the consistency of cluster GPU resource status is ensured, and the resource utilization of the entire cluster is improved.

[0160] In some optional implementations, the cluster GPU utilization management device 300 further includes a monitoring and adjustment module, which is used to:

[0161] According to a preset idle resource monitoring cycle, regularly collect GPU resource usage monitoring indicator data of each of the business namespaces;

[0162] Compare the GPU resource usage monitoring indicator data with a preset resource recycling trigger condition to determine whether the resource recycling trigger condition is met;

[0163] When the resource recycling triggering condition is met, the GPU resource dynamic expansion and contraction mechanism is started to temporarily recycle the idle GPU resources to the public namespace;

[0164] While reclaiming the GPU resources, the resource management policy of the public namespace is updated, and the resource allocation policy is adjusted according to the resource demand priority of the business namespace.

[0165] By regularly monitoring the actual resource usage and resource demand changes of each business namespace, dynamic resource scheduling is achieved. At the same time, resource management strategies are optimized based on monitoring data to continuously improve GPU resource utilization.

[0166] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0167] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0168] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0169] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a cluster GPU utilization management method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0170] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the cluster GPU utilization management method.

[0171] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0172] By dividing the cluster into namespaces, resources can be isolated from each other; GPU resources are allocated to each business namespace according to the business GPU resource quota and the public GPU resource quota, so that resources can be allocated reasonably and the utilization rate of GPU resources can be improved; resource monitoring data of business GPU resources of each business namespace is collected in real time, and according to the resource monitoring data and the preset scheduling strategy, when there is an idle business namespace, the resource recovery process is triggered to recover the business GPU resources of the idle business namespace to the public namespace, so that resource quotas can be adjusted dynamically to avoid idle resources in each namespace, improve GPU resource utilization and reduce costs; according to the application information in the GPU resource application request, the current GPU resource usage data and the preset resource allocation priority rules, when the allocation conditions are met, the GPU resources in the public namespace are allocated to the target business namespace, so that GPU resources can be dynamically scheduled to maximize the utilization rate of GPU resources.

[0173] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the cluster GPU utilization management method as described above.

[0174] By dividing the cluster into namespaces, resources can be isolated from each other; GPU resources are allocated to each business namespace according to the business GPU resource quota and the public GPU resource quota, so that resources can be allocated reasonably and the utilization rate of GPU resources can be improved; resource monitoring data of business GPU resources of each business namespace is collected in real time, and according to the resource monitoring data and the preset scheduling strategy, when there is an idle business namespace, the resource recovery process is triggered to recover the business GPU resources of the idle business namespace to the public namespace, so that resource quotas can be adjusted dynamically to avoid idle resources in each namespace, improve GPU resource utilization and reduce costs; according to the application information in the GPU resource application request, the current GPU resource usage data and the preset resource allocation priority rules, when the allocation conditions are met, the GPU resources in the public namespace are allocated to the target business namespace, so that GPU resources can be dynamically scheduled to maximize the utilization rate of GPU resources.

[0175] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0176] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for managing cluster GPU utilization, characterized in that: The steps include: Determine a namespace division method according to a preset division condition, and divide the cluster into namespaces based on the namespace division method, wherein the namespaces include multiple business namespaces and a public namespace; Setting a service GPU resource quota for each service namespace, and obtaining a public GPU resource quota for the public namespace according to the service GPU resource quota; Allocate GPU resources to each of the service namespaces according to the service GPU resource quota and the public GPU resource quota to obtain service GPU resources; Collect resource monitoring data of service GPU resources of each service namespace in real time; According to the resource monitoring data and the preset scheduling policy, it is determined whether there is an idle service namespace, and if so, a resource recycling process is triggered to recycle the service GPU resources of the idle service namespace into the public namespace; A GPU resource application request of a target business namespace is received, and whether an allocation condition is met according to application information in the GPU resource application request, current GPU resource usage data, and a preset resource allocation priority rule is determined; if the allocation condition is met, the GPU resources in the public namespace are allocated to the target business namespace.

2. The cluster GPU utilization management method according to claim 1, characterized in that: The step of setting a service GPU resource quota for each service namespace and obtaining a public GPU resource quota for the public namespace according to the service GPU resource quota comprises: Acquire historical resource usage data and service priority information of each of the service namespaces, and obtain a weight coefficient of each of the service namespaces according to the service priority information; Based on the historical resource usage data and the weight coefficient, predicting the resource usage growth rate of each of the business namespaces in a preset time period; Calculate the maximum GPU quota of each of the business namespaces using an optimization algorithm according to the resource usage growth rate and the weight coefficient; Determine a minimum GPU quota for each of the service namespaces based on the maximum GPU quota; Using the maximum GPU quota and the minimum GPU quota as the service GPU resource quota of the corresponding service namespace; The public GPU resource quota is calculated according to the maximum GPU quota and the minimum GPU quota.

3. The method for managing cluster GPU utilization according to claim 2, characterized in that: The step of allocating GPU resources to each of the service namespaces according to the service GPU resource quota and the public GPU resource quota to obtain the service GPU resources comprises: Receiving an application request for GPU resources from the business namespace, and obtaining a GPU application quantity for the business namespace according to the application request; If the GPU application quantity is greater than the maximum GPU quota, reject the application request and allocate GPU resources according to the maximum GPU quota; If the GPU application quantity is less than the minimum GPU quota, determining whether the idle public GPU resource quota in the public namespace is greater than the minimum GPU quota; When the public GPU resource quota is greater than the minimum GPU quota, idle public GPU resources are scheduled from the public namespace and allocated to the corresponding service namespace.

4. The method for managing cluster GPU utilization according to claim 3, characterized in that: The step of determining whether there is an idle service namespace according to the resource monitoring data and the preset scheduling strategy, and if so, triggering a resource recycling process to recycle the service GPU resources of the idle service namespace into the public namespace as a public resource pool comprises: Compare the resource monitoring data with the GPU resource quota information of the business namespace to obtain the GPU resource utilization rate and GPU resource idle rate of the business namespace; When the GPU resource usage rate is less than a preset usage rate threshold, or the GPU resource idle rate is higher than a preset idle rate threshold, the corresponding service namespace is set as an idle service namespace; Obtaining priority information of tasks running in the idle service namespace, and sorting the tasks from low to high according to the priority information to obtain a task sorting result; Traversing the task sorting results starting from the task with the lowest priority, and determining whether scheduling the service GPU resources occupied by the currently traversed task to the public namespace will exceed the upper limit of the public GPU resource quota of the public namespace; If the upper limit is not exceeded, the business GPU resources occupied by the currently traversed task are scheduled to the public namespace, and the GPU resource usage data of the corresponding business namespace is updated; The resource recycling process is completed until all the tasks of the idle business namespace are checked or the GPU resources of the public namespace have reached the quota upper limit.

5. The cluster GPU utilization management method according to claim 1, characterized in that: The step of judging whether an allocation condition is met according to the application information in the GPU resource application request, the current resource usage data and the preset resource allocation priority rule, and if the allocation condition is met, allocating the GPU resources in the public namespace to the applied business namespace comprises: Obtaining current GPU resource usage data of the target service namespace and application information in the GPU resource application request, wherein the application information includes the number of GPUs currently applied for, usage duration, and priority parameters; Calculate the priority score of the target service namespace according to the preset resource allocation priority rule and the current GPU resource usage data; Determine whether the number of GPUs currently applied for exceeds the maximum GPU quota of the target business namespace, and if so, reject the application and return a prompt message; Determine whether the number of available GPUs in the public namespace meets the application requirements of the target service namespace; When the application requirements of the target business namespace are not met, GPU resources are scheduled from idle resources of other business namespaces to the public namespace according to the priority scores from low to high; When the GPU resources in the public namespace still cannot meet the application requirements, the application is rejected and a prompt message indicating insufficient resources is returned; When the number of available GPUs in the public namespace meets the application requirements, a corresponding number of GPU resources are allocated from the public namespace to the target service namespace.

6. The method for managing cluster GPU utilization according to claim 1, characterized in that: After the step of allocating the GPU resources in the public namespace to the target service namespace, the method further includes: Regularly obtain resource usage, quota usage and service priority data of each of the service namespaces; Calculating the resource balance of each of the service namespaces according to the resource utilization rate, the quota utilization rate and the service priority data; Determining whether there is an imbalance in resource allocation according to the resource balance degree; If there is an imbalance in resource allocation, the number of GPU resources migrated between different business namespaces is calculated based on the resource balancing algorithm; Migrate the number of migrated GPU resources to the corresponding business namespace in an incremental migration manner; After the migration is completed, the resource usage rate and quota usage rate of each of the migrated business namespaces are obtained as updated resource data; According to the updated resource data, it is determined whether the resources of each business namespace have reached a balanced state. If not, the migration is continued until the resources are balanced.

7. The method for managing cluster GPU utilization according to claim 1, characterized in that: After the step of allocating the GPU resources in the public namespace to the target service namespace, the method further includes: According to a preset idle resource monitoring cycle, regularly collect GPU resource usage monitoring indicator data of each of the business namespaces; Compare the GPU resource usage monitoring indicator data with a preset resource recycling trigger condition to determine whether the resource recycling trigger condition is met; When the resource recycling triggering condition is met, the GPU resource dynamic expansion and contraction mechanism is started to temporarily recycle the idle GPU resources to the public namespace; While reclaiming the GPU resources, the resource management policy of the public namespace is updated, and the resource allocation policy is adjusted according to the resource demand priority of the business namespace.

8. A cluster GPU utilization management device, characterized in that: include: A partitioning module, used to determine a namespace partitioning method according to a preset partitioning condition, and partition a cluster into namespaces based on the namespace partitioning method, wherein the namespaces include a plurality of business namespaces and a public namespace; A setting module, used to set a business GPU resource quota for each business namespace, and obtain a public GPU resource quota for the public namespace according to the business GPU resource quota; A resource allocation module, configured to allocate GPU resources to each of the service namespaces according to the service GPU resource quota and the public GPU resource quota to obtain service GPU resources; A collection module, used for collecting resource monitoring data of service GPU resources of each service namespace in real time; A resource recovery module, used to determine whether there is an idle service namespace according to the resource monitoring data and a preset scheduling policy, and if so, trigger a resource recovery process to recover the service GPU resources of the idle service namespace into the public namespace; The resource scheduling module is used to receive a GPU resource application request of a target business namespace, and determine whether an allocation condition is met based on the application information in the GPU resource application request, current GPU resource usage data, and a preset resource allocation priority rule; if the allocation condition is met, the GPU resources in the public namespace are allocated to the target business namespace.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the cluster GPU utilization management method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the cluster GPU utilization management method according to any one of claims 1 to 7 are implemented.

Images