Distribution method and system for improving GPU (Graphics Processing Unit) resource utilization rate, medium and program product

Abstract

The invention provides an allocation method and system for improving the GPU resource utilization rate, a medium and a program product, and relates to the technical field of data processing. The method comprises the following steps of: constructing a hybrid resource pool which comprises a divided resource pool divided according to set computing power and video memory granularity and a whole spare resource pool retaining a complete GPU resource unit; then, a to-be-processed task is obtained from an interface end, and the computing power scale and continuity demand data of the to-be-processed task are determined through a task analysis model obtained based on multi-factor machine learning training; finally, according to the data, computing power resources are distributed from the hybrid resource pool according to a preset method, so that diversified task requirements are met, efficient and flexible distribution and utilization of GPU resources are achieved, and the overall efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an allocation method, system, medium and program product for improving GPU resource utilization. Background Art

[0002] With the rapid development of modern science and technology, the demand for computing power in fields such as artificial intelligence, big data analysis, and graphics and image processing has exploded. GPUs (Graphics Processing Units) have been widely used in many computing-intensive tasks due to their powerful parallel computing capabilities. In large-scale computing environments such as cloud computing and data centers, it is often necessary to process a large number of tasks from different users or different applications at the same time. These tasks vary significantly in computing power requirements and resource continuity requirements. How to effectively allocate GPU resources and improve resource utilization in such complex and diverse task scenarios has become an important issue that needs to be urgently addressed in the current field of computing technology.

[0003] The traditional GPU resource allocation method mainly utilizes resource pool technology. A relatively fixed granularity division method is usually adopted in GPU resource allocation. This fixed granularity division method realizes the preliminary sharing and allocation management of resources to a certain extent, and can meet the basic needs of some routine tasks.

[0004] However, this traditional resource pool allocation technology has obvious limitations. Due to the granularity limitation of traditional resource pool technology, when facing large tasks that require a large amount of continuous GPU resources, fixed small-granularity resource division is difficult to meet their needs, seriously affecting the execution speed and computing performance of the task. Summary of the invention

[0005] The present application provides an allocation method, system, medium and program product for improving GPU resource utilization, which are used to improve resource utilization efficiency when facing large tasks that require a large amount of continuous GPU resources.

[0006] In the first aspect, the present application provides an allocation method for improving GPU resource utilization, which is applied to a resource allocation system, the method comprising: constructing a hybrid resource pool including a divided resource pool and a whole standby resource pool, the divided resource pool being a resource pool in which GPU resources are divided according to set computing power granularity and video memory granularity, and the whole standby resource pool being a resource pool that retains a preset number of complete GPU resource units; obtaining a task to be processed from an interface end, and determining computing power scale data and continuity requirement data required for the task to be processed through a task analysis model, the task analysis model being obtained by machine learning training based on input data characteristics, algorithm complexity, data processing flow and historical task execution records of multiple tasks in advance; and allocating corresponding computing power resources from the divided resource pool and the whole standby resource pool according to a preset method based on the computing power scale data and the continuity requirement data.

[0007] By adopting the above technical solution, a hybrid resource pool can be built to flexibly allocate resources according to task requirements. The divided resource pool meets routine tasks with small granularity, and the entire spare resource pool is used to cope with tasks with large computing power and continuous requirements. The task analysis model accurately evaluates the task computing power and continuity requirements to make the allocation more precise. The two work together to give full play to the parallel computing capabilities of the GPU, improve resource utilization, and avoid idle or wasted resources.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the step of allocating corresponding computing power resources from the divided resource pool and the entire standby resource pool according to a preset method based on the computing power scale data and the continuity requirement data specifically includes: if the computing power scale required by the task to be processed is less than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, then the task to be processed is allocated to the divided resource pool for processing; if the computing power scale required by the task to be processed is greater than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, then the task to be processed corresponding to the part of the task to be processed whose computing power scale exceeds the set computing power threshold is allocated to the entire standby resource pool for processing.

[0009] By adopting the above technical solution, tasks with low computing power and low continuous demand are allocated to the divided resource pool, which can efficiently utilize scattered resources. Tasks with high computing power and low continuous demand are partially transferred to the entire backup resource pool to ensure continuous calculation of key parts and avoid resource fragmentation affecting efficiency. This pool processing method can better adapt to different types of tasks, reduce task queuing waiting time, speed up task processing, and improve the overall system throughput.

[0010] In combination with some embodiments of the first aspect, in some embodiments, if the computing power scale required by the task to be processed is greater than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, then after the step of allocating the task to be processed corresponding to the part of the task to be processed whose computing power scale exceeds the set computing power threshold to the entire spare resource pool for processing, it also includes: if the continuity requirement data of the task to be processed is greater than the set continuity degree threshold, then the task to be processed is first allocated to the partitioned resource pool for preliminary calculation work of the task to be processed; after searching for a complete GPU resource unit matching the task to be processed in the entire spare resource pool, suspending the task processing work executed in the partitioned resource pool; migrating the computing status information data of the task to the complete GPU resource unit and resuming task execution.

[0011] By adopting the above technical solution, for tasks with high continuity requirements, preliminary calculations are first performed in the divided resource pool, and the flexible resources are used to quickly start the task, reducing the overall waiting time. After finding a matching complete GPU resource unit, the computing status information data is migrated to ensure seamless task execution. This not only fully utilizes the rapid response advantage of the divided resource pool, but also meets the high requirements of the task for continuous resources, improving the efficiency of task execution.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of migrating the computing status information data of the task to the complete GPU resource unit and resuming the task execution, it also includes: real-time statistics on the resource usage of the divided resource pool and the entire backup resource pool; when the utilization rate of the divided resource pool is lower than a first preset threshold for a continuous set period of time, merging a preset part of the divided resources and transferring them to the entire backup resource pool; when the utilization rate of the entire backup resource pool is lower than a second preset threshold for a continuous set period of time, dividing part of the complete GPU resource unit according to the set computing power granularity and transferring them to the divided resource pool.

[0013] By adopting the above technical solution, when the utilization rate of the divided resource pool is low, some resources are merged and transferred to the whole standby resource pool to optimize resource allocation. When the utilization rate of the whole standby resource pool is low, resources are split to the divided resource pool to improve resource flexibility. Through dynamic adjustment, resource allocation is always kept in a better state, adapting to task changes, further improving resource utilization and reducing system operating costs.

[0014] In combination with some embodiments of the first aspect, in some embodiments, after the step of migrating the computing status information data of the task to the complete GPU resource unit and resuming task execution, it also includes: obtaining the priority information of the task to be processed; if the mixed resource pool cannot meet the requirements of the high-priority task to be processed, pausing the low-priority task being processed in the divided resource pool; and reclaiming the resources occupied by the low-priority task and allocating them to the task to be processed corresponding to the high priority.

[0015] By adopting the above technical solution, task priority information is obtained. When high-priority tasks are short of resources, low-priority tasks are suspended to recycle resources, ensuring that important tasks are processed first and avoiding delays in critical tasks due to resource competition. This priority-based resource scheduling strategy can better meet the service quality requirements of different tasks and improve the system's responsiveness to urgent and important tasks.

[0016] In combination with some embodiments of the first aspect, in some embodiments, before the step of migrating the computing status information data of the task to the complete GPU resource unit and resuming task execution, it also includes: saving the processing status information data of the task to be processed in a unified format according to the data structure characteristics of the task to be processed; and determining the integrity of the data through a set encoding mechanism.

[0017] By adopting the above technical solution, the processing status information data is uniformly saved according to the task data structure, which is convenient for system identification and management. The coding mechanism is set to ensure data integrity and prevent data loss or damage. When the task is migrated or the resources are adjusted, the processing status information is accurately transmitted and restored to maintain the continuity of task execution.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of allocating corresponding computing power resources from the divided resource pool and the entire standby resource pool according to a preset method based on the computing power scale data and the continuity demand data, it also includes: real-time collection of multi-dimensional performance indicators of GPU resources; inputting the multi-dimensional performance indicators into a resource health assessment model to obtain the health of GPU resources, wherein the resource health assessment model is obtained by machine learning training based on multiple historical performance data and corresponding resource health status labels; when it is detected that the health of the GPU resource unit is lower than a preset health threshold, the task load carried by the target GPU resource unit is automatically reduced.

[0019] By adopting the above technical solutions, the task volume is dynamically adjusted according to the health level to avoid unhealthy resource units affecting the task execution effect, and timely intervention is made before potential problems occur in resources to prevent task failure or calculation errors. The accuracy and stability of task calculations are guaranteed, the service life of GPUs is extended, and system downtime caused by hardware failures is reduced.

[0020] In a second aspect, the present application provides a resource allocation system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the resource allocation system to perform the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a resource allocation system, causes the resource allocation system to execute the method described in the first aspect and any possible implementation manner of the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a resource allocation system, the resource allocation system executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By adopting the technical means of building a hybrid resource pool and applying a task analysis model, the technical problem of the traditional GPU resource allocation method with fixed granularity and difficulty in meeting the needs of diversified tasks is effectively solved, thereby achieving the technical effect of accurately allocating GPU resources and improving resource utilization.

[0024] 2. Due to the adoption of technical means to process tasks in pools based on computing power scale and continuity demand thresholds, the technical problem of inefficiency caused by the inability of traditional resource allocation to flexibly adapt to different types of tasks has been effectively solved, thereby achieving the technical effect of reducing task waiting time, improving system throughput, and accelerating the task processing process.

[0025] 3. Due to the adoption of technical means of suspending low-priority tasks based on task priority to reclaim resources, the technical problem of delays in key tasks caused by resource competition is effectively solved, thereby achieving the technical effect of ensuring priority processing of important tasks, meeting the service quality requirements of different tasks, and improving system responsiveness and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of an allocation method for improving GPU resource utilization in an embodiment of the present application; Figure 2 is another flow chart of the allocation method for improving GPU resource utilization in an embodiment of the present application; Figure 3It is a schematic diagram of a physical device structure of a resource allocation system in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, the following is a description of the process of the method provided by this implementation. Figure 1 , which is a flow chart of an allocation method for improving GPU resource utilization in an embodiment of the present application.

[0030] S101, constructing a hybrid resource pool including a divided resource pool and a whole standby resource pool, wherein the divided resource pool is a resource pool in which GPU resources are divided according to a set computing power granularity and a video memory granularity, and the whole standby resource pool is a resource pool in which a preset number of complete GPU resource units are reserved; The resource allocation system first conducts a comprehensive evaluation of GPU resources, including hardware parameters such as GPU model, number of computing cores, video memory capacity, and bandwidth. Based on these parameters, combined with the requirements of common tasks in actual application scenarios, reasonable computing power granularity and video memory granularity are set. Specifically, deep learning training tasks can be used, and the computing power granularity can be determined based on the computational complexity of the neural network layer. For the convolution layer, due to its large amount of calculation and certain regularity, it can be divided as a relatively independent computing power unit, and the appropriate computing power granularity is determined based on factors such as the convolution kernel size and the number of input and output channels. For the fully connected layer, since its calculation method is different from that of the convolution layer, the corresponding computing power granularity can be set according to the number of neurons and the number of connection weights. In terms of video memory granularity, considering the storage requirements of deep learning model parameters, intermediate calculation results, and training data, it is divided according to the scale of the model and the batch size of the data. For example, for a large image classification model, the model parameters may need to occupy a large amount of video memory space. The video memory can be divided into larger granularity to store the model parameters. For the intermediate calculation results in the training process, such as gradient information, a smaller video memory granularity can be set according to the data size and update frequency.

[0031] The construction of the divided resource pool is mainly realized through the system's virtualization technology or a dedicated resource management driver. The resource allocation system logically divides the GPU's computing resources and video memory resources according to the set granularity. For example, the GPU's computing core is divided into several small computing units, each of which represents a computing power granularity; the video memory is divided into memory blocks of corresponding sizes, each of which corresponds to a video memory granularity. These divided resource units can be independently allocated and managed to form a divided resource pool to meet the diverse resource requirements of different tasks. The system determines the preset number of complete GPU resource units to be reserved as a whole standby resource pool based on the overall GPU resource scale, the frequency of occurrence of large continuous tasks in historical tasks, and the system's load balancing requirements. For example, in a cluster with 100 GPUs, after data analysis, it is found that on average, one large task that requires a large amount of continuous resources will appear in every 10 tasks, and the system requires a quick response to such tasks during peak loads. Based on this, the system may reserve 10-20 complete GPU resource units as a whole standby resource pool. The management of the entire backup resource pool is relatively independent, ensuring that the GPU resource units therein are always available and not occupied by scattered tasks.

[0032] In some embodiments, the standby reserved ratio of the entire resource pool can be determined when constructing a hybrid resource pool to meet the diverse task requirements and achieve efficient and flexible allocation and utilization of GPU resources, mainly based on the analysis of historical task data and the prediction of future task volume. The total amount of tasks in each time unit within the period forms a sequence of total amount of tasks , and then calculate the average value of the total task sequence ,

[0033] Calculate the standard deviation of the total task series ,

[0034] The standard deviation can reflect the fluctuation of the total amount of tasks around the average value, and help evaluate the stability and degree of change of the task volume. , the task volume is , fitting a straight line based on historical data ,in

[0035] in For the The serial number of the time unit, It means setting time The b is the average value of the time unit serial number, and the b refers to the ordinate value of the intersection of the historical data fitting straight line and the y-axis.

[0036]

[0037] Determine the future The amount of prediction tasks per time unit , , calculate the average proportion of the preset scale tasks to the total tasks in the set time period based on historical task data ; Based on the predicted task volume , calculate the future The amount of tasks in a time unit :

[0038] Get the average number of complete GPU resources corresponding to historical large-scale tasks , the total number of all available full GPU resources is 、The divided resource pool can undertake the equivalent number of complete GPU resources corresponding to the preliminary work of large-scale tasks ; then the standby reservation ratio of the entire resource pool is :

[0039] By analyzing historical task data, we can more accurately predict the demand for complete GPU resources for future tasks, so as to reasonably reserve a proportion of the entire resource pool and avoid wasting resources by over-reserving the entire resource pool. The calculation of this proportion comprehensively considers factors such as the demand for complete GPU resources in future large-scale tasks, the ability of the divided resource pool to undertake the preliminary work of large-scale tasks, and the overall number of complete GPU resources available, so as to determine the proportion of the entire resource pool that needs to be reserved to cope with large-scale tasks, so as to ensure that there are enough continuous GPU resources available when there are large-scale tasks.

[0040] S102, obtaining a task to be processed from the interface end, and determining the computing power scale data and continuity demand data required for the task to be processed through a task analysis model, wherein the task analysis model is obtained by machine learning training in advance based on input data characteristics, algorithm complexity, data processing flow and historical task execution records of multiple tasks; The resource allocation system interacts with the external environment through a variety of interfaces to obtain tasks to be processed. These interfaces include but are not limited to network interfaces, storage interfaces, and specific task submission interfaces. The system performs format parsing and integrity checks on the input data of the task. At the same time, the system obtains basic information about the task, such as the task type, task source, and the urgency of the task (if identified). According to the task type and the characteristics of the input data, the system performs preliminary data preprocessing operations. For large-scale data tasks, data block processing may be performed to divide the data into small blocks suitable for system processing to improve data processing efficiency. For data that requires specific format conversion, such as converting image data from RGB format to grayscale format, the system performs corresponding conversion operations at this time to ensure the consistency and availability of the data in subsequent analysis and processing.

[0041] The training data of the task analysis model comes from the detailed records of a large number of historical tasks, including the input data characteristics of the task (such as the distribution form of the data, the numerical range, the feature dimension, etc.), the algorithm complexity (determined by analyzing the algorithm type, the number of calculation steps, the loop nesting depth, etc. adopted by the task), the data processing flow (extracting features from the operation sequence of the task execution, the data flow, the intermediate result generation method, etc.), and the actual resource usage and execution time during the task execution process. When receiving the task to be processed, the model takes the input data characteristics, algorithm complexity, data processing flow and other information of the task as input, and uses the pre-trained machine learning algorithm for analysis and prediction. For deep learning tasks, the model analyzes the structure of the neural network model (such as the number of layers, the number of neurons in each layer, the type of activation function, etc.), combines the characteristics of the input data (such as image size, the number of samples in the data set, etc.), and predicts the computing power scale required for the task through complex calculations and model reasoning. At the same time, according to the data dependency and operation sequence in the task processing flow, the degree of resource continuity demand of the task is judged. During the task execution process, the system introduces a real-time feedback mechanism to feed back the actual resource usage and execution performance data of the task to the task analysis model in real time. The model dynamically adjusts and optimizes itself based on this feedback information to adapt to the ever-changing task mode and environment. For example, if it is found that the actual computing power requirement of a task is higher than the model prediction value, the model will analyze the reason. It may be due to the particularity of the input data that leads to an increase in algorithm complexity, or unexpected computing overhead during the task execution. Based on these analysis results, the model automatically adjusts the algorithm parameters or introduces new feature variables to retrain itself.

[0042] S103. Allocate corresponding computing resources from the divided resource pool and the entire standby resource pool according to the computing power scale data and the continuity demand data in accordance with a preset method.

[0043] The system pre-sets the computing power threshold and continuity requirement threshold. These two thresholds are empirical values ​​determined through statistical analysis and performance testing of a large amount of historical task data. When the computing power scale of the task to be processed is less than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, the system determines that the task is a low computing power and low continuity requirement task. At this time, the resource allocation system allocates the task to the divided resource pool for processing.

[0044] The divided resource pool is preset with a resource allocation table, which is used to record the status of each resource unit (idle, occupied), the computing power granularity and video memory granularity, the currently assigned tasks and other information. When assigning tasks, the system searches for idle resource units that meet the conditions in the resource allocation table according to the computing power and video memory requirements of the task. For example, if the task requires a small computing power (such as less than 100 GFLOPS) and limited video memory (such as less than 1GB), the system will find the corresponding idle resource units with small computing power granularity and small video memory granularity in the divided resource pool, and assign the task to these units for execution. If the computing power scale of the task to be processed is greater than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, the system will assign the subtasks corresponding to the part of the task whose computing power scale exceeds the set computing power threshold to the entire standby resource pool for processing. For the remaining tasks, if their computing power requirements are within the capacity of the divided resource pool, they will still be assigned to the divided resource pool for execution. In this way, the large computing power advantage of the entire standby resource pool can be fully utilized to process key tasks, while the flexibility of the divided resource pool can be used to process other parts, thereby improving overall efficiency.

[0045] In the resource allocation process, the system adopts a strategy that combines first-in-first-out and priority scheduling. For tasks of the same priority, resources are allocated in order of arrival; for high-priority tasks, the system gives priority to allocating resources to them, even if it needs to pause or preempt the resources of low-priority tasks being executed. The resource allocation system assigns a priority identifier to each task. The priority can be determined based on factors such as the urgency of the task, the importance of the task source (such as tasks submitted by critical business applications have a higher priority), or the priority specified by the user. At the same time, the resource allocation system monitors the tasks to which resources have been allocated in real time, and records information such as the execution progress of the tasks, resource usage (such as the computing power currently used, the amount of video memory occupied, the data transmission rate, etc.). If it is found that the resource requirements of a task have changed (such as an increase or decrease in computing power requirements due to intermediate results in the data processing process), the system dynamically adjusts the resources allocated to the task.

[0046] In some embodiments, in order to fully and accurately grasp the operating status of GPU resources, the resource allocation system uses special monitoring tools and technologies to collect multi-dimensional performance indicators of GPU resources in real time. These performance indicators cover multiple key aspects, such as the utilization rate of computing cores, which can reflect the current busyness of GPU computing resources; the occupancy rate of video memory, which is crucial to understanding the utilization of video memory resources; data transmission bandwidth, which reflects the speed and efficiency of data transmission within the GPU and between external devices; and hardware-related indicators such as temperature and power consumption. Excessive temperature may affect the stability and life of the GPU, and power consumption is related to energy consumption. By collecting these multi-dimensional performance indicators, the system can obtain detailed information about the GPU resources during operation. The resource health assessment model is built based on machine learning technology. Its training data comes from the collection of multiple historical performance data of GPU resources in different operating scenarios in the past. At the same time, each historical performance data corresponds to a corresponding resource health status label. These historical performance data cover the performance of GPU resources under various task loads, different environmental conditions, etc. During the training process, the machine learning algorithm will deeply analyze the intrinsic relationship between these data and mine the characteristic patterns that can characterize the health status of GPU resources. For example, when the computing core is under high load for a long time and the temperature continues to rise, it may indicate that the resource is under pressure and the health is declining; or when the memory occupancy rate fluctuates frequently and the data transmission bandwidth is unstable, it may also be related to potential resource problems. Through the learning of a large amount of historical data, the model gradually has the ability to accurately evaluate the health of GPU resources based on multi-dimensional performance indicators collected in real time. When new performance indicator data is input into the model, the model can quickly calculate and output the health value of the current GPU resource based on the learned patterns and rules. This value can intuitively reflect the overall health of the GPU resource. The system pre-sets a reasonable preset health threshold. This threshold is obtained through the analysis of a large amount of GPU operation data and actual experience summary. It is used as a standard for judging whether the GPU resource is in a healthy state. When the system detects that the health of a GPU resource unit is lower than this preset health threshold, it means that the GPU resource unit may have performance degradation, potential failure risk or is already in an unstable state. At this time, in order to ensure the accuracy and stability of task calculations and avoid task failures or calculation errors due to GPU resource problems, the system will automatically start the task volume adjustment mechanism. Specifically, the system will reduce the amount of tasks assigned to the target GPU resource unit, migrate some tasks to other healthy GPU resource units for processing (if there are available resources), or queue or reschedule tasks based on the overall resource situation.

[0047] In this way, the system can intervene in time before potential problems occur in GPU resources, effectively preventing unhealthy resource units from affecting task execution. It also helps to extend the service life of the GPU, reduce system downtime caused by hardware failures, and ensure the stable operation of the entire computing system.

[0048] In the embodiment of the present application, a hybrid resource pool including a divided resource pool and a whole standby resource pool is constructed, and a task analysis model based on multi-factor machine learning training is used to determine the task computing power scale and continuity demand data, and then resources are allocated according to the data pool. This series of technical means effectively solves many drawbacks of traditional GPU resource allocation methods, such as fixed granularity, inability to accurately assess task requirements, and inflexible resource allocation, and realizes more flexible and adaptable GPU resource allocation to cope with diverse task scenarios and meet the requirements of different tasks for computing power and continuity.

[0049] After combining the above content, the following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the allocation method for improving GPU resource utilization in an embodiment of the present application.

[0050] S201: If the computing power scale required by the task to be processed is less than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, the task to be processed is allocated to the divided resource pool for processing; The resource allocation system first needs to conduct a detailed evaluation and analysis of the tasks to be processed. The computing power scale data obtained by the system through the task analysis model will be converted into standardized computing units. At the same time, the system will quantify the continuity demand data into specific numerical indicators, such as the task's requirements for continuous resource usage time, the strength of dependencies in the data processing process, etc. The system will compare these data with the pre-set computing power threshold and continuity threshold. The setting of these thresholds is based on a large amount of historical data analysis and performance testing. For example, the computing power threshold may be set to 30% of the computing power of a single GPU, and the continuity threshold may be set to 1 hour of continuous computing time requirements.

[0051] When the system confirms that the task belongs to the type of low computing power and low continuity requirements, it will start a refined resource allocation process. First, the system will establish a real-time resource status diagram in the divided resource pool. The status diagram contains information such as the current load of each resource pool, the remaining computing power capacity, and the size of available video memory. The system uses a multi-dimensional matching algorithm to match the specific requirements of the task with the characteristics of the resource unit. For example, if the task requires 2TFLOPS of computing power and 4GB of video memory, the system will search for the minimum set of resource units that meet these conditions. During the resource allocation process, the system will take load balancing factors into consideration. By monitoring the usage of each resource unit in real time, the system will give priority to resource units with lower loads to avoid overloading some units while other units are idle.

[0052] S202: If the computing power scale required by the pending task is greater than the set computing power threshold and the continuity demand data is less than the set continuity degree threshold, the pending task corresponding to the portion of the pending task whose computing power scale exceeds the set computing power threshold is allocated to the entire standby resource pool for processing; When the computing power scale of the task to be processed is greater than the set computing power threshold, it means that the task requires a large amount of GPU computing power resources, and the continuity demand data is less than the threshold, indicating that the requirement for resource continuity is not high. In order to make rational use of resources, the system can adopt the following technical means for processing: 1) The system first retrieves the detailed information of the task to be processed, locates the specific task module or component whose computing power scale exceeds the computing power threshold, for example, determines that a certain network layer of the task needs to occupy a large amount of computing power resources; 2) Then, the system calls the task splitting component in the task analysis model to split the located super-threshold task module into two subtasks: one subtask contains the computing work of the super-threshold part, and the other subtask contains the remaining computing work; 3) Then, the system determines whether the GPU resource unit in the entire backup resource pool meets the computing power requirements of the super-threshold subtask; if it does, the subtask is directly submitted to the target GPU resource unit for execution; if it does not, the super-threshold subtask continues to be split until it can be submitted to a GPU resource unit in the entire backup resource pool for execution; 4) Finally, the system submits the super-threshold subtask to the entire backup resource pool for processing, and keeps the other subtask containing the remaining calculation in the divided resource pool. After the entire backup resource pool has processed the super-threshold subtask, the calculation results of the two subtasks are merged to complete the final processing of the task to be processed.

[0053] This technical solution of splitting the super-threshold task module and submitting it to the entire backup resource pool can effectively separate the part with high computing power demand from the part with low computing power demand, rationally utilize the advantages of different resource pools, avoid large tasks occupying the divided resource pool while affecting the processing of small tasks, thereby improving system resource utilization.

[0054] S203: If the continuity requirement data of the task to be processed is greater than the set continuity degree threshold, the task to be processed is first allocated to the divided resource pool to perform preliminary calculation work on the task to be processed; When it is detected that the continuity demand data of the task to be processed is greater than the preset threshold, it means that the task has high requirements for the continuity and continuous occupation of GPU computing resources. In order to give full play to the resource scheduling flexibility of the divided resource pool, the system adopts the following technical means for processing: 1) The system first evaluates and determines the initial resource requirements of the task in the divided resource pool based on the data input volume, algorithm complexity and other information of the task to be processed, such as how many GPU computing units are required; 2) Then, the system calls the resource scheduling module to find and lock the GPU computing units that meet the initial resource requirements from the divided resource pool; 3) Then, the system submits the task to be processed to the locked GPU computing unit to start the initial calculation work, such as data preprocessing, feature extraction and other initialization stage processing flow; 4) At the same time, the system enables the monitoring component of the task to monitor the task execution status and resource occupancy in real time, so as to prepare for the subsequent resource expansion or computing migration in time.

[0055] By quickly responding to and starting initial calculations for continuous tasks through the divided resource pool, the waiting and preparation time for tasks can be shortened, and the subsequent restart of calculations after the entire block of resources is ready can be accelerated, thereby improving the response speed and efficiency of the system in processing continuous tasks.

[0056] S204, after searching the entire spare resource pool for a complete GPU resource unit that matches the task to be processed, suspending the task processing work executed in the divided resource pool; When the initial calculation of the continuous task is started, the system adopts the following technical means to continue to allocate the entire GPU resource to it: 1) The system continuously detects the availability and operating status of all GPU resource units in the entire backup resource pool to determine the idle GPU resources that can currently meet the task computing power and continuity requirements; 2) Then, the system compares the detailed computing power requirement parameters of the task to be processed with the computing power range, video memory size and other information of each GPU resource in the entire backup resource pool, and searches for one or more GPU resource units that fully match the task to be processed; 3) After the match is successful, the system immediately locks the target GPU resource and sends a pause instruction to the relevant scheduling module in the divided resource pool to suspend the processing of the GPU computing power unit currently occupied by the continuous task; 4) At the same time, notify the task monitoring component to prepare for subsequent computing state migration operations.

[0057] This search, match and pause processing technology solution can accurately and seamlessly find the entire GPU resource required for continuous tasks, coordinate with the processing work in the divided resource pool, and make system-level preparations for the migration of computing status and rapid restart.

[0058] S205, migrating the computing status information data of the task to the complete GPU resource unit and resuming task execution; After finding a complete GPU resource unit that matches the task to be processed, the resource allocation system needs to suspend the task and migrate its computing state information data to the new GPU resource unit. The specific technical means can be adopted as follows: Save the computing state information of the task: The system calls the task processing interface, and according to the task type and data structure characteristics, takes a snapshot of the current computing state, parameter information, intermediate results and other processing data of the task, and serializes and encodes them in a unified data format to generate a computing state information data packet, which also contains the priority metadata of the task, resource usage status and other information. Then select a suitable state migration scheme: Considering the requirements of different tasks for the amount of state data and migration delay, the system provides a variety of computing state migration schemes to choose from, such as fast migration based on memory sharing, reliable migration based on network transmission, etc. For tasks with a small amount of computing state data and sensitive to delay, the state information can be written into shared memory for fast migration; for tasks with a large amount of data, a network-based transmission scheme is used to ensure reliability. Suspend the task on the original GPU resource: The system sends a suspend instruction to the task handler on the original GPU resource to suspend the computing work of the task. At the same time, the task data reading interface is notified to stop providing new data, write the intermediate state data into the cache, and then transfer the calculation state data to the new GPU resource. The system sends the encoded calculation state information data to the allocated complete GPU resource unit through the selected state migration scheme, and loads it into the resource unit memory. Resume task execution: After receiving the calculation state information, the task manager on the new GPU resource decodes and deserializes it to restore the execution context of the task. The task processing interface restarts the task processing flow, continues to process subsequent data from the received intermediate state, and the task calculation is seamlessly restored.

[0059] S206, collecting statistics on resource usage of the divided resource pool and the entire standby resource pool in real time; To achieve a dynamic balance between resource pools, the resource allocation system needs to monitor the usage of the two resource pools in real time. Specific technical means may include: statistics on the usage of the divided resource pools. The system continuously detects the occupancy status of each divided GPU resource unit, counts the number or proportion of different types of resource units occupied, and calculates the overall resource usage of the divided resource pool. Statistics on the usage of the entire standby resource pool: monitor the changes in the number of complete GPU resource units in the current entire standby resource pool, record the number of newly allocated and released resource units, maintain the real-time resource occupancy table of the resource pool, and count the usage of the entire standby resource pool. Set the usage statistics time window: Considering that resource usage may fluctuate instantaneously, the average usage over a certain time span (for example, 5 minutes) should be used as a statistical indicator to avoid too frequent adjustments to the resource pool. Persistently record resource pool usage: The system periodically (for example, every minute) writes the usage statistics of the two resource pools into a log or database for persistent storage. After the system is restarted, historical usage information can continue to be obtained.

[0060] S207, when the usage rate of the divided resource pool is lower than the first preset threshold value for a continuous set period of time, a preset part of the divided resources is merged and transferred to the whole standby resource pool; Set the low utilization threshold of the divided resource pool: Analyze the utilization fluctuation range of the divided resource pool and the system resource utilization target based on historical resource usage data, and determine a reasonable low utilization threshold, such as 30%. Re-detect the duration of the utilization rate below the threshold: The system continuously detects the utilization rate of the divided resource pool. If the utilization rate is lower than the low threshold of 30% for a certain period of time (such as 20 minutes), it is considered that the resource utilization rate is low and the resource pool needs to be adjusted. Determine the number or proportion of divided resources to be merged: The system selects idle resources for merging according to a certain strategy based on the real-time usage status of various divided resource units. For example, select the resource type with the lowest utilization rate and merge 20% of its resources. Perform the merging of divided resources: The system calls the resource management interface to unbind the selected divided resources, release the occupied tasks, and reorganize them into larger-granularity resource units according to a certain scale. Transfer to the whole-block standby resource pool: The large-granularity divided resources after reorganization and merging are marked as whole-block GPU resources by the system, and are transferred to the unified whole-block standby resource pool by the resource management module to prepare for subsequent tasks that require a large amount of continuous GPU computing power.

[0061] S208: When the usage rate of the entire standby resource pool is lower than a second preset threshold value for a continuous set period of time, part of the complete GPU resource units are divided according to the set computing power granularity and transferred to the divided resource pool.

[0062] The resource allocation system needs to monitor the resource usage of the entire standby resource pool in real time, count the number of running tasks, resource occupancy and other information, and calculate the utilization rate. Through the utilization rate data obtained by continuous monitoring, the resource allocation system can determine that within the set time window, if the average utilization rate of the entire standby resource pool continues to be lower than the preset lower threshold, for example, lower than 15%, it can be determined that the resource utilization rate is too low and it is necessary to reallocate and adjust the resources in the resource pool. According to the total amount of the resource pool and the used amount data, the resource allocation system can calculate the number of complete GPU resource units that need to be divided this time according to a certain ratio, for example, 10% of the total amount of the complete resource pool during the period when the utilization rate is lower than the threshold can be divided. The calculation of the number needs to take into account the expected growth trend of task demand. The resource allocation system selects complete GPU resource units with a longer idle time from the entire standby resource pool for division to avoid frequent interruption of the tasks being processed. The resource allocation system calls the GPU virtualization program to divide the selected complete GPU resource units according to the predetermined computing power granularity scheme. The divided GPU resource units are marked as divided resources by the resource allocation system, and the metadata information of the original complete GPU is recorded, including the unbinding relationship, operating status, etc., to facilitate subsequent resource scheduling and management. The resource allocation system removes the divided and marked GPU resource units from the entire spare resource pool and adds them to the divided resource pool, and uniformly schedules and manages them with the original divided resources to meet the needs of various small-grained concurrent tasks. The resource allocation system can optimize the computing power granularity scheme of divided resources by continuously analyzing task requirements and resource usage, and realize dynamic adjustment of resource configuration, so that it can better adapt to changes in task requirements and improve resource utilization efficiency.

[0063] In the embodiment of the present application, a hybrid resource pool is constructed, including a divided resource pool divided according to the set computing power and video memory granularity and a whole spare resource pool that retains the complete GPU resource unit. A task analysis model based on multi-factor machine learning training is used to determine the task computing power scale and continuity demand data, and resources are allocated according to the data according to a preset method. At the same time, real-time statistics of resource usage are collected, and resource configuration is dynamically adjusted according to the utilization rate of different resource pools. Operations such as task calculation state migration are also involved. The problems of fixed granularity and low resource utilization in traditional GPU resource allocation methods are effectively solved, and the effects of accurate allocation of GPU resources, improved resource utilization, enhanced system task processing capabilities and overall efficiency are achieved. Diversified task requirements are met and the flexibility and adaptability of GPU resource management are improved.

[0064] The resource allocation system in the embodiment of the present invention is described below from the perspective of hardware processing. Figure 3 , which is a schematic diagram of a physical device structure of a resource allocation system in an embodiment of the present application.

[0065] It should be noted that Figure 3 The structure of the resource allocation system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0066] like Figure 3 As shown, the resource allocation system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. Various programs and data required for system operation are also stored in RAM 303. CPU 301, ROM 302 and RAM 303 are connected to each other through a bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0067] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0068] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0069] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0071] Specifically, the resource allocation system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the allocation method for improving GPU resource utilization provided by the above embodiment is implemented.

[0072] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the resource allocation system described in the above embodiment; or may exist independently without being assembled into the resource allocation system. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the resource allocation system, the resource allocation system implements the allocation method for improving GPU resource utilization provided in the above embodiment.

[0073] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0074] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0075] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A method for improving GPU resource utilization, applied to a resource allocation system, characterized in that: The method comprises: Constructing a hybrid resource pool including a divided resource pool and a whole standby resource pool, wherein the divided resource pool is a resource pool in which GPU resources are divided according to a set computing power granularity and a video memory granularity, and the whole standby resource pool is a resource pool in which a preset number of complete GPU resource units are reserved, wherein the set computing power granularity refers to the smallest unit for dividing computing resources, and the video memory granularity refers to the size of the video memory unit divided in combination with the GPU video memory capacity; Obtain the task to be processed from the interface end, and determine the computing power scale data and continuity requirement data required for the task to be processed through a task analysis model, wherein the task analysis model is obtained by machine learning training based on the input data characteristics, algorithm complexity, data processing flow and historical task execution records of multiple tasks in advance. The computing power scale data refers to the computing power required to complete the task to be processed, and the continuity requirement data refers to the degree of requirement for continuous occupation and use of GPU resources during the execution of the task; According to the computing power scale data and the continuity demand data, corresponding computing power resources are allocated from the divided resource pool and the entire spare resource pool in accordance with a preset method.

2. The method according to claim 1, characterized in that: The step of allocating corresponding computing resources from the divided resource pool and the entire standby resource pool according to the computing power scale data and the continuity demand data according to a preset method specifically includes: If the computing power scale required by the task to be processed is less than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, the task to be processed is allocated to the divided resource pool for processing; If the computing power scale required by the pending tasks is greater than the set computing power threshold and the continuity requirement data is less than the set continuity degree threshold, the pending tasks corresponding to the part of the pending tasks whose computing power scale exceeds the set computing power threshold will be allocated to the entire backup resource pool for processing.

3. The method according to claim 2, characterized in that If the computing power scale required by the pending task is greater than the set computing power threshold and the continuity demand data is less than the set continuity degree threshold, after the step of allocating the pending tasks corresponding to the portion of the pending tasks whose computing power scale exceeds the set computing power threshold to the entire standby resource pool for processing, the step further includes: If the continuity requirement data of the task to be processed is greater than the set continuity degree threshold, the task to be processed is first allocated to the divided resource pool to perform preliminary calculation work on the task to be processed; After searching the entire spare resource pool for a complete GPU resource unit matching the task to be processed, suspending the task processing work executed in the divided resource pool; The computing status information data of the task is migrated to the complete GPU resource unit and the task execution is resumed.

4. The method according to claim 3, characterized in that After the step of migrating the computing status information data of the task to the complete GPU resource unit and resuming the task execution, the method further includes: Real-time statistics of resource usage of the divided resource pool and the entire standby resource pool; When the usage rate of the divided resource pool is lower than the first preset threshold value for a continuous set period of time, a preset part of the divided resources are merged and transferred to the whole standby resource pool; When the usage rate of the entire standby resource pool is lower than a second preset threshold for a continuous set period of time, part of the complete GPU resource units are divided according to the set computing power granularity and transferred to the divided resource pool.

5. The method according to claim 3, characterized in that: After the step of migrating the computing status information data of the task to the complete GPU resource unit and resuming the task execution, the method further includes: Obtaining priority information of the task to be processed; If the mixed resource pool cannot meet the needs of the high-priority tasks to be processed, the low-priority tasks being processed in the divided resource pool are suspended; The resources occupied by the low-priority tasks are recovered and allocated to the pending tasks corresponding to the high-priority tasks.

6. The method according to claim 3, characterized in that Before the step of migrating the computing status information data of the task to the complete GPU resource unit and resuming the task execution, the method further includes: The processing status information data of the task to be processed is saved in a unified format according to the data structure characteristics of the task to be processed; The integrity of the data is ensured through the set encoding mechanism.

7. The method according to claim 1, characterized in that After the step of allocating corresponding computing resources from the divided resource pool and the entire standby resource pool according to the computing power scale data and the continuity demand data in a preset method, the method further includes: Collect multi-dimensional performance indicators of GPU resources in real time; Input the multi-dimensional performance indicators into a resource health evaluation model to obtain the health of GPU resources, wherein the resource health evaluation model is obtained by machine learning training based on multiple historical performance data and corresponding resource health status labels in advance; When it is detected that the health of a GPU resource unit is lower than a preset health threshold, the amount of tasks carried by the target GPU resource unit is automatically reduced.

8. A resource allocation system, characterized in that: The resource allocation system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the resource allocation system to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a resource allocation system, the resource allocation system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is executed on a resource allocation system, the resource allocation system is caused to perform the method according to any one of claims 1 to 7.

Images