Dynamic scheduling method for GPU resources in cloud environment

Abstract

The invention relates to the technical field of GPU resource dynamic scheduling, in particular to a cloud environment GPU resource dynamic scheduling method, which comprises the following steps: monitoring operation indexes of GPU instances in a cloud environment in real time, collecting GPU video memory allocation records, GPU video memory fragmentation degree magnitude and page migration activity count, and predicting task resource demand total quantity through statistics. And generating a video memory state evaluation result. According to the method, the resource pool adjustment decision and the task matching degree list are jointly optimized, the instance models and the number are screened under the double constraints of the instance starting time parameter and the release cost parameter, and operation delay and extra cost caused by blind capacity expansion and contraction are reduced. Based on a strong binding mechanism started by a video memory pre-allocation instruction and a task process, in combination with continuous tracking of a resource consumption trajectory, a video memory peak deviation and an execution time deviation are fed back in real time, and a closed-loop scheduling optimization link is formed.

Description

Technical Field

[0001] The present invention relates to the technical field of GPU resource dynamic scheduling, and in particular to a cloud environment GPU resource dynamic scheduling method. Background Art

[0002] The field of GPU resource dynamic scheduling technology focuses on the real-time allocation, optimization, and rebalancing of graphics processing unit (GPU) resources in a distributed computing environment, especially in cloud computing, edge computing, and high-performance computing scenarios, to solve problems such as low resource utilization and increased response delay caused by fluctuating task requirements, hardware heterogeneity, and multi-tenant competition.

[0003] In the existing technology, resource pool adjustment strategies are mostly based on a single indicator (such as the total memory gap) to trigger expansion and contraction, ignoring the dynamic coupling relationship between instance cold start time and release cost, which can easily cause operational oscillation. For example, after quickly expanding high-configuration instances to cope with short-term peak loads, redundant resources are forced to be held for a long time due to excessively high release costs, pushing up operating costs. Task scheduling usually allocates resources based on static priority or polling mechanisms, and does not include instance status change time in scheduling decisions, resulting in high-priority tasks being forced to wait due to delays in the start of the target instance. Therefore, improvements are needed. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and propose a cloud environment GPU resource dynamic scheduling method.

[0005] In order to achieve the above object, the present invention adopts the following technical solution, a cloud environment GPU resource dynamic scheduling method, comprising the following steps: Monitor the operating indicators of each GPU instance in the cloud environment in real time, collect GPU memory allocation records, GPU memory fragmentation values ​​and page migration activity counts, and generate memory status evaluation results by statistically predicting the total task resource requirements; Based on the memory status evaluation result, the predicted total resource demand is compared with the total capacity configuration of the current cloud environment GPU resource pool to obtain the resource capacity gap or redundancy. Based on the resource capacity gap or redundancy, the startup time parameters of the newly added GPU instances and the associated cost parameters of the released GPU instances are queried and evaluated to establish a GPU resource pool adjustment decision. Based on the memory status evaluation result, access the list of tasks to be scheduled, extract the GPU memory requirement of each task, generate a task and GPU matching list, select a target GPU instance for each task in the task queue to be processed based on the task and GPU matching list and combine the time information of the instance status change in the GPU resource pool adjustment decision, and establish a scheduling sequence for the tasks to be executed; Based on the scheduling sequence of tasks to be executed, the task allocation information and the corresponding GPU instance identifier are sent to the GPU resource manager of the cloud environment, and a GPU video memory pre-allocation or locking instruction is issued according to the sequence content, and a task distribution execution result list is obtained. Based on the task distribution execution result list, the task process is triggered on the target GPU instance to start, and the GPU resource consumption during the task operation is continuously tracked to obtain the GPU scheduling execution performance record.

[0006] Preferably, the steps of obtaining the video memory status evaluation result are: Call the GPU instance's video memory allocation monitoring interface to obtain the video memory allocation record of each GPU instance in the time window, calculate the standard deviation of the free area between consecutive video memory blocks to generate the video memory fragmentation degree value, traverse the kernel log to count the number of page migrations per unit time to obtain the page migration activity count, and form a basic feature set; Based on the basic feature set, an autoregressive prediction model is constructed to calculate the total resource demand of future time window tasks. The expression is: ; in, is the total amount of task resource requirements, The video memory allocation record for the previous time window. is the rate of change of the value of video memory fragmentation over time, Counts the migration activity for the current page. is the autoregressive coefficient fitted by historical data, is the white noise error term; The total resource requirement of the task and the confidence interval threshold In contrast, if ,in If the prediction benchmark value is calculated based on the sliding average of historical data, the prediction is determined to be valid and a video memory status evaluation result is generated.

[0007] Preferably, the steps for obtaining the resource capacity gap or redundancy are: Obtain the total amount of task resource requirements recorded in the video memory status evaluation result, and at the same time access the resource pool configuration database to extract the total capacity configuration parameters of the current GPU resource pool to generate a resource requirement-capacity matching data set; Based on the resource demand-capacity matching data set, the absolute difference between the total task resource demand and the total capacity configuration is calculated, and the difference result is compared with the preset capacity fluctuation threshold. If the absolute value of the difference exceeds the capacity fluctuation threshold and the total demand is greater than the total capacity, it is determined to be a resource capacity gap. If the absolute value of the difference exceeds the capacity fluctuation threshold and the total demand is less than the total capacity, it is determined to be a resource capacity redundancy, and a resource capacity gap or redundancy is generated.

[0008] Preferably, the steps of obtaining the GPU resource pool adjustment decision are: Accessing the instance specification metadata database provided by the cloud service provider, extracting a list of instance models that meet the current gap or redundancy range and associated startup time parameters and unit time holding cost parameters according to the numerical range of the resource capacity gap or redundancy, and generating an instance specification-cost data set; Based on the instance specification-cost data set, the candidate instance models are arranged in ascending order according to the startup time parameter, and the instance models to be released are arranged in descending order according to the unit time holding cost parameter, and the priority weight ratio of the startup time to the holding cost is set to 3:1, and the top three expansion instance models with the shortest startup time and the top three reduction instance models with the highest holding cost are screened out to generate a priority sorting list; According to the priority sorting list, the number of instances for expansion is calculated by dividing the gap by the single instance capacity specification and rounding up, and the number of instances for reduction is calculated by dividing the redundancy by the minimum release unit of the reduction instance and rounding down, and a GPU resource pool adjustment decision including instance model, quantity and execution timing is generated.

[0009] Preferably, the steps of obtaining the task and GPU matching list are: Traversing the GPU instance memory allocation records in the memory status evaluation result, calculating the number of continuous free memory blocks of each GPU instance, counting the coefficient of variation of the free block size, defining the ratio of the coefficient of variation to the total memory capacity of the instance as a fragmentation factor, and extracting the average delay time of page migration operations from the system log to generate an instance fragmentation-migration feature set; Based on the instance fragmentation-migration feature set, the matching degree between the computing task and the GPU instance; For each task, filter the GPU instances based on the matching degree, and sort them in descending order to generate a task and GPU matching degree list.

[0010] Preferably, the steps for obtaining the scheduling sequence of tasks to be executed are: Based on the task and GPU matching list, the scheduling priority of the task on the instance is calculated; Traverse all tasks, filter out instances with the highest scheduling priority and a state change time less than or equal to the maximum tolerable delay threshold of each task, and generate a scheduling sequence of tasks to be executed in descending order of scheduling priority.

[0011] Preferably, the steps for obtaining the task distribution execution result list are: Parse each record in the to-be-executed task scheduling sequence, extract the task ID, GPU instance ID, pre-allocated video memory capacity and planned execution timestamp, verify the current video memory availability and online status of the GPU instance ID, and generate a task-instance binding relationship table with validity identification; Based on the task-instance binding relationship table, a video memory operation instruction message is constructed, and instructions are sent in batches according to the planned execution timestamp sequence through the resource manager's REST API to generate an instruction delivery log with a time sequence mark; Listen to the asynchronous response message of the resource manager, match the transaction ID in the instruction delivery log, parse the operation status code in the response message, retry the failed instruction up to 3 times according to the error type, and generate a task distribution execution result list.

[0012] Preferably, the steps of obtaining the GPU scheduling execution performance record are: Parsing the successful records in the task distribution execution result list, extracting the task ID, GPU instance ID, and occupied video memory block address, sending a process start command to the target GPU instance, and generating a task process handle mapping table; Based on the task process handle mapping table, the video memory usage, computing power utilization and process survival time of the task are collected at intervals of 500 milliseconds, and the three are aligned by timestamp and merged to generate a resource consumption trajectory data set with time series marks; According to the resource consumption trajectory data set, the absolute deviation between the video memory peak and the pre-allocated capacity, and the relative deviation between the actual execution time and the planned time are calculated, and the deviation value is associated with the resource consumption trajectory and written into the database to generate a GPU scheduling execution performance record containing deviation indicators and original monitoring data.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: The present invention jointly optimizes the resource pool adjustment decision and the task matching list, screens the instance model and quantity under the dual constraints of the instance startup time parameter and the release cost parameter, and reduces the operation delay and additional cost caused by blind expansion and contraction. Based on the strong binding mechanism of video memory pre-allocation instructions and task process startup, combined with the continuous tracking of resource consumption trajectory, real-time feedback of video memory peak deviation and execution time deviation is formed to form a closed-loop scheduling optimization link. This logic quantifies video memory continuity through fragmentation factors and evaluates hardware stability through page migration overhead, improves the matching accuracy of tasks and instances, and reduces task interruptions caused by video memory fragmentation or migration delays. At the same time, the priority sorting list is used to drive the generation of scheduling sequences. On the premise of ensuring that high-demand tasks are executed first, the waiting time factor is used to dynamically correct the scheduling strategy, alleviate the problem of long-tail task hunger, and achieve a balance between resource utilization and task fairness. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] See also Figure 1 The present invention provides a technical solution, a cloud environment GPU resource dynamic scheduling method, comprising the following steps: Monitor the operating indicators of each GPU instance in the cloud environment in real time, collect GPU memory allocation records, GPU memory fragmentation values ​​and page migration activity counts, and generate memory status evaluation results by statistically predicting the total task resource requirements; Based on the memory status evaluation results, the predicted total resource demand is compared with the total capacity configuration of the current cloud environment GPU resource pool to obtain the resource capacity gap or redundancy. Based on the resource capacity gap or redundancy, the startup time parameters of the newly added GPU instances and the associated cost parameters of the released GPU instances are queried and evaluated to establish GPU resource pool adjustment decisions. Based on the memory status evaluation results, access the list of tasks to be scheduled, extract the GPU memory requirements of each task, generate a task and GPU matching list, and based on the task and GPU matching list, select the target GPU instance for each task in the queue of pending tasks and establish a scheduling sequence for pending tasks in combination with the time information of instance status changes in the GPU resource pool adjustment decision. Based on the scheduling sequence of tasks to be executed, the task allocation information and the corresponding GPU instance identifier are sent to the GPU resource manager of the cloud environment. According to the sequence content, GPU video memory pre-allocation or locking instructions are issued, and the task distribution execution result list is obtained. Based on the task distribution execution result list, the task process is triggered on the target GPU instance. The GPU resource consumption during the task operation is continuously tracked to obtain the GPU scheduling execution performance record.

[0017] The steps to obtain the video memory status evaluation results are as follows: Call the GPU instance's video memory allocation monitoring interface to obtain the video memory allocation record of each GPU instance in the time window, calculate the standard deviation of the free area between consecutive video memory blocks to generate the video memory fragmentation degree value, traverse the kernel log to count the number of page migrations per unit time to obtain the page migration activity count, and form a basic feature set; Based on the basic feature set, an autoregressive prediction model is constructed to calculate the total resource demand of tasks in the future time window. The expression is: ; in, is the total amount of task resource requirements, The video memory allocation record for the previous time window. is the rate of change of the value of video memory fragmentation over time, Counts the migration activity for the current page. is the autoregressive coefficient fitted by historical data, is the white noise error term; Compare the total task resource requirements with the confidence interval threshold In contrast, if ,in If the prediction benchmark value is calculated based on the sliding average of historical data, the prediction is determined to be valid and a video memory status evaluation result is generated.

[0018] Specifically, by calling the monitoring API interface of each GPU instance provided by the cloud platform, for example, using the function interface provided by NVIDIA's NVML library, setting the time window to 60 seconds, periodically obtain the detailed video memory usage information of each active GPU instance, record the starting address, size and allocation timestamp of the currently allocated video memory block, and form a video memory allocation record list for each instance. At the same time, the system traverses the list to calculate the size of all consecutive free video memory blocks, obtains an array of free block sizes, and calculates the standard deviation of this array. The standard deviation is the value of the degree of video memory fragmentation. For example, if an instance has free blocks [100MB, 50MB, 150MB], the mean is 100MB, and the standard deviation is , then the fragmentation value is 40.82. Then, the system accesses the Linux kernel log system (such as reading / var / log / kern.log or using the journalctl command), filters the page migration event log entries related to the target GPU device, and counts the total number of page migrations that occurred in the same 60-second time window to obtain the page migration activity count. For example, by searching for a specific kernel message pattern (such as a log line containing "pagemigration" and the GPU device ID), the count is obtained to obtain 5 migrations. Finally, the collected video memory allocation records of each instance, the calculated video memory fragmentation value, and the counted page migration activity count are integrated and organized in the format of [instance ID, timestamp, video memory allocation record list, video memory fragmentation value, page migration activity count] to form a basic feature set for subsequent predictive analysis.

[0019] formula: , the formula is useful in that it combines historical video memory usage ( )、Dynamic changes of video memory fragmentation( ) and the intensity of page migration activity ( ), an autoregressive model is constructed to predict the total GPU task resource demand in a short time window in the future. Compared with relying solely on historical allocations, this model takes into account the health status and usage pressure of the video memory, improves the accuracy and foresight of the prediction, and helps to adjust the resource pool size more timely.

[0020] The parameter is obtained by querying the last time window recorded in the basic feature set (for example, The size of all allocated video memory blocks in the list is accumulated. In the time window, the video memory allocation record of a GPU instance shows that a total of 5 tasks are allocated, and the video memory occupied is 2.5GB, 1.8GB, 3.2GB, 0.5GB, and 2.0GB respectively. GB.

[0021] The parameter is obtained by first obtaining the current time window. The value of the video memory fragmentation level (get e.g. 55.3) and the previous time window The value of the video memory fragmentation level (e.g. 40.82), then calculate the difference between the two and divide it by the length of the time window (e.g. 60 seconds). The calculation formula is: .For example, MB / sec. This positive value indicates increasing fragmentation.

[0022] The steps to obtain the parameter are as follows: This parameter represents the current time window The page migration activity count observed during the period is obtained by querying the current time window recorded in the underlying feature set (e.g. For example, in the current time window From the kernel log statistics, the number of page migrations is 8. .

[0023] The steps for obtaining parameters are as follows: these parameters are autoregressive coefficients obtained by fitting historical data, reflecting the weight of each feature on the total future resource demand. The acquisition method is to collect the basic feature set data of the past N time windows (including , , , ) and the actual total resource requirements of each window , build a linear regression model, use the least squares method to solve the multivariate linear regression equation, and get the optimal coefficient estimate. For example, collect data from the past 100 time windows, perform regression analysis, and get the fitting coefficients: , , The setting of these coefficients is based on the strength of the correlation between each factor in the historical data and future demand. Regression analysis will automatically find the optimal weights to minimize the forecast error. These coefficients will be regularly refitted and updated as new data is added to adapt to environmental changes.

[0024] The steps to obtain the parameter are as follows: the parameter represents the random fluctuation or white noise error term that the model cannot explain, and its expected value is 0. In actual prediction, it is usually taken as 0 for point estimation, or based on the residual distribution obtained by regression analysis (for example, calculating the standard deviation of the residual ) to estimate its possible range. For example, if the residual standard deviation obtained by regression analysis is 0.5GB, then it can be considered that It follows a normal distribution with a mean of 0 and a standard deviation of 0.5. In this calculation, we use point estimates, .

[0025] Calculation process: Substitute specific values ​​for calculation: Known GB, MB / sec= GB / sec, ; calculate ; Calculate the total amount of task resource requirements : ; ; ; ; The results show that based on the amount of video memory allocated in the previous time window, the increasing trend of current video memory fragmentation, and the activeness of page migration activities, the model predicts that the total resource demand of GPU tasks in the next time window will be approximately 9.635GB. It will serve as a key basis for evaluating whether the current resource pool capacity is sufficient.

[0026] formula middle, The steps to obtain the parameter are as follows: This parameter is the total predicted value of the future time window task resource demand calculated by the autoregressive prediction model. It can be seen from the calculation that, GB.

[0027] The parameter is obtained by using the sliding average method based on the total resource demand data of historical tasks to calculate the predicted benchmark value. It represents the average level of recent resource demand. The calculation method is to select the actual resource demand (or the verified predicted value) of the most recent M time windows. , calculate their average. For example, select M=10, that is, use the data of the past 10 time windows. For example, the actual demand volume (in GB) of these 10 windows is: [9.8, 9.5, 10.1, 9.9, 9.7, 9.6, 9.4, 10.0, 9.8, 9.7], then GB.

[0028] The steps to obtain the parameter are as follows: This parameter is the confidence interval threshold, which is used to define the predicted value With reference value The acceptable deviation range between the two. The setting basis is usually the statistical characteristics of the historical forecast error, for example, it can be set to k times the standard deviation of the historical forecast error. Collect historical data at multiple time points and , calculate the absolute value sequence of the difference between them, and then calculate the standard deviation of this sequence . Set the confidence level, for example, the k value corresponding to the normal distribution of 95% confidence level is about 1.96. For example, the standard deviation calculated based on historical error data GB, set k = 2 (corresponding to 95% confidence level, rounded to facilitate calculation), then . Calculation process: GB. This threshold setting means that if the model's prediction value deviates from the recent average demand level by less than 0.5 GB, the prediction is considered reliable. This threshold can be adjusted according to the requirements for prediction stability. The higher the requirement, the more reliable the prediction. The smaller the value.

[0029] Compare and judge: Known GB GB GB; Compute the absolute value of the difference: ； The calculated absolute value of the difference and the threshold For comparison: ; The judgment condition is established. This result shows that the total amount of task resource requirements predicted by the model (9.635GB) and the predicted benchmark value calculated based on the moving average of historical data The absolute difference between the two (0.115 GB) is less than the preset confidence interval threshold. (0.5GB). Therefore, the prediction result obtained by the autoregressive model is determined Based on this effective prediction result, the system generates a memory status evaluation result, which will include a confirmed effective total task resource demand forecast value. GB, and possibly other status information (such as the current degree of fragmentation, page migration activity, etc.), for subsequent resource capacity analysis and task scheduling decisions.

[0030] The steps to obtain resource capacity gap or redundancy are: Obtain the total task resource demand recorded in the video memory status evaluation result, access the resource pool configuration database, extract the total capacity configuration parameters of the current GPU resource pool, and generate a resource demand-capacity matching data set; Based on the resource demand-capacity matching data set, the absolute difference between the total task resource demand and the total capacity configuration is calculated, and the difference result is compared with the preset capacity fluctuation threshold. If the absolute value of the difference exceeds the capacity fluctuation threshold and the total demand is greater than the total capacity, it is determined to be a resource capacity gap. If the absolute value of the difference exceeds the capacity fluctuation threshold and the total demand is less than the total capacity, it is determined to be a resource capacity redundancy, and a resource capacity gap or redundancy is generated.

[0031] Specifically, the system first extracts key data items from the memory status evaluation results generated in the previous stage, namely the total amount of verified future time window task resource requirements , for example, the obtained value is GB, at the same time, the system initiates a query request to the configuration management database (CMDB) or the cloud platform resource management API to access the resource pool configuration database that records the detailed information of the current GPU resource pool. The database stores the specifications (such as model, memory size) and status (such as running, stopped) of all GPU instances in the resource pool. The system traverses all GPU instances with a status of "running", accumulates their memory capacity parameters, and obtains the total capacity configuration parameters of the current GPU resource pool. For example, if there are two running instances in the resource pool, each with 16GB of video memory, the total capacity configuration is GB, and finally, the total amount of task resource requirements And the calculated total capacity configuration Combined to form a structured record, such as the key-value pair format {'predicted_demand_Q': 9.635, 'total_capacity_C': 32.0}, this record is the resource demand-capacity matching dataset.

[0032] Based on the resource demand-capacity matching dataset generated in the previous step, for example, containing the record {'predicted_demand_Q': 9.635, 'total_capacity_C': 32.0}, the system first calculates the total resource demand of the task. Configuration with total capacity The absolute difference between them is calculated by directly finding the absolute value of the difference between them, that is, GB, then compare this absolute difference result (22.365GB) with a preset capacity fluctuation threshold. The capacity fluctuation threshold is set to avoid frequent adjustments to the resource pool due to small, temporary mismatches between supply and demand. Its value can be determined based on statistical analysis of historical supply and demand differences. For example, collect the absolute difference every minute in the past month. , calculate the standard deviation of these difference data, for example, the standard deviation is 1.5GB, to allow a certain normal fluctuation range, set the threshold to 1.5 times the standard deviation, that is, the capacity fluctuation threshold = GB, the system compares the calculated absolute difference of 22.365GB with the threshold of 2.25GB and finds , indicating that the difference is significant and exceeds the normal fluctuation range. It is necessary to further determine whether it is a gap or redundancy. At this time, check the original difference GB, because the difference is negative (that is, the total demand is less than the total capacity), it is determined that there is currently resource capacity redundancy, and its value is 22.365GB. If the original difference is positive and the absolute difference exceeds the threshold, it is determined to be a resource capacity gap. Finally, the system generates a resource capacity gap or redundancy based on the determination result. In this case, the generated result is "Resource capacity redundancy: 22.365GB".

[0033] The steps to obtain the GPU resource pool adjustment decision are as follows: Access the instance specification metadata database provided by the cloud service provider, and extract the instance model list that meets the current gap or redundancy range and the associated startup time parameters and unit time holding cost parameters according to the numerical range of the resource capacity gap or redundancy, and generate an instance specification-cost data set; Based on the instance specification-cost data set, candidate instance models are sorted in ascending order by the startup time parameter, and instance models to be released are sorted in descending order by the unit time holding cost parameter. The priority weight ratio of startup time to holding cost is set to 3:1, and the top three expansion instance models with the shortest startup time and the top three reduction instance models with the highest holding cost are screened to generate a priority sorting list; According to the priority sorting list, the number of instances for expansion is calculated by dividing the gap by the single instance capacity specification and rounding up. The number of instances for reduction is calculated by dividing the redundancy by the minimum release unit of the reduced instance and rounding down. A GPU resource pool adjustment decision is generated that includes the instance model, quantity, and execution timing.

[0034] Specifically, the system first calls the resource query interface provided by the cloud service provider to access its instance specification metadata database, which contains detailed technical specifications and pricing information of various GPU instances. Then, based on the specific value of the resource capacity gap or redundancy determined in the previous step, for example, if there is a resource capacity redundancy of 22.365GB, the system selects the instances that can be considered for release from the list of GPU instances running in the current resource pool (the list contains information such as instance ID, instance type, startup time, and unit time cost, which is usually obtained from the internal resource management system or cloud platform API). The screening criteria are the instance type and its associated unit time holding cost parameter, for example Filter out all running instances and their types and costs, such as [{'instance_id':'i-1', 'type':'TypeQ', 'capacity_gb':16, 'cost_per_hour':1.0}, {'instance_id':'i-2', 'type':'TypeQ', 'capacity_gb':16, 'cost_per_hour':1.0}], and organize and compile the specification information (type, capacity, startup time, holding cost) of these instances into a structured list, which is the instance specification-cost dataset.

[0035] Based on the instance specification-cost dataset generated in the previous step, for example, it contains the running instance information in the redundant scenario [{'instance_id':'i-1', 'type':'TypeQ', 'capacity_gb':16, 'startup_time_min':8, 'cost_per_hour':1.0}, {'instance_id':'i-2', 'type':'TypeQ', 'capacity_gb':16, 'startup_time_min':8, 'cost_per_hour':1.0}, min': 8, 'cost_per_hour': 1.0}], the system performs different sorting and filtering logics based on the current determination of resource capacity redundancy (requires capacity reduction) or resource capacity gap (requires capacity expansion). In the current redundant scenario (reduction in capacity), the system will sort the involved instances (or instance types, if the cost is aggregated by type) in descending order according to the unit time holding cost parameter in the instance specification-cost data set, and give priority to the instance with the highest cost for release. If there are multiple instances with the same cost, they can be sorted twice according to the longest running time or other strategies (such as the latest startup time). Sorting. In this example, the two instances have the same cost. Sorting by instance ID yields ['i-2', 'i-1'] or ['i-1', 'i-2']. Then, the top three instance models (if multiple models are involved) or instance IDs with the highest holding cost (in this example, all candidate instances) are selected as candidates to be released. In this case, the candidate instances are i-1 and i-2. If it is a capacity expansion scenario, the startup time parameter in the instance specification is sorted in ascending order. The top three instance models with the shortest startup time are selected as candidates. The priority weight ratio of startup time to holding cost is set to 3:1. This weight setting is based on business considerations. For example, when expanding capacity, quickly obtaining computing power (startup time weight is 3) is more important than cost (weight is 1). When shrinking capacity, it is more important to release high-cost instances first (cost weight is 3) than their shutdown speed (startup time, weight is 1). However, in the specific screening of this step, according to the description, the top three are selected after sorting by a single indicator (expansion in ascending time, and shrinkage in descending cost), and finally a priority sorting list is formed that includes preferred instances (models for expansion and specific instance IDs for shrinkage) and their key parameters (startup time or cost).

[0036] The system performs specific instance quantity calculation and decision-making based on the priority list generated in the previous step and the determined resource capacity gap or redundancy. In the scaling-down scenario where the current resource capacity redundancy is 22.365 GB, the system selects the top-ranked instance from the priority list (including candidate instances i-1 and i-2, both of TypeQ, cost $1.0 / hr) for processing, selects instance i-1 (or i-2), determines its capacity specification as 16 GB, and defines the minimum release unit of the scaled-down instance as the capacity of a single instance, that is, 16 GB. Then, the number of instances that need to be scaled down is calculated by dividing the redundancy (22.365 GB) by the minimum release unit of the scaled-down instance (16 GB), and rounding the result down, that is, floor(22.365 / 16)=floor(1.397)=1, which means that one TypeQ instance needs to be released. If the scenario is a resource capacity gap, for example, the gap is 20GB, and the preferred expansion instance model determined by the priority sorting list is TypeQ (capacity 16GB), then the method for calculating the number of expansion instances is to divide the gap (20GB) by the single instance capacity specification (16GB), and round up the result, that is, ceil(20 / 16)=ceil(1.25)=2, indicating that 2 TypeQ instances need to be added. Finally, the system integrates the calculation results and generates a clear GPU resource pool adjustment decision, which includes the adjustment action (addition or removal), instance model (when expanding) or instance ID (when shrinking), the calculated number of instances, and the planned execution time (for example, immediate execution or execution in a specified time window). For the current shrinking case, the generated decision is: {adjustment action: removal, instance ID: i-1, instance type: TypeQ, quantity: 1, execution sequence: immediate}.

[0037] The steps to obtain the task and GPU matching list are as follows: Traverse the GPU instance memory allocation records in the memory status evaluation results, calculate the number of consecutive free memory blocks of each GPU instance, and count the coefficient of variation of the free block size ,in is the standard deviation, is the mean, and the coefficient of variation is related to the total memory capacity of the instance The fragmentation factor is defined as ,At the same time, the average delay time of page migration operations is extracted from the system log to generate the instance fragmentation-migration feature set; Based on the instance fragmentation-migration feature set, the matching degree between the calculation task and the GPU instance is expressed as: ; in, For the matching degree, For GPU instances The fragmentation factor, For example Available video memory capacity, For the task The memory requirement of To fit the attenuation coefficient of video memory allocation efficiency, For example The average delay time of page migration; For each task, filter and GPU instance, where To preset the matching threshold, generate a task and GPU matching list in descending order of matching.

[0038] Specifically, the formula: middle, The steps to obtain the parameter are as follows: This parameter represents the GPU instance The standard deviation of the current sizes of all consecutive free video memory blocks. First, obtain the total free memory by calling the video memory monitoring interface, and obtain the information of each free block by analyzing the internal video memory management data structure or using specific tools to obtain the instance Get the size list of all the consecutive free video memory blocks, for example, get the list [1.5GB, 0.5GB, 2.0GB]. Then, calculate the standard deviation of the list. Calculation process: First calculate the mean GB, then calculate the variance , and finally take the square root to get the standard deviation GB.

[0039] The steps to obtain the parameter are as follows: This parameter represents the GPU instance The average size of all the current continuous free video memory blocks. The acquisition method is the same as The first step is to obtain a list of consecutive free memory block sizes and directly calculate their average values. For example, for the list [1.5GB, 0.5GB, 2.0GB], GB.

[0040] The steps to obtain the parameter are as follows: This parameter represents the GPU instance This parameter is a fixed specification parameter of the instance and can be obtained by querying the instance specification metadata database provided by the cloud service provider or through the monitoring interface when the instance is started. For example, for an NVIDIA A100 40GB instance, its total video memory capacity is GB.

[0041] Calculation process: Substitute specific values ​​to calculate the fragmentation factor : Known GB GB GB; Calculate the coefficient of variation: ; Calculate the fragmentation factor : ; This step also requires obtaining the average delay time of the page migration operation. The acquisition method is: continuously monitor the system kernel log, filter the page migration event log entries specific to the GPU device (can be filtered by device PCIID or driver log identifier), record the start and end timestamps of each migration operation, calculate the duration, and then calculate the average of all recorded migration durations within a set time window (for example, the past 5 minutes). For example, if 3 page migrations are observed in the past 5 minutes, with durations of 15ms, 25ms, and 20ms respectively, the average latency is ms.

[0042] The final generated instance The instance shard-migration feature set is {'instance_id': k, 'F_k': 0.0117, 'M_k': 20}.

[0043] This result shows that the example The fragmentation factor is 0.0117 and the average delay of page migration is 20ms. Smaller values ​​indicate less fragmentation (relative to its total capacity). The smaller the value, the higher the page migration efficiency. These two indicators together form the basis for evaluating whether the current state of the GPU instance is suitable for carrying new tasks.

[0044] formula: ; (The integral calculation result is: ); but ; The benefit of the formula is that it takes into account multiple key factors that affect the efficiency of tasks running on a specific GPU instance, and designs a comprehensive matching Calculation model. It is inversely proportional to the fragmentation factor and page migration delays , which means lower fragmentation, faster migration, and higher matching. At the same time, through a , Task Requirements and allocative efficiency decay The integral term quantifies the possibility and efficiency of successfully allocating the required video memory and penalizes the situation where the available space is close to the task demand, so that the matching degree can more accurately reflect the potential of the task to start and run smoothly on the instance.

[0045] The steps to obtain the parameters are as follows: The fragmentation factor is obtained from the instance fragmentation-migration feature set generated by calculation. For example, .

[0046] The steps to obtain the parameter are as follows: This parameter represents the GPU instance The current real-time available video memory capacity. This value is obtained by calling the real-time video memory monitoring interface. For example, the current monitoring instance The available video memory is GB.

[0047] The steps to obtain the parameter are as follows: the parameter represents the task to be scheduled The declared GPU memory requirement. This value is specified by the task submitter when submitting the task, or estimated through static analysis of the task code or historical running data analysis. For example, the task Declare when submitting GB video memory.

[0048] The steps to obtain the parameter are as follows: This parameter is the decay coefficient of the memory allocation efficiency, which reflects the speed at which the allocation success rate or efficiency decreases when the requested memory size is close to the available memory. This coefficient needs to be fitted through experiments or historical data analysis based on the behavior of the specific GPU model and the driver / memory manager. Collect historical data and record it at different available memory Next, request different sizes The success or failure of the task allocation or the time taken to allocate the task. When it decreases, the allocation success rate decreases, or the allocation time increases. Fit an exponential decay model and solve it to get For example, by performing nonlinear regression analysis on the historical allocation data of A100 GPU under a specific load, we can get (Unit: 1 / GB). Higher A large value indicates that allocation efficiency drops sharply when video memory is close to full.

[0049] The steps to obtain the parameter are as follows: This parameter represents the GPU instance The average delay time of page migration is obtained from the instance fragmentation-migration feature set generated by calculation. For example, ms.

[0050] Calculation process: Substitute specific values ​​to calculate the matching degree : Known GB GB (1 / GB) ms; Compute the exponential term: ; ; Compute the value of the integral part: ; Compute the logarithmic term: ; Calculate the matching degree : ; ; ; ; The results show that the task (Requires 8GB) and GPU instance The matching score of (15GB available, fragmentation factor 0.0117, migration delay 20ms) is 10.43. This value itself has no absolute unit, and its relative size is used to compare the adaptability of different instances to the same task, or the adaptability of the same instance to different tasks. The higher the score, the better the instance. The current state (low fragmentation, sufficient contiguous space possibility, low migration overhead) is more suitable for running tasks. .

[0051] formula: and ; The steps to obtain the parameters are as follows: The amount of video memory required. For example GB.

[0052] The steps to obtain the parameters are as follows: The current available video memory capacity. For example GB.

[0053] The steps to obtain the parameters are as follows: the parameters are calculated tasks With examples For example .

[0054] The parameter acquisition steps are as follows: This parameter is a preset matching threshold, which is used to filter out instances with low matching. This threshold should be based on the historical task performance and matching score. Collect a large number of tasks in different instances (with different Score) on the running data (such as completion time, GPU utilization, etc.), drawing performance indicators Observe whether there is a scatter plot or curve of changes. When the score is lower than this value, the task performance will drop significantly or the failure rate will increase significantly. A value near this critical point is selected as the threshold For example, by analyzing historical data, it is found that when When the value is lower than 6.0, the task running time increases significantly, so set .

[0055] To the task and examples Filter: Known GB GB

[0056] Check the first condition: ; The conditions are established.

[0057] Check the second condition: ; The conditions are established.

[0058] Since both conditions are met, the instance Considered a task The system needs to repeat this process for all available GPU instances to find a qualified candidate instance for the task Find all instances that meet the condition. For example, there are also instances ,That GB, ,but (8<=10) holds, (7.5>6.0) holds, so the example is also qualified. For example ,That GB, ,but (8<=20) holds, but (5.5>6.0) does not hold, so the example After finding all eligible instances, sort them by their matching degree. The scores are sorted in descending order.

[0059] For example, a qualifying instance is (score 10.43) and (score 7.5), the sorted list is [(k, 10.43), (k', 7.5)]. This result shows that for the task , Example and All of them are candidates that meet the basic video memory requirements and are in good condition. The matching degree of the task and GPU is higher and is a better choice. The final generated task and GPU matching degree list provides a list of qualified GPU instances sorted from high to low by matching degree for each task to be scheduled. For example, The list is [(k, 10.43), (k', 7.5)] for subsequent scheduling decisions.

[0060] The steps to obtain the task scheduling sequence to be executed are: Calculate tasks based on task and GPU matching list In the example The scheduling priority on is calculated as: ; in, is the scheduling priority, For the task With examples The matching degree, For example The state change time, is the system basic scheduling overhead constant (calibrated through stress testing), For the task The accumulated waiting time in the current queue, is the total number of tasks to be processed in the queue, To prevent overflow of zero wait time for a very small positive number, For the task The accumulated waiting time in the current queue; Traverse all tasks and filter each task with the highest scheduling priority. An instance of the maximum tolerable delay threshold of a task generates a scheduling sequence of tasks to be executed in descending order of scheduling priority.

[0061] Specifically, the formula: The benefit of the formula is that it provides a quantitative priority calculation method for task scheduling decisions, which not only takes into account the task With examples Technical matching (reflecting operational efficiency and stability), and also combines the instant availability of instances (measured by state change time and scheduling overhead embodied), and introduced the waiting time of the task As a factor to measure the urgency of a task, by combining the waiting time of a task with its relative waiting situation in the entire queue (normalized by the square root sum of the waiting times of other tasks), a comprehensive consideration of task fairness and urgency is achieved, so that high-priority tasks (high matching degree, fast instance preparation, and long waiting time) can be scheduled first.

[0062] The steps to obtain the parameter are as follows: With GPU instances The matching score of is derived from the calculation result of the previous step "task and GPU matching list". For example, according to the previous calculation, task With examples The matching degree .

[0063] The steps to obtain the parameter are as follows: This parameter represents the GPU instance The state change time is the time required for the instance to accept new tasks. If the instance is currently in a stable running state and idle, then If the instance is in the process of being started based on a resource pool adjustment decision, is the estimated remaining startup time, which can be obtained from the cloud platform API or resource manager. Currently ready, then Second.

[0064] The steps to obtain the parameter are as follows: This parameter is the system basic scheduling overhead constant, which represents the minimum time overhead inherent in the scheduling system to process a task allocation request, and is independent of the specific task and instance. This constant is calibrated by stress testing the scheduling system: under different loads, a large number of simple task scheduling requests with negligible execution time are sent to the system, and the end-to-end time from the issuance of the request to the confirmation of the allocation completion (excluding the actual execution time of the task) is measured, and the average value or P99 value of these times is calculated. For example, through stress testing, the average time taken by the system to complete a basic scheduling operation is Second.

[0065] The steps to obtain the parameters are as follows: The cumulative waiting time in the pending queue from the time the task was submitted to the current time. The scheduler records the timestamp of the task when it enters the queue and subtracts the timestamp from the current time each time the priority is calculated. For example, the task has been waiting in the queue for 120 seconds, then Second.

[0066] The parameter is obtained as follows: This parameter represents the total number of tasks in the current pending task queue. The scheduler maintains the pending task queue in real time, and the value can be obtained by directly counting the number of tasks in the queue. For example, if there are 5 tasks waiting to be scheduled in the current queue, then .

[0067] The parameter is obtained as follows: This is a very small positive constant used to prevent the situation where the waiting time of all tasks is zero (for example, when the queue is just established) or the denominator is zero during the calculation process. Its value should be much smaller than the typical waiting time or scheduling time unit. Just set a fixed small value. For example, set Second.

[0068] The steps to obtain the parameter are as follows: The scheduler needs to traverse all tasks in the current queue. tasks, get the cumulative waiting time of each task For example, the waiting times of the five tasks in the queue are: Task 1 ( ) Seconds, Task 2 Seconds, Task 3 Seconds, Task 4 Seconds, Task 5 Second.

[0069] Calculation process: Substitute specific numerical calculation tasks In the example Scheduling priority on : Known Second, Second, Second, , the waiting time of all tasks in the pending queue are [120, 60, 180, 30, 90] seconds respectively, Second.

[0070] Calculate the waiting time related items: Calculate the total waiting time: ; Calculate the sum of the square roots of the waiting times: ; Computational tasks Relative waiting time factor: ; Calculate items related to matching degree and instance preparation time: ; Calculate the final scheduling priority : ; ; The results show that the task On GPU instances The scheduling priority score on is 114.29. This score comprehensively reflects the matching degree between the task and the instance, the readiness of the instance, and the waiting urgency of the task itself. The scheduler will Calculate its priority score on all eligible instances and combine it with the priority scores of other tasks to determine the final scheduling assignment. The task-instance pairing with a higher priority score is more likely to be executed first.

[0071] The scheduling priority of each task on its candidate GPU instance calculated in the previous step , the system then traverses all the tasks to be scheduled, and for each task , from its candidate instance list (i.e., satisfying and The final screening is carried out in the instance) and the screening criteria include two aspects: one is the instance status change time must be less than or equal to the maximum tolerance delay threshold allowed by the task. Second, the instance must have the highest scheduling priority among all candidate instances that meet the first time constraint. The maximum tolerable delay threshold of a task is specified when the task is submitted or preset according to the task type (for example, the threshold for interactive tasks is low, and the threshold for batch tasks is high). It defines the maximum time that a task can wait for an instance to be ready from the current moment. For example, the maximum tolerable delay threshold of task j is 300 seconds, and the threshold of instance k is seconds, instance k' seconds, both of which satisfy , if instance k And the instance k' , then instance k is selected as the target execution instance of task j because it has the highest priority. After the system determines the unique best target instance for each task to be scheduled, it collects these (task ID, target instance ID, scheduling priority) triples and finally selects them according to the scheduling priority. The values ​​of are globally sorted in descending order to form the final task scheduling sequence to be executed.

[0072] The steps to obtain the task distribution execution result list are as follows: Parse each record in the task scheduling sequence to be executed, extract the task ID, GPU instance ID, pre-allocated video memory capacity and planned execution timestamp, verify the current video memory availability and online status of the GPU instance ID, and generate a task-instance binding relationship table with validity identification; Based on the task-instance binding relationship table, the memory operation instruction message is constructed, and the instructions are sent in batches according to the planned execution timestamp sequence through the resource manager's REST API to generate an instruction delivery log with a time sequence mark; Listen to the asynchronous response message of the resource manager, match the transaction ID in the instruction delivery log, parse the operation status code in the response message, retry the failed instruction up to 3 times according to the error type, and generate a task distribution execution result list.

[0073] Specifically, the system processes each scheduling record in the task scheduling sequence to be executed in sequence. For example, when processing the record (task_j, instance_k, P_jk), it first extracts the task identifier (task ID) as 'j', the target GPU instance identifier (GPU instance ID) as 'k', and the pre-allocated video memory capacity required by the task (i.e. the demand for the task). , for example, 8GB), and determine a planned execution timestamp based on the sequence order or priority, such as setting it to be executed within a few seconds after the current time. Then, before actually issuing the binding instruction, the system performs a final verification. The system initiates a real-time query to the resource manager or cloud platform monitoring interface to obtain the latest status of instance 'k', verify whether it is online (for example, the status is 'Running' or 'Active') and operational, and check its current real-time available video memory capacity Is it still greater than or equal to the pre-allocated video memory capacity (8GB) required by task 'j'? The binding relationship between the task and the instance is considered valid only when the instance is online and the video memory meets the requirements. The system records this information (task ID, instance ID, requested video memory, and planned timestamp) together with the validity verification result (Boolean value True or False) to form a task-instance binding relationship table with validity identification.

[0074] According to the task-instance binding relationship table with validity identification generated in the previous step, the system filters out all binding records marked as valid. For each valid record, for example, task 'j' is bound to instance 'k' and requests 8GB of video memory, the system constructs a specific video memory operation instruction message based on this. The message is usually in JSON or XML format and contains all the parameters required to perform the operation, such as {"transaction_id":"unique-tx-id-123","operation":"reserve_gpu_memory","target_instance":"k","requester_task":"j","memory_size_gb":8,"lock_mode":"exclusive"}, A uniquely generated transaction ID is included for subsequent tracking. Subsequently, the system sorts these instruction messages according to the planned execution timestamp in the record, and organizes instructions with the same or similar timestamps into batches (if the resource manager API supports batch operations). By calling the RESTful API interface provided by the resource manager (for example, sending a request to the POST / v1 / gpu / instances / {instance_id} / memory / reservations endpoint), these constructed instruction messages are sent in chronological order. Each time an instruction (or a batch of instructions) is successfully sent, the system records an entry in the instruction delivery log, including the precise timestamp of the sending, the transaction ID used, the target instance ID, the task ID, and the complete instruction message content, to generate an instruction delivery log with a time stamp.

[0075] The system starts a background process or service, and continuously listens to the asynchronous response messages sent back by the resource manager through a specified mechanism (such as a message queue, callback URL, or polling status interface). Each time a response message is received, the system first parses the message content, extracts the transaction ID, and uses this transaction ID to find the matching original instruction record in the instruction issuance log to associate the response with its corresponding task and instance. Next, the operation status code indicating the operation result in the response message (for example, HTTP status code 200 indicates success, 4xx or 5xx indicates failure) and the error information text that may be contained are parsed. If the status code indicates that the operation failed, the system will determine whether the failure can be retried based on the predefined error type. For example, for errors indicating that resources are temporarily unavailable (such as status code 503) or temporary network problems, they are It is set as a retryable error, while errors indicating invalid parameters (such as status code 400) or non-existent resources (such as status code 404) are set as non-retryable errors. For retryable failed instructions, as long as the cumulative number of retries has not reached the set upper limit (for example, a maximum of 3 times), the system will arrange to resend the instruction at a later time (fixed interval or exponential backoff strategy can be used) and update the retry count. If the instruction is finally successfully executed (successful for the first time or successful after retries), or still fails after reaching the maximum number of retries, or encounters a non-retryable error, the system will record the final result (including task ID, instance ID, success or failure status, allocated memory block address information when successful, error details and number of retries when failed), summarize the final distribution processing results of all tasks, and generate a task distribution execution result list.

[0076] The steps to obtain GPU scheduling execution performance records are as follows: Parse the successful records in the task distribution execution result list, extract the task ID, GPU instance ID, and occupied video memory block address, send a process start command to the target GPU instance, and generate a task process handle mapping table; Based on the task process handle mapping table, the video memory usage, computing power utilization, and process survival time of the task are collected at 500 millisecond intervals. The three are aligned by timestamp and merged to generate a resource consumption trajectory dataset with time series tags. According to the resource consumption trajectory dataset, the absolute deviation between the peak value of video memory and the pre-allocated capacity, and the relative deviation between the actual execution time and the planned time are calculated. The deviation value is associated with the resource consumption trajectory and written into the database to generate a GPU scheduling execution efficiency record containing deviation indicators and original monitoring data.

[0077] Specifically, the system first parses the task distribution execution result list, filters out the records with the status of "success", and for each successful record, extracts the corresponding task ID, target GPU instance ID, and the address of the successfully pre-allocated or locked video memory block returned by the resource manager (if any). Then, the system uses this information, combined with the execution logic of the task (for example, the task's executable file path, startup parameters, required environment variables, etc.), to construct a process startup command, which is sent to the node corresponding to the specified GPU instance ID through an appropriate remote execution mechanism (such as executing the command line after SSH connection, calling the container orchestration system API such as KubernetesJobAPI, or through a proxy service deployed on the target instance). The command must contain the necessary environment variables. Environment variables are set, such as specifying the specific GPU device index assigned to the task through `CUDA_VISIBLE_DEVICES`, and passing the pre-allocated video memory block address (if the application needs to use it directly) to the task process. After successfully starting the task process, the execution environment on the target instance returns a unique handle that identifies the process (such as the operating system's process ID (PID), container ID, or job ID). The system collects these task IDs and the corresponding process handles and instance IDs, and generates and maintains a task process handle mapping table, such as `{'task_id_j': {'process_handle': 'pid_12345', 'instance_id': 'k'}, ...}`.

[0078] Based on the maintained task process handle mapping table, the system starts a periodic monitoring process. For each active task process handle recorded in the table, the system collects the real-time resource consumption data of the process on the specified GPU instance at a fixed time interval of 500 milliseconds through remote query or by using the monitoring agent deployed on the instance. The specific collection content includes: querying the actual GPU memory usage (units such as GB) associated with the process handle and the computing unit utilization (percentage) of the GPU core by calling the NVIDIA Management Library (NVML) interface or executing the `nvidia-smi` command. At the same time, by querying the process management information of the operating system (for example, checking the ` / proc / [PID]` directory or using the `ps` command), it is confirmed whether the process is still alive and calculates its cumulative running time since startup (process survival time). The system associates the video memory usage, computing power utilization, process survival status (such as Boolean value True / False) and accumulated running time collected each time with the precise timestamp of the collection, and merges the data collected at the same timestamp into one record. These records are continuously accumulated to form a complete and time-stamped resource consumption trajectory dataset for each task, such as `[{'timestamp':'ts1', 'task_id':'j', 'instance_id':'k', 'memory_gb':7.5, 'utilization_pct':85, 'alive':True, 'runtime_sec':60.5}, {'timestamp':'ts2', ...}]`.

[0079] After the task is completed (i.e., the process is no longer alive) or the preset maximum monitoring time is reached, the system processes the resource consumption trajectory dataset generated by the task. First, it traverses the video memory usage sequence in the dataset and finds its maximum value, which is the video memory peak value. Then, it compares the video memory peak value with the pre-allocated capacity initially requested by the task (e.g., 8GB requested by task j), and calculates the absolute deviation between the two, for example, `|peak 7.8GB-pre-allocated 8GB|=0.2GB`. Then, the actual execution time of the task is determined from the dataset, that is, the time difference between the start of the process and the last timestamp of the survival detection. At the same time, the planned execution time of the task is obtained (this planned time usually refers to the estimated execution time, which needs to be provided by the task submitter or estimated based on historical data). , for example, the planned execution time is 1200 seconds), the relative deviation between the actual execution time (for example, 1150 seconds) and the planned execution time is calculated, for example, `(1150-1200) / 1200=-0.0417` or -4.17%. If the planned execution time is not available, this deviation is not calculated. Finally, the system associates the two performance indicators, the absolute deviation between the calculated video memory peak and the pre-allocated capacity, and the relative deviation between the actual execution time and the planned time, with the basic information of the task (task ID, instance ID, start / end time, etc.) and the complete resource consumption trajectory data set (or its storage location reference), and writes them into the specified performance database or log storage as a whole record to generate a GPU scheduling execution performance record containing deviation indicators and original monitoring data.

[0080] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A cloud environment GPU resource dynamic scheduling method, characterized in that: The following steps are involved: Monitor the operating indicators of each GPU instance in the cloud environment in real time, collect GPU memory allocation records, GPU memory fragmentation values ​​and page migration activity counts, and generate memory status evaluation results by statistically predicting the total task resource requirements; Based on the memory status evaluation result, the predicted total resource demand is compared with the total capacity configuration of the current cloud environment GPU resource pool to obtain the resource capacity gap or redundancy. Based on the resource capacity gap or redundancy, the startup time parameters of the newly added GPU instances and the associated cost parameters of the released GPU instances are queried and evaluated to establish a GPU resource pool adjustment decision. Based on the memory status evaluation result, access the list of tasks to be scheduled, extract the GPU memory requirement of each task, generate a task and GPU matching list, select a target GPU instance for each task in the task queue to be processed based on the task and GPU matching list and combine the time information of the instance status change in the GPU resource pool adjustment decision, and establish a scheduling sequence for the tasks to be executed; Based on the scheduling sequence of tasks to be executed, the task allocation information and the corresponding GPU instance identifier are sent to the GPU resource manager of the cloud environment, and a GPU video memory pre-allocation or locking instruction is issued according to the sequence content, and a task distribution execution result list is obtained. Based on the task distribution execution result list, the task process is triggered on the target GPU instance to start, and the GPU resource consumption during the task operation is continuously tracked to obtain the GPU scheduling execution performance record.

2. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps of obtaining the video memory status evaluation result are as follows: Call the GPU instance's video memory allocation monitoring interface to obtain the video memory allocation record of each GPU instance in the time window, calculate the standard deviation of the free area between consecutive video memory blocks to generate the video memory fragmentation degree value, traverse the kernel log to count the number of page migrations per unit time to obtain the page migration activity count, and form a basic feature set; Based on the basic feature set, an autoregressive prediction model is constructed to calculate the total resource demand of future time window tasks. The expression is: ； in, is the total amount of task resource requirements, The video memory allocation record for the previous time window, is the rate of change of the value of video memory fragmentation over time, Counts the migration activity for the current page. is the autoregressive coefficient fitted by historical data, is the white noise error term; The total resource requirement of the task is compared with the confidence interval threshold In contrast, if ,in If the prediction benchmark value is calculated based on the sliding average of historical data, the prediction is determined to be valid and a video memory status evaluation result is generated.

3. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps for obtaining the resource capacity gap or redundancy are: Obtain the total amount of task resource requirements recorded in the video memory status evaluation result, and at the same time access the resource pool configuration database to extract the total capacity configuration parameters of the current GPU resource pool to generate a resource requirement-capacity matching data set; Based on the resource demand-capacity matching data set, the absolute difference between the total task resource demand and the total capacity configuration is calculated, and the difference result is compared with the preset capacity fluctuation threshold. If the absolute value of the difference exceeds the capacity fluctuation threshold and the total demand is greater than the total capacity, it is determined to be a resource capacity gap. If the absolute value of the difference exceeds the capacity fluctuation threshold and the total demand is less than the total capacity, it is determined to be a resource capacity redundancy, and a resource capacity gap or redundancy is generated.

4. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps for obtaining the GPU resource pool adjustment decision are as follows: Accessing the instance specification metadata database provided by the cloud service provider, extracting a list of instance models that meet the current gap or redundancy range and associated startup time parameters and unit time holding cost parameters according to the numerical range of the resource capacity gap or redundancy, and generating an instance specification-cost data set; Based on the instance specification-cost data set, the candidate instance models are arranged in ascending order according to the startup time parameter, and the instance models to be released are arranged in descending order according to the unit time holding cost parameter, and the priority weight ratio of the startup time to the holding cost is set to 3:1, and the top three expansion instance models with the shortest startup time and the top three reduction instance models with the highest holding cost are screened out to generate a priority sorting list; According to the priority sorting list, the number of instances for expansion is calculated by dividing the gap by the single instance capacity specification and rounding up, and the number of instances for reduction is calculated by dividing the redundancy by the minimum release unit of the reduction instance and rounding down, and a GPU resource pool adjustment decision including instance model, quantity and execution timing is generated.

5. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps for obtaining the task and GPU matching list are as follows: Traversing the GPU instance memory allocation records in the memory status evaluation result, calculating the number of continuous free memory blocks of each GPU instance, counting the coefficient of variation of the free block size, defining the ratio of the coefficient of variation to the total memory capacity of the instance as a fragmentation factor, and extracting the average delay time of page migration operations from the system log to generate an instance fragmentation-migration feature set; Based on the instance fragmentation-migration feature set, the matching degree between the computing task and the GPU instance; For each task, filter the GPU instances based on the matching degree, and sort them in descending order to generate a task and GPU matching degree list.

6. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps for obtaining the scheduling sequence of tasks to be executed are: Based on the task and GPU matching list, the scheduling priority of the task on the instance is calculated; Traverse all tasks, filter out instances with the highest scheduling priority and a state change time less than or equal to the maximum tolerable delay threshold of each task, and generate a scheduling sequence of tasks to be executed in descending order of scheduling priority.

7. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps for obtaining the task distribution execution result list are as follows: Parse each record in the to-be-executed task scheduling sequence, extract the task ID, GPU instance ID, pre-allocated video memory capacity and planned execution timestamp, verify the current video memory availability and online status of the GPU instance ID, and generate a task-instance binding relationship table with validity identification; Based on the task-instance binding relationship table, a video memory operation instruction message is constructed, and instructions are sent in batches according to the planned execution timestamp sequence through the resource manager's REST API to generate an instruction delivery log with a time sequence mark; Listen to the asynchronous response message of the resource manager, match the transaction ID in the instruction delivery log, parse the operation status code in the response message, retry the failed instruction up to 3 times according to the error type, and generate a task distribution execution result list.

8. The cloud environment GPU resource dynamic scheduling method according to claim 1, characterized in that: The steps for obtaining the GPU scheduling execution performance record are as follows: Parsing the successful records in the task distribution execution result list, extracting the task ID, GPU instance ID, and occupied video memory block address, sending a process start command to the target GPU instance, and generating a task process handle mapping table; Based on the task process handle mapping table, the video memory usage, computing power utilization and process survival time of the task are collected at intervals of 500 milliseconds, and the three are aligned by timestamp and merged to generate a resource consumption trajectory data set with time series marks; According to the resource consumption trajectory data set, the absolute deviation between the video memory peak and the pre-allocated capacity, and the relative deviation between the actual execution time and the planned time are calculated, and the deviation value is associated with the resource consumption trajectory and written into the database to generate a GPU scheduling execution performance record containing deviation indicators and original monitoring data.

Images