Method and system for releasing acceleration capability of physical GPU (Graphic Processing Unit) of cloud server

Abstract

The invention discloses a cloud server physical GPU acceleration capability release method and system, and belongs to the technical field of computer GPU capability release, and the method specifically comprises the steps: collecting GPU operation parameters in real time, carrying out the preprocessing, storing the GPU operation parameters in a shared state database, constructing a lightweight virtualization abstraction layer on a physical GPU, dividing the physical GPU into a plurality of logic units, and storing the logic units in a shared state database; the method comprises the following steps: establishing a physical GPU, constructing an isolation layer, utilizing a three-dimensional resource segmentation strategy, adopting a priority queue, a jump mechanism and a delay tolerance mechanism in a scheduling process, dynamically allocating logic unit resources of the physical GPU, accelerating release of GPU resources, and realizing unification and real-time updating of physical GPU state information among nodes of a cloud server through a distributed state synchronization mechanism; through dynamic task scheduling, a jumping mechanism, a delay tolerance mechanism and distributed state synchronization, efficient GPU resource management is realized, and GPU capability release is accelerated.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer GPU capacity release, and specifically to a method and system for releasing the physical GPU acceleration capacity of a cloud server. Background Art

[0002] In cloud computing, GPUs (graphics processing units) are widely used for high-performance computing tasks such as deep learning, scientific computing, and graphics rendering due to their powerful parallel computing capabilities. However, in distributed cloud server environments, GPU resource status is often managed independently, lacking global coordination and resulting in inefficient scheduling.

[0003] Existing technologies primarily rely on static partitioning or preset allocations for GPU resource management, which suffer from rigid scheduling and difficulty adapting to dynamic task demands. Therefore, implementing efficient dynamic allocation and recycling mechanisms for GPU resources to improve GPU utilization in cloud servers has become a pressing issue in the cloud computing field. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention proposes a method and system for releasing the physical GPU acceleration capabilities of cloud servers, which realizes efficient GPU resource management through dynamic task scheduling, jump mechanism, delay tolerance mechanism and distributed state synchronization.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] Methods for releasing the physical GPU acceleration capabilities of cloud servers include:

[0007] Collect GPU operating parameters in real time and store them in a shared state database;

[0008] Building a lightweight virtualization abstraction layer on the physical GPU, dividing the physical GPU into multiple logical units, and building an isolation layer between the logical units;

[0009] Based on the three-dimensional resource partitioning strategy, in the resource scheduling process, priority queues, skipping mechanisms and delay tolerance mechanisms are used to allocate resources to the logical units according to the task priority, resource request and scheduling requirements;

[0010] Through the distributed state synchronization mechanism, the physical GPU state information is synchronized and updated between each node of the cloud server.

[0011] Specifically, the method of constructing a lightweight virtualization abstraction layer on the physical GPU, dividing the physical GPU into multiple logical units, and constructing an isolation layer between the logical units includes:

[0012] Load virtualization services within the cloud server operating system, manage the mapping between physical GPUs and logical units, design a unified API interface, and inject virtualization context management logic;

[0013] Determine the partitioning strategy based on the performance parameters of the physical GPU;

[0014] Use container or micro-VM technology to create multiple independent operating environments on the physical GPU;

[0015] Each operating environment is a logical unit, configured with independent drivers, libraries, and scheduling interfaces to form an isolated sandbox.

[0016] Specifically, the three-dimensional resource partitioning strategy allocates resources to the logical unit in the resource scheduling process according to the priority, resource request and scheduling requirements of the task using a priority queue, a skip mechanism and a delay tolerance mechanism, including:

[0017] Use three-dimensional resource segmentation strategy to divide resources and standardize the divided resources;

[0018] Get task requests, calculate task priorities, and insert tasks into the priority queue according to their priorities;

[0019] The scheduler traverses the tasks in the priority queue in turn and searches for a logical unit in the resource pool that meets the three-dimensional resource requirements based on the task requirements;

[0020] For tasks that cannot fully meet the demand temporarily, a delay tolerance mechanism is enabled. When the resource matching changes, the position of each task in the queue is adjusted in time.

[0021] For tasks that cannot be matched with resources temporarily, the task status is adjusted to the waiting state and retained in the priority queue until resources are released or the scheduling policy is adjusted and then re-matched. If the task exceeds the preset delay tolerance time in the waiting state, re-evaluation is triggered.

[0022] Specifically, obtaining a task request, calculating the task priority, and inserting the task into the priority queue according to the priority includes:

[0023] Get task requests, add urgency and response deadline labels to each task, and format the tasks;

[0024] Perform a multi-dimensional priority assessment on the formatted tasks, including the urgency of the task, resource matching, and task deadline. Scores are assigned based on each dimension, and a comprehensive priority score is calculated.

[0025] Create a priority queue based on the total priority score of the tasks.

[0026] Specifically, the scheduler sequentially traverses the tasks in the priority queue and searches for a logical unit that meets the three-dimensional resource requirements in the resource pool according to the task requirements, including:

[0027] Traverse the tasks in the priority queue, read the task with the highest current priority, analyze the three-dimensional resource requirements of the task with the highest priority, and calculate the minimum resource threshold required for the task;

[0028] Traverse the currently available logical unit resource pool, preliminarily screen the logical units that meet the minimum requirements of the task, calculate the comprehensive matching score for each logical unit, and select the best matching object;

[0029] Based on the preliminary screening and matching, a resource allocation simulation is performed to verify whether the logical unit can maintain stability during task execution. If the current logical unit meets all requirements, the logical unit is allocated to the task. Otherwise, the jump mechanism is entered to reallocate the logical unit.

[0030] Specifically, the jump mechanism includes:

[0031] When a logic unit fails to meet task requirements, analyze the failure cause and determine the current resource competition environment;

[0032] According to the game model and the obtained Nash equilibrium, the optimal resource dimension is selected for jumping;

[0033] After jumping to the new resource dimension, the new logical unit is re-matched and verified to check whether it meets the task requirements. If so, the matching is completed; if not, the above steps are repeated until a matching logical unit is found or the maximum number of jumps is reached.

[0034] Specifically, for tasks that cannot fully meet the demand temporarily, the delay tolerance mechanism is enabled. When the resource matching changes, the position of each task in the queue is adjusted in time, including:

[0035] Set dynamic delay tolerance time windows based on task importance, system load, and service level agreements;

[0036] Within the delay tolerance period, the system continuously collects the status of each logical unit of the GPU and monitors resource release and task completion in real time.

[0037] Re-split or merge logical units to optimize resource allocation, periodically try to re-match tasks and resources, and immediately execute resource allocation when new resources are detected to be available or can meet task requirements after adjustment;

[0038] If a logical unit that meets the requirements cannot be matched within the tolerance time, a second evaluation is performed on the timed-out task to decide whether to extend the tolerance time or perform degradation processing.

[0039] Specifically, the three-dimensional resource segmentation strategy is used to divide resources, including dividing resources according to the dimensions of computing power, memory capacity, and data transmission or bandwidth.

[0040] A cloud server physical GPU acceleration capability release system, used to implement the cloud server physical GPU acceleration capability release method, including: a data processing module, a partitioning and isolation module, an acceleration resource release module, and a state synchronization module;

[0041] The data processing module is used to collect GPU operating parameters in real time, perform pre-processing, and store them in a shared state database;

[0042] The partitioning and isolation module is used to build a lightweight virtualization abstraction layer on the physical GPU, divide the physical GPU into multiple logical units, and build an isolation layer between the logical units;

[0043] The accelerated resource release module, based on the three-dimensional resource segmentation strategy, uses priority queues, skipping mechanisms and delay tolerance mechanisms to allocate resources to the logical units according to the task priority, resource request and scheduling requirements during the resource scheduling process;

[0044] The state synchronization module synchronizes and updates the physical GPU state information between each node of the cloud server through a distributed state synchronization mechanism.

[0045] Specifically, the accelerated resource release module includes: a resource segmentation unit, a logic matching unit and a secondary matching unit;

[0046] The resource segmentation unit divides resources using a three-dimensional resource segmentation strategy and standardizes the divided resources;

[0047] The logic matching unit is used to obtain task requests, calculate task priorities, and insert tasks into the priority queue according to their priorities;

[0048] The secondary matching unit enables a delay tolerance mechanism for tasks that cannot fully meet the requirements temporarily. For tasks that cannot match resources temporarily, the task status is adjusted to a waiting state, while being retained in the priority queue and re-matched after resources are released or the scheduling strategy is adjusted.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention proposes a method for releasing the physical GPU acceleration capabilities of a cloud server. By constructing a lightweight virtualization abstraction layer, the physical GPU is divided into multiple logical units, the allocation strategy is dynamically adjusted according to the task urgency and resource matching degree, a priority queue is established, and high-priority tasks obtain resources first. A waiting window is provided for non-urgent tasks, effectively alleviating the bottleneck of resource competition. In the scenario of resource matching failure, jump optimization is performed, and the task priority and status are dynamically adjusted according to the task evaluation results, thereby improving the success rate of resource matching. The real-time update of the GPU status between nodes is achieved through distributed state synchronization, ensuring the consistency and timeliness of global resource scheduling and improving the speed of GPU capacity release. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Flowchart of the method for releasing the physical GPU acceleration capability of a cloud server provided by the present invention;

[0052] Figure 2 The task matching flow chart provided by the present invention;

[0053] Figure 3 The GPU resource adjustment flow chart provided by the present invention;

[0054] Figure 4 A flow chart of the jump mechanism provided by the present invention;

[0055] Figure 5 This is a diagram of the system architecture for releasing the physical GPU acceleration capabilities of the cloud server provided by the present invention. DETAILED DESCRIPTION

[0056] The present application is described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but are not intended to limit the present application in any form. It should be noted that those skilled in the art may make several variations and improvements without departing from the scope of the present application. These all fall within the scope of protection of the present application.

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other and are all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flow chart. In addition, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish between the same items or similar items with basically the same functions and effects.

[0059] Unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification and in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the relevant listed items.

[0060] Example 1

[0061] See also Figure 1-Figure 3 The present invention provides an embodiment of a method for releasing the physical GPU acceleration capability of a cloud server, comprising the following specific steps:

[0062] Step S1: Collect GPU operating parameters in real time, perform preprocessing, and store them in a shared state database.

[0063] The GPU operating parameters include: GPU load, temperature, memory usage, power consumption, queue length, etc., to ensure that the cloud server can grasp the health status and usage of each GPU unit in real time. Data collection relies on the interface provided by the GPU driver to collect various GPU indicators with low latency and high frequency. The sampling period is set according to the application scenario and load conditions to ensure the timeliness of monitoring data. During high-load task operation, the sampling frequency is adjusted in time to capture more details; in the idle state, a lower sampling rate is used to reduce system overhead.

[0064] It should be noted that the preprocessing adopts existing technologies, including normalization, filtering, anomaly detection and other preprocessing operations on the collected GPU operating parameters. Data normalization and format conversion: the collected GPU operating parameters are processed in a unified unit; data storage and sharing: the preprocessed GPU operating parameters are stored in a dedicated shared status database; data warning and anomaly processing: when the collected GPU operating parameters exceed the preset threshold (such as abnormal temperature, excessive load), the alarm mechanism is immediately triggered. I will not go into details here.

[0065] Step S2: Build a lightweight virtualization abstraction layer on the physical GPU, divide the physical GPU into multiple logical units, and build an isolation layer.

[0066] The specific steps of step S2 are:

[0067] Step S201: Load the virtualization service in the cloud server operating system, manage the mapping relationship between the physical GPU and the logical unit, design a unified API interface, and inject the virtualization context management logic.

[0068] In this embodiment, physical GPU resources are mapped into multiple logical units with independent operating environments. Each logical unit achieves independence in resource, driver, and task scheduling through a context isolation mechanism. In this process, a one-to-many mapping relationship table is established between the physical GPU and the logical units, and is encapsulated and managed with a unified data structure. At the same time, a standardized API interface is designed to provide operational instructions such as task submission, resource allocation, status query, and context switching, ensuring that the scheduler or upper-level container platform can communicate seamlessly with the GPU virtualization layer, and injecting virtualization context management logic, which can realize the state preservation, recovery, switching, and rollback of tasks between multiple logical units.

[0069] Virtualization context management logic refers to the logical mechanism used to manage and maintain context data such as the operating status, resource usage information, and execution environment configuration of each logical execution unit or virtual GPU when executing tasks in a GPU virtualization environment. The API interface includes a resource application interface and a resource release interface. The resource application interface returns the identifier of an available logical unit; the resource release interface recycles the logical unit resources to the resource pool.

[0070] Step S202: Determine a partitioning strategy based on the performance parameters of the physical GPU.

[0071] In this embodiment, a partitioning strategy is selected based on GPU performance parameters, including: a level partitioning strategy, a weight partitioning strategy, a dynamic scaling strategy, and a heterogeneous adaptation strategy.

[0072] Step S203: Create multiple independent operating environments on the physical GPU through container or micro-virtual machine technology.

[0073] Step S204: Each operating environment is a logical unit, configured with independent drivers, libraries and scheduling interfaces to form an isolated sandbox.

[0074] Step S3: Using the three-dimensional resource partitioning strategy, during the scheduling process, a priority queue, a jump mechanism, and a delay tolerance mechanism are employed to dynamically allocate the logical unit resources of the physical GPU and accelerate the release of GPU resources.

[0075] like Figure 2 As shown, the specific steps of step S3 are:

[0076] Step S301: using a three-dimensional resource segmentation strategy to divide resources and standardize the divided resources;

[0077] In this embodiment, the three-dimensional resource segmentation strategy in step S301 includes: resource segmentation according to the computing power dimension, the video memory capacity dimension, and the data transmission / bandwidth dimension; in the computing power dimension, resource segmentation is performed based on indicators such as the number of GPU cores or stream processors and clock frequency; in the video memory capacity dimension, resources are subdivided into multiple sub-video memory areas based on the total video memory amount and bandwidth requirements; in the data transmission / bandwidth dimension, resources are segmented based on the GPU internal data transmission rate and I / O interface bandwidth;

[0078] Resource standardization: normalizes the metrics of different GPU models and GPUs produced by different manufacturers in these three dimensions to form a unified resource description model, facilitating comparison and matching during scheduling.

[0079] Step S302: Obtain a task request, calculate the task priority, and insert the task into the priority queue according to the priority;

[0080] The specific steps of step S302 are:

[0081] Step S3021: Obtain task requests, add urgency and response deadline tags to each task, and format the tasks;

[0082] In this embodiment, the original task request is converted into a standardized structure that can be recognized by the scheduler. Specifically, when a user or an upper-level service initiates a task request, the basic information of the task is first parsed, including resource requirements, task type, and context parameters. Then, two key tags are automatically injected into each task: an urgency tag and a response deadline tag. The urgency tag reflects the priority level of task processing; the response deadline tag indicates how long the task must be completed to constrain the scheduling time. All task information is structured and converted into a standard task format, which includes fields such as task identifier, resource requirement description, tag parameters, and status mark.

[0083] Step S3022: Perform a multi-dimensional priority evaluation on the formatted task, wherein the multi-dimensional evaluation includes: task urgency, resource matching, and task deadline, and score each dimension to calculate a total priority score.

[0084] Specifically, the urgency of the task is the core indicator, and high-urgency tasks are given higher priority; resource matching: evaluates the degree of matching between the task's demand for three-dimensional resources and the current available resource pool. If the resource demand just happens to or exceeds the standard configuration of the current resource pool, the priority needs to be lowered or enter a waiting state; task deadline: the latest completion time of the task is included in the priority calculation, and tasks closer to the deadline will receive higher priority.

[0085] Scoring criteria: Urgency, with a predefined score (e.g., high: 10, medium: 5, low: 2) directly affecting the basic priority of the task; Resource demand matching, which calculates the matching score between the demand and the free resources in the current resource pool; if the match is good, points are added; if the match is not good, points are subtracted; Deadline: Based on the remaining time until the task deadline, a certain amount of time urgency is given as bonus points; Finally, the scores of each task are combined to calculate a total priority score, which is used to sort the task in the queue.

[0086] Step S3023: Create a priority queue based on the total priority score of the tasks.

[0087] In this embodiment, a dynamically adjustable data structure is used to construct and maintain the priority queue, such as a max heap, a balanced binary tree, or a skip list, to ensure efficient insertion, search, and update performance in scenarios with high task concurrency and frequent queuing and dequeuing.

[0088] Step S303: The scheduler sequentially traverses the tasks in the priority queue and searches for a logical unit that meets the three-dimensional resource requirements in the resource pool according to the task requirements;

[0089] The specific steps of step S303 are:

[0090] Step S3031: traverse the tasks in the priority queue, read the task with the highest current priority, analyze the three-dimensional resource requirements of the task with the highest priority, and calculate the minimum resource threshold required by the task;

[0091] In this embodiment, the three-dimensional resource requirements of the parsing task include: computing power requirements, video memory requirements, and data transmission requirements; the minimum resource threshold required for the calculation task is to ensure that resource allocation does not affect task execution efficiency;

[0092] Step S3032: traverse the currently available logical unit resource pool, preliminarily screen the logical units that meet the minimum requirements of the task, calculate the comprehensive matching score for each logical unit, and select the best matching object;

[0093] In this embodiment, the scheduler traverses the currently available logical unit resource pool and preliminarily screens logical units that meet the minimum requirements of the task. For each logical unit, the scheduler calculates its resource utilization and makes an estimate, including whether it is sufficient to meet the current task, whether allocation will lead to resource waste or fragmentation, and whether there are other higher-priority tasks waiting for suitable logical units. For multiple feasible logical units, the scheduler calculates their comprehensive matching scores (such as priority and load balancing) and selects the best matching object.

[0094] Step S3033: Based on the preliminary screening and matching, a resource allocation simulation is performed to verify whether the logic unit can maintain stability during task execution. If the current logic unit meets all requirements, the logic unit is allocated to the task; otherwise, a jump mechanism is entered to reallocate the logic unit.

[0095] In this embodiment, based on the preliminary matching, a resource allocation simulation is performed to verify whether the logic unit can maintain stability during task execution, including: dynamically monitoring whether resources are sufficient, whether the load fluctuations of other tasks affect the current task, and whether there are resources that are about to be released and can be used;

[0096] like Figure 4 As shown, the jump mechanism in step S3033 includes:

[0097] 1) When a logic unit fails to meet task requirements, analyze the failure cause and determine the current resource competition environment;

[0098] In this embodiment, the core goals of the jump mechanism are to quickly find suitable resources, improve resource allocation efficiency, reduce fragmentation problems, and improve overall GPU resource utilization.

[0099] Reasons for the first match failure include: insufficient computing resources → jump to the next logical unit with stronger computing power; severe video memory fragmentation → jump to a logical unit with complete video memory space; insufficient bandwidth → jump to a logical unit with greater data transmission capacity;

[0100] 2) Based on the game model and the obtained Nash equilibrium, select the optimal resource dimension for jumping;

[0101] Using game theory methods, we can solve the Nash equilibrium of the game matrix. A Nash equilibrium is a state where, given the fixed strategies of other participants, no participant can unilaterally change their strategy to increase their profit. In a resource-competitive environment, the strategy combination corresponding to the Nash equilibrium is the optimal resource acquisition strategy for each logical unit. For example, if competition in the computing power dimension is fierce, but there are more available resources in the storage capacity dimension, we can choose to jump to the storage capacity dimension.

[0102] 3) After jumping to the new resource dimension, the new logical unit is re-matched and verified to check whether it meets the task requirements. If so, the matching is completed; if not, the above steps are repeated until a matching logical unit is found or the maximum number of jumps is reached.

[0103] In this embodiment, if the maximum number of hops is reached, a backup strategy can be adopted, including: resource scheduling optimization, such as dynamic priority adjustment; physical layer optimization, such as hardware accelerated redirection, near-field storage access, etc.; task splitting and reorganization, elastic scaling, etc.;

[0104] The application of the logic unit matching and jumping mechanism solves the problem of inefficient matching caused by resource fragmentation or load imbalance during GPU resource scheduling. Through intelligent matching strategies, dimensional jumping, historical record analysis and machine learning optimization, this mechanism can quickly locate the optimal GPU logic unit and reduce task waiting time; avoid inefficient allocation and resource fragmentation, and improve overall GPU resource utilization; adapt to various scheduling scenarios, and ensure that tasks can obtain stable computing resource support under high load conditions; it is particularly suitable for cloud computing, multi-tenant GPU sharing and large-scale AI task scheduling scenarios.

[0105] The advantage is: by solving the Nash equilibrium point of the game model, the global optimality of the resource allocation strategy in a competitive environment is ensured, avoiding the local optimal trap of the traditional greedy algorithm.

[0106] Step S304: For some high-priority tasks whose needs cannot be fully met by the current resource pool, a delay tolerance mechanism is enabled to ensure that the high-priority tasks are always at the front of the queue. When the resource matching changes, the position of each task in the queue can be adjusted in time.

[0107] The specific steps of step S304 include:

[0108] Step S3041: Setting a dynamic delay tolerance time window based on task importance, system load, and service level agreement;

[0109] In this embodiment, after receiving a task to be scheduled, the first step is to evaluate the importance level of the task, which is determined by the business type, urgency tag, etc.; secondly, the overall GPU resource load is monitored in real time, including the current occupancy rate, task congestion, and logic unit availability, to determine whether it is in a high-pressure operation stage; combined with the latest response time or maximum delay upper limit of the task, the delay tolerance time window is dynamically calculated through a rule model or a weighted decision model.

[0110] Step S3042: within the delay tolerance time, continuously collect the status of each logic unit of the GPU, and monitor the resource release and task execution completion in real time.

[0111] Step S3043: Re-split or merge the logical units, optimize resource allocation, and periodically try to re-match tasks and resources. When it is detected that new resources are available or can meet task requirements after adjustment, resource allocation is immediately performed.

[0112] In this embodiment, when a task is within the delay tolerance time window and cannot be matched with suitable resources, the resource utilization, idle fragments and task waiting queue status of the current logical unit are periodically evaluated to determine whether there is a problem of uneven resource allocation or resource fragmentation. If it is detected that some logical units are in a low-load state, some of their resources are released and merged with other idle units to form a new resource block; conversely, if a logical unit has a high load but the resource allocation is too coarse-grained, it is divided more finely to generate multiple smaller logical units to adapt to lightweight tasks.

[0113] Step S3044: If a logical unit that meets the requirements cannot be matched within the tolerance time, a secondary evaluation is performed on the timed-out task to determine whether to extend the tolerance time or perform degradation processing.

[0114] In this embodiment, when a logical unit that meets the requirements cannot be matched, the task is notified to enter a further waiting state, and the task priority is appropriately lowered within the range allowed by the system; or a backup scheduling strategy is triggered, such as partial resource downgrade allocation or cross-node resource scheduling, to ensure that the task is not completely shelved.

[0115] The benefits of the delay tolerance mechanism are: improving resource utilization by allowing tasks to wait and reallocate resources in a short period of time, avoiding resource waste caused by instantaneous resource shortages and making full use of GPU resources dynamically released by the system; improving task response time. Under high load conditions, the delay tolerance mechanism avoids direct task rejection and ensures that high-priority tasks can be allocated as soon as possible after resources are released, thereby improving overall response time.

[0116] Step S305: For tasks that cannot be matched with resources temporarily, the task status is adjusted to the waiting state, and it is retained in the priority queue until the resources are released or the scheduling strategy is adjusted and then re-matched. If the task exceeds the preset delay tolerance time in the waiting state, a re-evaluation is triggered, and even its priority is adjusted to ensure that the task does not lose service quality due to waiting too long.

[0117] In this embodiment, when a task fails to find a logical unit that meets its three-dimensional resource requirements at the current moment, the task will not be discarded or rejected immediately. Instead, its status will be set to waiting and it will continue to remain in the priority queue. At the same time, the timestamp of entering the waiting state will be recorded. In this state, the task still participates in scheduling and sorting, but needs to wait for resource release or configuration optimization. The waiting time of the task and the system load changes are continuously monitored, and the preset delay tolerance time is used as the constraint threshold. When the task waiting time exceeds the threshold, the re-evaluation logic is triggered, and the current resource status, the original priority of the task, the timeliness of the task, etc. are comprehensively considered to determine whether its priority score needs to be adjusted, or whether the task needs to be downgraded, split or migrated.

[0118] The overall benefits of the steps are: efficient use of GPU resources, dynamic allocation ensures that each task can quickly obtain the logical unit that meets the needs, and timely release ensures resource recycling, thereby greatly improving overall resource utilization; real-time response and flexible scheduling, through dynamic monitoring and real-time feedback, so that task scheduling not only responds quickly, but also can automatically adjust the allocation strategy according to the load situation, adapting to multi-tenant and load peak scenarios; reducing system latency and resource waste, timely allocation and release effectively reduce task waiting time and resource idle time, optimize system performance, and improve user experience; improve the robustness of the scheduling system: real-time monitoring and anomaly detection mechanisms enable the system to take quick measures in the face of emergencies to ensure scheduling continuity and system stability.

[0119] like Figure 3 As shown in the figure, the scheduler is responsible for unified management and coordination of resource allocation, task scheduling, etc., receives adjustment suggestions given by the prediction model, and ultimately decides how to update or allocate system resources; resource monitoring monitors the usage of various system resources in real time to form usable status information; the prediction model uses historical data and algorithms to analyze or predict the real-time data input by resource monitoring, and outputs adjustment suggestions for the current system load or resource status; the execution unit performs corresponding operations according to the instructions issued by the prediction model or the scheduler.

[0120] exist Figure 3The specific flow process is as follows: Requesting three-dimensional status (scheduler → resource monitoring). The scheduler needs to understand the current three-dimensional resource status of the system (such as GPU computing power, video memory, bandwidth, etc.) or other important indicators, so it sends a request to the resource monitoring; Inputting the current status (resource monitoring → prediction model). The resource monitoring transmits the collected real-time status information (such as load, temperature, utilization rate, etc.) to the prediction model for analysis or prediction; Outputting adjustment suggestions (prediction model → scheduler). The prediction model provides optimization or adjustment suggestions after integrating the current status and historical data; Updating the allocation strategy (scheduler → resource monitoring). The scheduler combines the prediction model's suggestions and its own scheduling strategy (including priority rules, service level agreements, etc.) to determine a new resource allocation or scheduling plan, and issues the new allocation strategy or scheduling command to the resource monitoring. The resource monitoring executes or records this update and provides the latest resource availability information. The resource monitoring and prediction model continuously interact. The resource monitoring continuously updates the prediction model with the latest resource usage. After obtaining new data, the prediction model again outputs new scheduling suggestions, forming a closed loop. Finally, the execution unit completes the actual computing task or resource change operation according to the above scheduling instructions.

[0121] Step S4: Through the distributed state synchronization mechanism, the physical GPU state information is unified and updated in real time among the nodes of the cloud server.

[0122] The specific steps of the distributed state synchronization mechanism are:

[0123] Step S401: Each cloud server node deploys a monitoring agent, uses the GPU driver interface to collect the operating parameters of the local GPU in real time, and pre-processes the collected raw data to form standardized status information.

[0124] Ensure that the status data format provided by each node is consistent to facilitate subsequent cross-node data fusion.

[0125] Step S402: Each node stores the processed GPU state data in a local cache or lightweight database, and pushes updates to the global state pool, defining a data structure containing key information such as GPU identification, current state, and sampling time;

[0126] Step S403: using a distributed message queue to push the status information of each node to the global status management service in real time;

[0127] Specifically, each physical node or virtualization unit running a GPU task will periodically generate a local status snapshot, including key operating parameters such as resource utilization, task execution status, load changes, and fault warnings. It uses a distributed message queue as a data transfer channel, asynchronously writes node status data into the queue as a message body, and allows the global status management service to pull updates by subscription.

[0128] Step S404: To address the inconsistency problem of cross-node data, a distributed consistency protocol or a timestamp-based conflict resolution mechanism is used to perform consistency verification on the state information.

[0129] Step S405: The real-time updated status information is immediately fed back to each node scheduler through a push notification mechanism. At the same time, synchronization delay and data consistency issues are monitored and feedback optimization is performed.

[0130] The benefits of these steps are: ensuring that scheduling decisions are based on the latest and accurate data, improving resource allocation efficiency, continuously optimizing the synchronization mechanism, and reducing task delays and load imbalances caused by state lags.

[0131] Example 2

[0132] See also Figure 5 , another embodiment provided by the present invention: a cloud server physical GPU acceleration capacity release system, comprising: a data processing module, a partitioning and isolation module, an acceleration resource release module and a state synchronization module;

[0133] The data processing module is used to collect GPU operating parameters in real time, perform pre-processing, and store them in a shared state database;

[0134] The partitioning and isolation module is used to build a lightweight virtualization abstraction layer on the physical GPU, divide the physical GPU into multiple logical units, and build an isolation layer;

[0135] The accelerated resource release module uses a three-dimensional resource segmentation strategy and adopts a priority queue, a jump mechanism, and a delay tolerance mechanism during the scheduling process to dynamically allocate logical unit resources of the physical GPU and accelerate the release of GPU resources;

[0136] The state synchronization module realizes the unification and real-time update of physical GPU state information among various nodes of the cloud server through a distributed state synchronization mechanism.

[0137] Accelerated resource release module, including: resource segmentation unit, logic matching unit and secondary matching unit;

[0138] The resource segmentation unit divides resources using a three-dimensional resource segmentation strategy and standardizes the divided resources;

[0139] The logic matching unit is used to obtain task requests, calculate task priorities, and insert tasks into the priority queue according to their priorities;

[0140] The secondary matching unit enables a delay tolerance mechanism for tasks that cannot fully meet the requirements temporarily. For tasks that cannot match resources temporarily, the task status is adjusted to a waiting state, while being retained in the priority queue and re-matched after resources are released or the scheduling strategy is adjusted.

[0141] In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0142] The above-described specific embodiments further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for releasing the physical GPU acceleration capability of a cloud server, characterized in that: include: Collect GPU operating parameters in real time and store them in a shared state database; Building a lightweight virtualization abstraction layer on the physical GPU, dividing the physical GPU into multiple logical units, and building an isolation layer between the logical units; Based on the three-dimensional resource partitioning strategy, in the resource scheduling process, priority queues, skipping mechanisms and delay tolerance mechanisms are used to allocate resources to the logical units according to the task priority, resource request and scheduling requirements; Through the distributed state synchronization mechanism, the physical GPU state information is synchronized and updated between each node of the cloud server.

2. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 1, wherein: The method of constructing a lightweight virtualization abstraction layer on the physical GPU, dividing the physical GPU into multiple logical units, and constructing an isolation layer between the logical units includes: Load virtualization services within the cloud server operating system, manage the mapping between physical GPUs and logical units, design a unified API interface, and inject virtualization context management logic; Determine the partitioning strategy based on the performance parameters of the physical GPU; Use container or micro-VM technology to create multiple independent operating environments on the physical GPU; Each operating environment is a logical unit, configured with independent drivers, libraries, and scheduling interfaces to form an isolated sandbox.

3. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 1, wherein: The three-dimensional resource partitioning strategy allocates resources to the logical units in the resource scheduling process according to the priority, resource request and scheduling requirements of the tasks using a priority queue, a skipping mechanism and a delay tolerance mechanism, including: Use three-dimensional resource segmentation strategy to divide resources and standardize the divided resources; Get task requests, calculate task priorities, and insert tasks into the priority queue according to their priorities; The scheduler traverses the tasks in the priority queue in turn and searches for a logical unit in the resource pool that meets the three-dimensional resource requirements based on the task requirements; For tasks that cannot fully meet the demand temporarily, a delay tolerance mechanism is enabled. When the resource matching changes, the position of each task in the queue is adjusted in time. For tasks that cannot be matched with resources temporarily, the task status is adjusted to the waiting state and retained in the priority queue until resources are released or the scheduling policy is adjusted and then re-matched. If the task exceeds the preset delay tolerance time in the waiting state, re-evaluation is triggered.

4. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 3, wherein: The process of obtaining a task request, calculating the task priority, and inserting the task into the priority queue according to the priority includes: Get task requests, add urgency and response deadline labels to each task, and format the tasks; Perform a multi-dimensional priority assessment on the formatted tasks, including the urgency of the task, resource matching, and task deadline. Scores are assigned based on each dimension, and a comprehensive priority score is calculated. Create a priority queue based on the total priority score of the tasks.

5. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 3, wherein: The scheduler sequentially traverses the tasks in the priority queue and searches for a logical unit in the resource pool that meets the three-dimensional resource requirements according to the task requirements, including: Traverse the tasks in the priority queue, read the task with the highest current priority, analyze the three-dimensional resource requirements of the task with the highest priority, and calculate the minimum resource threshold required for the task; Traverse the currently available logical unit resource pool, preliminarily screen the logical units that meet the minimum requirements of the task, calculate the comprehensive matching score for each logical unit, and select the best matching object; Based on the preliminary screening and matching, a resource allocation simulation is performed to verify whether the logical unit can maintain stability during task execution. If the current logical unit meets all requirements, the logical unit is allocated to the task. Otherwise, the jump mechanism is entered to reallocate the logical unit.

6. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 5, wherein: The jumping mechanism includes: When a logic unit fails to meet task requirements, analyze the failure cause and determine the current resource competition environment; According to the game model and the obtained Nash equilibrium, the optimal resource dimension is selected for jumping; After jumping to the new resource dimension, the new logical unit is re-matched and verified to check whether it meets the task requirements. If so, the matching is completed; if not, the above steps are repeated until a matching logical unit is found or the maximum number of jumps is reached.

7. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 3, wherein: For tasks that cannot fully meet the demand temporarily, the delay tolerance mechanism is enabled. When the resource matching changes, the position of each task in the queue is adjusted in time, including: Set dynamic delay tolerance time windows based on task importance, system load, and service level agreements; Within the delay tolerance period, the system continuously collects the status of each logical unit of the GPU and monitors resource release and task completion in real time. Re-split or merge logical units to optimize resource allocation, periodically try to re-match tasks and resources, and immediately execute resource allocation when new resources are detected to be available or can meet task requirements after adjustment; If a logical unit that meets the requirements cannot be matched within the tolerance time, a second evaluation is performed on the timed-out task to decide whether to extend the tolerance time or perform degradation processing.

8. The method for releasing the physical GPU acceleration capability of a cloud server according to claim 3, wherein: The three-dimensional resource segmentation strategy is used to divide resources, including dividing resources according to the dimensions of computing power, memory capacity, and data transmission or bandwidth.

9. A cloud server physical GPU acceleration capability release system, configured to implement the cloud server physical GPU acceleration capability release method according to any one of claims 1 to 8, characterized in that: include: Data processing module, partitioning and isolation module, accelerated resource release module and state synchronization module; The data processing module is used to collect GPU operating parameters in real time, perform pre-processing, and store them in a shared state database; The partitioning and isolation module is used to build a lightweight virtualization abstraction layer on the physical GPU, divide the physical GPU into multiple logical units, and build an isolation layer between the logical units; The accelerated resource release module, based on the three-dimensional resource segmentation strategy, uses priority queues, skipping mechanisms and delay tolerance mechanisms to allocate resources to the logical units according to the task priority, resource request and scheduling requirements during the resource scheduling process; The state synchronization module synchronizes and updates the physical GPU state information between each node of the cloud server through a distributed state synchronization mechanism.

10. The cloud server physical GPU acceleration capability release system according to claim 9, characterized in that: The accelerated resource release module includes: a resource segmentation unit, a logic matching unit and a secondary matching unit; The resource segmentation unit divides resources using a three-dimensional resource segmentation strategy and standardizes the divided resources; The logic matching unit is used to obtain task requests, calculate task priorities, and insert tasks into the priority queue according to their priorities; The secondary matching unit enables a delay tolerance mechanism for tasks that cannot fully meet the requirements temporarily. For tasks that cannot match resources temporarily, the task status is adjusted to a waiting state, while being retained in the priority queue and re-matched after resources are released or the scheduling strategy is adjusted.

Images