Gpu virtualization system based on cuda cross-level translation and multi-pooling scheduling

Abstract

The application discloses a CUDA cross-level translation and multi-pooling scheduling GPU virtualization system, relates to the technical field of GPU virtualization system, and comprises an instruction translation module, a resource monitoring module, a scheduling decision module and a dynamic load balancing module; the instruction translation module is used for intercepting CUDA calling instructions of a GPU instance, and performing real-time translation according to the difference between the required CUDA version and the physical GPU driver version of an application. Through CUDA cross-level translation and multi-pooling scheduling, the application significantly improves the performance and flexibility of the GPU virtualization system, realizes real-time compatible translation of cross-version CUDA instructions, shortens the game starting time, and simultaneously ensures efficient resource scheduling under burst load through a dynamic load balancing mechanism, avoids user queuing and game lag, in addition, a priority sorting mechanism improves resource utilization and reduces operation cost, and a QoS feedback module further improves user experience through real-time monitoring and optimization, and ensures the continuous stability and efficiency of the service.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of GPU virtualization systems, in particular to a GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling. BACKGROUND

[0002] With the rapid development of cloud computing and virtualization technology, GPU virtualization technology has become a key technology for efficient sharing of expensive hardware resources and improving service flexibility. Traditional GPU virtualization schemes (such as pass-through, API forwarding, and full virtualization) have achieved certain results in general cloud computing environments, but when applied to specific high-demand scenarios, their limitations are increasingly prominent, making it difficult to meet the stringent performance and service experience requirements.

[0003] Specifically, in a cloud game platform that provides services to a large number of users, especially when handling the burst GPU load caused by user-initiated real-time game session requests and game scene switching, existing GPU virtualization technologies face serious challenges:

[0004] 1. Compatibility and startup delay issues: Cloud game platforms need to support a large number of games developed using different versions of CUDA libraries in different years. Traditional virtualization schemes often compromise when trying to support old CUDA applications (requiring old drivers) and new games (requiring new drivers), or rely on inefficient software emulation layers, resulting in long game session startup times and poor user experience. Users expect to enter a smooth picture within a few seconds after clicking "Start Game", but existing solutions often have driver switching or translation overhead as a bottleneck when handling applications with a large span of CUDA versions.

[0005] 2. Resource scheduling rigidity under burst load: Cloud gaming has a significant instantaneous peak load characteristic. For example, during the launch of a popular new game, a promotion activity at a whole hour, or a live broadcast of a major e-sports event, a large number of users may initiate game requests at the same time; complex scene switching within the game can also instantaneously increase GPU power demand. Traditional GPU virtualization scheduling mechanisms based on static allocation or simple time slice polling are slow to react, with clear barriers between resource pools, and cannot dynamically aggregate and allocate scattered GPU fragment resources within milliseconds to respond to burst demand, resulting in queuing of user requests, game lag, and even service denial during peak periods.

[0006] 3. Resource fragmentation and utilization bottleneck: In order to meet the differentiated needs of different games for GPU models and computing power, cloud platforms usually deploy multiple models of GPU hardware to form a heterogeneous resource pool. The existing virtualization scheme lacks efficient cross-physical GPU, cross-resource pool fine-grained resource integration and scheduling capability, which easily causes idle and fragmentation of computing power resources (especially high-end GPU computing power), and is difficult to maximize the overall hardware utilization rate while guaranteeing the user SLA (Service Level Agreement), significantly increasing the operating cost of the platform.

[0007] Therefore, in the application scenarios of real-time session startup and burst load response of cloud game platforms, a new GPU virtualization technology is needed to solve the CUDA cross-version compatibility problem, achieve millisecond-level elastic resource scheduling, and effectively improve the utilization rate of heterogeneous GPU resource pool. The performance bottleneck and low efficiency of the existing general GPU virtualization scheme in such scenarios have become the main obstacle to the improvement of cloud game service quality and cost optimization. SUMMARY

[0008] The purpose of the present application is to provide a GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling to solve the problems raised in the background art.

[0009] To achieve the above purpose, the present application provides a GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling, which includes an instruction translation module for intercepting CUDA call instructions of a GPU instance and performing real-time translation according to the difference between the required CUDA version of an application and the physical GPU driver version;

[0010] A resource monitoring module is used to collect load data of a physical GPU resource pool in real time, and the load data includes memory occupancy, GPU core utilization and data bandwidth throughput;

[0011] A scheduling decision module is used to receive data from the resource monitoring module and make resource allocation decisions. The scheduling decision module includes:

[0012] A cross-level translation overhead value is calculated, which takes into account the physical GPU driver version number, the required CUDA version number of the application, the instruction complexity coefficient, the current instruction number and the reference instruction number threshold;

[0013] A preset translation overhead threshold is compared with the cross-level translation overhead value. If the threshold is exceeded, the instruction translation module starts a multi-version binary rewriting mechanism;

[0014] A resource load pressure value is calculated, which takes into account the required bandwidth, memory occupancy, GPU core utilization, data bandwidth throughput and corresponding weight coefficients;

[0015] A preset load threshold is compared with the resource load pressure value, and if the threshold is exceeded, a scheduling instruction is triggered to the dynamic load balancing module;

[0016] A dynamic load balancing module is configured to schedule tasks to heterogeneous nodes across physical GPUs and resource pools according to the scheduling instruction of the scheduling decision module.

[0017] Compared with the prior art, the present application has the following advantages:

[0018] 1. The present application executes a multi-version binary rewriting mechanism according to the threshold of the scheduling decision module, realizes real-time compatible translation of cross-version CUDA instructions, shortens the startup time of old games in a new driver environment, and thus completely eliminates the compatibility bottleneck of high game startup delay mentioned in the background technology, and meets the user's second-level startup demand in the cloud game platform.

[0019] 2. The present application schedules tasks to heterogeneous nodes across physical GPUs and resource pools according to the scheduling instruction of the scheduling decision module through the dynamic load balancing module, so that the system can complete the task scheduling across the GPU resource pool in a very short time during the peak period of burst traffic, and ensure the stability of the game frame rate.

[0020] 3. The present application can reduce the idle rate of high-end GPUs by executing the priority sorting mechanism of the scheduling decision module, thereby improving the comprehensive utilization rate of heterogeneous resource pools, achieving the purpose of integrating fragmented resources, breaking through the utilization bottleneck of resource fragmentation and high-end GPU idling mentioned in the background technology, and thus reducing the operating cost of the cloud game platform.

[0021] 4. The present application dynamically optimizes the task scheduling path and translation strategy by monitoring the frame rate and delay in real time through the QoS feedback module, automatically triggers scheduling strategy optimization and translation path adjustment when the quality of experience declines, realizes dynamic difficulty balance, so that users of different levels can obtain adaptive training intensity, thereby avoiding the experience imbalance problem caused by static difficulty and user frustration, and improving the user retention rate in the cloud game platform. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 It is a whole module deployment schematic diagram of the GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] In a specific embodiment, the present embodiment takes the online of popular games in a cloud game platform as an application scenario, the game is developed using CUDA 9.2, and the platform GPU resource pool is deployed with multiple architecture hardware (A100 / V100 / T4), the driver version is CUDA 12.0, and a GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling is provided, and the overall module deployment of the system is as shown in Figure 1

[0025] Instruction translation module: deployed in the GPU virtualization layer, used to intercept the CUDA call instructions of the GPU instance, and perform real-time translation according to the difference between the required CUDA version of the application and the physical GPU driver version. Through real-time translation, cross-version CUDA instruction compatibility is achieved, significantly shortening the startup time of old games in a new driver environment, meeting the user's second-level startup demand in the cloud game platform, and improving the user experience.

[0026] Resource monitoring module: used to collect the load data of the physical GPU resource pool in real time, the load data including the video memory occupancy rate , GPU core utilization rate and data bandwidth throughput , sampling frequency 100ms / time. Real-time data collection provides accurate basis for scheduling decision, ensures reasonable resource allocation, avoids resource waste and overload, and improves the overall stability and response speed of the system.

[0027] Scheduling decision module: running in the central control node, used to receive the data of the resource monitoring module and make resource allocation decisions. Through intelligent decision-making, the resource allocation is dynamically adjusted according to the real-time load condition, ensuring that the system can still maintain stable performance under high load, improving resource utilization efficiency and system flexibility.

[0028] Dynamic load balancing module: used to schedule tasks to heterogeneous nodes across physical GPUs and resource pools according to the scheduling instructions of the scheduling decision module. Realize the scheduling of heterogeneous nodes across physical GPUs and resource pools, ensure that the system can respond quickly during peak burst traffic, balance the load, avoid user queuing and game lag, and improve user satisfaction and system service quality.

[0029] QoS feedback module: used to collect the frame rate and delay of user sessions in real time, and calculate the service quality score according to the formula, dynamically adjust the scheduling strategy and optimize the translation path. Through real-time monitoring of user session quality, dynamically adjusting the scheduling strategy and translation path, ensuring the continuous stability of service quality, improving user retention rate and platform competitiveness.

[0030] ​The following is described in conjunction with a detailed embodiment, which is applied to the application scenario of popular games going online in a cloud game platform. When the game is opened on the first day, 50,000 user requests are instantly poured in. The actual working process and core control logic of the system are as follows:

[0031] ①First, the instruction translation module intercepts the CUDA call instruction stream and parses the parameters to obtain the required CUDA version number of the application , the physical GPU driver version number and the current instruction number . At the same time, the scheduling decision module is used to calculate the cross-level translation overhead value according to the formula . This formula is used to quantify the overhead of cross-version CUDA instruction translation by considering factors such as driver version difference, instruction complexity and instruction quantity to determine whether to start the multi-version binary rewriting mechanism. This helps the system to make appropriate decisions when facing different translation needs to improve translation efficiency and performance. is the instruction complexity coefficient, defined as the quantitative translation difficulty of a single CUDA instruction. The larger the value, the higher the translation overhead. Based on instruction type classification: basic API instructions (such as memory allocation, data copy): =1.0 (baseline value); compute-intensive instructions (such as kernel function calls): =1.5-2.0 (opcode reconstruction required); version difference sensitive instructions (such as old version exclusive API): =2.5+ (deep binary rewriting required). At the same time, if binary rewriting is enabled and verification fails (sandbox running timeout / error), the value of the instruction is increased (such as +0.3), and the software simulation layer is triggered. is the reference instruction number threshold, defined as the maximum number of sub-instructions allowed for a single API call. If it exceeds the limit, binary rewriting is triggered. The default value is set according to historical data. And preset the translation overhead threshold , compare the cross-level translation overhead value with the translation overhead threshold . When > , the multi-version binary rewriting mechanism of the instruction translation module is triggered. Specifically:

[0032] Intercept the CUDA API call instruction stream issued by the GPU instance, parse the instruction characteristics: the required CUDA version number of the application , the physical GPU driver version number and the current instruction number ;

[0033] If , API mapping is enabled. Specifically:

[0034] Search for a function with the same name as the source function in the target driver version supported by the physical GPU: if a completely matching function is found, record its memory address, if no function with the same name is found, retrieve a function variant with similar functions;

[0035] Establish a correspondence table between the source function and the target function, including three mapping types: direct mapping, variant mapping and parameter adaptation mapping, where direct mapping refers to one-to-one binding of functions with the same name, variant mapping refers to the association between functions with different names but equivalent functions, and parameter adaptation mapping refers to functions with different parameter structures, and the system automatically fills in default parameters to adapt to the call;

[0036] Intercept the original function instruction called by the application, modify the call address to the target function address, and if the number of target function parameters increases, automatically insert default parameter values;

[0037] Execute the single function call after redirection in the isolation environment, verify two core indicators, including function return value consistent with expectations and memory operation without out-of-bound, if verification fails, upgrade to binary rewriting mechanism and mark translation overhead increase;

[0038] After verification, inject the mapping relationship into the execution stream, and all subsequent same function calls automatically jump to the new address;

[0039] If , enable binary rewriting, specifically:

[0040] Load the version cross-mapping table, and execute replacement of incompatible operation codes and reconstruction of memory access patterns instruction by instruction, while preserving the original number of instructions Invariable;

[0041] Verify the rewritten results, run the rewritten instructions in the sandbox, and ensure that the new cross-level translation overhead value ≤ , otherwise enable the software simulation layer;

[0042] Generate translation metadata and synchronize it to the scheduling queue of the dynamic load balancing module.

[0043] Through the above technical solutions, real-time compatible translation of cross-version CUDA instructions is realized, which shortens the startup time of old games in a new driver environment, thereby completely eliminating the compatibility bottleneck of high game startup delay mentioned in the background technology, and meeting the user's second-level startup demand in the cloud game platform.

[0044] ②At the same time, the player's concentrated login causes the V100 resource pool load to surge, and the video memory occupancy rate increases. At this time, the resource monitoring module collects the load data of the physical GPU resource pool in real time, including the video memory occupancy rate GPU core utilization and data bandwidth throughput . And according to the demand bandwidth calculate the current bandwidth demand ratio Then through the scheduling decision module, according to the formula calculate the resource load pressure value , the formula is used to calculate the load pressure of the current GPU resource pool, by weighting multiple load indicators, a value reflecting the overall load condition is obtained. This helps the system to accurately judge the load situation of the resource pool, so as to decide whether to need to carry out load balancing operation to cope with sudden load or optimize resource utilization. 、 and are weight coefficients, wherein the weight coefficients are initialized based on historical load data or empirical values, for example: if memory-intensive tasks are dominant→ the value is larger; if compute-intensive tasks are dominant→ the value is larger; if data throughput sensitive tasks are dominant→ the value is larger. At the same time, the constraint condition is At the same time, the preset load threshold , the resource load pressure value and the load threshold are compared, when > , trigger the scheduling instruction to the dynamic load balancing module. Through the dynamic load balancing module, the tasks are scheduled to the heterogeneous nodes across the physical GPU and across the resource pool, specifically:

[0045] read the translation metadata output by the instruction translation module, and mark the task constraints, if the rewrite_level of the marked task is set to‌DEEP‌, limit the task to be scheduled to‌compat_pool‌for execution, if the version identifier V_a‌in the marked task is less than 11‌, the GPU of‌Ampere architecture‌will be disabled;

[0046] scan the resource pool and collect real-time data of each pool, including memory occupancy , GPU core utilization , data bandwidth throughput and network delay ;

[0047] According to the resource load pressure value , grading is carried out, when , execute horizontal migration, specifically:

[0048] According to the compatibility constraints in the translation metadata, low-load target nodes that meet the network delay less than 20ms, resource margin greater than the migration task demand, and hardware support for the required instruction set version of the task are screened out;

[0049] Suspend all execution threads of the task on the source node, and capture the complete execution context, including register state, video memory data pointer mapping, and unfinished instruction queue;

[0050] Only copy the modified pages since the last checkpoint, use a compression transmission protocol to reduce bandwidth occupation, and ensure data integrity through block verification;

[0051] Allocate video memory space on the target node and load the transmitted data, while rebuilding the execution context, including restoring the register state, remapping the video memory address space, and redirecting the function call pointer;

[0052] Continue executing the task from the frozen point on the target node, keep the original task ID unchanged, and route all new requests to the target node;

[0053] After confirming that the task is running stably on the target node, release the video memory and computing resources occupied by the source node, and update the resource pool state record.

[0054] When , perform vertical expansion, specifically:

[0055] According to the formula , calculate the number of instances to be added , where is the total number of instances, and filter resources from the compatible resource pool;

[0056] Lock enough video memory space, computing unit quota, and virtual channel bandwidth in the target resource pool;

[0057] Load the standard GPU virtual machine template, inject task-specific configurations, and then start the instance and connect to the scheduling network.

[0058] Through the above technical solutions, the system can complete task scheduling across GPU resource pools in a very short time when a popular game is launched, ensuring stable game frame rate, thereby solving the scheduling rigidity problem of user queuing and game lag caused by sudden load in the cloud game platform mentioned in the background technology.

[0059] ③And when the high-end A100 GPU has a high idle rate and the low-end T4 GPU is overloaded, the dynamic load balancing module performs a priority sorting operation, specifically:

[0060] According to the formula , calculate the resource gap value of the GPU instance This formula helps the system assess the load capacity and remaining potential of different GPU nodes or resource pools, providing a basis for prioritization and load balancing.

[0061] According to the formula Calculate scheduling priority This formula is used to calculate task scheduling priority. By comprehensively considering resource gaps and other relevant factors and assigning different weights, it determines the execution order of tasks. This helps the system prioritize more urgent or important tasks when resources are limited, improving resource utilization efficiency and user experience. , and Dynamic weights;

[0062] according to Tasks are sorted from high to low priority, and high-priority tasks are scheduled to low-load GPU nodes first.

[0063] The above technical solutions can reduce the idle rate of high-end GPUs, thereby improving the overall utilization rate of heterogeneous resource pools, achieving the goal of fragmented resource integration, breaking through the bottleneck of resource fragmentation and high-end GPU idle utilization mentioned in the background technology, and thus reducing the operating cost of cloud gaming platforms.

[0064] ④ In addition, the frame rate of user sessions is collected in real time through the QoS feedback module. and delay And according to the formula Calculate service quality score ,in This is the minimum tolerable frame rate, i.e., the lowest frame rate acceptable to the user experience. Then, a preset service quality score threshold is defined. Service quality score With service quality score threshold Compare them, when they appear < In this case, the scheduling strategy is dynamically adjusted and the translation path is optimized.

[0065] By using the above technical solutions, task scheduling paths and translation strategies are dynamically optimized. When the experience quality declines, scheduling strategy optimization and translation path adjustment are automatically triggered to achieve dynamic difficulty balance. This allows users of different skill levels to obtain appropriate training intensity, fundamentally avoiding the experience imbalance problem caused by user frustration due to static difficulty, thereby improving user retention rate in cloud gaming platforms.

[0066] To sum up, the application improves the performance and flexibility of the GPU virtualization system by CUDA cross-level translation and multi-pooling scheduling, realizes real-time compatible translation of cross-version CUDA instructions, shortens the game starting time, and ensures efficient resource scheduling under burst load through a dynamic load balancing mechanism, avoids user queuing and game lag, in addition, the priority sorting mechanism improves the resource utilization and reduces the operating cost, and the QoS feedback module further improves the user experience through real-time monitoring and optimization, and ensures the continuous stability and efficiency of the service.

[0067] The basic principle, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling, characterized in that, include: The instruction translation module is used to intercept CUDA call instructions from the GPU instance and perform real-time translation based on the difference between the CUDA version required by the application and the physical GPU driver version. The resource monitoring module is used to collect load data of the physical GPU resource pool in real time. The load data includes video memory utilization, GPU core utilization, and data bandwidth throughput. The scheduling decision module is used to receive data from the resource monitoring module and make resource allocation decisions; the scheduling decision module includes: Calculate the cross-level translation overhead value, taking into account the physical GPU driver version number, the CUDA version number required by the application, the instruction complexity coefficient, the current number of instructions, and the reference instruction number threshold; A preset translation overhead threshold is set, and the cross-level translation overhead value is compared with the translation overhead threshold. If the threshold is exceeded, the instruction translation module is triggered to start the multi-version binary rewrite mechanism. Calculate the resource load pressure value, taking into account the required bandwidth, video memory utilization, GPU core utilization, data bandwidth throughput and corresponding weighting coefficients; A preset load threshold is set, and the resource load pressure value is compared with the load threshold. If the threshold is exceeded, a scheduling command is triggered to the dynamic load balancing module. The dynamic load balancing module is used to schedule tasks to heterogeneous nodes across physical GPUs and resource pools according to the scheduling instructions of the scheduling decision module.

2. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 1, characterized in that, The specific steps for the instruction translation module to initiate the multi-version binary rewrite mechanism are as follows: Intercept the CUDA API call instruction stream issued by the GPU instance and parse the instruction characteristics: the CUDA version number required by the application, the physical GPU driver version number, and the current number of instructions; If the version differences are within the allowable range, then enable API mapping; If the version difference exceeds the allowed range, binary rewriting will be enabled.

3. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 2, characterized in that, The specific steps for enabling API mapping are as follows: Search for functions with the same name as the source function in the target driver version supported by the physical GPU. If a function with an exact match is found, record its memory address. If no function with the same name is found, search for function variants with similar functionality. Establish a mapping table between source functions and target functions, which includes three mapping types: direct mapping, variant mapping, and parameter adaptation mapping. Direct mapping refers to a one-to-one binding between functions with the same name, variant mapping refers to the association between functions with different names but equivalent functions, and parameter adaptation mapping refers to functions with different parameter structures, where the system automatically fills in default parameters to adapt to the call. Intercept the raw function instructions called by the application, modify the call address to the address of the target function, and automatically insert default parameter values ​​if the number of parameters in the target function increases. In an isolated environment, a single function call after redirection is executed to verify two core metrics: the function return value is consistent with the expectation and the memory operation does not exceed the bounds. If the verification fails, the binary rewrite mechanism is upgraded and the translation overhead is marked as increased. Once the verification is successful, the mapping relationship will be injected into the execution flow, and all subsequent calls to the same function will automatically jump to the new address.

4. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 2, characterized in that, The specific steps for enabling binary rewriting are as follows: Load the version cross-mapping table and execute instruction by instruction to replace incompatible opcodes and reconstruct memory access patterns, while keeping the original number of instructions unchanged; Verify the rewrite results by running the rewritten instructions in the sandbox to ensure that the new cross-level translation overhead value does not exceed the preset threshold; otherwise, enable the software simulation layer. Generate translated metadata and synchronize it to the scheduling queue of the dynamic load balancing module.

5. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 1, characterized in that, The dynamic load balancing module is also used to perform priority sorting operations, specifically: Calculate the resource deficit value of the GPU instance; Calculate scheduling priority, taking into account resource gap value, task waiting time and resource demand urgency, and assign dynamic weights; Tasks are sorted from highest to lowest scheduling priority, with high-priority tasks scheduled to low-load GPU nodes first.

6. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 4, characterized in that, The specific steps for the dynamic load balancing module to schedule tasks to heterogeneous nodes across physical GPUs and resource pools are as follows: Read the translation metadata output by the instruction translation module and mark the task constraints. If the rewrite_level of the marked task is set to DEEP, the task is restricted to be scheduled to compat_pool for execution. If the version identifier V_a in the marked task is less than 11, the GPU of the Ampere architecture will be disabled. Scan the resource pool and collect real-time data for each pool, including video memory usage, GPU core utilization, data bandwidth throughput, and network latency. The system is categorized according to resource load pressure values. When the load pressure value is in the first range, horizontal migration is performed, and when the load pressure value is in the second range, vertical expansion is performed.

7. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 6, characterized in that, The specific steps for the dynamic load balancing module to perform horizontal migration are as follows: Based on the compatibility constraints in the translated metadata, low-load target nodes that meet the requirements of network latency less than 20ms, resource margin greater than the migration task requirements, and hardware support for the instruction set version required by the task are selected. Pause all execution threads of the task on the source node and capture the complete execution context, including register state, video memory data pointer mapping, and incomplete instruction queue; Only modified pages since the last checkpoint are copied, a compressed transmission protocol is used to reduce bandwidth consumption, and data integrity is ensured through block verification. Allocate video memory space on the target node and load the transmitted data, while rebuilding the execution context, including restoring register state, remapping video memory address space and redirecting function call pointers; Continue executing the task from the freeze point on the target node, keeping the original task ID unchanged, and route all new requests to the target node; After confirming that the task is running stably on the target node, release the GPU memory and computing resources occupied by the source node and update the resource pool status record.

8. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 6, characterized in that, The specific steps for the dynamic load balancing module to perform vertical expansion are as follows: Based on the overload level, calculate the number of new instances needed and select resources from the compatibility resource pool; Lock in sufficient video memory space, computing unit quotas, and virtual channel bandwidth in the target resource pool; Load a standard GPU virtual machine template, inject task-specific configuration, then start the instance and connect it to the scheduling network.

9. The GPU virtualization system based on CUDA cross-level translation and multi-pooling scheduling according to claim 1, characterized in that, The system also includes a QoS feedback module, used to collect the frame rate and latency of user sessions in real time, calculate the service quality score according to a formula, dynamically adjust the scheduling strategy, and optimize the translation path. Specifically: Real-time acquisition of frame rate and latency of user sessions; Calculate the Quality of Service score, taking into account the current frame rate, minimum tolerable frame rate, and latency; A preset service quality score threshold is set. The service quality score is compared with the service quality score threshold. If it is lower than the threshold, the scheduling strategy is dynamically adjusted and the translation path is optimized.

Images