Distributed CUDA virtualization method for mixed environment
By building a unified virtualization interface layer and supporting RDMA networks, the bottleneck problems of GPU resource scheduling and communication in hybrid environments are solved, achieving efficient CUDA resource management and fault recovery, and improving computing performance and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-29
- Publication Date
- 2026-03-10
AI Technical Summary
In hybrid environments, virtual machines and containers struggle to achieve unified scheduling and context migration of GPU resources across nodes. Traditional RPC communication methods result in high latency and high CPU overhead, impacting GPU computing performance.
A unified virtualization interface layer is built to intercept CUDA requests. Combined with RDMA network, resource mapping and routing decisions are performed to achieve unified management and efficient data transmission of distributed CUDA resources. It supports seamless compatibility between virtual machines and containers and performs task migration in case of failure.
It achieves GPU resource pooling and efficient scheduling in hybrid environments, improves computing performance, ensures the continuity and reliability of computing tasks, and has strong fault tolerance.
Smart Images
Figure CN121636054A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of virtualization technology, and more particularly to a method for distributed CUDA virtualization in hybrid environments. Background Technology
[0002] With the development of cloud computing and containerization technologies, the hybrid deployment of virtual machines and containers has become the norm in data centers. In this environment, efficiently sharing and remotely accessing physical GPU resources, especially to meet the CUDA computing needs of tasks such as AI training and scientific computing, is a major challenge. In existing technologies, virtual machines directly access exclusive GPU resources, which lacks elasticity; although containers can access GPUs through device plugins, it is difficult to achieve unified resource scheduling and context migration across nodes in a distributed environment.
[0003] Meanwhile, traditional Remote Procedure Call (RPC) communication suffers from high latency and high CPU overhead when transmitting CUDA instructions and data over the network, resulting in reduced GPU computing performance. Therefore, there is an urgent need for a distributed CUDA virtualization solution that can shield against the differences in underlying hybrid environments, achieve global resource pooling, and provide efficient data transmission. The logic of generating pages, modifying them, and then re-displaying them leads to lower display efficiency and higher overhead. Summary of the Invention
[0004] The purpose of this invention is to provide a method for distributed CUDA virtualization in hybrid environments. By abstracting the CUDA library and performing virtualization on top of the CUDA library interface layer, a unified virtualization layer management method is proposed to support the unified use and management of hybrid virtual machine and container environments. Furthermore, it combines high-performance RDMA networks to efficiently support distributed environments, improve the global scheduling and allocation capabilities of CUDA resources in distributed environments, and enhance system efficiency.
[0005] This invention provides the following solution:
[0006] A method for distributed CUDA virtualization in hybrid environments includes:
[0007] Build a unified virtualization interface layer and intercept CUDA requests from virtual machines and containers to extract key information;
[0008] Based on the extracted key information, query the distributed resource mapping table to obtain the system resource status that best matches the current CUDA request;
[0009] Routing decisions are made based on key information from CUDA requests and the status of system resources to determine the target execution device, perform fault detection on the target execution device, and trigger rescheduling.
[0010] Virtualization resource management and handle allocation, and the establishment of a data transmission channel between CUDA requests and target execution devices based on the RDMA network;
[0011] Determine the status of the data transmission channel and perform node fault recovery.
[0012] Furthermore, a distributed virtualization layer is built on top of the CUDA Driver API layer. This layer is implemented through a dynamic link library and supports CUDA call interception for both virtual machines and containers. When an application in a virtual machine or container calls the CUDA API, the virtualization library is loaded through the environment variable LD_PRELOAD, thereby intercepting all CUDA function calls and extracting key information. This key information includes: call context identifier, API function type, parameter data size and memory requirements, and device access mode. The call context identifier is used to distinguish requests from different virtual machines or containers, and the API function type is marked as a stateful request or a stateless request based on the function semantics.
[0013] Furthermore, the resource mapping table records real-time dynamic information of all GPU devices in the cluster, including: the identifier of each GPU device, the address of the node it is located on and the memory usage, the real-time utilization of the computing unit, the health status of the device and the network communication latency data between nodes. The health status of the device includes normal, overload or fault.
[0014] For stateful requests, locate the historically bound GPU and confirm its availability; for stateless requests, further, for stateful requests, use the historically bound GPU device whose availability has been confirmed as the target execution device.
[0015] For stateless requests, a load balancing algorithm is used to select the device with the lightest overall load from the list of available GPU candidates as the target execution device. The overall load is calculated by the following formula: Overall load value = α × computing unit utilization + β × memory utilization + γ × normalized network latency, where α, β, and γ are configurable weight coefficients, and α + β + γ = 1.
[0016] Furthermore, if the memory utilization rate of the target execution device is greater than 90%, or the computing unit utilization rate is greater than 95%, or the device response times out, the target execution device is determined to be in a fault state; otherwise, the target execution device is determined to be in a normal state.
[0017] If the target execution device is in a faulty state, a rescheduling is triggered within the same node, where the same node refers to the physical server that initiated the current request.
[0018] Furthermore, the specific process of rescheduling is as follows: Within the current node, firstly, GPUs that have been marked as faulty are excluded, and all GPUs with normal health status and sufficient remaining video memory to meet the needs of this request are selected to form a local backup device list; then, from the above backup device list, the GPU with the lowest real-time utilization of the computing unit is selected as the new target execution device; finally, the routing decision is updated, and the current CUDA request is rerouted to this newly selected GPU for execution. If there is no available GPU in the current node, the global scheduler selects an available GPU on another node.
[0019] Furthermore, a virtual handle is dynamically allocated to each newly created CUDA context. The virtual handle is a globally unique identifier generated by a cryptographic hash function, whose inputs include a user identifier, a timestamp, and a random number. For stateful requests to create a new context, its virtual handle is bound to the selected target GPU. For stateless requests, a temporary binding relationship is established between the handle and the device.
[0020] Furthermore, an available connection to the target node is obtained from a pre-established RDMA connection pool. The target node is the node where the target execution device is located. The connection pool maintains high-bandwidth, low-latency communication links between all computing nodes.
[0021] Depending on the size of the data to be transmitted, the interaction method with the target GPU is adaptively selected: for small-scale requests with data size less than 4KB, a message-based transmission mode is adopted, which encapsulates the CUDA API call instructions and parameters into data packets and sends them directly to the virtualization server on the remote node for parsing and execution; for large-scale data operations with data size greater than or equal to 4KB, a direct memory access mode is adopted: first, the memory region of the data to be transmitted is registered on the local node, and then the data is directly written to the target GPU's video memory space through RDMA write operations, or directly read from the target GPU's video memory through RDMA read operations.
[0022] Furthermore, if cross-node communication failure is detected due to the inability to establish communication with the target node through the RDMA connection pool, or if a node fault is detected through continuous heartbeat monitoring, the data transmission channel status is determined to be abnormal; otherwise, the data transmission channel status is determined to be normal. Specifically, the node fault is detected through continuous heartbeat monitoring by losing three heartbeats consecutively at 2-second intervals.
[0023] Furthermore, when the data transmission channel is in an abnormal state, a node fault recovery process is triggered. The core of this process is that the global resource scheduler takes over all CUDA computing tasks carried on the faulty node. The scheduler first queries the resource mapping table to quickly locate a healthy and resource-sufficient standby node in the current cluster. Using the checkpoint mechanism, the GPU memory data, CUDA context, and stream processor state that need to be migrated from the faulty node are migrated to the selected standby node through the RDMA network. For large-scale state data, the RDMA memory direct access mode is used first for fast migration.
[0024] The above solution achieves the following beneficial technical effects:
[0025] This invention effectively solves the challenge of GPU resource isolation and sharing in hybrid virtual machine and container environments by constructing a unified distributed CUDA virtualization layer. It achieves unified pooling, intelligent scheduling, and high-availability management of all physical GPU resources within the cluster, significantly improving resource utilization and task throughput. By intercepting CUDA requests and combining real-time resource status for routing decisions, the system can dynamically match the optimal execution device for computing tasks and use RDMA networks for low-overhead data transmission, ensuring near-native computing performance. Furthermore, this method possesses strong fault tolerance and fault recovery capabilities, automatically migrating tasks in the event of node or device failure, ensuring the continuity and reliability of distributed computing tasks, and providing efficient and stable underlying support for large-scale, hybrid GPU computing clusters. Attached Figure Description
[0026] Figure 1 This is a flowchart illustrating a method for distributed CUDA virtualization in a hybrid environment, provided by one or more embodiments of the present invention.
[0027] Figure 2 This is a flowchart of the method for determining the target execution device of the present invention.
[0028] Figure 3 This is a flowchart of the method for establishing a data transmission channel according to the present invention. Detailed Implementation
[0029] To more clearly illustrate the present invention, the following description, in conjunction with preferred embodiments and accompanying drawings, further explains the invention. Similar components in the drawings are indicated by the same reference numerals. Those skilled in the art should understand that the specific description below is illustrative rather than restrictive and should not be construed as limiting the scope of protection of the present invention.
[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0031] Specifically, this method addresses the CUDA virtualization needs in hybrid virtual machine and container environments by constructing a unified virtualization layer and combining it with RDMA network support to achieve unified scheduling and management of CUDA resources in a distributed environment.
[0032] Please see Figure 1 As shown, the method for distributed CUDA virtualization in hybrid environments includes:
[0033] Step S101: Build a unified virtualization interface layer and intercept CUDA requests from virtual machines and containers to extract key information.
[0034] Specifically, a distributed CUDA virtualization layer is built on top of the CUDA Driver API layer. This layer is implemented through a dynamic link library and supports CUDA call interception for both virtual machines and containers. When an application in a virtual machine or container calls the CUDA API, the virtualization library is loaded through the environment variable LD_PRELOAD, thereby intercepting all CUDA function calls and extracting key information. This key information includes: call context identifier, API function type, parameter data size and memory requirements, and device access mode. The call context identifier is used to distinguish requests from different virtual machines or containers, and the API function type is marked as a stateful request or a stateless request based on the function semantics.
[0035] Specifically, stateful requests refer to function calls that create or modify the CUDA runtime context state, such as context creation (cuCtxCreate), module loading (cuModuleLoad), and memory allocation (cuMemAlloc); stateless requests refer to computation-related function calls that do not depend on a specific context state, such as kernel function startup (cuLaunchKernel).
[0036] Specifically, by constructing a unified virtualization interface layer and intercepting CUDA requests, seamless compatibility and centralized management of virtual machine and container environments are achieved. This layer can extract key information, such as call context identifiers and API function types, thus providing accurate basis for subsequent resource scheduling. This helps reduce the management complexity of hybrid environments, improves the transparency and controllability of CUDA resource access, and ensures that applications can run efficiently in distributed environments without modification, enhancing the system's flexibility and scalability.
[0037] Please continue reading. Figure 1 As shown, the method for distributed CUDA virtualization in hybrid environments further includes:
[0038] Step S102: Based on the extracted key information, query the distributed resource mapping table to obtain the system resource status that best matches the current CUDA request.
[0039] Specifically, the distributed resource mapping table is maintained by the global scheduler and records the real-time dynamic information of all GPU devices in the cluster, including: the identifier of each GPU device, the address of the node it is located on and the memory usage, the real-time utilization of the computing unit, the health status of the device and the network communication latency data between nodes. The health status of the device includes normal, overload or fault.
[0040] For stateful requests, locate the historically bound GPU and confirm its availability; for stateless requests, obtain a list of available GPU candidates with light current load and low network latency.
[0041] For example, in this embodiment, the system uses quantified thresholds to explicitly define "light load" and "low network latency," thus serving as an objective basis for filtering the list of available GPU candidates. The specific filtering criteria are as follows:
[0042] Light load: refers to a GPU device that meets both of the following conditions: compute unit utilization is consistently below 75%, and memory utilization is below 80%;
[0043] Low network latency: This means that during the current measurement period, the network communication latency between the node where the GPU device is located and the requesting node is within the lowest 30th percentile of the latency between all nodes in the cluster.
[0044] When generating the candidate list, the system will filter out all GPU devices that simultaneously meet the above-mentioned "light load" condition and the "low" network latency standard based on real-time data in the resource mapping table.
[0045] Specifically, by querying the distributed resource mapping table, the system dynamically obtains the real-time status of GPU devices within the cluster, including information such as load, memory usage, and network latency. This enables the system to intelligently match resources based on request type; for example, maintaining context consistency for stateful requests and selecting low-load devices for stateless requests. This mechanism optimizes resource utilization, reduces request response time, and improves the overall system reliability and performance, ensuring that computing tasks are executed efficiently in a distributed environment.
[0046] Please continue reading. Figure 1 As shown, the method for distributed CUDA virtualization in hybrid environments further includes:
[0047] Step S103: Based on the key information of the CUDA request and the system resource status, a routing decision is made to determine the target execution device, fault detection is performed on the target execution device, and rescheduling is triggered.
[0048] Specifically, routing decisions are made based on key information and resource status, and combined with fault detection and rescheduling mechanisms, ensuring that CUDA requests are directed to the optimal execution device. By monitoring device status in real time (such as memory usage and compute unit utilization), the system can promptly identify faults and trigger local or global rescheduling, avoiding single points of failure and performance bottlenecks. This significantly improves the system's fault tolerance and availability, ensuring the continuity and stability of computing tasks, and maintaining efficient operation even during resource fluctuations or partial device malfunctions.
[0049] Please see Figure 2 As shown, the target execution device determination method includes:
[0050] Step S201: Based on the key information of the CUDA request and the system resource status, a routing decision is made to determine the target execution device.
[0051] Specifically, for stateful requests, the GPU device whose availability has been confirmed in the past will be used as the target execution device.
[0052] For stateless requests, a load balancing algorithm is used to select the device with the lightest overall load from the list of available GPU candidates as the target execution device. The overall load is calculated by the following formula: Overall load value = α × computing unit utilization + β × memory utilization + γ × normalized network latency, where α, β, and γ are configurable weight coefficients, and α + β + γ = 1.
[0053] Specifically, the normalized network latency refers to the process of mapping the original network latency value measured within the cluster to the interval [0, 1] through mathematical transformation. The specific calculation method is as follows:
[0054] During system initialization or periodically (e.g., every 5 minutes), the network latency values between all available GPU nodes in the current cluster and the currently requesting node are measured and recorded, forming a latency set L={l1,l2,...,ln}. From this set, the global minimum latency lmin and the global maximum latency lmax are determined. For any GPU device in the candidate list, its corresponding normalized network latency d is calculated using the formula:
[0055] d=(li−lmin) / (lmax−lmin);
[0056] Where li is the original network latency between the node where the GPU device is located and the currently requesting node;
[0057] The weighting coefficients α, β, and γ are used to adjust the relative importance of computational load, memory load, and network latency in the overall load assessment. Their specific values can be configured according to the actual workload characteristics of the cluster. In a typical cluster configuration example dominated by computationally intensive tasks, priority can be given to ensuring the balance of computational resources, for example, setting α=0.6, β=0.2, and γ=0.2. For a communication-intensive task cluster involving a large amount of data exchange, the weight of network latency can be appropriately increased, for example, setting α=0.3, β=0.2, and γ=0.5. The system administrator can set the weighting coefficients through the configuration file before system startup.
[0058] Please see Figure 2 As shown, the target execution device determination method further includes:
[0059] Step S202: Perform fault detection on the target execution device and trigger rescheduling.
[0060] Specifically, if the memory utilization of the target execution device is greater than 90% or the computing unit utilization is greater than 95% or the device response times out, the target execution device is determined to be in a fault state; otherwise, the target execution device is determined to be in a normal state.
[0061] If the target execution device is in a faulty state, a rescheduling is triggered within the same node. The same node refers to the physical server that initiated the current request. For example, if a container is running on a server with the IP address 192.168.1.10, then "same node" refers to the server itself and does not involve other machines on the network.
[0062] The specific rescheduling process is as follows: Within the current node, firstly, GPUs marked as faulty are excluded, and all GPUs in good health with sufficient remaining video memory to meet the current request requirements are selected to form a local backup device list; then, from the above backup device list, the GPU with the lowest real-time utilization of the computing unit is selected as the new target execution device; finally, the routing decision is updated, and the current CUDA request is rerouted to this newly selected GPU for execution. If there are no available GPUs within the current node, the global scheduler intervenes and selects an available GPU from other nodes within the cluster for rescheduling.
[0063] Please continue reading. Figure 1 As shown, the method for distributed CUDA virtualization in hybrid environments further includes:
[0064] Step S104: Virtualization resource management and handle allocation, and establish a data transmission channel between CUDA requests and the target execution device based on the RDMA network.
[0065] Specifically, through virtualized resource management and handle allocation, and the establishment of RDMA data transfer channels, dynamic resource binding and efficient communication are achieved. Virtual handles ensure the uniqueness and traceability of the context, while RDMA connection pools support low-latency, high-bandwidth data transfer. Adaptive transfer modes (such as message mode or direct memory access) optimize processing based on data size, reducing CPU overhead and network latency. This improves the throughput and response speed of distributed CUDA computing, enhancing the efficiency of large-scale data processing.
[0066] Please see Figure 3 As shown, the methods for establishing a data transmission channel include:
[0067] Step S301, Virtualization resource management and handle allocation.
[0068] Specifically, a virtual handle is dynamically allocated to each newly created CUDA context. The virtual handle is a globally unique identifier generated by a cryptographic hash function, whose inputs include a user identifier, a timestamp, and a random number. For stateful requests to create a new context, its virtual handle is bound to the selected target GPU. For stateless requests, a temporary binding relationship is established between the handle and the device, but this relationship can be adjusted in subsequent requests based on load changes. All remote context states managed through virtual handles (such as loaded modules, allocated video memory, etc.) are maintained and executed by the virtualization server on the target node.
[0069] Please continue reading. Figure 3 As shown, the method for establishing the data transmission channel further includes:
[0070] Step S302: Establish a data transmission channel between CUDA request and target execution device based on RDMA network.
[0071] Specifically, an available connection to the target node is obtained from a pre-established RDMA connection pool. The target node is the node where the target execution device is located. The connection pool maintains high-bandwidth, low-latency communication links between all computing nodes.
[0072] Depending on the size of the data to be transmitted, the interaction method with the target GPU is adaptively selected: For small-scale requests with data size less than 4KB, a message-based transmission mode is adopted, which encapsulates the CUDA API call instructions and parameters into data packets and sends them directly to the virtualization server on the remote node (a resident process responsible for parsing and executing the received CUDA instructions) for parsing and execution; For large-scale data operations with data size greater than or equal to 4KB, a direct memory access mode is adopted: first, the memory region of the data to be transmitted is registered on the local node, and then the data is directly written to the target GPU's video memory space through RDMA write operations, or directly read from the target GPU's video memory through RDMA read operations.
[0073] Please continue reading. Figure 1 As shown, the method for distributed CUDA virtualization in hybrid environments further includes:
[0074] Step S105: Determine the status of the data transmission channel and perform node fault recovery.
[0075] Specifically, if cross-node communication failure is detected due to the inability to establish communication with the target node through the RDMA connection pool, or if a node failure is detected through continuous heartbeat monitoring, the data transmission channel status is determined to be abnormal; otherwise, the data transmission channel status is determined to be normal. The detection of node failure through continuous heartbeat monitoring specifically refers to the loss of 3 consecutive heartbeats at 2-second intervals.
[0076] When the data transmission channel is abnormal, a node failure recovery process is triggered. The core of this process is that the global resource scheduler takes over all CUDA computing tasks hosted on the failed node. The scheduler first queries the distributed resource mapping table to quickly locate a healthy and resource-sufficient standby node in the current cluster. Using a checkpoint mechanism, it migrates the GPU memory data, CUDA context, and stream processor states that need to be migrated from the failed node to the selected standby node in batches and efficiently via the RDMA network. For large-scale state data, the RDMA direct memory access mode is prioritized for rapid migration to minimize service interruption time.
[0077] Specifically, by monitoring the data transmission channel status and implementing node failure recovery, the system ensures rapid response and recovery in abnormal situations. Heartbeat detection and communication failure identification can promptly trigger the global scheduler to take over tasks and migrate status data to backup nodes. This provides high availability and disaster recovery capabilities, minimizes service interruption time, ensures the robustness and business continuity of the distributed environment, and enables the CUDA virtualization system to continue operating stably even when facing node failures.
[0078] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is impossible to exhaustively list all the implementation methods here. All obvious variations or modifications derived from the technical solutions of the present invention are still within the protection scope of the present invention.
Claims
1. A method for distributed CUDA virtualization for mixed environments, characterized in that, Comprise: Build a unified virtualization interface layer, and intercept CUDA requests of virtual machines and containers to extract key information; Query a distributed resource mapping table based on the extracted key information to obtain the system resource state that best matches the current CUDA request; Make a routing decision based on the key information of the CUDA request and the system resource state to determine the target execution device, perform fault detection on the target execution device, and trigger rescheduling; Manage virtualized resources and allocate handles, and establish a data transmission channel between the CUDA request and the target execution device based on an RDMA network; Determine the state of the data transmission channel and perform node fault recovery.
2. The method for distributed CUDA virtualization for mixed environments according to claim 1, wherein, A distributed virtualization layer is built above the CUDA Driver API layer, which is implemented through a dynamic link library and supports CUDA call interception of virtual machines and containers. When an application in a virtual machine or container calls a CUDA API, the virtualization library is loaded through the environment variable LD_PRELOAD, thereby intercepting all CUDA function calls and extracting key information, including: call context identifier, API function type, parameter data size and memory requirement, and device access mode. The call context identifier is used to distinguish requests from different virtual machines or containers, and the API function type is marked as a stateful request or a stateless request according to the function semantics.
3. The method for distributed CUDA virtualization for mixed environments according to claim 2, wherein, The resource mapping table records real-time dynamic information of all GPU devices in the cluster, including: identifiers of each GPU device, addresses of the nodes where the devices are located, memory usage, real-time utilization of computing units, health status of the devices, and network communication delay data between nodes. The health status of the devices includes normal, overload, or fault; For stateful requests, locate the historically bound GPU and confirm its availability; for stateless requests, obtain a list of available GPU candidates with low current load and low network delay.
4. The method for distributed CUDA virtualization for mixed environments according to claim 3, wherein, For stateful requests, use the historically bound GPU device whose availability is confirmed as the target execution device. For stateless requests, use a load balancing algorithm to select the device with the lightest comprehensive load from the list of available GPU candidates as the target execution device. The comprehensive load is calculated by the following formula: comprehensive load value = α × computing unit utilization + β × memory usage + γ × normalized network delay, where α, β, and γ are configurable weight coefficients, and α + β + γ = 1.
5. The method for distributed CUDA virtualization for mixed environments according to claim 4, wherein, If the memory usage of the target execution device is greater than 90% or the computing unit utilization is greater than 95% or the device response times out, the target execution device is determined to be in a fault state, otherwise, the target execution device is determined to be in a normal state. If the target execution device is in a fault state, trigger rescheduling within the same node, where the same node refers to the physical server from which the current request is initiated.
6. The method for distributed CUDA virtualization for mixed environments according to claim 5, wherein, The specific process of the rescheduling is as follows: in the current node, first, the GPUs that have been marked as fault state are excluded, all GPUs in a healthy state and with sufficient remaining video memory to meet the request demand are screened out, and a local backup device list is formed; then, from the backup device list, a GPU with the lowest real-time utilization of the computing unit is selected as a new target execution device; finally, the routing decision is updated, and the current CUDA request is rerouted to the newly selected GPU for execution, and if there is no available GPU in the current node, an available GPU on another node is selected by the global scheduler.
7. The method for distributed CUDA virtualization for mixed environments according to claim 6, wherein, A virtual handle is dynamically allocated for each newly created CUDA context, which is a globally unique identifier generated by an encryption hash function, and the input of the encryption hash function includes a user identifier, a timestamp and a random number. For a stateful request, the virtual handle of the newly created context is bound to the selected target GPU, and for a stateless request, a temporary binding relationship between the handle and the device is established.
8. The method for distributed CUDA virtualization for mixed environments according to claim 7, wherein, An available connection with the target node is obtained from a pre-established RDMA connection pool, the target node is a node where the target execution device is located, and the connection pool maintains high-bandwidth and low-latency communication links between all computing nodes. According to the size of the data to be transmitted, the interaction mode with the target GPU is adaptively selected: for small-scale requests with a data size less than 4KB, a message-based transmission mode is used, the CUDA API call instruction and parameters are packaged into a data packet, and the data packet is directly sent to the virtualization server on the remote node for analysis and execution; for large-scale data operations with a data size greater than or equal to 4KB, a memory direct access mode is used: first, the memory region of the data to be transmitted is registered in the local node, and then the data is directly written into the video memory space of the target GPU through an RDMA write operation, or the data is directly read from the video memory of the target GPU through an RDMA read operation.
9. The method for distributed CUDA virtualization for mixed environments according to claim 8, wherein, If cross-node communication fails due to the failure to establish communication with the target node through the RDMA connection pool, or node failure is found through continuous heartbeat monitoring, it is determined that the data transmission channel state is abnormal, otherwise, it is determined that the data transmission channel state is normal; the node failure found through continuous heartbeat monitoring is that 3 consecutive heartbeats are lost at an interval of 2 seconds.
10. The method for distributed CUDA virtualization for mixed environments according to claim 9, wherein, When the data transmission channel state is abnormal, a node failure recovery process is triggered, and the core of the process is that the global resource scheduler takes over all CUDA computing tasks carried on the failed node. The scheduler first queries the resource mapping table to quickly locate a backup node that is healthy and has sufficient resources in the current cluster, and then uses the checkpoint mechanism to migrate the GPU video memory data, CUDA context and stream processor state of the failed node to the selected backup node through the RDMA network. For large-scale state data, the RDMA memory direct access mode is preferentially used for fast migration.