A container-oriented GPU pass-through method and system

Abstract

The application discloses a container-oriented GPU transparent transmission method, comprising: after the host computer is started, traversing the GPU device to build a global hardware resource account book; receiving a container application, allocating an independent video memory addressing interval, a computing power time slice quota and a hardware operation permission based on the account book, and binding a container process identifier; calling a kernel memory management unit to establish a dedicated table mapping of a physical GPU address and a container virtual address, locking the permission and registering a virtual GPU device node, completing the transparent transmission mounting; continuously collecting hardware and business data, dynamically adjusting the computing power quota, and triggering fault isolation and self-healing in an abnormal state; after the container task is completed, the page table mapping is released, the data is emptied, the register and the permission are reset, and the resources are recycled into the account book. The application can realize multi-container safe sharing, elastic scheduling and fault self-healing while retaining the original performance of the physical GPU, and greatly improves the resource utilization and the cluster stability.

Description

Technical Field

[0001] This application relates to the field of GPU resource scheduling technology, and more specifically, to a container-oriented GPU pass-through method and system. Background Technology

[0002] With the rapid development of high-computing applications such as artificial intelligence, big data analytics, and real-time graphics rendering, GPUs have become a core computing resource. Container technology, with its advantages of lightweight design, rapid deployment, and elastic scaling, is gradually becoming the mainstream architecture for supporting computing applications. However, the container ecosystem natively lacks efficient abstraction and scheduling capabilities for dedicated hardware such as GPUs. How to achieve low-loss access, secure isolation, and elastic reuse of GPU resources in a container environment has become a key bottleneck restricting the large-scale containerization of high-end computing applications.

[0003] Current mainstream containerized GPU resource scheduling solutions mainly include GPU virtualization slicing and traditional hardware passthrough. GPU virtualization slicing relies on a virtualization middleware layer for instruction forwarding and resource simulation. While it supports multi-container sharing, it suffers from inherent computational cost and transmission latency, typically resulting in a 10%–30% performance drop, making it unsuitable for high-throughput, low-latency, precision computing tasks. Traditional hardware passthrough, while preserving the native performance of the physical GPU, often employs a static binding model where each container exclusively uses a single GPU. This coarse-grained resource scheduling fails to support multi-container sharing and reuse, leading to low hardware utilization. Furthermore, traditional passthrough lacks kernel-level resource isolation, fine-grained quota management, and permission constraints. When multiple containers run concurrently, it is prone to anomalies such as computational preemption, memory data crosstalk, and unauthorized hardware access. It also does not support dynamic mounting and unmounting of runtime resources, exhibits poor compatibility with heterogeneous GPUs from multiple brands, lacks fault tolerance and self-healing capabilities, and suffers from overall technical shortcomings including insufficient security, poor flexibility, and high operational and maintenance costs. Summary of the Invention

[0004] In view of at least one defect or improvement need in the prior art, this application provides a container-oriented GPU pass-through method and system, which can solve at least one of the problems existing in the background art.

[0005] To achieve the above objectives, according to the first aspect of this application, a container-oriented GPU pass-through method is provided, the method comprising: After the host machine starts up, it completes the initialization of the PCIe bus and the underlying GPU driver, disables the system's native virtualization and vGPU slicing mechanism, traverses the physical GPU devices and verifies the device's legality and hardware status, collects the bus address, video memory capacity, number of stream processors and interrupt resource parameters, and builds a global GPU hardware resource ledger. Receive resource request parameters for container services, allocate independent video memory addressing range, computing time slice quota and hardware operation permissions to the target container based on the global GPU hardware resource ledger, generate a unique resource identifier and bind it to the container process identifier; The kernel memory management unit interface is called to establish a dedicated page table mapping between physical GPU addresses and container virtual addresses, lock the access permissions of the mapping range, register virtual GPU device nodes compatible with native drivers inside the container, and complete the GPU hardware pass-through mounting. Continuously collect GPU hardware status data and container service operation data, dynamically adjust the computing power time slice quota according to real-time load, and trigger fault isolation and self-healing processes in stages when anomalies are detected. After the container task is completed, the page table mapping relationship is released, the video memory data and task cache are cleared, the GPU hardware registers and permission configurations are reset, all computing power and video memory resources are reclaimed and reinstated into the global GPU hardware resource ledger.

[0006] Furthermore, in the aforementioned container-oriented GPU pass-through method, the construction of a global GPU hardware resource ledger specifically includes: Based on the kernel PCIe bus scanning interface, the base address register address, video memory physical address space, device identification code, interrupt number, and direct memory access channel address of the target physical GPU are read. Invalid devices of integrated graphics cards and virtual graphics cards are filtered out, and a unique hardware resource ledger is established.

[0007] Furthermore, in the aforementioned container-oriented GPU pass-through method, the establishment of a dedicated page table mapping between the physical GPU address and the container virtual address specifically includes: The physical GPU's memory space, register operation space, and direct memory access data transfer space are fully mapped to the container's independent address space. The mapping process preserves the hardware's native addressing rules and transfer timing, and does not perform instruction rewriting or data simulation operations.

[0008] Furthermore, in the above-described container-oriented GPU pass-through method, the dynamic adjustment of the computing power time slice quota specifically includes: The physical memory address segmentation locking technology is used to divide the entire GPU physical memory into multiple independent and continuous addressing ranges, and allocate dedicated memory partitions for each container. By locking the addressing boundaries of each partition through kernel page table permissions, containers are restricted to accessing only the video memory address space bound to themselves.

[0009] Furthermore, in the above-described container-oriented GPU pass-through method, the dynamic adjustment of the computing power time slice quota further includes: Based on the GPU hardware clock cycle and stream processor scheduling mechanism, the computing clock cycle is divided into equal time slices; Dynamically allocate computing resources based on container service priority and real-time load weight, and limit the upper limit of computing power usage of a single container.

[0010] Furthermore, in the aforementioned container-oriented GPU pass-through method, the step of triggering a graded fault isolation and self-healing process upon detecting an anomaly specifically includes: Construct a two-dimensional data acquisition link between kernel mode and container user mode to collect data on video memory usage, stream processor utilization, temperature, power consumption, and task queue length; Establish a three-level fault diagnosis model to distinguish between minor anomalies, moderate blockages, and severe faults; For severe failures, immediately isolate the faulty container and the faulty GPU to prevent the failure from spreading.

[0011] Furthermore, the aforementioned GPU pass-through method for containers, wherein the step of triggering fault isolation and self-healing processes in a tiered manner upon detecting an anomaly, further includes: For self-healing hardware and software anomalies, it automatically clears invalid video memory data, restores the default state of hardware registers, repairs abnormal page table mappings, and restarts the local scheduling process, achieving fault self-healing without manual intervention, while retaining fault logs and self-healing records.

[0012] According to a second aspect of this application, a container-oriented GPU passthrough system is also provided, comprising: The hardware initialization and device compliance verification module is used to complete the initialization of the PCIe bus and GPU underlying driver after the host machine starts up, disable the system's native virtualization and vGPU slicing mechanism, traverse physical GPU devices and verify the legality of the devices and hardware status, collect bus address, video memory capacity, number of stream processors and interrupt resource parameters, and build a global GPU hardware resource ledger. The resource partitioning and quota configuration module is used to receive resource application parameters for container services, allocate independent video memory addressing range, computing time slice quota and hardware operation permissions to the target container based on the global GPU hardware resource ledger, generate a unique resource identifier and bind it to the container process identifier. The kernel address mapping and virtual device registration module is used to call the kernel memory management unit interface to establish a dedicated page table mapping between physical GPU addresses and container virtual addresses, lock the access permissions of the mapping range, register virtual GPU device nodes compatible with native drivers inside the container, and complete the GPU hardware pass-through mounting. The real-time monitoring and adaptive optimization module is used to continuously collect GPU hardware status data and container service operation data, dynamically adjust the computing power time slice quota according to the real-time load, and trigger fault isolation and self-healing processes in stages when an anomaly is detected. The resource unloading and device reset module is used to release the page table mapping relationship, clear the video memory data and task cache, reset the GPU hardware registers and permission configuration, reclaim all computing power and video memory resources and re-include them in the global GPU hardware resource ledger after the container task ends.

[0013] According to a third aspect of this application, a container-oriented GPU pass-through device is also provided, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of any of the methods described above.

[0014] According to a fourth aspect of this application, a storage medium is also provided that stores a computer program executable by a container-oriented GPU passthrough device, which, when run on the container-oriented GPU passthrough device, causes the container-oriented GPU passthrough device to perform the steps of any of the methods described above.

[0015] In summary, compared with the prior art, the above-described technical solutions conceived in this application can achieve the following beneficial effects: The container-oriented GPU pass-through method provided in this application can eliminate the traditional virtualization intermediate forwarding layer, and achieve secure sharing of multiple containers and fine-grained resource isolation while retaining the native computing power of physical GPUs. It supports runtime elastic scheduling and seamless fault self-healing, greatly improves GPU hardware resource utilization and cluster operation stability, and reduces the deployment and maintenance costs of heterogeneous computing environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a container-oriented GPU pass-through method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a multi-level heterogeneous in-memory computing converged architecture design provided in an embodiment of this application; Figure 3 This is a schematic diagram of the overall workflow provided for an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Furthermore, the technical features involved in the various embodiments described below can be combined with each other as long as they do not conflict with each other.

[0019] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0020] Figure 1 A flowchart illustrating a container-oriented GPU pass-through method provided in this application embodiment is shown below. Figure 1 As shown in the figure, this application provides a GPU pass-through method for containers, including the following steps: After the host machine starts up, it completes the initialization of the PCIe bus and the underlying GPU driver, disables the system's native virtualization and vGPU slicing mechanism, traverses the physical GPU devices and verifies the device's legality and hardware status, collects the bus address, video memory capacity, number of stream processors and interrupt resource parameters, and builds a global GPU hardware resource ledger. Receive resource request parameters for container services, allocate independent video memory addressing range, computing time slice quota and hardware operation permissions to the target container based on the global GPU hardware resource ledger, generate a unique resource identifier and bind it to the container process identifier; The kernel memory management unit interface is called to establish a dedicated page table mapping between physical GPU addresses and container virtual addresses, lock the access permissions of the mapping range, register virtual GPU device nodes compatible with native drivers inside the container, and complete the GPU hardware pass-through mounting. Continuously collect GPU hardware status data and container service operation data, dynamically adjust the computing power time slice quota according to real-time load, and trigger fault isolation and self-healing processes in stages when anomalies are detected. After the container task is completed, the page table mapping relationship is released, the video memory data and task cache are cleared, the GPU hardware registers and permission configurations are reset, all computing power and video memory resources are reclaimed and reinstated into the global GPU hardware resource ledger.

[0021] Specifically, this application provides a container-oriented GPU passthrough method. This method can be applied to containerized computing cluster systems based on Docker or Kubernetes. The system adopts a four-layer decoupled architecture, including a container service layer, a resource management middleware layer, a kernel passthrough adaptation layer, and a bottom hardware layer. The specific implementation steps of this method are as follows: First, hardware initialization and device compliance verification are performed. After the host machine powers on, the system completes the initialization of the PCIe bus and the underlying GPU drivers, and actively disables the system's native virtualization support and vGPU slicing mechanism to avoid interference from the virtualization intermediate layer on the hardware passthrough path. Then, using the PCIe bus scanning interface provided by the kernel, all physical GPU devices within the server are scanned, and the legitimacy, hardware health status, and driver compatibility of each device are verified, filtering out non-target devices such as integrated graphics cards or virtual graphics cards. After successful verification, the system collects core hardware parameters for each physical GPU, such as bus address, memory capacity, number of stream processors, and interrupt resources, and constructs a global GPU hardware resource ledger based on these parameters, providing standardized data support for subsequent resource allocation and address mapping.

[0022] Secondly, the resource partitioning and quota configuration steps are executed. When the container service layer receives a resource request initiated by a container service, the request parameters carry information such as the amount of GPU memory required by the service, the proportion of computing power, the task priority, and the runtime. Based on the global GPU hardware resource ledger, the resource management middleware layer uses a dynamic partitioning algorithm to allocate independent GPU memory addressing ranges, computing power time slice quotas, and hardware operation permissions to the target container. Specifically, the system uses physical GPU memory address segmentation locking technology to divide the entire GPU physical memory into multiple independent and continuous addressing ranges, allocating dedicated GPU memory partitions to each container; at the same time, based on the GPU hardware clock cycle and stream processor scheduling mechanism, the computing clock cycle is divided into equal time slices, and the computing power time slice quota is dynamically allocated in combination with service priority and real-time load weight. In addition, the system constructs a two-way control system of hardware operation blacklists and whitelists, which only grants basic permissions such as data read and write and model calculation to ordinary service containers, while blocking high-risk low-level operations such as register modification and frequency adjustment. After resource allocation is completed, the system generates a unique resource identifier and access key, and binds the resource to the process identifier of the target container to achieve a one-to-one correspondence between the resource and the target container, thus avoiding resource sharing conflicts.

[0023] Next, the kernel address mapping and virtual device registration steps are executed. The system calls the kernel memory management unit interface to establish a one-to-one, offset-free page table mapping relationship between the physical GPU's memory space, register operation space, and DMA data transfer space and the container's virtual address. This mapping process completely preserves the hardware's native addressing rules and transmission timing, without performing any instruction rewriting or data simulation operations, ensuring that container GPU instructions can directly reach the physical hardware. While establishing the mapping, the system locks the access permissions of the mapped range, prohibiting the container from accessing addresses outside the mapped range. Afterward, the system registers a lightweight virtual GPU device node in the container's internal file system. This node is compatible with mainstream native driver interfaces such as CUDA or ROCm, requiring no modification to the upper-layer business code. Finally, the device communication link is initialized, completing the GPU hardware pass-through mounting, allowing applications within the container to directly access the physical GPU's computing resources through standard interfaces.

[0024] Subsequently, real-time monitoring and adaptive optimization steps are executed. The system constructs a dual-dimensional data acquisition link between kernel mode and container user mode, continuously collecting GPU hardware status data and container service operation data. Hardware status data includes memory usage, stream processor utilization, temperature, power consumption, bus status, and abnormal interrupt information; container service operation data includes GPU instruction call frequency, task queue length, transmission latency, and resource quota utilization. The system analyzes this data in real time to determine load fluctuations and device health status, and dynamically adjusts the computing time slice quota based on the real-time load to balance resource consumption among multiple containers. Meanwhile, when an anomaly is detected, the system performs graded processing based on a three-level fault judgment model: for minor anomalies such as instruction timeouts or short-term overloads, task retry and resource throttling are performed; for moderate anomalies such as queue backlogs or task blocking, computing resources are dynamically expanded and invalid tasks are cleaned up; for severe faults such as hardware errors, driver anomalies, or memory crashes, the faulty container and faulty GPU are immediately isolated to prevent the spread of the fault and trigger a self-healing process, such as clearing invalid memory data, restoring the default state of hardware registers, repairing abnormal page table mappings, or restarting the local scheduling process, to achieve fault self-healing without manual intervention, while retaining fault logs and self-healing records for traceability.

[0025] Finally, the resource unloading and device reset steps are performed. When the container task ends normally or the container is destroyed, the system automatically releases the page table mapping relationship, clears the business data and task cache in the video memory, resets the GPU hardware registers, scheduling status and permission configuration, reclaims all computing power and video memory resources, and re-includes the reclaimed resources in the global GPU hardware resource ledger, marking them as available. This enables the cyclic reuse of hardware resources within the cluster, supporting the elastic scheduling needs of other container services in the future.

[0026] The container-oriented GPU pass-through method provided in this application can eliminate the traditional virtualization intermediate forwarding layer, and achieve secure sharing of multiple containers and fine-grained resource isolation while retaining the native computing power of physical GPUs. It supports runtime elastic scheduling and seamless fault self-healing, greatly improves GPU hardware resource utilization and cluster operation stability, and reduces the deployment and maintenance costs of heterogeneous computing environments.

[0027] Optionally, the container-oriented GPU pass-through method provided in this application embodiment, wherein the construction of a global GPU hardware resource ledger specifically includes: Based on the kernel PCIe bus scanning interface, the base address register address, video memory physical address space, device identification code, interrupt number, and direct memory access channel address of the target physical GPU are read. Invalid devices of integrated graphics cards and virtual graphics cards are filtered out, and a unique hardware resource ledger is established.

[0028] Optionally, the container-oriented GPU pass-through method provided in this application embodiment, wherein establishing a dedicated page table mapping between the physical GPU address and the container virtual address specifically includes: The physical GPU's memory space, register operation space, and direct memory access data transfer space are fully mapped to the container's independent address space. The mapping process preserves the hardware's native addressing rules and transfer timing, and does not perform instruction rewriting or data simulation operations.

[0029] Optionally, the GPU pass-through method for containers provided in this application embodiment, wherein dynamically adjusting the computing power time slice quota specifically includes: The physical memory address segmentation locking technology is used to divide the entire GPU physical memory into multiple independent and continuous addressing ranges, and allocate dedicated memory partitions for each container. By locking the addressing boundaries of each partition through kernel page table permissions, containers are restricted to accessing only the video memory address space bound to themselves.

[0030] Optionally, the container-oriented GPU pass-through method provided in this application embodiment further includes, in addition to, dynamically adjusting the computing power time slice quota: Based on the GPU hardware clock cycle and stream processor scheduling mechanism, the computing clock cycle is divided into equal time slices; Dynamically allocate computing resources based on container service priority and real-time load weight, and limit the upper limit of computing power usage of a single container.

[0031] Optionally, the GPU pass-through method for containers provided in this application embodiment, wherein the step of triggering a graded fault isolation and self-healing process when an anomaly is detected, specifically includes: Construct a two-dimensional data acquisition link between kernel mode and container user mode to collect data on video memory usage, stream processor utilization, temperature, power consumption, and task queue length; Establish a three-level fault diagnosis model to distinguish between minor anomalies, moderate blockages, and severe faults; For severe failures, immediately isolate the faulty container and the faulty GPU to prevent the failure from spreading.

[0032] Optionally, the container-oriented GPU pass-through method provided in this application, wherein the step of triggering a graded fault isolation and self-healing process when an anomaly is detected, further includes: For self-healing hardware and software anomalies, it automatically clears invalid video memory data, restores the default state of hardware registers, repairs abnormal page table mappings, and restarts the local scheduling process, achieving fault self-healing without manual intervention, while retaining fault logs and self-healing records.

[0033] Specifically, this application adopts a four-layer layered decoupled architecture, such as Figure 2 As shown, the architecture, from top to bottom, consists of the container service layer, the resource management middleware layer, the kernel passthrough adaptation layer, and the underlying hardware layer. This architecture abandons the traditional virtualization simulation forwarding mechanism, preserving the native computing power performance of physical GPUs through hardware passthrough. Simultaneously, it leverages a layered management mechanism to overcome the technical shortcomings of traditional bare-metal passthrough solutions, such as lack of isolation, scheduling, and fault tolerance. Each layer has independent responsibilities, is decoupled and collaborative, and operates bidirectionally, adaptable to dynamic and elastic computing power scheduling scenarios involving multiple containers, multiple heterogeneous GPUs, and multiple node clusters. The specific composition, functions, and interaction logic of each layer are as follows: (1) Low-level hardware layer This layer forms the foundation for computing power, including server physical hardware, PCIe bus architecture, operating system kernel, underlying hardware drivers, GPU cards, etc., fully supporting native hardware resources such as video memory, SM stream processors, hardware registers, DMA transfer channels, and hardware interrupts. This layer does not undergo any hardware modification or virtualization simulation, completely preserving the native hardware operating mechanism and providing underlying hardware support for low-loss GPU pass-through.

[0034] (2) Kernel pass-through adapter layer This layer, deployed in the host kernel space, is the core layer for achieving low-loss GPU hardware passthrough. It primarily includes a GPU hardware scanning submodule, a kernel page table mapping submodule, and a device registration submodule. This layer directly calls the kernel MMU unit and the PCIe bus interface to complete accurate GPU hardware parameter parsing, a one-to-one mapping between physical addresses and container virtual addresses without offset, and shields interference mechanisms such as native system virtualization and vGPU slicing. It skips all intermediate forwarding links, allowing container GPU instructions to directly reach the physical hardware. Simultaneously, it unifies the underlying interaction interface for heterogeneous GPUs, enabling basic adaptation capabilities for various types of GPU hardware.

[0035] (3) Resource management intermediate layer This layer serves as the core of system innovation and management, employing a collaborative deployment approach between kernel and user modes. It completely decouples hardware resources from upper-layer container services, achieving refined resource management without compromising hardware passthrough performance. This layer integrates GPU resource quota partitioning, hardware permission management, monitoring, and fault tolerance submodules. It is responsible for logically partitioning, permission binding, anomaly isolation, and resource reclamation of the physical GPU resources after passthrough, solving the technical problems of traditional passthrough solutions such as inability to share resources among multiple containers, resource preemption, data crosstalk, and fault propagation. Simultaneously, it interfaces with container cluster scheduling interfaces, providing upper-layer services with elastic scaling and fault self-healing capabilities.

[0036] (4) Container service layer This layer serves as the business application bearer, built upon blacklists / whitelists and the mainstream Kubernetes container ecosystem. It includes container application load balancing, container resource management plugins, and container cluster scheduling interfaces. This layer is unaware of underlying hardware differences and kernel pass-through logic; it simply requests and invokes GPU computing resources on demand through standardized interfaces. Relying on the middleware management capabilities, it achieves automatic matching, dynamic scheduling, and idle resource reclamation of GPU resources, supporting the elastic deployment and high-concurrency operation of containerized computing services.

[0037] This system forms a closed-loop operating logic: the underlying hardware provides native computing power, the kernel layer implements low-loss pass-through, the middle layer implements controllable shared scheduling, and the upper layer implements elastic business support. Each layer has unidirectional dependence and bidirectional collaboration.

[0038] like Figure 3 The diagram illustrates the overall technical implementation steps of the container-oriented GPU pass-through method provided in this application embodiment: S1. Hardware Initialization and Device Compliance Verification: After the host machine boots up, it completes the initialization of the PCIe bus and the underlying GPU driver, disables the system's native virtualization and vGPU slicing mechanism to avoid interference from native mechanisms with the hardware passthrough process. It then traverses all physical GPU devices through the kernel scanning interface, verifying device legitimacy, hardware status, and driver compatibility. It collects core parameters such as bus address, memory capacity, number of SM units, and interrupt resources to build a global GPU hardware resource ledger.

[0039] S2. Business-driven resource partitioning and quota configuration: Receives resource request parameters such as memory usage, computing power ratio, task priority, and runtime of container services. Based on the global resource ledger, it adopts a dynamic partitioning algorithm to allocate independent memory addressing ranges, computing power time slice quotas, and hardware operation permissions to the target container. It generates a unique resource identifier ID and access key, and binds the container PID to achieve a one-to-one correspondence between resources and containers, avoiding resource misuse conflicts.

[0040] S3. Kernel Address Mapping and Virtual Device Registration: The kernel MMU interface is invoked to establish a dedicated page table mapping between the GPU physical address and the container's virtual address, locking access permissions for the mapped range and prohibiting out-of-bounds access across ranges. Inside the container, a virtual GPU device node compatible with native drivers is registered, the device communication link is initialized, and low-loss GPU hardware pass-through mounting is completed.

[0041] S4. Real-time Status Monitoring and Adaptive Resource Optimization: Continuously collects GPU hardware and container service operation status data, determines load fluctuations and device health status in real time, dynamically adjusts computing time slice allocation based on real-time load, balances resource usage across multiple containers, and ensures stable parallel operation of multiple services. When abnormal faults are detected, a tiered fault isolation and self-healing process is triggered to ensure continuous and stable system operation.

[0042] S5. Service completion, resource unloading, and device status reset: After the container task is completed or destroyed, the system automatically removes the page table mapping relationship, clears the video memory data and task cache, resets the GPU hardware registers, scheduling status, and permission configuration, reclaims all computing power and video memory resources, and re-includes the reclaimed resources in the global resource ledger to achieve hardware resource recycling.

[0043] The GPU pass-through method for containers provided in this application includes the following core content: (1) Low-loss hardware address mapping pass-through PCIe Hardware Resource Precision Resolution Mechanism: Based on the Linux kernel PCIe bus scanning interface, it traverses server bus devices and accurately reads core hardware parameters such as the BAR register address, physical memory address space, device identification code, interrupt number, and DMA channel address of the target physical GPU. It filters out invalid devices such as integrated graphics cards and virtual graphics cards, establishes a unique GPU hardware resource ledger, and provides standardized hardware data support for subsequent accurate address mapping and resource allocation, avoiding device mapping errors and resource matching anomalies.

[0044] Kernel-mode one-to-one physical address pass-through mapping: Abandoning the traditional virtualization layer and resource simulation forwarding architecture, this approach uses the kernel MMU memory management unit to perform page table remapping, establishing a one-to-one, offset-free mapping relationship between the host GPU's physical address and the container's virtual address. This completely maps the physical GPU's memory space, register operation space, and DMA data transfer space to the container's independent address space. The mapping process fully preserves the hardware's native addressing rules, data bit width, and transmission timing, without performing instruction rewriting or data simulation operations, thus ensuring the native performance of hardware pass-through from the underlying layer.

[0045] Lightweight device node registration mechanism: Lightweight virtual GPU device nodes are generated within the container file system, natively compatible with mainstream driver interfaces such as CUDA and ROCm, allowing for adaptation without modifying upper-layer business code. The device nodes only handle instruction pass-through and data interaction functions, without consuming additional system resources, ensuring lightweight and low-overhead operation of the GPU pass-through process.

[0046] (2) Multi-container GPU resource isolation and fine-grained quotas Hardware-level physical memory partitioning and isolation mechanism: Unlike traditional software-based statistical memory quota schemes, this application employs physical memory address segmentation and locking technology. Based on business resource requirements, the entire GPU physical memory is divided into multiple independent and contiguous addressable ranges, allocating dedicated memory partitions to each container. Kernel page table permissions lock the address boundaries of each partition, restricting containers to accessing only their own bound memory address space. This eliminates crosstalk, out-of-bounds read / write, and data overwriting issues between containers, achieving hardware-level memory isolation. This mechanism supports both fixed and dynamic quota partitioning modes to adapt to diverse business load scenarios.

[0047] Dynamic time-sharing rate limiting mechanism for computing power: Based on the GPU hardware clock cycle and SM stream processor scheduling mechanism, a refined time-sharing scheduling model for computing power is constructed. The GPU computing clock cycle is divided into equal time slices, and computing power resources are dynamically allocated in combination with container business priority and real-time load weight, accurately limiting the upper limit of computing power occupation of a single container. It enables on-demand allocation of computing power for differentiated business scenarios such as training and inference, avoiding task blocking caused by a single container monopolizing hardware computing power for a long time. It also supports short-term scaling up during sudden load surges and automatic scaling down during peak load declines, balancing computing power utilization and business response time.

[0048] Hardware operation permission hierarchical isolation mechanism: A two-way control system of blacklists and whitelists for hardware operations is established to achieve fine-grained hierarchical control of GPU operation permissions. Basic permissions such as data read / write, model calculation, and routine inference are only granted to ordinary business containers, while high-risk low-level operations such as register modification, frequency adjustment, power consumption configuration, and device reset are blocked. Privileged maintenance containers are granted limited high-level permissions. The system audits container hardware operation behavior in real time, intercepts unauthorized commands and generates alarm records, avoiding hardware parameter tampering and device failure caused by abnormal operations.

[0049] (3) Status monitoring and fault-tolerant recovery Real-time monitoring and acquisition via dual kernel and user space links: Constructing a dual-dimensional data acquisition link between kernel space and container user space, acquiring GPU hardware status and container service operation data at millisecond level. The hardware dimension covers memory usage, SM utilization, temperature, power consumption, bus status and abnormal interrupt information, while the service dimension includes GPU instruction call frequency, task queue length, transmission latency and resource quota utilization. Redundant data is removed through a data filtering mechanism to ensure monitoring accuracy and lightweight system operation.

[0050] Layered Fault Identification and Isolation Mechanism: A three-tiered fault judgment model is established to distinguish between three types of abnormal scenarios: minor anomalies, moderate blocking, and severe failures, enabling differentiated fault handling. For minor anomalies such as instruction timeouts and short-term overloads, task retries and resource throttling are implemented. For moderate anomalies such as queue backlogs and task blocking, computing resources are dynamically expanded and invalid tasks are cleaned up. For severe failures such as hardware errors, driver anomalies, and memory crashes, the faulty container and faulty GPU are immediately isolated to prevent the fault from spreading and ensure the stable operation of other service containers.

[0051] Seamless fault self-healing and resource reset mechanism: For self-healing hardware and software anomalies, the system automatically executes a lightweight resource reset process, clearing invalid video memory data, restoring hardware registers to their default states, repairing abnormal page table mappings, and restarting the local scheduling process, achieving fault self-healing without manual intervention. Simultaneously, complete fault logs and self-healing records are retained, enabling fault traceability and status review, providing data support for operation and maintenance optimization and strategy iteration.

[0052] The technical solution of this application possesses significant technological advancements and engineering application value, effectively addressing various core defects of existing technologies and generating multi-dimensional technological gains. Through a kernel-level low-loss direct-pass innovation mechanism, this application greatly eliminates the computing power loss and transmission latency caused by virtualization architecture, resolving the 10%~30% computing power loss and transmission latency issues present in traditional virtualization solutions. This enables container GPU computing performance to reach the native level of physical machines, breaking through the performance bottleneck of traditional vGPU solutions that cannot support high-precision, low-latency, and high-throughput computing services, and fully meeting the application needs of high-order computing scenarios such as AI large-scale model training, real-time graphics rendering, and high-frequency inference computation.

[0053] The container-oriented GPU pass-through method provided in this application has the following advantages: 1. Kernel-level low-loss pass-through innovative mechanism: This application breaks through the existing technical paradigm of GPU virtualization slicing and software simulation forwarding, completely abandons the virtualization intermediate forwarding layer, and innovatively adopts a kernel-mode physical address one-to-one offset-free mapping mechanism. Relying on the host machine's MMU memory management unit and PCIe bus hardware capabilities, it completely maps the physical GPU's video memory space, register operation space and DMA transfer space to the container's independent address space.

[0054] 2. Innovative hardware-software collaborative isolation and sharing mechanism: Without sacrificing the original performance of the hardware, this application solves the stability problems of traditional exclusive pass-through solutions, such as the inability of multiple containers to share resources, resource contention, data crosstalk, and unauthorized operations, through multiple mechanisms such as hardware-level isolation of video memory, dynamic rate limiting of computing power, and hierarchical management of permissions. It enables secure parallel reuse of multiple containers on a single GPU hardware, greatly improves the utilization rate of hardware resources, and effectively reduces the hardware investment cost of computing power clusters.

[0055] 3. Innovative mechanism for heterogeneous unified adaptation and dynamic hot scheduling: This application solves the technical shortcomings of traditional GPU pass-through solutions, such as static binding, poor compatibility, insufficient elasticity, and lack of self-healing ability. It achieves unified and compatible deployment of multiple types of GPU hardware by relying on the heterogeneous adaptation layer, realizes elastic scaling and automated operation and maintenance of computing power by relying on hot scheduling and intelligent cluster scheduling capabilities, and improves the stability of cluster operation by relying on the layered fault self-healing mechanism. It significantly reduces the deployment, adaptation and operation and maintenance costs of heterogeneous computing power clusters and has strong versatility, scalability and large-scale industrial application value.

[0056] This application also provides a container-oriented GPU pass-through system, including: The hardware initialization and device compliance verification module is used to complete the initialization of the PCIe bus and GPU underlying driver after the host machine starts up, disable the system's native virtualization and vGPU slicing mechanism, traverse physical GPU devices and verify the legality of the devices and hardware status, collect bus address, video memory capacity, number of stream processors and interrupt resource parameters, and build a global GPU hardware resource ledger. The resource partitioning and quota configuration module is used to receive resource application parameters for container services, allocate independent video memory addressing range, computing time slice quota and hardware operation permissions to the target container based on the global GPU hardware resource ledger, generate a unique resource identifier and bind it to the container process identifier. The kernel address mapping and virtual device registration module is used to call the kernel memory management unit interface to establish a dedicated page table mapping between physical GPU addresses and container virtual addresses, lock the access permissions of the mapping range, register virtual GPU device nodes compatible with native drivers inside the container, and complete the GPU hardware pass-through mounting. The real-time monitoring and adaptive optimization module is used to continuously collect GPU hardware status data and container service operation data, dynamically adjust the computing power time slice quota according to the real-time load, and trigger fault isolation and self-healing processes in stages when an anomaly is detected. The resource unloading and device reset module is used to release the page table mapping relationship, clear the video memory data and task cache, reset the GPU hardware registers and permission configuration, reclaim all computing power and video memory resources and re-include them in the global GPU hardware resource ledger after the container task ends.

[0057] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0058] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0059] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0060] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0061] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0062] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0063] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0064] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0065] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0066] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A container-oriented GPU pass-through method, characterized in that, Includes the following steps: After the host machine starts up, it completes the initialization of the PCIe bus and the underlying GPU driver, disables the system's native virtualization and vGPU slicing mechanism, traverses the physical GPU devices and verifies the device's legality and hardware status, collects the bus address, video memory capacity, number of stream processors and interrupt resource parameters, and builds a global GPU hardware resource ledger. Receive resource request parameters for container services, allocate independent video memory addressing range, computing time slice quota and hardware operation permissions to the target container based on the global GPU hardware resource ledger, generate a unique resource identifier and bind it to the container process identifier; The kernel memory management unit interface is called to establish a dedicated page table mapping between physical GPU addresses and container virtual addresses, lock the access permissions of the mapping range, register virtual GPU device nodes compatible with native drivers inside the container, and complete the GPU hardware pass-through mounting. Continuously collect GPU hardware status data and container service operation data, dynamically adjust the computing power time slice quota according to real-time load, and trigger fault isolation and self-healing processes in stages when anomalies are detected. After the container task is completed, the page table mapping relationship is released, the video memory data and task cache are cleared, the GPU hardware registers and permission configurations are reset, all computing power and video memory resources are reclaimed and reinstated into the global GPU hardware resource ledger.

2. The container-oriented GPU pass-through method according to claim 1, characterized in that, The construction of the global GPU hardware resource ledger specifically includes: Based on the kernel PCIe bus scanning interface, the base address register address, video memory physical address space, device identification code, interrupt number, and direct memory access channel address of the target physical GPU are read. Invalid devices of integrated graphics cards and virtual graphics cards are filtered out, and a unique hardware resource ledger is established.

3. The container-oriented GPU pass-through method according to claim 1, characterized in that, The establishment of a dedicated page table mapping between physical GPU addresses and container virtual addresses specifically includes: The physical GPU's memory space, register operation space, and direct memory access data transfer space are fully mapped to the container's independent address space. The mapping process preserves the hardware's native addressing rules and transfer timing, and does not perform instruction rewriting or data simulation operations.

4. The container-oriented GPU pass-through method according to claim 1, characterized in that, The dynamic adjustment of the computing power time slice quota specifically includes: The physical memory address segmentation locking technology is used to divide the entire GPU physical memory into multiple independent and continuous addressing ranges, and allocate dedicated memory partitions for each container. By locking the addressing boundaries of each partition through kernel page table permissions, containers are restricted to accessing only the video memory address space bound to themselves.

5. The container-oriented GPU pass-through method according to claim 4, characterized in that, The dynamic adjustment of the computing power time slice quota also includes: Based on the GPU hardware clock cycle and stream processor scheduling mechanism, the computing clock cycle is divided into equal time slices; Dynamically allocate computing resources based on container service priority and real-time load weight, and limit the upper limit of computing power usage of a single container.

6. The container-oriented GPU pass-through method according to claim 1, characterized in that, The step-by-step triggering of fault isolation and self-healing processes upon detecting an anomaly specifically includes: Construct a two-dimensional data acquisition link between kernel mode and container user mode to collect data on video memory usage, stream processor utilization, temperature, power consumption, and task queue length; Establish a three-level fault diagnosis model to distinguish between minor anomalies, moderate blockages, and severe faults; For severe failures, immediately isolate the faulty container and the faulty GPU to prevent the failure from spreading.

7. The container-oriented GPU pass-through method according to claim 6, characterized in that, The step of triggering fault isolation and self-healing processes in a tiered manner upon detecting an anomaly also includes: For self-healing hardware and software anomalies, it automatically clears invalid video memory data, restores the default state of hardware registers, repairs abnormal page table mappings, and restarts the local scheduling process, achieving fault self-healing without manual intervention, while retaining fault logs and self-healing records.

8. A container-oriented GPU pass-through system, characterized in that, include: The hardware initialization and device compliance verification module is used to complete the initialization of the PCIe bus and GPU underlying driver after the host machine starts up, disable the system's native virtualization and vGPU slicing mechanism, traverse physical GPU devices and verify the legality of the devices and hardware status, collect bus address, video memory capacity, number of stream processors and interrupt resource parameters, and build a global GPU hardware resource ledger. The resource partitioning and quota configuration module is used to receive resource application parameters for container services, allocate independent video memory addressing range, computing time slice quota and hardware operation permissions to the target container based on the global GPU hardware resource ledger, generate a unique resource identifier and bind it to the container process identifier. The kernel address mapping and virtual device registration module is used to call the kernel memory management unit interface to establish a dedicated page table mapping between physical GPU addresses and container virtual addresses, lock the access permissions of the mapping range, register virtual GPU device nodes compatible with native drivers inside the container, and complete the GPU hardware pass-through mounting. The real-time monitoring and adaptive optimization module is used to continuously collect GPU hardware status data and container service operation data, dynamically adjust the computing power time slice quota according to the real-time load, and trigger fault isolation and self-healing processes in stages when an anomaly is detected. The resource unloading and device reset module is used to release the page table mapping relationship, clear the video memory data and task cache, reset the GPU hardware registers and permission configuration, reclaim all computing power and video memory resources and re-include them in the global GPU hardware resource ledger after the container task ends.

9. A container-oriented GPU pass-through device, characterized in that, The method includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, It stores a computer program executable by a container-oriented GPU passthrough device, which, when run on the container-oriented GPU passthrough device, causes the container-oriented GPU passthrough device to perform the steps of the method according to any one of claims 1 to 7.

Images