Interface interception-based GPU resource isolation method, device, equipment and medium
By injecting an interception library before the application process starts, intercepting and rewriting the video memory query interface, and constructing a quota-constrained view, the problem of GPU resource isolation schemes failing when drivers are upgraded or replaced in the existing technology is solved, achieving stable and reliable multi-tenant resource isolation and improving the predictability of services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA INFORMATION SAFETY RES INST CO LTD
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-16
Smart Images
Figure CN122220091A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer resource management technology, and in particular to GPU resource isolation methods, apparatus, devices and media based on interface interception. Background Technology
[0002] With the widespread application of artificial intelligence and scientific computing, GPUs have become critical computing infrastructure. To improve resource utilization, cloud service providers and enterprise data centers commonly adopt deployment models where multiple processes or containers share a single physical GPU. However, GPUs natively lack fine-grained multi-tenant resource isolation capabilities, leading to resource contention among tenants and affecting service stability and predictability. Existing GPU resource isolation solutions primarily intercept memory allocation and kernel emission calls at the GPU driver interface level, combining this with external quota management services to implement resource limits. While such solutions achieve low-level enforcement, they have serious drawbacks: their interception points are deeply coupled to specific GPU driver versions and kernel module implementations. When the driver is upgraded or the GPU model is changed, the interception logic is highly susceptible to failure, resulting in unreliable quota control. Therefore, a new method is urgently needed that does not rely on low-level driver implementations and can simultaneously ensure consistency between memory observability and enforcement, in order to build a stable, reliable, and auditable GPU resource isolation sandbox in user space. Summary of the Invention
[0003] In view of this, the purpose of this application is to provide a GPU resource isolation method, apparatus, device and medium based on interface interception, which achieves "runtime entry controllability" through pre-loading interception, reduces the dependence on the consistency of the underlying call chain, and intercepts and rewrites the memory query interface to construct a memory view subject to quota constraints, so that the application's observation results are forced to be consistent with the allocation.
[0004] This application provides a GPU resource isolation method based on interface interception, the GPU resource isolation method including: Before the application process starts, a runtime interception library is injected into the address space of the application process through a preloading mechanism; wherein, the GPU runtime interface configured for the runtime interception library includes a memory query interface, a memory allocation interface, a memory release interface, and a kernel launch interface. When the application process receives a request to call the video memory query interface, the runtime interception library queries the management terminal for the video memory quota information of the current tenant on the target GPU device. Based on the video memory quota information, the total video memory and available amount parameters returned by the video memory query interface are corrected to construct a video memory observation view constrained by the quota. When a request is received from the application process to call the video memory allocation interface, the runtime interception library performs a pre-allocation verification process before calling the corresponding function of the GPU native runtime library to determine whether to allocate video memory to the application process.
[0005] In one possible implementation, the pre-allocation verification process is performed through the following steps: The system queries the management terminal for the current available quota of the application process and checks whether the requested quota is less than or equal to the current available quota. If so, the allocation is performed by calling the GPU's native runtime library; If not, return the GPU native computing platform standard out-of-memory error code and do not allocate memory.
[0006] In one possible implementation, after successful allocation, the GPU resource isolation method further includes: The system establishes a mapping relationship between the allocated device pointer and the current allocation amount and stores it in a local mapping table, and sends an accounting request to the management terminal to update the tenant's usage.
[0007] In one possible implementation, after sending the accounting request to the management terminal, the GPU resource isolation method further includes: When the system receives feedback that the payment request has been rejected by the management, the runtime interception library performs a rollback operation.
[0008] In one possible implementation, the GPU resource isolation method further includes, when the video memory release interface is invoked: When the application process receives a request to call the video memory release interface, the runtime interception library determines the corresponding allocation in the local mapping table based on the device pointer to be released, deletes the corresponding mapping entry, sends an account release request to the management terminal, and calls the GPU native runtime library to perform the release process.
[0009] In one possible implementation, the GPU resource isolation method further includes, for the purpose of invoking the Kernel launch interface: The runtime interception library determines whether to request a launch license based on a preset strategy and the current quota status, before the corresponding function in the GPU native runtime library. If so, a permission token is simultaneously sent to the management terminal. After obtaining permission from the management terminal, the GPU native runtime library is invoked to perform kernel launch processing.
[0010] In one possible implementation, when the management terminal is unavailable or communication fails, the runtime interception library performs any of the following operations according to a preset degradation strategy: It maintains memory quota control, rejects all kernel launch requests, and enables local caching policies, allowing a limited number of launches based on the most recent valid license.
[0011] This application also provides a GPU resource isolation device based on interface interception, the GPU resource isolation device comprising: The preloading module is used to inject the runtime interception library into the address space of the application process before the application process starts through a preloading mechanism; wherein the runtime interception library is configured with a GPU runtime interface including a memory query interface, a memory allocation interface, a memory release interface, and a kernel launch interface. The video memory query module is used to query the management terminal for the video memory quota information of the current tenant on the target GPU device when the application process calls the video memory query interface. Based on the video memory quota information, the total video memory and available amount parameters returned by the video memory query interface are corrected to construct a video memory observation view constrained by the quota. The video memory allocation module is used to determine whether to allocate video memory to the application process when the application process calls the video memory allocation interface. Before calling the corresponding function of the GPU native runtime library, the runtime interception library performs a pre-allocation verification process.
[0012] This application embodiment also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the GPU resource isolation method based on interface interception described above are performed.
[0013] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the GPU resource isolation method based on interface interception as described above.
[0014] This application provides a GPU resource isolation method, apparatus, device, and medium based on interface interception. The GPU resource isolation method includes: injecting a runtime interception library into the address space of the application process through a preloading mechanism before the application process starts; wherein the GPU runtime interface configured in the runtime interception library includes a memory query interface, a memory allocation interface, a memory release interface, and a kernel emission interface; when a request from the application process to call the memory query interface is received, the runtime interception library queries the management terminal for the memory quota information of the current tenant on the target GPU device, and corrects the total memory and available memory parameters returned by the memory query interface based on the memory quota information, thereby constructing a memory observation view constrained by the quota; when a request from the application process to call the memory allocation interface is received, the runtime interception library performs a pre-allocation verification process before calling the corresponding function of the GPU native runtime library to determine whether to allocate memory to the application process. By preloading interception, "runtime entry point controllability" is achieved, reducing the dependence on the consistency of the underlying call chain. Furthermore, the memory query interface is intercepted and rewritten to construct a memory view subject to quota constraints, thereby forcing the application's observations to be consistent with the allocation.
[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 A flowchart illustrating a GPU resource isolation method based on interface interception provided in an embodiment of this application; Figure 2 This is one of the structural schematic diagrams of a GPU resource isolation device based on interface interception provided in an embodiment of this application; Figure 3 This is a second schematic diagram of a GPU resource isolation device based on interface interception provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.
[0019] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of computer resource management technology.
[0020] Research has revealed that GPUs have become critical computing infrastructure in the context of widespread applications of artificial intelligence and scientific computing. To improve resource utilization, cloud service providers and enterprise data centers commonly adopt deployment models where multiple processes or containers share a single physical GPU. However, GPUs inherently lack fine-grained multi-tenant resource isolation capabilities, leading to resource contention among tenants and impacting service stability and predictability. Existing GPU resource isolation solutions primarily intercept memory allocation and kernel emission calls at the GPU driver interface level, combined with external quota management services to implement resource limits. While such solutions achieve low-level enforcement, they suffer from serious flaws: their interception points are deeply coupled to specific GPU driver versions and kernel module implementations. When the driver is upgraded or the GPU model is changed, the interception logic is highly susceptible to failure, resulting in unreliable quota control. Therefore, a new method is urgently needed that does not rely on low-level driver implementations and can simultaneously ensure consistency between memory observability and enforcement, in order to build a stable, reliable, and auditable GPU resource isolation sandbox in user space.
[0021] Based on this, embodiments of this application provide a GPU resource isolation method, apparatus, device, and medium based on interface interception. By preloading interception, "runtime entry point controllability" is achieved, reducing the dependence on the consistency of the underlying call chain. Furthermore, the memory query interface is intercepted and rewritten to construct a memory view subject to quota constraints, thereby forcing the application's observation results to be consistent with the allocation.
[0022] Please see Figure 1 , Figure 1 This is a flowchart illustrating a GPU resource isolation method based on interface interception, provided as an embodiment of this application. Figure 1 As shown in the embodiments of this application, the GPU resource isolation method includes: S101: Before the application process starts, the runtime interception library is injected into the address space of the application process through a preloading mechanism.
[0023] In this step, before starting the application process, preloading parameters are configured in its runtime environment to ensure that the runtime interception library is loaded by the dynamic linker first; at the same time, runtime parameters for identity binding and resource domain division are configured, such as tenant / container / process identifier, device identifier, management terminal address, and policy switch.
[0024] It should be noted that the application process is a CUDA ecosystem application or framework process running on a container or host machine, which initiates memory queries, memory allocation / release, and kernel launch requests through the CUDA Runtime interface.
[0025] The runtime interception library is configured with CUDA Runtime API including a memory query interface, a memory allocation interface, a memory release interface, and a kernel launch interface.
[0026] Here, the runtime interception library is injected into the user-space dynamic library of the application process in a preloaded manner. It is used to intercept CUDA Runtime interface calls within the application process and forward them to the real runtime library when the quota conditions are met.
[0027] S102: When the application process receives a request to call the video memory query interface, the runtime interception library queries the management terminal for the video memory quota information of the current tenant on the target GPU device, and corrects the total video memory and available amount parameters returned by the video memory query interface based on the video memory quota information, thereby constructing a video memory observation view constrained by the quota.
[0028] In this step, when a request to call the memory query interface is received from the application process, the runtime interception library queries the management end for the memory quota information of the current tenant on the target GPU device. Based on the memory quota information, the total memory and available memory parameters returned by the memory query interface are corrected to construct a memory observation view constrained by the quota.
[0029] Here, the runtime interception library first initiates a synchronous query request to the management end to obtain the current tenant's quota limit and current usage on the target GPU device. Based on the quota limit and current usage, it corrects the total and available memory parameters returned by the memory query interface, and then returns the information to the application process. Therefore, the quota limit and current usage obtained by the application process through this interface no longer reflect the actual capacity of the physical GPU, but strictly correspond to the boundary of the granted, safely usable memory resources. This implements the runtime interception library's collaborative interception and correction of the memory query interface in the CUDA Runtime API, ensuring consistency between the memory status observed by the application process and its actual allocable capacity.
[0030] In a specific implementation, the runtime interception library completes the following preparations: establishing a binding between the tenant identifier (e.g., container ID `tenant_id="ns-abc123"`) and the target GPU device index (e.g., `device_id=0`); initiating an initial quota synchronization request to the management end to obtain the initial quota configuration for the tenant on `device_id=0`, and caching the quota data in the local storage structure. When the application process calls the video memory query interface, the interception library replaces the fields of the total video memory and available memory parameters returned by the video memory query interface according to the video memory quota information to complete the correction.
[0031] S103: When a request to call the video memory allocation interface is received from the application process, the runtime interception library performs a pre-allocation verification process before calling the corresponding function of the real CUDA runtime library to determine whether to allocate video memory to the application process.
[0032] In this step, when a request to call the video memory allocation interface is received from the application process, the runtime interception library performs a pre-allocation verification process before calling the corresponding function of the real CUDA runtime library to determine whether to allocate video memory to the application process.
[0033] In one possible implementation, the pre-allocation verification process is performed through the following steps: The system queries the management terminal for the current available quota of the application process and checks whether the requested quota is less than or equal to the current available quota. If so, it calls the real CUDA runtime library to perform the allocation. If not, it returns a CUDA standard out-of-memory error code and does not perform the allocation.
[0034] Here, when an application process requests video memory allocation, the runtime interception library first obtains the current available quota from the management end before calling the actual CUDA runtime library. If the requested amount is greater than the available quota, the runtime interception library directly returns the runtime standard allocation failure error code and does not proceed to the actual allocation. If the requested amount is not greater than the available quota, the runtime interception library calls the actual CUDA runtime library to perform the allocation.
[0035] It should be noted that the actual CUDA runtime library is the `libcudart.so` dynamic link library provided by NVIDIA CUDA Toolkit v11.0 or later; the runtime interception library parses and calls the corresponding functions in this library using the `dlsym(RTLD_NEXT, "function name")` method.
[0036] In one possible implementation, after successful allocation, the GPU resource isolation method further includes: establishing a mapping relationship between the allocated device pointer and the current allocation amount and storing it in a local mapping table, and sending an accounting request to the management terminal to update the tenant's used amount.
[0037] Here, when the device pointer is returned after a successful allocation, the runtime interception library writes the device pointer and the amount allocated this time into the local mapping table (for obtaining the size when releasing), and reports the "accounting event" to the management terminal to update the used quota.
[0038] In this application, the device pointer-allocation amount mapping and the accounting / debiting mechanism can accurately recover quotas upon release, avoiding quota vacancy and resource waste caused by meter drift after long-term operation.
[0039] In one possible implementation, after sending the accounting request to the management terminal, the GPU resource isolation method further includes: when receiving feedback that the accounting request has been rejected by the management terminal, the runtime interception library performs a rollback operation.
[0040] Here, when a concurrent race condition causes the management to refuse to record the transaction, the runtime interception library performs a rollback: it calls the actual release and cleans up the mapping table entries, and then returns an allocation failure error code to the application process.
[0041] It should be noted that the process for handling refused billing is the same as the process for handling refused billing described above.
[0042] In one possible implementation, the GPU resource isolation method further includes, when the video memory release interface is invoked: When the application process receives a request to call the video memory release interface, the runtime interception library determines the corresponding allocation amount in the local mapping table based on the device pointer to be released, deletes the corresponding mapping entry, sends an billing request to the management terminal, and calls the real CUDA runtime library to perform the release process.
[0043] Here, when an application process requests to release a device pointer, the runtime interception library queries the mapping table for the allocated amount corresponding to that pointer, deletes the entry, and reports a "billing event" to the management end to reduce the used quota. Subsequently, it calls the actual runtime library to complete the release and returns the actual error code. This "device pointer-allocated amount" mapping achieves precise reclamation during the release phase, preventing quota drift.
[0044] In one possible implementation, the GPU resource isolation method further includes, for the purpose of invoking the Kernel launch interface: The runtime interception library determines whether to apply for a launch license based on a preset strategy and the current quota status before the corresponding function in the real CUDA runtime library. If so, it synchronously launches a license token to the management terminal. After obtaining the license granted by the management terminal, it calls the real CUDA runtime library to perform kernel launch processing.
[0045] Here, when the application process calls the Kernel launch interface, the runtime interception library determines whether a new launch license token or time quota needs to be requested before launch, based on the policy and the current quota status. If so, the runtime interception library synchronously requests a license from the management end. Only after the license is successfully granted does the runtime interception library call the actual launch function to complete the launch.
[0046] It should be noted that the preset strategy refers to a set of structured control identifiers loaded and persisted according to preset configuration parameters during the runtime interception library initialization phase. The value of the identifier directly determines whether the interception library performs quota awareness, view rewriting, accounting entry and exit, or license verification for specific behaviors in the CUDARuntime interface (especially memory query, allocation, release, and Kernel emission). The switches are orthogonal to each other and take effect independently, together forming the programmable execution surface of the multi-tenant GPU resource isolation strategy. The default policy must include at least a LaunchPolicyDescriptor (LPD), which contains at least the following fields: `policy_type`: Policy type identifier, such as `"token_based"` (token bucket), `"time_quota"` (time slice quota), `"burst_cap"` (burst cap), `"priority_gated"` (priority gated), etc.; `enforcement_mode`: Enforcement mode, with values such as `"strict"` (strict blocking), `"adaptive"` (adaptive degradation), `"monitor_only"` (monitor only), etc.
[0047] In one possible implementation, when the management terminal is unavailable or communication fails, the runtime interception library performs any of the following operations according to a preset degradation strategy: maintain only video memory quota control, reject all kernel launch requests, and enable a local caching strategy to allow a limited number of launches based on the most recent valid license.
[0048] This application provides a GPU resource isolation method based on interface interception. The GPU resource isolation method includes: injecting a runtime interception library into the address space of the application process through a preloading mechanism before the application process starts; wherein the CUDA Runtime API configured in the runtime interception library includes a memory query interface, a memory allocation interface, a memory release interface, and a kernel emission interface; when a request from the application process to call the memory query interface is received, the runtime interception library queries the management terminal for the memory quota information of the current tenant on the target GPU device, and corrects the total memory and available memory parameters returned by the memory query interface based on the memory quota information, thereby constructing a memory observation view constrained by the quota; when a request from the application process to call the memory allocation interface is received, the runtime interception library performs a pre-allocation verification process before calling the corresponding function of the real CUDA runtime library to determine whether to allocate memory to the application process. By preloading interception, "runtime entry point controllability" is achieved, reducing the dependence on the consistency of the underlying call chain. Furthermore, the memory query interface is intercepted and rewritten to construct a memory view subject to quota constraints, thereby forcing the application's observations to be consistent with the allocation.
[0049] Please see Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of a GPU resource isolation device based on interface interception provided in an embodiment of this application; Figure 3 This is a second schematic diagram of a GPU resource isolation device based on interface interception, provided as an embodiment of this application. Figure 2 As shown, the GPU resource isolation device 200 includes: The preloading module 210 is used to inject the runtime interception library into the address space of the application process through a preloading mechanism before the application process starts; wherein the runtime interception library is configured with a CUDA Runtime API including a memory query interface, a memory allocation interface, a memory release interface, and a kernel emission interface. The video memory query module 220 is used to query the management terminal for the video memory quota information of the current tenant on the target GPU device when the application process calls the video memory query interface. Based on the video memory quota information, the total video memory and available amount parameters returned by the video memory query interface are corrected to construct a video memory observation view constrained by the quota. The video memory allocation module 230 is used to determine whether to allocate video memory to the application process when the application process calls the video memory allocation interface. This is done by the runtime interception library before calling the corresponding function of the real CUDA runtime library.
[0050] Furthermore, the video memory allocation module 230 performs a pre-allocation verification process through the following steps: The system queries the management terminal for the current available quota of the application process and checks whether the requested quota is less than or equal to the current available quota. If so, the allocation is performed by calling the actual CUDA runtime library; If not, a CUDA standard out-of-memory error code is returned and no allocation is performed.
[0051] Furthermore, such as Figure 3 As shown, the GPU resource isolation device 200 also includes an update module 240, which is used for: The system establishes a mapping relationship between the allocated device pointer and the current allocation amount and stores it in a local mapping table, and sends an accounting request to the management terminal to update the tenant's usage.
[0052] Furthermore, such as Figure 3 As shown, the GPU resource isolation device 200 also includes a rollback execution module 250, which is used for: When the system receives feedback that the payment request has been rejected by the management, the runtime interception library performs a rollback operation.
[0053] Furthermore, such as Figure 3 As shown, the GPU resource isolation device 200 also includes a video memory release module 260, which is used for: When the application process receives a request to call the video memory release interface, the runtime interception library determines the corresponding allocation amount in the local mapping table based on the device pointer to be released, deletes the corresponding mapping entry, sends an billing request to the management terminal, and calls the real CUDA runtime library to perform the release process.
[0054] Furthermore, such as Figure 3 As shown, the GPU resource isolation device 200 also includes a kernel launch module 270, which is used for: The runtime interception library determines whether to apply for a launch license based on a preset strategy and the current quota status, before the corresponding function in the real CUDA runtime library. If so, a permission token is synchronously sent to the management terminal. After obtaining the permission granted by the management terminal, the actual CUDA runtime library is invoked to perform the kernel emission process.
[0055] This application provides a GPU resource isolation device based on interface interception. The GPU resource isolation device includes: a preloading module, used to inject a runtime interception library into the address space of the application process before the application process starts through a preloading mechanism; wherein the CUDA Runtime API configured in the runtime interception library includes a memory query interface, a memory allocation interface, a memory release interface, and a kernel emission interface; a memory query module, used to query the management terminal for the memory quota information of the current tenant on the target GPU device when a request from the application process to call the memory query interface is received, and to correct the total memory and available memory parameters returned by the memory query interface based on the memory quota information, thereby constructing a memory observation view constrained by the quota; and a memory allocation module, used to perform a pre-allocation verification process before calling the corresponding function of the real CUDA runtime library when a request from the application process to call the memory allocation interface is received, to determine whether to allocate memory to the application process. By preloading interception, "runtime entry point controllability" is achieved, reducing the dependence on the consistency of the underlying call chain. Furthermore, the memory query interface is intercepted and rewritten to construct a memory view subject to quota constraints, thereby forcing the application's observations to be consistent with the allocation.
[0056] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.
[0057] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the GPU resource isolation method based on interface interception in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.
[0058] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the GPU resource isolation method based on interface interception in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.
[0059] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0060] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0061] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0062] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0063] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0064] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A GPU resource isolation method based on interface interception, characterized in that, The GPU resource isolation method includes: Before the application process starts, a runtime interception library is injected into the address space of the application process through a preloading mechanism; wherein, the GPU runtime interface configured for the runtime interception library includes a memory query interface, a memory allocation interface, a memory release interface, and a kernel launch interface. When the application process receives a request to call the video memory query interface, the runtime interception library queries the management terminal for the video memory quota information of the current tenant on the target GPU device. Based on the video memory quota information, the total video memory and available amount parameters returned by the video memory query interface are corrected to construct a video memory observation view constrained by the quota. When a request is received from the application process to call the video memory allocation interface, the runtime interception library performs a pre-allocation verification process before calling the corresponding function of the GPU native runtime library to determine whether to allocate video memory to the application process.
2. The GPU resource isolation method according to claim 1, characterized in that, The pre-allocation verification process is carried out through the following steps: The system queries the management terminal for the current available quota of the application process and checks whether the requested quota is less than or equal to the current available quota. If so, the allocation is performed by calling the GPU's native runtime library; If not, return the GPU native computing platform standard out-of-memory error code and do not allocate memory.
3. The GPU resource isolation method according to claim 2, characterized in that, After successful allocation, the GPU resource isolation method further includes: The system establishes a mapping relationship between the allocated device pointer and the current allocation amount and stores it in a local mapping table, and sends an accounting request to the management terminal to update the tenant's usage.
4. The GPU resource isolation method according to claim 3, characterized in that, After sending the accounting request to the management terminal, the GPU resource isolation method further includes: When the system receives feedback that the payment request has been rejected by the management, the runtime interception library performs a rollback operation.
5. The GPU resource isolation method according to claim 1, characterized in that, The GPU resource isolation method further includes the following for when the video memory release interface is called: When the application process receives a request to call the video memory release interface, the runtime interception library determines the corresponding allocation in the local mapping table based on the device pointer to be released, deletes the corresponding mapping entry, sends an account release request to the management terminal, and calls the GPU native runtime library to perform the release process.
6. The GPU resource isolation method according to claim 1, characterized in that, The GPU resource isolation method further includes the following for invoking the Kernel emission interface: The runtime interception library determines whether to request a launch license based on a preset strategy and the current quota status, before the corresponding function in the GPU native runtime library. If so, a permission token is simultaneously sent to the management terminal. After obtaining permission from the management terminal, the GPU native runtime library is invoked to perform kernel launch processing.
7. The GPU resource isolation method according to claim 6, characterized in that, When the management terminal is unavailable or communication fails, the runtime interception library performs any of the following operations according to a preset degradation strategy: It maintains memory quota control, rejects all kernel launch requests, and enables local caching policies, allowing a limited number of launches based on the most recent valid license.
8. A GPU resource isolation device based on interface interception, characterized in that, The GPU resource isolation device includes: The preloading module is used to inject the runtime interception library into the address space of the application process before the application process starts through a preloading mechanism; wherein the runtime interception library is configured with a GPU runtime interface including a memory query interface, a memory allocation interface, a memory release interface, and a kernel launch interface. The video memory query module is used to query the management terminal for the video memory quota information of the current tenant on the target GPU device when the application process calls the video memory query interface. Based on the video memory quota information, the total video memory and available amount parameters returned by the video memory query interface are corrected to construct a video memory observation view constrained by the quota. The video memory allocation module is used to determine whether to allocate video memory to the application process when the application process calls the video memory allocation interface. Before calling the corresponding function of the GPU native runtime library, the runtime interception library performs a pre-allocation verification process.
9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the GPU resource isolation method based on interface interception as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the GPU resource isolation method based on interface interception as described in any one of claims 1 to 7.