A method and system for dividing GPU space resources based on persistent threads

Abstract

The application discloses a GPU space resource partitioning method and system based on a persistent thread, and belongs to the field of high-performance computing and computer system structure.The method extracts original PTX code from a binary module by dynamically intercepting a CUDA interface at a driver layer; a static compiling conversion technology is used to recombine an instruction stream, construct a persistent thread structure, and innovatively introduce a "kernel fusion" mechanism in the persistent thread; through instruction body multiplication and virtual index remapping, the static demand of a single physical thread block for registers and shared memory is artificially doubled, so that the single physical thread block reaches the upper limit of hardware resources of a streaming multiprocessor; finally, a kernel starting interface is intercepted, and the original kernel is transparently replaced by the converted kernel.The application effectively solves the defect that traditional persistent threads cannot realize strict physical exclusive on a streaming multiprocessor due to unsaturated resource occupation, and realizes fine-grained and deterministic partitioning and isolation of GPU space resources without modifying user source code.

Description

Technical Field

[0001] This invention belongs to the field of resource isolation and virtualization in computer system architecture, and specifically relates to a method and system for GPU space resource partitioning based on persistent threads. Background Technology

[0002] Graphics Processing Units (GPUs), as high-throughput parallel computing devices, consist of multiple Streaming Multiprocessors (SMs) as their core computing units. In mainstream general-purpose computing programming models such as CUDA, computational tasks are organized into a hierarchical structure composed of grids, thread blocks, and threads. When a kernel function is started on the host side, the GPU's internal hardware scheduler is responsible for distributing thread blocks from the grid to various SMs for execution. During this process, the hardware scheduler's resource allocation is opaque and uncontrollable. The scheduler dynamically determines which thread blocks are scheduled to which SM based primarily on the available resources of each SM (such as the number of register files, shared memory size, and thread slots) and the resource requirements of the thread blocks. More importantly, the GPU's execution model follows a non-preemptive "run-to-complete" mechanism. Once a thread block is scheduled to execute on an SM, it will occupy the SM's computational and storage resources for an extended period. Unless the thread block finishes execution and voluntarily exits, the hardware scheduler cannot forcibly reclaim its resources, and other waiting thread blocks cannot preempt the use of that SM.

[0003] Leveraging the non-preemptive characteristic of GPU thread blocks—"once uploaded, they are exclusively occupied"—existing research proposes software scheduling methods based on persistent threads (PT) to achieve GPU spatial resource isolation. Traditional persistent thread methods launch a limited number of thread blocks, keeping them resident on the task queue (SM) for extended periods. These resident thread blocks run an infinite loop, dynamically acquiring and executing logical tasks by atomically accessing the task queue in global memory. This approach essentially replaces the GPU hardware scheduler's task distribution with software-level task allocation, attempting to physically occupy a corresponding number of SM resources by controlling the number of launched physical thread blocks, thereby preventing interference from other tasks in a multi-tasking environment and achieving spatial isolation.

[0004] However, in practical applications, directly using traditional persistent threads for space resource allocation has significant limitations, primarily manifested in the isolation failure problem caused by "resource saturation." A streaming multiprocessor has a large register file and shared memory resources, while a single ordinary thread block written by the user often has relatively small resource requirements. According to the GPU hardware scheduling policy, if the resources on a single SM are sufficient to accommodate multiple thread blocks, the hardware scheduler will attempt to schedule multiple thread blocks to the same SM for parallel execution to mask memory access latency. Therefore, even if the developer starts N persistent thread blocks and attempts to occupy N SMs, because a single thread block cannot fill all the resources of an SM, the hardware scheduler may compress these N thread blocks and schedule them to fewer than N SMs (e.g., two thread blocks sharing one SM), or insert thread blocks of other uncontrolled tasks into the remaining resource gaps. This causes "logical SM exclusivity" to fail to translate into "physical SM exclusivity," undermining the determinism of space isolation. To solve this problem, a technique that can significantly increase the resource utilization of a single persistent thread is needed. By merging kernel threads, the computational tasks and resource requirements of multiple logical thread blocks are combined into a single physical thread block. This artificially multiplies the static requirements of this physical thread block for registers and shared memory, bringing them to or near the hardware resource limits of the SM (Memory Module). Thus, when this "kernel-merged persistent thread" is loaded into the SM, it physically excludes the coexistence of any other thread blocks, ensuring that starting N thread blocks precisely and exclusively occupies N SMs, achieving strict hardware-level space resource allocation. Summary of the Invention

[0005] The purpose of this invention is to provide a GPU space resource partitioning method and system based on persistent threads. It obtains the original kernel code by dynamically intercepting the CUDA driver interface, reassembles the instruction stream using static compilation and transformation technology to construct persistent threads, and innovatively introduces a "kernel fusion" mechanism to perform resource inflation processing on physical thread blocks. This solves the problem that traditional persistent thread methods have low resource utilization of a single thread block and cannot reach the hardware resource limit of streaming multiprocessors, which leads to the hardware scheduler compressing and scheduling multiple thread blocks on the same streaming multiprocessor or being unable to exclude interference from other tasks, thus making it difficult to achieve strict and deterministic physical space resource isolation.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A GPU space resource partitioning method based on persistent threads includes the following steps: Step 1: During program runtime, hijack the module loading function in the CUDA driver interface to intercept the original PTX code from the CUDA executable binary. Step 2: Perform static analysis and compilation conversion on the captured original PTX code. Based on the preset resource allocation strategy, convert the ordinary threads in the original PTX into ordinary persistent threads or kernel-fused persistent threads, and generate the modified PTX code. Step 3: Hijack the kernel startup function in the CUDA driver interface to prevent the original kernel from starting. Based on the current GPU resource status or user configuration, use the modified PTX code to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving the allocation of space resources for GPU streaming multiprocessors.

[0007] A further improvement of this invention is that, in step 1, the module loading function in the CUDA driver interface is hijacked during program execution to intercept the original PTX code from the CUDA executable binary, including: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing the CUDA binary data nvFatbin; read the header information of this segment, verify the magic number to confirm the data format; traverse each entry in nvFatbin, identify the text segment containing PTX assembly code; if the data segment is compressed, perform the decompression operation first, and finally obtain the unobfuscated, string-based original PTX code; Step 1.3 establishes a mapping relationship between the original kernel function names and the extracted original PTX code, and maintains a lookup tree for storing the transformed kernel functions. This mapping relationship enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel fusion persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

[0008] A further improvement of the present invention is that, in step 1.2, nvFatbin refers to a multi-architecture hybrid binary data container embedded in the ".nv_fatbin" segment of the host program executable file or dynamic link library; the memory layout of the ".nv_fatbin" segment begins with a Fat Binary Header structure containing a specific magic number; the multi-architecture hybrid binary data container encapsulates PTX intermediate code text or ELF format binary machine code generated for different GPU computing capabilities, aiming to enable a single program to be compatible with multiple GPU architectures.

[0009] A further improvement of this invention is that, in step 2, static analysis and compilation conversion are performed on the intercepted original PTX code. Based on a preset resource allocation strategy, ordinary threads in the original PTX are converted into ordinary persistent threads or kernel-fused persistent threads, and modified PTX code is generated, including: Step 2.1: Perform lexical analysis on the original PTX code string obtained in Step 1 to identify the entry instructions, parameter declarations, register declarations, and function body instruction segments of the kernel functions; parse the original kernel's hardware resource requirements, including the required shared memory size and the number of registers, and scan the code for instructions that reference hardware built-in variables. Step 2.2: Inject control parameters for persistent threads into the parameter list segment of the PTX code; at the same time, inject general-purpose registers for storing virtual block indexes and synchronization predicates into the register declaration segment. Step 2.3: Inject logic at the function entry point to obtain the task ID via addition instructions, and construct an infinite loop structure of "obtain task - perform calculation - check and exit"; use regular expressions to replace the instruction to read the hardware block index register in the original code with the instruction to read the software virtual register calculated from the global task ID, thereby decoupling physical scheduling from logical execution. Step 2.4: When generating persistent threads for kernel fusion, within the aforementioned persistent loop, the original kernel's computation instruction sequence is copied N times, where N≥2. In each copied instruction block, the virtual thread index is remapped through offset calculation, enabling a single physical thread block to continuously process the data of N logical thread blocks. This increases the resource density of a single physical thread block, forces the GPU hardware scheduler to reduce the number of thread blocks residing on the same streaming multiprocessor, and achieves resource exclusivity at the streaming multiprocessor level. Step 2.5: The compiled ordinary persistent thread and the kernel-fused persistent thread insert their handles into the maintained search tree.

[0010] A further improvement of this invention is that, in step 2.2, persistent thread refers to a software-defined thread execution mode that breaks through the default "start-execute-exit" lifecycle of the GPU hardware scheduler. In the software-defined thread execution mode, the physical thread block scheduled to the streaming multiprocessor does not exit immediately after completing the currently assigned logical task, but remains active through injected loop control instructions. The physical thread block continuously reuses registers and shared memory resources to process multiple logical tasks until all tasks are completed, thereby achieving long-term exclusive use and control of the physical computing resources of a specific streaming multiprocessor.

[0011] A further improvement of this invention is that, in step 2.4, kernel fusion refers to a static instruction stream reorganization technique; this technique merges and encapsulates the instruction sequences, register contexts, and shared memory declarations that originally belonged to N independent logical thread blocks into the instruction space of the same physical kernel function; by copying function bodies and remapping virtual indexes, the original N groups of logical thread indices are mapped to a unique physical thread space, and the address offsets of the dynamic shared memory required by each group of logical threads are accumulated, thereby generating a "super kernel" that is equivalent to the sum of N original kernels in terms of resource requirements; Kernel fusion persistent threads refer to a saturated resource-exclusive execution mode built using kernel fusion technology. In this mode, the launched physical thread block not only resides continuously in the streaming multiprocessor through a persistent loop, but also, due to its internal integration of N times the instruction body and resource requirements processed by kernel fusion, the register usage or shared memory occupancy of a single physical thread block reaches or approaches the hardware resource limit of the streaming multiprocessor. This extremely high resource density forces the GPU hardware scheduler to be unable to schedule any other thread blocks on the same streaming multiprocessor, thereby forcibly implementing a strict spatial isolation effect of "one streaming multiprocessor running only one specified thread block" at the physical level.

[0012] A further improvement of this invention is that, in step 3, the kernel startup function in the CUDA driver interface is hijacked to prevent the original kernel from starting. Based on the current GPU resource status or user configuration, the modified PTX code is used to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving the allocation of space resources for the GPU streaming multiprocessor, including: Step 3.1: After intercepting the cuLaunchKernel function call, use the original kernel function handle as the key to search for the compiled ordinary persistent thread and kernel-fused persistent thread handles in the maintained search tree; based on the resource availability of the GPU streaming multiprocessor, dynamically decide whether to use the ordinary persistent thread or the kernel-fused persistent thread for startup; Step 3.2: Allocate new memory space to build the modified parameter list; inject the original grid dimension and block dimension as ordinary scalar parameters into the new parameter list so that the persistent thread can rebuild the logical index at the software level; inject the global task counter pointer, total number of tasks and auxiliary state array pointer; if it is determined that the kernel fusion version is started, increase the dynamic shared memory request size to meet the superposition requirements of multiple logical thread blocks for shared memory after fusion. Step 3.3: Ignore the grid dimension in the original request, set the physical grid dimension to be started to the number of persistent threads to be started; use the new kernel handle selected in Step 3.1, the reorganized parameter list in Step 3.2, and the reset grid dimension to call the original cuLaunchKernel to complete the final kernel startup, so that the physical thread blocks reside precisely and persistently on the specified number of streaming multiprocessors.

[0013] A GPU space resource partitioning system based on persistent threads, comprising: The original PTX code interception unit hijacks the module loading function in the CUDA driver interface during program runtime and intercepts the original PTX code from the CUDA executable binary file; The original PTX processing unit performs static analysis and compilation transformation on the captured original PTX code. Based on the preset resource allocation strategy, it converts the ordinary threads in the original PTX into ordinary persistent threads or kernel-fused persistent threads and generates modified PTX code. The spatial resource partitioning unit hijacks the kernel startup function in the CUDA driver interface, preventing the original kernel from starting. Based on the current GPU resource status or user configuration, it uses modified PTX code to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving spatial resource partitioning for GPU streaming multiprocessors.

[0014] A further improvement of this invention is that, in the original PTX code interception unit, the module loading function in the CUDA driver interface is hijacked during program runtime to intercept the original PTX code from the CUDA executable binary file, including: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing the CUDA binary data nvFatbin; read the header information of this segment, verify the magic number to confirm the data format; traverse each entry in nvFatbin, identify the text segment containing PTX assembly code; if the data segment is compressed, perform the decompression operation first, and finally obtain the unobfuscated, string-based original PTX code; Step 1.3 establishes a mapping relationship between the original kernel function names and the extracted original PTX code, and maintains a lookup tree for storing the transformed kernel functions. This mapping relationship enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel fusion persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the GPU space resource partitioning method based on persistent threads.

[0016] Compared with the prior art, the present invention has at least the following beneficial technical effects: This invention provides a GPU space resource partitioning method and system based on persistent threads. By dynamically intercepting and parsing CUDA binary modules at the driver layer and using just-in-time (JIT) compilation technology to statically reorganize the PTX instruction stream, it innovatively introduces a kernel fusion mechanism into persistent threads. Through instruction body multiplication and virtual index remapping, it multiplies the static demand of a single physical thread block on registers and shared memory, effectively solving the defect of traditional persistent threads that cannot achieve strict physical exclusivity on streaming multiprocessors due to insufficient resource saturation. This saturation-style resource saturation strategy forces the GPU hardware scheduler to precisely map and lock physical thread blocks onto a specified number of streaming multiprocessors, thereby achieving fine-grained and deterministic partitioning of GPU space resources without modifying user source code or recompiling the application. This eliminates resource fragmentation and mutual interference in multi-task concurrent environments, significantly improving the computational stability and service quality of critical tasks. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method of the present invention.

[0019] Figure 2 This is a structural block diagram of the system of the present invention. Detailed Implementation

[0020] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0021] In the description of this invention, it should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0025] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] Example 1 This invention provides a GPU space resource allocation method based on persistent threads, comprising the following steps: Step 1: During program runtime, hijack the module loading function in the CUDA driver interface to intercept the original PTX code from the CUDA executable binary. Step 2: Perform static analysis and compilation conversion on the captured original PTX code. Based on the preset resource allocation strategy, convert the ordinary threads in the original PTX into ordinary persistent threads or kernel-fused persistent threads, and generate the modified PTX code. Step 3: Hijack the kernel startup function in the CUDA driver interface to prevent the original kernel from starting. Based on the current GPU resource status or user configuration, use the modified PTX code to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving the allocation of space resources for GPU streaming multiprocessors.

[0027] In step 1 of this embodiment, the module loading function in the CUDA driver interface is hijacked during program runtime to intercept the original PTX code from the CUDA executable binary, including: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing the CUDA binary data nvFatbin; read the header information of this segment, verify the magic number to confirm the data format; traverse each entry in nvFatbin, identify the text segment containing PTX assembly code; if the data segment is compressed, perform the decompression operation first, and finally obtain the unobfuscated, string-based original PTX code; Step 1.3 establishes a mapping relationship between the original kernel function names and the extracted original PTX code, and maintains a lookup tree for storing the transformed kernel functions. This mapping relationship enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel fusion persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

[0028] In step 1.2 of this embodiment, nvFatbin refers to a multi-architecture hybrid binary data container embedded in the ".nv_fatbin" segment of the host program executable file or dynamic link library; the memory layout of the ".nv_fatbin" segment starts with a Fat Binary Header structure containing a specific magic number; the multi-architecture hybrid binary data container encapsulates PTX intermediate code text or ELF format binary machine code generated for different GPU computing capabilities, aiming to enable a single program to be compatible with multiple GPU architectures.

[0029] In step 2 of this embodiment, static analysis and compilation conversion are performed on the intercepted original PTX code. According to a preset resource allocation strategy, ordinary threads in the original PTX are converted into ordinary persistent threads or kernel-fused persistent threads, and modified PTX code is generated, including: Step 2.1: Perform lexical analysis on the original PTX code string obtained in Step 1 to identify the entry instructions, parameter declarations, register declarations, and function body instruction segments of the kernel functions; parse the original kernel's hardware resource requirements, including the required shared memory size and the number of registers, and scan the code for instructions that reference hardware built-in variables. Step 2.2: Inject control parameters for persistent threads into the parameter list segment of the PTX code; at the same time, inject general-purpose registers for storing virtual block indexes and synchronization predicates into the register declaration segment. Step 2.3: Inject logic at the function entry point to obtain the task ID via addition instructions, and construct an infinite loop structure of "obtain task - perform calculation - check and exit"; use regular expressions to replace the instruction to read the hardware block index register in the original code with the instruction to read the software virtual register calculated from the global task ID, thereby decoupling physical scheduling from logical execution. Step 2.4: When generating persistent threads for kernel fusion, within the aforementioned persistent loop, the original kernel's computation instruction sequence is copied N times, where N≥2. In each copied instruction block, the virtual thread index is remapped through offset calculation, enabling a single physical thread block to continuously process the data of N logical thread blocks. This increases the resource density of a single physical thread block, forces the GPU hardware scheduler to reduce the number of thread blocks residing on the same streaming multiprocessor, and achieves resource exclusivity at the streaming multiprocessor level. Step 2.5: The compiled ordinary persistent thread and the kernel-fused persistent thread insert their handles into the maintained search tree.

[0030] In step 2.2 of this embodiment, persistent thread refers to a software-defined thread execution mode that breaks through the default "start-execute-exit" lifecycle of the GPU hardware scheduler. In the software-defined thread execution mode, the physical thread block scheduled to the streaming multiprocessor does not exit immediately after completing the currently assigned logical task, but remains active through injected loop control instructions. The physical thread block continuously reuses registers and shared memory resources to process multiple logical tasks until all tasks are completed, thereby achieving long-term exclusive use and control of the physical computing resources of a specific streaming multiprocessor.

[0031] In step 2.4 of this embodiment, kernel fusion refers to a static instruction stream reorganization technique. This technique merges and encapsulates the instruction sequences, register contexts, and shared memory declarations that originally belonged to N independent logical thread blocks into the instruction space of the same physical kernel function. By copying function bodies and remapping virtual indexes, the original N groups of logical thread indices are mapped to a unique physical thread space, and the address offsets of the dynamic shared memory required by each group of logical threads are accumulated, thereby generating a "super kernel" that is equivalent to the sum of N original kernels in terms of resource requirements. Kernel fusion persistent threads refer to a saturated resource-exclusive execution mode built using kernel fusion technology. In this mode, the launched physical thread block not only resides continuously in the streaming multiprocessor through a persistent loop, but also, due to its internal integration of N times the instruction body and resource requirements processed by kernel fusion, the register usage or shared memory occupancy of a single physical thread block reaches or approaches the hardware resource limit of the streaming multiprocessor. This extremely high resource density forces the GPU hardware scheduler to be unable to schedule any other thread blocks on the same streaming multiprocessor, thereby forcibly implementing a strict spatial isolation effect of "one streaming multiprocessor running only one specified thread block" at the physical level.

[0032] In step 3 of this embodiment, the kernel startup function in the CUDA driver interface is hijacked to prevent the original kernel from starting. Based on the current GPU resource status or user configuration, the modified PTX code is used to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving the allocation of space resources for the GPU streaming multiprocessor, including: Step 3.1: After intercepting the cuLaunchKernel function call, use the original kernel function handle as the key to search for the compiled ordinary persistent thread and kernel-fused persistent thread handles in the maintained search tree; based on the resource availability of the GPU streaming multiprocessor, dynamically decide whether to use the ordinary persistent thread or the kernel-fused persistent thread for startup; Step 3.2: Allocate new memory space to build the modified parameter list; inject the original grid dimension and block dimension as ordinary scalar parameters into the new parameter list so that the persistent thread can rebuild the logical index at the software level; inject the global task counter pointer, total number of tasks and auxiliary state array pointer; if it is determined that the kernel fusion version is started, increase the dynamic shared memory request size to meet the superposition requirements of multiple logical thread blocks for shared memory after fusion. Step 3.3: Ignore the grid dimension in the original request, set the physical grid dimension to be started to the number of persistent threads to be started; use the new kernel handle selected in Step 3.1, the reorganized parameter list in Step 3.2, and the reset grid dimension to call the original cuLaunchKernel to complete the final kernel startup, so that the physical thread blocks reside precisely and persistently on the specified number of streaming multiprocessors.

[0033] Example 2 This invention provides a GPU space resource partitioning method based on persistent threads. By dynamically intercepting and parsing CUDA binary modules at the driver layer and using just-in-time (JIT) compilation technology to statically reorganize the PTX instruction stream, it innovatively introduces a kernel fusion mechanism into persistent threads. Through instruction body multiplication and virtual index remapping, it multiplies the static demand of a single physical thread block on registers and shared memory, effectively solving the defect of traditional persistent threads that cannot achieve strict physical exclusivity on streaming multiprocessors due to insufficient resource saturation. This saturation-style resource saturation strategy forces the GPU hardware scheduler to precisely map and lock physical thread blocks to a specified number of streaming multiprocessors, thereby achieving fine-grained and deterministic partitioning of GPU space resources without modifying user source code or recompiling the application. This eliminates resource fragmentation and mutual interference in multi-task concurrent environments, significantly improving the computational stability and service quality of critical tasks.

[0034] Please see Figure 1 The present invention provides a GPU space resource allocation method based on persistent threads, comprising the following steps: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing CUDA binary data (nvFatbin). Read the header information of this segment and verify the magic number to confirm the data format. Traverse each entry in nvFatbin to identify the text segment containing PTX assembly code. If the data segment is compressed, perform decompression first, ultimately obtaining the unobfuscated, string-based raw PTX code.

[0035] Step 1.3 establishes a mapping between the original kernel function names and the extracted PTX code, and maintains a lookup tree to store the transformed kernel functions. This mapping enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel-fused persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

[0036] PTX (Parallel Thread Execution) refers to a low-level virtual machine and instruction set architecture defined by NVIDIA for parallel computing. As an intermediate representation (IR), it is independent of specific GPU hardware generations and describes the entry point, parameter list, register declaration, memory model, and computational logic of GPU kernel functions in the form of text assembly instructions. During the execution of CUDA programs, PTX code is just-in-time (JIT) compiled by the GPU driver into target hardware-specific executable binary machine code (SASS).

[0037] nvFatbin refers to a multi-architecture hybrid binary data container embedded in the ".nv_fatbin" segment of the host program's executable file or dynamic link library. The memory layout of the ".nv_fatbin" segment begins with a Fat Binary Header structure containing a specific magic number (0xBA55ED50). The multi-architecture hybrid binary data container encapsulates PTX intermediate code text or ELF format binary machine code generated for different GPU computing capabilities, aiming to enable a single program to be compatible with multiple GPU architectures.

[0038] Step 2.1: Perform lexical analysis on the PTX code string obtained in Step 1.2 to identify the kernel function's entry instruction (.entry), parameter declaration (.param), register declaration (.reg), and function body instruction segment; parse the original kernel's hardware resource requirements, including the required shared memory size (.shared) and the number of registers, and scan the code for reference instructions to hardware built-in variables (such as %ctaid, %tid, %ntid); Step 2.2: Inject control parameters for persistent threads into the parameter list segment of the PTX code; at the same time, inject general-purpose registers for storing virtual block indexes and synchronization predicates into the register declaration segment. Step 2.3: Inject logic at the function entry point to obtain the task ID via addition instructions, and construct an infinite loop structure of "obtain task - perform calculation - check and exit"; use regular expressions to replace the instruction in the original code that reads the hardware block index register (%ctaid.x / y / z) with the instruction that reads the software virtual register calculated from the global task ID, thereby decoupling physical scheduling from logical execution; Step 2.4: When generating persistent threads for kernel fusion, the original kernel's computation instruction sequence is copied N times (N≥2) within the aforementioned persistent loop. In each copied instruction block, the virtual thread index is remapped through offset calculation, enabling a single physical thread block to continuously process the data of N logical thread blocks. This increases the resource density of a single physical thread block, forces the GPU hardware scheduler to reduce the number of thread blocks residing on the same streaming multiprocessor, and achieves resource exclusivity at the streaming multiprocessor level.

[0039] Step 2.5: The handles of the compiled ordinary persistent threads and kernel-fused persistent threads are inserted into the search tree maintained in Step 1.

[0040] Persistent threads refer to a software-defined thread execution mode that breaks through the default "start-execute-exit" lifecycle of the GPU hardware scheduler. In this mode, the physical thread block scheduled to the streaming multiprocessor does not exit immediately after completing the currently assigned logical task, but remains active through injected loop control instructions. The physical thread block continuously reuses registers and shared memory resources to process multiple logical tasks until all tasks are completed, thereby achieving long-term exclusive use and control of the physical computing resources of a specific streaming multiprocessor.

[0041] Kernel fusion refers to a static instruction stream reconfiguration technique. This technique merges and encapsulates the instruction sequences, register contexts, and shared memory declarations that originally belonged to N (N≥2) independent logical thread blocks into the instruction space of the same physical kernel function. By copying function bodies and remapping virtual indexes, the original N groups of logical thread indices are mapped to a unique physical thread space, and the address offsets of the dynamic shared memory required by each group of logical threads are accumulated, thereby generating a "super kernel" that is equivalent to the sum of the N original kernels in terms of resource requirements.

[0042] Kernel fusion persistent threads refer to a saturated resource-exclusive execution mode built using kernel fusion technology. In this mode, the launched physical thread block not only resides continuously in the streaming multiprocessor through a persistent loop, but also, due to its internal integration of N times the instruction body and resource requirements processed by kernel fusion, the register usage or shared memory occupancy of a single physical thread block reaches or approaches the hardware resource limit of the streaming multiprocessor. This extremely high resource density forces the GPU hardware scheduler to be unable to schedule any other thread blocks on the same streaming multiprocessor, thereby forcibly implementing a strict spatial isolation effect of "one streaming multiprocessor running only one specified thread block" at the physical level.

[0043] Step 3.1: After intercepting the cuLaunchKernel function call, use the original kernel function handle as the key to search for the compiled ordinary persistent thread and kernel-fused persistent thread handles in the search tree maintained in step 1.3; based on the resource availability of the GPU streaming multiprocessor, dynamically decide whether to use the ordinary persistent thread or the kernel-fused persistent thread for startup; Step 3.2: Allocate new memory space to construct the modified parameter list; inject the original grid dimensions (GridDim X / Y / Z) and block dimensions (BlockDim) as ordinary scalar parameters into the new parameter list so that the persistence thread can rebuild the logical index at the software level; inject the global task counter pointer, total number of tasks, and auxiliary state array pointer; if it is determined that the kernel fusion version is being started, increase the dynamic shared memory request size to meet the superposition requirements of multiple logical thread blocks for shared memory after fusion; Step 3.3: Ignore the grid dimension in the original request, set the physical grid dimension to be started to the number of persistent threads to be started; use the new kernel handle selected in Step 3.1, the reorganized parameter list in Step 3.2, and the reset grid dimension to call the original cuLaunchKernel to complete the final kernel startup, so that the physical thread blocks reside precisely and persistently on the specified number of streaming multiprocessors.

[0044] After completing the above steps, GPU space resources can be precisely and finely divided. Subsequently, a task manager can be added according to application needs to dynamically adjust the number of persistent threads launched at runtime. This concludes the GPU space resource allocation method based on persistent threads.

[0045] Example 3 Please see Figure 2 The present invention provides a GPU space resource partitioning system based on persistent threads, comprising: The original PTX code interception unit hijacks the module loading function in the CUDA driver interface during program runtime and intercepts the original PTX code from the CUDA executable binary file; The original PTX processing unit performs static analysis and compilation transformation on the captured original PTX code. Based on the preset resource allocation strategy, it converts the ordinary threads in the original PTX into ordinary persistent threads or kernel-fused persistent threads and generates modified PTX code. The spatial resource partitioning unit hijacks the kernel startup function in the CUDA driver interface, preventing the original kernel from starting. Based on the current GPU resource status or user configuration, it uses modified PTX code to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving spatial resource partitioning for GPU streaming multiprocessors.

[0046] In the original PTX code interception unit of this embodiment, the module loading function in the CUDA driver interface is hijacked during program runtime to intercept the original PTX code from the CUDA executable binary, including: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing the CUDA binary data nvFatbin; read the header information of this segment, verify the magic number to confirm the data format; traverse each entry in nvFatbin, identify the text segment containing PTX assembly code; if the data segment is compressed, perform the decompression operation first, and finally obtain the unobfuscated, string-based original PTX code; Step 1.3 establishes a mapping relationship between the original kernel function names and the extracted original PTX code, and maintains a lookup tree for storing the transformed kernel functions. This mapping relationship enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel fusion persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

[0047] Example 4 The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a GPU space resource partitioning method based on persistent threads.

[0048] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0049] This application is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0050] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0051] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0052] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0053] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A GPU space resource partitioning method based on persistent threads, characterized in that, Includes the following steps: Step 1: During program runtime, hijack the module loading function in the CUDA driver interface to intercept the original PTX code from the CUDA executable binary. Step 2: Perform static analysis and compilation conversion on the captured original PTX code. Based on the preset resource allocation strategy, convert the ordinary threads in the original PTX into ordinary persistent threads or kernel-fused persistent threads, and generate the modified PTX code. Step 3: Hijack the kernel startup function in the CUDA driver interface to prevent the original kernel from starting. Based on the current GPU resource status or user configuration, use the modified PTX code to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving the allocation of space resources for GPU streaming multiprocessors.

2. The GPU space resource allocation method based on persistent threads according to claim 1, characterized in that, In step 1, the module loading function in the CUDA driver interface is hijacked during program runtime to intercept the original PTX code from the CUDA executable binary, including: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing the CUDA binary data nvFatbin; read the header information of this segment, verify the magic number to confirm the data format; traverse each entry in nvFatbin, identify the text segment containing PTX assembly code; if the data segment is compressed, perform decompression first, and finally obtain the unobfuscated, string-based original PTX code; Step 1.3 establishes a mapping relationship between the original kernel function names and the extracted original PTX code, and maintains a lookup tree for storing the transformed kernel functions. This mapping relationship enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel fusion persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

3. The GPU space resource allocation method based on persistent threads according to claim 2, characterized in that, In step 1.2, nvFatbin refers to a multi-architecture hybrid binary data container embedded in the ".nv_fatbin" segment of the host program's executable file or dynamic link library; the memory layout of the ".nv_fatbin" segment begins with a Fat Binary Header structure containing a specific magic number; the multi-architecture hybrid binary data container internally encapsulates PTX intermediate code text or ELF format binary machine code generated for different GPU computing capabilities, aiming to enable a single program to be compatible with multiple GPU architectures.

4. The GPU space resource allocation method based on persistent threads according to claim 2, characterized in that, In step 2, static analysis and compilation transformation are performed on the captured original PTX code. Based on a preset resource allocation strategy, ordinary threads in the original PTX are converted into ordinary persistent threads or kernel-fused persistent threads, and modified PTX code is generated, including: Step 2.1: Perform lexical analysis on the original PTX code string obtained in Step 1 to identify the entry instructions, parameter declarations, register declarations, and function body instruction segments of the kernel functions; parse the original kernel's hardware resource requirements, including the required shared memory size and the number of registers, and scan the code for instructions that reference hardware built-in variables. Step 2.2: Inject control parameters for persistent threads into the parameter list segment of the PTX code; at the same time, inject general-purpose registers for storing virtual block indexes and synchronization predicates into the register declaration segment. Step 2.3: Inject logic at the function entry point to obtain the task ID via addition instructions, and construct an infinite loop structure of "obtain task - perform calculation - check and exit"; use regular expressions to replace the instruction to read the hardware block index register in the original code with the instruction to read the software virtual register calculated from the global task ID, thereby decoupling physical scheduling from logical execution. Step 2.4: When generating persistent threads for kernel fusion, within the aforementioned persistent loop, the original kernel's computation instruction sequence is copied N times, where N≥2. In each copied instruction block, the virtual thread index is remapped through offset calculation, enabling a single physical thread block to continuously process the data of N logical thread blocks. This increases the resource density of a single physical thread block, forces the GPU hardware scheduler to reduce the number of thread blocks residing on the same streaming multiprocessor, and achieves resource exclusivity at the streaming multiprocessor level. Step 2.5: The compiled ordinary persistent thread and the kernel-fused persistent thread insert their handles into the maintained search tree.

5. The GPU space resource allocation method based on persistent threads according to claim 4, characterized in that, In step 2.2, persistent thread refers to a software-defined thread execution mode that breaks through the default "start-execute-exit" lifecycle of the GPU hardware scheduler. In the software-defined thread execution mode, the physical thread block scheduled to the streaming multiprocessor does not exit immediately after completing the currently assigned logical task, but remains active through injected loop control instructions. The physical thread block continuously reuses registers and shared memory resources to process multiple logical tasks until all tasks are completed, thereby achieving long-term exclusive use and control of the physical computing resources of a specific streaming multiprocessor.

6. The GPU space resource allocation method based on persistent threads according to claim 4, characterized in that, In step 2.4, kernel fusion refers to a static instruction stream reorganization technique. This technique merges and encapsulates the instruction sequences, register contexts, and shared memory declarations that originally belonged to N independent logical thread blocks into the instruction space of the same physical kernel function. By copying function bodies and remapping virtual indexes, the original N groups of logical thread indices are mapped to a unique physical thread space, and the address offsets of the dynamic shared memory required by each group of logical threads are accumulated, thereby generating a "super kernel" that is equivalent to the sum of the N original kernels in terms of resource requirements. Kernel-fused persistent threads refer to a saturated resource-exclusive execution mode built using kernel fusion technology. In this mode, the started physical thread block not only resides in the streaming multiprocessor through a persistent loop, but also, due to the N times the instruction body and resource requirements processed by kernel fusion integrated inside, the register usage or shared memory occupation of a single physical thread block reaches or approaches the hardware resource limit of the streaming multiprocessor. This extremely high resource consumption density forces the GPU hardware scheduler to be unable to schedule any other thread blocks on the same streaming multiprocessor, thus physically enforcing a strict spatial isolation effect of "one streaming multiprocessor running only one specified thread block".

7. The GPU space resource allocation method based on persistent threads according to claim 1, characterized in that, In step 3, the kernel startup function in the CUDA driver interface is hijacked to prevent the original kernel from starting. Based on the current GPU resource status or user configuration, the modified PTX code loads and starts a regular persistent thread or a kernel-fused persistent thread, thereby achieving the allocation of space resources for the GPU streaming multiprocessor, including: Step 3.1: After intercepting the cuLaunchKernel function call, use the original kernel function handle as the key to search for the compiled ordinary persistent thread and kernel-fused persistent thread handles in the maintained search tree; based on the resource availability of the GPU streaming multiprocessor, dynamically decide whether to use the ordinary persistent thread or the kernel-fused persistent thread for startup; Step 3.2: Allocate new memory space to build the modified parameter list; inject the original grid dimension and block dimension as ordinary scalar parameters into the new parameter list so that the persistent thread can rebuild the logical index at the software level; inject the global task counter pointer, total number of tasks and auxiliary state array pointer; if it is determined that the kernel fusion version is started, increase the dynamic shared memory request size to meet the superposition requirements of multiple logical thread blocks for shared memory after fusion. Step 3.3: Ignore the grid dimension in the original request, set the physical grid dimension to be started to the number of persistent threads to be started; use the new kernel handle selected in Step 3.1, the reorganized parameter list in Step 3.2, and the reset grid dimension to call the original cuLaunchKernel to complete the final kernel startup, so that the physical thread blocks reside precisely and persistently on the specified number of streaming multiprocessors.

8. A GPU space resource partitioning system based on persistent threads, characterized in that, include: The original PTX code interception unit hijacks the module loading function in the CUDA driver interface during program runtime and intercepts the original PTX code from the CUDA executable binary file; The original PTX processing unit performs static analysis and compilation transformation on the captured original PTX code. Based on the preset resource allocation strategy, it converts the ordinary threads in the original PTX into ordinary persistent threads or kernel-fused persistent threads and generates modified PTX code. The spatial resource partitioning unit hijacks the kernel startup function in the CUDA driver interface, preventing the original kernel from starting. Based on the current GPU resource status or user configuration, it uses modified PTX code to load and start a normal persistent thread or a kernel-fused persistent thread, thereby achieving spatial resource partitioning for GPU streaming multiprocessors.

9. A GPU space resource partitioning system based on persistent threads according to claim 8, characterized in that, The original PTX code interception unit hijacks the module loading function in the CUDA driver interface during program runtime, intercepting the original PTX code from the CUDA executable binary, including: Step 1.1: Intercept the cuModuleGetFunction function call using dynamic link library injection or symbol table hooking techniques; Step 1.2: In the interception function, based on the passed module handle, traverse the loaded module images in the process's memory space, searching for the ".nv_fatbin" segment containing the CUDA binary data nvFatbin; read the header information of this segment, verify the magic number to confirm the data format; traverse each entry in nvFatbin, identify the text segment containing PTX assembly code; if the data segment is compressed, perform decompression first, and finally obtain the unobfuscated, string-based original PTX code; Step 1.3 establishes a mapping relationship between the original kernel function names and the extracted original PTX code, and maintains a lookup tree for storing the transformed kernel functions. This mapping relationship enables the subsequent interception of cuLaunchKernel to quickly index the corresponding pre-compiled ordinary persistent thread kernel or kernel fusion persistent thread kernel based on the original kernel function handle, thereby achieving transparent replacement.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a GPU space resource partitioning method based on persistent threads according to any one of claims 1-7.

Images