GPU parallel thread block adjustment and kernel function scheduling optimization method and system

Abstract

The invention discloses a GPU (Graphics Processing Unit) parallel thread block adjustment and kernel function scheduling optimization method and system. The method comprises the following steps: acquiring GPU equipment attributes and kernel function execution attributes; determining a feasible thread block size solution space of each kernel function based on GPU equipment attributes and kernel function execution attributes, and selecting a thread block size which enables the load balance degree evaluation model value to be maximum as optimal execution configuration of the kernel function; on the basis of the determined optimal thread block size of each kernel function, dividing an execution grid of each kernel function into a plurality of kernel function slices, and establishing a concurrent execution relationship between the kernel functions; a CUDA graph is constructed in a mixed mode according to the established concurrent execution relation between the kernel functions and data dependence; and by taking an actual model of particle simulation software as a test object, respectively comparing GPU occupancy rates before and after thread block adaptive adjustment, multi-stream concurrent scheduling and hybrid CUDA graph scheduling optimization, kernel function execution time and overall simulation efficiency, and outputting an optimized particle simulation result.

Description

Technical Field

[0001] This invention belongs to the field of GPU optimization technology, and specifically relates to a method and system for GPU parallel thread block adjustment and kernel function scheduling optimization. Background Technology

[0002] High-power microwave (HPM) devices can generate, carry, and transmit microwave signals at high power levels, playing an irreplaceable and crucial role in many cutting-edge fields such as national defense, aerospace, and high-energy physics. However, the internal physical processes of HPM devices are extremely complex, exhibiting highly nonlinear characteristics. Traditional theoretical analysis alone is insufficient to fully and accurately explain their internal physical mechanisms; and accurately diagnosing the coupling mechanism of the electron beam and electromagnetic field within the device through experimental methods faces numerous difficulties. Against this backdrop, first-principles particle simulation methods have become a core technical means to solve this problem. By discretizing and coupling the Maxwell equations with the Newton-Lorentz equations, particle simulation methods can accurately reproduce the trajectory of charged particles under the influence of a self-consistent electromagnetic field, simulating complex physical processes whose analytical formulas are difficult to derive, providing a powerful tool for revealing the complex physical processes within HPM devices. After years of development, particle simulation methods have become one of the key tools for plasma research and understanding, and are widely used in defense fields such as controlled nuclear fusion, vacuum electronics, and cryogenic plasma.

[0003] With the development of HPM technology, researchers have an increasingly urgent need for performance optimization and the development of novel HPM devices, leading to a rapid increase in the computational complexity of particle simulation methods. Simulations using traditional CPU (Central Processing Unit) architectures often take weeks to simulate a large-scale HPM device, severely hindering research progress and failing to meet researchers' demands for rapid computation. my country has long faced strict technological blockades from Western countries in the field of high-end electromagnetic simulation software, resulting in a very challenging development situation. In 2016, the Institute of Plasma and Microwave Electronics at Xi'an Jiaotong University successfully developed XEMPIC, a full electromagnetic particle simulation software employing conformal mesh generation technology, based on the UNIPIC software. However, the current solution still suffers from low parallelism and uneven resource utilization, leaving significant room for optimization. Based on the above situation, this paper takes XEMPIC software as the research object, analyzes the bottlenecks in the existing architecture, and constructs a GPU (Graphics Processing Unit) based parallel thread block adjustment and kernel function scheduling mechanism, which significantly improves the GPU parallel computing efficiency of XEMPIC software, provides faster and more powerful support for scientific research, and helps my country break through the technological blockade in high-power microwave devices and related national defense high-tech fields, and achieve independent innovation and development.

[0004] Traditional particle simulation methods have significant shortcomings, specifically: 1. The CPU uses a serial architecture, which has a small number of cores, limited parallel computing power, low memory bandwidth, and high data transmission latency. It is difficult to support the massive parallel computing tasks of large-scale simulation, resulting in slow simulation speed, low efficiency, and susceptibility to computational bottlenecks, thus failing to meet the requirements of efficient simulation.

[0005] 2. The traditional method of using SM (Streaming Multiprocessor) utilization to reflect GPU load status is not entirely reasonable. SM utilization is a sufficient but not necessary condition for GPU load balancing and cannot characterize global load balancing characteristics. The fixed thread block size allocation strategy is prone to causing idle and wasted thread block resources, especially for memory-restricted kernel functions, which will also cause a significant increase in runtime. Ultimately, this limits the overall computing performance in heterogeneous computing environments and makes it impossible to achieve optimal execution efficiency for each kernel function.

[0006] 3. The default streaming execution mode is difficult to adapt to the computational characteristics and resource requirements of different kernel functions. The sequential scheduling of functions results in limited GPU utilization, low parallelism, and an inability to fully utilize the interdependencies between CUDA streams and kernel functions. Kernel function execution overhead is high, and the scheduling flexibility between tasks is poor. 4. The immediate kernel function startup mode is adopted. Every time the <<<>>> syntax is called, the conversion from Runtime API to Driver API is triggered, which introduces a significant scheduling delay. The kernel function startup overhead is large and the efficiency is low. In recent years, researchers have attempted to migrate particle simulation processes from CPUs to GPUs using NVIDIA's CUDA platform to achieve corresponding optimization designs, and have achieved some success. Traditional particle simulation methods mainly involve simply translating and migrating CPU code to GPU code, hoping to obtain accurate simulation results solely through the powerful parallel computing capabilities of the GPU.

[0007] As the PIC algorithm continues to develop, the calculations involved are becoming increasingly complex, and the requirements for computing power are becoming increasingly higher. In traditional particle simulation methods, when faced with the need to accurately simulate a large-sized device, the use of simple GPU parallel simulation methods has improved the running efficiency, but it has also resulted in a large amount of unreasonable allocation and idle waste of GPU resources. This has greatly increased the research cost and has also limited the research efficiency of researchers to a certain extent. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for GPU parallel thread block adjustment and kernel function scheduling optimization to solve the above-mentioned problems.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for GPU parallel thread block adjustment and kernel function scheduling optimization, comprising: Obtain GPU device properties and kernel function execution properties; Based on GPU device attributes and kernel function execution attributes, and in accordance with the triple constraints of thread bundle alignment, total register limit, and total shared memory limit, the feasible thread block size solution space for each kernel function is determined. The thread block size that maximizes the load balancing evaluation model value is selected as the optimal execution configuration for that kernel function. Based on the determined optimal thread block size for each kernel function, the execution grid of each kernel function is divided into multiple kernel function slices, establishing the concurrent execution relationship between kernel functions; Based on the established concurrent execution relationships and data dependencies among kernel functions, a CUDA graph is constructed using a hybrid mode. Using the actual model of the particle simulation software as the test object, the GPU utilization, kernel function execution time, and overall simulation efficiency are compared before and after optimization by thread block adaptive adjustment, multi-stream concurrent scheduling, and hybrid CUDA graph scheduling. The optimized particle simulation results are then output.

[0010] Furthermore, obtaining GPU device attributes and kernel function execution attributes includes: The device attributes such as the number of SMs, the maximum number of thread blocks per SM, and the warp size are obtained through cudaDeviceProps. The kernel attributes such as register usage and shared memory usage per thread of the kernel function are extracted using cudaFuncGetAttributes.

[0011] Furthermore, the constraints based on GPU device attributes and kernel function execution attributes, and according to the triple constraints of thread bundle alignment, total register limit, and total shared memory limit, include: Thread bundle alignment ensures that the size of the thread block must be an integer multiple of the warp size; register limitation ensures that the total number of registers used by the thread block cannot exceed the device limit; shared memory limitation ensures that the shared memory used by the thread block cannot exceed the device limit. Based on the above three conditions, the thread block solution space of this kernel function is initialized, and the optimal load balancing degree and optimal thread block size are initialized to zero.

[0012] Furthermore, determining the feasible thread block size solution space for each kernel function, and selecting the thread block size that maximizes the load balancing evaluation model value as the optimal execution configuration for that kernel function, includes: A GPU load balancing evaluation model is constructed by weighting the product of warp occupancy, thread block occupancy, and grid occupancy. If the current load balancing is greater than the optimal load balancing, the optimal load balancing and optimal thread block size are updated to the current load balancing and thread block size, respectively. Otherwise, the next solution is traversed until all solutions are found. The final optimal thread block size is the thread block size that best balances the GPU's SM load.

[0013] Furthermore, the warp utilization rate reflects the ratio of the number of warps actually executed concurrently on the SM to the maximum number of warps supported by the hardware; the thread block utilization rate shows the ratio of the number of thread blocks executed concurrently on the SM to the maximum number of thread blocks supported by the hardware, representing the ratio of the number of concurrent thread blocks on the SM to the maximum capacity of the hardware; and the grid utilization rate refers to the ratio of the total number of thread blocks in the grid to the maximum number of concurrent thread blocks that all SMs on the GPU can support.

[0014] Furthermore, the establishment of concurrent execution relationships between kernel functions, based on the determined optimal thread block size for each kernel function, involves dividing the execution grid of each kernel function into multiple kernel function slices, including: When scheduling kernel functions in units of kernel function slices, the thread block allocation information of the previous algorithm for the kernel function is obtained, and the kernel function slice size is set to the number of SMs in the GPU. First, the kernel function is divided into multiple kernel function slices, then a CUDA stream is allocated to each slice for concurrent execution, and finally the execution results in all CUDA streams are synchronized and summarized into the final execution result of the entire kernel function, thus completing the execution of the kernel function.

[0015] Furthermore, the construction of the CUDA graph using a hybrid mode based on the established concurrent execution relationships and data dependencies between kernel functions includes: First, the remaining modules with static parameters, such as electromagnetic field propulsion and nodal field calculation, are captured using the Flow Capture API to generate an initial static subgraph, and the particle simulation calculation process is started by loading this static subgraph. If particle generation is detected, the kernel function of the particle propulsion module needs to be added to the static graph. The CUDA graph node of the particle propulsion step is manually constructed by calling the cudaGraphAddKernelNode function, and the dependency relationship with the static subgraph is established by the cudaGraphAddDependencies function. If the change in the number of particles does not exceed the thread size currently configured for the particle propulsion kernel function, no changes are made, and the CUDA graph is loaded for particle simulation calculations. If the change in the number of particles exceeds the thread size, the mesh parameters of the manually constructed particle propulsion nodes in the second stage are updated through the cudaGraphKernelNodeSetParams function.

[0016] Secondly, the present invention provides a GPU parallel thread block adjustment and kernel function scheduling optimization system, comprising: The data acquisition module is used to acquire GPU device attributes and kernel function execution attributes; The configuration module is used to determine the feasible thread block size solution space for each kernel function based on GPU device attributes and kernel function execution attributes, and according to the triple constraints of thread bundle alignment, total register limit, and total shared memory limit. The thread block size that maximizes the load balancing evaluation model value is selected as the optimal execution configuration for that kernel function. The relationship establishment module is used to divide the execution grid of each kernel function into multiple kernel function slices based on the determined optimal thread block size of each kernel function, and to establish the concurrent execution relationship between kernel functions; The comparison output module is used to construct a CUDA graph in a hybrid mode based on the established concurrent execution relationship and data dependency between kernel functions. Using the actual model of the particle simulation software as the test object, it compares the GPU utilization, kernel function execution time and overall simulation efficiency before and after optimization by thread block adaptive adjustment, multi-stream concurrent scheduling and hybrid CUDA graph scheduling, and outputs the optimized particle simulation results.

[0017] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the GPU parallel thread block adjustment and kernel function scheduling optimization method.

[0018] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the GPU parallel thread block adjustment and kernel function scheduling optimization method.

[0019] Compared with the prior art, the present invention has the following technical effects: This invention initializes the kernel function's thread block solution space based on three conditions for thread block size, and initializes the optimal load balancing degree and optimal thread block size to zero. Finally, it iterates through the solution space and calculates the final optimal thread block size under the current thread block conditions according to the algorithm formula. This optimal thread block size is the one that best balances the GPU's SM load. Theoretically, using this thread block size results in the shortest kernel function execution time and the highest efficiency in utilizing GPU resources.

[0020] This invention analyzes the task dependencies of particle simulation algorithms and proposes a multi-stream concurrency strategy combining kernel function slicing and resource complementary scheduling to achieve task-level parallel acceleration. Parallelism analysis is performed on the serial logic between modules. By slicing kernel functions and utilizing CUDA streams, multiple kernel functions can be executed simultaneously. This not only solves the problem that a single kernel function cannot fully utilize the concurrency characteristics of CUDA streams, but also saves CUDA stream space, improves software parallelism, and accelerates software simulation speed. This invention's updated hybrid CUDA graph construction mode combines the convenience of stream capture with the flexibility of manual construction, significantly reducing the overhead of kernel function startup beyond the actual GPU execution and the time consumed by a series of preprocessing operations such as API calls. The more grids, the better the speedup. Attached Figure Description

[0021] Figure 1 GPU utilization diagram.

[0022] Figure 2Flowchart of the thread block adjustment algorithm for GPU load balancing.

[0023] Figure 3 Kernel function topology diagram of XEMPIC software.

[0024] Figure 4 A diagram illustrating the execution of kernel function slices.

[0025] Figure 5 CUDA stream scheduling flowchart after slicing a single kernel function.

[0026] Figure 6 A diagram illustrating two kernel functions that access an array sequentially.

[0027] Figure 7 The execution order of the two kernel functions after slicing.

[0028] Figure 8 Improved execution order.

[0029] Figure 9 Kernel Launch's share of kernel function execution time.

[0030] Figure 10 Hybrid mode construction of CUDA graphs in XEMPIC software.

[0031] Figure 11 Add particle propulsion nodes to the static CUDA graph.

[0032] Figure 12 Flowchart of this invention. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings: Example 1, please refer to Figure 12 This invention provides a method for GPU parallel thread block adjustment and kernel function scheduling optimization, comprising: Obtain GPU device properties and kernel function execution properties; Based on GPU device attributes and kernel function execution attributes, and in accordance with the triple constraints of thread bundle alignment, total register limit, and total shared memory limit, the feasible thread block size solution space for each kernel function is determined. The thread block size that maximizes the load balancing evaluation model value is selected as the optimal execution configuration for that kernel function. Based on the determined optimal thread block size for each kernel function, the execution grid of each kernel function is divided into multiple kernel function slices, establishing the concurrent execution relationship between kernel functions; Based on the established concurrent execution relationships and data dependencies among kernel functions, a CUDA graph is constructed using a hybrid mode. Using the actual model of the particle simulation software as the test object, the GPU utilization, kernel function execution time, and overall simulation efficiency are compared before and after optimization by thread block adaptive adjustment, multi-stream concurrent scheduling, and hybrid CUDA graph scheduling. The optimized particle simulation results are then output.

[0034] This invention abandons the single SM occupancy rate evaluation standard and establishes an evaluation mechanism that takes into account both global load characteristics and local resource utilization, optimizes the allocation of thread block size, avoids thread block resource waste and excessive invalid memory access time, and ensures optimal overall computing performance in heterogeneous computing environments. Breaking away from the default sequential scheduling mode of stream execution, this approach improves GPU (CUDA stream resource) utilization and enhances task scheduling flexibility by coordinating the scheduling of multiple CUDA streams, based on the computational characteristics and resource requirements of different kernel functions and the dependencies between kernel functions. This maximizes the degree of parallelism. Optimize the API (Application Programming Interface) conversion process in instant startup mode, reduce the frequency of conversion from Runtime API to Driver API, significantly reduce the scheduling latency of kernel function startup, reduce preparation and execution overhead, and improve kernel function call efficiency.

[0035] It possesses advantages and characteristics such as global load balancing representation capability, good heterogeneous parallelism, dynamic scheduling flexibility, and startup overhead optimization. This makes this method a powerful optimization tool suitable for various large-scale multi-grid device simulations.

[0036] Example 2: This invention provides a method for GPU parallel thread block adjustment and kernel function scheduling optimization, including: To address the current software's static configuration strategy of fixed thread block size allocation in kernel function thread block allocation, which leads to suboptimal execution efficiency of different kernel functions in heterogeneous computing environments, this paper proposes an adaptive thread block size adjustment algorithm based on GPU load balancing. This algorithm dynamically matches kernel function characteristics with GPU hardware parameters to achieve global load balancing. Furthermore, to address the current software's default stream-serial execution mode during kernel function execution, where kernel functions are scheduled sequentially, resulting in limited GPU resource utilization, this paper analyzes the task dependencies of the particle simulation algorithm and proposes a multi-stream concurrency strategy combining kernel function slicing and resource-complementary scheduling to achieve task-level parallel acceleration. Finally, to address the current software's immediate startup mode during kernel function startup, each call to the <<<>>> syntax triggers a conversion from the Runtime API to the Driver API, introducing significant scheduling latency, this section designs a hybrid-mode CUDA graph construction strategy to address the kernel function startup overhead problem, enabling the predefinition of static computation flows and efficient updating of dynamic parameters.

[0037] The proposed GPU load balancing-based adaptive thread block size adjustment algorithm analyzes the impact of using fixed thread block configurations in kernel functions within a program. It designs an adaptive dynamic adjustment algorithm for thread blocks based on GPU load balancing (with a weighting of 4:3:3 for Warp utilization, thread block utilization, and grid utilization), taking GPU hardware parameters and kernel function attributes as inputs, thereby improving GPU resource utilization.

[0038] The proposed kernel function parallel algorithm based on multi-stream concurrency performs parallelism analysis on the serial logic between modules. By analyzing the task dependencies of the particle simulation algorithm, CUDA streams are used as the basic unit for asynchronous task scheduling. Kernel functions are sliced, and CUDA streams are used to enable multiple kernel functions to be executed simultaneously. A multi-stream concurrency strategy combining kernel function slicing and resource complementary scheduling is proposed to achieve task-level parallel acceleration.

[0039] The proposed hybrid kernel function scheduling algorithm for constructing CUDA graphs analyzes the latency overhead caused by real-time kernel function scheduling. By predefining a series of kernel function startups and memory operations as a static computation graph, and leveraging the characteristics of CUDA graph technology combined with particle simulation methods, a hybrid CUDA graph construction mode of static capture plus dynamic update is proposed. This mode combines the convenience of flow capture with the flexibility of manual construction, significantly improving computational efficiency.

[0040] Specific procedures: In actual simulations using XEMPIC software, the introduction of GPU parallel computing significantly accelerated the simulation speed, but issues such as insufficient resource utilization still constrained the software's development. This invention focuses on the lifecycle of kernel functions, primarily addressing three processes: kernel function thread block allocation, kernel function slicing and multi-stream parallel processing, and kernel function startup. It sequentially designs a GPU load-balancing thread block adjustment algorithm, a multi-stream concurrent kernel function parallel algorithm, and a hybrid CUDA graph construction kernel function scheduling algorithm to accelerate and optimize particle simulation. Taking the RBWO model as an example, the invention first designs an adaptive thread block adjustment algorithm to achieve load balancing among GPU stream processors, allocating appropriate thread block resources to each kernel function. After thread block resource allocation, multi-stream parallel execution between kernel functions is implemented to improve task parallelism. After establishing the parallel relationship logic, the functions are ready to start and run the simulation. Considering the latency issues caused by real-time kernel function scheduling, a hybrid two-step CUDA graph construction scheduling algorithm is proposed. (The logical relationship of the three steps is: first, allocate resources; then, adjust the logical running relationship between different functions; and finally, reduce startup overhead according to the characteristics of each function.) The specific implementation is as follows: (1) Thread block adjustment algorithm for GPU load balancing This invention, based on SM occupancy rate, combines the three levels of the CUDA thread model and uses... Figure 1 The three-dimensional occupancy rates shown describe the GPU's load. Warp occupancy reflects the ratio of the actual number of concurrently executed warps on the SM to the maximum number of warps supported by the hardware. Thread block occupancy shows the ratio of the number of concurrently executed thread blocks on the SM to the maximum number of thread blocks supported by the hardware, representing the ratio of the number of concurrent thread blocks on the SM to the maximum hardware capacity. Mesh occupancy is the ratio of the total number of thread blocks in the mesh to the maximum number of concurrent thread blocks supported by all SMs on the GPU. This invention will use the weighted product of these three occupancy rates to describe the GPU's load balancing, i.e.:

[0041] in =0.4, =0.3, =0.3, the weights are determined through orthogonal experiments, prioritizing instruction-level parallelism and global load balancing. The thread block adjustment algorithm takes GPU hardware parameters and kernel function attributes as input (input data), and solves for the optimal thread block size through resource constraint checks and GPU load balancing optimization. The algorithm flow is as follows: Figure 2 As shown, the specific steps are as follows: For each kernel function, the algorithm first obtains device attributes such as the number of SMs, the maximum number of thread blocks per SM, and the warp size through cudaDeviceProps. Then, it uses cudaFuncGetAttributes to extract kernel function attributes such as register usage and shared memory consumption for each thread. (Data is obtained through functions.) Secondly, the size of a thread block needs to meet three constraints: first, thread bundle alignment, meaning the size of the thread block must be an integer multiple of the warp size; second, register limitation, meaning the total number of registers used by the thread block cannot exceed the device's maximum limit; and third, shared memory limitation, meaning the shared memory used by the thread block cannot exceed the device's maximum limit.

[0042] The thread block solution space of the kernel function is initialized based on the above three conditions, and the optimal load balancing degree and optimal thread block size are initialized to zero.

[0043] Finally, the solution space is traversed, and the GPU load balancing is calculated according to the above formula when the kernel function is run under the current thread block condition. If the current load balancing is greater than the optimal load balancing, the optimal load balancing and optimal thread block size are updated to the current load balancing and thread block size, respectively. Otherwise, the next solution is traversed until the traversal is complete. The final optimal thread block size is the thread block size that makes the GPU's SM load most balanced. Theoretically, using this thread block size results in the shortest kernel function execution time and the highest resource utilization efficiency for the GPU. (Processing data output results) Kernel-based parallel algorithm for multi-stream concurrency The main loop of the XEMPIC software follows strict temporal logic: each time step sequentially executes core steps such as electromagnetic field propagation, particle motion updates, and current and charge distribution. While this serial mode ensures computational accuracy, it neglects the potential parallelism between tasks. By performing topological sorting on the kernel functions in the XEMPIC software, the following was obtained: Figure 3 The topology diagram shown, combined with the kernel function implementation, reveals the following kernel function relationships: First, there are the kernel functions Mphi and Mzr. The former depends on the magnetic field in the φ direction and the electric fields in the z and r directions from the previous step to update the magnetic field in the φ direction of the current step, while the latter depends on the magnetic fields in the z and r directions and the electric field in the φ direction from the previous time step to update the magnetic fields in the z and r directions of the current step. There are no memory write conflicts between them, allowing them to execute in parallel; therefore, this is called the dependent kernel function group. Second, there are the three kernel functions for the particle propulsion step. They all update the global particle array, but different threads update different elements in the particle array. There are no memory write conflicts between threads, allowing concurrency through data partitioning; therefore, this is called the weakly dependent kernel function group. Finally, the kernel function Ephi must execute after the kernel function Mzr completes because the propulsion of the former depends on the updated magnetic field data in the z and r directions of the latter, resulting in strict timing constraints; therefore, this is called the strongly dependent kernel function group. (The software's kernel functions are topologically sorted based on their interrelationships.) Kernel function slicing and splitting algorithm After allocating the thread block size for the kernel function, this method begins slicing the kernel function (within a slice). When scheduling in units of kernel function slices, a completed thread block only needs to wait for other thread blocks in its kernel function slice to finish executing before releasing computational resources. Taking kernel function K as an example, the thread block allocation for the kernel function in the previous step is obtained and divided into slices as follows: Figure 4 For a grid of the size shown, initially, thread blocks in the range of (0, …, size-1) of K are initiated, then thread blocks in the range of (size, 2×size-1) are initiated, and so on. This staged execution method of kernel function slices is called offset-based execution. To better utilize limited resources, the kernel function slice size is set to the number of SMs in the GPU. Figure 5 This demonstrates the CUDA stream scheduling process after a single kernel function is sliced. First, the kernel function is divided into multiple kernel function slices. Then, a CUDA stream is allocated to each slice for concurrent execution, thereby increasing concurrency, fully utilizing CUDA stream resources, and reducing the execution time overhead of the kernel function. Finally, the execution results of all CUDA streams are synchronized and aggregated into the final execution result of the entire kernel function, completing the kernel function execution.

[0044] b) Algorithm for concurrent execution of kernel functions The kernel function slicing and distribution scheduling strategy in this invention provides greater flexibility for task scheduling. Slicing not only solves the problem of a single kernel function not being able to fully utilize the concurrency characteristics of CUDA streams, but also makes it possible to apply optimization strategies for multi-tasking systems to the current program. For example, Figure 6The two kernel functions K1 and K2 shown access the same array sequentially for different calculations. It can be seen that to ensure the correctness of the calculation results, operation A must be completed before operation a. However, there is no strict order between B and a (as shown by the red dashed line in the figure). Therefore, by slicing the kernel function to generate multiple kernel function fragments with different offsets, operation A can be executed first, followed by operations a and B simultaneously, then operation b, and so on. The kernel function slicing and scheduling strategy improves the parallelism probability between kernel functions in this way. As long as the total computational and memory resources used by multiple kernel function slices are within the theoretical range of the GPU, they can run simultaneously.

[0045] Concurrent kernel function execution can improve resource utilization per SM (System Controller). In the CUDA programming model, parallel execution is implemented at the thread block level. Thread blocks can be allocated on an SM as long as it has sufficient resources to meet its needs. There are three types of resource constraints on each SM: the number of registers, the size of shared memory, and the number of threads. Each thread block consumes a certain amount of these resources. Given a kernel function K, the resource usage for each type is as follows:

[0046] Where tsz represents the number of threads in a thread block, i.e., the size of the thread block. treg represents the number of registers used by each thread. bsmem is the size of shared memory used in the thread block. Creg, Csmem, and Cthr represent the total number of available registers, shared memory, and threads on a single SM, respectively. The resource usage of the kernel function is denoted as r(k), which is determined by the most scarce resource in each thread block, i.e., the maximum value of the three resources used by the thread block.

[0047]

[0048] In addition, each architecture of the GPU has an upper limit on the number of concurrent thread blocks per SM, denoted as Cblk.

[0049] For a set of kernel functions K=(K0, K1, …, KN-1), if their resource usage cannot satisfy the concurrent execution of multiple instances of the same kernel function, but can allow the concurrent execution of instances of different kernel functions, then this set of kernel functions is said to have complementary resource usage.

[0050] Table 1 shows the resource usage of all kernel functions and the amount of available GPU resources in the runtime environment, obtained using the Nsight Compute performance analysis tool in execution order. Taking the two consecutive kernel functions AccumulateI and AccumulateRho in the particle propulsion step as examples, the two kernel functions accumulate the current and charge after particle motion into the base grid for subsequent field propulsion. Each thread indexes one particle, and the kernel functions use atomic addition operations to accumulate global current and charge data. Therefore, there is no dependency between the multiple sub-kernel functions after splitting the kernel function. One thread block of the AccumulateI kernel function uses 640 threads, accounting for 62.5% of the maximum available threads per thread block in the GPU device. Therefore, the sub-kernel functions of AccumulateI cannot be executed concurrently with the sub-kernel functions of AccumulateI. One thread block of the AccumulateRho kernel function uses 33 registers, accounting for 51.6% of the maximum available registers per thread block in the GPU device. Similarly, it cannot be executed concurrently with the sub-kernel functions of AccumulateRho. However, the kernel functions AccumulateI and AccumulateRho should be able to run concurrently, as the combined number of threads (896) and registers (58) is still within device limitations. The remaining overlap possibilities are consistent with the analysis of these two kernel functions.

[0051] Table 1 Kernel Function Resource Usage Table

[0052] Next, we will use these two kernel functions as examples to describe this concurrent execution strategy based on resource complementarity. To illustrate this method more clearly, we will use P to represent the kernel function AccumulateI and Q to represent the kernel function AccumulateRho. Figure 7 The execution process of two kernel functions after slicing is demonstrated. In the program, we schedule the slices of each kernel function in a breadth-first search order. Since two instances of kernel function P cannot be executed simultaneously, this scheduling method cannot fully utilize complementary resources.

[0053] Based on the analysis of kernel function resource usage, this section proposes an improved execution order as follows: Figure 8 As shown. Since kernel functions P and Q have complementary resource consumption, a depth-first search is used to sequentially and alternately schedule the slices of each kernel function in parallel. That is, P0 is scheduled first, then Q0 can be executed in parallel with P1, and this extends to all subsequent sub-kernel functions.

[0054] As can be seen, the improved method only requires two streams to schedule two kernel functions, which not only overlaps the execution time of the two kernel functions, but also saves the extra overhead of creating multiple CUDA streams. The theoretical speedup of this method can be calculated as follows.

[0055] We use TP and TQ to represent the execution time of a slice (Pi and Qi) of kernel functions P and Q, respectively, and assume that they both have N slices. For two kernel functions with different numbers of slices, the extra slices are still executed in the original manner. Therefore, the total execution time is...

[0056] The execution time of the improved scheduling method can be estimated as follows:

[0057] Here, E represents the overhead of concurrent execution. E is introduced because some kernel functions may interfere with each other during concurrent execution. In actual execution, we observed that the average execution time of the kernel function AccumulateI is approximately 1.5 times that of the kernel function AccumulateRho. Assuming that the two kernel functions do not interfere with each other during concurrency, i.e., E=1, substituting these values ​​into the formula yields the theoretical speedup:

[0058] (3) Kernel function scheduling algorithm for hybrid construction of CUDA graph The startup process of CUDA kernel functions is a systematic process involving CPU and GPU collaboration, multi-level API conversion, and complex resource scheduling. The performance overhead generated during the startup phase can be quantitatively evaluated using two core metrics: Kernel Overhead and CPU Launch Overhead. After completing the above two processes, the startup overhead of each kernel function during the operation of the XEMPIC software was analyzed using Nsight System, as shown in Table 2.

[0059] Table 2 Startup overhead of kernel functions

[0060] Figure 9 This demonstrates a comparison between the kernel launch overhead and the actual execution time of all kernel functions during the XEMPIC software runtime.

[0061] Analysis of the proportions of each component reveals that for short kernel functions like EAxis, scheduling overhead even exceeds the actual execution time of the kernel function. For longer kernel functions, although execution time still dominates, scheduling overhead is on the same order of magnitude as the actual execution time and cannot be ignored. The root cause of this overhead lies in the dynamic nature of the immediate-startup mode: each kernel function call requires re-parsening the mesh / thread block configuration, checking resource availability, and updating the hardware context. For software like XEMPIC that involves numerous repetitive iterations, this repetitive overhead can be mitigated through the pre-compilation mechanism of the CUDA graph. All operation sequences in a single time step are converted into a static graph structure, parsed and optimized only during the first iteration, with subsequent iterations directly reusing the pre-generated execution plan.

[0062] In summary, considering the startup and execution of kernel functions, and the fact that the software's computational flow exhibits both static and dynamic characteristics—for example, the electromagnetic field propulsion module has fixed parameters, while the particle propulsion module requires kernel function configuration adjustments due to dynamic changes in the number of particles—this invention proposes a hybrid CUDA graph construction mode combining static capture and dynamic updates. This mode integrates the convenience of flow capture with the flexibility of manual construction to reduce kernel function startup time. The hybrid mode construction is implemented in three stages, such as... Figure 10 As shown.

[0063] a) In the first stage, there are no particles in the device space at the start of the simulation, and the particle propulsion module is not effective. Therefore, the remaining modules with static parameters, such as electromagnetic field propulsion and nodal field calculation, are first captured using the Stream Capture API to generate an initial static subgraph, and the particle simulation calculation process is started by loading this static subgraph.

[0064] b) In the second stage, if particle generation is detected, the kernel function of the particle propulsion module needs to be added to the static graph. Since the number of particles will continue to change, the mesh size of the kernel function needs to be dynamically adjusted. Therefore, the cudaGraphAddKernelNode function is called to manually construct the CUDA graph nodes for the particle propulsion step, and the cudaGraphAddDependencies function is used to establish dependencies with the static subgraph, such as... Figure 11 As shown.

[0065] c) In the third stage, since the mesh parameters are calculated by rounding up, if the change in the number of particles does not exceed the thread size currently configured for the particle propulsion kernel function, no changes are made, and the CUDA graph is loaded for particle simulation calculations. If the change in the number of particles exceeds the thread size, the mesh parameters of the particle propulsion nodes manually constructed in the second stage are updated through the cudaGraphKernelNodeSetParams function to avoid recapturing the entire CUDA graph.

[0066] (4) Compare the final output results Taking the RBWO model as an example, the kernel functions in the XEMPIC software were tested and compared.

[0067] a) GPU load balancing thread block adjustment algorithm: Compared to the default configuration, the adaptive adjustment algorithm calculates a thread block size that is generally closer to the optimal thread block size. The GPU utilization obtained using the adaptive adjustment algorithm is closer to the optimal situation, and in most kernel functions, the GPU utilization of this method is better than the default configuration. The GPU utilization during the execution of all kernel functions reached over 50%, indicating that the thread block size calculated by the adaptive adjustment algorithm can better utilize the limited computing resources of the GPU compared to the default configuration. Furthermore, the invention also measured the average execution time of each kernel function in actual operation using the default thread block size and the thread block size calculated by the adaptive adjustment algorithm. As shown in Table 3, it can be seen that the average execution time of 9 out of 12 kernel functions was improved by more than 10%, with speedup ratios reaching 2.73 and 3.23 in the PtclAdvance and EAxis kernel functions, respectively.

[0068] Table 3 Comparison of default and optimal kernel function configurations

[0069] Kernel-based parallel algorithms for multi-stream concurrency: i. Kernel Function Slicing and Stream Splitting Algorithm: Table 4 compares the execution time of kernel functions under two different strategies: no slicing and stream splitting, and slicing and stream splitting. The data shows that the execution mode of slicing and allocating multiple CUDA streams can effectively reduce the execution time of kernel functions compared to the default single CUDA stream execution mode. The ClearCurrent kernel function achieves the highest speedup of 1.35 times.

[0070] Table 4. Time Comparison of Kernel Function Default Execution and Slice-based Execution

[0071] ii. Concurrent Execution Algorithm between Kernel Functions: Concurrent scheduling was performed on the two kernel functions AccumulateI and AccumulateRho for the particle propulsion step. Table 5 shows the execution time and speedup ratio of the particle propulsion step before and after the improvement for different particle numbers. It can be seen that the speedup ratio increases continuously with the increase of the number of particles. This is because with the increase of the number of particles, the number of slices of the kernel function also increases, and the number of slices that can be executed overlappingly between the two kernel functions is also greater.

[0072] Table 5. Acceleration ratio of particle propulsion steps at different particle sizes

[0073] c) Hybrid Kernel Function Scheduling Algorithm for Constructing CUDA Graphs: Comparing the time overhead of immediate startup and startup using CUDA graphs. Table 6 shows that the simulation efficiency of all three models is significantly improved after adopting CUDA graphs. The RBWO model achieves a 1.37x performance improvement, with the simulation time decreasing from 2630 seconds to 1920 seconds; the MILO (Magnetically Insulated Transmission Line Oscillator) model achieves a 1.22x performance improvement, and the diode model achieves a 1.19x performance improvement.

[0074] Table 6 Comparison of execution time between programs using CUDA graphs and those not using CUDA graphs

[0075] In another embodiment of the present invention, a GPU parallel thread block adjustment and kernel function scheduling optimization system is provided, which can be used to implement the above-mentioned GPU parallel thread block adjustment and kernel function scheduling optimization method. Specifically, the system includes: The data acquisition module is used to acquire GPU device attributes and kernel function execution attributes; The configuration module is used to determine the feasible thread block size solution space for each kernel function based on GPU device attributes and kernel function execution attributes, and according to the triple constraints of thread bundle alignment, total register limit, and total shared memory limit. The thread block size that maximizes the load balancing evaluation model value is selected as the optimal execution configuration for that kernel function. The relationship establishment module is used to divide the execution grid of each kernel function into multiple kernel function slices based on the determined optimal thread block size of each kernel function, and to establish the concurrent execution relationship between kernel functions; The comparison output module is used to construct a CUDA graph in a hybrid mode based on the established concurrent execution relationship and data dependency between kernel functions. Using the actual model of the particle simulation software as the test object, it compares the GPU utilization, kernel function execution time and overall simulation efficiency before and after optimization by thread block adaptive adjustment, multi-stream concurrent scheduling and hybrid CUDA graph scheduling, and outputs the optimized particle simulation results.

[0076] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0077] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a GPU parallel thread block adjustment and kernel function scheduling optimization method.

[0078] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the GPU parallel thread block adjustment and kernel function scheduling optimization method in the above embodiments.

[0079] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for optimizing GPU parallel thread block adjustment and kernel function scheduling, characterized in that, include: Obtain GPU device properties and kernel function execution properties; Based on GPU device attributes and kernel function execution attributes, and in accordance with the triple constraints of thread bundle alignment, total register limit, and total shared memory limit, the feasible thread block size solution space for each kernel function is determined. The thread block size that maximizes the load balancing evaluation model value is selected as the optimal execution configuration for that kernel function. Based on the determined optimal thread block size for each kernel function, the execution grid of each kernel function is divided into multiple kernel function slices, establishing the concurrent execution relationship between kernel functions; Based on the established concurrent execution relationships and data dependencies among kernel functions, a CUDA graph is constructed using a hybrid mode. Using the actual model of the particle simulation software as the test object, the GPU utilization, kernel function execution time, and overall simulation efficiency are compared before and after optimization by thread block adaptive adjustment, multi-stream concurrent scheduling, and hybrid CUDA graph scheduling. The optimized particle simulation results are then output.

2. The GPU parallel thread block adjustment and kernel function scheduling optimization method according to claim 1, characterized in that, The acquisition of GPU device attributes and kernel function execution attributes includes: The device attributes such as the number of SMs, the maximum number of thread blocks per SM, and the warp size are obtained through cudaDeviceProps. The kernel attributes such as register usage and shared memory usage per thread of the kernel function are extracted using cudaFuncGetAttributes.

3. The GPU parallel thread block adjustment and kernel function scheduling optimization method according to claim 1, characterized in that, The aforementioned constraints, based on GPU device attributes and kernel function execution attributes, and adhering to the triple constraints of thread beam alignment, total register limit, and total shared memory limit, include: Thread bundle alignment ensures that the size of the thread block must be an integer multiple of the warp size; register limitation ensures that the total number of registers used by the thread block cannot exceed the device limit; shared memory limitation ensures that the shared memory used by the thread block cannot exceed the device limit. Based on the above three conditions, the thread block solution space of this kernel function is initialized, and the optimal load balancing degree and optimal thread block size are initialized to zero.

4. The GPU parallel thread block adjustment and kernel function scheduling optimization method according to claim 3, characterized in that, The process of determining the feasible thread block size solution space for each kernel function and selecting the thread block size that maximizes the load balancing evaluation model value as the optimal execution configuration for that kernel function includes: A GPU load balancing evaluation model is constructed by weighting the product of warp occupancy, thread block occupancy, and grid occupancy. If the current load balancing is greater than the optimal load balancing, the optimal load balancing and the optimal thread block size are updated to the current load balancing and thread block size, respectively. Otherwise, the next solution is traversed until the traversal is complete. The final optimal thread block size is the thread block size that makes the GPU's SM load most balanced.

5. The GPU parallel thread block adjustment and kernel function scheduling optimization method according to claim 4, characterized in that, The warp utilization rate reflects the ratio of the number of warps actually executed concurrently on the SM to the maximum number of warps supported by the hardware; the thread block utilization rate shows the ratio of the number of thread blocks executed concurrently on the SM to the maximum number of thread blocks supported by the hardware, representing the ratio of the number of concurrent thread blocks on the SM to the maximum capacity of the hardware; the grid utilization rate refers to the ratio of the total number of thread blocks in the grid to the maximum number of concurrent thread blocks that all SMs on the GPU can support.

6. The GPU parallel thread block adjustment and kernel function scheduling optimization method according to claim 1, characterized in that, Based on the determined optimal thread block size for each kernel function, the execution grid of each kernel function is divided into multiple kernel function slices, and the concurrent execution relationship between kernel functions is established, including: When scheduling kernel functions in units of kernel function slices, the thread block allocation information of the previous algorithm for the kernel function is obtained, and the kernel function slice size is set to the number of SMs in the GPU. First, the kernel function is divided into multiple kernel function slices, then a CUDA stream is allocated to each slice for concurrent execution, and finally the execution results in all CUDA streams are synchronized and summarized into the final execution result of the entire kernel function, thus completing the execution of the kernel function.

7. The GPU parallel thread block adjustment and kernel function scheduling optimization method according to claim 1, characterized in that, The process of constructing a CUDA graph using a hybrid mode based on the established concurrent execution relationships and data dependencies between kernel functions includes: First, the remaining modules with static parameters, such as electromagnetic field propulsion and nodal field calculation, are captured using the Flow Capture API to generate an initial static subgraph, and the particle simulation calculation process is started by loading this static subgraph. If particle generation is detected, the kernel function of the particle propulsion module needs to be added to the static graph. The CUDA graph node of the particle propulsion step is manually constructed by calling the cudaGraphAddKernelNode function, and the dependency relationship with the static subgraph is established by the cudaGraphAddDependencies function. If the change in the number of particles does not exceed the thread size currently configured for the particle propulsion kernel function, no changes are made, and the CUDA graph is loaded for particle simulation calculations. If the change in the number of particles exceeds the thread size, the mesh parameters of the manually constructed particle propulsion nodes in the second stage are updated through the cudaGraphKernelNodeSetParams function.

8. A GPU parallel thread block adjustment and kernel function scheduling optimization system, characterized in that, include: The data acquisition module is used to acquire GPU device attributes and kernel function execution attributes; The configuration module is used to determine the feasible thread block size solution space for each kernel function based on GPU device attributes and kernel function execution attributes, and according to the triple constraints of thread bundle alignment, total register limit, and total shared memory limit. The thread block size that maximizes the load balancing evaluation model value is selected as the optimal execution configuration for that kernel function. The relationship establishment module is used to divide the execution grid of each kernel function into multiple kernel function slices based on the determined optimal thread block size of each kernel function, and to establish the concurrent execution relationship between kernel functions; The comparison output module is used to construct a CUDA graph in a hybrid mode based on the established concurrent execution relationship and data dependency between kernel functions. Using the actual model of the particle simulation software as the test object, it compares the GPU utilization, kernel function execution time and overall simulation efficiency before and after optimization by thread block adaptive adjustment, multi-stream concurrent scheduling and hybrid CUDA graph scheduling, and outputs the optimized particle simulation results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the GPU parallel thread block adjustment and kernel function scheduling optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the GPU parallel thread block adjustment and kernel function scheduling optimization method as described in any one of claims 1 to 7.

Images