A multi-level GPU load balancing method and system for DFPT analysis

Abstract

The present invention discloses a multi-level GPU load balancing method and system for DFPT analysis, which belongs to the field of computer high-performance computing numerical simulation. The method first determines the batch load through pre-running based on the data set of density functional perturbation analysis. Secondly, a first-level load division is performed according to the batch load: load balancing division is performed between cards to achieve load balancing between GPU cards. Finally, a second-level load division is performed: load balancing division is performed within the card to achieve load balancing within the GPU card. The system includes a load determination module, a first-level load division module and a second-level load division module. The present invention improves the problem that the existing load balancing strategy is difficult to cope with density functional perturbation analysis after GPU acceleration due to the lack of targeted optimization, thereby improving resource utilization and computing efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of computer high-performance computing numerical simulation, and in particular relates to a multi-level GPU load balancing method for density functional perturbation (DFPT) analysis. Background Art

[0002] Density Functional Theory (DFT) is a method used to study the electronic structure of multi-electron systems and has widespread applications in physics and chemistry. DFT is one of the most commonly used methods in computational materials science and computational chemistry, particularly for studying the properties of molecules and condensed matter. Perturbation Theory (PT) is a mathematical method used to find approximate solutions to related problems from exact solutions. It is often used to calculate complex Hamiltonians in quantum physics. Physical properties measured in experiments are directly related to the system's quantum response, or perturbations. These responses, which are related to second-order or higher-order derivatives of the total energy, can be calculated using density functional perturbation theory (DFPT) within the framework of homogeneous quantum mechanics. This quantum perturbation theory can accurately predict many fundamental physical phenomena, such as superconductivity, vibrational frequency or phonon dispersion, polarizability, and harmonic and anharmonic Raman spectroscopy. The density functional perturbation analysis method is implemented in the computational software FHI-aims. To achieve faster execution, heterogeneous parallelization is used, utilizing GPUs for accelerated computation. When using multiple GPU cards for density functional perturbation analysis, the task load on each card is easily unbalanced due to the lack of targeted optimization.

[0003] Load imbalance in density functional perturbation analysis not only reduces resource utilization and computational efficiency, but also wastes energy and space. Existing load balancing methods only consider load balancing between processes during CPU runtime, but are not applicable to situations where GPU acceleration is used. Furthermore, existing load balancing strategies are relatively simple, considering only load balancing between processes and ignoring the load on the card after GPU acceleration. These issues make existing load balancing methods difficult to apply to density functional perturbation analysis after GPU acceleration. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the existing technology and provide a multi-level GPU load balancing method and system for DFPT analysis. This method can effectively improve the problem that the existing load balancing strategy is difficult to cope with the density functional perturbation analysis after GPU acceleration due to the lack of targeted optimization.

[0005] A multi-level GPU load balancing method for DFPT analysis includes the following steps:

[0006] Step 1: Determine the batch load through a preliminary run based on the data set of density functional perturbation analysis.

[0007] Step 2: Perform first-level load division based on batch load, and perform load balancing between cards to achieve load balancing between GPU cards.

[0008] Step 3: Secondary load division, perform load balancing within the card to achieve load balancing within the GPU card.

[0009] Furthermore, in step 1, the load division basis determined by pre-operation is specifically:

[0010] 1.1. For the dataset to be subjected to density functional perturbation analysis, perform a pre-run using the same number of processes and GPU cards as the actual run.

[0011] 1.2. For the pre-run density functional perturbation analysis, the operation logic is used to ensure that one batch of data is run each time for loop iteration, thereby obtaining the load of each batch.

[0012] 1.3. Use the load of each batch as the load partitioning weight, and set a unique index for each running batch and its weight in a one-to-one correspondence.

[0013] Furthermore, in step 2, the first-level load division is performed to realize load balancing between GPU cards by performing load balancing between cards. Specifically:

[0014] 2.1. Use the load partitioning weight and unique index of each batch described in step 1 to determine the load partitioning weights of all batches in the actual operation.

[0015] 2.2. Use the inter-card load partitioning algorithm to complete the inter-card load partitioning.

[0016] Furthermore, the inter-card load partitioning algorithm described in step 2.2 specifically includes:

[0017] 2.2.1. Obtain the weight information and geometric center coordinates of the current batch on all GPU cards through inter-process communication.

[0018] 2.2.2. Use the geometric center coordinates and weight information of all batches to be divided to perform a binary partition on the GPU set so that the weights of the left and right half of the GPU set are balanced.

[0019] 2.2.3. Repeat the above binary division recursively until the weight of each GPU card is balanced relative to the total weight.

[0020] 2.2.4. Through inter-process communication, each GPU card obtains the divided batch information.

[0021] Furthermore, in step 3, the secondary load division is performed to realize load balancing within the GPU card by performing load balancing within the card:

[0022] 3.1. Based on the load balancing between cards in step 2, two data structures, a batch structure and a thread block structure, are designed for each GPU card.

[0023] Batch structure: used to represent the weight and index information of each batch; thread block structure: used to represent the total weight and index information allocated to each thread block in the card.

[0024] 3.2. Use the card load partitioning algorithm to partition the card load.

[0025] Furthermore, the algorithm for dividing the load within the card in step 3.2 is specifically as follows:

[0026] 3.2.1. Sort the batch structure set described in 3.1 from large to small according to the weight information.

[0027] 3.2.2. Based on the number of thread blocks used, create a minimum heap of corresponding size using the thread block structure described in 3.1 as the key and the total weight allocated in the thread block.

[0028] 3.2.3. Take the thread block with the smallest total weight from the top of the minimum heap, assign the batch with the largest weight in the set to this thread block, update the total weight of the thread block, and then put the updated thread block back into the minimum heap.

[0029] 3.2.4. Repeat 3.2.1 to 3.2.4 until all batches are distributed.

[0030] On the other hand, the present invention also provides a multi-level GPU load balancing system for DFPT analysis, including a load determination module, a first-level load division module and a second-level load division module.

[0031] The load determination module determines the batch load through a pre-run.

[0032] The first-level load division module performs load balancing between cards according to batch loads.

[0033] The secondary load division module performs load balancing division within a card after achieving load balancing between cards.

[0034] Beneficial effects of the present invention:

[0035] The present invention improves the problem that the existing load balancing strategy is difficult to deal with the density functional perturbation analysis after GPU acceleration due to the lack of targeted optimization by using a multi-level GPU load balancing adjustment method, thereby improving resource utilization and computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the overall flow chart of the method of the present invention;

[0037] Figure 2 This is a flow chart of the first-level load balancing method of the present invention;

[0038] Figure 3 This is a flow chart of secondary load balancing in the method of the present invention;

[0039] Figure 4 This is a flow chart of the first-level load balancing algorithm in the method of the present invention;

[0040] Figure 5 This is a flow chart of the secondary load balancing algorithm in the method of the present invention;

[0041] Figure 6 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0043] Existing load balancing methods only consider inter-process load balancing during CPU runtime, but are not applicable to GPU-accelerated scenarios. Furthermore, existing load balancing strategies are relatively simple, considering only inter-process load balancing and ignoring the internal load of the card after GPU acceleration. These issues make existing load balancing methods difficult to apply to density functional perturbation analysis after GPU acceleration.

[0044] The core idea of ​​the embodiment of the present invention for improving the load imbalance situation of the existing density functional perturbation analysis load balancing strategy after using GPU is: according to the load weight under GPU calculation, load adjustment is performed comprehensively through multi-level load division steps to achieve overall load balance.

[0045] The embodiment of the present invention performs multi-level load balancing optimization on the density functional perturbation analysis module using GPU acceleration by redistributing the task load between GPU cards and within GPU cards. When the density functional perturbation analysis module uses multiple GPU cards for calculation, there will be 128 parallel execution thread blocks on each GPU card. The operation strategy of the density functional perturbation analysis module is to divide the data set into multiple batches for operation, and each GPU card will be allocated multiple batches for calculation. On a single GPU card, the multiple batches that need to be calculated will be redistributed to each parallel execution thread block, and each thread block executes multiple batches. It can be seen that the density functional perturbation analysis module, in the GPU operation mode, not only needs to balance the load between multiple cards, but also needs to load balance the thread blocks within each GPU card due to parallel execution to achieve overall load balancing.

[0046] Figure 1 A schematic flow chart of a multi-level GPU load balancing method for DFPT analysis provided by the present invention. Figure 1 As shown, the method includes steps S101-S103:

[0047] The logic of the load balancing method of the original density functional perturbation analysis module when running on the CPU is as follows: during the first run, the load information of the batch responsible for each process running on the CPU is recorded, and then the recorded load information is used as the partitioning weight to divide the load so that the overall load of the batch on each process is roughly the same, thereby achieving the effect of load balancing.

[0048] The embodiment of the present invention improves the load partitioning method of the density functional perturbation analysis module when running on a GPU. The improvement steps include:

[0049] Step S101: Pre-run the data set that will be subjected to density functional perturbation analysis using the same number of processes and GPU cards as the actual run. Specific running logic is used for the pre-run density functional perturbation analysis to ensure that one batch of data is run through a loop iteration each time, thereby obtaining the load condition of each batch. The load condition of each batch is used as the load partitioning weight w. A unique index is set for each calculation batch, corresponding to its weight one by one, and the result is output to a file for use during the actual run.

[0050] Specifically, the pre-run density functional perturbation analysis module uses specific operating logic to change the original multi-threaded block parallel operation in the GPU card to a single-threaded block operation mode, so as to measure the load information of each batch on the dataset under GPU operation as a weight, and set a unique index to bind the batch to its weight. In order to speed up the operation speed of this single-threaded block mode, the original density functional perturbation analysis module was cut down and exited immediately after recording the load information, which greatly reduced the running time and improved the speed of pre-run.

[0051] Step S102, first-level load division, inter-card load balancing division to achieve inter-GPU load balancing. Use the batch unique index and its weight w obtained in step S101 as the load division basis to perform the first-level load division. The goal is to make the total weight W_GPUi of the batches allocated to each GPU card approximately equal, such as Figure 2 shown.

[0052] Specifically, first, during the actual first run, the unique index and weight information of the data set are read and mapped to each batch during the run, representing the load of each batch. The first level of partitioning is performed using the inter-card load partitioning algorithm, such as Figure 4 shown.

[0053] The specific process of the load balancing algorithm is as follows:

[0054] (1) The geometric center coordinates of the batch are calculated based on the three-dimensional coordinates of the integration points in the batch, and the three-dimensional geometric center coordinates of each batch are stored in the batch information structure together with the batch weight information.

[0055] (2) Inter-process communication is used to ensure that each process, i.e., the GPU card, has a complete set of batch information structures.

[0056] (3) Use the geometric centers and weight information of all batches to be divided to perform binary partitioning on the GPU set, so that the total batch weights of the left and right half GPU sets are balanced relative to the total weight of all batches.

[0057] (4) Recursively perform the above binary division on the left and right sides until the total batch weight on each GPU card is balanced relative to the total weight of all batches.

[0058] (5) Through inter-process communication, each GPU card obtains the divided batch information.

[0059] Step S103, secondary load division, performs intra-card load balancing division to achieve load balancing between thread blocks within the GPU card. Based on the primary load division result of step S102, the batches allocated to each GPU card are secondary distributed so that the sum of the weights W_blocki of the batches in each thread block on each GPU card is approximately equal, such as Figure 3 shown.

[0060] Specifically, during the first actual run, the system initializes two data structures to support load balancing. The first data structure represents each computational batch and contains a unique identifier for identifying the batch and a weight representing the load of the batch. The second data structure represents the thread block and includes an identifier for identifying the thread block, a variable for tracking the total weight of the batches assigned to the thread block, and a list to store all batches assigned to the thread block. Load balancing is then performed using an adaptive load balancing algorithm, such as Figure 5 shown.

[0061] The specific process of the load balancing algorithm is as follows:

[0062] (1) Initialize a structure array representing all computation batches. For each batch, assign a unique identifier and a weight value, and then sort the batches in descending order based on the weight.

[0063] (2) Create an array of thread block structures whose size is equal to the number of thread blocks in the current GPU. Use this array to construct a minimum heap using the total weight assigned to the thread block batch as the key.

[0064] (3) Iteratively select the thread block with the smallest total weight from the minimum heap, assign the batch with the largest weight after sorting to this thread block, and update the total weight of this thread block. This process is repeated until all batches are assigned.

[0065] (4) The allocation results are stored in an array, which records the batch identifier and quantity allocated to each thread block.

[0066] (5) The allocation result is then passed to the GPU execution environment. Each thread block executes the corresponding computing task according to the batch identifier and quantity assigned to it.

[0067] The embodiment of the present invention analyzes the reasons for the failure of load balancing of the original density functional perturbation analysis module in GPU mode, and adopts a multi-level load balancing method to achieve overall load balancing of the density functional perturbation analysis module in GPU operation mode.

[0068] On the other hand, the present invention also provides a multi-level GPU load balancing system for DFPT analysis, including a load determination module, a first-level load division module and a second-level load division module, such as Figure 6 shown.

[0069] The load determination module determines the batch load through a pre-run.

[0070] The first-level load division module performs load balancing between cards according to batch loads to achieve load balancing between GPU cards.

[0071] The secondary load division module performs intra-card load balancing after achieving inter-card load balancing, thereby achieving intra-GPU card load balancing.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A multi-level GPU load balancing method for DFPT analysis, characterized in that: The following steps are involved: Step 1: Determine the batch load through a pre-run based on the data set of density functional perturbation analysis. The specific process is as follows: 1.

1. For the dataset to be subjected to density functional perturbation analysis, perform a pre-run using the same number of processes and GPU cards as the actual run; 1.

2. For the pre-run density functional perturbation analysis, the operation logic is used to ensure that one batch of data is run for each iteration to obtain the load of each batch; 1.

3. Use the load of each batch as the load partitioning weight, and set a unique index for each running batch and its weight in a one-to-one correspondence; Step 2: Perform first-level load division based on batch load: Perform load balancing between cards to achieve load balancing between GPU cards. The specific process is as follows: 2.

1. Using the load partitioning weight and unique index of each batch described in step 1, determine the load partitioning weights of all batches in the actual operation; 2.

2. Use the inter-card load partitioning algorithm to complete the inter-card load partitioning; Step 3: Perform secondary load division: Perform load balancing division within the card to achieve load balancing within the GPU card. The specific process is as follows: 3.

1. Based on the load balancing between GPU cards in step 2, two data structures, a batch structure and a thread block structure, are designed for each GPU card. The batch structure is used to represent the weight and index information of each batch; the thread block structure is used to represent the total weight and index information allocated to each thread block in the card; 3.

2. Use the card load partitioning algorithm to partition the card load.

2. The multi-level GPU load balancing method for DFPT analysis according to claim 1, wherein the inter-card load partitioning algorithm described in step 2.2 is specifically implemented as follows: 2.2.

1. Obtain the weight information of the current batch and the geometric center coordinates of the batch on all GPU cards through inter-process communication; 2.2.

2. Use the geometric center coordinates and weight information of all batches to be divided to perform a binary partition on the GPU set, so that the weights of the left and right half of the GPU set are balanced; 2.2.

3. Recursively perform the above binary division until the weight of each GPU card is balanced relative to the total weight; 2.2.

4. Through inter-process communication, each GPU card obtains the divided batch information.

3. A multi-level GPU load balancing method for DFPT analysis according to claim 2, characterized in that: The specific process of the card load division algorithm described in step 3.2 is as follows: 3.2.

1. Sort the set of batch structures described in 3.1 from largest to smallest according to weight information; 3.2.

2. Based on the number of thread blocks used, create a minimum heap of the corresponding size using the thread block structure described in 3.1 as the key and the total weight allocated within the thread block. 3.2.

3. Take the thread block with the smallest total weight from the top of the min-heap, assign the batch with the largest weight in the set to this thread block, update the total weight of the thread block, and then put the updated thread block back into the min-heap; 3.2.

4. Repeat 3.2.1 to 3.2.4 until all batches are distributed.

4. A multi-level GPU load balancing system for DFPT analysis, used to implement the load balancing method according to any one of claims 1 to 3, characterized in that: It includes a load determination module, a primary load division module and a secondary load division module; The load determination module determines the batch load through a pre-run; The first-level load division module performs load balancing between cards according to batch load; The secondary load division module performs load balancing division within a card after achieving load balancing between cards.

Images