A load balancing method for multi-level parallel large-scale coupled programs

Abstract

The application discloses a load balancing method for multi-level parallel large-scale coupled programs, comprising the following steps: based on a function insertion mechanism, combining a TLS, a thread cache mechanism and a measurement perception algorithm, low-overhead performance tracking and function relationship call statistics are performed on a large-scale multi-level parallel coupled program; based on resource configuration and layout, a multi-component performance prediction model is constructed by fusing component-level resource characteristics; and component layout is automatically and parallelly generated in a lightweight manner, so that adaptive resource allocation is performed in two dimensions of the shortest time and the minimum machine time. The load balancing method for multi-level parallel large-scale coupled programs provided by the application solves the pain points of large-scale multi-level parallel programs, such as too large monitoring overhead, difficulty in depicting the influence of physical layout in a performance model and inability of multi-target optimization in resource allocation, so that the optimal configuration of resources in a large-scale computing environment is realized.

Description

Technical Field

[0001] This invention relates to the field of high-performance computing technology, and in particular to a load balancing method for large-scale coupled programs with multi-level parallelism. Background Technology

[0002] With the rapid development of high-performance computing technology, complex scientific and engineering computations have increasingly relied on large-scale coupled programs. These programs typically integrate multiple independent models (i.e., components) describing different physical processes or computational logics within the same computational framework, simulating complex real-world systems through collaborative work. In terms of operational mechanisms, large-scale coupled programs often contain extremely complex physical or mathematical driving factors, and there is frequent data exchange and deep logical dependencies between components. To fully utilize the computing power of modern supercomputers, these programs generally adopt a multi-level parallel architecture of "component-process-thread," which means that the performance of the entire system depends not only on the computational efficiency of individual components but also on the depth of multi-level parallel collaboration, cross-component communication latency, and the distribution of underlying computing resources.

[0003] As a typical manifestation and highly challenging application of large-scale coupled procedures in the field of Earth science, the Earth System Model (ESM) has become a key support for simulating global climate evolution and predicting extreme weather events. By coupling multiple complex physical component models such as the atmosphere, ocean, sea ice, waves, and land surface, it achieves a holistic simulation of the dynamics, physical processes, and interactions of various spheres of the Earth.

[0004] Large-scale coupled programs, exemplified by FIO-ESM (the Earth System Model developed by the First Institute of Oceanography, Ministry of Natural Resources), are a typical example that fully embodies the aforementioned multi-level parallelism characteristics and has been deployed on tens of thousands of computing cores. In large-scale computing environments, achieving automated load balancing for coupled models such as FIO-ESM is crucial for improving computing resource utilization and simulation efficiency, but it also faces significant challenges. Existing technologies mainly suffer from the following limitations:

[0005] First, performance measurement methods struggle to balance low overhead and high coverage. Programs like FIO-ESM operate on massive scales, and traditional performance analysis tools often incur extremely high runtime overhead in hybrid multi-level parallel modes, significantly altering the program's execution characteristics (i.e., the "uncertainty effect"). Furthermore, manually embedding monitoring code into the source code is not only inefficient but also fails to cover the dynamic changes in complex cross-component communication, coupler scheduling, and deep internal function call relationships. Therefore, achieving an automated, low-intrusion, lightweight measurement method for multi-level parallel modes is a prerequisite for obtaining accurate performance metrics.

[0006] Secondly, existing performance modeling methods lack a comprehensive consideration of "component layout" and "resource allocation." The execution efficiency of coupled programs is not only limited by the number of cores allocated to each component, but also profoundly affected by the spatial arrangement of components on physical nodes and the fine-grained allocation of computing resources (such as the number of threads and memory bandwidth allocation). Traditional load balancing models often simplify components into isolated tasks, ignoring the performance fluctuations caused by intra-node resource contention and inter-node communication latency under different layouts. This results in insufficient accuracy and poor generalization of model predictions in complex physical layout scenarios.

[0007] Finally, resource allocation goals are singular and lack flexibility. In practical scientific research and business applications, users' needs for HPC system resource configuration exhibit clear diversity: in scenarios with extremely high timeliness requirements, such as weather forecasting, the goal is to pursue the "shortest runtime"; while in long-term climate simulations or budget-constrained tasks, the goal shifts to maximizing resource utilization efficiency, i.e., "minimum CPU-Hours". Existing automated load balancing methods typically only perform heuristic searches for a single goal or rely on manual trial and error to adjust parameters, lacking an efficient mechanism that can automatically generate optimal component layouts and adaptively allocate resources for complex coupled systems based on different optimization dimensions specified by the user. Summary of the Invention

[0008] In view of the aforementioned shortcomings of existing technologies, the technical problem to be solved by this invention is that existing performance measurement methods struggle to balance low overhead and high coverage, existing performance modeling methods lack comprehensive consideration of "component layout" and "resource allocation," and resource allocation objectives are singular and inflexible. This invention provides a load balancing method for large-scale coupled multi-level parallel programs. For complex applications such as FIO-ESM, large-scale coupled programs represent a closed-loop adaptive optimization scheme from low-overhead awareness at the bottom layer, cross-dimensional modeling at the middle layer, to multi-objective decision-making at the upper layer. This method addresses the pain points of excessive overhead during monitoring, the difficulty in characterizing the impact of physical layout in performance models, and the inability to optimize resource allocation across multiple objectives in large-scale multi-level parallel programs, thereby achieving optimal resource allocation in large-scale computing environments.

[0009] To achieve the above objectives, this invention provides a load balancing method for large-scale coupled programs with multi-level parallelism, comprising the following steps:

[0010] Based on the function instrumentation mechanism, combined with TLS, thread caching mechanism and measurement awareness algorithm, low-overhead performance tracing and function relationship call statistics are performed on large-scale multi-level parallel coupled programs.

[0011] Based on resource allocation and layout, and combined with the coupled program performance information obtained by function instrumentation mechanism, a multi-component performance prediction model that integrates component-level resource characteristics is constructed.

[0012] The system automatically generates component layouts in a parallel and lightweight manner, embeds a performance prediction model into a genetic search algorithm, and performs adaptive resource allocation based on two dimensions: shortest time and minimum machine time.

[0013] Furthermore, based on function instrumentation mechanisms, combined with TLS, thread caching mechanisms, and measurement-aware algorithms, low-overhead performance tracing and function call statistics are performed on large-scale, multi-level parallel coupled programs. Specifically, this includes the following steps:

[0014] For multi-level parallel coupled programs including components, processes, and threads, a hook function is designed to separate multi-level parallel state awareness and measurement awareness. Each worker thread is allocated a thread-local time stack (time_stk), a function address stack (func_addr_stk), a component module stack (module_stk), as well as a thread-local cache and a thread-local aggregation space. Among these, the parallel awareness of self_addr and father_addr is implemented through the parallel state awareness of func_addr_stk.

[0015] During the hooking process, the probe start time (shell_start_time), function start time (self_start_time), and function end time (self_end_time) of each thread are recorded. Corresponding parallel state awareness processing is performed on the current function address (self_addr), parent function address (father_addr), and component module identifier (current_module) of the serial / parallel regions. Based on the measurement-aware separation mechanism, the function call count (times), function execution time (acctime), and probe envelope time (sheltime) are calculated.

[0016] A two-dimensional hashmap storage structure is constructed using self_addr and father_addr as two-dimensional keys. It is thread-local cached and thread-local aggregated for hash keys, and combines hook functions to store function performance statistics during program execution.

[0017] Furthermore, the specific way in which the function address stack func_addr_stk performs parallel state awareness operations is as follows: within the parallel area, the parent-child call relationship is restored by "each thread independently maintaining the call relationship stack", and outside the parallel area, the serial call context is copied by "synchronously pushing and popping all thread stacks", taking into account both the isolation of multi-threaded calls and the consistency of call relationship restoration when switching between serial and parallel calls.

[0018] Furthermore, the thread caching mechanism is as follows: to address the time-consuming problem of calculating two-dimensional hash keys, the system stores the function address mapping relationship and hash index in a thread-private cache; at the same time, each thread maintains a two-dimensional HashMap storage structure based on address keys locally, accumulates the call frequency and execution time in real time, and performs global reduction aggregation only at specific synchronization points or when the program ends.

[0019] Furthermore, based on resource allocation and layout, a multi-component performance prediction model integrating component-level resource characteristics is constructed, specifically including the following steps:

[0020] Based on low-overhead performance analysis methods, component-level result analysis is performed to obtain historical running samples of multi-component coupled programs; based on the number of processes and threads of each component, a multi-dimensional feature space containing component-level features and global statistical features is constructed; at the same time, a layout matrix is ​​constructed according to the component layout grouping information, and layout-aware features are calculated.

[0021] A second-order polynomial expansion is performed on the multidimensional feature space to capture the nonlinear interaction between features. After standardization by StandardScaler, the RidgeCV model is used for fitting, and the optimal regularization coefficient alpha is automatically searched to obtain the main model for performance prediction.

[0022] The prediction residuals of the main model on the training data are calculated, and the HuberRegressor robust regression model is used to fit the prediction residuals to correct systematic bias. The median residuals of each group are calculated by layout, and the prediction results of different layouts are differentiated by shrinkage weighting. The optimal combination of residual model fusion weights, layout correction strength and shrinkage coefficient is determined by grid search to minimize the MAPE index.

[0023] Furthermore, the historical running samples of the multi-component coupled program include the number of processes {component}_process, the number of threads {component}_thread, the component layout information layout, and the corresponding total program running time tot_time for each component.

[0024] Furthermore, component-level features include the process-thread product (workload), the resource proportion of each component, the logarithmic transformation value, and the reciprocal value.

[0025] Furthermore, global statistical features include total resources, imbalance between components, and maximum resource share.

[0026] Furthermore, the automatic parallel lightweight generation of component layouts adaptively allocates resources based on two dimensions: shortest time and minimum machine time. This includes the following steps:

[0027] Obtain the component set, component identifier, total core count constraint, process candidate set, and thread candidate set of the multi-component coupled program to be optimized;

[0028] The component layout is encoded to represent it as a standardized layout code that can be used for searching, constraint verification, and performance prediction; and the duplicate and invalid layouts are reduced by using a parallel lightweight layout generation method.

[0029] For each component layout candidate, a genetic algorithm is executed to search for the minimum running time and minimum machine time as optimization objectives. An initial population that meets the total core count constraint is generated based on domain knowledge constraints, with each individual consisting of the number of processes and threads of each component. Then, after multiple generations of evolution, the predicted running time is calculated as the fitness of all individuals in the population using a prediction model in each generation. Elite individuals are retained, and the next generation is generated through crossover and mutation. Individuals that do not meet the constraints are repaired or randomly regenerated. An early stopping mechanism is supported, and the evolution is terminated when there is no improvement for several consecutive generations. After the genetic algorithm converges, local refinement is performed on the best individual until no further improvement is possible. Finally, the globally optimal running time scheme and machine time scheme are selected from all layout patterns and output.

[0030] Furthermore, the parallel lightweight layout generation method specifically includes using the central initial component block as the starting point for layout generation, expanding subsequent component blocks only within the neighborhood of the currently generated component block according to a preset relative offset rule to narrow the candidate layout search space; performing boundary and occupancy validity checks on candidate positions during the expansion process, retaining only layout branches that meet the constraints to continue generation; performing rotational symmetry and axisymmetric equivalence checks on the generation results and deleting duplicate layouts to reduce repeated searches and storage of isomorphic layouts; and allocating different expansion branches to parallel tasks for separate execution, and then streaming the results of each branch to achieve rapid generation of component layouts and reduce the time overhead of layout search.

[0031] Technical effect

[0032] This invention presents a load balancing method for large-scale coupled programs with multi-level parallelism. It is a low-overhead probe-based measurement method that enables efficient performance tracking of coupled programs in a multi-level parallel mode. Based on performance modeling of resource configuration and component layout, a high-precision correlation model between resource mapping schemes and program execution time is established, providing scientific quantitative evaluation support for adaptive resource allocation of coupled systems. Based on lightweight component layout parallel generation and adaptive resource allocation, a heuristic optimization algorithm is used to automatically search and generate optimal configuration layouts in two dimensions: "shortest runtime" and "minimum computation time," while satisfying total resource constraints. Taking the Earth System Model FIO-ESM as an example, with a core budget of 1024, the default configuration scheme has a runtime of 160.553 seconds and a computation time of 164406.272 core-seconds. Based on the optimal configuration scheme of this invention, the runtime is reduced to 97.374 seconds and the computation time is reduced to 94405.0533 core-seconds.

[0033] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a preferred embodiment of the load balancing method for a large-scale coupled program with multi-level parallelism according to the present invention.

[0035] Figure 2 This is a schematic diagram of the ATM component results of a low-overhead probe-based performance analysis method for load balancing of large-scale coupled multi-level parallel programs, which is a preferred embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of a parallel lightweight layout generation method for a load balancing method for large-scale coupled programs with multi-level parallelism, according to a preferred embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of the adaptive resource allocation process of a load balancing method for large-scale coupled programs with multi-level parallelism, according to a preferred embodiment of the present invention.

[0038] Figure 5 This is a schematic diagram illustrating the adaptive resource allocation result of a load balancing method for large-scale coupled programs with multi-level parallelism, according to a preferred embodiment of the present invention. Detailed Implementation

[0039] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0040] In the following description, specific details, such as particular internal procedures and techniques, are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will appreciate that the invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of the invention with unnecessary detail.

[0041] This invention provides a load balancing method for large-scale coupled programs with multi-level parallelism, comprising the following steps:

[0042] Based on the function instrumentation mechanism, combined with TLS, thread caching mechanism and measurement awareness algorithm, low-overhead performance tracing and function relationship call statistics are performed on large-scale multi-level parallel coupled programs.

[0043] Based on resource allocation and layout, a multi-component performance prediction model integrating component-level resource characteristics is constructed.

[0044] Automatically generate component layouts in parallel and with lightweight design, and adaptively allocate resources based on both shortest time and minimum machine time.

[0045] like Figure 1 As shown, the specific steps include:

[0046] Step 1: For multi-level parallel coupled programs (components, processes, threads), design hook instrumentation functions that separate multi-level parallel state awareness and measurement awareness; allocate thread-local time stacks (time_stk), function address stacks (func_addr_stk), component module stacks (module_stk), as well as thread-local caches and thread-local aggregation spaces for each worker thread; during the hook processing, record the probe start time (shell_start_time), function start time (self_start_time), and function end time (self_end_time) for each thread; and perform corresponding parallel state awareness processing on the current function address (self_addr), parent function address (father_addr), and component module identifier (current_module) in the serial / parallel regions respectively; based on the measurement awareness separation mechanism, calculate the function call count (times), function execution time (acctime), and probe envelope time (sheltime); among which, the function address stack (func_addr_stk)... The parallel state awareness operation is performed as follows: within the parallel zone, the parent-child call relationship is restored by "each thread independently maintaining the call relationship stack", and outside the parallel zone, the serial call context is copied by "synchronously pushing and popping all thread stacks", which takes into account both the isolation of multi-threaded calls and the consistency of call relationship restoration when switching between serial and parallel calls.

[0047] Step 2: Construct a two-dimensional hashmap storage structure using the current function address `self_addr` and the parent function address `father_addr` as two-dimensional keys. This structure features thread-local caching and aggregation for hash keys, and incorporates hook functions to store function performance statistics during program execution. The thread caching mechanism addresses the time-consuming calculation of two-dimensional hash keys by storing the function address mapping and hash index in a thread-private cache. Simultaneously, each thread maintains a local two-dimensional hashmap storage structure based on address keys, accumulating call frequency and execution time in real time, and performing global reduction and aggregation only at specific synchronization points or when the program ends.

[0048] Step 3: Based on the low-overhead performance analysis method, perform component-level result analysis to obtain historical running samples of multi-component coupled programs, including the number of processes {component}_pes, the number of threads {component}_thread, the component layout information layout, and the corresponding total program running time tot_time for each component;

[0049] Based on the number of processes and threads of each component, a multi-dimensional feature space is constructed, which includes component-level features and global statistical features. The component-level features include the process-thread product (workload), the resource proportion of each component, the logarithmic transformation value, and the reciprocal value.

[0050] Global statistical features include total resources, imbalance between components, and maximum resource share; at the same time, a layout matrix is ​​constructed based on component layout grouping information to calculate layout-aware features such as the number of aggregated cores within a group, standard deviation between groups, and peak ratio.

[0051] Step 4: Perform a second-order polynomial expansion on the multidimensional features to capture the nonlinear interaction between features. After standardization (StandardScaler), use the RidgeCV model for fitting and automatically search for the optimal regularization coefficient alpha to obtain the main model for performance prediction.

[0052] Step 5: Calculate the prediction residuals of the main model on the training data, and use the HuberRegressor robust regression model to fit the residuals to correct systematic bias; further calculate the median residuals of each group according to the layout, and perform differential calibration on the prediction results of different layouts using a shrinkage weighting method; determine the optimal combination of residual model fusion weights, layout correction strength and shrinkage coefficient through grid search to minimize the mean absolute percentage error (MAPE) index.

[0053] Through the training process of the above performance prediction model, combined with feature engineering and model adjustment in step 5, we obtain the model with the smallest mape, which can be used in subsequent steps.

[0054] like Figure 4 As shown, in step 6, obtain the component set, component identifier, total core count constraint, process candidate set, and thread candidate set of the multi-component coupled program to be optimized.

[0055] Step 7: Encode the component layout, unifying it into a standardized layout code that can be used for searching, constraint validation, and performance prediction; and reduce duplicate and invalid layouts through a lightweight generation strategy. For example, Figure 3As shown, the parallel lightweight layout generation method is as follows: taking the central initial component block as the starting point for layout generation, and expanding subsequent component blocks only within the neighborhood of the currently generated component block according to a preset relative offset rule to reduce the candidate layout search space; during the expansion process, boundary validity and occupancy validity checks are performed on the candidate positions, and only layout branches that meet the constraints are retained for continued generation; rotational symmetry and axisymmetric equivalence judgments are performed on the generation results and duplicate layouts are deleted to reduce the repeated search and storage of isomorphic layouts; and different expansion branches are assigned to parallel tasks for execution, and the results of each branch are then merged in a streaming manner, thereby achieving rapid generation of component layouts and reducing the time overhead of layout search.

[0056] Step 8: For each component layout candidate, a genetic algorithm is executed to search for the "minimum running time" and "minimum machine time" as optimization objectives. An initial population that meets the total core count constraint is generated based on domain knowledge constraints (such as the allowed number of threads for each component, the requirement that the number of processes must be even). Each individual consists of the number of processes and threads for each component. Then, after multiple generations of evolution, the predicted running time is calculated as the fitness for all individuals in the population using a prediction model in each generation. Elite individuals are retained, and the next generation is generated through crossover (randomly selecting parent genes for each component) and mutation (randomly replacing the process or thread value of a single component according to probability). Individuals that do not meet the constraints are repaired or randomly regenerated. An early stopping mechanism is supported, and the evolution is terminated when there is no improvement for several consecutive generations. After the genetic algorithm converges, local refinement is performed on the best individual (traversing all candidate values ​​for each component and greedily replacing the configuration that reduces the objective function) until no further improvement is possible. Finally, the globally optimal running time scheme and machine time scheme are selected from all layout patterns and output.

[0057] The following specific examples illustrate the load balancing method for large-scale coupled programs with multi-level parallelism provided by this invention.

[0058] This embodiment uses the Earth System Model FIO-ESM as its application background. FIO-ESM comprises multiple components such as atmosphere, ocean, sea ice, and land, and its operation employs a multi-level parallel model involving components, processes, and threads. For this model, this embodiment provides a low-overhead probe-based measurement method to achieve accurate perception of the performance of various components of a coupled program in a large-scale environment. The specific steps of this method are as follows:

[0059] Step 1.1: Build a probe runtime library that supports multi-level parallelism. Create the core file `hook.c`, and design a metadata structure in the hook function that supports dual identification of MPI process ID (RankID) and OpenMP thread ID (ThreadID). Utilize a memory pool to pre-allocate non-conflicting statistical node space for different threads, and pre-configure support for two-dimensional key-value pairs.<self_addr, father_addr> Its high-speed storage logic is built from the ground up to support a runtime architecture that supports hybrid parallel systems.

[0060] Step 1.2, full instrumentation of hybrid parallel code based on compiler mechanism. During the FIO-ESM compilation stage, by injecting the GCC / ICC -finstrument-functions option into the Makefile or build script, the compiler is forced to insert probe hooks at the entry and exit points of MPI communication functions, OpenMP computing kernels, and coupler scheduling functions of each component; at the same time, the linker embeds the probe static library or dynamic library generated in Step 1 into the target program, so that the program can cross the "process-thread" boundary at runtime and automatically capture all function-level execution details, including those inside and outside the parallel region.

[0061] Step 1.3: Initialize the multi-level measurement context for multi-level parallel sensing. During the MPI initialization phase after FIO-ESM startup (such as after MPI_Init), the probe library automatically acquires the physical topology information of the current process and uses thread-local storage (TLS) to establish a private measurement environment for all potential OpenMP worker threads within the process, including the function address stack func_addr_stk, the time stack time_stk, and the module stack module_stk. Through this layout of "one global table per process, one private stack per thread," both unified archiving of process-level data and complete isolation of thread-level execution flow are ensured.

[0062] Step 1.4, serial-parallel state identification at the function entry point. When the program executes to the instrumentation location and calls __cyg_profile_func_enter, the probe first determines in real time whether the current code is in the "MPI single-threaded serial zone" or the "OpenMP multi-threaded parallel zone" by calling the OpenMP runtime status interface: If in the serial zone, the main thread, while performing stack pushing, notifies all slave threads in a sleeping or standby state to synchronously copy the call stack top information of the main thread through atomic signals; if in the parallel zone, each slave thread independently records its own thread number and maintains a private call stack, ensuring that when the program enters the OpenMP parallel task, all newly started threads can accurately identify their parent function call context from the serial zone.

[0063] Step 2.1: Recovery of Hybrid Call Relationships and Performance Statistics at Function Exit. When `__cyg_profile_func_exit` is triggered at the function exit, the probe executes the corresponding logic based on the current parallel state: within the parallel zone, the thread restores the parent-child mapping between `self_addr` and `father_addr` through the private `func_addr_stk`, accurately capturing the load differences of each branch in a multi-threaded environment; upon exiting the parallel zone and returning to serial execution, the probe triggers a synchronized stack pop of the main thread and all child threads, cleaning up redundant temporary contexts. This process records the execution time of each level of the parallel structure in real time and calculates various metrics using a measurement-aware separation mechanism, as shown in the following formula:

[0064] The execution time of the function under test:

[0065] Probe envelope overhead time:

[0066] Number of calls:

[0067] Step 2.2, Layered Storage and Lock-Free Optimization Based on Hybrid Parallelism. To address the memory contention problem caused by tens of thousands of threads simultaneously updating statistical data in hybrid parallelism, the probe adopts a "thread-level fast recording—global-level batch merging" model. Each OpenMP thread first retrieves the corresponding hash node through its private thread-local cache and accumulates execution time and call frequency within its lock-free thread-private space. Only when a specific thread's private storage space reaches a threshold or encounters an MPI synchronization barrier is the aggregated data atomically flushed to the global HashMap of its respective MPI process by the main thread or a designated worker thread. This achieves data aggregation while maintaining hybrid parallelism efficiency.

[0068] Step 2.3, Process and component-level result output for coupled architecture. At the end of the FIO-ESM simulation task, the probe runtime library traverses the global storage structure of each MPI process and, combined with the identifiers in `module_stk`, automatically decomposes the performance data into statistical reports for different physical components such as the atmosphere, ocean, and coupler; this provides high-confidence raw data for subsequent adaptive resource allocation and layout optimization based on the hybrid parallel characteristics of each component. For example... Figure 2 The image shows an example of the measurement results from the ATM component.

[0069] The probe was applied to the FIO-ESM f09_g16 test case, simulating 6n days, and performance modeling was performed based on five components: atmosphere (atm), land (lnd), sea ice (ice), ocean (ocn), and coupler (cpl). The steps included:

[0070] Step 3.1: Based on the historical runtime data of the FIOESM test cases, extract the number of processes (e.g., atm_pes, lnd_pes), the number of threads (e.g., atm_thread, lnd_thread), component layout information, and the corresponding total application time (tot_time) for each component. Perform a logarithmic transformation on the total program runtime (y=ln(1+tot_time)) to improve the numerical stability of model training; simultaneously, use the StratifiedShuffleSplit algorithm to divide the dataset into a training set (208 records), a validation set (14 records), and a test set (15 records) (divided according to the ratio of 88% training, 6% validation, and 6% test), and use StandardScaler to normalize the input process and thread data.

[0071] Step 3.2: Construct a multi-dimensional feature space that integrates component-level features and global layout features. Based on the number of basic processes and threads of each component, extract multi-dimensional physical and statistical features, as shown in Table 1 and Table 2:

[0072] Component-level features: Calculate the workload of each component (product of the number of processes and the number of threads), process / thread ratio, resource allocation (such as the proportion of a single component's processes to the total number of processes), and the reciprocal and logarithmic transformation values ​​of the corresponding values.

[0073] Global statistical features: Calculate the total number of processes, threads, and workload, as well as the resource imbalance between components (maximum value minus minimum value) and the maximum resource share.

[0074] Layout-aware features: The discrete component layout is parsed into a standardized layout matrix, and the number of aggregate cores, the average core value, the maximum core value, the standard deviation between groups, and the peak ratio (maximum value / mean value) within the same layout group are calculated.

[0075] Table 1 Component-level and Global Features

[0076]

[0077] Table 2 Layout-aware features

[0078]

[0079] Step 4: Construct the main prediction model based on second-order polynomial expansion. To capture the nonlinear interaction between the resource configurations of each component, the multidimensional features obtained in Step 8 are expanded using second-order polynomial features (degree=2). The model automatically searches for the optimal regularization coefficient (alpha=50.0 in this embodiment) from the set of candidate regularization coefficients [10, 50, 100, 500, 1000, 5000, 10000] through built-in cross-validation to obtain the main regression model and output the preliminary prediction runtime.

[0080] Step 5.1: Construct a residual correction model based on robust regression. To correct the systematic bias of the main model, calculate the prediction residual of the main model on the training data (i.e., the difference between the actual running time and the predicted time of the main model). Using the original multidimensional features (without polynomial expansion) generated in Step 9 as input, fit the residual using the Huber robust regression model (with parameters epsilon=1.05, alpha=1e-4). This model is robust to outliers, effectively corrects the systematic bias of the main model, and is not affected by abnormal samples.

[0081] Step 5.2: Perform layout-based differential Shrinkage calibration and model fusion.

[0082] The median residuals of the training samples in each group were calculated based on the layout, and a shrinkage weighting formula was used. Differential calibration is performed on the prediction results (wherein) (This refers to the number of samples for the layout in the training set). Layouts with more samples receive larger correction weights, while layouts with fewer samples have their correction weights automatically reduced to avoid overfitting. Simultaneously, a grid search is performed on the hybrid hyperparameters on the training set: traversing the residual model fusion weights (range [0, 1), step size 0.05), the layout correction strength scale (range [0, 1.5), step size 0.05), and the shrinkage coefficients (set {0.5, 1, 2, 3}), with the goal of minimizing MAPE. The optimal combination determined in this embodiment is: primary_weight=1.00, residual_weight=0.00, scale=1.15, shrinkage=3.0.

[0083] Step 5.3: Output prediction results and model error evaluation. Input the test set data into the fusion verification model after optimization in step 12, restore the prediction results to the real physical time space through exponential transformation (expm1), and compare them with the actual running time for evaluation. Testing shows that the prediction model constructed in this embodiment performs excellently on the test set, and its core error metrics are as follows:

[0084] The mean absolute percentage error (MAPE) was 5.83%.

[0085] The median absolute percentage error (Median APE) was 4.36%.

[0086] Coefficient of determination (R) 2 It reached 0.9888.

[0087] Test results show that the model can accurately predict the execution time of large-scale programs with complex layouts while maintaining extremely low inference overhead, providing a reliable quantitative basis for subsequent studies.

[0088] Step 6: Given a total CPU core count budget of 1024, a component set of {atm(1), lnd(2), ice(3), ocn(4), cpl(5)}, and thread candidate values ​​of {1, 2, 4}, the process search range is [32, 1024] with a step size of 2 (only even values ​​are considered). This includes the following steps:

[0089] Step 7: On a square grid with a side length of 20 (i.e., the number of components × 4), each component is abstracted as a module occupying a 2×2 grid. After placing the first module at the center of the grid, subsequent modules must be adjacent to the already placed modules in the up, down, left, and right directions (there are 3 candidate offset positions in each direction, for a total of 12 candidate points), and must not overlap with existing modules. An iterative depth-first search is used to traverse all valid 5-module placement sequences. To improve efficiency, the effective branches of the second layer (about 8 in this example) are first distributed to multiple processes to expand each subtree in parallel. After all branches have been searched, the results are merged, and deduplication is accelerated by hashing: the MD5 hash value of each candidate pattern is calculated, and the hash values ​​after three rotations (90° / 180° / 270°) and two axisymmetric transformations (horizontal flip, vertical flip) are also calculated. If the hash of any transformation already exists in the included set, it is determined to be a duplicate pattern and is removed. Finally, all the deduplicated unequal layout patterns are obtained, extended and aligned, and saved as a NumPy array file for downstream optimization. The layout generation time was 5.12 seconds.

[0090] Step 8.1: Load the performance prediction model saved during the training phase (including the RidgeCV main model, polynomial transformer, residual model and layout correction parameters), and at the same time read the candidate layouts from the layout pattern file. After filtering by domain knowledge constraints, retain the layout patterns that have been verified to be runnable.

[0091] Step 8.2, Initial Population Generation. For each layout pattern, a constrained random backtracking algorithm is used to generate 64 initial individuals. Specifically, a thread value is randomly selected for each component, and then a process value is randomly selected sequentially according to the component order. Backtracking search ensures that the sum of the physical cores in each group does not exceed the total budget of 1024 cores. Combinations that do not meet the constraints are pruned and discarded, and are corrected by a repair function after generation.

[0092] Step 8.3, Fitness Assessment. For each individual in the population, first calculate its physical core usage; if it exceeds the total budget of 1024, a large penalty value is assigned. Next, check the core balance of each parallel group; individuals with an imbalance ratio exceeding 10% are also penalized. After constraint checks, call the performance prediction model to calculate the estimated runtime for this configuration. The final fitness score is: when the goal is "minimum runtime", the predicted time is taken directly; when the goal is "minimum machine time", the product of the predicted time and the number of physical cores is taken.

[0093] Step 8.4, Genetic Evolution. The 8 elite individuals with the best fitness in each generation directly advance to the next generation; the remaining individuals are generated by randomly selecting two parents from the elite pool, performing a crossover operation that randomly selects parent genes component by component, and then randomly mutating the process or thread value of each component with a probability of 0.20. Individuals that do not meet the constraints after mutation are repaired or randomly regenerated and replaced. The maximum iteration is 40 generations; if the optimal fitness does not improve for 5 consecutive generations, early termination is triggered.

[0094] Step 8.5, Local Refinement. After the genetic algorithm converges, a deterministic local search is performed on the globally optimal individual: traversing all combinations of candidate process values ​​and thread values ​​for each component, if the fitness strictly decreases after replacement, the replacement is accepted, and this process is repeated until all candidate values ​​for all components cannot be further improved.

[0095] Step 8.6, Global Optimal Selection and Output. From the search results of all layout patterns, select the globally optimal solution for both runtime and machine time objectives. The final result of this embodiment is:

[0096] Minimum runtime scheme: Using layout [(1,4),(2,4),(3,4),(4,4),(5,4)], total cores: 992, predicted runtime: 97.3736 seconds, configured as atm=144*4, lnd=144*2, ice=160*2, ocn=384*1, cpl=64*1;

[0097] Minimum runtime scheme: Using the layout [(1,4),(2,4),(3,4),(4,4),(5,4)], with a total of 896 cores, the predicted runtime is 105.3627 seconds, and the runtime is 94405.0533 core-seconds. The configuration is atm=128*4, lnd=128*1, ice=120*2.

[0098] ocn=384*1,cpl=72*1;

[0099] Default configuration: Layout [(1,),(2,),(3,),(4,),(5,)], total cores 1024, runtime 160.553 seconds, machine time 164406.272 cores / seconds, configured as atm=1024*1, lnd=1024*1, ice=1024*1, ocn=1024*1.

[0100] cpl=1024*1, the result is as follows Figure 5 As shown.

[0101] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A load balancing method for large-scale coupled programs with multi-level parallelism, characterized in that, Includes the following steps: Based on the function instrumentation mechanism, combined with TLS, thread caching mechanism and measurement awareness algorithm, low-overhead performance tracing and function relationship call statistics are performed on large-scale multi-level parallel coupled programs. Based on resource allocation and layout, and combined with the coupled program performance information obtained by function instrumentation mechanism, a multi-component performance prediction model that integrates component-level resource characteristics is constructed. The system automatically generates component layouts in a parallel and lightweight manner, embeds a performance prediction model into a genetic search algorithm, and performs adaptive resource allocation based on two dimensions: shortest time and minimum machine time.

2. The load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 1, characterized in that, Based on function instrumentation, combined with TLS, thread caching mechanisms, and measurement-aware algorithms, low-overhead performance tracing and function call statistics are performed on large-scale, multi-level parallel coupled programs. Specifically, the following steps are included: For multi-level parallel coupled programs including components, processes, and threads, a hook function is designed to separate multi-level parallel state awareness and measurement awareness. Each worker thread is allocated a thread-local time stack (time_stk), a function address stack (func_addr_stk), a component module stack (module_stk), as well as a thread-local cache and a thread-local aggregation space. Among these, the parallel awareness of self_addr and father_addr is implemented through the parallel state awareness of func_addr_stk. During the hooking process, the probe start time (shell_start_time), function start time (self_start_time), and function end time (self_end_time) of each thread are recorded; and corresponding parallel state awareness processing is performed on the current function address (self_addr) and parent function address (father_addr) of the serial / parallel region respectively; based on the measurement-aware separation mechanism, the function call count (times), function execution time (acctime), and probe envelope time (sheltime) are calculated. A two-dimensional hashmap storage structure is constructed using self_addr and father_addr as two-dimensional keys. It is thread-local cached and thread-local aggregated for hash keys, and combines hook functions to store function performance statistics during program execution.

3. The load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 2, characterized in that, The specific way the function address stack func_addr_stk performs parallel state awareness operations is as follows: within the parallel area, the parent-child call relationship is restored by "each thread independently maintaining the call relationship stack", and outside the parallel area, the serial call context is copied by "synchronously pushing and popping all thread stacks", which takes into account both the isolation of multi-threaded calls and the consistency of call relationship restoration when switching between serial and parallel.

4. The load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 2, characterized in that, The thread caching mechanism is as follows: to address the time-consuming problem of calculating two-dimensional hash key-value pairs, the system stores the function address mapping relationship and hash index in a thread-private cache; at the same time, each thread maintains a two-dimensional HashMap storage structure based on address key-value pairs locally, accumulates the call frequency and execution time in real time, and performs global reduction aggregation only at specific synchronization points or when the program ends.

5. The load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 1, characterized in that, Based on resource allocation and layout, a multi-component performance prediction model that integrates component-level resource characteristics is constructed, specifically including the following steps: Based on low-overhead performance analysis methods, component-level result analysis is performed to obtain historical running samples of multi-component coupled programs; based on the number of processes and threads of each component, a multi-dimensional feature space containing component-level features and global statistical features is constructed; at the same time, a layout matrix is ​​constructed according to the component layout grouping information, and layout-aware features are calculated. A second-order polynomial expansion is performed on the multidimensional feature space to capture the nonlinear interaction between features. After standardization by StandardScaler, the RidgeCV model is used for fitting, and the optimal regularization coefficient alpha is automatically searched to obtain the main model for performance prediction. The prediction residuals of the main model on the training data are calculated, and the prediction residuals are fitted with the HuberRegressor robust regression model to correct systematic bias. The median residuals of each group are calculated by layout, and the prediction results of different layouts are differentiated by shrinkage weighting. The optimal combination of residual model fusion weights, layout correction strength and shrinkage coefficient is determined by grid search to minimize the MAPE index.

6. The load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 5, characterized in that, Historical runtime samples of multi-component coupled programs include the number of processes {component}_process, the number of threads {component}_thread, the component layout information layout, and the corresponding total program runtime tot_time for each component.

7. A load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 5, characterized in that, The component-level features include the process-thread product (workload), the resource proportion of each component, the logarithmic transformation value, and the reciprocal value.

8. A load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 5, characterized in that, The global statistical features include total resources, imbalance between components, and maximum resource share.

9. A load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 1, characterized in that, Automatically generate component layouts in a parallel and lightweight manner, and adaptively allocate resources based on both the shortest time and minimum machine time. Specifically, this includes the following steps: Obtain the component set, component identifier, total core count constraint, process candidate set, and thread candidate set of the multi-component coupled program to be optimized; The component layout is encoded to represent it as a standardized layout code that can be used for searching, constraint verification, and performance prediction; and a parallel lightweight layout generation method is used to reduce duplicate and invalid layouts. For each component layout candidate, a genetic algorithm is executed to search for the "minimum running time" and "minimum machine time" as optimization objectives. An initial population that meets the total core number constraint is generated based on domain knowledge constraints, and each individual consists of the number of processes and threads of each component. Then, after multiple generations of evolution, the predicted running time is calculated as the fitness of all individuals in the population by calling the prediction model in each generation. Elite individuals are retained, and the next generation is generated through crossover and mutation. Individuals that do not meet the constraints are repaired or randomly generated again. An early stopping mechanism is supported, and the evolution is terminated when there is no improvement for several consecutive generations. After the genetic algorithm converges, local refinement is performed on the best individual until no further improvement is possible. Finally, the globally optimal running time scheme and machine time scheme are selected from all layout patterns and output.

10. A load balancing method for large-scale coupled programs with multi-level parallelism as described in claim 9, characterized in that, The parallel lightweight layout generation method specifically includes: taking the central initial component block as the starting point for layout generation; expanding subsequent component blocks only within the neighborhood of the currently generated component block according to a preset relative offset rule to reduce the candidate layout search space; performing boundary and occupancy validity checks on candidate positions during the expansion process, retaining only layout branches that meet the constraints to continue generation; performing rotational symmetry and axisymmetric equivalence checks on the generated results and deleting duplicate layouts to reduce repeated searches and storage of isomorphic layouts; and allocating different expansion branches to parallel tasks for separate execution, and then streaming the results of each branch to achieve rapid generation of component layouts and reduce the time overhead of layout search.

Images