Dynamic load balancing method and device for parallel numerical simulation and storage medium

Abstract

The invention relates to the field of numerical simulation, and provides a dynamic load balancing method and device for parallel numerical simulation and a storage medium. The method comprises the following steps: constructing a load graph based on discrete grids of a computational domain; extracting a multi-dimensional feature vector for each vertex; processing the multi-dimensional feature vector by using a pre-trained graph neural network model to predict the load weight of each vertex in parallel calculation; according to the load weight and the grid connection relation of each vertex in parallel computing, generating a dynamic partitioning scheme through a graph partitioning tool, and mapping partitions to a plurality of processors; monitoring the load unbalance degree of each processor in real time in the numerical simulation process, and triggering a re-partitioning operation when the unbalance degree exceeds a preset threshold value; and repeating the steps of load prediction, partitioning and monitoring until the simulation is finished. According to the technical scheme provided by the invention, the accuracy and adaptive capacity of load balancing in parallel numerical simulation are improved, so that the calculation efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of numerical simulation, and in particular to a dynamic load balancing method, apparatus and storage medium for parallel numerical simulation. Background Technology

[0002] Parallel numerical simulations, such as computational fluid dynamics, structural mechanics, and multiphysics simulations, are crucial tools for solving complex scientific and engineering problems. They achieve high-precision solutions within an acceptable timeframe by discretizing large-scale computational domains into a mesh and distributing the mesh to multiple processors for parallel processing. In this process, load balancing is a key factor determining parallel efficiency and computational resource utilization. An ideal load-balanced state requires all processors to complete computational tasks simultaneously, avoiding overall performance degradation due to idle processors.

[0003] Currently, in both industry and academia, graph-based static partitioning methods are the mainstream technique for load distribution in parallel numerical simulations. These methods partition the computational grid once before simulation begins, based on its topology (connectivity), and maintain this partitioning scheme throughout the computation. The fundamental assumption is that the computational cost of each grid cell is the same or negligible.

[0004] However, as the complexity of simulation problems increases, the existing technical solutions have gradually revealed their inherent limitations. First, they cannot adapt to dynamically changing loads during simulation. In real-world unsteady, nonlinear, or adaptive mesh refinement (AMR) simulations, the computational load in different regions changes drastically over time or during the solution process. Static partitioning cannot sense or respond to these changes, causing early balanced partitioning to fail in subsequent calculations, resulting in severe load imbalance. Second, it is difficult to accurately quantify and integrate complex load heterogeneity. Computational load depends not only on mesh geometry but also on the combined influence of various factors such as physical field states (e.g., shock waves, phase transition interfaces), material properties, and local convergence. Traditional static partitioning methods typically only consider simple static indicators such as the number of mesh vertices or elements, lacking the ability to deeply integrate and model multidimensional, time-varying load characteristics. This leads to significant deviations between their partitioning criteria and actual computational overhead, making it difficult to achieve true load balance in complex simulations. Summary of the Invention

[0005] This application provides a dynamic load balancing method, apparatus, and storage medium for parallel numerical simulation. By dynamically predicting load weights and adjusting partitions in real time through graph neural networks, the accuracy and adaptability of load balancing in parallel numerical simulation are improved, thereby increasing computational efficiency.

[0006] On the one hand, this application provides a dynamic load balancing method for parallel numerical simulation, the method comprising: A load graph is constructed based on a discrete grid of the computational domain, wherein the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections. For each vertex, a multi-dimensional feature vector is extracted, which includes geometric features, physical field features, and computational state features; The multidimensional feature vectors are processed using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computing. Based on the load weight of each vertex in parallel computing and the mesh connectivity, a dynamic partitioning scheme is generated using a graph partitioning tool, and the partitions are mapped to multiple processors. During numerical simulation, the load imbalance of each processor is monitored in real time. When the imbalance exceeds a preset threshold, a repartitioning operation is triggered. Repeat the load prediction, partitioning, and monitoring steps described above until the simulation ends.

[0007] On the other hand, this application provides a dynamic load balancing device for parallel numerical simulation, the device comprising: A construction module is used to construct a load graph based on a discrete grid of the computational domain. The vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections. The extraction module is used to extract a multi-dimensional feature vector for each vertex, the feature vector including geometric features, physical field features and computational state features; The prediction module is used to process the multidimensional feature vector using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computing. The generation module is used to generate a dynamic partitioning scheme based on the load weight of each vertex in parallel computing and the mesh connectivity, and to map the partitions to multiple processors. The monitoring module is used to monitor the load imbalance of each processor in real time during numerical simulation. When the imbalance exceeds a preset threshold, a repartitioning operation is triggered. The iteration module is used to repeat the above load prediction, partitioning, and monitoring steps until the simulation ends.

[0008] Thirdly, this application provides an apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the technical solution of the dynamic load balancing method for parallel numerical simulation described above.

[0009] Fourthly, this application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described dynamic load balancing method for parallel numerical simulation.

[0010] As can be seen from the technical solution provided in this application, on the one hand, by constructing a load graph and extracting multi-dimensional dynamic features such as geometry, physical field, and computational state of the grid cells, and using a graph neural network model for learning and prediction, this method can deeply integrate multi-source heterogeneous factors affecting computational costs, thereby generating load weights that more accurately reflect the actual computational overhead of the cells. This provides a more accurate input basis for subsequent partitioning, improving the quality of the partitioning scheme from the source. On the other hand, the method of this application is not a one-time static partitioning, but rather monitors the load imbalance in real time during the simulation process and dynamically triggers repartitioning operations. This allows the partitioning scheme to adaptively adjust to changes in load distribution, migrating computational tasks from heavily loaded processors to lightly loaded processors, thereby maintaining a high and stable load balance level throughout the simulation cycle, reducing processor idle waiting time, and improving the utilization rate of computing resources and the overall solution speed. Thirdly, the method of this application combines a graph neural network model with classic graph partitioning tools to form a complete, data-driven adaptive load balancing closed-loop process. This process reduces the reliance on human experience and fixed rules in traditional methods, enabling it to automatically learn load patterns in complex simulations and autonomously make partitioning decisions, thus enhancing the applicability and intelligence of the method in complex and ever-changing scenarios. In summary, the technical solution of this application effectively improves the accuracy and adaptability of load balancing in parallel numerical simulations by dynamically predicting load weights and adjusting partitions in real time through graph neural networks, thereby significantly improving computational efficiency. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a dynamic load balancing method for parallel numerical simulation provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the dynamic load balancing device for parallel numerical simulation provided in the embodiments of this application; Figure 3 This is a schematic diagram of the device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0015] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0016] Currently, graph-based static partitioning methods are the mainstream technique for load distribution in parallel numerical simulations in both industry and academia. These methods partition the computational grid once before simulation begins, based on its topology (connectivity), and maintain this partitioning scheme throughout the computation. The fundamental assumption is that the computational cost of each grid cell is the same or negligible. However, as the complexity of simulation problems increases, the inherent limitations of these existing techniques have become increasingly apparent. First, they cannot adapt to dynamically changing loads during simulation. In real-world unsteady, nonlinear, or adaptive mesh refinement (AMR) simulations, the computational load in different regions changes drastically over time or during the solution process. Static partitioning cannot detect or respond to these changes, causing early balanced partitioning to fail in subsequent calculations, resulting in severe load imbalance. Second, it is difficult to accurately quantify and integrate complex load heterogeneity. Computational load depends not only on the grid geometry but also on a combination of factors such as physical field states (e.g., shock waves, phase transition interfaces), material properties, and local convergence. Traditional static partitioning methods typically only consider simple static indicators such as the number of mesh vertices or cells, lacking the ability to deeply integrate and model multidimensional and time-varying load characteristics. This results in a significant deviation between the partitioning criteria and the actual computational overhead, making it difficult to achieve true load balancing in complex simulations.

[0017] To address the aforementioned problems in the prior art, this application proposes a dynamic load balancing method for parallel numerical simulation, the flowchart of which is attached. Figure 1 As shown, the main steps include S101 to S106, which are detailed below: Step S101: Construct a load graph based on the discrete grid of the computational domain, where the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to the grid topology connection relationships.

[0018] In parallel numerical simulations, the computational domain is first discretized into a mesh containing a large number of elements (e.g., tetrahedrons, hexahedrons, etc.) or nodes. Traditional static partitioning methods only focus on the geometric topology of the mesh, while this application constructs a load graph based on the discrete mesh of the computational domain, where the vertices of the load graph correspond to mesh elements or nodes, and the edges correspond to the mesh topological connections.

[0019] Specifically, the process of constructing the load graph is as follows: Each grid cell or node is mapped to a vertex of the graph. If two grid cells share an edge or a face (i.e., are physically adjacent), an undirected edge is created between their corresponding vertices. In this way, the entire topology of the computational grid is completely mapped to a graph data structure. ,in, It is a set of vertices. It is a set of edges. This graph structure not only describes the geometry of the computational domain, but more importantly, it will become the skeleton for subsequent graph neural networks to perceive and transmit load information.

[0020] Step S102: Extract a multi-dimensional feature vector for each vertex, wherein the feature vector includes geometric features, physical field features and computational state features.

[0021] A major drawback of traditional partitioning methods is that they approximate the load using only simple, static metrics (such as cell volume or adjacency count), which fails to reflect the load heterogeneity caused by the complexity of physical processes and the dynamic nature of numerical solutions in real simulations. To address this issue, this application extracts a comprehensive, dynamic feature vector for each vertex in the load graph. As an embodiment of this application, the extraction of a multi-dimensional feature vector for each vertex can be achieved through the following steps S1021 to S1023, detailed below: Step S1021: Dynamically acquire the geometric features of each vertex, including mesh density, mesh volume, and mesh quality index.

[0022] Mesh density is quantified by calculating the degree of a vertex, i.e., the number of edges connected to that vertex, reflecting the tightness of the cell's connection within the mesh. Mesh volume directly calculates the volume occupied by the mesh cell in physical space. Mesh quality metrics are a set of parameters that measure the quality of mesh cell shapes, such as cell type (e.g., triangle, quadrilateral) and skewness, the latter obtained by calculating geometric relationships such as the ratio of the inscribed sphere radius to the circumscribed sphere radius. Poor-quality meshes can lead to difficulties in numerical solutions, thus increasing the computational load.

[0023] Step S1022: Real-time acquisition of physical field characteristics, including the physical quantity values ​​and their gradient magnitudes at the current time step.

[0024] This is crucial for capturing dynamic changes in load. For example, in computational fluid dynamics simulations, it is necessary to obtain the physical quantity values ​​within each cell at the current time step in real time, such as density. ,pressure ,temperature And so on. Furthermore, the rate of change of a physical quantity in space often better reflects computational complexity. Therefore, it is necessary to calculate the magnitude of the gradient of a physical quantity, such as the velocity gradient. or stress gradient In high gradient regions (e.g., shock waves, boundary layers), solvers typically require finer processing or more iterations.

[0025] Step S1023: Record the computation state characteristics, including the number of local iterations and the local time step size in the previous time step.

[0026] This feature comes directly from the solver's internal state. The local iteration count records the number of iterations required by the nonlinear or linear solver in the previous time step to satisfy the convergence criteria for the region where the element is located. A high number of iterations usually means high computational cost for that region. The local time step size is particularly important in explicit time integration methods or adaptive time stepping methods, as it reflects the maximum time step allowed for the element to maintain numerical stability.

[0027] By integrating the aforementioned geometric features, physical field features, and computational state features, a high-dimensional feature vector is constructed for each vertex. This vector can comprehensively and dynamically characterize its potential load in parallel computing.

[0028] Step S103: Use a pre-trained graph neural network model to process the multidimensional feature vectors to predict the load weight of each vertex in parallel computing.

[0029] After obtaining the feature vectors, the core challenge is how to fuse the vertex's own features with its neighborhood information in the graph structure to accurately predict the load. Traditional machine learning models, such as simple linear regression or isolated processing of each vertex's features, are inadequate for this task. This application employs an advanced model, Graph Neural Networks (GNNs), to address this problem. Specifically, the pre-trained graph neural network model is used to process the multi-dimensional feature vectors to predict the load weight of each vertex in parallel computation, which can be achieved through steps S1031 and S1032, as detailed below: Step S1031: Training the graph neural network model.

[0030] The goal of graph neural network model training is to train a GNN model that can accurately map features to load weights, specifically including the following implementation schemes a, b, c, and d.

[0031] a. Constructing the training dataset: The dataset contains thousands of mesh graph samples from different simulation scenarios. Each sample has its true load weight label. Crucially, this information is obtained through several methods, including: real-time monitoring of the computation time of each processor during historical simulation runs; inferring the actual computation time of each grid cell based on processor time and cell allocation relationships; and normalizing the actual computation time to use it as the true load weight label. For example, if the total computation time of all cells on a processor is... And the processor contains M If there are [number] units, then the computation time for each unit can be approximated as [time]. Then normalize it to a standard range (e.g., between 0 and 1) as .

[0032] b. Initial Feature Embedding: Initialize the vertex feature vectors... The input is a multilayer perceptron (MLP) that undergoes an initial transformation, projecting it into a higher-dimensional, more expressive hidden space to obtain the initial hidden features of the vertices. .

[0033] c. Graph Vertex Feature Update: This is the core of GNN. This application uses a Graph Attention Network (GAT) to calculate the attention weights between vertices, in order to adaptively aggregate neighborhood information. Specifically, for vertices... i and his neighbors The GAT layer calculates the attention coefficient:

[0034] in, It is a weight matrix. It is an attention vector transpose This represents the concatenation operation. Then, the coefficients are normalized using the softmax function to obtain the attention weights. .vertex i The new features are obtained by weighted aggregation of their neighbor features: .

[0035] To enhance the stability of feature representations using a multi-head attention mechanism, this application concatenates or averages the outputs of multiple independent attention mechanisms (heads):

[0036] in, KIt refers to the number of heads. After multiple rounds of such message passing, the vertex features will contain information about its multi-hop neighborhood.

[0037] Residual connections are used to avoid gradient vanishing during training. In deep GNNs, the input and output of the l-th layer are added together, normalized, and then passed to the next layer. .

[0038] d. Load weight coefficient prediction and loss calculation: After K-layer GNN message passing, the final vertex features are obtained. Then, based on the updated feature regression, the load weights are predicted via another MLP regression head: .

[0039] During training, the loss function is calculated using the mean squared error: And use optimizers such as Adam to minimize the loss until the model converges.

[0040] Step S1032: Make predictions using a pre-trained GNN model.

[0041] During the simulation inference phase, the load graph constructed at the current time step and the extracted multidimensional feature vectors are input into the pre-trained GNN model. The model automatically executes the feature update process and outputs the value of each vertex. Predictive calculation of load weight This weight is a scalar that precisely reflects the relative computational cost of that mesh cell under the current simulation conditions. Step S104: Based on the load weight of each vertex in parallel computing and the mesh connectivity, generate a dynamic partitioning scheme using a graph partitioning tool, and map the partitions to multiple processors.

[0042] After obtaining the precise load weight for each grid cell, the next crucial step is to generate a dynamic partitioning scheme using a graph partitioning tool based on the load weight and grid connectivity, and then map the partitions to multiple processors. The goal of this step is to divide the computational task (i.e., the set of grid cells) into several partitions and allocate them to different processors, so that the total load across the partitions is as balanced as possible, while minimizing the communication overhead between partitions (typically related to the number of edges cut at the partition boundaries).

[0043] In specific implementation, this application preferably uses a mature graph partitioning tool library (e.g., METIS, ParMETIS, or Zoltan, etc.) to implement this process. That is, based on the load weight of each vertex in parallel computing and the mesh connectivity, a dynamic partitioning scheme is generated by the graph partitioning tool, and the partitions are mapped to multiple processors. This can be achieved through the following steps S1041 to S1043: Step S1041: Input the load weight as the vertex weight into the graph partitioning tool interface.

[0044] For each vertex predicted by the GNN in step S103 Load weight The weight of that vertex is input to the graph partitioning tool. Simultaneously, the load graph constructed in step S101... The connection relationships between vertices and edges are defined. This is a significant improvement over traditional partitioning methods, which typically set all vertex weights to be the same or are based solely on simple geometric information. In contrast, the weights provided in this application are dynamically predicted values ​​that reflect the actual computational cost, thus providing high-quality input for partitioning tools.

[0045] Step S1042: Generate a partitioning scheme that minimizes communication overhead based on the processor quantity constraint.

[0046] Call the graph partitioning tool's API (e.g., METIS_PartGraphKway function in METIS) and specify the target number of partitions P (usually equal to the number of available processors). The core algorithm of the partitioning tool (e.g., a multi-level graph partitioning algorithm) aims to minimize edge cuts (i.e., the sum of the weights of edges connecting different partitions) as one of its main optimization objectives, while strictly constraining the balance of the sum of vertex weights in each partition (i.e., the total computational load). Its mathematical objective can be expressed as: while satisfying Under the premise of minimizing ,in, It is the set of edges that have been cut. This is the edge weight (usually set to 1 or based on physical connection strength). This ensures low inter-processor communication while maintaining load balancing.

[0047] Step S1043: Map the new partition to the processor cluster using the parallel communication library and update the task allocation synchronously.

[0048] The graph partitioning tool outputs the partition number to which each vertex belongs. In parallel programs, this partitioning result needs to be used to map the new partitions to the processor cluster. This involves data migration using parallel communication libraries such as MPI (Message Passing Interface): each processor, according to the new partitioning scheme, sends mesh cell data that does not belong to its new partition to the target processor and receives data belonging to its new partition. After this process is complete, all processors synchronously enter the new computation phase. To achieve a smooth transition, techniques such as double buffering can be used to avoid computational interruptions.

[0049] Step S105: Monitor the load imbalance of each processor in real time during the numerical simulation. When the imbalance exceeds the preset threshold, trigger the repartitioning operation.

[0050] Static partitioning schemes gradually become ineffective in dynamically changing simulations. Therefore, this application designs a closed-loop feedback control mechanism, specifically: during the numerical simulation, the load imbalance of each processor is monitored in real time, and when the imbalance exceeds a preset threshold, a repartitioning operation is triggered. As an embodiment of this application, triggering the repartitioning operation can be implemented through steps S1051 to S1053, as detailed below: Step S1051: Periodically calculate the load imbalance coefficient.

[0051] During the simulation, the computation time of all processors is collected periodically (e.g., after every N time steps). Assuming there are P processors, their computation times are represented as a vector. Then, calculate the load imbalance coefficient. ,in, It is the maximum processor time. This is the average processor time. (Coefficient) K It intuitively reflects the degree of system load imbalance. K =1 indicates perfect equilibrium. K The larger the value, the more severe the imbalance.

[0052] Step S1052: Dynamically calculate the threshold based on the accuracy requirements of the current simulation stage. .

[0053] Setting a fixed threshold, while simple, lacks adaptability. This application preferably employs a dynamic threshold strategy. The settings need to balance the overhead and benefits of repartitioning. Based on the accuracy requirements of the current simulation phase, the threshold is dynamically calculated. For example, in the early stages of simulation or during periods of drastic changes in the physical field, where load balancing is critical and more frequent repartitioning is permissible, a smaller load balancer can be set. (For example, =1.2). When the simulation tends to stabilize, the requirements can be appropriately relaxed, and a larger value can be set. (For example, =1.5) to reduce unnecessary repartitioning overhead. Accuracy requirements can be indirectly determined based on parameters such as time step and residual convergence criteria.

[0054] Step S1053: Determine and trigger repartitioning.

[0055] The calculated imbalance coefficient is compared with the dynamic threshold. Repartitioning is automatically triggered. Once triggered, the simulation process will pause the current calculation and jump to step S101, restarting the complete process of load graph construction, feature extraction, load prediction, and partitioning based on the latest mesh and physical field states.

[0056] Step S106: Repeat the load prediction, partitioning, and monitoring steps until the simulation ends.

[0057] Repeat the load prediction, partitioning, and monitoring steps described above until the simulation ends. This demonstrates that the method in this application is not a one-off action, but a continuously running, automated optimization process throughout the entire simulation lifecycle. It ensures that regardless of how the load dynamically changes, the system can maintain a high level of parallel computing efficiency through periodic sensing, decision-making, and adjustments.

[0058] In the above embodiments, the threshold is dynamically calculated. This can be further optimized as follows: Collect historical load fluctuation data over a recent period and calculate fluctuation statistics (e.g., standard deviation, coefficient of variation) using a sliding window; based on the fluctuation statistics and simulation accuracy metrics, adaptively adjust the threshold using a preset formula to balance partition frequency and overhead. For example, a simple adaptive formula could be... ,in, It is the basic threshold. This is the standard deviation of recent load fluctuations. It is an adjustment factor. When the load fluctuates greatly, the threshold is automatically increased to avoid excessively frequent repartitioning.

[0059] In large-scale simulations, repartitioning may fail due to network or data errors. Therefore, Figure 1 The example method may also include the following fault-tolerant processing steps S1071 to S1073: Step S1071: When partitioning fails, roll back to the previous stable partition state to ensure simulation continuity.

[0060] When the repartitioning process (step S104) is interrupted due to an anomaly (such as an MPI communication error, loss of critical files, or timeout), the monitoring process immediately detects the error signal. The system does not terminate directly but first rolls back to the previous stable partitioning state. This "previous stable partitioning state" refers to the successfully running, validated partitioning scheme and corresponding simulation data state of the system before the start of this failed repartitioning operation. The rollback operation is achieved by quickly switching the task allocation mapping table. The main process (or root process) of the parallel program broadcasts the rollback instruction to all worker processes. Each process, according to the instruction, discards any new partitioning data received during the failed repartitioning process and continues to use the old partitioning mapping for subsequent calculations. This ensures that the computational task can immediately continue under a known, stable configuration, avoiding system crashes or computational interruptions due to partitioning failures, thus ensuring simulation continuity.

[0061] Step S1072: Recover simulation data through a checkpoint mechanism to avoid data loss or inconsistency.

[0062] The integrity of simulation data in the "rollback state" is ensured by a checkpointing mechanism. The system periodically (e.g., after each successful repartitioning, or at fixed simulation steps) persistently stores the simulation state (including mesh geometry data, physics data, time step information, etc. on each processor) as checkpoints to a highly reliable parallel file system (e.g., Lustre, GPFS, etc.) or the node's local SSD. Each checkpoint contains a timestamp and a consistency checksum (e.g., CRC32 or MD5).

[0063] When a rollback is triggered, the system reads the latest checkpoint data corresponding to the timestamp of the "previous stable partition state" from storage. Before loading the data, a checksum verification is performed to ensure the data has not been corrupted or lost. Each processor reads its assigned portion of the checkpoint data in parallel. Then, based on the rolled-back partitioning scheme, the complete local simulation state is reconstructed in memory. This includes restoring the values ​​of various physical quantities of the mesh elements (e.g., density, pressure, velocity) to the checkpoint time and restoring the solver's internal state (e.g., residuals, iteration counts). This step ensures that the simulation can restart from a completely consistent and correct data state, effectively avoiding data inconsistencies that may arise from partitioning failures.

[0064] Step S1073: Re-trigger the load forecasting and partitioning process to regenerate a balanced partitioning scheme.

[0065] After resuming stable operation, the system can log the failure (including error type, time of occurrence, etc.) and attempt a simple analysis of the cause of the failure. For example, if the timeout is caused by transient network congestion, the system may briefly delay or retry before the next repartition; if the timeout is due to abnormal values ​​output by the load prediction model, a reasonableness check of the model output can be triggered.

[0066] The ultimate goal of fault tolerance is not to maintain an old partition that may no longer be balanced, but to regenerate a balanced partitioning scheme. Therefore, after the system has been running stably for a period of time (e.g., continuing to calculate for several more time steps, or waiting for network conditions to improve), the complete dynamic load balancing process will be retried. That is, starting again from step S101 (building the load graph), feature extraction (S102), GNN load prediction (S103), graph partitioning (S104), and monitoring (S105) will be performed based on the latest grid and physical field states. This re-triggering can employ a more conservative strategy, such as extending the communication timeout or increasing the re-partitioning trigger threshold (θ), to reduce the risk of failure again.

[0067] When the number of processors is extremely large (e.g., more than ten thousand cores), the following optimization can be adopted: A hierarchical partitioning strategy is employed, first coarsely partitioning the global grid to reduce global communication overhead, and then subdividing local subdomains to optimize load distribution. For example, a simple algorithm can be used to first divide the grid into coarse partitions equal to the number of supernodes, and then the proposed method can be used for fine partitioning within each supernode. Simultaneously, distributed graph neural network inference is implemented through an asynchronous training mechanism, where each computing node independently executes model inference and periodically synchronizes model parameters to maintain consistency. This avoids the inference process becoming a performance bottleneck.

[0068] From the above appendix Figure 1 As illustrated by the example of a dynamic load balancing method for parallel numerical simulation, on the one hand, by constructing a load graph and extracting multi-dimensional dynamic features such as the geometry, physical field, and computational state of the mesh cells, and using a graph neural network model for learning and prediction, this method can deeply integrate multi-source heterogeneous factors affecting computational costs, thereby generating load weights that more accurately reflect the actual computational overhead of the cells. This provides a more accurate input basis for subsequent partitioning, improving the quality of the partitioning scheme from the source. On the other hand, the method in this application is not a one-time static partitioning, but rather monitors the load imbalance in real time during the simulation process and dynamically triggers repartitioning operations. This allows the partitioning scheme to adaptively adjust to changes in load distribution, migrating computational tasks from heavily loaded processors to lightly loaded processors, thereby maintaining a high and stable load balance level throughout the simulation cycle, reducing processor idle waiting time, and improving the utilization of computing resources and the overall solution speed. Thirdly, the method in this application combines a graph neural network model with classic graph partitioning tools to form a complete, data-driven adaptive load balancing closed-loop process. This process reduces the reliance on human experience and fixed rules in traditional methods, enabling it to automatically learn load patterns in complex simulations and autonomously make partitioning decisions, thus enhancing the applicability and intelligence of the method in complex and ever-changing scenarios. In summary, the technical solution of this application effectively improves the accuracy and adaptability of load balancing in parallel numerical simulations by dynamically predicting load weights and adjusting partitions in real time through graph neural networks, thereby significantly improving computational efficiency.

[0069] Please see the appendix Figure 2 This application provides a dynamic load balancing device for parallel numerical simulation. The device may include a construction module 201, an extraction module 202, a prediction module 203, a generation module 204, a monitoring module 205, and an iteration module 206, as detailed below: Module 201 is used to construct a load graph based on a discrete grid of the computational domain, wherein the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections. The extraction module 202 is used to extract a multi-dimensional feature vector for each vertex, wherein the feature vector includes geometric features, physical field features and computational state features; The prediction module 203 is used to process multidimensional feature vectors using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computing. The generation module 204 is used to generate a dynamic partitioning scheme based on the load weight of each vertex in parallel computing and the mesh connectivity, and to map the partitions to multiple processors. The monitoring module 205 is used to monitor the load imbalance of each processor in real time during the numerical simulation process. When the imbalance exceeds the preset threshold, a repartitioning operation is triggered. Iteration module 206 is used to repeat the above load prediction, partitioning and monitoring steps until the simulation ends.

[0070] From the above appendix Figure 2 As illustrated by the example of a dynamic load balancing device for parallel numerical simulation, on the one hand, by constructing a load graph and extracting multi-dimensional dynamic features such as the geometry, physical field, and computational state of the grid cells, and using a graph neural network model for learning and prediction, this method can deeply integrate multi-source heterogeneous factors affecting computational costs, thereby generating load weights that more accurately reflect the actual computational overhead of the cells. This provides a more accurate input basis for subsequent partitioning, improving the quality of the partitioning scheme from the source. On the other hand, the method in this application is not a one-time static partitioning, but rather monitors the load imbalance in real time during the simulation process and dynamically triggers repartitioning operations. This allows the partitioning scheme to adaptively adjust to changes in load distribution, migrating computational tasks from heavily loaded processors to lightly loaded processors, thereby maintaining a high and stable load balance level throughout the simulation cycle, reducing processor idle waiting time, and improving the utilization of computing resources and the overall solution speed. Thirdly, the method in this application combines a graph neural network model with classic graph partitioning tools to form a complete, data-driven adaptive load balancing closed-loop process. This process reduces the reliance on human experience and fixed rules in traditional methods, enabling it to automatically learn load patterns in complex simulations and autonomously make partitioning decisions, thus enhancing the applicability and intelligence of the method in complex and ever-changing scenarios. In summary, the technical solution of this application effectively improves the accuracy and adaptability of load balancing in parallel numerical simulations by dynamically predicting load weights and adjusting partitions in real time through graph neural networks, thereby significantly improving computational efficiency.

[0071] Figure 3 This is a schematic diagram of the structure of a device provided in one embodiment of this application. For example... Figure 3As shown, the device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for a dynamic load balancing method for parallel numerical simulation. When the processor 30 executes the computer program 32, it implements the steps described in the above embodiment of the dynamic load balancing method for parallel numerical simulation, for example... Figure 1 The steps S101 to S106 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the construction module 201, extraction module 202, prediction module 203, generation module 204, monitoring module 205, and iteration module 206 are shown.

[0072] For example, the computer program 32 for a dynamic load balancing method in parallel numerical simulation mainly includes: constructing a load graph based on a discrete grid of the computational domain, wherein the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections; extracting multi-dimensional feature vectors for each vertex, wherein the feature vectors include geometric features, physical field features, and computational state features; processing the multi-dimensional feature vectors using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computation; generating a dynamic partitioning scheme using a graph partitioning tool based on the load weight of each vertex in parallel computation and the grid connection relationship, and mapping the partitions to multiple processors; monitoring the load imbalance of each processor in real time during the numerical simulation, and triggering a repartitioning operation when the imbalance exceeds a preset threshold; repeating the above load prediction, partitioning, and monitoring steps until the simulation ends. The computer program 32 can be divided into one or more modules / units, which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 32 in device 3. For example, computer program 32 can be divided into the functions of a construction module 201, an extraction module 202, a prediction module 203, a generation module 204, a monitoring module 205, and an iteration module 206 (a module in the virtual device). The specific functions of each module are as follows: Construction module 201 is used to construct a load graph based on a discrete grid of the computational domain, wherein the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to the grid topology connections; Extraction module 202 is used to extract multi-dimensional feature vectors for each vertex, wherein the feature vectors include geometric features, physical field features, and computational state features; Prediction module 203... 03 is used to process multi-dimensional feature vectors using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computing; generation module 204 is used to generate a dynamic partitioning scheme based on the load weight of each vertex in parallel computing and the mesh connectivity, and map the partitions to multiple processors; monitoring module 205 is used to monitor the load imbalance of each processor in real time during numerical simulation, and trigger a repartitioning operation when the imbalance exceeds a preset threshold; iteration module 206 is used to repeat the above load prediction, partitioning and monitoring steps until the simulation ends.

[0073] Device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of device 3 and does not constitute a limitation on device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the device may also include input / output devices, network access devices, buses, etc.

[0074] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0075] The memory 31 can be an internal storage unit of the device 3, such as a hard disk or RAM of the device 3. The memory 31 can also be an external storage device of the device 3, such as a plug-in hard disk, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the device 3. Furthermore, the memory 31 can include both internal and external storage units of the device 3. The memory 31 is used to store computer programs and other programs and data required by the device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0077] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0078] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0079] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0082] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the above-described embodiments of this application can also be implemented by a computer program instructing related hardware. The computer program for the dynamic load balancing method for parallel numerical simulation can be stored in a storage medium. When the computer program is executed by a processor, it can implement the steps of the above-described method embodiments, namely, constructing a load graph based on a discrete grid of the computational domain, wherein the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections; extracting multi-dimensional feature vectors for each vertex, wherein the feature vectors include geometric features, physical field features, and computational state features; processing the multi-dimensional feature vectors using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computation; generating a dynamic partitioning scheme using a graph partitioning tool based on the load weight of each vertex in parallel computation and the grid connection relationship, and mapping the partitions to multiple processors; monitoring the load imbalance of each processor in real time during the numerical simulation, and triggering a repartitioning operation when the imbalance exceeds a preset threshold; repeating the above load prediction, partitioning, and monitoring steps until the simulation ends. Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of storage media can be appropriately added or removed according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media do not include electrical carrier signals and telecommunication signals.

[0083] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A dynamic load balancing method for parallel numerical simulation, characterized in that, The method includes: A load graph is constructed based on a discrete grid of the computational domain, wherein the vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections. For each vertex, a multi-dimensional feature vector is extracted, which includes geometric features, physical field features, and computational state features; The multidimensional feature vectors are processed using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computing. Based on the load weight of each vertex in parallel computing and the mesh connectivity, a dynamic partitioning scheme is generated using a graph partitioning tool, and the partitions are mapped to multiple processors. During numerical simulation, the load imbalance of each processor is monitored in real time. When the imbalance exceeds a preset threshold, a repartitioning operation is triggered. Repeat the load prediction, partitioning, and monitoring steps described above until the simulation ends.

2. The dynamic load balancing method for parallel numerical simulation according to claim 1, characterized in that, The extraction of multi-dimensional feature vectors for each vertex includes: The geometric features of each vertex of the load graph are dynamically acquired, including mesh density, mesh volume, and mesh quality index. Real-time acquisition of physical field characteristics, including the physical quantity values ​​and their gradient magnitudes at the current time step; Record the computational state characteristics, which include the number of local iterations and the local time step size in the previous time step.

3. The dynamic load balancing method for parallel numerical simulation according to claim 2, characterized in that, The mesh density in the geometric features is dynamically determined by calculating the number of adjacent cells at each vertex. The mesh quality indicators include cell type and skewness, and are generated by a real-time mesh analysis algorithm.

4. The dynamic load balancing method for parallel numerical simulation according to claim 1, characterized in that, The graph neural network model employs a graph attention network and achieves feature updates through the following steps: Calculate attention weights between vertices to adaptively aggregate neighborhood information; Enhance the stability of feature representations using a multi-head attention mechanism; Avoid gradient vanishing during training by using residual connections.

5. The dynamic load balancing method for parallel numerical simulation according to claim 1, characterized in that, The triggering of the repartitioning operation includes: Calculate the load imbalance factor periodically. The time consumption constitutes a vector. ; Based on the accuracy requirements of the current simulation phase, the threshold is dynamically calculated. The accuracy requirement is determined based on the time step and grid resolution parameters; when Automatic repartitioning will be triggered at any time.

6. The dynamic load balancing method for parallel numerical simulation according to claim 1, characterized in that, The partitioning achieved through the graph partitioning tool includes: Input the load weights as vertex weights into the graph partitioning tool interface; Generate a partitioning scheme that minimizes communication overhead based on processor quantity constraints; The new partition is mapped to the processor cluster using a parallel communication library, and task assignments are updated synchronously.

7. The dynamic load balancing method for parallel numerical simulation according to claim 1, characterized in that, The method also includes the following fault tolerance steps: When partitioning fails, the simulation will revert to the previous stable partition state to ensure continuity. Simulation data is recovered through a checkpoint mechanism to avoid data loss or inconsistency; Re-trigger the load forecasting and partitioning process to regenerate a balanced partitioning scheme.

8. A dynamic load balancing device for parallel numerical simulation, characterized in that, The device includes: A construction module is used to construct a load graph based on a discrete grid of the computational domain. The vertices of the load graph correspond to grid cells or nodes, and the edges correspond to grid topology connections. The extraction module is used to extract a multi-dimensional feature vector for each vertex, the feature vector including geometric features, physical field features and computational state features; The prediction module is used to process the multidimensional feature vector using a pre-trained graph neural network model to predict the load weight of each vertex in parallel computing. The generation module is used to generate a dynamic partitioning scheme based on the load weight of each vertex in parallel computing and the mesh connectivity, and to map the partitions to multiple processors. The monitoring module is used to monitor the load imbalance of each processor in real time during numerical simulation. When the imbalance exceeds a preset threshold, a repartitioning operation is triggered. The iteration module is used to repeat the above load prediction, partitioning, and monitoring steps until the simulation ends.

9. An apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images