Graph reinforcement learning based topology network global optimization method

Abstract

This invention discloses a global optimization method for topology networks based on graph reinforcement learning, relating to the fields of computational geometry and mesh optimization. This method models the topology optimization process as a Markov decision process: the topology network containing singularities and corners is transformed into a heterogeneous graph representation, constructing a graph state; a reinforcement learning agent fused with a graph neural network selects a topology editing action based on the current graph state; after executing the action, the topology structure and graph state are updated, and an immediate reward is calculated based on changes in global mesh quality indicators; the agent is batch-trained through experience replay and temporal difference learning, enabling it to learn a mapping strategy from graph states to optimal actions; the trained policy network is deployed and applied to the automatic optimization of new topologies. This invention achieves full automation of the topology optimization process, effectively avoids the nonlocal negative impacts of local modifications, learns globally optimal optimization strategies, significantly improves mesh quality and optimization efficiency, and is suitable for simulation mesh generation for aircraft.

Description

Technical Field

[0001] This invention relates to the fields of computational geometry and mesh optimization, and more specifically, to a global optimization method for topological networks based on graph reinforcement learning. Background Technology

[0002] In engineering simulation fields such as finite element analysis and computational fluid dynamics, global quality optimization of the topology network is a crucial step in ensuring simulation accuracy and computational convergence. The initial topology network (such as the fragmented quadrilateral mesh generated by decomposing the frame field region) typically contains a large number of singularities, corners, and topological lines, and its quality directly determines the reliability of subsequent simulation results.

[0003] Currently, the mainstream topology optimization method mainly relies on manual adjustments by experts. A typical process includes: diagnosing global quality defects based on mesh quality indicators (orthogonality, smoothness, etc.) and locating key singularities; performing local topology editing operations (such as moving, merging, and splitting) on ​​singularities; verifying the changes in global mesh quality after modification and determining whether non-local negative effects have occurred; and iterating repeatedly until the engineering quality requirements are met.

[0004] However, existing technologies have the following drawbacks:

[0005] The diagnosis is highly complex: it requires experts to have a deep understanding of the quality indicators, the location of global key singularities depends on experience, and it is difficult to identify systematically.

[0006] Nonlocal effects are uncontrollable: The topological network is highly coupled, and local benign modifications can easily be transmitted through topological connections to cause malign effects in distant places, lacking predictability;

[0007] The operation is cumbersome and inefficient: manually executing the topology transformation process is time-consuming, and repeated trial and error can easily lead to human error.

[0008] Lack of a global optimization perspective: Relying on local heuristic adjustments, it is unable to systematically evaluate and optimize the overall structure of the topology network;

[0009] Poor adaptability: Expert experience is difficult to transfer to different types of topologies, and its adaptability to complex geometric scenes is limited.

[0010] Therefore, there is an urgent need for a topology network optimization method that can be automated, globally optimized, and has good generalization ability. Summary of the Invention

[0011] This invention aims to address the following major problems existing in the prior art: topology optimization relies on expert experience, and the diagnosis of key singularities lacks a systematic approach; local modifications cause non-local negative effects that are difficult to predict and control; manual operations are cumbersome, inefficient, and prone to errors, with a low degree of automation; and there is a lack of systematic evaluation and global optimization strategies for the overall structure of the topology network.

[0012] To achieve the above-mentioned objectives, this invention provides a global optimization method for topological networks based on graph reinforcement learning, the method comprising:

[0013] Environment initialization: Input the initial topology network to be optimized, which includes singular points, corner points and topology lines connecting nodes; initialize the graph reinforcement learning environment, including defining the master policy network and target network of the fused graph neural network, defining the set of topology editing actions and initializing the experience replay pool and hyperparameters;

[0014] Graph state construction: The initial topology network is transformed into a heterogeneous graph representation, and the feature information of nodes and edges in the graph is extracted to form the current graph state;

[0015] Agent decision-making: The reinforcement learning agent, which integrates graph neural networks, selects the topology editing action to be executed based on the current graph state through an ε-greedy policy;

[0016] Action execution and state transition: Execute the selected topology editing action to perform geometric editing on the current topology network, update the structure of the topology network and the corresponding heterogeneous graph state, and obtain the next graph state;

[0017] Reward Calculation: Calculate the immediate reward based on the change in global mesh quality metrics before and after performing the selected topology editing action;

[0018] Experience storage: The current graph state, the selected topology editing action, the immediate reward, the next graph state, and the termination flag are combined into an experience tuple and stored in the experience replay pool;

[0019] Batch training: Randomly sample batches of experience from the experience replay pool, use temporal difference learning to update the parameters of the main policy network, so that the main policy network learns the mapping policy from graph state to optimal topology editing action;

[0020] Target network update: Every preset number of steps, the parameters of the main policy network are copied to the target network to stabilize the training process;

[0021] Model Deployment: The converged master policy network is used as the optimal policy model and applied to the global optimization task of the initial topology network to be processed, outputting the optimized topology network.

[0022] Existing topology optimization methods rely on manual expert intervention, lacking systematic diagnosis and global optimization capabilities, and are inefficient and prone to errors. To address this issue, this invention decomposes the topology optimization process into steps such as environment initialization, graph state construction, agent decision-making, action execution and state transition, reward calculation, experience storage, batch training, target network update, and model deployment, constructing a complete automated optimization process based on graph reinforcement learning. Specifically, graph state construction transforms the topology network into a heterogeneous graph, enabling the agent to perceive the global structure; the reinforcement learning agent selects actions based on the current state; reward calculation provides feedback based on quality changes; batch training learns the optimal mapping strategy through experience replay and the target network; and finally, deployment achieves automatic optimization of the new topology. This fully automates the topology optimization process, avoiding manual intervention, improving optimization efficiency and consistency, and enhancing optimization quality through a global learning strategy.

[0023] Preferably, the graph state construction step specifically includes:

[0024] Singular points and corner points in the initial topology are defined as two types of nodes;

[0025] Define the topological lines connecting nodes as edges;

[0026] Encode feature information for each node: the features of singular points include three-dimensional coordinates and valence state, and the features of corner points include three-dimensional coordinates and the type of boundary they belong to;

[0027] Encode feature information for each edge, including length, curvature, angle with adjacent edges, and quality index of the grid cell associated with that edge;

[0028] Construct a heterogeneous graph data structure containing a node set, an edge set, and its feature matrix as the current graph state.

[0029] To effectively encode the topological network into graph states, enabling the agent to fully understand the geometric and connectivity features of the topology and provide a foundation for decision-making, this invention defines a specific method for constructing the graph states: singular points and corner points are defined as two types of nodes, and topological lines are defined as edges; features such as coordinates, valence states, and boundary types are encoded for nodes; features such as length, curvature, included angle, and quality of associated meshes are encoded for edges; ultimately, a heterogeneous graph containing a set of nodes, a set of edges, and a feature matrix is ​​formed. Through refined feature encoding, the graph neural network can extract rich local and global topological information, providing high-quality input for the agent's accurate decision-making, thereby improving the effectiveness of the optimization strategy.

[0030] Preferably, the set of topology editing actions includes the following four types of atomic-level operations:

[0031] Fine-tuning singular point positions: Adjusting the coordinates of singular points based on neighborhood geometric features;

[0032] Merging adjacent topological blocks: Eliminating redundant singularities and merging adjacent topological blocks into one;

[0033] Topological edge splitting: Introducing new singular pairs on a topological edge to split an edge into multiple edges;

[0034] Irregular area template reconstruction: Redraw the locally distorted areas using a predefined mesh template;

[0035] The above operations together constitute a discrete action space for reinforcement learning agents to choose from.

[0036] This invention defines four types of topology editing actions: fine-tuning singular point positions, merging adjacent topological blocks, splitting topological edges to introduce new singular point pairs, and reconstructing templates for irregular regions. These operations cover key optimization requirements such as local adjustments, structural simplification, enhanced control, and distortion repair, constituting a discrete action space. Through this refined action space, the agent can achieve complex topology optimization in a combinatorial manner while ensuring the feasibility of the learning process; the atomicity of the actions facilitates validity verification and reward allocation.

[0037] Preferably, the instant reward is calculated as follows:

[0038] ;

[0039] in, For instant rewards, Rewards for quality improvement Rewards for efficiency Punishment for mistakes.

[0040] The immediate reward consists of three parts: quality improvement reward, efficiency reward, and error penalty, comprehensively reflecting the optimization effect, speed, and legitimacy. This multi-objective reward design guides the agent to pursue efficiency while improving quality and suppressing ineffective operations, thereby learning optimization strategies that meet engineering requirements.

[0041] Preferably, the calculation method for the quality improvement reward is as follows:

[0042] ;

[0043] in, As a reward amplification factor, The difference in the total global mesh quality score before and after performing the selected topology editing action;

[0044] ;

[0045] in, For orthogonality, For smoothness, Aspect ratio, denoted as the total number of quadrilaterals, and i as the grid cell number.

[0046] Among them, multiple quality indicators are integrated by weighted summation, so that the reward signal is closely coupled with the actual requirements of engineering simulation for grid quality, guiding the agent to prioritize the improvement of key quality dimensions.

[0047] Preferably, in the experience storage step, each experience tuple stored in the experience replay pool is represented as:

[0048] ;

[0049] in, This is the current state of the diagram. The identifier for the selected topology editing action in the current step. For instant rewards, The next graph state obtained after performing the selected topology editing action. This is the end label for iterative execution.

[0050] The format of the experience tuple includes the current graph state, the selected action, the immediate reward, the next graph state, and a termination flag, covering all the information required for reinforcement learning training. The unified experience format facilitates batch sampling and training, while the introduction of the termination flag enables the agent to distinguish termination states, correctly handle future reward calculations, and improve training stability.

[0051] Preferably, the batch training step includes the following sub-steps:

[0052] A batch of experience tuples are randomly sampled from the experience replay pool;

[0053] In each empirical tuple The input is fed into the target network to obtain the Q-value predictions of all candidate topology editing actions output by the target network; for each The maximum value is selected from the Q-value predictions of all candidate topology editing actions. This maximum value represents the present value estimate of the future cumulative reward that can be obtained by performing the optimal topology editing action in the next graph state. The state of the next graph obtained after performing the selected topology editing action;

[0054] Multiply the maximum value by the discount factor This yields the present value of future rewards; the iteration ends when the label is displayed. When true, it indicates that the current graph state is a terminated state. In this case, the present value of future rewards is set to zero, and only immediate rewards are used. Perform calculations;

[0055] Will Input the main policy network and obtain the Q-value prediction vectors of all candidate topology editing actions output by the main policy network. This represents the current state of the graph.

[0056] Extracting the Q-value prediction vectors of all candidate topology editing actions output by the main policy network and... The corresponding Q value is denoted as the current Q value. The identifier for the topology editing action selected in the current step;

[0057] Calculate the temporal difference loss between the current Q-value and the target Q-value, and add a regularization term;

[0058] The parameters of the master policy network are updated by using the backpropagation algorithm, enabling the master policy network to learn the mapping strategy from graph states to optimal topology editing actions.

[0059] The process involves random sampling from the experience pool; inputting the next graph state into the target network to obtain the Q-values ​​of all actions and taking the maximum value; calculating the target Q-value by combining the immediate reward and discount factor; inputting the current graph state into the main network to obtain the current Q-value; calculating the loss of both and backpropagating to update the main network parameters. Through temporal difference learning and the target network mechanism, stable and efficient policy updates are achieved; random sampling breaks down empirical correlations and improves sample efficiency; and the delayed update of the target network reduces training oscillations.

[0060] Preferably, the target Q value is calculated as follows:

[0061] ;

[0062] in, For the target Q value, For the target network in The maximum value among all candidate topology editing action Q values ​​output below. Indicates in The topology editing action selected by the reinforcement learning agent.

[0063] Preferably, the model deployment step specifically includes:

[0064] Load the main policy network after training convergence;

[0065] Input the initial topology network to be processed, and execute the graph state construction, reinforcement learning agent decision-making, action execution and state transition and reward calculation steps in sequence, iterating until the preset termination condition is met;

[0066] Output the globally optimized topology network for subsequent engineering simulation analysis.

[0067] The process involves loading the converged master policy network, inputting a new initial topology network, and automatically iterating through steps such as graph state construction, decision-making, action execution, and reward calculation until a termination condition is met, outputting an optimized topology network. This enables ready-to-use engineering deployment, significantly shortening the optimization cycle of new topologies and improving the efficiency of CAE simulation preprocessing.

[0068] Preferably, in the action execution and state transition steps, a geometric validity verification sub-step is included before executing the selected topology editing action:

[0069] Verify that the topology blocks to be merged are adjacent and do not overlap;

[0070] Verify whether the topological edges to be split are connected, and whether they are topologically connected after splitting;

[0071] Verify whether adjusting node coordinates leads to mesh cell distortion or penetration of geometric boundaries;

[0072] If the verification fails, the topology editing action is marked as invalid, triggering an error penalty and the topology editing action is not executed before proceeding to the next step.

[0073] The algorithm incorporates a geometric validity verification sub-step before action execution. This sub-step verifies whether merged blocks are adjacent and non-overlapping, whether splitting edges guarantee topological connectivity, and whether node movement causes distortion. If the verification fails, the operation is marked as invalid and a penalty is triggered, preventing the action from being executed. This pre-validation check prevents the execution of invalid operations, ensuring the topological network remains valid throughout the optimization process and improving the robustness and reliability of the method. Simultaneously, the error penalty mechanism guides the agent to avoid selecting illegal actions.

[0074] One or more technical solutions provided by this invention have at least the following technical effects or advantages:

[0075] Automated global optimization: Without expert intervention, the intelligent agent automatically diagnoses key singularities and performs optimization operations, solving the problem of tedious and inefficient manual operations;

[0076] Avoiding non-local negative impacts: Based on global graph feature learning optimization strategies, the chain reaction of local modifications can be predicted, ensuring the quality of the global topology;

[0077] Global optimal strategy learning: Get rid of local heuristic adjustments and learn the global optimal strategy through reinforcement learning to systematically improve mesh quality (especially improve singularity layout and block connectivity).

[0078] Strong adaptability and generalization: GNN efficiently extracts topological features, and the agent can handle initial topologies from different sources and of different types, adapting to diverse engineering scenarios;

[0079] Optimization and efficiency improvement: Experience replay and target network mechanism accelerate training convergence, and optimization can be completed quickly after deployment, reducing engineering costs;

[0080] Quality and stability assurance: Based on the optimal strategy learned, consistent high-quality optimization results are output for similar topological scenarios, thereby improving simulation reliability. Attached Figure Description

[0081] The accompanying drawings, which are provided to further illustrate embodiments of the invention and constitute a part of this invention, are not intended to limit the scope of the invention.

[0082] Figure 1 This is a flowchart of a global optimization method for topological networks based on graph reinforcement learning. Detailed Implementation

[0083] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other.

[0084] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0085] Example 1;

[0086] Please refer to Figure 1 , Figure 1 This is a flowchart of a graph reinforcement learning-based global optimization method for topology networks. The complete process of implementing global optimization of topology networks using this method includes the following steps:

[0087] 1) Environment initialization: Input the initial topology structure to be optimized and initialize the graph reinforcement learning environment;

[0088] 2) Graph state construction: Transform the topological network into a heterogeneous graph representation and extract the feature information of nodes and edges;

[0089] 3) Agent decision-making: The reinforcement learning agent fused with GNN selects the optimal topology editing action based on the current graph state;

[0090] 4) Action execution and state transition: Execute the selected topology editing operation to update the topology network structure and heterogeneous graph state;

[0091] 5) Reward Calculation: Calculate the immediate reward based on the change in the global grid quality index after the operation;

[0092] 6) Experience storage: Stores graph state transition experience to provide data support for agent policy optimization;

[0093] 7) Batch training: The policy network is trained in batches using a graph reinforcement learning algorithm;

[0094] 8) Target network update: Periodically synchronize the parameters of the main policy network to the target network to ensure training stability;

[0095] 9) Model Deployment: Apply the trained optimal policy to the new initial topology optimization task.

[0096] This invention models the global topology optimization process as a Markov Decision Process (MDP) and integrates Graph Neural Networks (GNNs) and reinforcement learning agents to learn the globally optimal optimization strategy. It includes: a state space where the topology network is transformed into a heterogeneous graph representation, with nodes including singularities and corners, and edges representing topological lines; node features encode coordinates, valence states, etc., while edge features encode length, curvature, angle, and associated grid quality indices. An action space defining a discrete set of atomic-level topology editing operations, including fine-tuning singularity positions, merging adjacent topology blocks, splitting topology edges to introduce new singularity pairs, and reconstructing templates for irregular regions. A reward function calculating immediate rewards based on changes in global grid quality indices (orthogonality, smoothness, aspect ratio) before and after an action, guiding the agent to learn the globally optimal strategy. State transitions updating the topology network structure and corresponding heterogeneous graph states after executing a topology editing operation. Global and local features of the topology are extracted through graph neural networks, and combined with reinforcement learning (such as the DQN algorithm), the agent learns a mapping function from graph states to optimization actions, maximizing long-term cumulative rewards. The training process includes stabilization techniques such as experience replay and target network updates, and the trained policy network is finally deployed and applied to the automatic optimization of the new topology.

[0097] Detailed steps:

[0098] Step 1: Environment Initialization:

[0099] Execution subject: training system; Execution content: input initial topology structure (including singular points, corner points, and topology line information); define policy network and target network for fusion GNN; define set of topology editing actions; initialize core components.

[0100] Specific implementation:

[0101] Initial topology input: A fragmented quadrilateral structure mesh generated by the frame field decomposition method, containing basic information such as node (singularity, corner) coordinates, valence state, edge (topology line) length, and shape.

[0102] Network definition (the main / target network structures are exactly the same): Input: heterogeneous graph data of the topology network; Output: execution probability of all topology editing actions;

[0103] Model structure: includes graph convolutional layers (extracting local correlation features between nodes and edges), graph attention layers (adaptively focusing on key nodes and edges), fully connected layers (converting graph features into action decisions), and dropout layers (preventing overfitting).

[0104] Action set definition: 4 types of atomic operations, covering the core requirements of topology optimization:

[0105] Operation 1: Fine-tune the location of singular points (adjust coordinates based on neighborhood geometry features and optimize local grid distribution);

[0106] Operation 2: Merge adjacent topology blocks (eliminate redundant singularities and simplify the topology).

[0107] Operation 3: Split the topological edges (introduce new singular point pairs to enhance local optimization control);

[0108] Operation 4: Irregular area template reconstruction (applying a predefined template to repair distorted areas).

[0109] Initialize core components: Initialize the main policy network and target network (parameters initially consistent); initialize the optimizer and experience replay pool; define hyperparameters (discount factor, ... -greedy, batch training size, target network update steps, etc.).

[0110] Step 2: Graph State Construction:

[0111] Execution Module: Graph State Extraction Module; Execution Content: Transforms the topological network into a standardized heterogeneous graph representation, encoding global and local features; Specific Implementation:

[0112] Heterogeneous graph construction: Nodes are divided into two categories: singular nodes and corner nodes, and edges are topological lines connecting nodes;

[0113] Node feature encoding: Singular point features include 3D coordinates and valence state (number of connected edges); Corner point features include 3D coordinates and the type of boundary they belong to.

[0114] Edge feature encoding: includes length, curvature, angle with adjacent edges, and quality indicators of associated grid cells;

[0115] State format: {"graph": heterogeneous graph data (node ​​set, edge set, feature matrix)}.

[0116] Step 3: Agent Decision Making:

[0117] Execution subject: Reinforcement learning agent fused with GNN; Execution content: Based on The -greedy strategy selects the optimal topology editing action;

[0118] Decision-making mechanism:

[0119] Exploration phase (random probability < -greedy): Randomly selects a topology editing action to expand the search space;

[0120] Utilization phase (random probability ≥ -greedy): Select the optimal action with the highest Q value (optimal for improving global quality);

[0121] Decay: Gradually reduce the exploration probability as training progresses to improve decision stability.

[0122] Step 4: Action Execution and State Transition

[0123] Execution Module: Topology Editing Execution Module; Execution Content: Edits the current topology network according to the selected action and updates the heterogeneous graph state;

[0124] Before executing an action, verify its geometric validity (e.g., merging topological blocks must ensure no overlap, and splitting edges must ensure topological connectivity) to avoid invalid operations.

[0125] Step 5: Reward Calculation

[0126] Execution Module: Reward Calculation Module;

[0127] Execution content: Calculate the comprehensive reward based on the global grid quality change;

[0128] Reward Design:

[0129] The formula for calculating instant rewards is as follows:

[0130] ;

[0131] in, For instant rewards, Rewards for quality improvement Rewards for efficiency Punishment for mistakes.

[0132] Quality Improvement Rewards:

[0133] ;

[0134] in, The difference in the total global mesh quality score before and after performing the selected topology editing action. =10.0 is the reward amplification factor.

[0135] ;

[0136] In the above formula For orthogonality, For smoothness, Aspect ratio, is the total number of quadrilaterals, 0.4, 0.3 and 0.3 are the weight percentages of orthogonality, smoothness and aspect ratio, respectively, and i is the grid cell number.

[0137] Efficiency Rewards:

[0138] ;

[0139] in, To determine the optimal number of operation steps threshold, This represents the current iteration step. =2.0 is used as the weight to encourage efficient optimization.

[0140] Error penalty:

[0141] =5.0 (Triggered when an invalid operation is performed, such as merging non-adjacent topological blocks or splitting illegal edges).

[0142] Step 6: Experience Storage

[0143] Execution subject: Graph reinforcement learning algorithm; Execution content:

[0144] empirical tuples Store in the playback pool. This is the current state of the diagram. The identifier for the selected topology editing action in the current step. For instant rewards, The next graph state obtained after performing the selected topology editing action. This is the end label for iterative execution.

[0145] Step 7: Batch Training

[0146] Execution subject: Graph reinforcement learning algorithm; Execution content: Batch training is performed by randomly sampling through a replay pool and estimating future rewards for the target network;

[0147] Specific implementation:

[0148] Random sampling of the experience replay buffer:

[0149] The experience replay buffer serves as the agent's memory, storing a large amount of state transition experience. During each training session, a small batch of experience data is randomly selected from this buffer for learning.

[0150] Optimal future reward estimation for the target network:

[0151] The target network is a lagged replica of the main network, used to calculate a stable estimate of future rewards. First, the state of the next graph... The input is fed into the target network to obtain the Q-value predictions for all possible topology-filling actions. Then, for each next graph state, the action with the maximum value (which can be obtained through comparison using the max function) is selected, representing the optimal future reward for that graph state. Finally, the maximum Q-value is multiplied by a discount factor. , represents the present value of future rewards. For states that cause the task to terminate, the future rewards are zero, and only the immediate rewards are calculated.

[0152] Target Q value calculation:

[0153] The calculation formula is as follows, where, For the target Q value, For immediate rewards, that is, the reinforcement learning agent's step at that time. After the action is completed, the environment immediately provides numerical feedback. As a discount factor, For the target network in The execution probability of the optimal topology filling template action under the given state.

[0154] ;

[0155] Calculation of the current Q-value of the main network:

[0156] First, set the current Input the main network, obtain Q-value estimates for all actions through the network's forward propagation, and then use the gather operation to select the Q-value corresponding to the actual action to be performed from all the Q-values. .

[0157] Calculation of time-series differential loss:

[0158] Huber loss function Using Smooth L1 Loss provides better robustness to outliers and is more stable for training compared to mean squared error; temporal difference error. : Calculates the difference between the current Q-value and the target Q-value, reflecting the accuracy of the agent's prediction; regularization term :Add to Regularization prevents overfitting and penalizes excessively large network weights. The specific formula is as follows.

[0159] ;

[0160] Backpropagation and parameter optimization:

[0161] The gradient of the network parameters with respect to the loss is calculated through backpropagation of the loss function.

[0162] Step 8: Target network update:

[0163] Execution entities: main policy network and target network; Execution content: when the preset number of update steps is reached, the parameters of the main policy network are copied to the target network to ensure training stability.

[0164] Step 9: Model Deployment

[0165] Execution subject: The optimal policy network after training; Execution content: Load the policy network that has been trained and converged, input the new initial topology, automatically execute steps 2-6, and output the globally optimized topology.

[0166] Logical relationship: Enables immediate use after training, improving optimization efficiency and generalization ability in engineering scenarios.

[0167] In this embodiment of the invention, the corresponding technical terms are defined as follows:

[0168] Topological network: refers to a graph structure composed of nodes and edges, used to represent the macroscopic connectivity of finite element meshes. In this invention, the topological network specifically refers to the macroscopic structural representation of a quadrilateral mesh generated by the frame field decomposition method.

[0169] Singularities are nodes in a quadrilateral mesh whose valence state is not equal to 4. Valence state refers to the number of edges connecting the node. Singularities are a key factor affecting mesh quality, and their distribution directly affects the smoothness and orthogonality of the mesh.

[0170] Corner points: These are nodes located on the boundaries of a geometric model, such as the leading edge and trailing edge of an aircraft wing. The location and distribution of corner points determine the accuracy of the mesh's fit to the geometric shape.

[0171] Topology lines: These are lines that connect nodes and represent the connections between mesh cells. The length, curvature, and angle of topology lines reflect the local geometric characteristics of the mesh.

[0172] In this embodiment of the invention, the specific implementation of graph state construction is as follows:

[0173] Node feature extraction: Singular point features: 3D coordinates (x,y,z) are directly read from the grid node; valence is obtained by counting the number of edges connected to the node, with a typical value range of 2-8.

[0174] Corner features: 3D coordinates (x, y, z) are read directly; the boundary type is determined by the position of the corner in the geometric model, including leading edge, trailing edge, wing root, wingtip, etc., and is encoded as a one-hot vector.

[0175] Edge feature extraction:

[0176] Length: Euclidean distance between the two endpoints of the topology line, with units consistent with the model scale;

[0177] Curvature: The geometric curvature along a topological line, calculated by the angle between adjacent line segments;

[0178] Angle with adjacent edge: The angle between the topology line and its adjacent edge at the node, reflecting the smoothness of the mesh;

[0179] Quality metrics for associated mesh cells: the weighted average of the orthogonality, smoothness, and aspect ratio of all mesh cells involved in the topology line.

[0180] Heterogeneous graph construction: Using graph neural network libraries such as PyTorch Geometric or DGL, the node feature matrix, edge feature matrix, and adjacency matrix are encapsulated into heterogeneous graph data objects, which serve as state inputs for reinforcement learning.

[0181] In this embodiment of the invention, the specific rules and thresholds for geometric legality verification are as follows:

[0182] Merge operation verification:

[0183] To verify whether two topological blocks are adjacent, check if the number of shared edges is ≥1.

[0184] Verify whether the merged regions are self-intersecting: Calculate the boundaries of the new regions and check for edge intersections;

[0185] Verify the quality of the merged mesh cells: Estimate the orthogonality of all cells in the new region. If the orthogonality of any cell is <0.5, it is considered invalid.

[0186] Verification of the splitting operation:

[0187] Verify whether the edge to be split is connected: check whether both endpoints of the edge belong to the current graph;

[0188] Verify the topological connectivity after partitioning: The two newly generated sub-edges after partitioning must ensure that the connectivity of the original region remains unchanged, that is, the region must not be divided into two disconnected parts;

[0189] Verify the location of the new singular point: The new location must be between the two endpoints of the original edge, and the distance from the endpoints of the original edge must not be less than 0.1 times the length of the original edge;

[0190] Node movement verification:

[0191] Verify whether the geometric boundary has been penetrated: the shortest distance between the new location and the boundary of the geometric model is greater than 0.1 times the average side length of the mesh;

[0192] Verify whether distortion occurs: After the movement, the aspect ratio of all involved grid cells must not exceed 10.

[0193] This invention constructs a complete automated process from environment initialization to model deployment. Users only need to input the initial topology network, and the system automatically executes a series of steps, including graph state construction, agent decision-making, action execution, state transition, and reward calculation, without manual intervention. The specific implementation of the deployment phase is further clarified: after loading the trained policy network, a new initial topology network is input, and the system automatically iterates through optimization steps until the optimization result is output. An end-to-end automated process design is achieved. This invention defines an atomic-level set of topology editing actions, enabling the agent to perform operations such as fine-tuning singularity positions and merging topology blocks, but these operations are automatically selected and executed by the agent. An agent autonomous decision-making mechanism is implemented. The agent decision-making steps adopt an ε-greedy strategy, enabling the agent to automatically balance exploration and exploitation, gradually learning to select the optimal action. Through a reinforcement learning framework, human experience is transformed into a learnable policy network, enabling the computer to autonomously complete the optimization process that originally required repeated trial and error by experts, thus achieving full-process automation. Automated global optimization is achieved, solving the problem of tedious and inefficient manual operation.

[0194] This invention details the construction method of graph states: the entire topological network is transformed into a heterogeneous graph, where nodes contain singularities and corners, and edges contain topological lines and their features. This representation method preserves the global structural information of the topological network, enabling the agent to see the entire network, not just local regions. Node features encode information such as coordinates and valence states, while edge features encode length, curvature, angle, and associated grid quality. These features allow the agent to understand the potential impact of local modifications that may be transmitted through topological connections, achieving a global perspective representation of graph states. The fusion graph neural network design of this invention enables the agent to automatically extract global and local features of the topology through graph convolutional layers and graph attention layers, modeling long-range dependencies between nodes. This capability allows the agent to predict, during decision-making, the potential impact of performing an action in the current region on distant regions through topological connections, thereby avoiding local optimization traps and realizing the feature extraction capability of graph neural networks. The reward function of this invention is calculated based on the change in the total global grid quality score, covering the orthogonality, smoothness, and aspect ratio of all grid cells. This means that when an agent optimizes a local region, if it causes a decrease in the quality of distant regions, the global score will decrease accordingly. The reward signal will punish this behavior, thus achieving reward guidance for the global quality index. Through long-term learning objectives that accumulate rewards to maximize benefits, the agent gradually learns to select modifications that are beneficial to the global picture and avoid actions that improve the local area but harm the global picture. This achieves the avoidance of non-local negative impacts and solves the problem of chain reactions caused by local modifications.

[0195] This invention achieves the learning of the globally optimal policy by using mathematical modeling of Markov decision processes, policy learning mechanisms of reinforcement learning, the goal of maximizing long-term cumulative rewards, and the weight design of global quality indicators, thus getting rid of local heuristic adjustments.

[0196] Example 2;

[0197] Based on Embodiment 1, the present invention will be described in detail below with reference to an aircraft example:

[0198] In engineering simulation fields such as aircraft design and flow field analysis, the quality of the finite element mesh directly affects the calculation accuracy. Taking aircraft wing design as an example, if the orthogonality deviation of the boundary layer mesh on the wing surface exceeds 15%, it will lead to a lift coefficient calculation error of more than 5%; insufficient mesh smoothness at the wing-body junction will cause spurious stress concentration, with errors reaching 20%. Currently, mesh optimization for such critical components is still mainly done manually by experts. Optimizing a typical wing mesh requires 3-5 working days and is highly dependent on personal experience, with results from different experts varying by more than 30%. To address this problem, Embodiment 2 of this invention provides a global optimization method for aircraft structural topology networks based on graph reinforcement learning, including the following steps:

[0199] Obtain an initial finite element mesh model of an aircraft component, which includes a wing, fuselage, or tail structure. The initial mesh model contains a fragmented quadrilateral structure mesh generated by the frame field decomposition method.

[0200] The mesh topology network of the aircraft components is transformed into a heterogeneous graph representation, and a graph state is constructed, in which singular points and corner points in the mesh nodes are used as nodes of the graph, and mesh topology lines are used as edges of the graph.

[0201] A reinforcement learning agent incorporating graph neural networks is constructed to model the aircraft mesh topology optimization process as a Markov decision process.

[0202] The agent selects a mesh topology editing action based on the current graph state, and updates the mesh topology and graph state after executing the action.

[0203] Real-time rewards are calculated based on changes in the quality indicators of the aircraft mesh before and after the operation. These quality indicators include mesh orthogonality, smoothness, and aspect ratio.

[0204] The agent is trained in batches using graph reinforcement learning algorithms to learn the mapping strategy from graph states to optimal actions.

[0205] The trained policy network is deployed and applied to the mesh topology optimization task of new aircraft components, and the optimized mesh model is output for subsequent simulation analysis of aircraft aerodynamic performance or structural strength.

[0206] In this embodiment of the invention, the aircraft component is a wing structure, and its initial mesh topology includes leading-edge singularities, trailing-edge corner points, and wing surface topology lines that reflect the geometric characteristics of the wing. In the graph state construction step, the node features also include geometric feature parameters such as wing surface curvature and airfoil thickness distribution.

[0207] In this embodiment of the invention, the topology editing action includes specialized operations for the characteristics of the aircraft mesh:

[0208] Fine-tune the singular point positions along the airfoil spanwise to optimize the boundary layer mesh distribution;

[0209] Redundant singularities in the wing leading edge region are merged to simplify the topology;

[0210] By splitting the topological edges at the wing-body junction and introducing new singular point pairs, the local optimization control is enhanced.

[0211] Template reconstruction is performed on irregular areas such as flaps and ailerons to repair distorted mesh cells.

[0212] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0213] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A global optimization method for topological networks based on graph reinforcement learning, characterized in that, The method includes: Environment initialization: Input the initial topology network to be optimized, which includes singular points, corner points and topology lines connecting nodes; initialize the graph reinforcement learning environment, including defining the master policy network and target network of the fused graph neural network, defining the set of topology editing actions and initializing the experience replay pool and hyperparameters; Graph state construction: The initial topology network is transformed into a heterogeneous graph representation, and the feature information of nodes and edges in the graph is extracted to form the current graph state; Agent decision-making: The reinforcement learning agent, which integrates graph neural networks, selects the topology editing action to be executed based on the current graph state through an ε-greedy policy; Action execution and state transition: Execute the selected topology editing action to perform geometric editing on the current topology network, update the structure of the topology network and the corresponding heterogeneous graph state, and obtain the next graph state; Reward Calculation: Calculate the immediate reward based on the change in global mesh quality metrics before and after performing the selected topology editing action; Experience storage: The current graph state, the selected topology editing action, the immediate reward, the next graph state, and the termination flag are combined into an experience tuple and stored in the experience replay pool; Batch training: Randomly sample batches of experience from the experience replay pool, use temporal difference learning to update the parameters of the main policy network, so that the main policy network learns the mapping policy from graph state to optimal topology editing action; Target network update: Every preset number of steps, the parameters of the main policy network are copied to the target network to stabilize the training process; Model Deployment: The converged master policy network is used as the optimal policy model and applied to the global optimization task of the initial topology network to be processed, outputting the optimized topology network.

2. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, The graph state construction steps specifically include: Singular points and corner points in the initial topology are defined as two types of nodes; Define the topological lines connecting nodes as edges; Encode feature information for each node: the features of singular points include three-dimensional coordinates and valence state, and the features of corner points include three-dimensional coordinates and the type of boundary they belong to; Encode feature information for each edge, including length, curvature, angle with adjacent edges, and quality index of the grid cell associated with that edge; Construct a heterogeneous graph data structure containing a node set, an edge set, and its feature matrix as the current graph state.

3. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, The set of topology editing actions includes the following four types of atomic operations: Fine-tuning singular point positions: Adjusting the coordinates of singular points based on neighborhood geometric features; Merging adjacent topological blocks: Eliminating redundant singularities and merging adjacent topological blocks into one; Topological edge splitting: Introducing new singular pairs on a topological edge to split an edge into multiple edges; Irregular area template reconstruction: Redraw the locally distorted areas using a predefined mesh template; The above operations together constitute a discrete action space for reinforcement learning agents to choose from.

4. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, The instant reward is calculated as follows: ； in, For instant rewards, Rewards for quality improvement Rewards for efficiency Punishment for mistakes.

5. The global optimization method for topological networks based on graph reinforcement learning according to claim 4, characterized in that, The calculation method for the quality improvement reward is as follows: ； in, As a reward amplification factor, The difference in the total global mesh quality score before and after performing the selected topology editing action; ； in, For orthogonality, For smoothness, Aspect ratio, denoted as the total number of quadrilaterals, and i as the grid cell number.

6. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, In the experience storage step, each experience tuple stored in the experience replay pool is represented as: ； in, This is the current state of the diagram. The identifier for the selected topology editing action in the current step. For instant rewards, The next graph state obtained after performing the selected topology editing action. This is the end label for iterative execution.

7. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, The batch training step includes the following sub-steps: A batch of experience tuples are randomly sampled from the experience replay pool; In each empirical tuple The input is fed into the target network to obtain the Q-value predictions of all candidate topology editing actions output by the target network; for each The maximum value is selected from the Q-value predictions of all candidate topology editing actions. This maximum value represents the present value estimate of the future cumulative reward that can be obtained by performing the optimal topology editing action in the next graph state. The state of the next graph obtained after performing the selected topology editing action; Multiply the maximum value by the discount factor To obtain the present value of future rewards; When the iteration execution ends, the label is displayed. When true, it indicates that the current graph state is a terminated state. In this case, the present value of future rewards is set to zero, and only immediate rewards are used. Perform calculations; Will Input the main policy network and obtain the Q-value prediction vectors of all candidate topology editing actions output by the main policy network. This represents the current state of the graph. Extracting the Q-value prediction vectors of all candidate topology editing actions output by the main policy network and... The corresponding Q value is denoted as the current Q value. The identifier for the topology editing action selected in the current step; Calculate the temporal difference loss between the current Q-value and the target Q-value, and add a regularization term; The parameters of the master policy network are updated by using the backpropagation algorithm, enabling the master policy network to learn the mapping strategy from graph states to optimal topology editing actions.

8. The global optimization method for topological networks based on graph reinforcement learning according to claim 7, characterized in that, The target Q value is calculated as follows: ； in, For the target Q value, For the target network in The maximum value among all candidate topology editing action Q values ​​output below. Indicates in The topology editing action selected by the reinforcement learning agent.

9. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, The model deployment steps specifically include: Load the main policy network after training convergence; Input the initial topology network to be processed, and execute the graph state construction, reinforcement learning agent decision-making, action execution and state transition and reward calculation steps in sequence, iterating until the preset termination condition is met; Output the globally optimized topology network for subsequent engineering simulation analysis.

10. The global optimization method for topological networks based on graph reinforcement learning according to claim 1, characterized in that, In the action execution and state transition steps, a geometric validity verification sub-step is included before executing the selected topology editing action: Verify that the topology blocks to be merged are adjacent and do not overlap; Verify whether the topological edges to be split are connected, and whether they are topologically connected after splitting; Verify whether adjusting node coordinates leads to mesh cell distortion or penetration of geometric boundaries; If the verification fails, the topology editing action is marked as invalid, triggering an error penalty and the topology editing action is not executed before proceeding to the next step.

Images