Structural grid quality optimization method based on deep reinforcement learning

Abstract

The application discloses a structure grid quality optimization method based on deep reinforcement learning, relates to the field of computer aided engineering and computational fluid dynamics preprocessing, and models the position optimization problem of singular points in a structure grid as a sequential decision process, constructs an interaction framework of a reinforcement learning intelligent agent and a grid environment, extracts grid features through a state perception module, an intelligent agent strategy network outputs singular point position actions, the environment updates the grid and calculates a reward value based on quality change after execution, and the interaction experience is stored to update the strategy network. In order to accelerate the training convergence, a pre-trained expert strategy network is introduced, the recommended action is introduced into a reward function to guide the exploration direction of the intelligent agent. After the training is completed, the fixed strategy network can be quickly deployed to optimization of a new geometric model. The automatic global optimization of the singular point layout is realized, the grid quality and optimization efficiency are significantly improved, the trained model has good generalization ability, and can be widely applied to structure grid generation.

Description

Technical Field

[0001] This invention relates to the field of computer-aided engineering and computational fluid dynamics preprocessing, specifically to a method for optimizing the quality of structured meshes based on deep reinforcement learning. Background Technology

[0002] In the preprocessing stage of computational fluid dynamics (CFD) and computer-aided engineering (CAE), the quality of the structured mesh directly affects the accuracy, stability, and convergence speed of the simulation. Structured meshes are typically generated using a partitioned topology method, dividing the complex computational domain into multiple quadrilateral (2D) or hexahedral (3D) topological blocks. The intersections of these blocks are called singularities. Singularities are nodes that connect at least 4 (2D) or 6 (3D) topological blocks. Their location and distribution determine the orientation, orthogonality, and smoothness of the mesh lines, and are key factors affecting mesh quality.

[0003] Currently, the placement and optimization of singular points heavily rely on the manual experience of engineers. A typical process involves engineers manually pre-setting the initial positions of singular points based on the geometry, generating an initial mesh, and then using quality control tools (such as Jacobian, orthogonality, and aspect ratio metrics) to identify areas with poor quality. The engineers then manually and repeatedly adjust the singular point coordinates for trial optimization until the mesh quality reaches an acceptable standard. This process has the following significant drawbacks:

[0004] (1) High-dimensional nonlinear optimization problem: The movement of multiple singular points affects each other. Adjusting one point may improve the local quality but worsen other areas, forming a high-dimensional, strongly coupled, non-convex optimization problem. Manual adjustment makes it difficult to find the global optimal solution.

[0005] (2) Reliance on expert experience and trial and error: Even experienced engineers still need to make a lot of trial and error for new configurations or complex areas. The process is time-consuming, tedious, and the results vary from person to person, lacking standardization.

[0006] (3) Lack of systematic automation methods: The automated optimization functions of existing commercial software (such as Laplace smoothing) mainly target internal mesh nodes. They usually dare not move singular points, which are the foundation of the topology, to avoid topology collapse or negative volume meshes. The application of existing AI technology in mesh optimization is mostly focused on mesh smoothing. There is still a lack of intelligent decision-making systems that can understand the topology, perceive the geometric environment, and take mesh quality as the direct optimization target. Summary of the Invention

[0007] This invention aims to overcome the shortcomings of existing technologies and provide a structural mesh quality optimization method based on deep reinforcement learning. Its core objective is to model the singularity point location optimization problem as a sequential decision-making process, utilize the autonomous exploration capability of reinforcement learning to learn the optimal movement strategy, and introduce an expert network constructed using deep learning for directional guidance. This significantly reduces ineffective exploration, shortens training time, and achieves automated, intelligent, and global optimization of structural mesh quality.

[0008] To achieve the above-mentioned objectives, this invention provides a method for optimizing the quality of structured meshes based on deep reinforcement learning, the method comprising:

[0009] Step S1: Obtain the initial structure mesh to be optimized, and determine the singular points and initial coordinates of the singular points in the initial structure mesh;

[0010] Step S2: Extract the state features of the current structure mesh and construct the state vector. Wherein, the state vector The current coordinates of the singular points in the current structured mesh are included; in the first iteration, the current structured mesh is the initial structured mesh, and the current coordinates of the singular points are the initial coordinates.

[0011] Step S3: Convert the state vector Input the policy network of the pre-trained or initialized reinforcement learning agent, and output the displacement action of the singular point. The reinforcement learning agent includes a policy network and a value network.

[0012] Step S4: Based on the displacement action Move the singular point, update the current coordinates of the singular point to the new coordinates, and regenerate the updated structure mesh based on the updated singular point coordinates;

[0013] Step S5: Calculate the quality index of the updated structured mesh, and calculate the reward value based on the change between the quality index and the quality index of the structured mesh before the move. ;

[0014] Step S6: Re-extract the state vector based on the updated structured mesh, denoted as... ; The interaction experience of the current iteration round The experience data is stored as a set of experience data in the experience pool;

[0015] Step S7: Based on the interaction experience data stored in the experience pool, update the parameters of the policy network and value network of the reinforcement learning agent using a reinforcement learning algorithm;

[0016] Step S8: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] As the current state vector for the next iteration, repeat steps S2 to S7 until the preset termination condition is met, and obtain the optimized structure mesh.

[0017] This method decomposes the optimization process into seven steps: state awareness (S2), agent decision-making (S3), environment execution (S4), reward calculation (S5), experience storage (S6), policy update (S7), and iterative loop (S8). This transforms the manual trial-and-error process into a machine-learnable sequential decision-making problem, laying the foundation for subsequent reinforcement learning solutions. It addresses the limitations of existing manual optimization methods in handling high-dimensional, strongly coupled singularity optimization problems and the lack of systematic automation methods. By constructing a complete automated closed-loop process from grid input to optimization output, it achieves sequential decision modeling for singularity optimization.

[0018] Preferably, the state vector constructed in step S2 It must include at least one of the following features:

[0019] Normalized spatial coordinates of singularities;

[0020] Orthogonality index of grid cells within the neighborhood of a singular point;

[0021] Statistical characteristics of smoothness of all grid cells.

[0022] In this context, the agent needs a comprehensive understanding of the grid state to make informed decisions. However, grid data is high-dimensional and complex, making it difficult to use directly as input. This method extracts three types of features: normalized coordinates of singular points (geometric location), neighborhood orthogonality (local quality), and global smoothness (global quality). These features form a low-dimensional representation that comprehensively reflects the grid state, enabling the agent to understand the current grid quality. A state vector containing singular point coordinates, local quality features, and global statistical features is constructed, achieving digital perception of the grid environment.

[0023] Preferably, the displacement action output in step S3 A continuous action space representation is used, specifically the displacement of each singular point in the x, y, and z directions. By directly outputting the continuous displacement of each singular point in the x, y, and z directions, the agent can perform fine adjustments in any direction, improving optimization accuracy.

[0024] Preferably, in step S4, when regenerating the updated structured mesh, the mesh generation algorithm is invoked based on the updated singular point coordinates to recalculate the coordinates of all mesh nodes. The updated structured mesh includes all nodes of the entire field mesh. After the singular points move, a new mesh needs to be generated quickly to evaluate the effect, but regenerating the entire field mesh is computationally intensive, and the mesh validity needs to be guaranteed. This method invokes the mesh generation algorithm based on the updated singular point coordinates to recalculate the coordinates of all mesh nodes, ensuring the topological integrity and geometric validity of the new mesh, providing accurate objects for the next round of evaluation. This achieves fast and reliable mesh updates based on the updated singular point coordinates.

[0025] Preferred reward value The calculation method is as follows:

[0026] ;

[0027] in, , and These are the weighting coefficients. and These are the quality indices of the structural mesh before and after the singularity point movement. The expert strategy network recommends actions. This is a penalty for when the grid is invalid.

[0028] The reward function consists of three parts: Encourage quality improvement Guide the intelligent agent to follow the expert's direction. Ineffective grid points are penalized, and weight balancing ensures that the agent balances quality improvement and stability during exploration. A multi-objective reward function integrating quality improvement, expert guidance, and ineffective grid penalty is constructed to guide the agent in learning the desired behavior.

[0029] Preferably, the method further includes an expert network guidance mechanism:

[0030] A pre-trained expert policy network is trained through supervised learning, and its training data is based on an expert dataset generated after perturbing the coordinates of singular points.

[0031] During the training process of the reinforcement learning agent, the expert policy network is based on the state vector. The output expert policy network suggests actions A reward function is introduced to guide the exploration direction of the reinforcement learning agent. A pre-trained expert policy network learns basic movement directions, and expert-suggested actions are incorporated into the reward function during training. This allows the agent to explore promising directions from the outset, avoiding blind random attempts and accelerating convergence. This significantly shortens training time and improves sample efficiency.

[0032] Preferably, the reinforcement learning algorithm used in step S7 is the PPO algorithm, which jointly updates the policy network and value network of the reinforcement learning agent by sampling data from the experience pool. By employing the PPO (Proximal Policy Optimization) algorithm, the policy update magnitude is limited through importance sampling and pruning mechanisms, while simultaneously updating both the policy network and the value network, ensuring the stability and sample efficiency of the training process. This achieves stable updating and convergence of the policy network.

[0033] Preferably, the termination condition in step S8 includes at least one of the following:

[0034] The preset maximum number of iterations has been reached;

[0035] The improvement in the quality index of the structured mesh within consecutive preset rounds is less than a preset threshold.

[0036] Reward Value Converge to a stable range. Set a maximum number of iterations to prevent infinite loops, set a threshold for quality improvement to determine convergence, and set a reward value convergence condition. Through a combination of multiple conditions, ensure timely stopping when the optimum or convergence is reached.

[0037] Preferably, the method further includes step S9:

[0038] The strategy deployment steps include:

[0039] Step S91: Obtain the policy network of the trained and fixed reinforcement learning agent to obtain the fixed policy network;

[0040] Step S92: Obtain the first structured mesh to be optimized, and determine the singular points in the first structured mesh and their initial coordinates;

[0041] Step S93: Obtain the current structured mesh based on the first structured mesh, extract the state features of the current structured mesh, and construct the first state vector;

[0042] Step S94: Input the first state vector into the fixed policy network and output the first displacement action for the singular point;

[0043] Step S95: Move the singular point according to the first displacement action, obtain the updated singular point coordinates, and update and generate the second structure mesh based on the updated singular point coordinates;

[0044] Step S96: Extract the second state vector based on the second structure mesh, use the second state vector as the state vector input for the next iteration, and use the second structure mesh as the current structure mesh for the next iteration;

[0045] Step S97: Repeat steps S93 to S96 until the preset deployment termination condition is met, and output the optimized structure mesh.

[0046] The trained policy network is fixed, and for a new first-structured grid, four core steps are iteratively executed: state awareness (S93), decision-making (S94), movement update (S95), and state update (S96), looping until the termination condition is met, outputting an optimized grid. No experience storage or network updates are performed throughout the process. This enables rapid deployment that is ready to use immediately after training, and the deployment process requires no manual intervention.

[0047] Preferably, the deployment termination condition in step S97 includes at least one of the following:

[0048] The preset maximum number of deployment iterations has been reached;

[0049] The quality index of the second structural mesh reaches the preset threshold;

[0050] The improvement in the quality index of the second structured mesh within a preset number of consecutive iterations is less than a preset threshold. A maximum number of iterations is set to prevent infinite loops, a quality index threshold is set to ensure that requirements are met, and a quality improvement threshold is set to determine convergence. Multiple conditions are combined to ensure timely stopping when the optimization objective is reached or convergence is achieved.

[0051] Because this invention employs a core technical solution of optimizing singular points into a sequential decision-making process and innovatively introduces deep learning expert networks to guide training, it can achieve the following significant technical effects:

[0052] Fully automated and intelligent optimization: The intelligent agent completely replaces manual trial and error, and can make optimization decisions autonomously, completely liberating engineers from tedious and repetitive work. The optimization process requires no human intervention.

[0053] Powerful global optimization capability: The exploratory nature of deep reinforcement learning enables it to effectively escape local optima and find a better distribution of singular points in a high-dimensional, non-convex search space than traditional gradient methods, thereby obtaining a higher quality grid.

[0054] Significantly improves optimization efficiency: The expert network guidance mechanism provides high-quality initial exploration directions for reinforcement learning agents, avoids a large number of random and invalid attempts, and greatly reduces the time cost of artificial intelligence methods in engineering applications.

[0055] Excellent generalization performance: The trained agent's policy learns the universal mapping relationship between grid states and singular point movements, and can transfer its optimization capabilities to unseen geometric models with similar topological features, achieving immediate use after training.

[0056] The optimization results are repeatable and standardized: Based on the deterministic neural network model, the optimization results are completely consistent for the same initial conditions, avoiding the randomness caused by manual operation and facilitating the establishment of a standardized grid generation process. Attached Figure Description

[0057] The accompanying drawings, which are provided to further illustrate embodiments of the invention and constitute a part of this invention, are not intended to limit the scope of the invention.

[0058] Figure 1 This is a flowchart of a structured mesh quality optimization method based on deep reinforcement learning. Detailed Implementation

[0059] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other.

[0060] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0061] Example 1;

[0062] Please refer to Figure 1 , Figure 1 The present invention provides a method for optimizing the quality of structured meshes based on deep reinforcement learning, characterized in that the method includes:

[0063] Step S1: Obtain the initial structure mesh to be optimized, and determine the singular points and initial coordinates of the singular points in the initial structure mesh;

[0064] Step S2: Extract the state features of the current structure mesh and construct the state vector. Wherein, the state vector The current coordinates of the singular points in the current structured mesh are included; in the first iteration, the current structured mesh is the initial structured mesh, and the current coordinates of the singular points are the initial coordinates.

[0065] Step S3: Convert the state vector Input the policy network of the pre-trained or initialized reinforcement learning agent, and output the displacement action of the singular point. The reinforcement learning agent includes a policy network and a value network.

[0066] Step S4: Based on the displacement action Move the singular point, update the current coordinates of the singular point to the new coordinates, and regenerate the updated structure mesh based on the updated singular point coordinates;

[0067] Step S5: Calculate the quality index of the updated structured mesh, and calculate the reward value based on the change between the quality index and the quality index of the structured mesh before the move. ;

[0068] Step S6: Re-extract the state vector based on the updated structured mesh, denoted as... ; The interaction experience of the current iteration round The experience data is stored as a set of experience data in the experience pool;

[0069] Step S7: Based on the interaction experience data stored in the experience pool, update the parameters of the policy network and value network of the reinforcement learning agent using a reinforcement learning algorithm;

[0070] Step S8: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] As the current state vector for the next iteration, repeat steps S2 to S7 until the preset termination condition is met, and obtain the optimized structure mesh.

[0071] The technical principle of this invention is to reconstruct grid optimization as an interactive learning process between the agent and the grid environment. Specifically:

[0072] Markov Decision Process Modeling: The iterative optimization process of singular points is modeled as a Markov Decision Process (MDP), where the state is the feature representation of the current grid, the action is the displacement of the singular point, and the reward is the change in grid quality.

[0073] Reinforcement learning solution: A reinforcement learning agent is used as the solver for the MDP. An optimal policy is learned through trial and error, which can be adjusted based on the current grid state. Choose actions that maximize long-term cumulative rewards. (i.e., singular point movement), realizing an end-to-end mapping from grid state perception to movement decision.

[0074] Expert Network Guidance: To accelerate training convergence, an expert policy network is pre-trained. This network learns basic movement directions from expert data generated by perturbations through supervised learning. During reinforcement learning training, the actions suggested by the expert network are incorporated into the reward function to guide the agent to explore promising regions and avoid a large amount of ineffective random exploration.

[0075] Closed-loop optimization: Through a closed-loop iteration of state perception → agent decision-making → environment execution → reward calculation → experience storage → policy update, the agent continuously evolves and eventually learns an optimization policy that can be universally applied to similar geometric models.

[0076] This invention achieves its objective of intelligently, automatically, and globally optimizing the quality of structural meshes through the synergistic effect of the following technical features:

[0077] At the problem modeling level: the singularity optimization problem, which traditionally relies on manual trial and error, is transformed into a machine-learnable Markov decision process, laying the foundation for intelligent solution. This is a prerequisite for achieving automation.

[0078] At the state awareness level: By constructing multi-dimensional features, the agent can understand the mesh quality, providing an information basis for subsequent decision-making. This is the guarantee that the agent can make reasonable decisions.

[0079] At the decision-making and execution level: A continuous action space and a high-precision grid update mechanism are employed, enabling the agent to precisely control the location of singularities and obtain feedback in real time, forming a closed loop. This is the technical means to achieve precise optimization.

[0080] At the learning guidance level: through expert network guidance and a carefully designed reward function, learning efficiency is significantly improved, enabling the agent to acquire effective strategies within a reasonable timeframe. This is key to solving the pain point of excessively long training times in engineering projects.

[0081] At the algorithm implementation level: stable algorithms such as PPO are used to ensure the convergence and stability of the training process. This guarantees the feasibility of the technical solution.

[0082] At the engineering application level: It provides complete training termination conditions and deployment processes, enabling the technical solutions to be truly implemented and transformed from the laboratory to engineering practice.

[0083] In summary, this invention, through a complete technical chain of state perception → agent decision-making → environmental execution → reward feedback → policy learning → deployment and application, transforms the trial-and-error process that originally relied on human experience into a machine-learnable intelligent decision-making system. It fundamentally solves the technical problems of existing technologies that rely on expert experience, lack automation methods, and are difficult to optimize globally, and realizes the intelligent, automated, and global optimization of structured grid quality.

[0084] The steps of this invention will be described in detail below:

[0085] (1) Complete method flow:

[0086] This method, from initial mesh input to optimized output mesh, includes the following steps performed sequentially:

[0087] 1) Initialization: Input the initial structure mesh with singular points.

[0088] 2) State awareness: Extract the features of the current grid and encode them into a state vector.

[0089] 3) Agent decision-making: The reinforcement learning agent outputs the displacement action to the specified singular point based on the state vector.

[0090] 4) Environment execution and mesh update: Execute actions, move singularities, and regenerate the entire field mesh.

[0091] 5) Reward Calculation and State Update: Evaluate the quality of the new grid, calculate the reward, and extract the new state.

[0092] 6) Loop optimization and learning: Store interaction experience and use it to update the agent's policy; return to step 2), forming a closed loop, until the termination condition is met.

[0093] 7) Strategy deployment: The pre-trained optimal strategy is applied to the rapid optimization of the new geometric model.

[0094] (2) Method principle:

[0095] The principle behind this method is to reconstruct mesh optimization as an interactive learning process between the agent and the mesh environment. The agent learns a policy through trial and error, which adapts to the current mesh state. Choose the displacement action that maximizes the long-term cumulative reward (i.e., the continuous improvement of mesh quality). (Singularity movement). Its learning objective is to maximize expected return.

[0096] (3) Detailed introduction of each step

[0097] Step 1: Initialization:

[0098] Execution subject: User / preprocessing system; Execution content and object: Input the initial structure mesh and determine the initial coordinates of singular points. This step is the starting point of the optimization process.

[0099] Step 2: State Awareness

[0100] Execution subject: Environment state extraction module; Execution content and object: Feature extraction of the current mesh. State vector. It needs to fully reflect the mesh quality and geometric context, including:

[0101] a) Singular point coordinates: the normalized positions of all singular points.

[0102] b) Local quality characteristics: Orthogonality of mesh cells in the neighborhood of each singular point.

[0103] c) Global statistical characteristics: smoothness of the entire grid cell.

[0104] Logical relationship: Output of this step The input for step 3 is the digital perception of the grid environment.

[0105] Step 3: Agent Decision Making:

[0106] Executor: Policy network of the reinforcement learning agent; Execution content: Policy network (Actor) receives states. Output displacement action Target of execution: displacement action It acts on a selected singularity in the current grid. This invention preferably employs a continuous action space, that is, directly outputting the action space of each target singularity. Continuous displacement in the direction ( This method offers high decision-making accuracy and aligns with physical intuition. Logical relationship: This step is the core decision-making stage of intelligent optimization; it generates the execution instructions for step 4 based on the perception results from step 2.

[0107] Step 4: Environment Execution and Mesh Update

[0108] Execution subject: the environment's mesh generator; Execution content and object: based on displacement actions. Move the corresponding singular points. Then, invoke the existing mesh generation algorithm, and based on the new singular point layout, recalculate and generate the coordinates of all mesh nodes in the entire field to obtain the new mesh. Logical relationship: This step transforms the agent's abstract decisions into actual changes in mesh geometry, providing the evaluation object for step 5.

[0109] Step 5: Reward Calculation and Status Update

[0110] Execution entities: the environment's reward calculation module and state extraction module;

[0111] Execution content: Reward calculation: Calculate the reward based on the newly generated grid. ,

[0112] ， in, ,

[0113] and These are the weighting coefficients. and These are the quality indices of the structural mesh before and after the singularity point movement. Actions recommended by expert network This is the penalty for an invalid mesh, specifically the significant penalty incurred when a singularity moves, causing the mesh to become invalid.

[0114] Optional sub-step: Expert network guidance—a key means of improving training efficiency in this invention;

[0115] To accelerate convergence in the early stages of training, an expert policy network pre-trained through supervised learning can be introduced. This expert network generates an expert dataset based on perturbations. In the early stages of training, an expert guidance mechanism is introduced, allowing the expert network to guide the policy based on... The suggested singularity movement direction guides the agent's own output movement vector; if the displacement action... Actions recommended by experts If the direction is consistent, a reward will be given.

[0116] State update: Repeat step 2 on the new mesh to generate a new state. .

[0117] Logical relationship: This step generates a learning signal. and the input for the next iteration ,and , Together, they constitute a complete interactive experience.

[0118] Step 6: Iterative Optimization and Learning:

[0119] Execution entity: Learning algorithm of reinforcement learning agent (PPO);

[0120] Execution content: (a) Experience storage: store the quadruple (b) Policy update: Periodically sample data and use reinforcement learning algorithms to update the parameters of the policy network and value network.

[0121] Logical relationship: This step is the process of self-evolution of the intelligent agent. It uses the feedback from step 5 to improve the decision-making ability of step 3, forming a complete perception-decision-execution-learning closed loop.

[0122] Step 7: Strategy Deployment:

[0123] Execution subject: The trained optimal policy network; Execution content and object: Fix the parameters of the optimal policy network after training convergence. For a new geometric model, load this network and execute steps 1 to 5 (excluding the network update in step 6). The agent can then quickly output an optimized action sequence based on the learned policy.

[0124] Logical Relationship: This step demonstrates the ultimate value and generalization ability of the method, enabling automated optimization that is ready to be trained and used immediately.

[0125] The specific methods for extracting state features in step S2 include:

[0126] Singular point coordinate normalization methods: For example, normalization can be performed using the minimum and maximum values ​​of global coordinates, or normalization can be performed based on the geometric bounding box, etc. The embodiments of the present invention will not elaborate on or limit the corresponding methods.

[0127] The specific calculation method of the orthogonality index is as follows: for example, the deviation of the interior angle of the grid cell from 90°, the cosine value of the included angle between adjacent sides, etc. The embodiments of the present invention will not elaborate or limit the corresponding calculation methods.

[0128] The specific calculation of smoothness statistical characteristics, such as the gradient of the area / volume change of adjacent grid cells, the change of node curvature, etc., are not described or limited in the embodiments of the present invention.

[0129] The dimension of the feature vector: Specifically, it can be 20-dimensional features extracted for each singular point, 10-dimensional statistical features for the whole field, the dimension of the total state vector, etc. The embodiments of this invention will not elaborate or limit the corresponding details.

[0130] In this embodiment of the invention, the specific structure of the policy network can be:

[0131] Network architecture: For example, whether to use a multilayer perceptron (MLP) or a convolutional neural network (CNN), the number of neurons in each layer, and the activation functions (ReLU, tanh, etc.).

[0132] Input / output dimensions: The input dimension is the same as the state vector dimension, and the output dimension is 3 × the number of singular points (each singular point is displaced in the x, y, and z directions).

[0133] Network parameter initialization methods: For example, Xavier initialization or He initialization can be used. The specific structure will not be described or limited in this embodiment of the invention.

[0134] In this embodiment of the invention, the specific structure of the value network can be:

[0135] Network architecture: Similar to or sharing the underlying feature extraction layer with the policy network;

[0136] Output dimension: a single scalar representing the state value;

[0137] Relationship with the policy network: whether parameters are shared, whether training is independent; the specific structure is not described or limited in this embodiment of the invention.

[0138] In this embodiment of the invention, the mesh generation algorithm may employ the following algorithm:

[0139] The mesh generation algorithm used, for example, whether it is based on transfinite interpolation (TFI), elliptic partial differential equation solving or algebraic mesh generation methods, is not described or limited in this embodiment of the invention.

[0140] The specific process of mesh regeneration after singular point movement: how to ensure mesh topology invariance and how to handle boundary constraints are not elaborated or limited in the embodiments of this invention.

[0141] Criteria for determining invalid meshes: For example, negative volume elements, excessive element distortion, mesh self-intersection, etc., are not elaborated or limited in the embodiments of this invention.

[0142] In this embodiment of the invention, the perturbation method during the generation of the expert dataset can be, for example, adding Gaussian noise (mean 0, standard deviation 5%-15% of the grid size) to the singular point coordinates. The scale of the expert data can be, for example, generating 100,000 perturbation samples. The definition and acquisition of the optimal action: how to select actions that can improve quality from the perturbation samples as labels; the training parameters of supervised learning: learning rate, batch size, number of training rounds, optimizer selection, etc. This embodiment of the invention does not elaborate on or limit the relevant details.

[0143] Example 2;

[0144] Building upon Example 1, Example 2 of this invention provides a structured mesh quality optimization method based on deep reinforcement learning, which can be widely applied to the preprocessing stage of aerodynamic simulation analysis of aircraft. Aircraft geometry typically includes complex curved surface features such as wings, fuselages, wingtips, and engine nacelles. The generation of structured meshes in these areas highly depends on the proper arrangement of singularities. Taking a typical unmanned aerial vehicle (UAV) as an example, the mesh quality requirements at locations such as the wing-fuselage junction, wingtip vortex region, and air intake lip are extremely high, directly affecting the accuracy of simulation results such as aerodynamic coefficient calculation, boundary layer separation prediction, and shock wave capture.

[0145] Specific methods include:

[0146] Step S1 (Initialization): Input the initial structure mesh of the UAV's 3D geometric model and determine the initial coordinates of singular points in key areas such as the wing leading edge, wingtip, and fuselage connection.

[0147] Step S2 (State Awareness): Extract the spatial coordinates of singular points (such as the wing spanwise position), local orthogonality index (affecting boundary layer resolution), and global smoothness characteristics (affecting shock wave capture capability);

[0148] Step S3 (Agent Decision): The reinforcement learning agent outputs the displacement action of the singular point at the wing-fuselage connection according to the current grid state, so as to improve the grid orthogonality in this area;

[0149] Step S4 (Mesh Update): Based on the updated singular point coordinates, the full-field mesh is regenerated using the Transcendental Interpolation (TFI) method to ensure a smooth transition of mesh lines in the wingtip region;

[0150] Step S5 (Reward Calculation): Calculate the minimum value of the Jacobian determinant and the orthogonality deviation of the new grid, and compare it with the value before the move. At the same time, introduce expert network suggestions for action to guide the process.

[0151] Steps S6-S7 (Learning and Updating): Store the interaction experience and update the policy network using the PPO algorithm, so that the agent gradually learns to place better singular points in the wingtip vortex region.

[0152] Step S8 (Iterative Optimization): Repeat the above process until convergence to obtain the optimal singular point layout suitable for this UAV model;

[0153] Step S9 (Policy Deployment): Fix the trained policy network and apply it to the rapid optimization of new UAV models (such as different airfoils and different aspect ratios) to achieve one-time training and multiple deployments.

[0154] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0155] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A structured mesh quality optimization method based on deep reinforcement learning, characterized in that, The method includes: Step S1: Obtain the initial structure mesh to be optimized, and determine the singular points and initial coordinates of the singular points in the initial structure mesh; Step S2: Extract the state features of the current structure mesh and construct the state vector. Wherein, the state vector The current coordinates of the singular points in the current structured mesh are included; in the first iteration, the current structured mesh is the initial structured mesh, and the current coordinates of the singular points are the initial coordinates. Step S3: Convert the state vector Input the policy network of the pre-trained or initialized reinforcement learning agent, and output the displacement action of the singular point. The reinforcement learning agent includes a policy network and a value network. Step S4: Based on the displacement action Move the singular point, update the current coordinates of the singular point to the new coordinates, and regenerate the updated structure mesh based on the updated singular point coordinates; Step S5: Calculate the quality index of the updated structured mesh, and calculate the reward value based on the change between the quality index and the quality index of the structured mesh before the move. ; Step S6: Re-extract the state vector based on the updated structured mesh, denoted as... ; The interaction experience of the current iteration round ( , , , Experience data is stored as a set of experience data in the experience pool; Step S7: Based on the interaction experience data stored in the experience pool, update the parameters of the policy network and value network of the reinforcement learning agent using a reinforcement learning algorithm; Step S8: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] As the current state vector for the next iteration, repeat steps S2 to S7 until the preset termination condition is met, and obtain the optimized structure mesh.

2. The structured mesh quality optimization method based on deep reinforcement learning according to claim 1, characterized in that, The state vector constructed in step S2 It must include at least one of the following features: Normalized spatial coordinates of singularities; Orthogonality index of grid cells within the neighborhood of a singular point; Statistical characteristics of smoothness of all grid cells.

3. The structured mesh quality optimization method based on deep reinforcement learning according to claim 1, characterized in that, The displacement action output in step S3 A continuous motion space representation is used, specifically the displacement of each singular point in the x, y, and z directions.

4. The structured mesh quality optimization method based on deep reinforcement learning according to claim 1, characterized in that, In step S4, when regenerating the updated structural mesh, the mesh generation algorithm is called based on the updated singular point coordinates to recalculate the coordinates of all mesh nodes. The updated structural mesh includes all nodes of the entire field mesh.

5. The structured mesh quality optimization method based on deep reinforcement learning according to claim 1, characterized in that, Reward Value The calculation method is as follows: ； in, , and These are the weighting coefficients. and These are the quality indices of the structural mesh before and after the singularity point movement. The expert strategy network recommends actions. This is a penalty for when the grid is invalid.

6. The structured mesh quality optimization method based on deep reinforcement learning according to claim 1, characterized in that, The method also includes an expert network guidance mechanism: A pre-trained expert policy network is trained through supervised learning, and its training data is based on an expert dataset generated after perturbing the coordinates of singular points. During the training process of the reinforcement learning agent, the expert policy network is based on the state vector. The output expert policy network suggests actions A reward function is introduced to guide the exploration direction of the reinforcement learning agent.

7. The structured mesh quality optimization method based on deep reinforcement learning according to claim 1, characterized in that, The reinforcement learning algorithm used in step S7 is the PPO algorithm, which performs joint updates of the policy network and value network of the reinforcement learning agent by sampling data from the experience pool.

8. The method for optimizing structured mesh quality based on deep reinforcement learning according to claim 1, characterized in that, The termination condition in step S8 includes at least one of the following: The preset maximum number of iterations has been reached; The improvement in the quality index of the structured mesh within consecutive preset rounds is less than a preset threshold. Reward Value It converges to a stable range.

9. The method for optimizing structured mesh quality based on deep reinforcement learning according to claim 1, characterized in that, The method further includes step S9: The strategy deployment steps include: Step S91: Obtain the policy network of the trained and fixed reinforcement learning agent to obtain the fixed policy network; Step S92: Obtain the first structured mesh to be optimized, and determine the singular points in the first structured mesh and their initial coordinates; Step S93: Obtain the current structured mesh based on the first structured mesh, extract the state features of the current structured mesh, and construct the first state vector; Step S94: Input the first state vector into the fixed policy network and output the first displacement action for the singular point; Step S95: Move the singular point according to the first displacement action, obtain the updated singular point coordinates, and update and generate the second structure mesh based on the updated singular point coordinates; Step S96: Extract the second state vector based on the second structure mesh, use the second state vector as the state vector input for the next iteration, and use the second structure mesh as the current structure mesh for the next iteration; Step S97: Repeat steps S93 to S96 until the preset deployment termination condition is met, and output the optimized structure mesh.

10. The structured mesh quality optimization method based on deep reinforcement learning according to claim 9, characterized in that, The deployment termination condition in step S97 includes at least one of the following: The preset maximum number of deployment iterations has been reached; The quality index of the second structural mesh reaches the preset threshold; The improvement in the quality index of the second structured mesh within a consecutive preset number of rounds is less than a preset threshold.

Images