Evolutionary strategy-based physical information neural network system construction method

Abstract

The application provides an evolutionary strategy-based physical information neural network system construction method, which comprises the following steps: S1, obtaining a partial differential equation of a target field to be solved, a solution domain, initial conditions and boundary conditions; S2, obtaining an initial physical information neural network system and performing parameter initialization; S3, constructing an initial sampling strategy and an initial loss function; S4, collecting a plurality of sampling points from the solution domain according to the initial sampling strategy to construct a training set, and pre-training the physical information neural network system to be trained based on the constructed training set and the initial loss function; and S5, performing multi-round iterative training on the pre-trained physical information neural network system based on the initial loss function and the initial sampling strategy, and adjusting the weight of each loss of the loss function and the sampling strategy in the next round of iterative training by using a covariance matrix adaptive evolutionary strategy in each round of iterative training.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to the field of neural network model construction and optimization technology, and more specifically, to a method for constructing a physical information neural network system based on an evolutionary strategy. Background Technology

[0002] Partial differential equations (PDEs) are core mathematical tools for describing complex phenomena in natural and engineering systems, and are widely used in fields such as fluid mechanics, materials science, quantum mechanics, and earth sciences. Traditional numerical methods, such as the finite element method, finite difference method, and spectral method, have formed a relatively mature system by discretizing PDEs in space and time. However, these methods are prone to the "curse of dimensionality" in high-dimensional, nonlinear, and complex geometric scenarios, with computational costs increasing exponentially with dimensionality. Furthermore, they struggle to balance accuracy and stability when dealing with complex boundary conditions and multi-scale phenomena. In addition, traditional methods typically rely on regular meshes and require a thorough understanding of the fundamental equations, limiting their flexibility.

[0003] In recent years, the development of machine learning technology has provided new pathways for solving partial differential equations (PDEs). Among them, Physics-Informed Neural Networks (PINNs) can approximate solutions in meshless continuous domains by directly embedding physical constraints into the neural network loss function and combining it with automatic differentiation techniques to solve derivatives. While ensuring physical consistency, PINNs can efficiently approximate PDE solutions in data-scarce scenarios, effectively overcoming the limitations of traditional numerical methods in complex solution domains and high-dimensional problems. With its strong fitting and scalability capabilities, PINNs have demonstrated significant advantages in fields such as fluid mechanics, financial mathematics, and multi-scale modeling.

[0004] However, existing related research still has the following technical defects: (1) It is easy to get trapped in local minima, which leads to the failure of physical constraints. The objective function of PINNs has highly non-convex and multi-peak characteristics, which is formed by the nonlinear characteristics of network parameters coupled with PDEs residuals, initial conditions and boundary conditions. When only gradient-based optimization methods are used, the optimization process depends on local gradient information, which is easy to get trapped in saddle points or local minima, resulting in large local residuals, and even solutions that violate physical constraints locally. (2) Poor training stability and slow convergence speed. The gradient scales of different loss terms such as PDEs residuals, initial conditions and boundary conditions in PINNs are significantly different. Gradient scale mismatch can easily cause some loss terms to dominate training or gradient oscillation. The superposition of nonlinearity of network structure and long-term dependence in the time domain will further aggravate training oscillation, resulting in training instability and slow convergence speed. (3) It is impossible to dynamically balance physical loss terms and initial boundary condition terms. If each loss term is set to a fixed weight, it cannot adapt to the dynamic optimal trade-off between physical accuracy and initial boundary condition matching degree at different training stages. Fixed weights may overemphasize a single objective at a certain stage of training, such as overfitting the initial value / boundary condition in the early stage while ignoring the decrease of PDEs residuals, or vice versa, making it difficult to achieve a balance between convergence speed and final accuracy. (4) Fixed sampling point distribution results in insufficient constraints on local high residual regions. If the sampling points are kept uniform or fixed, the network has insufficient ability to constrain local complex structures (such as solution peaks, boundary layers, and oscillating regions) in the spatial and temporal domains during training. High residual regions cannot obtain sufficient gradient information to correct the network due to the scarcity of samples, making it difficult to eliminate local errors.

[0005] In summary, existing PINNs methods suffer from problems such as being prone to getting trapped in local minima leading to the failure of physical constraints, poor training stability and slow convergence speed, inability to dynamically balance physical loss and initial boundary conditions, and insufficient constraints on local high residual regions due to fixed sampling. Therefore, there is an urgent need for a PINNs construction method that can achieve global optimization, dynamically balance loss weights, adaptive sampling, and improve convergence stability, so as to significantly improve the accuracy, robustness, and generalization ability of solving high-dimensional nonlinear partial differential equations.

[0006] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0007] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method for constructing a physical information neural network system based on an evolutionary strategy.

[0008] The objective of this invention is achieved through the following technical solution:

[0009] According to a first aspect of the present invention, a method for constructing a physical information neural network system based on an evolutionary strategy is provided. This physical information neural network system is used to solve partial differential equations in a target domain. The method includes: S1, obtaining the partial differential equations in the target domain to be solved, along with their solution domain, initial conditions, and boundary conditions; S2, obtaining an initial physical information neural network system and initializing its parameters; S3, constructing an initial sampling strategy and an initial loss function, wherein the initial loss function is a weighted sum of the residual loss of the partial differential equation to be solved, the initial condition loss, and the boundary condition loss, wherein the initial weights of each loss are the same; S4, according to the initial sampling strategy... S1) Collect multiple sampling points from the solution domain to construct a training set. Based on the constructed training set and the initial loss function, pre-train the physical information neural network system to be trained until the loss of the physical information neural network system is within the preset loss range. S2) Based on the initial loss function and the initial sampling strategy, perform multiple rounds of iterative training on the pre-trained physical information neural network system. In each round of iterative training, adopt the covariance matrix adaptive evolution strategy to adjust the weight of each loss term in the loss function in the next round of iterative training with the goal of minimizing the loss. Optimize the sampling strategy with the goal of improving the prediction accuracy of the physical information neural network system for each region in the solution domain.

[0010] This solution can achieve at least the following beneficial technical effects:

[0011] This scheme enables the physical information neural network system to converge quickly to a stable state through pre-training, and adopts an adaptive evolution strategy of covariance matrix to simultaneously optimize the weight of loss function and sampling strategy during iterative training. It can adaptively balance various physical constraints, improve the prediction accuracy of each region in the solution domain, and effectively improve the convergence speed and overall accuracy of partial differential equation solving.

[0012] Optionally, in S4, the pre-training includes multiple iterations, wherein each iteration includes: S41, collecting multiple sampling points from the solution domain using an initial sampling strategy, and performing multiple iterations of training on the initial physical information neural network system using gradient descent based on all collected sampling points and the initial loss function; S42, initializing the relevant parameters of the covariance matrix adaptive evolution strategy, wherein the relevant parameters include at least step size, covariance matrix, evolution round, and population size, and optimizing the system parameters of the initial physical information neural network system trained in S41 using the covariance matrix adaptive evolution strategy in multiple rounds until the evolution round is reached.

[0013] This solution can achieve at least the following beneficial technical effects:

[0014] This approach combines gradient descent and covariance matrix adaptive evolution strategies in the pre-training stage for two-layer optimization, which can quickly converge the network parameters to a reasonable range, providing a stable and reliable initial model for subsequent iterative training, avoiding training divergence and improving overall solution efficiency.

[0015] Optionally, in S42, each round of optimization includes: using the system parameters of the current initial physical information neural network system as the mean, and generating a normal distribution of system parameters for this round based on the current step size and covariance matrix; randomly selecting multiple sets of system parameters from the normal distribution of system parameters for this round, wherein the number of system parameter sets collected is the same as the population size; collecting multiple sampling points from the solution domain based on the current sampling strategy, and configuring a physical information neural network system for each set of system parameters in the population of the current round; using each physical information neural network system configured in this round to predict the predicted solution for each sampling point collected in this round; based on all the predicted solutions of each physical information neural network system configured in this round, calculating the loss of each physical information neural network system configured in this round using the current loss function, and using the loss of each configured physical information neural network system as the fitness of the system parameters corresponding to that physical information neural network system; updating the parameters of the initial physical information neural network system based on all system parameters selected in this round and the fitness of each set of system parameters in the following manner.

[0016]

[0017] in, The system parameters of the neural network system represent the updated initial physical information. Indicates the lowest fitness After the system parameters are sorted in descending order of fitness, the group is located at the [number]th [position]. Bit-level system parameters express The weight coefficients are given by the fact that the sum of the weight coefficients corresponding to all system parameters is 1, and the weight coefficients corresponding to each group of system parameters are greater than 0.

[0018] This solution can achieve at least the following beneficial technical effects:

[0019] This scheme updates system parameters based on fitness ranking and weighted update rules, which can efficiently search for the optimal solution in the parameter space, improve the training stability and solution accuracy of the physical information neural network system, and avoid getting trapped in local optima.

[0020] Optionally, in S5, an adaptive evolutionary strategy based on the covariance matrix is ​​used to optimize the weights of each loss term in the loss function, including: initializing the relevant parameters of the adaptive evolutionary strategy based on the covariance matrix, wherein the relevant parameters include step size, covariance matrix, evolutionary round, and population size; iteratively executing the following steps until the evolutionary round is reached: configuring the mean of this round based on the weights of each loss term in the current loss function, and generating a normal distribution of loss weights for the current round based on the current step size and covariance matrix; randomly selecting multiple sets of loss weights from the normal distribution of loss weights for the current round, and configuring a loss function for each selected set of loss weights, wherein the number of selected weight sets is the same as the population size, and each set of loss weights includes the weights of partial differential equation residual loss, initial condition loss, and boundary condition loss; based on the current... The pre-sampling strategy collects multiple sampling points from the solution domain. Using all collected sampling points and each configured loss function, the current physical information neural network system is trained multiple times to obtain multiple trained physical information neural network systems corresponding one-to-one with the loss function. Based on the current sampling strategy, multiple sampling points are collected from the solution domain. Each trained physical information neural network system predicts the predicted solution for each sampling point. Based on all predicted solutions, the loss of each trained physical information neural network system is calculated using the current loss function. The loss of each trained physical information neural network system is used as the fitness of the corresponding loss weights. The weights of the loss function are updated based on all loss weights selected in this round and the fitness of each set of loss weights.

[0021] This solution can achieve at least the following beneficial technical effects:

[0022] This scheme employs an adaptive evolution strategy based on the covariance matrix to adaptively adjust the weights of each term in the loss function. This approach can optimize the residual loss, initial condition loss, and boundary condition loss of partial differential equations in a balanced manner, enabling various physical constraints to converge synchronously and improving the overall physical consistency and solution accuracy of the model.

[0023] Optionally, the weights of the loss function can be updated as follows:

[0024]

[0025]

[0026] in, This represents the mean. Indicates based on the first The loss weights of the loss function of the round are constructed from the first round. The average value of the wheel, Indicates based on the first The loss weights of the loss function of the round are constructed from the first round. The average value of the wheel, Indicates the first The step size of the round iteration, This represents the smallest fitness value in the k-th iteration. After the group loss weights are sorted in descending order of fitness, the one located at the th The loss weight of the bit, express The weighting coefficients, , and The weights of the partial differential equation residual loss, initial condition loss, and boundary condition loss are represented in that order.

[0027] This solution can achieve at least the following beneficial technical effects:

[0028] This scheme achieves stable optimization of loss weights by updating the number domain rules, ensuring that the weights are non-negative and have a reasonable ratio, further improving the adaptive balancing ability of the loss function, and enhancing the stability and solution effect of the training process.

[0029] Optionally, the sampling strategy is a sampling strategy that adjusts the sampling probability of each pre-divided region within the solution domain by introducing a temperature coefficient. The larger the temperature coefficient, the higher the sampling probability of the region with greater loss in the solution domain. In the initial sampling strategy, the temperature coefficient is 0, and the sampling probability of each region in the solution domain is consistent.

[0030] This solution can achieve at least the following beneficial technical effects:

[0031] The scheme adopts an adaptive sampling strategy based on temperature coefficient, which can dynamically adjust the sampling probability according to the loss of each region in the solution domain. Initial uniform sampling ensures global coverage and achieves accurate focusing on regions with significant residuals.

[0032] Optionally, in S5, an adaptive evolutionary strategy based on the covariance matrix is ​​used to adjust the temperature coefficient to optimize the sampling strategy. This includes: initializing the relevant parameters of the adaptive evolutionary strategy based on the covariance matrix, including the step size, covariance matrix, evolutionary round, and population size; iteratively executing the following steps until the iteration round is reached: using the current temperature coefficient as the mean, and generating a normal distribution of the temperature coefficient for the current round based on the current step size and covariance matrix; randomly selecting multiple temperature coefficients from the normal distribution of the temperature coefficient for the current round, and configuring a sampling strategy for each selected temperature coefficient, wherein the number of selected temperature coefficients is the same as the population size; for each sampling strategy configured in this round, collecting multiple sampling points from the solution domain to form a sample with the covariance matrix. The sampling strategy corresponds to a training set; the current physical information neural network system is trained once using the training set corresponding to each sampling strategy to obtain multiple trained physical information neural network systems that correspond one-to-one with the sampling strategy; multiple sampling points are uniformly sampled from the solution domain, and each trained physical information neural network system makes a prediction for each sampling point, and the loss of the trained physical information neural network system is calculated using the current loss function based on the prediction results of each trained physical information neural network system, wherein the loss of each trained physical information neural network system is the fitness of the temperature coefficient corresponding to the training set used for training; the mean is updated based on all temperature coefficients selected in this round and the fitness of each temperature coefficient.

[0033] This solution can achieve at least the following beneficial technical effects:

[0034] This scheme adaptively optimizes the temperature coefficient through an adaptive evolution strategy of the covariance matrix, which can dynamically increase the sampling density in the region of significant residuals, strengthen the constraints in the key region, and further accelerate the convergence speed and improve the solution accuracy.

[0035] Optionally, the sampling strategy can be configured as follows: the sampling probability of each pre-divided region in the solution domain can be determined as follows:

[0036]

[0037] in, This indicates the number of pre-divided regions in the solution domain. and This represents the index of a pre-divided region within the solution domain. and The terms in the solution domain are represented sequentially as follows: The region and the first Each region Indicates the i-th term in the solution domain Sampling probability of each region and The current physical information neural network pairs are represented sequentially. and Loss of sampling points in This represents the temperature coefficient.

[0038] Optionally, the mean can be updated as follows:

[0039]

[0040] in, Indicates the first The average value of the wheel, Indicates the first The average value of the wheel, Indicates the first The loss of the trained physical information neural network system corresponding to the temperature coefficient with the lowest fitness in each iteration. This represents the loss of a physical information neural network system trained using uniform sampling as the sampling strategy. Indicates the first The step length of the wheel.

[0041] Optionally, the target field can be any one of the following: fluid mechanics, materials science, quantum mechanics, and earth science.

[0042] According to a second aspect of the present invention, a method for solving partial differential equations in a target domain is proposed. The method includes: obtaining partial differential equations in the target domain; constructing a physical information neural network system for solving the partial differential equations in the target domain using the method described in the first aspect of the present invention based on the partial differential equations in the target domain; and using the constructed physical information neural network system to predict the solution at each sampling point in the solution domain of the partial differential equations in the target domain to solve the partial differential equations in the target domain.

[0043] This solution can achieve at least the following beneficial technical effects:

[0044] This invention provides a method for constructing a physical information neural network system based on an evolutionary strategy. By pre-training, adaptive loss weight optimization, and residual-driven sampling strategy, it solves the technical problems of traditional physical information neural networks in solving partial differential equations, such as being prone to getting trapped in local optima, constraint imbalance, low sampling efficiency, and insufficient solution accuracy. It can achieve efficient, stable, and high-precision solutions to complex partial differential equations in multiple fields.

[0045] Compared with the prior art, the advantages of the present invention are as follows:

[0046] This invention enhances the global search capability of gradient methods by introducing an adaptive evolutionary strategy based on the covariance matrix. It replaces fixed loss weights with dynamic variables that are adaptively optimized by the evolutionary strategy, and upgrades uniform sampling to optimizable importance sampling based on model residuals. These three elements work together to achieve a two-layer adaptive update of network parameter distribution and physical residual information. This effectively avoids local optima, dynamically balances physical constraints and initial boundary conditions, and accurately focuses on high residual regions, significantly improving the training stability, convergence speed, and solution accuracy of PINNs in solving complex nonlinear PDEs. Attached Figure Description

[0047] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0048] Figure 1 This is a schematic diagram of a method for constructing a physical information neural network system based on an evolutionary strategy according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of experimental data for solving one-dimensional convection equations using the baseline method according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of experimental data for solving one-dimensional convection equations using the physical information neural network system construction method proposed in this invention, according to an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of experimental data for solving the one-dimensional wave equation using the baseline method according to an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of experimental data for solving a one-dimensional wave equation using the physical information neural network system construction method proposed in this invention, according to an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0054] As mentioned in the background section, existing PINNs methods suffer from problems such as being prone to getting trapped in local minima, leading to the failure of physical constraints, poor training stability and slow convergence speed, inability to dynamically balance physical loss and initial boundary conditions, and insufficient constraints on local high residual regions due to fixed sampling. They cannot balance global optimization capability, training stability and solution accuracy in solving complex partial differential equations with high dimensions, nonlinearity and strong coupling, and are difficult to meet the requirements of robustness and generalization ability of models in real physical scenarios.

[0055] To address the aforementioned issues, this invention proposes a scheme for constructing a physical information neural network system based on an evolutionary strategy. This scheme enhances the global search capability of gradient optimization methods by introducing an adaptive evolutionary strategy for the covariance matrix. It adjusts the fixed loss weights into dynamic variables that are adaptively optimized by the evolutionary strategy, and upgrades traditional uniform sampling to optimizable importance sampling based on model residuals. Through the coordinated use of these three techniques, a two-layer adaptive update of network parameter distribution and physical residual information is achieved, effectively improving the stability and convergence efficiency of solving complex nonlinear partial differential equations.

[0056] Before detailing the embodiments of the present invention, a brief explanation of the Physical Information Neural Network (PINN) is provided. PINN is a meshless scientific computing framework for solving partial differential equations (PDEs). Its core is to embed physical laws into the neural network training process as soft constraints, achieving a deep integration of data-driven learning and physical priors. Unlike traditional supervised learning, PINN does not require a sample-label training set. During training, it randomly samples discrete points within the PDE solution domain and outputs the prediction street corresponding to each discrete point. Based on the sampled points and their corresponding prediction solutions, it calculates the equation residuals, boundary losses, and initial losses. By minimizing the total residuals, it drives the network to learn continuous solutions that satisfy physical consistency, rather than fitting the error between the predicted values ​​and the true labels.

[0057] According to one embodiment of the present invention, a method for constructing a physical information neural network system based on an evolutionary strategy is proposed. The physical information neural network system (i.e., physical information neural network, PINNs) is used to solve partial differential equations in a target domain, wherein the target domain includes, but is not limited to, fluid mechanics, materials science, quantum mechanics, and earth science. (See attached reference.) Figure 1 In summary, the method for constructing a physical information neural network system based on an evolutionary strategy includes steps S1, S2, S3, S4, and S5. To better understand this invention, each step will be described in detail below with reference to embodiments and accompanying drawings.

[0058] S1. Obtain the partial differential equations of the target domain to be solved, as well as its solution domain, initial conditions, and boundary conditions.

[0059] According to one embodiment of the present invention, the solution domain is used to define the spatiotemporal constraint range of the partial differential equation, clarify the effective solution space of the physical information neural network output, the initial conditions are used to give the state constraints of the physical system at the initial time, providing a time starting point for solving the equation, and the boundary conditions are used to limit the behavioral constraints of the physical system on the spatial boundary, ensuring that the network output conforms to the boundary constraint relationship of the real physical scene. The above information together constitutes the complete physical prior required for PINNs training, providing a standard constraint basis for subsequent sampling, network prediction, and residual calculation.

[0060] S2. Obtain the initial physical information of the neural network system and initialize its parameters.

[0061] According to an embodiment of the present invention, the Xavier uniform initialization method is used to initialize the parameters of the physical information neural network system. This initialization method can keep the variance of the input and output of each layer of the network stable, avoid gradient vanishing or gradient explosion caused by excessively large or small parameter initialization, provide a stable initial parameter distribution for subsequent evolutionary strategy optimization and physical residual training, and improve the overall convergence stability and training efficiency of the model.

[0062] S3. Construct an initial sampling strategy and an initial loss function. The initial loss function is a weighted sum of the residual loss, initial condition loss, and boundary condition loss of the partial differential equation to be solved, wherein each loss has the same initial weight.

[0063] According to an embodiment of the present invention, the sampling strategy is a sampling strategy that adjusts the sampling probability of each region in the solution domain by introducing a temperature coefficient. The larger the temperature coefficient, the higher the sampling probability of the region with greater loss in the solution domain. The initial sampling strategy has a temperature coefficient of 0, and the sampling probability of each region in the solution domain is consistent.

[0064] According to one embodiment of the present invention, PINNs achieve meshless PDE solving by embedding the residual terms of the physical equations into the loss function of a neural network. Defined on a spatiotemporal set... The system of partial differential equations described by equation constraints, boundary conditions, and initial conditions can be expressed as:

[0065]

[0066]

[0067]

[0068] In the formula, It is a solution to a partial differential equation. These are spatial coordinates. yes The boundary, It is a time coordinate. It refers to a time range. , , These represent operators defined by partial differential equations, initial conditions, and boundary conditions, respectively.

[0069] The physics-driven PINN first constructs a finite set of points. and its spacetime boundaries initial conditions Then, a parameterized neural network is used. Approximation is achieved by optimizing the residual loss. The residual loss is defined as follows:

[0070]

[0071]

[0072]

[0073]

[0074] in For sampling points, It is the output of the parameterized neural network. It represents the set of points inside the equation. It is a set of points selected based on initial conditions. It is the set of points selected on the boundary. The PDE residual term is the L2 error between the differential operator output of the network prediction and 0. The initial / boundary term is the L2 error between the network prediction value and the analytical / reference initial boundary value (known function or observation / numerical solution). , , These correspond to the loss weights of the residual loss, initial condition loss, and boundary condition loss, respectively. , , , These correspond to the residual loss, initial condition loss, boundary condition loss, and overall loss function of the equation, respectively. This represents the L2 norm.

[0075] S4. Collect multiple sampling points from the solution domain according to the initial sampling strategy to construct a training set. Based on the constructed training set and the initial loss function, pre-train the physical information neural network system to be trained until the loss of the physical information neural network system is within the preset loss range.

[0076] According to an embodiment of the present invention, the pre-training includes multiple iterations, wherein each iteration includes: S41, collecting multiple sampling points from the solution domain using an initial sampling strategy, and performing multiple iterations of training on the initial physical information neural network system using gradient descent based on all collected sampling points and an initial loss function; S42, initializing the relevant parameters of the covariance matrix adaptive evolution strategy, wherein the relevant parameters include at least step size, covariance matrix, evolution round, and population size, and optimizing the system parameters of the initial physical information neural network system trained by S41 using the covariance matrix adaptive evolution strategy in multiple rounds until the evolution round is reached. Specifically, by alternately executing the gradient descent and CMA-ES phases, the Adam optimizer is first used to update the network parameters through several rounds of gradient descent, allowing the network to quickly converge to near the basic feasible solution. In this phase, the overall loss is gradually reduced by calculating the gradient of the residuals at the sampling points. Next, a multivariate normal distribution with the current network parameters as the mean is constructed, and several sets of candidate network parameters are generated by sampling in conjunction with the covariance matrix. The PINN total loss value is calculated for each set of candidate parameters as the fitness, used to evaluate the parameters' performance. According to the weighted ranking strategy of CMA-ES, the set of candidate parameters with better fitness is used to update the mean vector, step size factor, and covariance matrix, making it more likely that the next sampling distribution will generate better-performing network parameters. Subsequently, the optimal candidate parameters updated by CMA-ES are used as the network's current parameters, replacing the original network parameters and putting the network into a new training state. By alternately executing the gradient descent and CMA-ES phases, a balance is achieved between the fine-grained optimization capability of gradient descent and the global search capability of the evolutionary strategy, thereby improving the network's accuracy and stability in solving complex PDE problems.

[0077] According to one embodiment of the present invention, each round of optimization includes: using the system parameters of the current initial physical information neural network system as the mean, and generating a normal distribution of system parameters for this round based on the current step size and covariance matrix; randomly selecting multiple sets of system parameters from the normal distribution of system parameters for this round, wherein the number of system parameter sets collected is the same as the population size; collecting multiple sampling points from the solution domain based on the current sampling strategy, and configuring a physical information neural network system for each set of system parameters in the population of the current round; using each physical information neural network system configured in this round to predict the predicted solution for each sampling point collected in this round; based on all the predicted solutions of each physical information neural network system configured in this round, calculating the loss of each physical information neural network system configured in this round using the current loss function, and using the loss of each configured physical information neural network system as the fitness of the system parameters corresponding to that physical information neural network system; updating the parameters of the initial physical information neural network system based on all system parameters selected in this round and the fitness of each set of system parameters in the following manner.

[0078]

[0079] in, The system parameters of the neural network system represent the updated initial physical information. Indicates the lowest fitness After the system parameters are sorted in descending order of fitness, the group is located at the [number]th [position]. Bit-level system parameters express The weight coefficients are given by the fact that the sum of the weight coefficients for all system parameters is 1, and the weight coefficient for each group of system parameters is greater than 0.

[0080] S5. Based on the initial loss function and initial sampling strategy, the pre-trained physical information neural network system is subjected to multiple rounds of iterative training. In each round of iterative training, the covariance matrix adaptive evolution strategy is adopted to adjust the weight of each loss term in the loss function in the next round of iterative training with the goal of minimizing the loss. The sampling strategy is optimized with the goal of improving the prediction accuracy of the physical information neural network system for each region in the solution domain.

[0081] According to an embodiment of the present invention, in S5, the weights of each loss term in the loss function are optimized using a covariance matrix adaptive evolution strategy, including: initializing relevant parameters of the covariance matrix adaptive evolution strategy, wherein the relevant parameters include step size, covariance matrix, evolution round, and population size; iteratively executing the following steps until the evolution round is reached: configuring the mean of this round based on the weights of each loss term in the current loss function, and generating a normal distribution of loss weights for the current round based on the current step size and covariance matrix; randomly selecting multiple sets of loss weights from the normal distribution of loss weights for the current round, and configuring a loss function for each selected set of loss weights, wherein the number of selected weight sets is the same as the population size, and each set of loss weights includes the weights of partial differential equation residual loss, initial condition loss, and boundary condition loss; and optimizing the weights of each loss term in the current sampling strategy based on the current sampling strategy. Multiple sampling points are collected in the solution domain. The current physical information neural network system is trained multiple times using all collected sampling points and each configured loss function to obtain multiple trained physical information neural network systems corresponding one-to-one with the loss function. Based on the current sampling strategy, multiple sampling points are collected from the solution domain. Each trained physical information neural network system predicts the predicted solution for each sampling point. Based on all predicted solutions, the loss of each trained physical information neural network system is calculated using the current loss function. The loss of each trained physical information neural network system is then used as the fitness of the corresponding loss weights. The weights of the loss function are updated based on all the loss weights selected in this round and the fitness of each set of loss weights. Specifically, the weights of the loss function are updated as follows:

[0082]

[0083]

[0084] in, This represents the mean. Indicates based on the first The loss weights of the loss function of the round are constructed from the first round. The average value of the wheel, Indicates based on the first The loss weights of the loss function of the round are constructed from the first round. The average value of the wheel, Indicates the first The step size of the round iteration, This represents the smallest fitness value in the k-th iteration. After the group loss weights are sorted in descending order of fitness, the one located at the th The loss weight of the bit, express The weighting coefficients, , and The weights of the partial differential equation residual loss, initial condition loss, and boundary condition loss are represented in that order.

[0085] According to one embodiment of the present invention, in S5, after the PINN model completes one forward inference, a candidate sampling point set is first constructed in the solution domain. The candidate points are generated in a uniform sampling manner. Then, the residual values ​​of the equations are calculated on the candidate point set using the current network parameters. The magnitude of the residuals reflects the degree of deviation in the physical solution of the region, thereby obtaining the current residual distribution of the network. Next, the residual distribution is normalized, and a temperature control parameter is introduced. The importance sampling probability is constructed based on the residual values. The temperature parameter can adjust the degree of influence of the residuals on the sampling probability; the higher the temperature, the more likely it is to concentrate on points with large residuals. To further improve the adaptability of the sampling distribution, CMA-ES is used to optimize the temperature hyperparameter: first, an initial temperature distribution is set, and an evolutionary strategy search space with temperature as the variable is constructed. Then, several sets of candidate temperature values ​​are generated by sampling through the multivariate normal distribution of CMA-ES, and a corresponding importance sampling probability distribution is generated based on each set of temperatures. Then, PINN is trained for several steps based on the sampling distribution, and the calculated residual values ​​are used as the fitness index of the candidate temperatures. Then, the candidate temperatures are sorted according to fitness, and the mean and step size of the temperature distribution are updated according to the CMA-ES strategy, so that the temperature coefficient gradually converges towards generating a higher quality sampling distribution. Finally, the optimal temperature value is selected, and a new set of sampling points is generated from the candidate point set using its corresponding residual importance sampling probability. This allows PINN to focus on areas with large physical solution errors during training, further improving the overall solution accuracy and convergence speed. After each generation of evolution, more training points can be resampled in the error set based on the physical residuals output by the current network, and added to the training set. The sampling strategy is controlled by a "temperature" hyperparameter τ, which is also optimized by CMA-ES. By optimizing the control point weights or generating new sampling positions, the constraint accuracy of PINN in high error regions is improved.Specifically, an adaptive evolutionary strategy based on the covariance matrix is ​​used to adjust the temperature coefficient to optimize the sampling strategy. This includes: initializing the relevant parameters of the adaptive evolutionary strategy based on the covariance matrix, including the step size, covariance matrix, evolutionary round, and population size; iteratively executing the following steps until the iteration round is reached: using the current temperature coefficient as the mean, and generating a normal distribution of the temperature coefficient for the current round based on the current step size and covariance matrix; randomly selecting multiple temperature coefficients from the normal distribution of the temperature coefficient for the current round, and configuring a sampling strategy for each selected temperature coefficient, wherein the number of selected temperature coefficients is the same as the population size; for each sampling strategy configured in this round, collecting multiple samples from the solution domain. Sample points are used to form a training set corresponding to the sampling strategy; the current physical information neural network system is trained once using the training set corresponding to each sampling strategy to obtain multiple trained physical information neural network systems that correspond one-to-one with the sampling strategy; multiple sampling points are uniformly sampled from the solution domain, and each trained physical information neural network system makes a prediction for each sampling point, and the loss of the trained physical information neural network system is calculated using the current loss function based on the prediction results, wherein the loss of each trained physical information neural network system is the fitness of the temperature coefficient corresponding to the training set used for training; the mean is updated based on all temperature coefficients selected in this round and the fitness of each temperature coefficient.

[0086] According to one embodiment of the present invention, the sampling probability of each pre-divided region in the solution domain is determined in the following manner:

[0087]

[0088] in, This indicates the number of pre-divided regions in the solution domain. and This represents the index of a pre-divided region within the solution domain. and The terms in the solution domain are represented sequentially as follows: The region and the first Each region Indicates the i-th term in the solution domain Sampling probability of each region and The current physical information neural network pairs are represented sequentially. and Loss of sampling points in This represents the temperature coefficient.

[0089] According to one embodiment of the present invention, the mean is updated in the following manner:

[0090]

[0091] in, Indicates the first +1 round average, Indicates the first The average value of the wheel, Indicates the first The loss of the trained physical information neural network system corresponding to the temperature coefficient with the lowest fitness in each iteration. This represents the loss of a physical information neural network system trained using uniform sampling as the sampling strategy. Indicates the first The step length of the wheel.

[0092] According to an embodiment of the present invention, a method for solving partial differential equations in a target domain is proposed. The method includes: obtaining partial differential equations in the target domain; constructing a physical information neural network system for solving the partial differential equations in the target domain using a physical information neural network system construction method based on an evolutionary strategy as described above; and using the constructed physical information neural network system to predict the solution of each sampling point in the solution domain of the partial differential equations in the target domain to solve the partial differential equations in the target domain.

[0093] To demonstrate the beneficial effects of this invention, the inventors verified it by comparing the prediction errors of a physical information neural network system constructed using the evolutionary strategy-based physical information neural network system construction method proposed in this invention with those constructed using a baseline method. The partial differential equations used for verification include a one-dimensional convection equation and a one-dimensional wave equation.

[0094] The following is a brief introduction to the one-dimensional convection equation and the one-dimensional wave equation.

[0095] 1. One-dimensional convection equations

[0096] One-dimensional convection equations are used to describe scalars (e.g., temperature, concentration, or momentum) in a fluid moving at a constant velocity The transport process during motion. This equation is a fundamental equation in fluid mechanics and transport phenomena. The expression for the one-dimensional convection equation is:

[0097]

[0098] The initial conditions are:

[0099]

[0100] The boundary conditions are:

[0101]

[0102] in, It is the convection coefficient. With... As the number of increases, the frequency of its solutions increases, making the approximation of PINNs more difficult. In the experiment, ... The analytical solution to this equation is as follows:

[0103]

[0104] 2. One-dimensional wave equation

[0105] One-dimensional wave equations are a class of hyperbolic partial differential equations used to describe the propagation of waves in one-dimensional space. They are commonly used in physics and engineering to model wave phenomena such as sound waves, seismic waves, and electromagnetic waves. Under periodic boundary conditions, the one-dimensional wave equation can be expressed as:

[0106]

[0107] The initial conditions are:

[0108]

[0109] The boundary conditions are:

[0110]

[0111] in For wave speed. In the experiment, it was set to... The analytical solution to this equation is as follows:

[0112]

[0113] See appendix Figure 2 With appendix Figure 3 , attached Figure 2 With appendix Figure 3 The figure shows experimental data for the one-dimensional convection equation, with appended figures. Figure 2 Figures showing experimental data for a physical information neural network system constructed using the baseline method are attached. Figure 2 Subgraphs (a) and (b) in the figure represent heatmaps of the prediction results and prediction errors of the physical information neural network system constructed using the baseline method in the solution domain. Figure 3 The accompanying figures show experimental data of a physical information neural network system constructed using the evolutionary strategy-based physical information neural network system construction method proposed in this invention. Figure 3 Subgraphs (a) and (b) in the figure represent the prediction results heatmap and prediction error heatmap of the physical information neural network system constructed using the evolutionary strategy-based physical information neural network system construction method proposed in this invention in the solution domain. See Appendix. Figure 4 With appendix Figure 5 , attached Figure 4 With appendix Figure 5 The experimental data for the one-dimensional wave equation are shown in the figure, with appended... Figure 4 Figures showing experimental data for a physical information neural network system constructed using the baseline method are attached. Figure 4 Subgraphs (a) and (b) in the figure represent heatmaps of the prediction results and prediction errors of the physical information neural network system constructed using the baseline method in the solution domain. Figure 5 The accompanying figures show experimental data of a physical information neural network system constructed using the evolutionary strategy-based physical information neural network system construction method proposed in this invention. Figure 5 Subgraphs (a) and (b) in the figure represent heatmaps of the prediction results and prediction errors of the physical information neural network system constructed using the evolutionary strategy-based physical information neural network system construction method proposed in this invention, in the solution domain. (See attached figure...) Figure 2 To be continued Figure 5 Experimental data show that, compared to the physical information neural network system constructed using the evolutionary strategy-based method proposed in this invention, the physical information neural network system constructed using this invention exhibits a smoother residual distribution in the solution domain, a significantly reduced residual extremum, and a markedly smaller residual abnormal region area. This also verifies that the residual-driven sampling and adaptive loss weight strategy proposed in this invention effectively strengthens the physical constraints on the difficult solution regions. The global search mechanism of CMA-ES avoids getting trapped in local optima during training, while importance sampling accurately tilts training resources towards residual abnormal regions. Simultaneously, the adaptive weight mechanism ensures the synchronous convergence of various loss terms. For different types of partial differential equation problems, this invention effectively reduces prediction errors, verifying the generality and robustness of this hybrid optimization strategy.

[0114] In summary, this invention enhances the global search capability of the gradient method by introducing an adaptive evolutionary strategy based on the covariance matrix, replaces the fixed loss weights with dynamic variables adaptively optimized by the evolutionary strategy, and upgrades uniform sampling to optimizable importance sampling based on model residuals. These three elements work together to achieve a two-layer adaptive update of network parameter distribution and physical residual information, effectively avoiding local optima, dynamically balancing physical constraints and initial boundary conditions, and accurately focusing on high residual regions. This significantly improves the training stability, convergence speed, and solution accuracy of PINNs in solving complex nonlinear PDEs.

[0115] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0116] This invention can be a system, method, electronic device, computing device, or computer program product. A computer program product mainly refers to a software product that implements this solution through a computer program.

[0117] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a physical information neural network system based on an evolutionary strategy, wherein the physical information neural network system is used to solve partial differential equations in a target domain, characterized in that, The method includes: S1. Obtain the partial differential equations of the target domain to be solved, as well as its solution domain, initial conditions, and boundary conditions; S2. Obtain the initial physical information of the neural network system and initialize its parameters; S3. Construct an initial sampling strategy and an initial loss function. The initial loss function is a weighted sum of the residual loss, initial condition loss, and boundary condition loss of the partial differential equation to be solved, wherein each loss has the same initial weight. S4. Collect multiple sampling points from the solution domain according to the initial sampling strategy to construct a training set. Based on the constructed training set and the initial loss function, pre-train the physical information neural network system to be trained until the loss of the physical information neural network system is within the preset loss range. S5. Based on the initial loss function and initial sampling strategy, the pre-trained physical information neural network system is subjected to multiple rounds of iterative training. In each round of iterative training, the covariance matrix adaptive evolution strategy is adopted to adjust the weight of each loss term in the loss function in the next round of iterative training with the goal of minimizing the loss. The sampling strategy is optimized with the goal of improving the prediction accuracy of the physical information neural network system for each region in the solution domain.

2. The method according to claim 1, characterized in that, In S4, the pre-training includes multiple iterations, wherein each iteration includes: S41. An initial sampling strategy is adopted to collect multiple sampling points from the solution domain, and based on all the collected sampling points and the initial loss function, the initial physical information neural network system is trained in multiple rounds using gradient descent. S42. Initialize the relevant parameters of the covariance matrix adaptive evolution strategy. The relevant parameters include at least the step size, covariance matrix, evolution round, and population size. Based on the initialized relevant parameters, the system parameters of the initial physical information neural network system trained by S41 are optimized in multiple rounds using the covariance matrix adaptive evolution strategy until the evolution round is reached.

3. The method according to claim 2, characterized in that, In S42, each round of optimization includes: The system parameters of the current initial physical information neural network system are used as the mean, and a normal distribution of the system parameters for this round is generated based on the current step size and covariance matrix. Multiple sets of system parameters are randomly selected from the normal distribution of system parameters in this round, wherein the number of system parameter sets collected is the same as the population size; Based on the current sampling strategy, multiple sampling points are collected from the solution domain, and a physical information neural network system is configured for each group of system parameters in the current round of the population. The neural network system for each physical information in this round uses the configuration to predict the predicted solution for each sampling point collected in this round. Based on all the predicted solutions of the physical information neural network system configured in each round, the loss of the physical information neural network system configured in each round is calculated using the current loss function, and the loss of the physical information neural network system configured in each round is used as the fitness of the system parameters corresponding to the physical information neural network system. Based on all the system parameters selected in this round and the fitness of each group of system parameters, the parameters of the initial physical information neural network system are updated in the following way; in, The system parameters of the neural network system represent the updated initial physical information. Indicates the lowest fitness After the system parameters are sorted in descending order of fitness, the group is located at the [number]th [position]. Bit-level system parameters express The weight coefficients are given by the fact that the sum of the weight coefficients for all system parameters is 1, and the weight coefficient for each group of system parameters is greater than 0.

4. The method according to claim 1, characterized in that, In S5, an adaptive evolutionary strategy based on the covariance matrix is ​​used to optimize the weights of each loss term in the loss function, including: Initialize the parameters of the adaptive evolution strategy for the covariance matrix, including step size, covariance matrix, evolution rounds, and population size; Iteratively execute the following steps until an evolutionary cycle is reached: The mean of the current round is configured based on the weights of each loss in the current loss function, and the loss weight normal distribution of the current round is generated based on the current step size and covariance matrix. Multiple sets of loss weights are randomly selected from the normal distribution of loss weights in the current round, and a loss function is configured for each set of selected loss weights. The number of weight sets selected is the same as the population size. Each set of loss weights includes the weights of partial differential equation residual loss, initial condition loss and boundary condition loss. Based on the current sampling strategy, multiple sampling points are collected from the solution domain. The current physical information neural network system is trained in multiple rounds using all the collected sampling points and each configured loss function to obtain multiple trained physical information neural network systems that correspond one-to-one with the loss function. Based on the current sampling strategy, multiple sampling points are collected from the solution domain. Each trained physical information neural network system is used to predict the predicted solution for each sampling point. Based on all the predicted solutions, the loss of each trained physical information neural network system is calculated using the current loss function. The loss of each trained physical information neural network system is used as the fitness of the loss weight corresponding to that trained physical information neural network system. The weights of the loss function are updated based on all the loss weights selected in this round and the fitness of each set of loss weights.

5. The method according to claim 4, characterized in that, Update the weights of the loss function as follows: in, This represents the mean. Indicates based on the first The loss weights of the loss function of the round are constructed from the first round. The average value of the wheel, Indicates based on the first The loss weights of the loss function of the round are constructed from the first round. The average value of the wheel, Indicates the first The step size of the round iteration, This represents the smallest fitness value in the k-th iteration. After the group loss weights are sorted in descending order of fitness, the one located at the th The loss weight of the bit, express The weighting coefficients, , and The weights of the partial differential equation residual loss, initial condition loss, and boundary condition loss are represented in that order.

6. The method according to claim 1, characterized in that, The sampling strategy is to adjust the sampling probability of each pre-divided region in the solution domain by introducing a temperature coefficient. The larger the temperature coefficient, the higher the sampling probability of the region with greater loss in the solution domain. The initial sampling strategy has a temperature coefficient of 0, and the sampling probability of each region in the solution domain is the same.

7. The method according to claim 6, characterized in that, In S5, an adaptive evolutionary strategy based on the covariance matrix is ​​used to adjust the temperature coefficient to optimize the sampling strategy, including: Initialize the parameters of the adaptive evolution strategy for the covariance matrix, including step size, covariance matrix, evolution rounds, and population size; Iteratively execute the following steps until the iteration round is reached: Using the current temperature coefficient as the mean, and based on the current step size and covariance matrix, generate a normal distribution of the temperature coefficient for the current round; Multiple temperature coefficients are randomly selected from the normal distribution of temperature coefficients in the current round, and a sampling strategy is configured for each selected temperature coefficient. The number of selected temperature coefficients is the same as the population size. For each sampling strategy configured in this round, multiple sampling points are collected from the solution domain to form a training set corresponding to that sampling strategy; The current physical information neural network system is trained once using the training set corresponding to each sampling strategy to obtain multiple trained physical information neural network systems that correspond one-to-one with the sampling strategy. Multiple sampling points are uniformly sampled from the solution domain. Each trained physical information neural network system makes a prediction for each sampling point. The loss of each trained physical information neural network system is calculated using the current loss function based on the prediction results of each trained physical information neural network system. The loss of each trained physical information neural network system is the fitness of the temperature coefficient corresponding to the training set used for training. The mean value is updated based on all temperature coefficients selected in this round and the fitness of each temperature coefficient.

8. The method according to claim 7, characterized in that, Configure the sampling strategy as follows: The sampling probability of each pre-divided region in the solution domain is determined as follows: in, This indicates the number of pre-divided regions in the solution domain. and This represents the index of a pre-divided region within the solution domain. and The terms in the solution domain are represented sequentially as follows: The region and the first Each region Indicates the i-th term in the solution domain Sampling probability of each region and The current physical information neural network pairs are represented sequentially. and Loss of sampling points in the middle, This represents the temperature coefficient.

9. The method according to claim 7, characterized in that, Update the mean as follows: in, Indicates the first The average value of the wheel, Indicates the first The average value of the wheel, Indicates the first The loss of the trained physical information neural network system corresponding to the temperature coefficient with the lowest fitness in each iteration. This represents the loss of a physical information neural network system trained using uniform sampling as the sampling strategy. Indicates the first The step length of the wheel.

10. The method according to claim 1, characterized in that, The target field can be any one of the following: fluid mechanics, materials science, quantum mechanics, and earth science.

11. A method for solving partial differential equations in a target domain, characterized in that, The method includes: Obtain the partial differential equations in the target domain; A physical information neural network system for solving partial differential equations in the target domain is constructed using the method described in any one of claims 1-7. A constructed physical information neural network system is used to predict the solution of the partial differential equation in the target domain at each sampling point in the solution domain, thereby solving the partial differential equation in the target domain.

12. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-11, 12.

13. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method according to any one of claims 1-11, 12.

Images