Neural network initialization design method based on meta-learning and embedded physical knowledge

Abstract

The application discloses a neural network initialization design method based on meta-learning and embedding physical knowledge, and is used for satellite component layout board temperature field reconstruction calculation, and comprises the following steps: S1, constructing a neural network embedding physical knowledge; S2, training the neural network by using meta-learning to obtain meta-initialization parameters of the neural network; S3, the neural network loads the meta-initialization parameters, reconstructs the temperature field of a satellite component layout board to be measured by using measuring point temperatures, constructs a loss function by using the measuring point temperatures, and obtains optimal performance parameters by minimizing the loss function, so that an optimal proxy model of the temperature field of the satellite component layout board to be measured is obtained. Therefore, in the method, the satellite component layout board temperature field can be effectively reconstructed and calculated in a short time by using fewer sample parameters and reducing the calculation data cost.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a neural network initialization design method based on meta-learning and embedded physical knowledge, which is used for temperature field reconstruction calculation of satellite component layout board. Background Technology

[0002] A neural network is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks to perform distributed parallel information processing. Relying on the complexity of the system, neural networks achieve information processing by adjusting the interconnections between a large number of internal nodes. With the continuous development of neural network learning technology, neural networks have achieved remarkable results in various fields in recent years, such as natural language processing, computer vision, and genetics. However, how to apply neural networks to scientific computing has remained a challenge.

[0003] Traditional numerical methods, such as the finite element method and the finite difference method, are mesh-based. Mesh creation is a complex geometric problem, and at higher dimensions, traditional numerical methods often suffer from the curse of dimensionality, making them unsolvable. Recent research has successfully utilized neural networks to solve differential equations. Among them, neural networks with embedded physical knowledge serve as a general framework for solving both forward and inverse differential equation problems. By encoding physical information into a loss function and then optimizing it, these neural networks establish a surrogate model of the solution, enabling prediction of the entire solution domain. Compared to traditional numerical methods, neural networks with embedded physical knowledge are meshless, avoiding the complex and tedious process of mesh creation and the potential dimensionality explosion problem. This saves solution time and computational costs, and allows for accurate prediction of all points within the entire domain.

[0004] However, when expensive data is not readily available, neural networks, as a data-driven method with embedded physical knowledge, struggle to achieve fast and accurate predictions, sometimes failing even after extensive training, thus increasing computational costs. For example, a satellite layout board requires multiple components, each of which can be considered a heat source. In practical use, uneven heat source distribution can affect the performance of the satellite layout board and, more seriously, may pose safety hazards. However, the heat distribution on the satellite layout board cannot be easily and accurately obtained. Therefore, a method is needed to reconstruct the temperature field of the satellite component layout board. Summary of the Invention

[0005] To at least partially solve the technical problems existing in the prior art, the present invention provides a neural network initialization design method based on meta-learning and embedded physical knowledge.

[0006] The technical solution of the present invention is as follows:

[0007] A neural network initialization design method based on meta-learning and embedded physical knowledge, the method being used for temperature field reconstruction calculation of satellite component layout plates, the method comprising:

[0008] Step S1: Construct a neural network with embedded physical knowledge;

[0009] Step S2: Train the neural network using meta-learning to obtain the meta-initialization parameters of the neural network;

[0010] Step S3: The neural network loads the meta-initialization parameters, reconstructs the temperature field of the satellite component layout board under test using the measured temperature, constructs a loss function using the measured temperature, and obtains the optimal performance parameters by minimizing the loss function through training, thereby obtaining the optimal surrogate model of the temperature field of the satellite component layout board under test.

[0011] Optionally, in the method, the temperature of the satellite component layout board under test satisfies the following control equation:

[0012]

[0013] Wherein, the boundary is: T = T0 or or

[0014] In the above formula, φ represents the intensity distribution, k represents the thermal conductivity coefficient within the region, T0 represents the temperature value of the isothermal boundary or convection boundary, and the temperature values ​​of some measuring points are obtained in advance before performing the method.

[0015] Optionally, the intensity distribution φ exists within the distribution range [φ min ,φ max ]Inside,

[0016] Step S2 includes randomly sampling within the distribution range to obtain a test value φ. i This is treated as higher-order prior information, and a loss function with embedded physical knowledge is constructed using each sample:

[0017] in,

[0018] or

[0019] in, and Both represent weight parameters.

[0020] Optionally, the equation obtained from each sampling can be considered as a task.

[0021] Step S2 includes updating the network parameters M times using the stochastic gradient descent algorithm under the same task, and then finding the optimal performance parameters under N tasks. This parameter is the meta-initialization parameter of the neural network.

[0022] Optionally, step S3 includes constructing a loss function using the temperature at the measuring point as follows:

[0023] in,

[0024]

[0025]

[0026] or

[0027] Then, minimize the loss through training. To obtain the optimal performance parameter θ ** Thus, the optimal surrogate model for the entire temperature field is obtained.

[0028] Optionally, in step S3, a stochastic gradient descent or adaptive momentum estimation algorithm is used to adjust the loss function. Optimization is performed to find the optimal performance parameter θ within the set iteration period N. ** .

[0029] Optionally, step S2 includes training the neural network to find the best performance parameter θ for each task. * ,in,

[0030] Under the same task, using SGD or Adam algorithms to update the neural network parameters θ, after the k-th update, it can be achieved... Description; Under different tasks, the neural network parameters θ are... The direction is updated, and the update step size ∈ adopts a linear strategy with steps ∈ = ∈ 0(1-k / N). The update method can be described as follows:

[0031] Optionally, in step S1, the depth of the neural network is D, D≥3, and initialization parameters are given to the neural network. The neural network includes an input layer, an output layer, and D-2 hidden layers.

[0032] Optionally, the neural network is represented as N θ (·), using a multilayer perceptron with an activation function, such that:

[0033] in,

[0034]

[0035]

[0036]

[0037] The main advantages of the technical solution of this invention are as follows:

[0038] This invention presents a neural network initialization design method based on meta-learning and embedded physical knowledge. It acquires the intensity distribution on the layout plate of the satellite component under test by using the control equation for the temperature of the layout plate. Then, it obtains the optimal performance parameters of the neural network through meta-learning, reconstructs the temperature field of the entire region using the measured temperature points, constructs a loss function for the associated neural network, and obtains the optimal performance parameters by minimizing the loss function, thereby acquiring the optimal surrogate model for the temperature field of the layout plate of the satellite component under test. This method can effectively reconstruct the temperature field of the satellite component layout plate in a shorter time using fewer sample parameters and reducing computational data costs.

[0039] Furthermore, based on the general method of this invention, three different meta-initialization parameters are obtained by learning from two types of prior distributions (zero-order distribution and higher-order distribution). After loading the meta-initialization parameters into the neural network with embedded physical knowledge, faster training speed and higher convergence accuracy can be achieved. The designed meta-initialization not only significantly accelerates the forward problem, i.e., solving differential equations, but also significantly accelerates the inverse problem of solving differential equations using small samples, i.e., recovering unknown parameters in the control equations. This greatly reduces the computational data cost and solves the problem of long computational training time, making it feasible to solve the control equations and predict unknown parameters using a small number of real training samples within the error range. Attached Figure Description

[0040] The accompanying drawings, which are provided to further illustrate embodiments of the invention and constitute a part of this invention, are illustrative of the invention and their descriptions serve to explain the invention, but do not constitute an undue limitation of the invention.

[0041] In the attached diagram:

[0042] Figure 1 This is a schematic diagram of the structure of a satellite component layout board according to one embodiment of the present invention;

[0043] Figure 2 This is a simplified flowchart illustrating a method according to one embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the reconstructed temperature field of the satellite component layout plate obtained by calculation in this embodiment;

[0045] Figure 4 This is a flowchart of the meta-learning initialization training process of a neural network initialization design method according to an embodiment of the present invention.

[0046] Figure 5 This is a schematic diagram of a neural network model with embedded physical knowledge according to one embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] Meta-learning is a learning method that learns how to learn and is widely used in few-shot learning. The widely used Reptile algorithm is a meta-initialization method that can quickly iterate and solve new tasks using a small number of samples after learning a task.

[0049] For neural networks that currently mostly use random or zero initialization of embedded physical knowledge, if meta-learning algorithms can be applied to neural networks with embedded physical knowledge, it can not only solve the dependence on training samples, but also achieve fast and accurate prediction of solutions to differential equations, greatly saving computational costs.

[0050] In this embodiment, meta-learning is applied to the initialization of neural networks with embedded physical knowledge. Meta-initialization is used to accelerate the solution of differential equations with a small number of training samples. This method is particularly useful for the temperature field reconstruction calculation of satellite component layout plates.

[0051] To more clearly illustrate the solution of the present invention, the following will take the calculation of the temperature field reconstruction of the satellite component layout plate as an example for detailed explanation.

[0052] In this embodiment, such as Figure 2 As shown, the method includes:

[0053] Step S1: Construct a neural network with embedded physical knowledge;

[0054] Step S2: Train the neural network using meta-learning to obtain the meta-initialization parameters of the neural network;

[0055] Step S3: The neural network loads the meta-initialization parameters, reconstructs the temperature field of the satellite component layout board under test using the measured temperature, constructs a loss function using the measured temperature, and obtains the optimal performance parameters by minimizing the loss function through training, thereby obtaining the optimal surrogate model of the temperature field of the satellite component layout board under test.

[0056] Specifically, the method in this embodiment, when solving the temperature field reconstruction calculation of the satellite component layout plate, such as... Figure 1 As shown, the satellite component layout board under test has multiple components distributed on it, and each component can be regarded as a heat source. The temperature of the satellite component layout board under test satisfies the following governing equation:

[0057]

[0058] Wherein, the boundary: T = T0 or or

[0059] In the above formula, φ represents the intensity distribution, k represents the thermal conductivity coefficient within the region, T0 represents the temperature value of the isothermal boundary or convection boundary, and the temperature values ​​of some measuring points are obtained in advance before performing the method.

[0060] It can be understood that the intensity distribution φ exists within the distribution range [φ min ,φ max ]Inside,

[0061] Step S2 includes randomly sampling within the distribution range to obtain a test value φ. i This is treated as higher-order prior information, and a loss function with embedded physical knowledge is constructed using each sample:

[0062] in,

[0063] or

[0064] in, and Both represent weight parameters.

[0065] Furthermore, when solving nonlinear differential equations in related or other fields, there are general methods based on the method described in this embodiment. Specifically, such general methods may include:

[0066] Step S1: Construct a neural network with embedded physical knowledge;

[0067] Step S2: Train the neural network using meta-learning to obtain the meta-initialization parameters of the neural network;

[0068] Step S3: The neural network loads the meta-initialization parameters and solves the differential equation.

[0069] In this general method, the parameterized nonlinear differential equation to be solved can be described as:

[0070]

[0071] When solving this equation, information about the zero-order or higher-order distribution can be obtained based on expert prior knowledge or scheme design.

[0072] The zeroth-order distribution contains the distribution information to be solved, and can be described as follows:

[0073] u(x,t)~p1(τ),x∈Ω,t∈[0,T].

[0074] Higher-order distributions contain higher-order information to be solved and can be described as follows:

[0075]

[0076] Both zero-order and higher-order distributions can serve as learning tasks for meta-learning algorithms. This invention extends the meta-learning algorithm Reptile from the original supervised learning method to supervised, unsupervised, and semi-supervised learning methods. Then, the extended New Reptile algorithm is applied to the initialization of neural networks with embedded physical knowledge, thereby improving the speed and accuracy of neural networks in solving differential equations.

[0077] Returning to the method of this embodiment, step S1 includes constructing a neural network of depth D, where D≥3, denoted as N. θ (·), and gives the neural network initialization parameters. The neural network consists of one input layer, one output layer, and D-2 hidden layers, which is a... arrive The mapping. Using a multilayer perceptron with an activation function, then:

[0078] in,

[0079]

[0080]

[0081]

[0082] Furthermore, in the method of this embodiment, each equation obtained from sampling is regarded as a task, and step S2 includes:

[0083] Under the same task, the network parameters are updated M times using the stochastic gradient descent algorithm, and then the optimal performance parameters are found for N tasks. This parameter is the meta-initialization parameter of the neural network.

[0084] Furthermore, step S2 includes training the neural network to find the best performance parameter θ for each task. * ,in,

[0085] Under the same task, using SGD or Adam algorithms to update the neural network parameters θ, after the k-th update, it can be achieved... Description; Under different tasks, the neural network parameters θ are... The direction is updated, and the update step size ∈ uses a linear strategy step. The update method can be described as follows:

[0086] In the general method based on this embodiment, when solving nonlinear differential equations in related or other fields, step S2 may include:

[0087] First, we determine the number of sampling learning tasks as L, and the task types for meta-learning training. When only zero-order prior distribution information is available, supervised learning is used for training, sampling the learning task τ from the task distribution p1(τ). k Then from task τ k Medium-sampled training points are used to establish a loss function based on the actual values ​​of the training points and the predictions of the neural network. When only high-order prior distribution information is available, an unsupervised learning method is used for training, sampling the learning task τ from the task distribution p2(τ). k Then from task τ k Mid-sampling is used to construct the loss function from training points based on PDE conditions and boundary conditions. When information on both the zero-order and higher-order distributions is available, a semi-supervised learning method is used for training. Task αL is selected for supervised learning, and the remaining tasks are used for unsupervised learning. The loss function for each specific task is established as described above.

[0088] Next, the neural network is trained to find the best performance parameter θ for each task. * Among them, under the same task, using SGD or Adam algorithms to update the neural network parameters θ, the k-th update can be achieved through... Description; Under different tasks, the neural network parameters θ are... The direction is updated, and the update step size ∈ adopts a linear strategy with steps ∈ = ∈ 0(1-k / N). The update method can be described as follows: Specific training methods such as Figure 3 As shown.

[0089] Returning to the method of this embodiment, step S3 includes constructing a loss function using the temperature at the measuring point as follows:

[0090] in,

[0091]

[0092]

[0093] or

[0094] Then, minimize the loss through training. To obtain the optimal performance parameter θ ** Thus, the optimal surrogate model for the entire temperature field is obtained.

[0095] Furthermore, in step S3, a stochastic gradient descent or adaptive momentum estimation algorithm is used to adjust the loss function. Optimization is performed to find the optimal performance parameter θ within the set iteration period N. ** .

[0096] For example, such as Figure 4 As shown in the figure, this figure is a schematic diagram of the temperature field of the reconstructed satellite component layout plate obtained by the method of this embodiment in a specific embodiment.

[0097] In the general method based on this embodiment, when solving nonlinear differential equations in related or other fields,

[0098] In step S3, the neural network will be loaded with meta-initialization parameters. The neural network needs to be constrained to meet physical limitations, primarily by constructing a loss function that incorporates embedded physical knowledge. loss function Including partial differential structure loss Boundary condition loss Initial condition loss and truth conditional loss The loss function can be expressed as:

[0099] in,

[0100]

[0101]

[0102]

[0103]

[0104] In the above formula, w f ,w i ,w b and w d These are weight parameters. and This represents the set of data points from the PDE, initial values, boundary values, and actual values.

[0105] At this time, in step S3, as Figure 5 As shown, stochastic gradient descent (SGD) or adaptive momentum estimation (Adam) algorithms are used to optimize the loss function. Optimize the system to find the optimal performance parameters within the set iteration period N.

[0106] After inputting the relevant parameters, the differential equation can be solved.

[0107] Therefore, in the neural network initialization design method of this embodiment, the intensity distribution on the layout plate of the satellite component under test is collected through the control equation of the temperature of the layout plate. Then, the optimal performance parameters of the neural network are obtained through meta-learning, and the temperature field of the entire region is reconstructed using the temperature of the measurement points. Subsequently, a loss function of the neural network is constructed, and the optimal performance parameters are obtained by minimizing the loss function, thereby obtaining the optimal surrogate model of the temperature field of the layout plate of the satellite component under test. In this method, the temperature field of the satellite component layout plate can be effectively reconstructed and calculated in a shorter time with fewer sample parameters and reduced computational data costs.

[0108] Furthermore, based on the general method of this invention, the meta-learning Reptile algorithm is extended and applied by combining the characteristics of neural networks with embedded physical knowledge. It is extended from supervised learning to supervised, unsupervised and semi-supervised algorithms, thus expanding the application scope and conditions of the meta-learning Reptile algorithm. At the same time, the meta-learning algorithm New Reptile is used to perform meta-initialization on neural networks with embedded physical knowledge, so as to accelerate the solution of forward and inverse differential equation problems and improve the prediction accuracy of neural networks using only a small number of expensive real training samples.

[0109] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Additionally, the terms "front," "back," "left," "right," "upper," and "lower" in this document refer to the placement shown in the accompanying drawings.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A neural network initialization design method based on meta-learning and embedded physical knowledge, characterized in that, The method is used for reconstructing the temperature field of a satellite component layout board. The method includes: Step S1: Construct a neural network with embedded physical knowledge; Step S2: Train the neural network using meta-learning to obtain the meta-initialization parameters of the neural network; Step S3: The neural network loads the meta-initialization parameters, reconstructs the temperature field of the satellite component layout board under test using the measurement point temperature, and constructs a loss function using the measurement point temperature. The optimal performance parameters are obtained by minimizing the loss function through training, thereby obtaining the optimal surrogate model of the temperature field of the satellite component layout board under test. In the method, the temperature of the satellite component layout board under test satisfies the following control equation: ； Wherein, the boundary: or or ; In the above formula, Indicates intensity distribution, Indicates the thermal conductivity coefficient within the region. The temperature value represents the isothermal boundary or convection boundary, and the temperature values ​​of some measuring points are obtained in advance before the method is executed; The intensity distribution Existing in the distribution range Inside, Step S2 includes randomly sampling within the distribution range to obtain a test value. This is treated as higher-order prior information, and a loss function with embedded physical knowledge is constructed using each sample: ,in, , or , in, and Both represent weight parameters.

2. The neural network initialization design method based on meta-learning and embedded physical knowledge according to claim 1, characterized in that, The equation obtained from each sampling is considered as a task. Step S2 includes updating the network parameters M times using the stochastic gradient descent algorithm under the same task, and then finding the optimal performance parameters under N tasks. This parameter is the meta-initialization parameter of the neural network.

3. The neural network initialization design method based on meta-learning and embedded physical knowledge according to claim 2, characterized in that, Step S3 includes constructing the following loss function using the temperature at the measuring point: ,in, ， ， or , Then, minimize the loss through training. Obtain the optimal performance parameters Thus, the optimal surrogate model for the entire temperature field is obtained.

4. The neural network initialization design method based on meta-learning and embedded physical knowledge according to claim 2, characterized in that, In step S3, stochastic gradient descent or adaptive momentum estimation algorithms are used to adjust the loss function. Optimize within the set iteration cycle. Find the optimal performance parameters .

5. The neural network initialization design method based on meta-learning and embedded physical knowledge according to claim 2, characterized in that, Step S2 includes training the neural network to find the best performance parameters for each task. ,in, Under the same task, either SGD or Adam algorithm can be used to update the neural network parameters. , No. After the update, it will be possible to... Description; Neural network parameters under different tasks by Update the direction and update the step size. Using a linear strategy step The update method can be described as follows: .

6. The neural network initialization design method based on meta-learning and embedded physical knowledge according to claim 1, characterized in that, In step S1, the depth of the neural network is D, D≥3, and initialization parameters are given to the neural network. The neural network includes an input layer, an output layer and D-2 hidden layers.

7. The neural network initialization design method based on meta-learning and embedded physical knowledge according to claim 6, characterized in that, The neural network is represented as follows: By using a multilayer perceptron with an activation function, we can achieve the following: ,in, ； ； 。

Images