A method and apparatus for tuning an improved network for a physical neural network

Abstract

The application provides a method and device for optimizing an improved network of a physical neural network. The method comprises adding an input scaling layer, an output scaling layer and a feature layer to the basic physical neural network to construct an improved physical neural network; using a plurality of observation points and the improved physical neural network to make a difference as a first loss function to perform first training on the improved physical neural network; and adding a residual term of the differential equation set to the first loss function to obtain a second loss function to perform second training. The method realizes the feasibility of using a physical neural network to solve a strongly rigid rate theory equation set, adds observation points as a supervised training term and performs pre-training to determine the optimization direction of the model, so that the model training has higher training efficiency. An optimization method for balancing each residual weight is adopted, so that the improved network has better regularization effect.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of machine learning, and in particular to a method and apparatus for tuning an improved network for physical neural networks. BACKGROUND

[0002] Currently, physical neural networks are a deep learning method that uses the residual of a differential equation set simulating a physical process as a loss term for neural network training, while ensuring the optimization direction and regularization, which can train a neural network model to realize the functions of forward predicting the solution of the differential equation set and inversely predicting the unknown parameters in the differential equation set. However, this technology has a problem: it is difficult for physical neural networks to correctly predict a class of differential equation sets with rigid maximum and a wide range of orders of magnitude covered by the solution.

[0003] Rate theory is an important method for simulating the microstructure evolution of material long-term irradiation damage, which can generally simulate the evolution process of various types of defects at the mesoscopic scale. However, the point defect mobile model based on this theory is a rigid ill-conditioned differential equation set composed of a large number of equations, and it is time-consuming to solve the rigid ill-conditioned equation set using numerical calculation methods, and it is not feasible to use traditional physical neural networks for training and prediction. SUMMARY

[0004] The present application describes a method and apparatus for tuning an improved network for physical neural networks, which can solve the above technical problems.

[0005] According to a first aspect, a method for tuning an improved network for physical neural networks is provided. The method comprises:

[0006] Modeling a physical process to obtain a system of differential equations, determining initial boundary values of the system of differential equations, physical parameters of the physical process, and values of a plurality of observation points; writing residuals of each term of the system of differential equations into a first loss function of a basic physical neural network constructed, training the basic physical neural network using the first loss function; adding an input scaling layer, an output scaling layer, and a feature layer to the basic physical neural network to construct an improved physical neural network; using a plurality of observation points and the improved physical neural network to perform a first round of training in pre-training of the improved physical neural network using a second loss function; after the first round of training in pre-training ends, obtaining values of loss terms of the second loss function and setting weights of the loss terms so that the loss terms are balanced to a first order of magnitude; using the second loss function to perform subsequent rounds of training in the pre-training until a preset ending condition is met; adding residual terms of the system of differential equations to the second loss function to obtain a third loss function for a first round of training in second training; after the first round of training in the second training ends, obtaining values of each loss term in the third loss function, and setting weights again so that the loss terms are balanced to a second order of magnitude; using the third loss function to perform subsequent rounds of training in the second training until the preset ending condition is met.

[0007] In one embodiment, the input scaling layer includes applying a first linear scaling function to t, i.e., t' = t / T, so that the order of t is O(1).

[0008] In one embodiment, the output scaling layer includes adding a second linear scaling function applied to the output of the neural network after the fully connected layer.

[0009] In one embodiment, the feature layer includes determining a function l(t) that makes the predicted solution satisfy the boundary conditions and adding a trend function trend(t) of the change of the predicted solution over time.

[0010] In one embodiment, adding the input scaling layer, the output scaling layer, and the feature layer to the basic physical neural network to construct the improved physical neural network includes adding by combining the input range of the system of differential equations, the output order of magnitude, the boundary conditions, and the change trend of the solution.

[0011] In one embodiment, after the first round of training, obtaining values of the second loss function and setting weights of loss terms of the second loss function so that the loss terms are balanced to a first order of magnitude includes obtaining loss function values of each term after the first round of training, and taking the inverse of the first loss function value as the weight.

[0012] In one embodiment, the second training ends to obtain a physical neural network optimization model, and using the physical neural network optimization model to make a prediction obtains an accurate differential equation solution.

[0013] In one embodiment, the second training ends to obtain a physical neural network optimization model, and using the physical neural network optimization model to make a prediction obtains an accurate differential equation solution.

[0014] According to a second aspect, an apparatus for tuning an improved network of a physical neural network is provided. The apparatus comprises:

[0015] A differential equation set establishing module is configured to model a physical process to obtain a differential equation set, determine initial boundary values of the differential equation set, physical parameters of the physical process, and values of a plurality of observation points.

[0016] A network training module is configured to write residuals of each term of the differential equation set into a first loss function of a constructed basic physical neural network, and train the basic physical neural network using the first loss function.

[0017] A network improvement module is configured to add an input scaling layer, an output scaling layer, and a feature layer to the basic physical neural network to construct an improved physical neural network.

[0018] A first training module is configured to use the plurality of observation points and the improved physical neural network to make a difference as a second loss function to perform a first round of training in pre-training of the improved physical neural network; after the first round of training in the pre-training ends, a value of a loss term of the second loss function is obtained, and a weight of the loss term is set to balance the loss term to a first order of magnitude; and the second loss function is used to perform subsequent rounds of training in the pre-training until a preset ending condition is met.

[0019] A second training module is configured to add residual terms of the differential equation set to the second loss function to obtain a third loss function to perform a first round of training in second training; after the first round of training in the second training ends, a value of each loss term in the third loss function is obtained, and the weight of the loss term is set again to balance the loss term to a second order of magnitude; and the second loss function is used to perform subsequent rounds of training in the second training until a preset ending condition is met.

[0020] In one embodiment, the network improvement module is specifically configured to:

[0021] The input scaling layer comprises: applying a first linear scaling function to t, i.e., t' = t / T, so that the order of magnitude of t is O(1).

[0022] The output scaling layer comprises adding a second linear scaling function to the output of the neural network after the fully connected layer.

[0023] The feature layer comprises determining a function l(t) that makes the predicted solution satisfy the boundary condition and adding a trend function trend(t) of the predicted solution over time.

[0024] In one embodiment, the network improvement module comprises adding by combining the input range, output order of magnitude, boundary condition and variation trend of the solution of the differential equation set.

[0025] In one embodiment, the first training module is specifically configured to: perform the first round of training in pre-training to obtain a loss function value of each item, and take the inverse of the first loss function value as a weight.

[0026] In one embodiment, the device further comprises a training module, which is specifically configured to: input predicted time data into the improved physical neural network model obtained after the second training, obtain a predicted physical parameter corresponding to the time data, and output the predicted physical parameter.

[0027] In the above method and device provided by the embodiments of the present specification, the feasibility of using a physical neural network to solve a strongly rigid rate theory equation set is realized by adding three special layers to the basic physical neural network. The observation points are added as supervised training items and pre-training is performed to determine the optimization direction of the model, so that the model training has higher training efficiency. The optimization method of balancing each residual weight is adopted to make the improved network have better regularization effect. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0029] Figure 1 A flowchart of an optimization method of an improved network for a physical neural network provided by an embodiment of the present specification is shown.

[0030] Figure 2 A basic physical neural network structure provided by an embodiment of the present specification is shown.

[0031] Figure 3 An improved physical neural network structure provided by an embodiment of the present specification is shown.

[0032] Figure 4A flowchart of a prediction method using the improved physical neural network model is shown in the embodiments of the present specification.

[0033] Figure 5 A structure diagram of an optimization device for the improved network of the physical neural network is shown in the embodiments of the present specification. DETAILED DESCRIPTION

[0034] The schemes provided in the present specification will be described below with reference to the drawings.

[0035] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0036] In the description of the embodiments of the present application, the words such as "exemplary", "for example", or "for instance" are used to mean serving as an example, instance or illustration. Any embodiment or design solution described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. In fact, the words such as "exemplary", "for example" or "for instance" are used merely to present related concepts in a specific manner.

[0037] In the description of the embodiments of the present application, the term "and / or" is merely used to describe an association relationship of associated objects, and can represent three relationships, for example, A and / or B, which can represent three cases of existence of A alone, existence of B alone, and existence of A and B simultaneously. In addition, unless otherwise specified, the term "multiple" means two or more.

[0038] In addition, the terms "first", "second" are used for description purposes only, and should not be interpreted or implied to indicate or suggest relative importance or implicitly indicate the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0039] Figure 1 A flowchart of an optimization method for the improved network of the physical neural network is shown in the embodiments of the present specification, as shown in the figure, the method comprises the following steps: Figure 1

[0040] In step S110, a differential equation set is obtained by modeling the physical process, and the initial boundary value of the differential equation set, the physical parameter of the physical process and the value of the observation point are determined.

[0041] In one embodiment, a plurality of partial differential equation sets are obtained by modeling the defect evolution process.​

[0042] In some embodiments, the step of determining the physical parameters of the plurality of differential equations is performed by determining the physical parameters of the plurality of differential equations.

[0043] In step S120, the residual of each term of the differential equations is written into a first loss function of the constructed basic physics-informed neural network, and the basic physics-informed neural network is trained using the first loss function.

[0044] The physics-informed neural network (PINN) adds a physical model to the traditional loss function to add physical information to the network. Without changing the back propagation, the PINN is trained by adding the initial control condition, the boundary condition and the physical balance equation to the traditional mean squared error (MSE) loss, so that it can be used to directly solve partial differential equations.

[0045] The PINN combines the advantages of data-driven machine learning and physical models, and can train a model that automatically satisfies physical constraints under the condition of a small amount of training data. The PINN has better generalization performance while ensuring accuracy and predicting important physical parameters of the model.

[0046] After the input time, the function is first approximated by a fully connected neural network, then the partial differential equation residual and the initial boundary value residual constraint are calculated using automatic differentiation technology, and are put into the loss function as a regularization term. Finally, the neural network connection weight parameters and the partial differential equation physical parameters are obtained by using the gradient descent method and other optimization algorithms.

[0047] The loss function is a mathematical function that measures the error between the predicted value and the true value. It is a key factor in deep learning, which can help evaluate the model and adjust the model parameters. Selecting the appropriate loss function can improve the performance of the model and help solve complex problems.

[0048] In one embodiment, the residual of each term of the established differential equations is added to the first loss function of the basic physics-informed neural network, and the neural network is trained.

[0049] In step S130, an input scaling layer, an output scaling layer and a feature layer are added to the basic physics-informed neural network to construct an improved physics-informed neural network.

[0050] The structure of the physics-informed neural network is the same as that of the traditional fully connected neural network, which is composed of multiple layers of fully connected neural networks. Figure 2 The structure of the basic physics-informed neural network provided by the embodiments of the present specification is shown as follows: Figure 2As shown, a nonlinear function is inserted between the neural networks of each layer to achieve the purpose of fitting a nonlinear model. The neurons of each layer are connected to the neurons of the previous layer, and the weights of each layer are optimized using the backpropagation algorithm. The loss function with physical information is the core of the physical information neural network structure. The loss function is the most important part of the backpropagation algorithm, which can be used to measure whether the neural network is well trained.

[0051] NN(t; θ) is the neural network, N(t; θ) is the output of the neural network, and the ordinary differential equation (ODE) system receives the output C1...C n In the ODE system, according to the received data, the loss function Loss is constructed by the residual terms of the differential equation system and the residual terms of the initial condition terms.

[0052] In one embodiment, the initial value of the differential equation system and the value of an observation point can be calculated to obtain the input range, output order of magnitude, boundary condition and solution trend of the differential equation system.

[0053] Figure 3 The improved physical neural network structure provided by the embodiments of the present specification is shown as follows: Figure 3 As shown, the improved physical neural network includes an input scaling layer, an output scaling layer, a feature layer and the like to construct the improved physical neural network.

[0054] On the basis of the basic physical neural network structure, the improved physical neural network is constructed. Using the improved physical neural network, it is necessary to complete the construction of the scaling scale, boundary condition and equation solution trend in the three-layer neural network newly added based on the physical neural network according to the input mathematical model.

[0055] 1) Add an input scaling layer, add a first linear scaling function 1 / T between the input time t and the neuron layer, so that t is fixed at an order of magnitude.

[0056] In one embodiment, the order of magnitude of t is O(1).

[0057] Since the input time t can be very long (for example, 1e8 seconds), that is, the irradiation defect evolution simulation time is long, the linear scaling function is applied to t, that is, t' = t / T, so that the order of magnitude of t is O(1).

[0058] In one embodiment, T can be taken as the maximum value of the time domain.

[0059] 2) Add an output scaling layer, add a second linear scaling function k n between the output layer of the neural network and the full differential layer to ensure the accuracy of the prediction result and speed up the training speed.

[0060] Because the neural network outputs C1, C2,..., C n with different sizes and large differences from each other, it will lead to the inaccuracy of the training results, and even unable to obtain the prediction results, and the traditional prediction time is long. Therefore, similar to the input scaling layer, another linear scaling function is added to the output of the neural network after the full connection layer. That is, C'1, C'2,..., C' n = k'1 * C1, k'2 * C2,..., k' n * C n . Wherein, k1, k2,..., k n are obtained from the mean value of the estimated solution C1, C2,..., C n , which can also be obtained by observation, and generally take its order of magnitude unit.

[0061] In one embodiment, the order of magnitude of the predicted solution of C1 is 10 20 , and the value of k1 can be 10 20 .

[0062] 3) Add a feature layer: add a feature layer between the output scaling layer and the input of the ODE system.

[0063] Specifically divided into two parts:

[0064] ① The function l(t) that makes the predicted solution satisfy the boundary condition:

[0065] Because the initial value of the ODE system is accurately known, fixing the initial value of the neural network prediction solution u(x; θ u ) as a known value can increase the accuracy and speed of neural network training.

[0066] First, the ODE system has boundary conditions:

[0067] u(t) = g(t), t ∈ Γ BC (1)

[0068] In order to make the predicted solution u(x; θ u ) satisfy the known initial time boundary condition, we construct u(t; θ) = g(t) + l(t)N(r; θ),

[0069] Where N(t; θ) is the output of the neural network, and the function l(x) satisfies the following two conditions:

[0070]

[0071] Γ BCFor the boundary condition of the ODE system, in this paper, it is c(T0) = c0. Therefore, l(x) can be selected as l(t) = t - T0, Such functions have the following characteristics:

[0072] When t = T0, l(x) = 0, at this time the predicted solution must be g(t), that is, the initial value c0.

[0073] ②Add the trend function trend(t) of the predicted solution over time:

[0074] In the present application, the linear trend of the solution can be obtained by the observation value of the initial value point T0 and another time point T1, and the linear trend value is constructed as follows:

[0075] trend(t) = k * t + b, (3)

[0076] Where the values of k and b can be calculated by the slope and intercept of the defect density values of the two time points T0 and T1. In the characteristic layer, the predicted solution of the neural network after the output scaling layer is added to trend(t), so that the predicted solution changes based on trend(t), avoiding the convergence difficulty caused by the large order of magnitude change of the solution, and the problem of large order of magnitude difference between different solutions.

[0077] In some embodiments, the following is selected:

[0078] l(t) = tanh(t - T0) * tanh(T1 - t), (4)

[0079] And reconstruct:

[0080] u(t; theta) = trend(t) + l(t) * N(t; theta), (5)

[0081] Finally, the construction of three special layers is completed. By improving the physical neural network, three special layers are added to the network, and the feasibility of using the physical neural network to solve the strongly stiff rate theory equation set is realized.

[0082] Step S140, using several observation points and the difference between the improved physical neural network as the second loss function to perform the first round of training in the pre-training of the improved physical neural network.

[0083] In one embodiment, the number densities of n kinds of defects corresponding to several time points are selected, and the difference between the n number densities and the output value of the improved physical neural network is used.

[0084] In one embodiment, the several observation points are a small number of observation points, and in a specific embodiment, 1-2 observation points can be used. The efficiency of model training is improved, the observation points are added as supervised training items for pre-training in the first training, the optimization direction of the model is determined, and the model training has higher training efficiency.

[0085] In one embodiment, the gradient descent algorithm is an adaptive momentum optimization method (Adam), and the learning rate is 0.02.

[0086] In one embodiment, the activation function is a swish function.

[0087] Step S150, after the first round of training in pre-training, the value of the second loss function loss term is obtained, and the weight of the loss term is set to balance the loss term to the first order of magnitude.

[0088] In one embodiment, the weight of the loss term is set to the inverse of each loss term.

[0089] In one embodiment, the loss term is balanced to O(1) to improve the model convergence speed and regularization effect, to ensure that the rate theory equation set with strong rigidity, large numerical changes, and wide coverage of the order of magnitude can be accurately predicted by the trained model.

[0090] Step S160, using the second loss function for subsequent rounds of training in pre-training until the preset end condition is met.

[0091] In one embodiment, the end condition is that the model test error is reduced to 1e -1 Below, or the training reaches 20000 steps.

[0092] Step S170, the residual term of the differential equation set is added to the second loss function to obtain a third loss function for second training.

[0093] The residual term of each differential equation obtained after pre-training is added to the second loss function to obtain a third loss function, and the second training is started based on the third loss function.

[0094] Step S180, after the first round of training in the second training, the value of each loss term in the third loss function is obtained, and the weight is set again to balance the loss term to the second order of magnitude.

[0095] In one embodiment, when training with the residual term, only one training is performed first to obtain the loss function value of each term, and then the inverse is taken as the weight, so that each loss function is in the same order of magnitude O(1), that is, the neural network training has better regularization effect and convergence efficiency.

[0096] In the neural network training process, it is noticed that the residuals calculated by each equation are quite different from each other, however, the residuals that are too different are used as the loss function of the neural network at the same time, which controls the convergence direction. In order to make the prediction result of the neural network meet the result of each equation, the loss functions of each equation can be balanced.

[0097] Balancing each loss function with weight: the specific method is to first train a model for one round, obtain the loss function values obtained by the residuals of each equation, take the inverse of these values as the loss function weight, and multiply it into the loss function formula of the neural network, so that each loss function value is balanced at the beginning of training, which can make the model converge faster and more accurately.

[0098] In one embodiment, after one second training is performed, a second loss function is obtained, and the weight of each loss term in the second loss function is set to balance the value of the second loss function to the same order of magnitude.

[0099] In one embodiment, the weight of the loss term is set to the inverse of each loss term.

[0100] In one embodiment, the loss term is balanced to O(1), which improves the model convergence speed and regularization effect.

[0101] Step S190, using the third loss function to perform subsequent rounds of second training until a preset end condition is met.

[0102] In one embodiment, the end condition is that the model test error is reduced to 1e -3 The following.

[0103] The above is the training process of the improved physical neural network, after the model training is completed, the model is used for prediction.

[0104] The evolution process of defects generated by irradiation damage of RAFM steel material is simulated, and the scheme of the application is described.

[0105] (1) Determine the mathematical model of the process to be simulated

[0106] In one embodiment, the differential equations of ODE are determined, and a point defect mobile model based on rate theory is used.

[0107] The evolution process of defects generated by irradiation damage of RAFM (low-activation ferrite martensite steel) steel material is simulated, and first the mathematical model describing the evolution process needs to be determined. In the embodiment, a point defect mobile model based on rate theory is used. Each equation of the model describes a type of defect evolution constraint, and the solution C1~C Nis a function of time as the independent variable and the number density of the ith type of defect (i = 1, 2, …, N) as the dependent variable. In addition, the relevant physical parameters of RAFM steel material need to be determined as parameters in the differential equation.

[0108] In one embodiment, the room temperature yield strength is 541 MPa, the tensile strength is 668 MPa, the elongation is 25%, the toughness impact energy is 200 J, the 600°C high temperature yield strength is 294 MPa, the tensile strength is 321 MPa, and the elongation is 29%.

[0109] (2) Establish an improved physical neural network

[0110] After completing the part that needs to be constructed in the physical neural network, according to the differential equations obtained in 1), the boundary conditions, initial values, input range to be simulated, estimated order of magnitude of the solution, and the change trend of the solution calculated according to the initial value and a single observation point are determined to construct the newly added input scaling layer, output scaling layer and feature layer. To ensure that the rate theory equation set with strong rigidity, numerical change and wide coverage of order of magnitude can be accurately predicted by the trained model.

[0111] (3) Use two optimization strategies to train the neural network

[0112] First, use a small number of observation points and the difference between the neural network outputs as a loss function item to improve the training efficiency, which is used in pre-training. Second, after the pre-training is completed, the equation residual is added to the training, and the weight of each loss term is set at the end of the first round of training, and the value is the inverse of each loss term, so that each loss term is in the same order of magnitude O(1), which improves the model convergence speed and regularization effect.

[0113] In the point defect mobile model example using rate theory, a small amount of numerical method calculation results can be used as a reference solution for prediction, or a small amount of experimental observation values can be used as a reference solution for prediction.

[0114] After the model training, a usable neural network model is finally obtained, which can receive the time point or other dependent variable of the simulated irradiation defect evolution as input and predict the accurate differential equation solution in a very short time. By providing an improved network and tuning method for predicting rate theory equation solutions based on physical neural networks, an improved physical neural network is trained using deep learning to accurately predict rate theory differential equation sets with rigidity and ill-conditioning.

[0115] Figure 4 The flowchart for predicting using the improved physical neural network model provided by the embodiments of the present specification is shown in FIG. 1. Figure 4As shown, taking the evolution process of defects generated by irradiation damage of RAFM steel material as an example, the method comprises the following steps:

[0116] 1) Load the improved physical neural network model. The model is a trained model.

[0117] 2) Input the predicted time point into the model.

[0118] 3) The model quickly predicts the density values of all defects at the time point and outputs, and draws a chart.

[0119] It can be understood that the present application can also be trained on other physical processes and predict the solutions of the system of differential equations.

[0120] Corresponding to the above method provided by the present application, the present application also provides a device. Figure 5 A structure schematic diagram of an optimization device for an improved network of a physical neural network provided by an embodiment of the present application is shown. As shown in the figure, Figure 5 The device 500 comprises:

[0121] A system of differential equations establishing module 510 is configured to model a physical process to obtain a system of differential equations, determine the initial boundary value of the system of differential equations, the physical parameters of the physical process, and the values of a plurality of observation points.

[0122] A network training module 520 is configured to write the residual error of each term of the system of differential equations into a first loss function of a constructed basic physical neural network, and train the basic physical neural network using the first loss function.

[0123] A network improvement module 530 is configured to add an input scaling layer, an output scaling layer, and a feature layer to the basic physical neural network to construct an improved physical neural network.

[0124] A first training module 540 is configured to use a plurality of observation points and the improved physical neural network to perform a first round of training in pre-training of the improved physical neural network using the difference as a second loss function; after the first round of training in pre-training ends, the value of the loss term of the second loss function is obtained and the weight of the loss term is set so that the loss term is balanced to the first order of magnitude; the second loss function is used to perform subsequent rounds of training in the pre-training until a preset end condition is met.

[0125] The second training module 550 is configured to add the residual term of the system of differential equations to the second loss function to obtain a third loss function for first round training in the second training. After the first round training in the second training ends, values of each loss term in the third loss function are obtained, and weights are set again to balance the loss terms to the second order of magnitude. The second loss function is used for subsequent round training in the second training until a preset end condition is met.

[0126] In one embodiment, the network improvement module 530 comprises:

[0127] The input scaling layer comprises: applying a first linear scaling function to t, i.e., t' = t / T, so that the order of t is O(1).

[0128] The output scaling layer comprises: adding a second linear scaling function applied to the output of the neural network after the fully connected layer.

[0129] The feature layer comprises: determining a function l(t) that makes the predicted solution satisfy the boundary condition and adding a trend function trend(t) of the change of the predicted solution over time.

[0130] In one embodiment, the first round training for pre-training obtains the loss function value of each term, and the inverse of the first loss function value is taken as the weight.

[0131] In one embodiment, the apparatus further comprises a training module 560, which is specifically configured to: use the improved physical neural network model obtained after the second training to input the predicted time data into the model to obtain the predicted physical parameters corresponding to the time data and output.

[0132] In one embodiment, the network improvement module 530 is specifically configured to add by combining the input range, output order of magnitude, boundary condition and change trend of the solution of the system of differential equations.

[0133] It should be noted that the description of the apparatus in Figure 5 may also refer to the description of the foregoing method.

[0134] According to another aspect of the embodiment, a computer readable storage medium is also provided, which stores a computer program, when the computer program is executed in a computer, the computer program causes the computer to execute the method described in combination with Figure 1 .

[0135] According to another aspect of the embodiment, a computer readable storage medium is also provided, which stores a computer program, when the computer program is executed in a computer, the computer program causes the computer to execute the method described in combination with Figure 1Those skilled in the art should be able to understand that the functions described in the above one or more examples can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium.

[0136] The above detailed description sets forth the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above detailed description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A method for optimizing an improved physical neural network, characterized in that, The method includes: A system of differential equations is obtained by modeling the physical process, and the initial and boundary values ​​of the system of differential equations, the physical parameters of the physical process, and the values ​​of several observation points are determined; the physical process is the evolution process of defects caused by radiation damage to the material. The residual of each term in the system of differential equations is written into the first loss function of the constructed basic physical neural network, and the basic physical neural network is trained using the first loss function. An improved physical neural network is constructed by adding an input scaling layer, an output scaling layer, and a feature layer to the basic physical neural network. The improved physical neural network is trained in the first round of pre-training by using the difference between several observation points and the improved physical neural network as a second loss function; the several observation points are the number density of n types of defects corresponding to several time points. After the first round of training in the pre-training, the value of the loss term of the second loss function is obtained and the weight of the loss term is set so that the loss term is balanced to the first order of magnitude; The second loss function is used to train subsequent rounds of the pre-training until the preset termination condition is met; The residual terms of the differential equation system are added to the second loss function to obtain the third loss function for the first round of training in the second training; After the first round of training in the second training, the value of each loss term in the third loss function is obtained, and the weights are set again to balance the loss terms to the second order of magnitude. The third loss function is used to train subsequent rounds of the second training until the preset termination condition is met; after the second training is completed, an improved physical neural network model is obtained, the predicted time point is input into the improved physical neural network model, and the density values ​​of various defects caused by the irradiation damage of the material at the predicted time point are obtained and output.

2. The method according to claim 1, characterized in that, The input scaling layer includes: applying a first linear scaling function to... ,Right now , making The order of magnitude is , The output scaling layer includes: adding a second linear scaling function applied to the output of the neural network after the fully connected layer. The feature layer includes: determining a function that ensures the predicted solution necessarily satisfies the boundary conditions. And add a trend function that predicts the solution over time. .

3. The method according to claim 1, characterized in that, The physical neural network based on this is improved by adding an input scaling layer, an output scaling layer, and a feature layer, including by incorporating the input range, output order of magnitude, boundary conditions, and solution variation trends of the differential equation system.

4. The method according to claim 1, characterized in that, After the first round of training in the pre-training is completed, the value of the loss term of the second loss function is obtained and the weight of the loss term is set so that the loss term is balanced to the first order of magnitude, including: performing the first round of training in the pre-training to obtain the loss function value of each term, and taking the reciprocal of the second loss function value as the weight.

5. The method according to claim 1, characterized in that, After the second training is completed, a physical neural network optimization model is obtained. The physical neural network optimization model is used to make predictions to obtain accurate solutions to differential equations.

6. A tuning device for an improved physical neural network, characterized in that, The device includes: The differential equation system establishment module is configured to model a physical process to obtain a differential equation system, and to determine the initial and boundary values ​​of the differential equation system, the physical parameters of the physical process, and the values ​​of several observation points; the physical process is the evolution process of defects caused by radiation damage to the material. The network training module is configured to write the residual of each term of the differential equation system into the first loss function of the constructed basic physical neural network, and use the first loss function to train the basic physical neural network; The network improvement module is configured to add an input scaling layer, an output scaling layer, and a feature layer to the basic physical neural network to construct an improved physical neural network. The first training module is configured to use the difference between several observation points and the improved physical neural network as a second loss function to perform the first round of pre-training on the improved physical neural network; the several observation points are the number density of n types of defects corresponding to several time points; after the first round of pre-training is completed, the value of the loss term of the second loss function is obtained and the weight of the loss term is set so that the loss term is balanced to a first order of magnitude; the second loss function is used to perform subsequent rounds of training in the pre-training until a preset termination condition is met. The second training module is configured to add the residual terms of the differential equation system to the second loss function to obtain a third loss function for the first round of training in the second training; after the first round of training in the second training is completed, the value of each loss term in the third loss function is obtained, and the weights are set again to balance the loss terms to the second order of magnitude; the second loss function is used to train subsequent rounds in the second training until the preset termination condition is met; after the second training is completed, an improved physical neural network model is obtained, the predicted time point is input into the improved physical neural network model, and the density values ​​of various defects caused by irradiation damage to the material at the predicted time point are obtained and output.

7. The apparatus according to claim 6, characterized in that, The input scaling layer includes: applying a first linear scaling function to... ,Right now , making The order of magnitude is ; The output scaling layer includes: adding a second linear scaling function applied to the output of the neural network after passing through the fully connected layer; The feature layer includes: determining a function that ensures the predicted solution necessarily satisfies the boundary conditions. And add a trend function that predicts the solution over time. .

8. The apparatus according to claim 6, characterized in that, The network improvement module is specifically configured to be added by combining the input range, output order of magnitude, boundary conditions, and solution trends of the differential equation system.

9. The apparatus according to claim 6, characterized in that, The first round of pre-training obtains the loss function value for each item, and the reciprocal of the first loss function value is taken as the weight.

10. The apparatus according to claim 6, characterized in that, It also includes a training module, which is specifically configured to: use the improved physical neural network model obtained after the second training to input the predicted time data into the model, obtain the predicted physical parameters corresponding to the time data, and output them.

Images