Data and physics hybrid drive neural network training method

Abstract

The invention discloses a data and physics hybrid drive neural network training method, relates to the technical field of artificial intelligence, and innovatively introduces a sliding window technology to dynamically adjust the weight of a data drive item. According to the method, an adjustable weight is multiplied in front of a data driving item, and a sliding window mechanism is used for dynamic parameter optimization, so that the influence between the data driving item and a physical driving item is balanced, and the requirements of different stages in the neural network training process are better met. The introduction of the sliding window technology in the method can perform feature extraction according to a period, and is especially suitable for a processing scene of time sequence data. By dynamically adjusting the window size, the window step length and the weight parameter, the method not only enhances the adaptability of the model to data change, but also can realize more flexible feature extraction and parameter optimization in different training stages, thereby further improving the prediction precision and convergence speed of the model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data and physics hybrid driven neural network training method. Background Art

[0002] Using neural networks for training to build proxy models as physical field generators can accurately predict physical field solutions under different working conditions, but the data-driven method used still relies on a large number of data samples obtained by pre-calculation of traditional numerical simulations. Existing studies have proposed a physics-driven training method for basic heat conduction and flow prediction problems, which can directly solve high-resolution complex flow fields without the need for data samples. Based on the original neural network physical field generator, the pre-calculated data is replaced by physical laws, partial differential equations and boundary conditions are introduced into the loss function, and the neural network is directly supervised to learn and ultimately accurately solve the physical field. Compared with traditional numerical simulation methods, this physics-driven deep learning method transforms the problem of directly solving the control equation into a loss function residual optimization problem to approach the partial differential equation solution. Multiple numerical experiments have shown that the physics-driven neural network method achieves first-order accuracy.

[0003] Based on the first two methods, a hybrid data and physics-driven method has been proposed to directly predict the physical field. The training process of the neural network model using the data-driven method and the physics-driven method was compared. The results show that the convergence of the two methods is significantly different in terms of the physical field approximation error and the residual of the control equation. Based on this observation, a hybrid data and physics-driven training method (combined data-driven andphysics-driven method) has been proposed to improve the convergence speed and solution accuracy. Through the weighted loss function, the reference sample and the physical equation can drive the model training at the same time. At present, several numerical experiments have been conducted to study the effectiveness of the hybrid method. Compared with the ordinary data-driven method, the introduction of the physical control equation can not only speed up the convergence speed, but also produce solutions that are more consistent with the intrinsic physical laws. Compared with the ordinary physics-driven method, even with rough or erroneous reference samples, this hybrid method can increase the convergence speed by about 50%.

[0004] However, in the current hybrid data and physics-driven training methods, the weights between two different loss function items are fixed values. When the neural network performs backpropagation according to the loss function, the gradient descent is based on the weights of the fixed ratio. In fact, the two loss items have different decline trends and amplitudes as the neural network is trained. If they are based on fixed weights, it will cause insufficient training, affect the training effect and speed, and even ultimately affect the prediction accuracy.

[0005] In the above hybrid data and physics-driven method, the setting of fixed weights limits the flexibility of the model and cannot adapt to the needs of different stages in the training process. In the early stages of training, data-driven terms may provide important initial directions, allowing the model to quickly converge to a reasonable range of solutions; in the later stages of training, physics-driven terms should dominate to ensure the accuracy of the solution under the constraints of physical laws. However, when the weights remain fixed, the model cannot dynamically adjust the influence of the two, which may lead to excessive reliance on data-driven terms in the early stages, and difficulty in fully meeting the high-precision requirements of physics-driven terms in the later stages.

[0006] Therefore, designing a mechanism that can adaptively adjust the weight ratio so that the model can balance data and physical loss terms at different training stages has become an important direction for improving the generalization ability and accuracy of the model. In addition, the adaptive weight mechanism can effectively deal with complex, multi-scale physical problems, dynamically adjust the weights of the two drivers during the training process, so that the model can more accurately follow the inherent physical laws while maintaining efficient convergence, thereby improving the model's predictive stability and adaptability to unknown working conditions. Summary of the invention

[0007] The present invention aims to solve the difference in convergence and accuracy between data-driven and physics-driven methods in existing physical field prediction neural network training. To this end, a hybrid training method combining data-driven and physics-driven is proposed, and the sliding window technology is innovatively introduced to dynamically adjust the weight of the data-driven term. This method balances the influence between data-driven and physics-driven terms by multiplying the data-driven term by an adjustable weight and using a sliding window mechanism for dynamic parameter optimization, thereby better adapting to the requirements of different stages in the neural network training process.

[0008] The introduction of sliding window technology in this method can extract features according to the period, which is particularly suitable for time series data processing scenarios. By dynamically adjusting the window size, window step and weight parameters, this method not only enhances the model's adaptability to data changes, but also enables more flexible feature extraction and parameter optimization at different training stages, further improving the model's prediction accuracy and convergence speed.

[0009] Compared with the traditional fixed weight method, the dynamic weight adjustment strategy of the present invention provides higher flexibility, which greatly enhances the adaptability of the model under different task conditions. At the same time, the adjustment parameters of the sliding window can be regulated according to the specific task requirements, thereby effectively improving the generalization ability and prediction stability of the neural network in a variety of physical scenarios. This design not only overcomes the limitations of traditional methods in accuracy and efficiency, but also provides an innovative solution for neural network modeling of complex physical problems in the future.

[0010] In order to solve the above technical problems, the present invention provides a technical solution: a data and physics hybrid driven neural network training method, comprising the following steps:

[0011] S1. In the data-driven approach, the loss function compares the difference between the training target and the output result as:

[0012]

[0013] Where T is the output temperature value of the grid point, T tar is the target temperature value of the grid point, and N is the number of grid points;

[0014] S2. In the physical drive method, the total physical loss is:

[0015]

[0016] S3. Define the overall error E and Laplace residual LR, respectively:

[0017]

[0018]

[0019] Where T is the output value, T true represents the true solution;

[0020] S4, combining E and LR as loss terms to improve physical consistency and increase convergence speed, using a weighted loss function:

[0021]

[0022] S5. Use a sliding window method to dynamically adjust the hyperparameter R to improve the training efficiency of the neural network. The method for dynamically adjusting the parameters of the sliding window includes the following steps:

[0023] Initialization: set the initial window size and parameter adjustment rules;

[0024] Model training: For each sliding window, the neural network model is trained using the data within the window;

[0025] Model evaluation: Evaluate the performance of the model on the test set and determine the final window parameter settings.

[0026] Furthermore, in step S5, an initial weight, a window value and a weight upper limit are assigned, and the weight is dynamically adjusted so that it does not exceed this upper limit. Thereafter, the total loss is equal to the weight multiplied by the data drive plus the physical drive.

[0027] Furthermore, in step S5, a method for updating the list is written, and two lists are initialized to record the data-driven and physical-driven data. The method for updating the list is called during the training of the main program, and the loss values ​​of the current data-driven and physical-driven are passed in. If the list length is greater than the window size, the first value of the list is deleted to ensure that the list is the same size as the window.

[0028] Furthermore, in step S5, after the sliding window is updated, the method of adjusting the weight is called. This method calculates the average value of the sliding window, so as to obtain the average value of the data-driven loss1 and the physical-driven loss2 within the set window size. Then, the average value of the physical drive is divided by the average value of (the data drive multiplied by the current weight) to obtain a new weight, which is the updated weight.

[0029] Furthermore, in step S5, as the training progresses, the sliding window is continuously updated, and each update calculates the average of the two lists, and then divides them to obtain a new weight, thereby achieving an effect of dynamically adjusting the weight.

[0030] Further, in step S1, the physics-driven method trains the model without labeled data to obtain the solution of the physical field. Based on Fourier's law, when the thermal conductivity is considered to be constant and there is no internal heat source, the physical law of heat conduction is described as the two-dimensional Laplace equation. This physical law drives the learning process through the loss function, which is defined as:

[0031]

[0032] Where N is the number of grid points.

[0033] Furthermore, in the loss function, the differential term is calculated by central difference:

[0034]

[0035] Where h is the distance between adjacent grids. The partial differential operator is constructed through convolution operation, and the central difference of temperature is calculated. The partial differential operators for x and y coordinates are designed respectively, and their weights are expressed as:

[0036] Furthermore, after the rotation and translation operations of the output temperature field, the second-order partial derivatives of the local quantity are calculated, and the central difference is written as:

[0037]

[0038] Among them, k m，n is the convolution kernel, d i，j is the central difference of the temperature values ​​T.

[0039] Furthermore, the boundary conditions of temperature are constrained by Dirichlet boundary conditions, where the temperature of the external and internal boundaries is kept constant;

[0040] The boundary conditions of the external and internal boundaries are implemented differently. The temperature of the external boundary is given a constant value, while the value of the internal boundary and the internal void area is constrained by the temperature loss function:

[0041]

[0042] Where M represents the number of all grid points in the hole, T BC，in is the constant temperature value of the inner boundary.

[0043] The advantages of the present invention over the prior art are that the present invention uses a sliding window to dynamically adjust the weight ratio of data-driven and physical-driven, effectively overcoming the deficiencies in adaptability and accuracy of fixed weights and random weighting methods. In contrast, the sliding window method can flexibly adjust the ratio of data-driven loss1 and physical-driven loss2 according to the convergence of the current model and the error distribution during training, ensuring that the two always maintain a reasonable weight balance at different stages. This dynamic balancing mechanism not only accelerates the convergence process of the model, but also significantly improves the training efficiency and prediction accuracy.

[0044] Since the sliding window technology can adapt to changes in different time scales and data distribution, this method has significant advantages in dealing with complex, multi-scale physical field problems. When achieving the expected prediction accuracy, the number of training steps required for the hybrid training method of the present invention is significantly reduced, thereby greatly saving hardware computing resources and time costs. In addition, the adaptability of the sliding window enables the model to adapt to changes in different working conditions more quickly, and even in the face of complex physical scenarios, it can ensure the stability and accuracy of the solution. Therefore, compared with the traditional weighted method, the present invention not only improves the performance of model training, but also provides an innovative solution for more efficient and resource-friendly physical field neural network modeling. DETAILED DESCRIPTION

[0045] The following is a further detailed description of a data and physics hybrid driven neural network training method of the present invention.

[0046] Working principle of the present invention:

[0047] By combining data-driven and physics-driven methods to train neural networks, and adjusting the weights of data-driven and physics-driven methods through the sliding window method to optimize the training of neural networks, the core of the sliding window dynamic parameter adjustment method is to use sliding windows to capture local patterns and trends in the residual data of the loss function. This method processes data by sliding a window on the data sequence of the loss function value. The data in the window is used to train the model, and the parameters of the window (such as size, step size, etc.) can be dynamically adjusted according to the characteristics of the data.

[0048] The present invention is described in detail.

[0049] In data-driven methods, the loss function compares the difference between the training target and the output result as:

[0050]

[0051] The physics-driven method trains the model without labeled data to obtain the solution of the physical field. Based on Fourier's law, when the thermal conductivity is considered constant and there is no internal heat source, the physical law of heat conduction can be described as the two-dimensional Laplace equation. This physical law drives the learning process through the loss function, which is defined as:

[0052]

[0053] In this loss function, the differential term is calculated by the central difference, and the partial differential operator is constructed through the convolution operation to calculate the central difference of the temperature. The partial differential operators for the x and y coordinates are designed respectively, and their weights are expressed as:

[0054]

[0055] After the rotation and translation operations of the output temperature field, the second-order partial derivatives of the local quantity are calculated, and the central difference can be written as:

[0056]

[0057] The boundary conditions are constrained by Dirichlet boundary conditions, where the temperature of the outer and inner boundaries is kept constant. The boundary conditions for the outer and inner boundaries are implemented differently. The temperature of the outer boundary is given a constant value, while the value of the inner boundary and the inner void region is constrained by a loss function:

[0058]

[0059] Therefore, the total physical loss is:

[0060]

[0061] In order to study the convergence process, the overall error (E) and Laplace residual (LR) are defined based on the original loss of the above method, which are defined as:

[0062]

[0063]

[0064] We propose to improve physical consistency and increase convergence speed by combining (E) and (LR) as loss terms. Because the magnitude difference between the two is too large, using the simple sum of the two errors as the total loss will lead to optimization deviation. To solve this problem, we use a weighted loss function:

[0065]

[0066] Through this weighted loss function, different loss terms can be easily scaled to equivalent magnitudes. The sliding window method can dynamically adjust this hyperparameter R to improve the efficiency of neural network training.

[0067] The method for dynamically adjusting parameters of a sliding window includes the following steps:

[0068] Initialization: Set the initial window size and parameter adjustment rules.

[0069] Model training: For each sliding window, the neural network model is trained using the data within the window.

[0070] Model evaluation: Evaluate the performance of the model on the test set and determine the final window parameter settings.

[0071] The specific process to achieve this function includes:

[0072] (1) Assign an initial weight, a window value, and a weight upper limit. The dynamic adjustment of the weight will not exceed this upper limit. After that, the total loss is equal to the weight multiplied by the data drive plus the physical drive.

[0073] (2) Write a method to update the list and initialize two lists to record the data-driven and physical-driven data. When the main program is training, call the method to update the list and pass in the current data-driven and physical-driven loss values. If the list length is greater than the window size, delete the first value in the list to ensure that the list is the same size as the window.

[0074] (3) After updating the sliding window (i.e., the list), the weight adjustment method is called. This method calculates the average value of the sliding window (list), thereby obtaining the average value of the data-driven loss1 and the physical-driven loss2 within the set window size. The average value of the physical-driven loss is then divided by the average value of (the data-driven loss multiplied by the current weight) to obtain a new weight, which is the updated weight.

[0075] (4) As training progresses, the sliding window is continuously updated (i.e., the method of calling the update list in step (2) and the method of adjusting the weight in step (3) are continuously repeated). Each update calculates the average of the two lists and divides them to obtain the new weight, thereby achieving the effect of dynamically adjusting the weight.

[0076] The specific implementation process of a data and physics hybrid driven neural network training method of the present invention is as follows:

[0077] The present invention relates to a neural network training technology using a generative model that combines data-driven and physical-driven methods, and in particular to a method for improving the efficiency of neural network training by adjusting the ratio between the two through a sliding window. For example, a generative model and a neural network are used to simulate and predict physical fields, and to simulate temperature values ​​at spatial points under constant heat conduction.

[0078] In this embodiment, we give a boundary condition initial condition, and then train the neural network model, the neural network model includes but is not limited to CNN, RNN, MLP, etc., and optimize the loss function of the neural network by combining the sliding window method. The loss function is composed of data-driven loss1 (the error between the output and the target, formula 1) and physical-driven loss2 (Laplace residual formula 2):

[0079] Formula 1:

[0080]

[0081] Formula 2:

[0082]

[0083] The total loss = weight*loss1+loss2. Through this weighted loss function, different loss terms can be scaled to equivalent magnitudes. Then, the weight value is dynamically adjusted through a sliding window to achieve the same number of training iterations. The physical residual and prediction error are both smaller than the original fixed weight method, and finally a temperature field that meets the boundary conditions is generated.

[0084] The present invention and its implementation methods are described above. Such description is not restrictive. If a person skilled in the art is inspired by it and creatively designs structures and implementation methods similar to the technical solution without departing from the purpose of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A data and physics hybrid driven neural network training method, characterized by: The following steps are included: S1. In the data-driven approach, the loss function compares the difference between the training target and the output result as: Where T is the output temperature value of the grid point, T tar is the target temperature value of the grid point, and N is the number of grid points; S2. In the physical drive method, the total physical loss is: S3. Define the overall error E and Laplace residual LR, respectively: Where T is the output value, T true represents the true solution; S4, combining E and LR as loss terms to improve physical consistency and increase convergence speed, using a weighted loss function: S5. Use a sliding window method to dynamically adjust the hyperparameter R to improve the training efficiency of the neural network. The method for dynamically adjusting the parameters of the sliding window includes the following steps: Initialization: set the initial window size and parameter adjustment rules; Model training: For each sliding window, the neural network model is trained using the data within the window; Model evaluation: Evaluate the performance of the model on the test set and determine the final window parameter settings.

2. A data and physics hybrid driven neural network training method according to claim 1, characterized in that: In step S5, an initial weight, a window value and a weight upper limit are assigned, and the dynamically adjusted weight will not exceed this upper limit. Thereafter, the total loss is equal to the weight multiplied by the data drive plus the physical drive.

3. A data and physics hybrid driven neural network training method according to claim 2, characterized in that: In step S5, write a method to update the list and initialize two lists to record the data-driven and physical-driven data. When the main program is trained, call the method to update the list and pass in the current data-driven and physical-driven loss values. If the list length is greater than the window size, delete the first value in the list to ensure that the list is the same size as the window.

4. The data and physics hybrid driven neural network training method according to claim 3 is characterized by: In step S5, after the sliding window is updated, the weight adjustment method is called. This method calculates the average value of the sliding window, so as to obtain the average value of the data-driven loss1 and the physical-driven loss2 within the set window size. Then, the average value of the physical drive is divided by the average value of (data drive multiplied by the current weight) to obtain a new weight, which is the updated weight.

5. A data and physics hybrid driven neural network training method according to claim 4, characterized in that: In step S5, as the training progresses, the sliding window is continuously updated, and each update calculates the average of the two lists, and then divides them to get the new weight, thereby achieving the effect of dynamically adjusting the weight.

6. A data and physics hybrid driven neural network training method according to any one of claims 1 to 5, characterized in that: In step S1, the physics-driven method trains the model without labeled data to obtain the solution of the physical field. Based on Fourier's law, when the thermal conductivity is considered constant and there is no internal heat source, the physical law of heat conduction is described as the two-dimensional Laplace equation. This physical law drives the learning process through the loss function, which is defined as: Where N is the number of grid points.

7. A data and physics hybrid driven neural network training method according to claim 6, characterized in that: In the loss function, the differential term is calculated by the central difference, and the partial differential operator is constructed by the convolution operation to calculate the central difference of the temperature. The partial differential operators for the x and y coordinates are designed respectively, and their weights are expressed as:

8. The data and physics hybrid driven neural network training method according to claim 7 is characterized by: After the rotation and translation operations of the output temperature field, the second-order partial derivatives of the local quantity are calculated, and the central difference is written as: Among them, k m，n is the convolution kernel, d i，j is the central difference of the temperature values ​​T.

9. A data and physics hybrid driven neural network training method according to claim 8, characterized in that: The boundary conditions for temperature are constrained by Dirichlet boundary conditions, where the temperature of the external and internal boundaries is kept constant; The boundary conditions of the external and internal boundaries are implemented differently. The temperature of the external boundary is given a constant value, while the value of the internal boundary and the internal void area is constrained by the temperature loss function: Where M represents the number of all grid points in the hole, T BC，in is the constant temperature value of the inner boundary.

Images