Training methods and apparatus, data processing methods, electronic devices and storage media

Abstract

A training method, apparatus, data processing method, electronic device, and storage medium are disclosed. The training processing method includes: acquiring at least one set of training input parameters; inputting the at least one set of training input parameters into a neural network for computation to obtain at least one set of first output parameters corresponding to the at least one set of training input parameters; acquiring at least one set of second output parameters corresponding to the at least one set of first output parameters, wherein the at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters respectively; based on the at least one set of second output parameters, acquiring the loss value of the neural network for the at least one set of first output parameters calculated by the loss function corresponding to the neural network; and adjusting the parameters of the neural network based on the loss value. This training method reduces the search space of the optimizer, eliminates the need for the optimizer to find the optimal solution to satisfy the initial and boundary conditions, improves training efficiency, and accelerates training convergence.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a neural network training method, a data processing method, a neural network training apparatus, a data processing apparatus, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] With the improvement of GPU capabilities, the software and hardware ecosystem supporting deep learning has developed rapidly. Using deep learning to solve scientific computing problems has become a trend, and methods for solving differential equations using deep learning are gradually emerging. Of particular note is a method called Physics Informed Neural Network (PINN), which has injected new vitality into the field of scientific computing. The core idea of ​​PINN is to use neural networks to represent the solutions to differential equations, and to continuously approximate numerical solutions by training the neural network, thus bringing some advantages that differ from traditional numerical methods. Summary of the Invention

[0003] This disclosure provides at least one embodiment of a method for training a neural network, comprising: acquiring at least one set of training input parameters; inputting the at least one set of training input parameters into the neural network for computational processing to obtain at least one set of first output parameters corresponding to the at least one set of training input parameters; acquiring at least one set of second output parameters corresponding to the at least one set of first output parameters, wherein the at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters respectively; based on the at least one set of second output parameters, acquiring the loss value of the neural network for the at least one set of first output parameters calculated by a loss function corresponding to the neural network; and adjusting the parameters of the neural network based on the loss value.

[0004] For example, in a neural network training method provided in at least one embodiment of this disclosure, the neural network is trained to solve differential equations that describe the motion and / or state of a target system.

[0005] For example, in a neural network training method provided in at least one embodiment of this disclosure, obtaining at least one set of second output parameters corresponding to the at least one set of first output parameters includes: obtaining boundary condition constraints and initial condition constraints corresponding to the target system; determining a set of intermediate output parameters corresponding to any set of first output parameters, wherein when a set of training input parameters corresponding to any set of first output parameters indicates that the set of training input parameters is located at the boundary of a preset region, the corresponding set of intermediate output parameters is set to the value specified by the boundary condition constraints; and determining a set of second output parameters corresponding to any set of first output parameters based on the corresponding set of intermediate output parameters and the initial condition constraints.

[0006] For example, in a neural network training method provided in at least one embodiment of this disclosure, determining a set of intermediate output parameters corresponding to any set of first output parameters includes: obtaining the corresponding set of intermediate output parameters according to the following formula: in, Let (p, v) represent the corresponding set of intermediate output parameters, (p, v) represent any set of training input parameters, p is in vector form, τ represents the time corresponding to p; G(p) represents the boundary condition constraint; the distance between p in any set of training input parameters and the boundary of the preset region is positively correlated with Q(p), and Q(p) = 0 when p is located at the boundary of the preset region; ⊙ represents the Hadamard product operation; x(p, τ; θ) represents a set of first output parameters corresponding to any set of training input parameters.

[0007] For example, in a neural network training method provided in at least one embodiment of this disclosure, each set of training input parameters includes a time sub-parameter. Determining a set of second output parameters corresponding to any set of first output parameters based on the corresponding set of intermediate output parameters and the initial condition constraints includes: when the time sub-parameter in any set of training input parameters indicates an initial time, setting the corresponding set of second output parameters to the value specified by the initial condition; when the time sub-parameter in any set of training input parameters indicates a time range from the initial time, determining the corresponding set of second output parameters based on the corresponding set of intermediate output parameters and the initial condition constraints; and when the time sub-parameter in any set of training input parameters indicates a time range after the preset time range, using the corresponding set of intermediate output parameters as the corresponding set of second output parameters.

[0008] For example, in a neural network training method provided in at least one embodiment of this disclosure, the corresponding set of second output parameters is obtained according to the following formula: in, This represents the corresponding set of second output parameters; T(τ;k) is a transition function used to adjust the preset time range through the transition parameter k, and T(τ;k) satisfies the following two conditions: x0(p) represents the initial condition constraint.

[0009] For example, in a neural network training method provided in at least one embodiment of this disclosure, the transition parameter k is positively correlated with the length of the preset time range.

[0010] For example, in a neural network training method provided in at least one embodiment of this disclosure, obtaining the loss value of the neural network for the at least one set of first output parameters, calculated by the loss function corresponding to the neural network, based on the at least one set of second output parameters, includes: obtaining the residual result corresponding to the at least one set of second output parameters according to the at least one set of second output parameters and the differential equation; and obtaining the loss value of the neural network according to the residual result corresponding to the at least one set of second output parameters.

[0011] For example, in a neural network training method provided in at least one embodiment of this disclosure, the loss function corresponding to the neural network is expressed as: in, Let θ represent the loss function, θ represent the parameters of the neural network, R(*) represent the residual calculation, and (p,τ) represent any set of training input parameters from the at least one set of training input parameters. Let M represent a set of second output parameters corresponding to any set of training input parameters, M represent the number of differential equations corresponding to the target system, and i be a positive integer. λ represents the accumulation operation. i This represents the residual weighting coefficient corresponding to the i-th differential equation. This represents the residual result calculated based on the i-th differential equation.

[0012] For example, in a neural network training method provided in at least one embodiment of this disclosure, obtaining the loss value of the neural network for the at least one set of first output parameters, calculated by the loss function corresponding to the neural network, based on the at least one set of second output parameters, includes: calculating the residual result corresponding to the at least one set of second output parameters according to the at least one set of second output parameters and the differential equation; calculating the data fitting loss value according to the output data and the label data, wherein the output data is the output obtained by inputting training data into the neural network, and the label data is the standard value corresponding to the training data; and obtaining the loss value of the neural network according to the residual result corresponding to the at least one set of second output parameters and the data fitting loss value.

[0013] For example, in a neural network training method provided in at least one embodiment of this disclosure, the neural network is a physical guided neural network.

[0014] For example, in a neural network training method provided in at least one embodiment of this disclosure, the target system is a fluid dynamics system, and the differential equation is obtained by transforming a partial differential equation from an Euler perspective, wherein the differential equation describes the fluid dynamics system from a Lagrange perspective.

[0015] For example, in a neural network training method provided in at least one embodiment of this disclosure, obtaining the at least one set of training input parameters includes: obtaining at least one set of training input parameters from the Lagrange perspective.

[0016] At least one embodiment of this disclosure provides a data processing method, comprising: acquiring at least one set of input parameters corresponding to a target system; inputting the at least one set of input parameters into a neural network to obtain at least one set of intermediate output results; performing transformation processing on the at least one set of intermediate output results to obtain at least one set of final output results corresponding one-to-one with the at least one set of intermediate output results, wherein the at least one set of final output results is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of intermediate output results respectively; wherein the neural network is at least partially trained according to the training method described in any embodiment of this disclosure.

[0017] This disclosure provides at least one embodiment of a neural network training apparatus, comprising: an acquisition unit configured to acquire at least one set of training input parameters; a computation processing unit configured to input the at least one set of training input parameters into the neural network for computation processing to obtain at least one set of first output parameters corresponding to the at least one set of training input parameters; a constraint processing unit configured to acquire at least one set of second output parameters corresponding to the at least one set of first output parameters, wherein the at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters respectively; a loss value calculation unit configured to acquire, based on the at least one set of second output parameters, a loss value of the neural network calculated by a loss function corresponding to the neural network for the at least one set of first output parameters; and an adjustment unit configured to adjust the parameters of the neural network based on the loss value.

[0018] At least one embodiment of this disclosure provides a data processing apparatus, including: an input acquisition unit configured to acquire at least one set of input parameters corresponding to a target system; and a processing unit configured to input the at least one set of input parameters into a neural network to obtain at least one set of output parameters; wherein the neural network is at least partially trained according to the training method described in any embodiment of this disclosure.

[0019] At least one embodiment of this disclosure provides an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to run the computer-executable instructions, wherein the computer-executable instructions, when run by the processor, implement a neural network training method or a data processing method according to any embodiment of this disclosure.

[0020] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement a neural network training method or a data processing method according to any embodiment of this disclosure. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0022] Figure 1 A schematic flowchart illustrating a data processing method provided for at least one embodiment of this disclosure;

[0023] Figure 2 This is a schematic diagram of a two-dimensional fluid simulation space provided in an embodiment of the present disclosure;

[0024] Figure 3 A structural diagram of a neural network provided in an embodiment of this disclosure;

[0025] Figure 4A and Figure 4B A comparison diagram of simulation results for a free liquid surface simulation provided in an embodiment of this disclosure;

[0026] Figures 5A to 5D A comparison diagram of simulation results for free liquid surface simulation provided in another embodiment of this disclosure;

[0027] Figure 6 A schematic flowchart illustrating a data processing method provided for at least one embodiment of this disclosure;

[0028] Figure 7 A schematic block diagram of a training device provided for at least one embodiment of this disclosure;

[0029] Figure 8 A schematic block diagram of a data processing apparatus provided in at least one embodiment of this disclosure;

[0030] Figure 9 A schematic diagram of an electronic device provided for at least one embodiment of this disclosure;

[0031] Figure 10 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of this disclosure;

[0032] Figure 11 This is a schematic diagram of a hardware environment provided for at least one embodiment of the present disclosure. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0034] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0035] With the development of science and technology, differential equations have evolved into an independent discipline, and physical laws are constantly appearing in the form of differential equations. Examples include the transport of matter, the propagation of sound in the air, the diffusion of heat, electromagnetic wave communication, the flow of water and air, and the matter-wave equation in quantum mechanics—the Schrödinger equation. Successes in physics have extended the theory and techniques of differential equations to a wide range of scientific fields, including chemistry, biology, and economics. Therefore, it is undeniable that differential equations play an extremely important role in science and engineering.

[0036] The conventional approach to solving problems using differential equations is as follows: first, analyze the problem; then, model the problem using differential equations; next, collect observational data and configure the key parameters and conditions of the model (such as initial conditions and boundary conditions of the differential equations); finally, solve the differential equations mathematically, usually by discretizing them in some way to convert them into a form that can be computed by a computer, and then obtain the solution.

[0037] Currently, solving differential equations has developed into a vast field, with many numerical calculation methods emerging, such as the finite difference method, the finite volume method, the finite element method, and so on.

[0038] The development of deep learning has injected new vitality into traditional methods. Compared to traditional numerical methods, physics-guided neural networks are a meshless method, and their advantages are:

[0039] 1) Better adaptability, enabling full utilization of collected data;

[0040] 2) Easy to implement, it can be easily implemented using existing deep learning toolchains and makes better use of GPUs (Graphics Processing Units) to perform parallel computing;

[0041] 3) Wider applicability, such as applicable to integral differential equations, stochastic partial differential equations, fractional partial differential equations, and even solving some mathematically indeterminate inverse problems.

[0042] For example, taking fluid surface simulation as an example, the fluid surface, also known as the free surface, is the interface between different fluids. A common example is the water surface, which is the boundary between water and air. Common fluid surface problems include the sloshing of fluids in containers during transportation, the movement of ocean waves, and dam failures.

[0043] Traditional numerical simulation methods in fluid dynamics can be broadly categorized into two types: mesh-based and mesh-free. Specifically, mesh-based algorithms include finite difference and finite element methods, while mesh-free algorithms include methods with smoothed particles. Mesh-based methods are suitable for calculations from an Eulerian perspective, while mesh-free methods are more commonly used for calculations from a Lagrangian perspective because the positions of fluid particles can severely distort the mesh. Generally, mesh-based methods can more accurately resolve multi-scale phenomena in flow, such as turbulence; while mesh-free methods can better calculate particle position information, such as at gas-liquid interfaces and shock waves.

[0044] Due to the rapid development of deep learning in recent years, fluid simulation methods based on neural networks have also gradually emerged. These methods utilize the universal approximation theorem of neural networks (neural networks can be used to approximate arbitrarily complex functions and can achieve arbitrarily high accuracy) to fit fluid observation data and learn physics from the data to perform simulations and predictions. Among them, physics-guided neural networks based on prior physical knowledge are a novel fluid simulation algorithm. Unlike other neural network algorithms, physics-guided neural network methods can use a system of partial differential equations describing fluid physics to constrain the neural network, thereby simulating fluids and making predictions, even without observational data.

[0045] Physics-guided neural networks (PHNs) have become a popular meshless method for computational fluid dynamics due to their ability to directly fit equations describing physics without the need for observational data. Compared to the iterative algorithms of traditional numerical methods, PHNs directly fit the entire spatial and temporal domains using neural networks. Unlike traditional methods that can only use approximate derivatives, PHNs use exact derivatives within an automatic differentiation framework, thus enabling more accurate calculations, especially for simulating nonlinear terms. Furthermore, the final result calculated by PHNs is a continuous function, allowing for numerical values ​​at any location without interpolation errors. This provides the advantage of being able to arbitrarily change the simulation resolution without recalculation.

[0046] The basic idea of ​​the physics-guided neural network method is to represent the neural network to be solved, transform the initial conditions, boundary conditions and partial differential equations (systems) constraints into the loss function of the neural network, and optimize the solution through the gradient descent method, so that the solution meets the physical constraint requirements.

[0047] However, the inability to properly satisfy the boundary and initial conditions of current physics-guided neural network methods can lead to significant computational errors. For example, the loss function of a physics-guided neural network can be expressed as:

[0048]

[0049] Here, L represents the loss function, R1 represents the residual of the partial differential equation (system), R2 represents the residual corresponding to the boundary conditions, R3 represents the residual corresponding to the initial conditions, and α and β are weighting coefficients used to help optimize the convergence of the loss function.

[0050] R2 and R3 correspond to data fitting, while R1 corresponds to the regularization of the neural network. This can cause uneven gradient descent of the loss function and excessively rapid changes (rigidity), which in turn affects the convergence of the entire optimization process and leads to poor final results.

[0051] For example, when using a physics-guided neural network to simulate fluids with a liquid surface, the physics-guided neural network method cannot meet the free surface condition (e.g., the pressure is zero on the free surface), resulting in poor simulation performance of fluids with a liquid surface.

[0052] Furthermore, current physics-guided neural network methods typically simulate fluids from an Eulerian perspective. However, Eulerian-based fluid simulations only provide velocity field information; calculating position requires additional computation, which cannot currently be directly implemented within the physics-guided neural network framework. Moreover, current physics-guided neural network methods require setting multiple initial and boundary points to simulate initial and boundary conditions. Initial points are used to calculate R², and boundary points are used to calculate R³. This computation consumes more resources and hinders model scalability.

[0053] This disclosure provides at least one embodiment of a neural network training method and apparatus, a data processing method and apparatus, an electronic device, and a storage medium. The neural network training method includes: acquiring at least one set of training input parameters; inputting the at least one set of training input parameters into the neural network for computational processing to obtain at least one set of first output parameters corresponding to the at least one set of training input parameters; acquiring at least one set of second output parameters corresponding to the at least one set of first output parameters, wherein the at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters respectively; based on the at least one set of second output parameters, acquiring the loss value of the neural network for the at least one set of first output parameters calculated by the loss function corresponding to the neural network; and adjusting the parameters of the neural network based on the loss value.

[0054] This training method forcibly adds initial and boundary constraints to the output of the neural network during the training phase, so that the final output automatically satisfies the initial and boundary conditions. The loss value is calculated using the output that satisfies the initial and boundary conditions to optimize the neural network, thereby reducing the optimizer's search space. The optimizer does not need to find the optimal solution to satisfy the initial and boundary conditions, which improves the accuracy of the neural network results, speeds up training, and accelerates the convergence of the loss function. In addition, it does not require setting multiple initial and boundary points, which improves the running efficiency, reduces resource consumption, and enhances the scalability of the model.

[0055] The neural network training method provided in at least one embodiment of this disclosure can be applied to the neural network training apparatus provided in at least one embodiment of this disclosure, which can be configured on an electronic device. The electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a hardware device such as a mobile phone or tablet computer.

[0056] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0057] Figure 1 This is a schematic flowchart illustrating a neural network training method provided for at least one embodiment of the present disclosure.

[0058] like Figure 1 As shown, at least one embodiment of the present disclosure provides a neural network training method including steps S10 to S50.

[0059] In step S10, at least one set of training input parameters is obtained.

[0060] In step S20, at least one set of training input parameters are input into the neural network for computation and processing to obtain at least one set of first output parameters corresponding to the at least one set of training input parameters.

[0061] In step S30, at least one set of second output parameters corresponding to at least one set of first output parameters are obtained.

[0062] For example, at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to at least one set of first output parameters respectively.

[0063] In step S40, based on at least one set of second output parameters, the loss value of the neural network calculated by the loss function corresponding to the neural network for at least one set of first output parameters is obtained.

[0064] In step S50, the parameters of the neural network are adjusted based on the loss value.

[0065] For example, repeat steps S10-S50 until the loss value meets the predetermined convergence condition, at which point the trained neural network is obtained.

[0066] For example, neural networks are trained to solve differential equations, meaning that a well-trained neural network can characterize differential equations and directly output the solution to be solved, which satisfies the physical constraints of the differential equations.

[0067] For example, a differential equation may include a single differential equation or a system of differential equations. Differential equations may include integral differential equations, stochastic partial differential equations, fractional partial differential equations, etc. For example, a well-trained neural network can represent a system of partial differential equations (PDEs). This disclosure does not impose any specific limitations on this.

[0068] For example, differential equations can describe the motion or state of a target system. For example, the target system can be a physical system, a chemical system, a biological system, etc. For example, differential equations can characterize the motion of matter in the target system, the state changes of the target system, etc. For example, referring to the above description, differential equations can be used to describe the propagation of sound in air, the diffusion of heat, the flow process of fluids, etc. This disclosure does not impose specific limitations on differential equations or target systems; any phenomenon that can be simulated using differential equations can be applied to the neural network training method provided in at least one embodiment of this disclosure.

[0069] For example, the neural network can be a physics-guided neural network. Alternatively, it can employ other neural network structures based on prior knowledge such as physics. This disclosure does not impose specific limitations on the type of neural network. For instance, the network structure of a physics-guided neural network can be a fully connected network. Of course, any other feasible network structure can also be used, and this disclosure does not impose specific limitations in this regard.

[0070] For example, in step S10, a set of training input parameters may include one or more input sub-parameters. The input sub-parameters can be determined based on the independent variables of the differential equation.

[0071] For example, when the target system is a two-dimensional fluid dynamics system, if a set of training input parameters is the training input parameters under the Euler perspective, then a set of training input parameters can be represented as: (i,j,t), where the input sub-parameter i represents the abscissa of the fluid particle, the input sub-parameter j represents the ordinate of the fluid particle, and the input sub-parameter t represents time, that is, the time sub-parameter.

[0072] For example, when the target system is a two-dimensional fluid dynamics system, if a set of training input parameters is the training input parameters under the Lagrange perspective, then a set of training input parameters can be represented as: (a,b,t), where the input sub-parameter a represents the label x-coordinate of the fluid particle, the input sub-parameter b represents the label y-coordinate of the fluid particle, and the input sub-parameter t represents time, that is, the time sub-parameter.

[0073] For example, a set of first output parameters and a set of second output parameters include multiple output sub-parameters, which are determined according to the dependent variable of the differential equation, that is, the numerical solution to be found in the differential equation.

[0074] For example, when the target system is a two-dimensional fluid dynamics system, if a set of training input parameters is (i,j,t), then multiple output sub-parameters can be represented as (u,v,P), where output sub-parameter u represents the lateral velocity of the fluid particle, output sub-parameter v represents the longitudinal velocity of the fluid particle, and output sub-parameter P represents the pressure of the fluid particle.

[0075] For example, when the target system is a two-dimensional fluid dynamics system, if a set of training input parameters is (a,b,t), then multiple output sub-parameters can be represented as: (i,j,u,v,P), where output sub-parameter i represents the abscissa of the fluid particle, output sub-parameter j represents the ordinate of the fluid particle, output sub-parameter u represents the lateral velocity of the fluid particle, output sub-parameter v represents the longitudinal velocity of the fluid particle, and output sub-parameter P represents the pressure of the fluid particle.

[0076] In other words, a set of training input parameters can be the independent variables selected in the differential equation, and a set of first output parameters and a set of second output parameters can come from the dependent variables in the differential equation. During the training phase, the set of first output parameters obtained after step S20, corresponding to the set of training input parameters, are the values ​​of the dependent variables estimated based on the parameters of the neural network. The set of second output parameters obtained after step S30, corresponding to the set of first output parameters, are the assumptions (ansatz) that can satisfy the boundary conditions and initial conditions.

[0077] For example, when the target system is a fluid dynamics system, the neural network can be trained by setting multiple fluid particles in the space where the fluid motion to be simulated is to be set at the same time. Each fluid particle corresponds to a set of training input parameters. The neural network is trained based on the multiple sets of training input parameters corresponding to the multiple fluid particles.

[0078] It should be noted that the number of training input parameter sets and the types of sub-parameters included in the training input parameters can be set as needed, and this disclosure does not impose specific restrictions on them.

[0079] For example, after obtaining at least one set of first output parameters of the neural network output through step S20, initial condition constraints and boundary condition constraints are added to these first output parameters so that the final output of the neural network satisfies the initial condition constraints and boundary condition constraints.

[0080] For example, step S30 may include: obtaining the boundary condition constraints and initial condition constraints corresponding to the target system; for any set of first output parameters, determining a set of intermediate output parameters corresponding to the set of first output parameters, wherein when a set of training input parameters corresponding to any set of first output parameters indicates that a set of training input parameters is located at the boundary of a preset region, the corresponding set of intermediate output parameters is set to the value specified by the boundary condition constraints; and determining a set of second output parameters corresponding to any set of first output parameters based on the corresponding set of intermediate output parameters and initial condition constraints.

[0081] For example, boundary conditions refer to the variation of variables or their derivatives with time and location when solving on the boundary of a predefined region. Boundary condition constraints indicate the constraints on the target system on the boundary of the predefined region. For example, initial condition constraints indicate the initial state of the entire target system. For differential equations, definite initial and boundary condition constraints are prerequisites for the entire problem to be well-posed, thus ensuring the existence and unique solution of the differential equation.

[0082] Boundary constraints and initial constraints can be specified according to the target system and the specific phenomena to be simulated. Boundary constraints and initial constraints can be represented in any known form, and this disclosure does not impose any restrictions on them.

[0083] For example, in step S10, multiple sets of training input parameters are obtained. Any set of training parameters is represented as (p, τ), where τ represents the time sub-parameter, and p is a vector representing the independent variables in the differential equation other than the time sub-parameter. For example, p represents the input coordinate space or features of the differential equation.

[0084] For example, in step S20, a set of training parameters (p, τ) are input into the neural network for processing to obtain a set of first output parameters x(p, τ; θ) corresponding to the set of training parameters (p, τ), where θ represents the parameters of the neural network.

[0085] For example, in a differential equation, the dependent variable in the formula can be represented as x(p,τ), which can also be understood as the true numerical solution corresponding to a set of training parameters (p,τ). x(p,τ;θ) represents the estimated value of x(p,τ) obtained through a neural network. For instance, for this set of first output parameters x(p,τ;θ), when applying boundary condition constraints, a set of intermediate output parameters corresponding to this set of first output parameters x(p,τ;θ) is determined based on the boundary condition constraints.

[0086] For example, a set of intermediate output parameters is determined according to the following formula 2.

[0087]

[0088] Here, G(p) represents the boundary condition constraint; the distance between p and the preset region boundary is positively correlated with Q(p), that is, the closer p is to the preset region boundary, the smaller Q(p) is, and Q(p) = 0 when p is located at the preset region boundary; ⊙ represents the Hadamard product operation.

[0089] For example, G(p) can be in the form of a vector group, and G(p) can include multiple initial conditions to set different initial states for multiple variables in the target system.

[0090] For example, as shown in Equation 2, when a set of training input parameters (p, τ) indicates that the set of training input parameters (p, τ) is located at the boundary of the preset region, that is, when p is located at the boundary of the preset region, Q(p) = 0, then That is, the corresponding set of intermediate output parameters Set to the value G(p) specified by the boundary condition constraint.

[0091] Of course, Formula 2 shows one possible implementation. Those skilled in the art can set other expressions, as long as they satisfy the condition that "when any set of training input parameters corresponding to a set of first output parameters indicates that a set of training input parameters is located at the boundary of a preset region, the corresponding set of intermediate output parameters is set to the value specified by the boundary condition constraint". This ensures that the intermediate output parameters corresponding to all sets of training input parameters satisfy the boundary condition constraint, without the need to set additional boundary points, reducing resource consumption, shrinking the search space of the optimizer, improving training efficiency, and accelerating model convergence.

[0092] Therefore, the corresponding set of intermediate output parameters obtained through the above steps To ensure that the boundary conditions are met.

[0093] For example, for this set of first output parameters x(p,τ;θ), when applying initial condition constraints, based on a set of intermediate output parameters... Given initial constraints, determine a set of second output parameters corresponding to any set of first output parameters x(p,τ;θ).

[0094] For example, determining a set of second output parameters corresponding to any set of first output parameters based on a corresponding set of intermediate output parameters and initial condition constraints can include: setting the corresponding set of second output parameters when the time sub-parameter τ in any set of training input parameters (p,τ) indicates the initial time. The value specified for the initial conditions; the time sub-parameter τ in any set of training input parameters (p, τ) indicates that when it falls within a preset time range starting from the initial moment, it is based on the corresponding set of intermediate output parameters. Based on the initial conditions and constraints, determine the corresponding set of second output parameters. In any set of training input parameters (p, τ), the time sub-parameter τ indicates that after a preset time range, the corresponding set of intermediate output parameters will be... As a corresponding set of second output parameters

[0095] For example, initial condition constraints typically specify the initial state of the target system. For instance, when the time sub-parameter τ indicates the initial time, such as τ = 0, a corresponding set of second output parameters is set. The value specified for the initial condition, so that the second output parameter The initial conditions are satisfied at the initial moment.

[0096] For example, a transition function T(τ;k) can be set to simulate the second output parameter. In a scenario where the initial condition constraint specifies a value at the initial moment, and its influence decreases over time, that is, the second output parameter... Gradually transition from the initial conditions to the intermediate output parameters within a preset time range

[0097] For example, a set of second output parameters is determined according to the following formula 3

[0098]

[0099] Here, T(τ; k) is a transition function and is used to adjust the length of the preset time range through the transition parameter k. T(τ; k) satisfies the following two conditions: x0(p) represents the initial condition constraint.

[0100] For example, the transition parameter k is positively correlated with the length of the preset time range. That is, the larger the transition parameter k, the second output parameter The slower the change speed of the transition from the initial conditions, and the longer the preset time range; conversely, the smaller the transition parameter k, the second output parameter The faster the change speed of the transition from the initial conditions, and the shorter the preset time range.

[0101] For example, when the time subparameter τ in any set of training input parameters (p, τ) indicates the initial moment, that is, τ = 0, then T(0; k) = 0, Therefore, at the initial moment, a set of second output parameters Is set to the value specified by the initial condition x0(p).

[0102] For example, when the time subparameter τ in a set of training input parameters (p, τ) indicates that it is within the preset time range starting from the initial moment, at this time 0 < T(τ; k) < 1, a set of second output parameters Is determined according to formula 3.

[0103] For example, when the time subparameter τ in a set of training input parameters (p, τ) indicates that it is after the preset time range starting from the initial moment, at this time T(τ; k) = 1, That is, a set of intermediate output parameters Is used as a set of second output parameters

[0104] For example, T(τ; k) is a transition function in a known form, the transition parameter k is preset and remains unchanged during the training process of the neural network.

[0105] For example, both the transition function T(τ; k) and the transition parameter k can be scalars. In one embodiment, the transition function T(τ; k) can be expressed as the following formula 4:

[0106]

[0107] Of course, the form of the transition function and the specific values ​​of the transition parameters can be set according to actual needs, and this disclosure does not impose any restrictions on this.

[0108] Of course, Equation 3 shows one possible implementation. Those skilled in the art can set other expressions to satisfy the requirement that the corresponding set of second output parameters is set to the values ​​specified by the initial conditions at the initial time. This ensures that the second output parameters corresponding to all sets of training input parameters satisfy the initial condition constraints, without the need to set additional initial points, reducing resource consumption, narrowing the search space of the optimizer, improving training efficiency, and accelerating model convergence.

[0109] Therefore, the second set of output parameters obtained through the above steps This is a hypothetical scenario that can simultaneously satisfy both boundary conditions and initial conditions.

[0110] After obtaining at least one set of second output parameters, the loss value is obtained based on these second output parameters.

[0111] For example, in some embodiments, the neural network can be trained directly without observation data. For example, step S40 may include: obtaining at least one set of residual results corresponding to the second output parameters based on at least one set of second output parameters and differential equations; and obtaining the loss value of the neural network based on the residual results corresponding to the at least one set of second output parameters.

[0112] The loss function for a neural network is expressed as:

[0113]

[0114] in, Let represent the loss function, θ represent the parameters of the neural network, R(*) represent the residual calculation, and (p,τ) represent any set of training input parameters from at least one set of training input parameters. Let M represent a set of second output parameters corresponding to any set of training input parameters, M represent the number of differential equations corresponding to the target system, and i be a positive integer. λ represents the accumulation operation. i This represents the residual weighting coefficient corresponding to the i-th differential equation. This represents the residual result calculated based on the i-th differential equation.

[0115] For example, according to Formulas 2 and 3 above, a set of second output parameters corresponding to a set of training input parameters (p, τ) can be obtained. Replace x(p,τ) in the differential equation with The differential equation to be fitted is obtained, and the loss value is calculated using Formula 5 based on the differential equation to be fitted, so as to adjust the parameters of the neural network.

[0116] Specifically, here, The dependent variable in the differential equation is represented as x(p,τ), which is obtained by referring to these two formulas. (The specific meanings of the parameters in the formulas are shown in Formulas 2 and 3, and will not be repeated here.) The x(p,τ) in the original differential equation is replaced with... The form of this is used to obtain the differential equation to be fitted.

[0117] In other words, the differential equation to be fitted and the original differential equation appear to have different structures, but they express the same meaning, that is, both describe the same state and / or motion of the same target system. Therefore, Became The estimated value obtained based on the neural network, Became The estimated value is obtained based on a neural network.

[0118] Given any set of training input parameters (p, τ) and the corresponding set of second output parameters... Substitute the differential equation to be fitted, calculate the residual result corresponding to each group of second output parameters, and add the residual results corresponding to each group of second output parameters to obtain the loss value of the neural network.

[0119] For example, the residual result corresponding to the second output parameter of each group can be calculated according to Formula 5.

[0120] Referring to Formulas 5 and 1, the neural network training method provided in at least one embodiment of this disclosure does not require additional data fitting for initial and boundary conditions, thereby reducing the resource consumption caused by setting additional boundary and initial points. It avoids the phenomenon in existing training methods where uneven gradient descent of the loss function and rapid changes (rigidity) affect the convergence of the entire optimization process and lead to poor final results, thus improving training efficiency and accelerating model convergence.

[0121] For example, in some embodiments, in addition to the loss due to physical constraints, a small amount of observation data can be incorporated during the training process to calculate the loss value for training the neural network.

[0122] For example, step S40 may include: calculating the residual results corresponding to the at least one set of second output parameters based on the at least one set of second output parameters and the differential equation; calculating the data fitting loss value based on the output data and the label data, wherein the output data is the output obtained by inputting the training data into the neural network, and the label data is the standard value corresponding to the training data; and obtaining the loss value of the neural network based on the residual results corresponding to the at least one set of second output parameters and the data fitting loss value.

[0123] For example, at this time, at least one set of residual results corresponding to the second output parameters can be calculated according to Formula 5; the data fitting loss value can be calculated based on a small amount of labeled observation data (i.e., training data with corresponding label data), for example, the training data is input into the neural network to obtain output data, and the data fitting loss value is calculated based on the output data and label data; the sum or weighted sum of at least one set of residual results corresponding to the second output parameters and the data fitting loss value is calculated, and the sum or weighted sum is used as the loss value of the neural network.

[0124] Next, in step S50, the loss function is optimized using an optimizer. For example, a first-order optimizer can be used, such as the Adam (Adaptive Momentum Estimation) optimizer, or a second-order optimizer can be used, such as the L-BFGS-B (Limited-Broyden Fletcher Goldfarb Shanno-bound) optimizer. Alternatively, a combination of first-order and second-order optimizers can be used, for example, when the first-order optimizer cannot optimize further, the second-order optimizer can be used to achieve better optimization results and find the optimal parameter values ​​θ of the neural network.

[0125] For example, when the target system is a fluid dynamics system, current fluid dynamics simulations, especially free surface simulations, typically use differential equations (or systems of equations) from an Euler perspective. The independent variables in these Euler-perspective differential equations include the position information of fluid particles (e.g., x-coordinate i, y-coordinate j) and time t, while the dependent variables include the velocity information of fluid particles (e.g., lateral velocity u, longitudinal velocity v), density information ρ, pressure P, etc. The Euler perspective is suitable for problems requiring observation of fluid information at a given location. However, when simulating free surfaces, the positions of fluid particles from an Euler perspective require additional calculation, and tracking or capturing the surface cannot be easily achieved within current neural network frameworks.

[0126] Therefore, at least one embodiment of this disclosure converts the partial differential equation from the Euler perspective into a differential equation from the Lagrange perspective, so that the differential equation describes the fluid dynamics system from the Lagrange perspective. This allows the trained neural network to directly obtain the position information of fluid particles and to easily set free surface conditions in the boundary conditions, thereby improving the simulation effect of the neural network on fluids with liquid surfaces.

[0127] The following example, using the simulation of the liquid surface of a liquid that meets the conditions of being incompressible and non-viscous, illustrates the execution process of the neural network training method provided in at least one embodiment of this disclosure.

[0128] It should be noted that, for ease of understanding and description, the following embodiments use two-dimensional or three-dimensional Euler equations as an example, and a schematic simulation scenario is set up for description. However, those skilled in the art can, according to actual needs, for example, when considering liquid viscosity, the differential equation can be adopted, and the initial conditions and boundary conditions can also be changed according to different simulation scenarios. This disclosure does not limit this.

[0129] For example, for liquids similar to water, we can make the physical assumptions that they are incompressible and inviscid; therefore, the motion of such liquids can be described by the incompressible Euler equations. In the case of gravity alone, the two-dimensional partial differential equations (Euler equations) describing the fluid dynamics system are shown in Equation 6:

[0130]

[0131] here, Let represent the partial derivative, i represent the x-coordinate position of the fluid particle, j represent the y-coordinate position of the fluid particle, t represent time, u represent the lateral velocity of the fluid particle, v represent the longitudinal velocity of the fluid particle, g represent gravity, P represent the pressure of the fluid particle, and ρ represent the density of the fluid particle. From Euler's perspective, for example, the dependent variables could be the fluid particle's position coordinates (i,j) and time t, while the independent variables could be the fluid particle's velocity (u,v) and pressure P.

[0132] This equation describes the fluid from an Euler perspective and contains only information about the fluid velocity field. Therefore, the specific locations of fluid particles, including free surfaces, require additional calculations, while this information can be directly obtained from a Lagrange perspective. Therefore, the Euler equation in Equation 6 can be transformed into a differential equation from a Lagrange perspective, as shown in Equation 7 below:

[0133]

[0134] Here, (a,b) represents the label of the fluid particle, (i,j) represents the position coordinates of the fluid particle, (u,v) represents the velocity of the fluid particle, τ represents time (different symbols are used to distinguish different perspectives), P represents the pressure of the fluid particle, and subscripts are used to represent partial derivatives, for example, i τ This represents the partial derivative of the position coordinate i of the fluid particle with respect to time τ.

[0135] It should be noted that the above conversion is only one possible mapping method. Those skilled in the art can also set more or other forms of converted differential equations according to actual needs, and this disclosure does not limit this.

[0136] For example, from a Lagrange perspective, the dependent variables are the fluid particle's label (a,b) and time τ, while the independent variables are the fluid particle's position (i,j), velocity (u,v), and pressure P.

[0137] For example, in step S10, multiple sets of training input parameters corresponding to multiple fluid particles are obtained. Each set of training input parameters includes the label sub-parameters (a, b) and time sub-parameter τ of the corresponding fluid particle.

[0138] Next, in step S20, multiple sets of training input parameters are input into the neural network for processing to obtain multiple sets of first output parameters x(p,τ;θ) corresponding to the multiple sets of training input parameters, where p=(a,b). For example, a set of first output parameters x(p,τ;θ)={i(p,τ;θ),j(p,τ);θ,u(p,τ;θ),v(p,τ;θ),P(p,τ;θ)}.

[0139] Next, to make the entire problem appropriate, in step S30, the boundary condition constraints and initial condition constraints of the fluid dynamics system are obtained. For example, assuming the simulation is of the motion of the free surface in an unsealed cuboid container (assuming no forward or backward motion of the container so that the target system can be described by two-dimensional equations), the initial condition constraint x0(p) can be expressed as:

[0140]

[0141] Here, we take i(a,b,0)=i0(a,b) as an example to illustrate the meaning of the parameters. For example, i(a,b,0)=i0(a,b) means that at the initial time (τ=0), the x-coordinate of the position of the fluid particle (a,b) is i0(a,b), and i0(a,b) is a certain known form.

[0142] For example, such as Figure 2 As shown, the length of the cuboid container is L, the height is H, the length direction is the horizontal axis direction (i(a,b,τ)), the height direction is the vertical axis direction (j(a,b,τ)), and the curve in physical space represents the possible state of the liquid surface at a certain time.

[0143] For example, the velocity of the fluid at the boundary of a predetermined region is 0, and the boundary positions of the fluid particles are constrained. Furthermore, for the free surface, the pressure of the fluid particles on the free surface is 0. Therefore, the resulting boundary condition constraint G(p) can be expressed as:

[0144]

[0145] Then, in step S30, a set of second input parameters corresponding to each set of first input parameters x(p,τ;θ) is calculated according to formulas 2 and 3 above.

[0146] For example, given a Q(p), in one embodiment, the dependent variable x(p,τ) of the differential equation and the corresponding The mapping relationship is shown in Formula 10:

[0147]

[0148] For example, as shown in the first formula in Formula 10, when a = L, Constraints that satisfy boundary conditions.

[0149] For example, and The mapping relationship is shown in Formula 11:

[0150]

[0151] Then, in step S40, replace 'i' in formula 7 with 'i' in formula 11. Replace j in formula 7 with j in formula 11. Thus, a system of differential equations for calculating the residuals is obtained, which is also the system of differential equations to be fitted, based on multiple sets of second output parameters. The loss value is calculated using this system of differential equations and Formula 5.

[0152] Then, in step S50, the parameters of the neural network are adjusted based on the loss value.

[0153] Repeat the above process until a neural network that meets the predetermined convergence condition is obtained.

[0154] Figure 3 This is a structural diagram of a neural network provided in an embodiment of the present disclosure.

[0155] like Figure 3 As shown, the neural network includes a fully connected network. A set of training input parameters (p,τ) = (a,b,τ) are input into the fully connected network to obtain a corresponding set of first output parameters x(p,τ;θ) = (i,j,u,v,P).

[0156] Then, referring to formulas 2 and 3 in step S30 above, a set of second output parameters corresponding to a set of first output parameters x(p,τ;θ) is calculated. Where (i0,j0,u0,v0,P0) represents the initial condition x0(p), and the partial derivative operations in the formula are implemented through an automatic differentiation framework.

[0157] Then, in step S40, based on multiple sets of second output parameters The loss value is calculated according to the loss function shown in Formula 5. The specific process is as described in step S40, and will not be repeated here.

[0158] Then, the parameters of the neural network are adjusted according to the loss value. The specific process is as described in step S50, and will not be repeated here.

[0159] For example, for a liquid that is physically assumed to be incompressible and inviscid, the three-dimensional partial differential equations (Euler equations) describing the fluid dynamics system under gravity alone are shown in Equation 12:

[0160]

[0161] Similar to the parameter definition in Formula 6, (i,j,z) represents the three-dimensional position coordinates of the fluid particle, and (u,v,w) represents the three-dimensional velocity of the fluid particle. From Euler's perspective, for example, the dependent variable can be the three-dimensional position coordinates (i,j,z) of the fluid particle and time t, and the independent variable can be the three-dimensional velocity (u,v,w) of the fluid particle and pressure P.

[0162] The Euler equations from the Euler perspective in Equation 12 are transformed into differential equations from the Lagrange perspective. The resulting set of differential equations is shown in Equation 13.

[0163]

[0164] Here, (a,b,c) represents the label of the fluid particle, (i,j,z) represents the position of the fluid particle, (u,v,w) represents the velocity of the fluid particle, τ represents time, P represents the pressure of the fluid particle, and the subscript letters represent the partial derivatives, for example, i τ This represents the partial derivative of the abscissa position i of the fluid particle with respect to time τ.

[0165] It should be noted that the above conversion is only one possible mapping method. Those skilled in the art can also set more or other forms of converted differential equations according to actual needs, and this disclosure does not limit this.

[0166] For example, from a Lagrange perspective, the dependent variables are the fluid particle's label (a,b,c) and time τ, while the independent variables are the fluid particle's position (i,j,z), velocity (u,v,w), and pressure P.

[0167] For example, assuming the simulation is of the motion of a free liquid surface in an unsealed cuboid container, the initial constraint x0(p) can be expressed as:

[0168]

[0169] The definition of the initial condition constraint x0(p) is the same as that of the two-dimensional system, and will not be repeated here.

[0170] For example, a cuboid container has a length of L, a height of H, and a depth of D. The velocity of the fluid at the boundary of a predetermined region is 0, and the boundary positions of the fluid particles are constrained. Furthermore, for the free surface, the pressure of the fluid particles on the free surface is 0. Therefore, the resulting boundary condition constraint G(p) can be expressed as:

[0171]

[0172] Then, in step S30, a set of second input parameters corresponding to each set of first input parameters x(p,τ;θ) is calculated according to formulas 2 and 3 above.

[0173] For example, given a Q(p), in one embodiment, the variables x(p,τ) of the differential equation and the corresponding The mapping relationship is shown in Formula 16:

[0174]

[0175] For example, and The mapping relationship is shown in Formula 17:

[0176]

[0177] Then, in step S40, 'i' in formula 13 is replaced with 'i' in formula 17. Replace j in formula 13 with j in formula 17. Thus, a system of differential equations for calculating the residuals is obtained, based on multiple sets of second output parameters. The loss value is calculated using this system of differential equations and Formula 5.

[0178] Then, in step S50, the parameters of the neural network are adjusted based on the loss value.

[0179] Repeat the above process until a neural network that meets the predetermined convergence condition is obtained.

[0180] The training method described in at least one embodiment of this disclosure, because it represents differential equations from a Lagrange perspective, enables the trained neural network to effectively calculate fluid simulation problems involving free surfaces and accurately calculate the position and velocity of the liquid surface. Furthermore, under the same parameters, compared to a physics-guided neural network trained using the method provided in at least one embodiment of this disclosure, the neural network exhibits better convergence speed, higher computational efficiency, and lower memory usage because it does not require optimization and fitting of initial and boundary points.

[0181] Figure 4A and Figure 4B A comparison diagram of simulation results for a free liquid surface simulation provided in an embodiment of this disclosure.

[0182] For example, the target system being simulated is a fluid dynamics system, and the physical phenomenon being simulated is the sloshing of a medium-to-high amplitude fluid surface in an unsealed container.

[0183] For example, by training the neural network using the contents described in Formulas 6 to 11 above, the following can be obtained: Figure 4A and Figure 4B The simulation results, labeled "This Application," are on the right side of the image. Furthermore, in... Figure 4A and Figure 4B In the image, the simulation results labeled "Traditional Physical Guidance Network" on the left represent the simulation results of the neural network obtained using the physical guidance network according to the traditional training method.

[0184] like Figure 4A and Figure 4B As shown, since the training method provided in at least one embodiment of this disclosure adds initial condition constraints and boundary condition constraints to the output of the neural network, the initial conditions and boundary conditions of the simulation results of "this application" are well satisfied, and the simulated liquid surface sloshing phenomenon is accurate and consistent with reality, while the simulation results obtained by the traditional method cannot simulate the free liquid surface.

[0185] Figures 5A to 5D A comparison diagram of simulation results for free liquid surface simulation provided in another embodiment of this disclosure.

[0186] exist Figures 5A to 5D In this simulation, the target system is a fluid dynamics system, and the physical phenomenon simulated is the "dam break" phenomenon, such as the change in water flow when a dam opens its gates to release water.

[0187] exist Figures 5A to 5D In the diagram, the simulation results marked "Traditional Physical Guidance Network" on the left represent the simulation results of a neural network obtained using a physical guidance network according to a traditional training method, while the simulation results marked "This Application" on the right represent the simulation results of a neural network trained using the training method provided in at least one embodiment of this disclosure.

[0188] Figure 5A The simulation results at the initial time (t=0) are described, such as... Figure 5A As shown, since the training method provided in at least one embodiment of this disclosure adds initial condition constraints to the output of the neural network, the simulation results of "this application" satisfy the initial conditions at the initial time, while the simulation results obtained by the traditional method do not satisfy the initial conditions well at the initial time.

[0189] Figures 5B to 5D Simulation results are described at t = 0.05s, t = 0.3s, and t = 0.7s. Based on... Figures 5B to 5D It is evident that, since the training method provided in at least one embodiment of this disclosure adds boundary condition constraints to the output of the neural network, the boundary conditions and initial conditions of the simulation results of "this application" are well satisfied, and the simulated "dam break" phenomenon is accurate and consistent with reality. In contrast, the simulation results obtained by the traditional method do not satisfy the boundary conditions well, and the error increases over time, eventually making it impossible to simulate the free liquid surface.

[0190] Through actual testing, the neural networks trained by the training method provided in at least one embodiment of this disclosure can achieve excellent liquid surface simulation effects and can capture phenomena such as nonlinear sloshing, impact and rebound. In contrast, traditional physical guidance neural networks cannot even simulate the correct fluid phenomena.

[0191] For example, at least one embodiment of this disclosure also provides a data processing method. Figure 6 This is a schematic flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0192] like Figure 6 As shown, the data processing method includes steps S60-S70.

[0193] In step S60, at least one set of input parameters corresponding to the target system is obtained.

[0194] In step S70, at least one set of input parameters is input into the neural network to obtain at least one set of intermediate output results.

[0195] In step S80, at least one set of intermediate output results are transformed to obtain at least one set of final output results that correspond one-to-one with at least one set of intermediate output results. The at least one set of final output results is obtained by adding boundary condition constraints and initial condition constraints to at least one set of intermediate output results respectively.

[0196] For example, the definition of the input parameters is the same as that of the training input parameters. For example, if the training input parameters are represented as (i,j,t), then the input parameters also include the horizontal coordinate of the fluid particle, the vertical coordinate of the fluid particle, and time t. The specifics will not be elaborated here.

[0197] For example, the neural network is trained at least in part according to the neural network training method described in at least one embodiment of the present disclosure.

[0198] The training methods for neural networks have been described above and will not be repeated here.

[0199] For example, the trained neural network can characterize the differential equations corresponding to the target system.

[0200] For example, a set of input parameters can be represented as (q, τ), where τ represents the time sub-parameter and q is a vector representing the independent variables in the differential equation other than the time sub-parameter. For example, a set of input parameters can be pre-set values.

[0201] For example, a set of intermediate output results can be represented as x(q,τ), which is the dependent variable of the differential equation.

[0202] For example, a set of final output results can be represented as That is, the numerical solution of the differential equation to be fitted to the target system, which satisfies the boundary condition constraints and the initial condition constraints. As mentioned earlier, the differential equation to be fitted is obtained by replacing x(p,τ) in the original differential equation with... The form of this is used to obtain the differential equation to be fitted.

[0203] For example, the conversion process can be implemented using the following formulas 18 and 19:

[0204]

[0205]

[0206] The meaning of the parameters in Formula 18 can be found in Formula 2, and will not be repeated here. The meaning of the parameters in Formula 19 can be found in Formula 3, and will not be repeated here.

[0207] Therefore, the differential equations represented in this data processing method are different from the original differential equations (sets) represented by traditional physical guided neural networks. The differential equations represented in this data processing method are differential equations (sets) to be fitted. As mentioned above, the differential equations (sets) to be fitted have a different structure than the original differential equations (sets), but express the same meaning.

[0208] For example, due to the high parallelism of neural networks and the relaxed requirements for computational precision, this data processing method can be executed in parallel on graphics processors, data processors, or neural network processors, and can be accelerated at mixed precision.

[0209] Furthermore, when the neural network is a physics-guided neural network, the data processing method also provides advantages such as more accurate calculation, especially for simulating nonlinear terms, due to the advantages brought by the physics-guided neural network.

[0210] At least one embodiment of this disclosure also provides a training apparatus for a neural network. Figure 7 This is a schematic block diagram of a training device provided for at least one embodiment of the present disclosure.

[0211] like Figure 7 As shown, the neural network training device 100 may include an acquisition unit 101, a computation processing unit 102, a constraint processing unit 103, a loss value calculation unit 104, and an adjustment unit 105. These components are interconnected via a bus system and / or other forms of connection mechanisms (not shown). For example, these modules can be implemented by hardware (e.g., circuit) modules, software modules, or any combination of both, as is the case in the following embodiments, and will not be repeated. For example, these units can be implemented by a central processing unit (CPU), a graphics processing unit (GPU), a tensor processor (TPU), a neural network processor (NPU), a data processor (DPU), an AI accelerator, a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions. It should be noted that... Figure 7 The components and structure of the training device 100 shown are merely exemplary and not limiting; the training device 100 may also have other components and structures as needed.

[0212] For example, acquisition unit 101 is configured to acquire at least one set of training input parameters.

[0213] For example, the arithmetic processing unit 102 is configured to input the at least one set of training input parameters into the neural network for arithmetic processing to obtain at least one set of first output parameters corresponding to the at least one set of training input parameters.

[0214] For example, constraint processing unit 103 is configured to obtain at least one set of second output parameters corresponding to the at least one set of first output parameters, wherein the at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters respectively.

[0215] For example, the loss value calculation unit 104 is configured to obtain the loss value of the neural network for the at least one set of first output parameters, calculated by the loss function corresponding to the neural network, based on the at least one set of second output parameters.

[0216] For example, adjustment unit 105 is configured to adjust the parameters of the neural network based on the loss value.

[0217] For example, a neural network training device is used to train a training network, which includes a neural network, a loss function, etc. (not shown), and an adjustment unit 105 is used to train the neural network to be trained in order to obtain a trained neural network.

[0218] It should be noted that the training network has the same structure and function as the neural network in the embodiments of the above-described neural network training method, and will not be described again here.

[0219] For example, the acquisition unit 101, the arithmetic processing unit 102, the constraint processing unit 103, the loss value calculation unit 104, and the adjustment unit 105 may include code and programs stored in memory; the processor may execute the code and programs to implement some or all of the functions of the acquisition unit 101, the arithmetic processing unit 102, the constraint processing unit 103, the loss value calculation unit 104, and the adjustment unit 105 as described above.

[0220] For example, the acquisition unit 101, the calculation and processing unit 102, the constraint processing unit 103, the loss value calculation unit 104, and the adjustment unit 105 can be dedicated hardware devices used to implement some or all of the functions of the acquisition unit 101, the calculation and processing unit 102, the constraint processing unit 103, the loss value calculation unit 104, and the adjustment unit 105 as described above. For example, the acquisition unit 101, the calculation and processing unit 102, the constraint processing unit 103, the loss value calculation unit 104, and the adjustment unit 105 can be a single circuit board or a combination of multiple circuit boards used to implement the functions described above.

[0221] In the embodiments of this application, the one or more circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) processor-executable firmware stored in the memory.

[0222] It should be noted that the acquisition unit 101 can be used to implement Figure 1 The arithmetic processing unit 102 can be used to implement step S10 as shown. Figure 1 As shown in step S20, the constraint processing unit 103 can be used to implement... Figure 1 The loss value calculation unit 104, as shown in step S30, can be used to implement... Figure 1 The adjustment unit 105 can be used to implement step S40 shown. Figure 1 The step S50 is shown. Therefore, for a detailed description of the functions that the acquisition unit 101, the calculation processing unit 102, the constraint processing unit 103, the loss value calculation unit 104, and the adjustment unit 105 can achieve, please refer to the relevant descriptions of steps S10 to S50 in the embodiments of the above data processing method; repeated details will not be repeated here. Furthermore, the data processing device 100 can achieve similar technical effects to the aforementioned data processing method, which will not be described further here.

[0223] It should be noted that, in the embodiments of this disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited; it can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0224] At least one embodiment of this disclosure also provides a data processing apparatus. Figure 8 This is a schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure.

[0225] like Figure 8 As shown, the data processing device 200 may include an input acquisition unit 201 and a processing unit 202. These components are interconnected via a bus system and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 8 The components and structure of the data processing apparatus 200 shown are merely exemplary and not limiting. The data processing apparatus 200 may also have other components and structures as needed.

[0226] For example, the input acquisition unit 201 is configured to acquire at least one set of input parameters corresponding to the target system.

[0227] For example, processing unit 202 is configured to input at least one set of input parameters into a neural network to obtain at least one set of output parameters.

[0228] For example, the neural network is at least partially trained according to the training method described in any embodiment of this disclosure. The specific training process can be referred to the relevant descriptions in the embodiments of the neural network training method described above, and repeated details will not be repeated.

[0229] At least one embodiment of this disclosure also provides an electronic device. Figure 9 This is a schematic diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0230] For example, such as Figure 9 As shown, the electronic device includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704. The processor 701, communication interface 702, and memory 703 communicate with each other via the communication bus 704. The processor 701, communication interface 702, and memory 703 can also communicate with each other via a network connection. This disclosure does not limit the type and function of the network. It should be noted that... Figure 7 The components of the electronic device shown are merely exemplary and not limiting; the electronic device may have other components depending on the actual application requirements.

[0231] For example, memory 703 is used to store computer-readable instructions non-transitory. When processor 701 executes the computer-readable instructions, it implements the neural network training method or data processing method according to any of the above embodiments. For specific implementations and explanations of the various steps of the neural network training method or data processing method, please refer to the embodiments of the neural network training method or data processing method described above, and will not be repeated here.

[0232] For example, other implementations of the neural network training method or data processing method implemented by the processor 701 executing computer-readable instructions stored in the memory 703 are the same as the implementations mentioned in the foregoing method embodiment section, and will not be repeated here.

[0233] For example, the communication bus 704 can be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0234] For example, communication interface 702 is used to enable communication between electronic devices and other devices.

[0235] For example, processor 701 and memory 703 can be located on the server side (or in the cloud).

[0236] For example, processor 701 can control other components in an electronic device to perform desired functions. Processor 701 can be a central processing unit (CPU), tensor processor (TPU), neural network processor (NPU), data processor (DPU), AI accelerator, or graphics processing unit (GPU)—a device with data processing and / or program execution capabilities; it can also be a digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The central processing unit (CPU) can be based on x86 or ARM architectures, etc.

[0237] For example, memory 703 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and processor 701 may execute the computer-readable instructions to implement various functions of the electronic device. Various application programs and various data may also be stored in the storage medium.

[0238] For example, a detailed description of the process by which an electronic device performs a neural network training method or a data processing method can be found in the relevant descriptions in the embodiments of the neural network training method or data processing method, and repeated descriptions will not be repeated.

[0239] Figure 10 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. For example, such as Figure 10 As shown, the storage medium 800 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 801 can be stored non-transitory. For example, when the computer-readable instructions 801 are executed by a processor, one or more steps of the data processing method or neural network training method described above can be performed.

[0240] For example, the storage medium 800 can be used in the aforementioned electronic device, such as the storage medium 800 may include the memory in the electronic device.

[0241] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0242] For example, the description of storage medium 800 can be found in the description of memory in the embodiments of the electronic device, and repeated descriptions will not be repeated here.

[0243] Figure 11 This is a schematic diagram of a hardware environment provided for at least one embodiment of the present disclosure. The electronic device provided in this disclosure can be applied to an Internet system.

[0244] use Figure 11 The computer system provided herein can implement the functions of the data processing apparatus and / or electronic device involved in this disclosure. Such computer systems may include personal computers, laptops, tablets, mobile phones, personal digital assistants, smart glasses, smartwatches, smart rings, smart helmets, and any smart portable or wearable device. A specific system in this embodiment uses a functional block diagram to explain a hardware platform including a user interface. This computer device can be a general-purpose computer device or a purpose-specific computer device. Both types of computer devices can be used to implement the data processing apparatus and / or electronic device of this embodiment. The computer system may include any components necessary to implement the image processing described herein. For example, the computer system can be implemented by a computer device through its hardware, software programs, firmware, and combinations thereof. For convenience, Figure 11 Although only one computer device is shown in the figure, the computer functions related to the information required for image processing described in this embodiment can be implemented in a distributed manner by a set of similar platforms, thus distributing the processing load of the computer system.

[0245] like Figure 11 As shown, the computer system may include a communication port 250 connected to a network for data communication. For example, the computer system can send and receive information and data through the communication port 250, enabling the computer system to exchange data via wireless or wired communication with other electronic devices. The computer system may also include a processor group 220 (i.e., the processor described above) for executing program instructions. The processor group 220 may consist of at least one processor (e.g., a CPU). The computer system may include an internal communication bus 210. The computer system may include different forms of program storage units and data storage units (i.e., the memory or storage media described above), such as a hard disk 270, read-only memory (ROM) 230, and random access memory (RAM) 240, capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor group 220. The computer system may also include an input / output component 260 for implementing input / output data flow between the computer system and other components (e.g., user interface 280, etc.).

[0246] Typically, the following devices can be connected to the input / output component 260: input devices such as touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices such as displays (e.g., LCD, OLED displays, etc.), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication interfaces.

[0247] Although Figure 11 A computer system with various devices is shown, but it should be understood that the computer system is not required to have all the devices shown, and alternatively, the computer system may have more or fewer devices.

[0248] The following points should be noted regarding this disclosure:

[0249] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0250] (2) For clarity, the thickness and dimensions of layers or structures are enlarged in the accompanying drawings used to describe embodiments of the invention. It will be understood that when an element such as a layer, film, region, or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element, or there may be intermediate elements present.

[0251] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0252] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for training a neural network, applied to a graphics processing unit, comprising: Obtain at least one set of training input parameters; The at least one set of training input parameters is input into the neural network to perform parallel computation processing using the graphics processor, thereby obtaining at least one set of first output parameters corresponding to the at least one set of training input parameters; At least one set of second output parameters corresponding to the at least one set of first output parameters are obtained, wherein the at least one set of second output parameters are obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters respectively, wherein the boundary condition constraints indicate the constraint situation of the target system on the boundary of a preset region, and the initial condition constraints indicate the initial state of the target system; Based on the at least one set of second output parameters, obtain the loss value of the neural network for the at least one set of first output parameters calculated by the loss function corresponding to the neural network; The parameters of the neural network are adjusted based on the loss value; The target system includes a fluid dynamics system, and the neural network is a neural network based on prior physical knowledge. The neural network is trained to simulate the motion of fluid particles in the fluid dynamics system. The motion simulation of fluid particles includes liquid surface simulation. The fluid dynamics system contains multiple fluid particles, each fluid particle corresponding to a set of training input parameters. Each set of training input parameters is used to characterize the label information and time information of the corresponding fluid particle from a Lagrange perspective. Each set of training input parameters corresponds to a first set of output parameters, which includes the position information of the fluid particle.

2. The training method according to claim 1, wherein, Obtaining at least one set of second output parameters corresponding to each of the at least one set of first output parameters includes: Obtain the boundary condition constraints and initial condition constraints corresponding to the target system; For any set of first output parameters, determine a set of intermediate output parameters corresponding to the set of first output parameters, wherein when a set of training input parameters corresponding to the set of first output parameters indicates that the set of training input parameters is located at the boundary of a preset region, the corresponding set of intermediate output parameters is set to the value specified by the boundary condition constraint. Based on the corresponding set of intermediate output parameters and the initial condition constraints, determine a set of second output parameters corresponding to any set of first output parameters.

3. The training method according to claim 2, wherein, For any set of first output parameters, determine a set of intermediate output parameters corresponding to the set of first output parameters, including: The corresponding set of intermediate output parameters is obtained according to the following formula: in, This represents the corresponding set of intermediate output parameters. Represents any set of training input parameters, In vector form, express The corresponding time; The boundary condition constraints are represented by the parameters in any set of training input parameters. The distance between and the boundary of the preset area and They show a positive correlation, and in When located at the boundary of the preset region, ; This represents the Hadamard product operation; This represents a set of first output parameters corresponding to any set of training input parameters. This represents the parameters of the neural network.

4. The training method according to claim 3, wherein, Each set of training input parameters includes a time sub-parameter. Based on the corresponding set of intermediate output parameters and the initial condition constraints, determine a set of second output parameters corresponding to any set of first output parameters, including: When the time sub-parameter in any set of training input parameters indicates the initial time, the corresponding set of second output parameters is set to the value specified by the initial condition constraint. When the time sub-parameter in any set of training input parameters is within a preset time range starting from the initial time, the corresponding set of second output parameters is determined based on the corresponding set of intermediate output parameters and the initial condition constraints. If the time sub-parameter in any set of training input parameters indicates that it is after the preset time range, the corresponding set of intermediate output parameters will be used as the corresponding set of second output parameters.

5. The training method according to claim 4, wherein, The corresponding set of second output parameters is obtained according to the following formula: in, This represents the corresponding set of second output parameters; This represents the corresponding set of intermediate output parameters; It is a transition function and is used to pass through transition parameters. Adjust the preset time range, The following two conditions must be met: ; This represents the initial condition constraint.

6. The training method according to claim 5, wherein, The transition parameters It is positively correlated with the length of the preset time range.

7. The training method according to claim 1, wherein, Based on the at least one set of second output parameters, obtaining the loss value of the neural network for the at least one set of first output parameters, calculated by the loss function corresponding to the neural network, includes: Based on the at least one set of second output parameters and the differential equation, obtain the residual results corresponding to the at least one set of second output parameters, wherein the differential equation describes the motion and / or state of the target system; The loss value of the neural network is obtained based on the residual results corresponding to the at least one set of second output parameters.

8. The training method according to claim 7, wherein, The loss function corresponding to the neural network is expressed as: in, Represents the loss function. The parameters of the neural network are represented. This indicates residual calculation. This refers to any one of the at least one set of training input parameters. Let M represent a set of second output parameters corresponding to any set of training input parameters, M represent the number of differential equations corresponding to the target system, and i be a positive integer. This indicates an accumulation operation. This represents the residual weighting coefficient corresponding to the i-th differential equation. This represents the residual result calculated based on the i-th differential equation.

9. The training method according to claim 1, wherein, Based on the at least one set of second output parameters, obtaining the loss value of the neural network for the at least one set of first output parameters, calculated by the loss function corresponding to the neural network, includes: Based on the at least one set of second output parameters and the differential equation, calculate the residual results corresponding to the at least one set of second output parameters, wherein the differential equation describes the motion and / or state of the target system; The data fitting loss value is calculated based on the output data and label data, wherein the output data is the output obtained by inputting the training data into the neural network, and the label data is the standard value corresponding to the training data; The loss value of the neural network is obtained based on the residual results corresponding to the at least one set of second output parameters and the data fitting loss value.

10. The training method according to any one of claims 1-9, wherein, The neural network is a physical information neural network.

11. The training method according to any one of claims 7-9, wherein, The differential equation is obtained by transforming the partial differential equation from the Euler perspective, and the differential equation describes the fluid dynamics system from the Lagrange perspective.

12. A data processing method, comprising: Obtain at least one set of input parameters corresponding to the target system; The at least one set of input parameters is input into the neural network to obtain at least one set of intermediate output results; The at least one set of intermediate output results are transformed to obtain at least one set of final output results that correspond one-to-one with the at least one set of intermediate output results. The at least one set of final output results are obtained by adding boundary condition constraints and initial condition constraints to the at least one set of intermediate output results respectively. The neural network is at least partially trained using the training method according to any one of claims 1-11.

13. A training device for a neural network, applied to a graphics processor, comprising: The acquisition unit is configured to acquire at least one set of training input parameters; The computation processing unit is configured to input the at least one set of training input parameters into the neural network to perform parallel computation processing using the graphics processor, thereby obtaining at least one set of first output parameters corresponding to the at least one set of training input parameters. The constraint processing unit is configured to obtain at least one set of second output parameters corresponding to the at least one set of first output parameters, wherein the at least one set of second output parameters is obtained by adding boundary condition constraints and initial condition constraints to the at least one set of first output parameters, wherein the boundary condition constraints indicate the constraint situation of the target system on the boundary of a preset region, and the initial condition constraints indicate the initial state of the target system; The loss value calculation unit is configured to obtain the loss value of the neural network for the at least one set of first output parameters, calculated by the loss function corresponding to the neural network, based on the at least one set of second output parameters. The adjustment unit is configured to adjust the parameters of the neural network based on the loss value; The target system includes a fluid dynamics system, and the neural network is a neural network based on prior physical knowledge. The neural network is trained to simulate the motion of fluid particles in the fluid dynamics system. The motion simulation of fluid particles includes liquid surface simulation. The fluid dynamics system contains multiple fluid particles, each fluid particle corresponding to a set of training input parameters. Each set of training input parameters is used to characterize the label information and time information of the corresponding fluid particle from a Lagrange perspective. Each set of training input parameters corresponds to a first set of output parameters, which includes the position information of the fluid particle.

14. A data processing apparatus, comprising: The input acquisition unit is configured to acquire at least one set of input parameters corresponding to the target system. The processing unit inputs the at least one set of input parameters into the neural network to obtain at least one set of output parameters; The neural network is at least partially trained according to the training method described in any one of claims 1-11.

15. An electronic device comprising: Memory stores computer-executable instructions non-transiently; The processor is configured to run computer-executable instructions. The computer-executable instructions are executed by the processor to perform the training method of the neural network according to any one of claims 1-11 or the data processing method according to claim 12.

16. A non-transitory computer-readable storage medium, wherein, The non-transitory computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the training method of the neural network according to any one of claims 1-11 or the data processing method according to claim 12.

Images