Molecular system steady state solving method based on depth equilibrium model
Through the deep equilibrium model and the Geometric Diffusion Bridge framework, the problem of deep learning difficulties and low accuracy in the evolution of geometric states of molecular systems is solved, and efficient and accurate molecular steady-state conformation solution and trajectory generation are achieved, which is significantly better than the present There are methods, especially in complex scenarios, to improve prediction efficiency and accuracy.
Patent Information
- Application Number
- CN202411108060.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-13
AI Technical Summary
Deep learning is difficult to learn when capturing the evolution of geometric states of molecular systems and has low accuracy. Machine learning force field prediction is heavily dependent on the availability and quality of intermediate labels when it is based on the force in the current geometric state of the system, and its application scenarios are limited.
A method for solving molecular system steady-state based on deep equilibrium model is proposed. The Geometric Diffusion Bridge (GDB) framework is adopted to construct data sets, preprocess samples, build SE(3) and other variable neural network models, train and minimize the target function. , simulated stochastic differential equations to generate molecular steady-state conformations that satisfy specific physical symmetry constraints.
Accurate and efficient molecular steady-state conformation solution and trajectory generation are achieved, which is significantly better than existing methods, especially in complex scenarios, improving prediction efficiency and accuracy, and enhancing the ability to capture the dynamics of geometric state evolution.
Smart Images

Figure CN119007843A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and specifically is a method for solving the steady-state of a molecular system based on a deep equilibrium model. Background Art
[0002] Capturing the evolution of a system's geometric state is crucial for multiple scientific fields. Typically, the geometric state describes the position of every object in the system at a specific time. As the system evolves, certain geometric states of interest, such as equilibrium states, can be determined. Predicting these specific states from the initial state is crucial to revealing the behavior and functionality of the system, thus providing valuable insights into important problems such as drug discovery, reaction modeling, and catalyst analysis in practice.
[0003] Despite its critical importance, accurately predicting the future geometric state of interest from the initial state of a complex system is non-trivial. Experimental approaches, while valuable for direct observation, often face obstacles under stringent environmental conditions and physical limitations of the instrument. Computational approaches attempt to simulate the dynamical evolution by solving Newton's equations of motion and obtaining the equilibrium state through iterative geometry optimization. While offering greater flexibility, such calculations are typically driven by first-principles methods or empirical laws, which either require significant computational costs or sacrifice accuracy.
[0004] In recent years, deep learning has become a key tool for scientific discovery in many fields, providing a new approach to the challenge of capturing the evolution of geometric states. A straightforward approach is to train a neural network to predict the target geometric state directly from the initial state. By appropriately encoding the intrinsic symmetries of the geometric state, equivariant networks are usually adopted and end-to-end supervised training is performed on paired data of the initial and target states. After the model is trained, its one-time forward propagation can significantly improve the prediction efficiency compared with traditional methods. However, this paradigm also forces the neural network to compress the iterative evolution into one step, making the learning process difficult and potentially leading to reduced accuracy. Another research route is to predict the force based on the current geometric state of the system through machine learning force fields (MLFFs). The trained MLFFs can simulate the evolution and facilitate geometry optimization to determine the target geometric state, showing a better efficiency-accuracy balance than first-principles methods. However, MLFFs heavily rely on the availability and quality of intermediate labels (such as potential energy) to ensure stability and accuracy, which limits their application in complex scenarios. Summary of the invention
[0005] Purpose of the invention: In order to solve the problems of difficult learning process and low accuracy when using deep learning to capture the evolution of geometric states, as well as the serious dependence on the availability and quality of intermediate labels (such as potential energy) and limited application scenarios when predicting forces based on the current geometric state of the system through machine learning force fields (MLFFs), the present invention proposes a method for solving the steady-state of molecular systems based on a deep equilibrium model, which uses a modified bridge matching framework to generate molecular steady-state conformations that meet specific physical symmetry constraints, named GDB (Geometric Diffusion Bridge) framework, which is used for accurate and efficient molecular steady-state conformation solution and trajectory generation.
[0006] Technical solution: A method for solving the steady-state of a molecular system based on a deep equilibrium model, comprising the following steps:
[0007] Constructing a data set, wherein the data in the data set are paired data of initial conformation-steady-state conformation; dividing the data in the data set into training samples and test samples;
[0008] The training samples are first sampled from time t to U(0,1), where U(0,1) represents a uniform distribution on the interval (0,1), and then the path probability X of the Brownian bridge is calculated. t ~N(X0+(X1-X0)t,σt(1-t)) to obtain the preprocessed training samples; where X t ∈R d represents the conformation at time t in the dynamic evolution, σ represents the noise scale, X0 and X1 are random variables representing the initial conformation and the steady-state conformation, respectively;
[0009] Construct SE(3) equivariant neural network models based on graph Transformer or spherical harmonics representation;
[0010] The SE(3) equivariant neural network model is trained using the preprocessed training samples until the SE(3) equivariant neural network model converges, thereby obtaining a trained SE(3) equivariant graph neural network model.
[0011] First, sample from the initial conformation distribution Π0 to generate a rough conformation; then sample from the joint distribution Π0,1 by simulating the following SDE, and obtain X1 at t=1, which is the desired steady-state conformation:
[0012] dX t =v θ (X t ,X0,t)dt+σdB t ,X0~Π0
[0013] In the formula, v θ (X t,X0,t) represents a time-dependent vector field of the trained SE(3) equivariant graph neural network model, Π0 represents the initial conformation distribution, X t ∈R d represents the conformation at time t in the dynamic evolution, B t It is a standard Brownian motion.
[0014] Furthermore, the training process of the SE(3) equivariant neural network model is equivalent to minimizing the following objective function L GDB (θ):
[0015]
[0016] Among them, P 1|t For a given X t The conditional distribution of X1 when Indicates X t Find the gradient.
[0017] Furthermore, the Brownian bridge is expressed as:
[0018]
[0019] The present invention also discloses a method for solving the steady state of a molecular system based on a deep equilibrium model, comprising the following steps:
[0020] Construct a data set, wherein the data in the data set includes several frames of trajectory data, and the trajectory data includes N frames (X0, X 1 / N ,...,X1),X i ∈R 3n ;
[0021] Construct SE(3) equivariant neural network models based on graph Transformer or spherical harmonics representation;
[0022] The SE(3) equivariant neural network model is trained using the following loss functions on each frame;
[0023]
[0024] After training, the following SDE is simulated by framing from the joint distribution Π 0,1 The obtained X1 is the desired steady-state conformation:
[0025]
[0026] Wherein, [Nt] indicates taking the integer part.
[0027] Beneficial effects: Compared with the prior art, the present invention has the following advantages:
[0028] (1) This invention introduces a novel and systematic generative modeling approach for bridging geometric states. From a probabilistic point of view, the evolution of geometric states actually reflects the transmission of their distribution. This invention establishes a GDB framework, which represents a predetermined diffusion process, with the initial and target geometric states as fixed endpoints, and is controlled by a rotation-translation equivariant transformation kernel, ensuring that the evolution follows intrinsic symmetries. This is achieved through Doobh-transformations, which simulate the evolution of geometric states and provide a new way to accurately and efficiently bridge geometric states;
[0029] (2) The GDB framework proposed in the present invention improves performance by modeling the joint state distribution of geometric states, accurately connecting geometric states, and fully utilizing available trajectories as a detailed description of dynamics. Mathematically, the present invention has proved that the joint distribution of geometric states between different time steps can be fully maintained through the (chained) equivariant diffusion bridge technique, which confirms its expressive power in connecting geometric states and emphasizes the importance of design choices made in the GDB framework proposed in the present invention. In addition, under mild and practical assumptions, the present invention has proved that in terms of convergence, the GDB framework proposed in the present invention can approximate the underlying dynamics of the evolution of the control geometric state trajectories with negligible errors, illustrating the completeness and practicality of the GDB framework proposed in the present invention in different scenarios. These advantages show the superiority of the GDB proposed in the present invention over existing methods;
[0030] (3) To verify its effectiveness and universality, the present invention has conducted extensive experiments covering different data modalities (simple molecules and adsorbent-catalyst complexes), scales (small, medium and large scales) and scenarios (with and without trajectory guidance). Numerical results show that the GDB framework proposed in the present invention consistently and significantly outperforms existing state-of-the-art machine learning methods. In particular, the present invention method even surpasses a strong machine learning force field baseline with more than 10 times the training data in the challenging structural relaxation task in OC22, while trajectory guidance can further enhance the performance of the present invention method. The significantly superior performance demonstrates the powerful ability of the GDB framework proposed in the present invention in capturing the complex evolutionary dynamics of geometric states, as well as its ability to identify valuable and critical geometric states of interest in key real-world challenges. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 Schematic diagram of the architecture of the neural network adopted in the present invention. DETAILED DESCRIPTION
[0032] The technical solution of the present invention is now further described in conjunction with the accompanying drawings and embodiments.
[0033] This embodiment proposes a method for solving the steady-state of a molecular system based on a deep equilibrium model. For a rough molecular conformation that is easily obtained, an equivariant graph neural network is trained by building a bridge, so that the trained equivariant graph neural network can iteratively repair this rough molecular conformation, thereby generating a molecular steady-state conformation that satisfies specific physical symmetry constraints. The framework is named GDB (Geometric Diffusion Bridge), and the molecular steady-state generated with higher accuracy can be used for efficient molecular steady-state solution and trajectory generation.
[0034] The bridge matching framework is constructed by constructing a pair with path probability Π = Π0Π1P (·|0,1) The associated stochastic differential equation (SDE), where P (·|0,1) represents a bridge process, Π0 represents the initial rough conformation distribution, also called the initial distribution; Π1 represents the steady-state conformation distribution that needs to be sampled, also called the terminal distribution. Note that in Π, the initial distribution and the terminal distribution are decoupled. Consider the coupled data (X0, X1)~Π 0,1 , where X0 and X1 represent random variables of initial conformation and steady-state conformation, respectively, which means that the initial and terminal distributions are not independent. The bridge matching framework should construct a random variable with path probability Π=Π 0,1 P (·|0,1) The associated SDE. Previous studies have found that the bridge matching framework cannot maintain the invariance of the coupled distribution. That is, the joint distribution of (X0,X1) is usually not Π 0,1 In order to solve this problem, the GDB framework of this embodiment constructs a path probability Π by fixing the initial point X0. (·|0) =Π (1|0) P (·|0,1) The associated stochastic differential equation, where Π (·|0) represents the conditional probability path distribution for a given initial point, Π (1|0) represents the conditional distribution of the steady-state conformation given an initial point.
[0035] Assume that the path probability P is given by the following SDE:
[0036] dX t =f(X t ,t)dt+σ(t)dB t
[0037] Among them, X t ∈R d represents the conformation at time t in the dynamic evolution, B t It is a standard Brownian motion.
[0038] According to the famous conclusion Doob h-transform in probability theory, the path probability P of a given starting point and end point can be obtained: (·|0,1) , which is given by the following SDE:
[0039]
[0040] For a given X0,X t Conditional Function The conditional expectation of Indicates X t Find the gradient.
[0041] The path probability P can be designed by ourselves. In this embodiment, we use f(X t ,t)=0, using neural network v θ (X t ,X0,t) to approximate The training process of the neural network is equivalent to minimizing the following objective function, namely:
[0042]
[0043] Among them, P 1|t For a given X t The conditional distribution of X1 when .
[0044] The optimal value of the neural network satisfies:
[0045]
[0046] After the training process is completed, the joint distribution Π can be obtained by simulating the following SDE 0,1 The obtained X1 is the desired steady-state conformation:
[0047] dX t =v θ (X t ,X0,t)dt+σdB t ,X0~Π0
[0048] Neural network is the v in SDE θ (X t ,X0,t), after training, we need to sample from the initial distribution Π0, that is, generate a rough conformation. Then simulate this SDE, and finally simulate to the time t=1 to obtain X1, which is a steady-state conformation. It can be seen that the process of sampling (or simulating SDE) is the process of generating a steady-state conformation.
[0049] In addition, the neural network should also take into account the symmetry of the system. Obviously, the likelihood of the molecule should not change under rotation or translation, so its distribution is SE(3) invariant. In order to satisfy this property, the vector field should satisfy some restrictions. In fact, if the prior distribution Π0 is SE(3) invariant and the transition kernel Π t|0 (X t |X0) is SE(3) equivariant, then the marginal distribution Π at any time t It is also SE(3) invariant.
[0050] If there is trajectory information in the dataset for training the neural network model, you can choose to use or not use trajectory information for training. If trajectory information is used for training, the effect will be better. However, since trajectory information is more expensive, many datasets do not have trajectory information.
[0051] The method for generating a molecular steady-state conformation without using trajectory information includes the following steps:
[0052] Step 1: Construct a data set, where the data in the data set are pairs of rough conformation and precise conformation; divide the constructed data set into training samples and test samples;
[0053] Step 2: First, sample the training samples in time from t to U(0,1), where U(0,1) represents a uniform distribution on the interval (0,1), and then follow the path probability X of the Brownian bridge. t ~N(X0+(X1-X0)t,σt(1-t)) for sampling, where X t ∈R d represents the conformation at time t in the dynamic evolution;
[0054] Step 3: Construct a neural network based on graph Transformer or spherical harmonics representation, such as Figure 1 As shown, the neural network of this embodiment is v in SDE θ (X t ,X0,t) item.
[0055] Step 4: Use the back propagation algorithm to train the neural network using the AdamW optimization algorithm.
[0056] The Brownian bridge representation used in this embodiment is:
[0057]
[0058] The loss function is expressed as:
[0059]
[0060] Among them, v θ (X t,X0,t) is a time-dependent vector field parameterized by the neural network θ, P 1,t Represents X t The joint distribution of X0.
[0061] Step 5: Use the Rdkit tool to generate a rough three-dimensional conformation for a molecule's 2D structure or chemical formula. Generate a steady-state conformation by simulating the following SDE. The obtained X1 is the desired steady-state conformation:
[0062] dX t =v θ (X t ,X0,t)dt+σdB t ,X0~Π0
[0063] Among them, v θ (X t ,X0,t) is a time-dependent vector field parameterized by a trained neural network θ, and Π0 represents the initial conformation distribution.
[0064] The above process can be simplified as follows:
[0065] The training mode of Algorithm2 without using trajectory information is as follows:
[0066] repeat;
[0067] (z0,z1)~Π 0,1 (X0,X1);
[0068] t~U(0,1);
[0069] ε~N(0,I);
[0070]
[0071] Gradient descent optimization step
[0072]
[0073] The method for generating a molecular steady-state conformation using trajectory information includes the following steps:
[0074] The dataset provides trajectory data consisting of several frames, including N frames (X0, X 1 / N ,…,X1),X i ∈R 3n . Assume that the true path measure is a Markov path probability. So the true path probability can be decomposed into:
[0075]
[0076] Construct SE(3) equivariant neural network models based on graph Transformer or spherical harmonics representation;
[0077] The SE(3) equivariant neural network model is trained using the following loss functions on each frame;
[0078]
[0079] After training, the following SDE is simulated by framing from the joint distribution Π 0,1 The obtained X1 is the desired steady-state conformation:
[0080]
[0081] Wherein, [Nt] indicates taking the integer part.
[0082] The above process can be simplified as follows:
[0083] The training mode of Algorithm3 using trajectory information is as follows:
[0084] repeat;
[0085] (z0,…,z N )~Π 0,…,1 (X0,…,X1);
[0086]
[0087] ε~N(0,I);
[0088]
[0089] Gradient descent optimization step
[0090]
[0091] The parameters required for the experiments are briefly provided below. For the Molecule3D and QM9 datasets, the bridge SDE is parameterized by extending the graph Transformer-based equivariant network to simultaneously encode the conditions of the time step and the initial geometric state.
[0092] During training, AdamW was used as the optimizer, and the hyperparameters ∈ were set to 1e-8 and (β1, β2) were set to (0.9, 0.999). The gradient clip norm was set to 5.0. The peak learning rate was set to 1e-4. The batch size was set to 512. The weight decay was set to 0.0. The model was trained for 500k steps, including a 30k step warm-up phase. After the warm-up phase, the learning rate decayed linearly to zero. The noise scale σ was set to 0.5. During inference, 10 time steps were used and the Euler solver was used. All models were trained on 16 NVIDIA V100 GPUs. The training process iterated 300 rounds with a batch size of 128, that is, each batch of 128 images; the initial value of the learning rate was 0.1, the learning rate was 0.01 from rounds 151 to 225, and the learning rate was 0.001 from rounds 226 to 300.
[0093] For the OC22 dataset, GemNet-OC is used to parameterize the bridge SDE, which also proves that the framework proposed in this embodiment is compatible with different base models. For training, AdamW is used as the optimizer, and the hyperparameters ∈ are set to 1e-8 and (β1, β2) are set to (0.9, 0.999). The gradient clipping norm is set to 10.0. The peak learning rate is set to 5e-4. The batch size is set to 64. The weight decay is set to 0.0. The model is trained for 200k steps. After the warm-up phase, the learning rate decays linearly to zero. The noise scale σ is set to 0.5. The trajectory length is set to N=10. At inference time, 10 time steps are used and the Euler solver is used. All models are trained on 8 NVIDIA A100 GPUs.
[0094] The following are the experimental results on the Molecule3D, QM9, and OC22 datasets:
[0095]
[0096]
[0097] Model ADwT index (%)↑ IS Baseline 44.77 SpinConv iterative solution 54.53 GemNet-dT iterative solution 59.68 GemNet-OC iterative solution 60.69 GemNet-OC direct prediction 60.44 GDB does not use trace information 62.4 GDB uses trace information 63.0
Claims
1. A method for solving the steady-state of a molecular system based on a deep equilibrium model, characterized by: The following steps are involved: Constructing a data set, wherein the data in the data set are paired data of initial conformation-steady-state conformation; dividing the data in the data set into training samples and test samples; The training samples are first sampled from time t to U(0,1), where U(0,1) represents a uniform distribution on the interval (0,1), and then the path probability X of the Brownian bridge is calculated. t ~N(X0+(X1-X0)t,σt(1-t)) to obtain the preprocessed training samples; where X t ∈R d represents the conformation at time t in the dynamic evolution, σ represents the noise scale, X0 and X1 are random variables representing the initial conformation and the steady-state conformation, respectively; Construct SE(3) equivariant neural network models based on graph Transformer or spherical harmonics representation; The SE(3) equivariant neural network model is trained using the preprocessed training samples until the SE(3) equivariant neural network model converges, thereby obtaining a trained SE(3) equivariant graph neural network model. First, sample from the initial conformation distribution Π0 to generate a rough conformation; then simulate the following SDE from the joint distribution Π 0,1 Sampling from the middle, we get X1 at t=1, which is the desired steady-state conformation: dX t =v θ (X t ,X0,t)dt+σdB t ,X0~Π0 In the formula, v θ (X t ,X0,t) represents a time-dependent vector field of the trained SE(3) equivariant graph neural network model, Π0 represents the initial conformation distribution, X t ∈R d represents the conformation at time t in the dynamic evolution, B t It is a standard Brownian motion.
2. The method for solving the steady-state of a molecular system based on a deep equilibrium model according to claim 1, characterized in that: The training process of the SE(3) equivariant neural network model is equivalent to minimizing the following objective function L GDB (θ): Among them, P 1|t For a given X t The conditional distribution of X1 when Indicates X t Find the gradient.
3. The method for solving the steady-state of a molecular system based on a deep equilibrium model according to claim 1, characterized in that: The Brownian bridge is represented as:
4. A method for solving the steady-state of a molecular system based on a deep equilibrium model, characterized in that: The following steps are involved: Construct a data set, wherein the data in the data set includes several frames of trajectory data, and the trajectory data includes N frames 9X0,X 1 / N ,...,X1),X i ∈R 3n ; Construct SE(3) equivariant neural network models based on graph Transformer or spherical harmonics representation; The SE(3) equivariant neural network model is trained using the following loss functions on each frame; After training, the following SDE is simulated by framing from the joint distribution Π 0,1 The obtained X1 is the desired steady-state conformation: Wherein, [Nt] indicates taking the integer part.
Citation Information
Patent Citations
Machine learning force field development data set sampling method
CN115169454A
Molecular generation method and device based on Bayesian flow network
CN118471369A
Action Detection Using Machine Learning Models
US20230360441A1
Trained machine learning model for forecasting molecular conformations
US20240249800A1
