Balanced distribution prediction for molecular systems

Abstract

A computing system for predicting an equilibrium distribution for a molecular system includes a processor that executes instructions using a portion of an associated memory to implement an equilibrium distribution prediction model. In an inference phase, the processor is configured to receive input data representing the molecular system, create a graphical representation of the molecular system, including location information for each atom in the molecular system, and input the graphical representation of the molecular system into a graphical neural network. The processor is further configured to receive as an output a plurality of predicted conformations of the molecular system from the graphical neural network, input each of the plurality of predicted conformations into a diffusion model, predict an equilibrium distribution for the conformations of the molecular system, and output the equilibrium distribution.

Description

BACKGROUND

[0001] In the field of computational chemistry, computer-based techniques have been developed to predict molecular properties through computer simulation. These molecular properties can have wide-ranging effects on the appearance and function of molecules or substances, and are therefore of intense interest in a variety of fields. For example, in the field of drug design, changes in molecular properties can affect the efficacy of a drug. In the field of drug discovery, molecular properties can affect the potential of a natural substance for therapeutic purposes. In the field of quantum chemistry, quantum mechanical calculations of the electronic contributions to the physical and chemical properties of molecules and substances are a fundamental area of research. As described below, there remains an opportunity to improve computational methods for predicting molecular properties, which will have useful applications far beyond the field of computational chemistry. SUMMARY

[0002] To address the problems discussed herein, a computing system and method for predicting an equilibrium distribution for a molecular system are provided. In one aspect, the computing system includes a processor that executes instructions using portions of an associated memory to implement an equilibrium distribution prediction model. In an inference phase, the processor is configured to receive input data representing a molecular system and create a graph representation of the molecular system, including position information for each atom in the molecular system. The processor is further configured to input the graph representation of the molecular system into a graph neural network and receive, as output from the graph neural network, a plurality of predicted conformations of the molecular system. The processor is further configured to input each of the plurality of predicted conformations into a diffusion model, predict an equilibrium distribution for the molecular system, and output the equilibrium distribution.

[0003] This summary is provided to introduce some concepts of the following in a simplified form. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all of the disadvantages mentioned in any portion of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0004] Figure 1 A schematic diagram of a computing system for predicting an equilibrium distribution for a molecular system is shown, in accordance with one embodiment of the present disclosure.

[0005] Figure 2A And Figure 2B A training phase of an equilibrium distribution prediction model of the computing system of Figure 1 is shown.

[0006] Figure 3 An example free energy landscape of an equilibrium distribution predicted by the computing system of Figure 1 is shown.

[0007] Figure 4 Inference phase of a trained balanced distribution prediction model of a computing system. Figure 1 Inference phase of a trained balanced distribution prediction model of a computing system.

[0008] Figure 5 A flowchart of a method for predicting a balanced distribution for a molecular system is shown in accordance with example implementations of the present disclosure.

[0009] Figure 6 An example computing environment in which embodiments of the present disclosure can be implemented is shown. DETAILED DESCRIPTION

[0010] With the increase in deep learning models, the prediction of molecular properties of molecular systems is rapidly advancing, leading to significant milestones, such as accurate protein structure prediction from amino acid sequences. However, the distribution of possible conformations of a molecular system (i.e., the energy landscape in the conformational space of a molecular system), which is more relevant to macroscopic properties than the single conformation of the lowest energy state of a molecular system, is less explored. Therefore, developing computational tools for predicting the microscopic states of molecules in the equilibrium ensemble would provide significant progress in understanding the properties and functions of molecular systems and enable the evaluation of macroscopic properties using statistical mechanical methods.

[0011] However, developing an efficient sampling method to predict the balanced distribution of a molecular system is computationally challenging. With the rapid progress of deep learning techniques, data-driven approaches have been successfully used to predict molecular structures corresponding to the lowest energy of a molecular system and have made breakthroughs in this field. For example, artificial intelligence (AI) structure prediction models have improved protein structure prediction accuracy to the atomic level, driving successful applications in structural biology. Fast computational docking methods based on deep neural networks have enabled ligand binding structure prediction, supporting fast virtual screening and drug engineering. Deep learning models have been designed to predict the relaxed structure of adsorbates on a catalyst surface. Although each of these advances indicates that deep learning provides a viable solution for understanding microscopic structures and states, the accurate prediction of the lowest energy structure of a molecular system only reveals a small part of the information required for an equilibrium molecular system. In fact, when a system reaches equilibrium, molecules can adopt different structures with probabilities controlled by an energy function. This balanced distribution is critical for studying statistical mechanical properties. For example, the properties and functionality of biological molecules can be inferred from various structures with different energies. In addition, determining the equilibrium distribution is required to calculate the free energy associated with many important macroscopic properties.

[0012] A computational method that directly and efficiently predicts the equilibrium distribution of conformational states of a molecular system from elementary descriptors (e.g., the amino acid sequence of a protein) would greatly enhance the understanding of the function and properties of the molecular system. However, for such a model to be practically useful, it would need to support independent sampling, which allows for efficient exploration of the microstates separated by energy barriers, such as in a conformational space with a complex energy landscape, and it would need to accurately predict the density states to efficiently compute the free energy of states of interest. Independent sampling overcomes the inherent difficulties in sequential sampling methods (e.g., Markov chain Monte Carlo (MCMC) or molecular dynamics (MD)) that result in lower sampling efficiency due to the non-identically distributed and correlated samples. Predicting the equilibrium distribution over all conformations is technically more challenging than predicting a single conformation corresponding to the most stable (e.g., lowest energy) state, especially when the distribution is highly complex due to anisotropy and multiple local optima.

[0013] Design Principles

[0014] In view of the problems discussed above, a computational system that utilizes an equilibrium distribution prediction model is provided. The computational system is suitable for predicting equilibrium distributions for molecular systems, as well as distributions for other types of systems that can be represented as graphs. The following discussion provides an overview of the theoretical foundations and design principles that the equilibrium distribution prediction model has been conceived. This discussion is followed by a detailed description of specific example embodiments of the equilibrium distribution prediction model.

[0015] Framework

[0016] Deep neural networks have been shown to be able to predict accurate molecular structures from descriptors of molecular systems . The equilibrium distribution prediction model described herein predicts the equilibrium distribution over the state space. To achieve this, the equilibrium distribution prediction model is based on a diffusion model: when the system is progressively down-scaled and injected with noise, the equilibrium distribution eventually becomes a standard Gaussian distribution, making it easier to predict and thus enabling efficient sampling. A forward diffusion process is implemented to reach the Gaussian distribution, followed by a subsequent reverse diffusion process to gradually recover the original equilibrium distribution.

[0017] Methods from the past that predict equilibrium distributions from simple distributions (e.g., simulated annealing using Monte Carlo methods and simulated annealing using Langevin dynamics) have the disadvantage of being computationally complex, slow to implement, and costly. As described herein, these challenges of predicting the equilibrium distribution of a molecular system can be reduced by formulating it as a stochastic process based on forward and reverse diffusion processes.

[0018] The forward and reverse diffusion processes can be viewed as a pair of mutually inverse stochastic processes that simulate a transformation between the equilibrium distribution and a simple distribution p simple that is independent of the system. The molecular system descriptors the equilibrium distribution as the initial distribution A stochastic process is constructed for it by gradually transforming p through a forward process simple Then, a corresponding backward process transforms p simple into the equilibrium distribution This defines the generation process of the equilibrium distribution prediction model.

[0019] For practical simulation, a deep neural network is trained through the forward process to predict the backward process from a given system descriptor Instead of predicting the target distribution directly from the system descriptor , the mutually inverse diffusion method described herein significantly reduces the problem difficulty. Since p simple is chosen to be independently samplable and has a closed-form density function, the equilibrium distribution prediction model can achieve independent sampling of the equilibrium distribution by simulating the backward process from p simple , and also provide the density function of the distribution by tracking the simulation process. Specifically, p is chosen to be a standard Gaussian distribution. The forward process is constructed as Langevin dynamics (Ornstein-Uhlenbeck process) with time-varying scaling coefficients β t that increase incrementally, whose stochastic differential equation (SDE) is shown in equation (1): where B t is a standard Brownian motion (i.e., Wiener process). The p simple generated by this forward process is more concentrated than the heating distribution in the heating-cooling simulation, and thus it is easier to obtain high-density samples. Therefore, the diffusion-based generative model described herein achieves efficient training and sampling with highly accurate results.

[0020] According to the theory of stochastic processes, the backward process is also a stochastic process that follows a reversed-time Markov step chain, whose SDE is shown in equation (2):

[0021] where t is the reversed time, is the distribution of the forward process corresponding to time t, is the Brownian motion in the reversed time. To recover simple from p by simulating this backward process, a scoring model is constructed using a deep neural network This scoring model is trained to predict the true scoring function t of each instantaneous distribution q

[0022] Since the model is defined on the conformation space, the trained graph neural network is implemented as a backbone of the equilibrium distribution prediction model that can predict the conformation of a molecular system, thus enabling molecular structure modeling and generalization across systems. The model Can describe descriptors for unseen molecular systems Generate equilibrium samples R0, i.e., simulate the inverse process of equation (2) for N+1 steps, where the time interval [0, τ] is uniformly discretized with step size h := τ / N, as shown in the following equation (3):

[0023] where index i corresponds to time t = ih, and β i := hβ t＝ih . See, Euler-Maruyama discretization of equation (2) directly gives with o(h) order discretization error. To have accurate discretization, h is small, so each β i is also small. Thus, the approximation relation and is obtained, which gives equation (3) without adding o(h) order discretization error. Since R N samples can be generated independently, the equilibrium distribution prediction model can produce statistically independent and identically distributed (IID) samples from the equilibrium distribution. This approach is more efficient than traditional simulation or sampling methods.

[0024] Fine-tune the equilibrium distribution prediction model with data

[0025] The score model can be trained from data by minimizing the score matching for each t. To produce a tractable loss function, first reformulate the intermediate edge distribution using the edge kernel. Then, the score function can be decomposed as:

[0026] i.e., Fisher's identity. The score matching loss becomes

[0027] where the second term in the last expression is a constant independent of θ. Optimizing the score matching loss is equivalent to minimizing the first term in the last expression. This term is shown in the following auxiliary equation (S1),

[0028] known as denoising score matching loss.

[0029] Once the conditional distribution q(R i | R0) is known, optimizing the denoising score matching loss formula (S1) is tractable, which fortunately leads to a closed-form for the forward process formula (1). In continuous time, previous literature gives the result: where and For the discretized expression, the review time interval [0, τ] is evenly divided into N + 1 points with step size h = τ / N, the step size i corresponds to the time t = ih, and β i := hβ t＝ih . From this, we get Correspondingly, Then, the required conditional distribution is: Then, the loss equation (S1) at time step i becomes Using the reparameterization of the Gaussian distribution q(R i | R0), where the loss is further reformed as: To balance the scales of the losses for different i ∈ {1, …, N}, the reference will scale the loss formula (S1) at step i by This ultimately leads to the data-based training loss: It should be understood that in this loss, the term for each i loss can be calculated independently and backpropagated with constant cost, without relying on the calculation before or after the step index i. This enables another efficient step-by-step supervision mode.

[0030] Physical information diffusion pre-training

[0031] Equilibrium distribution prediction models can be trained on a series of conformational data samples of molecular systems. However, it can be very expensive to collect sufficient experimental data and / or simulation data to well characterize the equilibrium distribution of various systems. To solve the problem of data scarcity, a pre-training algorithm (referred to as physical information diffusion pre-training (PIDP) in this paper) is implemented, which effectively optimizes the equilibrium distribution prediction model without data. Supervision comes from the energy function of each molecular system descriptor which is explicitly defined in the equilibrium distribution. It should be understood that the true score function The partial differential equation is called the Fokker-Planck equation. Thus, the scoring model is pre-trained by minimizing a loss function that enforces formula (4):

[0032] where λ1is the initial condition weighting, is a set of state points that evaluate the loss of step i. Ideally, are considered as points on a grid that spans the state space. But when the dimensionality of the state space is very high (like thousands for a protein), the number of grid points needed to maintain a constant discretization error grows exponentially, making this approach infeasible. However, the space of interest is a low-dimensional subspace composed of states with relatively low energy. Thus, the model can be effectively trained on these states as R0samples (for R i , samples from q(R i | R0) are drawn), which is much easier than obtaining data samples that follow the overall equilibrium distribution. In addition, stochastic estimators are used to reduce the complexity of computing high-order derivatives and derivatives of vector-valued functions. It should be understood that for each step i, the corresponding model obtains training loss that is independent of other steps and can be directly backpropagated. This step-wise supervised mode better leverages the design of the equilibrium distribution prediction model and enables effective pre-training.

[0033] Computing free energy

[0034] In addition to generating IID samples, the equilibrium distribution prediction model can also evaluate the density function of the distribution it defines, as shown in formula (6):

[0035] where D is the dimensionality of the state space, is the solution of an ordinary differential equation (ODE) shown in formula (7):

[0036] with initial condition R0, can be solved using an ODE solver. This provides another perspective of the distribution and enables more calculations related to the distribution to be called. For example, the free energy of a molecular system descriptor can be estimated using the following formula:

[0037] where the expectation can be estimated by using independent samples that can be easily generated using the equilibrium distribution prediction model. When is unknown, the free energy difference can still be computed by replacing with where is a density estimator (e.g., kernel density estimator) constructed using data samples that follow a Boltzmann distribution, because the partition functions cancel each other out in the computation. In contrast to traditional free energy computation methods, the equilibrium distribution prediction model does not rely on a harmonic approximation of the energy function near metastable states, and its IID sample generation achieves faster sample coverage over the relevant state space than MD. Moreover, because the equilibrium distribution prediction model also provides a density function, it enables the model to correct biases in the predicted distribution: where is the set of samples generated by and It should be understood that the normalization density function is not required after weighting over the normalized sample set.

[0038] To evaluate the density of the equilibrium distribution prediction model, it is noted that the model defines a distribution through a series of distribution transformations from p simple , via a reverse diffusion process. In the continuous time limit, the instantaneous distribution transformation in the generalized diffusion process dR t = f t (R t )dt + g t dB t is given by the Fokker-Planck equation (FPE), whose logarithmic form is shown in the following supplementary equation (S2):

[0039] where p t (R t ) is the distribution density function that evolves continuously under the process, denotes the Laplacian operator of the (scalar-valued) function f. The reverse process defined by the model is equation (2), where is approximated by the score model If expressed in forward time t, it is the following supplementary equation (S3):

[0040] where is the reverse of the Brownian motion. Thus, its FPE is shown in the following supplementary equation (S4):

[0041] where is the density function of the reverse process defined by the model at time step t, is the equilibrium distribution defined by the model. When the model is sufficiently learned, approximates and approximates Thus Also, the approximation can be used This transforms equation (S4) into the supplementary equation (S5):

[0042] This represents, in comparison to the FPE equation (S2), Also, it can be seen as dominated by the equivalent “deterministic diffusion process” defined by equation (7). Let R t be the solution of equation (7), which is a (deterministic) curve in the state space, then the total derivative with respect to time t (also called material / particle derivative) is:

[0043] In comparison to equation (S5), we obtain:

[0044] By integrating with respect to t, we obtain:

[0045] which is consistent with equation (6)

[0046] Scoring model At each time step, the goal is to match the corresponding score function According to the FPE equation (S2), the score function satisfies the differential equation:

[0047] Taking the gradient of the above equation gives:

[0048] becomes the equation for the score function To approximate Scoring model It is also required to satisfy this equation. To implement it, following the physics-informed neural network approach, the squared norm of the equation residual at each time step t is minimized:

[0049] Note that and The loss of the noise prediction model at time step t becomes:

[0050] To combine the loss terms of different time steps, the loss weight at time step t is set to be consistent with the data-based loss function, i.e., weighted by the inverse scale of The first term in equation (4) is obtained by time discretization and evaluating the loss on a set of samples.

[0051] Alternatively, for a weighting scheme with different t, the weight of time step t is inversely proportional to:

[0052] As long as each is normalized, FPE has no or no need of boundary conditions. For the initial condition, the score of the target equilibrium distribution is exactly given by the gradient of the energy function of the system, which enables the energy function to supervise the model. To achieve this initial condition, is minimized, and this loss is not imposed on discrete step i = 0, but on i = 1. This leads to the second term in equation (4).

[0053] Inverse design with equilibrium distribution prediction model

[0054] Another benefit of the equilibrium distribution prediction model is that it can directly predict the distribution conditioned on a given microscopic property c without the need to generate data of the conditional distribution and train the model again. This is necessary for the increasingly demanded inverse design task, where a state with a specific property, such as ground state energy gap and conductivity, is needed. From the data generation process of equation (2), all that needs to be adjusted is to convert the score function from to By Bayes' rule, the latter can be restructured as where the first term can be approximated by the learned (unconditional) score model. Therefore, only an extra model (i.e. property predictor / classifier) is needed, which is much easier to train than the generative model.

[0055] Interpolation between states

[0056] The equilibrium distribution prediction model can also reveal the reaction path between two given states, which can be used to discover the reaction coordinate or collective variable and find the intermediate metastable states in the transition. This is done by noting that the distribution transformation process behind the equilibrium distribution prediction model, equation (1), is equivalent to the process in equation (7) if is well approximated by This process is deterministic and reversible, thus establishing a correspondence between the real space and the latent space. One can map the two given states to the latent space, approximate the path in the latent space by linear interpolation, and map it back to the real space. Since the distribution in the latent space is Gaussian, which has convex contour lines, the linear interpolation path goes through the high probability / low energy region, thus providing an accurate prediction of the real reaction path.

[0057] Example Embodiments

[0058] In accordance with the principles discussed above, specific example embodiments of a balanced distribution prediction model in accordance with the present disclosure will now be described with reference to Figures 1 to 6

[0059] Reference is first made to Figure 1 The computing system 10 includes at least one computing device. The computing system 10 is shown as having a first computing device 14 including a processor 18 and a memory 22, and a second computing device 16 including a processor 20 and a memory 24. The illustrated implementation is exemplary in nature, and other configurations are possible. In the following description, the first computing device will be described as a server 14, and the second computing device will be described as a client computing device 16, and respective functions performed at each device will be described. It will be understood that in other configurations, the computing system 12 can include a single computing device performing the core functions of both the server 14 and the client computing device 16, and the first computing device can be a computing device other than a server. In other alternative configurations, functions described as being performed at the server 14 can instead be performed at the client computing device 16, and vice versa.

[0060] Continuing Figure 1 The processor 18 is configured to execute instructions using relevant portions of the memory 22 to implement a balanced distribution prediction model 26 deployed at the server 14. In general, the balanced distribution prediction model 26 predicts a conformational balanced distribution of a molecular system based on a score function (i.e., a gradient of a log of a density function) for each intermediate distribution within a predetermined range of time steps of a diffusion process by processing data 28 representing the molecular system. As described in detail below with reference to Figure 2A The data 28 representing the molecular system is a descriptor including at least one of an amino acid sequence a two-dimensional structure of the molecular system, and / or a chemical formula of the molecular system. Additionally or alternatively, the descriptor may be a three-dimensional structure of the molecular system, a protein name, a simplified molecular-input line-entry system (SMILES), etc. The data 28 can be stored in a database of molecular systems 30, such as UniProt, Swiss-Prot, Protein Research Foundation (PRF), etc. Upon receiving user input via a user interface 32 at the client computing device 16, the data 28 can be sent to a graph representation generator 34 included in the balanced distribution prediction model 26.

[0061] ​The graph representation generator 34 is configured to receive the input data 28. Based on the data 24 representing the molecular system, the graph representation generator 34 generates a graph representation 36 of the molecular system, including position information for each atom in the molecular system. As described below with reference to Figure 2A As described above, the pre-processing algorithm can be implemented to generate the graph representation. The graph representation 36 is then input to the graph neural network 38. The graph neural network 38 is configured to generate a plurality of predicted conformations 40 of the molecular system. The predicted conformations 40 can be stored at a predicted conformation database 42 as predicted conformation data 44.

[0062] Each predicted conformation 40 output from the graph neural network 38 is input to the diffusion model 46. As described in detail above and below, with reference to Figure 2A , 2B The equilibrium distribution 48 for the molecular system is predicted. Data 50 representing the predicted equilibrium distribution 48 can be stored in an equilibrium distribution database 52.

[0063] The equilibrium distribution 48 is output and can be viewed, for example, in a user interface 54 of a display 56 of the client computing device 16. In the embodiments described herein, it will be appreciated that the server 14 communicates with the client computing device 16 via a network 58.

[0064] Turning to Figure 2A and 2B which illustrate a training phase of the equilibrium distribution prediction model 26. As described above and as shown in Figure 2A Descriptors of the molecular system A graph representation 36 of the molecular system is generated by the pre-processing algorithm 60. The graph representation 36 includes a plurality of normal nodes connected by edges, as illustrated in the legend in Figure 2A Each normal node represents an atom in the molecular system. The graph representation 36 also includes one virtual node that is fully connected to all of the normal nodes of the graph representation 36 by virtual edges. It will be appreciated that the difference between the virtual node and the normal nodes is that the normal nodes represent atoms, while the virtual node is provided for computational purposes only and does not represent any physical component of the molecular system.

[0065] Based on the graph representation 36, the processor 12 of the computing system 10 is configured to provide a training data set 62 for the graph neural network 38, including a plurality of training data pairs, for example by computation or reading from a storage location in memory. Each training data pair includes a graph representation 36 of a molecular system and an energy parameter value 64 representing an energy change in the molecular system after an energy transformation (possibly due to, for example, molecular relaxation of the molecular system).

[0066] As described above, the graph neural network 38 is configured to generate a plurality of predicted conformations 40 of the molecular system, each predicted conformation being output from the graph neural network 38 and input to a diffusion model 46 as shown. Figure 2B The diffusion model 46 is a score-based generative diffusion model configured to introduce random noise at predetermined time steps based on a Gaussian distribution in a forward process and to reduce the noise in a reverse process. As discussed in detail in the “Framework” section, the forward process is constructed as Langevin dynamics (Ornstein-Uhlenbeck process) targeting p simple The diffusion model also implements a score model, an example of which is defined by equation (3) to predict a true score function for a plurality of distribution instances from the molecular system descriptors The score model can be trained by equation (4), for example, to minimize a loss function as described above with reference to the “Physical information scattering pre-training” section.

[0067] While the equilibrium distribution prediction model can be trained on a series of conformation data samples over a molecular system, it can be very expensive to collect sufficient experimental data and / or simulation data to fully characterize the equilibrium distribution of various systems. Therefore, the PIDP is implemented to optimize the equilibrium distribution prediction model without data. As described above, the diffusion model during training is supervised by the energy function of each molecular system descriptor according to the Boltzmann distribution and by molecular dynamics simulation data.

[0068] The diffusion model 46 is configured to vary the noise applied to the predicted conformations across a series of time steps, denoted as T1, T2, and T3 in Figure 2B The noise increases with time steps throughout the simulation. It will be appreciated that as the noise increases between time steps, there will be greater potential changes in the conformations predicted by the model during the reverse diffusion process. Through the progression of the series of time steps, the conformations of the molecular system will follow a simple distribution (such as a standard Gaussian distribution) such that during the reverse diffusion process, independent and identically distributed samples can be drawn and their density function evaluated, ultimately yielding the equilibrium distribution 48. The shaded regions of the equilibrium distribution 48 represent the free energy levels corresponding to different conformations of the molecular system, with the darkest shade indicating the stable region of lowest free energy. Figure 2B The two stable regions as shown can represent two different steady states of the molecular system, for example, the bound / unbound state conformations of an enzyme, or the open / closed states of a membrane channel.

[0069] Figure 3An exemplary free energy landscape of the equilibrium distribution 48 is shown. As described above, the shaded regions of the equilibrium distribution 48 indicate the free energy associated with different conformations of the molecular system. The darkest regions indicate regions with the lowest free energy, which correspond to metastable conformations of the molecular system, such as Figure 3 Conformation C2 is a semi-stable transient conformation of the molecular system, as it transitions between the metastable states of conformations CI and C3.

[0070] Figure 4 The inference phase of the equilibrium distribution prediction model 26 is shown in FIG. 4. As described in detail above with reference to Figure 1 、 Figure 2A and Figure 2B the equilibrium distribution prediction model 26 receives the molecular system descriptors The molecular system descriptors are used by the pre-processing algorithm 60 to create a graph representation 36 of the molecular system based on the molecular system descriptors, and the graph representation 36 is input into the diffusion model 46.

[0071] The equilibrium distribution prediction model 26 also contains an independent and identically distributed (IID) sampler 66 that performs conformation sampling of the equilibrium distribution 48 to produce statistically independent samples. One existing method of such statistical sampling uses a simulated annealing algorithm, which is a variant of the Metropolis-Hastings algorithm, to search the solution space and reach the target, complex, constructed distribution. Simulated annealing algorithms can be used to make multiple predictions from multiple initial states of the molecular system and result in a distribution of predicted conformations of a. The temperature is a parameter in the simulated annealing algorithm that controls how far in a direction away from the target (minimum or maximum) the algorithm is allowed to explore at any given time step. Typically, the temperature is lowered throughout the simulation to converge the solution. While useful, such existing simulated annealing algorithms are computationally complex, slow, and expensive to implement. This is due to the fact that the simulation of the distribution change is either approximate (if Markov chain Monte Carlo sampling is employed) or experiences particle degradation (if importance sampling is employed). In contrast to such simulated annealing methods, to avoid such computational complexity, the present disclosure utilizes a diffusion model that transforms the conformations of the molecular system from a Gaussian distribution to a complex target distribution through an explicit diffusion process that can be more accurately simulated.

[0072] The equilibrium distribution prediction model 26 also includes a density module 68 that is configured to determine a density function of the equilibrium distribution 48 based on the computed statistics. The diffusion process formulation enables estimation of the density function of the target distribution by solving an ordinary differential equation, as described in detail above in the “Computing the Free Energy” section.

[0073] Figure 5A flowchart of a method 500 for predicting equilibrium distributions for molecular systems is shown. The method 500 can be implemented by the hardware and software of the computing system 10 described above, or by other suitable hardware and software. At step 502, the method 500 can include pre-training a diffusion model by a physical information diffusion pre-training (PIDP) algorithm during a training phase. As described above, while equilibrium distribution prediction models can be trained on conformation data samples across a range of molecular systems, it can be very expensive to collect sufficient experimental data and / or simulation data to adequately characterize the equilibrium distributions of various systems. Thus, PIDP is implemented to optimize equilibrium distribution prediction models without data.

[0074] Proceeding from step 502 to step 504, the method 500 can further include supervising the diffusion model during training by an energy function according to a Boltzmann distribution. The supervision comes from an energy function of each molecular system descriptor which is well-defined in the equilibrium distribution. When the energy of a sample descriptor is unknown, the free energy difference can be computed by following a data sample of a Boltzmann distribution. In contrast to traditional free energy computation methods, the equilibrium distribution prediction model does not rely on a harmonic approximation of the energy function near a metastable state, and IID sample generation enables sample coverage over a relevant state space faster than MD.

[0075] Proceeding from step 504 to step 506, the method 500 can further include supervising the diffusion model during training with molecular dynamics (MD) simulation data. While it is inefficient to use dynamic processes to characterize distributions, using MD simulations can produce sufficient data as a supervising function in the equilibrium prediction distribution model.

[0076] At step 508, the method 500 can further include receiving input data representing a molecular system during an inference phase. As described in detail above, the data representing a molecular system can be a descriptor including at least one of: an amino acid sequence of the molecular system a two-dimensional structure of the molecular system and a chemical formula of the molecular system Additionally or alternatively, the descriptor may be a three-dimensional structure of the molecular system, a protein name, a SMILES, etc. The data can be stored in a molecular system database, such as UniProt, Swiss-Prot, Protein Research Foundation (PRF), etc.

[0077] Continuing from step 508 to step 510, the method 500 can further include creating a graphical representation of the molecular system including position information of each atom in the molecular system. As described above with reference to Figure 2AThe preprocessing algorithm can be implemented to generate a graph representation. The graph representation can include a plurality of ordinary nodes connected by edges, each ordinary node representing an atom in the molecular system. The graph representation can also include a virtual node that is fully connected to all ordinary nodes of the graph representation by virtual edges.

[0078] From step 510, the method 500 can further include inputting the graph representation of the molecular system to a graph neural network. The graph neural network can be configured to generate a plurality of predicted conformations of the molecular system, which can be stored as predicted conformation data at a predicted conformation database.

[0079] From step 512, the method 500 can further include receiving the plurality of predicted conformations of the molecular system as output from the graph neural network. From step 514, the method 500 can further include inputting each predicted conformation of the plurality of predicted conformations to a diffusion model. The diffusion model can be a score-based generative diffusion model configured to introduce random noise at predetermined time steps based on a Gaussian distribution in a forward process and reduce the noise in a reverse process. As described in detail above, the forward process can be constructed to target Langevin dynamics (Ornstein-Uhlenbeck process) defined as a Markov chain of steps, and the reverse process can also be a stochastic process following a reverse-time Markov chain of steps. The diffusion model can further implement a score model trained to predict a real score function of a plurality of distribution instances by minimizing a loss function from molecular system descriptors simple . Furthermore, the diffusion model can be configured to change a noise scale of each of the plurality of predicted conformations at a plurality of predetermined time steps. The advancement of the time steps to inject increasing levels of noise into the conformations of the molecular system driving system follows p simple , and utilizing the fractional model to simulate the reverse diffusion process enables the model to produce an equilibrium distribution.

[0080] From step 516, the method 500 can further include receiving a predicted equilibrium distribution of the molecular system. Data representing the predicted equilibrium distribution can be stored in an equilibrium distribution database. The equilibrium distribution can be outputted and viewed in a user interface of a display of a client computing device.

[0081] ​The balanced distribution prediction model described herein enables sampling without simulation or model retraining, which can be more efficient and feasible for large or complex molecular systems. It can also sample from multiple long-lived states, which can be difficult to access using traditional methods. Additionally, the model has the potential to accurately compute free energies, which is important in many areas of chemistry and physics. Finally, it can provide information for inverse design, aiding in the development of new substances or molecules with specific properties.

[0082] The use of physically-informed diffusion pre-training enables the model to satisfy physical constraint conditions and maintain superior performance in data-scarce situations, while diffusion-based generative modeling reduces complexity by breaking the problem into smaller units. The advanced neural network architecture of the present model, such as the use of graph neural networks, allows it to more effectively process and understand the descriptors of molecular systems. While the development of deep learning-based structure prediction techniques has rapidly advanced in recent years, elucidating the balanced distribution of the microscopic states of molecular systems has presented challenges. By developing an accurate balanced distribution prediction method and leveraging large simulation databases assembled by the simulation community and well-developed force fields, the balanced distribution prediction model described herein has the potential to accelerate the development of computational science and make significant contributions to the study of molecular systems.

[0083] In some embodiments, the methods and processes described herein can be tied to the computing system of one or more computing devices. In particular, such methods and processes can be implemented as a computer-application program or service, an application-programming interface (API), a library, and / or other computer-program product.

[0084] Figure 6 A non-limiting embodiment of a computing system 600 that can perform one or more of the above-described methods and processes is schematically illustrated. The computing system 600 is shown in simplified form. The computing system 600 can embody the above-described and Figure 1 The computer device 10 shown. The computing system 600 can take the form of one or more personal computers, server computers, tablet computer(s), home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone(s)), and / or other computing devices, and wearable computing devices such as smart watches and head-mounted augmented reality devices.

[0085] The computing system 600 includes a logic processor 602, volatile memory 604, and a non-volatile storage device 606. The computing system 600 can optionally include a display subsystem 608, input subsystem 610, communication subsystem 612, and / or other components not shown in FIG. 6. Figure 1

[0086] ​The logic processor 602 includes one or more physical devices configured to execute instructions. For example, the logic processor can be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions can be implemented to perform tasks, implement data types, transform the state of one or more components, achieve technical effects, or otherwise arrive at desired results.

[0087] The logic processor can include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logic processor can include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. Processors of the logic processor 602 can be single-core or multi-core, and the instructions executed thereon can be configured for sequential, parallel, and / or distributed processing. Individual components of the logic processor optionally can be distributed among two or more separate devices, which can be remotely located and / or configured for coordinated processing. Aspects of the logic processor can be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.

[0088] The non-volatile storage device 606 includes one or more physical devices configured to hold instructions executable by the logic processors to implement the methods and processes described herein. When such methods and processes are implemented, the state of non-volatile storage device 606 can be transformed— e.g., to hold different data.

[0089] The non-volatile storage device 606 can include physical devices differing in structure from one another: can include optical storage (e.g., CD, DVD, HD-DVD, Blu-ray Disc, etc.), semiconductor storage (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.) and / or magnetic storage (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), or other mass storage device technology. The non-volatile storage device 606 can include nonvolatile, dynamic, static, read / write, read-only, sequential-access, location-addressable, file-addressable, and / or content-addressable devices. It will be appreciated that the non-volatile storage device 606 is configured to hold instructions even when power is cut to the non-volatile storage device 606.

[0090] The volatile memory 604 can include physical devices comprising random-access memory. The volatile memory 604 is typically used by the logic processor 602 to temporarily store information during processing rather than having storage retain information after such processing. It will be appreciated that the volatile memory 604 does not store instructions for implementation of the methods and processes described herein.

[0091] Aspects of logic processor 602, volatile memory 604, and non-volatile storage device 606 can be integrated into one or more hardware-logic components. For example, such hardware-logic components can include field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), application- specific integrated circuits (ASICs), programmable and adapted standard products (PSS / ASSPs), system-on-chips (SOCs), and complex programmable logic devices (CPLDs).

[0092] The terms“module,”“program,” and“engine” can be used to describe an aspect of computing system 600 typically implemented in software by a processor using a portion of volatile memory. Thus, a module, program, or engine can be instantiated via logic processor 602 executing instructions held by non-volatile storage device 606, using a portion of volatile memory 604. It will be understood that different modules, programs, and / or engines can be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and / or engine can be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms“module,”“program,” and“engine” can encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.

[0093] Display subsystem 608, when included, can be used to present a visual representation of data held by non-volatile storage device 606. This visual representation can take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the non-volatile storage device, and thus transform the state of the non-volatile storage device, the state of display subsystem 608 can likewise be transformed to visually represent changes in the underlying data. Display subsystem 608 can include one or more display devices utilizing virtually any type of technology. Such display devices can be combined with logic processor 602, volatile memory 604, and / or non-volatile storage device 606 in a shared enclosure, or such display devices can be peripheral display devices.

[0094] Input subsystem 610, when included, can comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem can comprise or interface with selected natural user input (NUI) componentry. Such componentry can be integrated or peripheral, and the transduction and / or processing of input actions can be handled on- or off-board. Example NUI componentry can include a microphone for speech and / or voice recognition; an infrared, color, stereoscopic, and / or depth camera for machine vision and / or gesture recognition; a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition; as well as electric-field sensing componentry for assessing brain activity to determine intent; and / or any other suitable sensor.

[0095] Communication subsystem 612, when included, can be configured to communicatively couple various computing devices described herein to each other, and to other devices. Communication subsystem 612 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured for communication via a wireless telephony network, or a wired or wireless local- or wide-area network, such as an HDMI over Wi-Fi connection. In some embodiments, the communication subsystem can allow computing system 600 to communicate with other devices via a network, such as the Internet, for example.

[0096] The following paragraphs provide additional description of aspects of the present disclosure. One aspect provides a computing system for predicting an equilibrium distribution for a molecular system. The computing system can include a processor that executes instructions using portions of associated memory to implement an equilibrium distribution prediction model. The processor can be configured to, at an inference stage: receive input data representing a molecular system; create a graph representation of the molecular system including position information for each atom in the molecular system; input the graph representation of the molecular system into a graph neural network; receive, as output from the graph neural network, a plurality of predicted conformations of the molecular system; input each predicted conformation of the plurality of predicted conformations into a diffusion model; predict an equilibrium distribution for the molecular system; and output the predicted equilibrium distribution.

[0097] In this aspect, additionally or alternatively, the diffusion model can vary noise applied to the predicted conformations over a series of time steps, generating the equilibrium distribution.

[0098] In this aspect, additionally or alternatively, the equilibrium distribution prediction model can include an independent and identically distributed (IID) sampler module that generates statistically independent samples from the equilibrium distribution.

[0099] In this aspect, additionally or alternatively, the balanced distribution prediction model can include a density module that determines a density function for the balanced distribution.

[0100] In this aspect, additionally or alternatively, the diffusion model can be pre-trained by a physical information diffusion pre-training algorithm.

[0101] In this aspect, additionally or alternatively, the diffusion model can be trained to minimize a loss function.

[0102] In this aspect, additionally or alternatively, the diffusion model can be supervised during training by an energy function according to a Boltzmann distribution.

[0103] In this aspect, additionally or alternatively, the diffusion model can be supervised during training by molecular dynamics simulation data.

[0104] In this aspect, additionally or alternatively, the diffusion model can be a score-based generative diffusion model configured to introduce random noise based on a Gaussian distribution at predetermined time steps in a forward process.

[0105] In this aspect, additionally or alternatively, the data representing the molecular system can include at least one of: an amino acid sequence of the molecular system, a two-dimensional structure of the molecular system, a chemical formula, a three-dimensional structure of the molecular system, a protein name, and / or a simplified molecular-input line-entry specification (SMILES) of the molecular system.

[0106] Another aspect provides a method for predicting a balanced distribution for a molecular system. The method can include receiving input data representing the molecular system; creating a graph representation of the molecular system including position information for each atom in the molecular system; inputting the graph representation of the molecular system into a graph neural network; receiving a plurality of predicted conformations of the molecular system as output from the graph neural network; inputting each predicted conformation of the plurality of predicted conformations into a diffusion model; and receiving a predicted balanced distribution of the molecular system.

[0107] In this aspect, additionally or alternatively, the method can further include varying noise applied to the predicted conformations over a series of time steps, thereby generating the balanced distribution.

[0108] In this aspect, additionally or alternatively, the method can further include generating statistically independent samples from the balanced distribution by an independent and identically distributed (IID) sampler module.

[0109] In this aspect, additionally or alternatively, the method can further include determining a density function for the balanced distribution by a density module.

[0110] In this regard, additionally or alternatively, the method can further include pre-training the diffusion model by a physical information diffusion pre-training algorithm.

[0111] In this regard, additionally or alternatively, the method can further include training the diffusion model to minimize a loss function.

[0112] In this regard, additionally or alternatively, the method can further include supervising the diffusion model during training with an energy function according to a Boltzmann distribution.

[0113] In this regard, additionally or alternatively, the method can further include supervising the diffusion model during training with molecular dynamics simulation data.

[0114] In this regard, additionally or alternatively, the diffusion model can be a score-based generative diffusion model configured to introduce random noise based on a Gaussian distribution at predetermined time steps in a forward process.

[0115] Another aspect provides a computing system for predicting an equilibrium distribution for a molecular system. The computing system can include a processor that executes instructions using portions of an associated memory to implement an equilibrium distribution prediction model. The processor can be configured to, at a training phase: receive training phase input data representing a molecular system; create a graph representation of the molecular system by a pre-processing algorithm, including position information for each atom in the molecular system; receive energy parameter values for the graph representation; input a training data set into a graph neural network, the training data set including the graph representation of the molecular system and the energy parameter values for the graph representation; receive, as output from the graph neural network, a plurality of predicted conformations of the molecular system; input each predicted conformation of the plurality of predicted conformations into a diffusion model; supervise the diffusion model with an energy function according to a Boltzmann distribution; supervise the diffusion model with molecular dynamics simulation data; and receive, from the diffusion model, a predicted equilibrium distribution for the molecular system.

[0116] It should be understood that the configurations and / or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein can represent one or more of any number of processing strategies. As such, acts shown and / or described can be performed in the sequence shown, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes can be altered.

[0117] The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations described herein, and other features, functions, acts and / or properties disclosed herein, as well as any and all equivalents thereof.

Claims

1. A computing system for predicting an equilibrium distribution for a molecular system, comprising: a processor that executes instructions using a portion of an associated memory to implement an equilibrium distribution prediction model, wherein the processor is configured to, in an inference phase: receive input data representing a molecular system; create a graph representation of the molecular system including position information for each atom in the molecular system; input the graph representation of the molecular system into a graph neural network; receive, as output, a plurality of predicted conformations of the molecular system from the graph neural network; input each of the plurality of predicted conformations into a diffusion model; predict an equilibrium distribution for the molecular system; and output the predicted equilibrium distribution.

2. The computing system of claim 1, wherein the diffusion model varies noise applied to the predicted conformations over a series of time steps, thereby generating the equilibrium distribution.

3. The computing system of claim 1, wherein the equilibrium distribution prediction model includes an independent and identically distributed (IID) sampler module that generates statistically independent samples from the equilibrium distribution.

4. The computing system of claim 1, wherein the equilibrium distribution prediction model includes a density module that determines a density function for the equilibrium distribution.

5. The computing system of claim 1, wherein the diffusion model is pre-trained by a physical information diffusion pre-training algorithm.

6. The computing system of claim 1, wherein the diffusion model is trained to minimize a loss function.

7. The computing system of claim 1, wherein the diffusion model is supervised during training by an energy function according to a Boltzmann distribution.

8. The computing system of claim 1, wherein the diffusion model is supervised during training by molecular dynamics simulation data.

9. The computing system of claim 1, wherein the diffusion model is a score-based generative diffusion model configured to introduce random noise based on a Gaussian distribution at predetermined time steps in a forward process.

10. The computing system of claim 1, wherein the data representing a molecular system includes at least one of: an amino acid sequence of the molecular system, a two-dimensional structure of the molecular system, a chemical formula, a three-dimensional structure of the molecular system, a protein name, and / or a simplified molecular-input line-entry system (SMILES) of the molecular system.

11. A method for predicting an equilibrium distribution for a molecular system, the method comprising: receiving input data representing a molecular system; creating a graph representation of the molecular system including position information for each atom in the molecular system; inputting the graph representation of the molecular system into a graph neural network; receiving, as output, a plurality of predicted conformations of the molecular system from the graph neural network; inputting each of the plurality of predicted conformations into a diffusion model; and receiving a predicted equilibrium distribution for the molecular system. ​ ​ 12. The method of claim 11, further comprising: varying noise applied to the predicted conformations over a series of time steps, thereby generating the equilibrium distribution.

13. The method of claim 11, further comprising: generating statistically independent samples from the equilibrium distribution by an independent and identically distributed (IID) sampler module.

14. The method of claim 11, further comprising: determining a density function for the equilibrium distribution by a density module.

15. The method of claim 11, further comprising: pre-training the diffusion model by a physical information diffusion pre-training algorithm.

16. The method of claim 11, further comprising: training the diffusion model to minimize a loss function.

17. The method of claim 11, further comprising: supervising the diffusion model during training with an energy function according to a Boltzmann distribution.

18. The method of claim 11, further comprising: supervising the diffusion model during training with molecular dynamics simulation data.

19. The method of claim 11, wherein the diffusion model is a score-based generative diffusion model configured to introduce random noise based on a Gaussian distribution at predetermined time steps in a forward process.

20. A computing system for predicting an equilibrium distribution for a molecular system, comprising: a processor that executes instructions using portions of associated memory to implement an equilibrium distribution prediction model, wherein the processor is configured to, in a training phase: receive training phase input data representing a molecular system; create a graph representation of the molecular system by a pre-processing algorithm, including position information for each atom in the molecular system; receive energy parameter values for the graph representation; input a training data set into a graph neural network, the training data set including the graph representation of the molecular system and the energy parameter values for the graph representation; receive as output from the graph neural network a plurality of predicted conformations of the molecular system; input each of the plurality of predicted conformations into a diffusion model; supervise the diffusion model with an energy function according to a Boltzmann distribution; supervise the diffusion model with molecular dynamics simulation data; and receive from the diffusion model a predicted equilibrium distribution for the molecular system. ​

Images