Systems and methods for spatial graph convolution and applications thereof in drug discovery and molecular simulation

Abstract

The invention relates to systems and methods for spatial graph convolution and their use in drug discovery and molecular simulation. Systems and methods for spatial graph convolution in accordance with embodiments of the present invention are described. One embodiment includes a method for predicting a characteristic of a molecule, where the method includes performing a first set of graph convolutions with a spatial graph representation of a set of molecules, where the first set of graph convolutions is based on bonds between the set of molecules, performing a second set of graph convolutions with the spatial graph representation, where the second set of graph convolutions is based on bonds between the set of molecules. Wherein the second set of graph contours is based at least on the distance between each atom and other atoms in the set of molecules, performing graph aggregation with the spatial graph representation to produce a feature vector, and predicting a set of one or more features for the set of molecules based on the feature vector.

Description

[0001] This application is a divisional application based on the patent application with application number 201980027200.7, application date March 5, 2019, and invention name “Systems and methods for spatial graph convolution and their applications in drug discovery and molecular simulation”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 638,803, entitled “Spatial Graph Convolutions with Applications to Drag Discovery,” filed on March 5, 2018. The disclosure of U.S. Provisional Patent Application Serial No. 62 / 638,803 is incorporated herein by reference in its entirety. Technical Field

[0004] The present invention relates generally to predicting molecular properties, and more particularly to predicting molecular properties using spatial graph convolutions. Background Art

[0005] Most FDA-approved drugs are small organic molecules that induce a therapeutic response by bonding to a target biomacromolecule. Once bonded, the small molecule ligand either inhibits the bonding of other ligands or allosterically adjusts the conformation of the target as a whole. Therefore, bonding is critical to any behavior of a therapeutic ligand. In order to maximize the therapeutic effect of a molecule, its affinity or binding free energy (ΔG) for the desired target must be maximized while minimizing its affinity for other macromolecules. Historically, scientists have used cheminformatics and structure-based methods to model ligands and their targets, and most machine learning (ML) methods use features driven by domain expertise. Summary of the invention

[0006] Systems and methods for spatial graph convolutions according to embodiments of the present invention are described. One embodiment includes a method for predicting characteristics of molecules, wherein the method includes performing a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules, performing a second set of graph convolutions with the spatial graph representation, wherein the second set of graph convolutions is based at least on distances between each atom and every other atom in the set of molecules, performing graph aggregation with the spatial graph representation to produce a feature vector, and predicting a set of one or more characteristics for the set of molecules based on the feature vector.

[0007] In another embodiment, the method further comprises a step for receiving atomic information for a set of one or more molecules.

[0008] In yet another embodiment, the method further comprises a step for constructing a spatial graph representation of the collection of molecules.

[0009] In yet another embodiment, constructing the spatial graph representation includes generating a distance matrix and an adjacency tensor, wherein the distance matrix represents distances between atoms of the set of molecules and the adjacency tensor indicates a plurality of different edge types between the atoms.

[0010] In yet another embodiment, the set of molecules includes ligand molecules and target molecules, wherein the rows of the distance matrix are ordered by membership in the ligand and target molecules.

[0011] In yet another embodiment, the bonds between the collection of molecules comprise covalent bonds.

[0012] In another additional embodiment, the bonds between the collection of molecules include at least one of π-π stacking, hydrogen bonding, and hydrophobic contacts.

[0013] In another additional embodiment, the second set of graph convolutions is further based on bonds between the set of molecules.

[0014] In yet another embodiment, the first set of graph convolutions is based on a first set of bonds between sets of molecules, and the second set of graph convolutions is based on a second set of bonds between sets of molecules.

[0015] In yet another embodiment, the first set of keys is a subset of the second set of keys.

[0016] In yet another embodiment, performing the first set of graph convolutions includes performing a gated recurrent unit (GRU) operation at each layer of the graph convolutions.

[0017] In yet another embodiment, performing a first set of graph convolutions includes utilizing a first plurality of neural networks, wherein each neural network in the plurality of neural networks is for a different bond type.

[0018] In yet another additional embodiment, performing the second set of graph convolutions includes utilizing a second plurality of neural networks, wherein weights of the first plurality of neural networks are shared with the second plurality of neural networks.

[0019] In yet another additional embodiment, performing the second set of graph convolutions includes utilizing a second plurality of neural networks, wherein neural networks of the second plurality of neural networks utilize distance information regarding distances between atoms of the set of molecules.

[0020] In yet another embodiment, the set of molecules includes ligand molecules and target molecules, wherein graph aggregation is performed only on the ligand molecules.

[0021] In yet another embodiment, the set of characteristics includes whether a first molecule in the set of molecules bonds to a second molecule in the set of molecules.

[0022] A system and method for training a spatial convolutional graph model according to an embodiment of the present invention is described. One embodiment includes a method for training a spatial convolutional graph model. The method includes the following steps: performing a first set of graph convolutions with a spatial convolutional graph model of a set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules, performing a second set of graph convolutions with the spatial convolutional graph model, wherein the second set of graph convolutions is based at least on the distance between each atom and other atoms in the set of molecules, performing graph clustering with the spatial convolutional graph model, calculating a loss for the set of molecules based on the graph clustering, and updating the spatial convolutional graph model based on the calculated loss.

[0023] In yet another additional embodiment, the method further comprises a step for training a set of one or more neural networks using the layers of the spatial convolutional graph model to predict a set of one or more parameters of the force field.

[0024] In yet another additional embodiment, the set of parameters is associated with a set of one or more characteristics, wherein the set of characteristics includes at least one of charge, bonding, angles, and dihedrals.

[0025] Again in yet another embodiment, the spatial convolutional graph model is a first spatial convolutional graph model, wherein the method further comprises training a second spatial convolutional graph model to predict potential energy, and predicting the potential energy of the molecular system based on the first and second spatial convolutional graph models.

[0026] Again in yet another embodiment, training the second spatial convolutional graph model includes sharing a set of one or more layers between the first and second spatial convolutional graph models.

[0027] In yet another additional embodiment, the force field is in the Assisted Model Building with Energy Refinement (AMBER) functional form.

[0028] Again in another additional embodiment, the method further comprises the step of training an ensemble of one or more neural networks using the layers of the spatial convolutional graph model to predict the potential energy of the input molecular system.

[0029] In yet another further embodiment, the method further comprises a step for identifying a set of conformations of the set of molecules by minimizing a potential energy predicted by the spatial graph convolution model.

[0030] In another embodiment, the method further comprises a step for predicting the motion of the collection of molecules.

[0031] Additional embodiments and features are described in part in the following description, and in part will become apparent to those skilled in the art upon reading this specification, or may be learned by practicing the invention. A further understanding of the nature and advantages of the invention may be achieved by reference to the remainder of this specification and the accompanying drawings which constitute a part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] A more complete understanding of the specification and claims will be obtained with reference to the following figures and data graphs, which are presented as exemplary embodiments of the invention and should not be construed as a complete recitation of the scope of the invention.

[0033] Figure 1 Figure 2. Visual depiction of a gated graph neural network (GGNN).

[0034] Figure 2 A system providing modeling and forecasting according to some embodiments of the present invention is illustrated.

[0035] Figure 3 Modeling elements that provide modeling and prediction according to some embodiments of the present invention are illustrated.

[0036] Figure 4 A modeling application providing modeling and prediction according to some embodiments of the present invention is illustrated.

[0037] Figure 5 A process for performing staged convolution according to an embodiment of the present invention is conceptually illustrated.

[0038] Figure 6 Figure 2. Visual depiction of a multi-stage spatially gated graph neural network.

[0039] Figure 7 The first stage of a multi-stage spatially gated graph neural network according to various embodiments of the invention is illustrated.

[0040] Figure 8 The second stage of a multi-stage spatially gated graph neural network according to various embodiments of the invention is illustrated. DETAILED DESCRIPTION

[0041] The systems and methods according to various embodiments of the present invention are specifically designed for protein ligand binding affinity and achieve state-of-the-art performance thereof. Although many examples in this application are described with reference to bonding affinity, many molecular characteristics can be predicted without departing from the spirit of the present invention, such as but not limited to toxicity, solubility, and electronic properties. In many embodiments, the process utilizes a multi-stage process based on bond type and spatial distance to predict molecular characteristics. In some embodiments, a new metric is calculated, the regression enrichment factor To measure early enrichment of computational models for chemical data. Processing according to various embodiments of the present invention implements a cross-validation strategy based on structural homology clustering that can more accurately measure the generalization of the model, which crucially distinguishes the goal of machine learning for drug discovery from standard machine learning tasks.

[0042] The arc of drug discovery presents multi-parameter optimization problems spanning a vast range of scales. Critical parameters range from solubility (angstroms) to protein-ligand bonding (nanometers) to in vivo toxicity (meters). Historically, scientists have used cheminformatics and structure-based approaches to model ligands and their targets, and most machine learning (ML) approaches use features driven by domain expertise. Through feature learning (rather than feature engineering), deep neural networks are poised to outperform traditional physics-based and knowledge-based machine learning models for predicting molecular properties relevant to drug discovery.

[0043] Training most conventional DNN architectures requires large amounts of data: for example, ImageNet currently contains over 14,000,000 labeled images. In comparison, the largest publicly available datasets on the properties of drug-like molecules include: PDBBind 2017, which contains a sample of slightly more than 4,000 protein-ligand co-crystal structures and associated bonding affinity values; Tox21, which has nearly 10,000 small molecules and associated toxicity endpoints; QM8, with about 22,000 small molecules and associated electronic properties; and ESOL, which has slightly more than 1,000 small molecules and associated solubility values. The scarcity of high-quality scientific data requires innovative neural architectures for molecular machine learning.

[0044] Successful DNNs often exploit correlation structures in the data, such as pixel proximity in images. Predicting protein-ligand bonding affinities can be viewed as analogous to a computer vision problem. Just as neighboring pixels represent the closeness between physical objects, bonding pockets can be divided into a grid of voxels. Here, neighboring voxels represent neighboring atoms and blocks of empty space. Unfortunately, this 3D convolutional approach has several potential disadvantages. First, the input and hidden weights require more memory in three dimensions. Second, since parameters grow exponentially with the number of dimensions, the model suffers from the problem of the “curse of dimensionality.” The simplest qualitative models for affinity prediction and related tasks incorporate only features of the ligand, ignoring the macromolecular target(s).

[0045] Graph convolutional neural networks (GCNNs) similarly exploit the inherent structure of the data. By exploiting molecular structure and symmetry, graph convolutions can use fewer parameters. Graph convolutions can be symmetric for permutations and for the relative position of each adjacent node, thereby significantly reducing the number of model parameters. Models according to several embodiments of the present invention can generalize graph convolutions to include intramolecular interactions and non-covalent interactions between different molecules.

[0046] In several embodiments, processing may employ a staged gated graph neural network that distinguishes the derivation of differentiable bond atom types from the propagation of information between atoms through space. Processing according to some embodiments of the invention employs a more flexible model based on a new update rule using the distance from source to target atoms and feature maps of the target atoms. In some protein-ligand bond affinity benchmarks, incorporating target atom information directly into the information function can increase the signal.

[0047] In some embodiments, there are N nodes, each node f in A given graph with features and a single edge type can be represented as consisting of node features x and a symmetric adjacency matrix A, which specifies whether a pair of nodes belongs to each other's neighbor set N. Turning now to the attached figure, Figure 1 A visual depiction of a gated graph neural network (GGNN) is illustrated in . The GGNN 100 is illustrated with nodes (110-120) representing atoms and edges (130-140) representing bonds. In this example, the small molecule propionamide is chosen to illustrate the propagation of information between different update layers of the network. For the carbonyl carbon (110), the relevant row of the feature matrix x might be [1,0,0] to represent its elements, while the corresponding row of the adjacency matrix A might be [0,1,0,1,1] to indicate its bonds with three neighbor atoms.

[0048] In various embodiments, a graph convolution update entails applying a function at each node that takes the node and its neighbors as input and outputs a new set of features for each node. Graph convolutions are described in Gilmer et al., “Neural messagepassing for quantum chemistry,” Proceedings of the 34th International Conference on Machine Learning, the disclosure of which is incorporated herein by reference in its entirety. A graph convolution can be written as

[0049]

[0050] in represents the node feature of node i at hidden layer t, N(v i) represents the neighbors of node i, and U (t) and m (t) are the update and message functions at hidden layer t, respectively. In some embodiments, when there are multiple edge types, multiple message functions can be defined, m (t，e) , which is for edge types e∈[1,...,N et ]Message function at layer t.

[0051] In many embodiments, the update function at all layers of the model is the familiar gated recurrent unit (GRU). However, those skilled in the art will recognize that other update functions may be used without departing from the spirit of the invention. In some embodiments, the message function is a simple linear operation that is different for each edge type but the same across layers:

[0052]

[0053] For edge type e, A (e) is the adjacency matrix, and W (e) is the weight matrix.

[0054] The GGNN family of graph convolutional architectures includes efficient optimizations to reduce the complexity of graphs. Let d be the dimensionality of the internal hidden representation of each node, and n be the number of nodes in the graph. A single step of message passing for dense graphs requires Decompose the d-dimensional node embedding into k different dimensional embedding reduces this runtime to Since most molecules are sparse or relatively small graphs, these layers are usually In various embodiments, other optimizations may be employed, such as (but not limited to) utilizing spectral representations of graphs.

[0055] Unlike conventional FCNNs, which learn nonlinear combinations of hand-crafted features of the input, the updates described in accordance with some embodiments of the present invention learn nonlinear combinations of more basic features of a given atom and the features of its immediate neighbors. Information propagates through increasingly distant atoms with each graph convolution, while GRU enables information to be added selectively. Ultimately, the GCNN includes and fully exploits each node feature via the feature matrix x, and includes and fully exploits structural information via the adjacency matrix A. In both classification and regression settings, GCNNs can terminate with a "graph aggregation" step that sums the rows of the final embedding and is invariant to the node ordering. Subsequent FCNNs produce graphs of the desired size (f out ) output. This completes the starting point for the graph convolution update used in accordance with many embodiments of the present invention:

[0056]

[0057] In some embodiments, predicting affinities for multiple targets through GCNN can be achieved by training different models for each target or by training a single multi-task network. The latter setting according to the respective embodiments of the present invention can use the final weight matrix where T represents the number of targets in the dataset. The corresponding multi-task loss function will be the average binary cross entropy loss across targets,

[0058]

[0059] Systems and methods according to many embodiments of the present invention exploit structural information about the target in addition to the ligand. To motivate the architecture for more principled DNN predictors, the following notation and framework are used. The distance matrix Its entry R ij Represents atom i and atom j The distance between.

[0060] So far, the concept of adjacency has been strictly restricted to chemical bonds, as encoded in the symmetric matrix A. However, adjacency can also cover a wider range of neighbor types to include non-covalent interactions (e.g., π-π stacking, hydrogen bonds, hydrophobic contacts). Adjacency does not require domain expertise. In various embodiments, pairwise distances below a threshold can also be used. Regardless of the scheme adopted, the distance matrix R motivates the construction of an extended version of A. In this framework, A becomes a matrix of shape N×N×N et A tensor with N et Indicates the number of edge types.

[0061] If the line is pressed by atom i For either protein or ligand atomic membership, both A and R can be viewed as block matrices, where the diagonal blocks are self-edges (i.e., bonds and non-covalent interactions) from one ligand atom to another ligand atom or from one protein atom to another protein atom. Off-diagonal block matrices can encode edges from protein to ligand and from ligand to protein. For simplicity, only one edge type N is described. et =1 special case:

[0062]

[0063] Among them A ij is 1 for a neighbor and 0 otherwise, and Within this evil framework, spatial graph convolution (graph convolution based on the notion of adjacency based on Euclidean distance) can be expressed as a generalization of GGNN, characterized by the above-mentioned updates.

[0064] In addition to edge type generalization, processing according to many embodiments of the invention introduces non-linearity in the message part of the graph convolutional layer:

[0065]

[0066] where NN (e) is a neural network for each edge type e, and N (e) (h i ) represents the neighbors of edge type e for atom / node i.

[0067] In some embodiments, the concept of layer is generalized to the concept of stage that can span several layers of a given type. The method according to some embodiments of the present invention includes three main steps:

[0068] (1) covalent propagation only, (2) non-covalent and covalent dual propagation, and (3) ligand-based graph clustering. More generally, in some embodiments, processing performs a first propagation based on a first set of one or more edge types. The second stage according to various embodiments of the invention may include bond types and distance information separate from the bond information separate from the distance matrix. The distance information may include (but is not limited to) specific physical distance measurements and / or binned distances. The second stage according to various embodiments of the invention may include all edge types or a subset of edge types.

[0069] In some embodiments, stage (1) covalent propagation requires only the first slice A of the adjacency matrix (1) , if (atom i , atom j ), then the (i, j) entry contains 1 if there is a bond between them, and 0 otherwise. Intuitively, stage (1) computes a vector-valued atomic type for each of the N atoms in the system based on the local network of bonded atoms A new set of. The atom type according to various embodiments of the present invention is a feature map of each atom. Subsequently, stage (2) according to various embodiments of the present invention requires propagation based on two fully adjacent tensors A, which are the atom types of the vector values ​​calculated in (1) begins. While stage (1) computes new bond-based "atom types" for the amino acid and ligand atoms, stage (2) transfers bond and spatial information between atoms. For example, if stage (1) distinguishes amide carbonyl oxygens from ketone carbonyl oxygens, then stage (2) might indicate in the first layer that the carbonyl oxygen is also within 3 angstroms of a hydrogen bond donor. Finally, in stage (3), graph aggregation is performed only on the ligand atoms. In some embodiments, graph aggregation of only the ligands is computationally direct via the block matrix formulation described above.

[0070] In various embodiments, a new set of vector-valued atom types is computed for each atom in the system based on a local network of bonded atoms.

[0071] Phase 1:

[0072]

[0073] In many embodiments, the second stage needs to be propagated based on two fully adjacent tensors A, which are vector-valued atomic types calculated in (1) Start. Phase (1) calculates new bond-based "atom types" for both amino acids and ligand atoms, while phase (2) transfers bond and spatial information between atoms. In some embodiments, in addition to bond information, spatial information from a distance matrix is ​​also utilized in the second phase.

[0074] Phase 2:

[0075]

[0076] In various embodiments, graph aggregation is performed in the third stage (3).Processing according to many embodiments of the invention performs graph aggregation only on ligand atoms.

[0077] Phase 3:

[0078]

[0079] where i (b) , j (b) ,i (sp) , j (sp) is a key and spatial neural network, and represents the feature map of the jth atom at the end of stage 2.

[0080] More generally, given S stages and handling edge types discretely, the concepts of combining edge information, multiple edge types, distance information, and stages can be summarized in the following formula.

[0081] In one formulation according to various embodiments of the present invention, each edge type may be processed discretely:

[0082] Phase 1:

[0083]

[0084] Phase S:

[0085]

[0086] Among them, for example, K S Define the number of graph convolution layers in the Sth stage, Defined in KS The feature map of the i-th atom in the S-th stage after the graph convolutional layers, Indicates the number of edge types to be used in stage S (this is the edge type N et A subset of the total number), R ij represents the i-th row and j-th column entry of the matrix R (and therefore the atom / node v i and v j The distance between them), and NN (S，e) represents the neural network for the e-th edge type at stage S (although the same neural network can be used for a given edge type at different stages for weight tying), f S represents the dimension of the feature map of each atom at the end of stage S, and i (S) and j (S) In the collection of neural networks for stage S. It should be noted that since it has been pointed out that in each stage / set of graph convolutional layers, a subset of edge types and edge information can be utilized in any given case, so at any given stage, a given NN within the GRU function does not need to utilize distance information R ij , nor does it need to utilize soft or hard distance intervals. In other words, some of these phases can depend only on the bonds or other non-spatial relationships between atoms.

[0087] In another formulation according to various embodiments of the invention, the edge type may be a flexible or a continuous / dense edge type.

[0088] Phase 1:

[0089]

[0090] Phase S:

[0091]

[0092] Among them, for example, K S Define the number of graph convolution layers in the Sth stage, Defined in K S The feature map of the i-th atom in the S-th stage after the graph convolutional layers, R ij represents the i-th row and j-th column entry of the matrix R (and therefore the atom / node v i and v j The distance between (S) represents the neural network at stage S, e ij It represents the atom v i and v j The vector of pre-featured or learned edge features between S represents the dimension of the feature map of each atom at the end of stage S, and i(S) and j (S) In the clustering of neural networks for stage S. Although in the above formulas each stage uses a distance matrix R, the distance matrix R may be omitted from one or more stages. In many embodiments, a first subset of stages may utilize a distance matrix, while a second subset of stages may not involve a distance matrix at all. In some embodiments, the last layer of each stage may simply be set to Alternatively, or in combination, the last layer of each stage according to many embodiments of the present invention may be set to or

[0093] In some embodiments, e ij is not predefined but can itself be a differentiable function. In one example, will be the edge between atoms i and j at stage S and graph convolution layer 3. In this example, In many embodiments, e ij can be a neural network that depends on the feature maps of atoms i and j at a given stage after the graph convolutional layer at that stage. In some embodiments, e ij It can be a RNN. For example,

[0094] In the specific case of protein-ligand bonding, the graph aggregation operation will be defined after S stages as:

[0095] Aggregation phase:

[0096]

[0097] However, according to an embodiment of the present invention, N may be replaced by any subset of the nodes in the graph. Lig The sum of atoms is adjusted to suit the requirements of specific applications. In addition, according to some embodiments of the present invention, other types of graph aggregation neural networks can be used, such as replacing the above-mentioned ReLU nonlinearity with other nonlinearities (such as S-shaped, leaky ReLU, tanh, etc.), and other operations other than fully connected neural networks (such as recurrent neural networks) can be used.

[0098] System and method

[0099] exist Figure 22 shows a system for providing modeling and prediction according to some embodiments of the present invention. Network 200 includes a communication network 260. Communication network 260 is a network such as the Internet that allows devices connected to network 260 to communicate with other connected devices. Server systems 210, 240, and 270 are connected to network 260. Each of server systems 210, 240, and 270 is a group of one or more server computer systems that are connected to each other via an internal network, and the system performs the processing of providing cloud services to users through network 260. For the convenience of this discussion, a cloud service is one or more applications executed by one or more server systems to provide data and / or executable applications to devices through a network. Server systems 210, 240, and 270 are shown, each of which has three servers connected via an internal network. However, server systems 210, 240, and 270 may include any number of servers, and any other number of server systems may be connected to network 260 to provide cloud services, including but not limited to virtual server systems. According to various embodiments of the invention, the processes for modeling and predicting molecular properties are provided by one or more software applications executed on a single server system and / or a group of server systems communicating over the network 260 .

[0100] According to various embodiments of the present invention, a user may use personal devices 280 and 220 connected to network 260 to perform processing for modeling and predicting molecular properties. In the illustrated embodiment, personal device 280 is shown as a desktop computer connected to network 260 via a conventional "wired" connection. However, personal device 280 may be a desktop computer, a laptop computer, a smart television, an entertainment game console, or any other device connected to network 260 via a "wired" or "wireless" network connection. Mobile device 220 connects to network 260 using a wireless connection. A wireless connection is a connection to network 260 that uses radio frequency (RF) signals, infrared signals, or any other form of wireless signaling. In Figure 2 In the embodiment of the present invention, mobile device 220 is a mobile phone. However, without departing from the present invention, mobile device 220 may be a mobile phone, a personal digital assistant (PDA), a tablet computer, a smart phone, a virtual reality headset, an augmented reality headset, a mixed reality headset, or any other type of device connected to network 260 via a wireless connection. According to some embodiments of the present invention, the processing for modeling and predicting molecular properties is performed by a user device. It is readily appreciated that the specific computing system used for modeling and predicting molecular properties depends largely on the requirements of a given application and should not be considered to be limited to any specific computing system implementation (one or more).

[0101] exist Figure 3300 illustrates a modeling element according to several embodiments of the present invention. A modeling element according to many embodiments of the present invention may include (but is not limited to) one or more of a mobile device, a computer, a server, and a cloud service. Modeling element 300 includes a processor 310, a communication interface 320, and a memory 330.

[0102] Those skilled in the art will recognize that a particular modeling element may include other components that are omitted for the sake of brevity without departing from the present invention. Processor 310 may include, but is not limited to, a processor, a microprocessor, a controller, or a combination of processors, microprocessors, and / or controllers that executes instructions stored in memory 330 to manipulate data stored in memory. Processor instructions may configure processor 310 to perform processing according to certain embodiments of the present invention. Communication interface 320 allows modeling element 300 to transmit and receive data over a network based on instructions executed by processor 310.

[0103] Memory 330 includes modeling application 332, molecular data 334 and model data 336. Modeling application according to several embodiments of the present invention is used to model and predict the interaction of different molecules. In many embodiments, the interaction of modeling can be used to identify candidate molecules for further testing. In several embodiments, modeling application can use molecular data, which includes data generated from various sources, including (but not limited to) databases of molecular simulation and / or molecular properties. Model data 336 according to various embodiments of the present invention may include (but not limited to) data for spatial graphs, edge type neural networks and fully connected classifier networks. Models according to many embodiments of the present invention can be used for various purposes, such as (but not limited to) identifying the characteristic matrix of each atom of a molecular system and classifying the various characteristics of molecules, including but not limited to bonding / non-bonding, quantum properties, toxicity and solubility.

[0104] Although in Figure 3 A specific example of the modeling element 300 is illustrated in FIG. Figure 3 In embodiments of the present invention, any of a variety of modeling elements may be used to perform processes similar to those described herein, as appropriate to the requirements of a particular application.

[0105] The modeling application for identifying candidate ligands according to an embodiment of the present invention is Figure 4 The modeling application 400 includes a graph engine 405, a bond functionalizer 410, a spatial characterizer 415, an aggregation engine 420, and a classification engine 425. The modeling application according to many embodiments of the present invention can model the molecular data of a molecular set of a molecular system and predict the interactions between the molecules in the system.

[0106] In various embodiments, the graph engine can create a graph representation of a collection of molecules. The graph representation can include (but is not limited to) bond information, bond types, distance information, atomic properties, and binding affinities. The constructed graphical representation according to various embodiments of the present invention can include an adjacency matrix and / or a distance matrix.

[0107] The bond characterizer according to many embodiments of the present invention can generate a feature matrix for each atom of a molecular system based on the bond between the atom and each of its neighboring atoms. In many embodiments, the bond characterizer can operate on different types of bonds at different stages of the modeling and prediction process. In various embodiments, the bond characterizer can determine a first set of feature matrices for each atom based on covalent bonds in a first stage, and can be used to determine a second set of feature matrices for atoms based on other types of bonds. In addition to or in lieu of bond types, the spatial characterizer according to some embodiments of the present invention can also generate a feature matrix for each atom of a molecular system based on the spatial distance between different atoms. The bond characterizer according to various embodiments of the present invention can be combined with the spatial characterizer in some stages of the process to combine both bond and spatial information into the feature matrix of each atom. The bond and spatial characterizers can implement multiple neural networks for each bond type and / or each stage of the process.

[0108] The aggregation engine according to various embodiments of the present invention can be used to aggregate features generated from feature generators. In various embodiments, the aggregation engine can aggregate the final set of feature matrices generated by bond and / or spatial feature generators. In many embodiments, the aggregation engine operates on only one of the molecules (e.g., a ligand) of a molecular system.

[0109] In various embodiments, the classification engine can be used to classify or predict interactions between sets of molecules. The classification engine according to some embodiments of the present invention can implement a classifier, such as (but not limited to) a fully connected neural network (FCNN) and / or a random forest. In various embodiments, the classification engine takes as input a feature matrix generated by a characterizer and / or an aggregation engine, and outputs the likelihood that a set of molecules has a particular relationship (e.g., bonded / non-bonded, agonist / antagonist, etc.). Although in Figure 4 Specific examples of modeling applications are illustrated in , but according to embodiments of the present invention, any of a variety of modeling applications may be used to perform processes similar to those described herein, as appropriate to the requirements of a specific application.

[0110] Specific examples of systems, elements, and applications for modeling and predicting molecular characteristics using spatial convolution according to embodiments of the present invention are described above; however, those skilled in the art will recognize that according to embodiments of the present invention, any number of structures and systems may be appropriately utilized as required by a specific application.

[0111] exist Figure 5 500 conceptually illustrates a process for performing staged convolution according to an embodiment of the present invention. Process 500 receives (505) atomic information for one or more molecules in a system. In various embodiments, the molecules may include source (e.g., ligands) and target molecules. The atomic information according to several embodiments of the present invention includes various information about each molecule, including (but not limited to) distance information and chemical properties. In many embodiments, the atom type (each atom The process 500 establishes (510) a spatial graph representation of the ligand and the target molecule. The spatial graph representation according to various embodiments of the present invention includes a distance matrix and / or an adjacency matrix. The distance matrix according to some embodiments of the present invention includes distance information of each atom and each other atom in the system. In many embodiments, the adjacency matrix includes multiple layers (or dimensions), each of which indicates adjacencies along different edge types. For example, adjacencies in a first layer may indicate that two atoms share a covalent bond, while adjacencies in different layers indicate different bond types. In some embodiments, adjacencies do not indicate any specific bond, but rather indicate whether two atoms are within a threshold distance of each other.

[0112] Then, processing according to various embodiments of the present invention may perform multiple graph convolution stages. Process 500 performs (515) a graph convolution stage. In some embodiments, different graph convolution stages may be based on different subsets of information about the molecular system, where the information may include (but is not limited to) bond types, distance types, and / or other edge types. For example, some stages may be based only on bonding types, while other stages may be based on certain edge types and distance information. A single graph convolution stage according to various embodiments of the present invention may be based on subsets of different edge types, while different convolution stages are based on different subsets or complete sets of different edge types. For example, in some embodiments, an initial graph convolution stage is based on a first layer of an adjacency matrix indicating the presence of covalent bonds, while subsequent graph convolution stages are based on all bond types as well as atomic distances.

[0113] In multiple embodiments, each graph convolution stage generates a feature vector for each atom, thereby learning the features of each atom based on the input to each stage. The feature vectors according to various embodiments of the present invention can be used as input to subsequent graph convolution stages. Each graph convolution stage according to various embodiments of the present invention can utilize a collection of neural networks. In many embodiments, each separate neural network is trained for each edge type. In many embodiments, the neural network may include, but is not limited to, fully connected neural networks, convolutional networks, and recursive networks. According to various embodiments of the present invention, neural networks for specific edge types can be used across different stages to tie the weights of different stages together. However, in many embodiments, different neural networks are trained and utilized for each edge type and stage.

[0114] Process 500 determines (520) whether there are more stages to be performed. In many embodiments, a specified number of stages are performed. When the process determines that there are more stages to be performed, the process returns to step 515. When all stages have been completed, the process 500 performs (525) graph aggregation to aggregate information after the graph convolution stage. In many embodiments, the graph aggregation is performed only on the atoms of the ligand molecule. In various embodiments, the graph aggregation step is performed using a fully connected neural network trained to predict the classification of the ligand molecule. The classification can include (but is not limited to) the predicted ability to bond to the target molecule.

[0115] The above describes the specific processing of classifying atoms using spatial convolution according to an embodiment of the present invention; however, those skilled in the art will recognize that according to an embodiment of the present invention, any number of processes may be appropriately adopted according to the requirements of a specific application.

[0116] exist Figure 6 , a visual depiction of a multi-stage spatially gated graph neural network according to various embodiments of the invention is illustrated in . The first stage 605 requires only graph convolutions on bonds, which in more traditional forms of molecular modeling results in new node (atom) feature maps that are roughly similar to distinguishable atom types. The second stage 610 requires bond-based and spatial distance-based information propagation. In the third stage 615, a graph aggregation operation is performed on the ligand atoms, whose feature maps are derived from the bonded ligand information and spatial proximity to protein atoms. In contrast to molecular dynamics force fields, which for historical reasons have different force fields for ligands and proteins, which then have to interoperate (often poorly) in simulations, methods according to some embodiments of the invention can derive the physicochemical properties of biomolecular interactions from a unified framework.

[0117] To further illustrate, the first and second stages according to various embodiments of the present invention utilize different subsets of the fully adjacency tensor A. Figure 7The first stage according to various embodiments of the present invention is illustrated in FIG. In this example, only the covalent or bonded interaction edge types encoded in the first slice of the last dimension of the adjacency tensor A are utilized. Figure 8 The second stage according to various embodiments of the present invention is illustrated in . In this example, the entire range of the last dimension of the adjacency tensor A is utilized for bonded and non-bonded interaction edge types. In various embodiments, the second stage may also include spatial data describing the distances between various atoms.

[0118] Traditional measures of predictor performance are plagued by general problems and problems specific to drug discovery. For a regressor, R 2 (“coefficient of determination”) and root mean square error (RMSE) are both susceptible to outliers in a single data point. The RMSE of both classifiers and regressors takes neither the training data distribution nor invalid model performance into account. The area under the receiver operating characteristic curve can correct for the RMSE deficiency of a classifier. However, all of the above-mentioned metrics are global statistics that are weighted equally over all data points. This property is particularly undesirable in drug development, where the tails of the prediction distribution are of greatest concern: although model predictions are made for an entire library of millions of molecules, only the highest-scoring molecules will be purchased or synthesized.

[0119] In response, the cheminformatics community has adopted the concept of early enrichment, placing greater emphasis on the importance of the top performers of a model. Currently, this advance in early enrichment measurement is limited to classification and has not yet included regression analysis. χ Similarly, processing according to many embodiments of the present invention utilizes a new metric for early enrichment in regression For a given target:

[0120]

[0121] where the experimental (observed) measurement y of sample i is i According to the model (predicted) measurement of sample i In other words, compute the average z-score of the observed values ​​for the first x% of the scoring samples, rather than computing, for example, Its unit is the same as y i Same (ie, log(K i ) values). Unfortunately, this unnormalized approach depends on the distribution of the data set. For example, in log(K i ) is measured, if the maximum deviation from the mean is 1.0, then the best the model can do is to achieve a value of 1.0

[0122] Processing according to various embodiments of the invention is normalized by dividing by the standard deviation σ(y) of the data. This can allow comparison of model performance between data sets with common units of measurement but different variances of those measurements. Thus, the upper limit is equal to the right-hand side of (15), where the indexed set of molecules i constitutes a subset of the x·N most experimental molecules. This value depends on the distribution of the training data as well as the value x. is the average of the χ·N z scores, which are themselves real numbers of standard deviations away from the mean experimental activity. The χ·N value can therefore exceed 1.0, since this means that the χ percentage of the top predictors has an average standard deviation above the mean of 1.0.

[0123] Spatial graph convolutions exhibit state-of-the-art performance in affinity prediction. Spatial graph convolutions according to several embodiments of the present invention use a more principled deep learning approach. The input features are simply basic information about atoms, bonds, and distances. In various embodiments, this framework does not use traditional hand-crafted features such as hydrophobic effects, π stacking, or hydrogen bonding. Instead, high-level interaction "features" according to various embodiments of the present invention are learned through intermediate graph convolutional neural network layers. Although the input characterization is simpler, spatial graph convolutions according to various embodiments of the present invention can use the same relatively small amount of data as previous expertise-driven methods to learn an accurate mapping of protein-ligand structure to bonding free energy. If machine learning is to be applied to real-world drug discovery settings, then the ability of a given model to both interpolate within familiar regions of chemical space and generalize to less mapped regions must be measured.

[0124] In one aspect, the present disclosure provides a method and system for predicting molecular characteristics. The method comprises: (a) obtaining a molecular structure having N atoms and N et The molecular set represented by a graph with N edge types; (b) et (c) performing between 0 and S-1 additional graph convolutions, each with N et and (d) if feature prediction is desired, performing graph aggregation on a subgraph of the graph before further neural network layer processing.

[0125] The system includes one or more processors that are individually or collectively configured to: (a) obtain a et The molecular set represented by a graph with N edge types; (b) et (c) performing between 0 and S-1 additional graph convolutions, each with N etand (d) if feature prediction is desired, performing graph aggregation on a subgraph of the graph before further neural network layer processing.

[0126] In some embodiments, the representation of the set of molecules further comprises an NxN distance matrix. et The edge types may include bond types, salt bridges, pi stacks, distance bins, distance base sets, raw distances, or combinations thereof. In some embodiments, (b) further includes directly combining information from an NxN distance matrix. In some embodiments, (c) further includes performing graph convolution with an NxN distance matrix. In some embodiments, a subgraph of a graph may be a graph of one or more molecules.

[0127] In some embodiments, the subgraph of a graph can be 1 subgraph, 2 subgraphs, 3 subgraphs, 4 subgraphs, 5 subgraphs or more. In some embodiments, the subgraph of a graph can be 1 to 1,000,000 subgraphs, 1 to 10,000 subgraphs, 1 to 1,000 subgraphs, 1 to 100 subgraphs, 1 to 50 subgraphs, 1 to 20 subgraphs, 1 to 10 subgraphs or 1 to 5 subgraphs.

[0128] In one aspect, the present disclosure provides a method for predicting characteristics of molecules. The method includes: (a) performing a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules; (b) performing a second set of graph convolutions with the spatial graph representation, wherein the second set of graph convolutions is based at least on the distance between each atom and other atoms in the set of molecules; (c) performing graph aggregation with the spatial graph representation; and (d) predicting a set of one or more characteristics for the set of molecules. The set of molecules can be 1 molecule, 2 molecules, or more molecules.

[0129] In some embodiments, predicting a set of one or more characteristics includes evaluating data from graph aggregation. Processing according to many embodiments of the present invention can receive atomic information of a set of one or more molecules. The atomic information can be, but is not limited to: bond lengths within a molecule, bond strengths within a molecule, bond angles within a molecule, dihedral angles within a molecule, dihedral angles between two and / or more molecules, bond angles between two and / or more molecules, interatomic distances between atomic pairs of the same or two and / or more different molecules, interatomic angles between atomic triplets of the same or two and / or more different molecules, bond distances between two and / or more molecules, charge of a molecule, electronegativity of a molecule, dipoles of a given atomic pair, dipoles of a molecule, dipoles of a set of one or more molecules, and / or forces between two and / or more molecules. The forces between two and / or more molecules can be, but are not limited to: electrostatic, ionic attraction, intermediate dipole-dipole, weak London dispersion, hydrophobic interactions, hydrophilic interactions, van der Waals forces, hydrogen bonds, covalent bonds, metallic bonds, magnetic and / or physical. Atomic information of one or more atoms in a molecule may include, but is not limited to: chemical element, atomic number, number of protons, number of electrons, approximate mass, charge, diameter, shape, orbital shape, size, energy level, valence, magnetic moment and / or isotope.

[0130] In various embodiments, establishing a spatial graph representation may include generating a distance matrix and an adjacency tensor, wherein the distance matrix represents the distances between atoms of a set of molecules, and the adjacency tensor indicates a plurality of different edge types between atoms. The edge types may include, but are not limited to, covalent bonds, ionic bonds, polar bonds, metallic bonds, non-covalent bonds (e.g., π-π stacking), salt bridges, distance intervals (hard cutoff and / or expansion on Gaussian sums and / or other basis sets), and / or hydrogen bonds.

[0131] In many embodiments, the set of molecules includes ligand molecules and target molecules, wherein the rows of the distance matrix are ordered by membership in the ligand and target molecules.

[0132] In some embodiments, the bond between the molecular set comprises a covalent bond. In a plurality of embodiments, the bond between the molecular set comprises at least one of π-π stacking, hydrogen bonding, and hydrophobic contact. In some embodiments, the bond / edge between atoms in different molecules in the set comprises a distance between a soft or hard interval. In some embodiments, the molecular set may comprise a molecule. In some embodiments, the molecular set may comprise a molecule surrounded by solvent molecules.

[0133] In some embodiments, the second set of graph convolutions is further based on the bonds between the sets of molecules. In some embodiments, the first set of graph convolutions may be based on the first set of bonds between the sets of molecules, and the second set of graph convolutions may be based on the second set of bonds between the sets of molecules. In some embodiments, the first set of bonds is a subset of the second set of bonds.

[0134] A first set of performing graph convolutions according to many embodiments of the present invention includes performing operations at each layer of the graph convolution. In some embodiments, the operations may be, but are not limited to: a gated recurrent unit (GRU), a long short-term memory (LSTM), a gated linear unit (GLU), a recurrent neural network, a fully connected neural network, or a combination thereof. In multiple embodiments, a first set of performing graph convolutions includes performing a gated recurrent unit (GRU) operation at each layer of the graph convolution.

[0135] In several embodiments, performing a first set of graph convolutions includes utilizing a first plurality of neural networks, wherein each neural network in the plurality of neural networks can be used for a different bond type. In several embodiments, performing a second set of graph convolutions includes utilizing a second plurality of neural networks, wherein weights of the first plurality of neural networks can be shared with the second plurality of neural networks.

[0136] In several embodiments, performing the second set of graph convolutions includes utilizing a second plurality of neural networks, wherein neural networks of the second plurality of neural networks may utilize distance information regarding distances between atoms of the set of molecules.

[0137] In some embodiments, the collection of molecules includes ligand molecules and target molecules, wherein graph clustering can be performed only on the ligand molecules.

[0138] In some embodiments, the set of characteristics includes whether a first molecule in the set of molecules is bonded to a second molecule in the set of molecules.

[0139] In another aspect, the present disclosure provides a method for training a spatial convolutional graph model. The method includes: performing a first set of graph convolutions with a spatial convolutional graph model of a set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules; performing a second set of graph convolutions with the spatial convolutional graph model, wherein the second set of graph convolutions is based at least on the distance between each atom and other atoms in the set of molecules; performing graph aggregation with the spatial convolutional graph model; calculating the loss of the set of molecules; and updating the spatial convolutional graph model based on the calculated loss.

[0140] In some embodiments, the motion of a ensemble of molecules can be calculated by performing stochastic simulations, e.g., estimating solvation, estimating conformal entropy, exploring different conformations of both protein and ligand (e.g., "induced fit" bonding models). The predicted motions according to several embodiments of the present invention can be used to simulate key biophysical processes, such as (but not limited to) the bonding of ligands to proteins, which can depend on several properties that can be calculated by predicting the motion of a ensemble of molecules.

[0141] In certain embodiments, the processing and system may use the layers of the spatial convolutional graph model to train a set of one or more neural networks to predict a set of one or more parameters of a force field. The layers of the network may train a set of 1, 2, 3, 4, 5, 10, 20, 50, 100, 1000, 10000 or more neural networks to predict a set of one or more parameters of a force field. The layers of the network may train a set of 1-10000, 1-1000, 1-500, 1-100, 1-10 or 1-5 neural networks to predict a set of one or more parameters of a force field. The neural network may be used to predict a set of 1, 2, 3, 4, 5, 10, 20, 50, 100, 1000, 10000 or more parameters of a force field. The neural network may be used to predict a set of 1-10000, 1-1000, 1-500, 1-100, 1-10 or 1-5 parameters of a force field.

[0142] In various embodiments, a parameter set may be associated with a set of one or more characteristics, the set of characteristics including at least one of charge, bond angles, and dihedral angles. A parameter set may be associated with a set of 1, 2, 3, 4, 5, 10, 20, 50, 100, 1000, 10000, or more characteristics. A parameter set may be associated with a set of 1-10000, 1-1000, 1-500, 1-100, 1-10, or 1-5 characteristics.

[0143] In some embodiments, the spatial convolutional graph model may be a first spatial convolutional graph model, and the method further includes: training a second spatial convolutional graph model to predict potential energy; and predicting the potential energy of the molecular system based on the first and second spatial convolutional graph models.

[0144] In various embodiments, training the second spatial convolutional graph model includes sharing a set of one or more layers between the first and second spatial convolutional graph models. Training the second spatial convolutional graph model may include sharing a set of 1, 2, 3, 4, 5, 10, 20, 50, 100, 1000, 10000 or more layers between the first and second spatial convolutional graph models. Training the second spatial convolutional graph model may include sharing a set of 1-10000, 1-1000, 1-500, 1-100, 1-10 or 1-5 layers between the first and second spatial convolutional graph models.

[0145] In some embodiments, the force field is in the form of an Assisted Model Building with Energy Refinement (AMBER) function. In some embodiments, the force field function can be a complete neural potential and can have the characteristics of a fixed form potential such as, but not limited to, AMBER, CHARMM (with or without CMAP terms), GROMOS, AMOEBA, or a combination thereof.

[0146] In many embodiments, systems and processes can use the layers of the spatial convolutional graph model to train a collection of one or more neural networks to predict the potential energy of an input molecular system. The layers of the spatial convolutional graph model can be used to train a collection of 1, 2, 3, 4, 5, 10, 20, 50, 100, 1000, 10000 or more neural networks to predict the potential energy of an input molecular system. The layers of the spatial convolutional graph model can be used to train a collection of 1-10000, 1-1000, 1-500, 1-100, 1-10 or 1-5 neural networks to predict the potential energy of an input molecular system.

[0147] In various embodiments, a set of conformations of a set of molecules is identified by minimizing a potential energy predicted by a spatial graph convolutional model.

[0148] In another aspect, the present disclosure provides a system for predicting characteristics of molecules, the system comprising: one or more processors that are individually or collectively configured to perform a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules; perform a second set of graph convolutions with the spatial graph representation, wherein the second set of graph convolutions is based at least on the distance between each atom and every other atom in the set of molecules; perform graph clustering using the spatial graph representation; and predict a set of one or more characteristics for the set of molecules.

[0149] In another aspect, the present disclosure provides methods and systems for drug discovery. The method may include identifying candidate ligands from predicted molecular features using machine learning. In some embodiments, identifying candidate ligands includes performing a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules; performing a second set of graph convolutions with the spatial convolutional graph model, wherein the second set of graph convolutions is based at least on the distance between each atom and other atoms in the set of molecules; performing graph clustering with the spatial graph representation; predicting a set of one or more features of the set of molecules; and predicting candidate ligands from the set of one or more features of the set of molecules.

[0150] In some embodiments, the system includes one or more processors that are configured, individually or collectively, to: perform a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions are based on bonds between the set of molecules; perform a second set of graph convolutions with the spatial convolutional graph model, wherein the second set of graph convolutions are based at least on the distance between each atom and every other atom in the set of molecules; perform graph clustering using the spatial graph representation; predict a set of one or more features of the set of molecules; and predict a candidate ligand from the set of one or more features of the set of molecules.

[0151] In some embodiments, the set of one or more characteristics includes bonding affinity. In some embodiments, the set of one or more characteristics includes ligand conformation. In some embodiments, the set of one or more characteristics can be charge, toxicity, absorption, distribution, metabolism, elimination, CYP450 subtype inhibition, metabolic stability, membrane permeability, oral bioavailability, quantum electronic properties, solubility, LogD, or a combination thereof of the ligand.

[0152] In some embodiments, a pre-trained model can be used to help predict candidate ligands from a set of one or more features of a collection of molecules.

[0153] In various embodiments, in order to compare the performance of the proposed architecture with previous methods, performance was evaluated based on PDBBind 2007. In some embodiments, the PDBBind 2007 data set is split by (a) starting from a "refined" set including protein co-crystal structures and associated bonding free energy values; (b) removing the "core" set to form a test set, wherein (c) the remaining samples are used as training data. In the present disclosure, this training test split refers to "PDBBind 2007, improved training, core test" hereinafter, and the performance is compared with the RF score, X score and the network described in the present disclosure.

[0154] In the present disclosure, the present disclosure investigates a cross-validation strategy that, in some embodiments, splits all training data into three different folders with hierarchical clustering - training, validation, and test subsets based on pairwise structural and sequence homology as distance metrics. In various embodiments, the cross-validation for the benchmark uses a hyperparameter set.

[0155] Example

[0156] Provide the following example to illustrate but not limit the claimed invention.The performance of deep neural network algorithm is highly sensitive to the selected hyperparameters.This sensitivity emphasizes the importance of strict cross validation.In order to compare the performance of the proposed architecture with previous methods, performance was evaluated based on PDBBind 2007.In previous work, the PDBBind2007 data set was started by (a) from a "refined" set including 1300 protein co-crystal structures and associated bonding free energy values; (b) removing a "core" set including 195 samples to form a test set and split, wherein (c) the remaining 1095 samples were used as training data.In the present disclosure, this training test split refers to "PDBBind 2007, refined training, core test" hereinafter, and performance is compared with the network described in RF score, X score and the present disclosure.

[0157] In the present disclosure, the present disclosure investigates a cross-validation strategy that, in some embodiments, splits all training data into three different folders with hierarchical clustering - training, validation and test subsets - based on paired structural and sequence homology as distance metrics. In the standard PDBbind 2007 "Refine Training, Core Testing" benchmark, spatial graph convolutions achieve state-of-the-art performance reflected by several metrics. PotentialNet outperforms RF scores and X scores based on Pearson and Spearman correlation coefficients. The Pearson correlation scores for (7)-(14) are within the error of the reported TopologyNet scores, and TopologyNet is by far the best performing model in this benchmark. Cross-validation is performed on this (including all results reported in Tables I, II and III) so that the performance on the test set is recorded for the hyperparameter sets that perform best on different validation sets (Table VII). In contrast, the TopologyNet model is trained on a combination of the validation set and the training set and evaluated directly on the test set. Therefore, the performance for TopologyNet reflects a training validation type split, rather than a training validation test split, which may exaggerate the performance of the method. Tables IV-VI illustrate the comparative results for predicting quantum properties, toxicity, and solubility.

[0158] Table I. Benchmarks: PDBBind 2007, Refined Training, Core Test

[0159]

[0160] Table II. Benchmark: PDBBind 2007 refined, condensed sequence splits

[0161]

[0162] Table III. Benchmark: PDBBind 2007 refined, condensed sequence splits

[0163]

[0164] Table IV. Quantum property predictions using the QM8 dataset

[0165] network Effective MAE Testing MAE Spatial PotentialNet, phased 0.0120 0.0118(0.0003) Spatial PotentialNet, single update 0.0133 0.0131(0.0001) MPNN 0.0142 0.0139(0.0007) DTNN 0.0168 0.0163(0.0010)

[0166] Table V. Toxicity prediction using the Tox21 dataset

[0167]

[0168]

[0169] Table VI. Solubility prediction using the Delaney ESOL dataset

[0170] network Effective RMSE Test RMSE PotentialNet 0.517 0.490(0.014) Weave 0.549 0.553(0035) GraphConv 0.721 0.648(0.019) XGBoost 1.182 0.912(0.000)

[0171] Table VII. Hyperparameters used for the PotentialNet neural network

[0172]

[0173]

[0174] These examples are provided to illustrate, not to limit, the claimed invention.

[0175] Molecular dynamics with spatial graph convolution

[0176] While the neural network community is devoting more and more resources to generative models, such as generative adversarial networks (GANs) and variational autoencoders (VAEs), natural sciences have been studying a range of generative models since the 17th century: physics. Optical physics precludes the observation of real-time, atomically resolved behavior of biological macromolecules and the small molecule drugs bonded to them. While quantum mechanics (QM) imposes fundamental limits on such experimental observations, knowledge of its mathematical framework also grants the ability to theoretically model such atomic behavior with arbitrary precision.

[0177] The first Newtonian mechanics, based on three laws of motion and simple inverse square potentials, enabled incredibly accurate predictions of the future positions of objects given initial conditions. In the twentieth century, as measurements of micro- and nanoscale phenomena became increasingly common, quantum mechanics was invented to achieve unprecedented accuracy in predicting the long-term (if random) behavior of subatomic, atomic, and molecular systems.

[0178] Molecular dynamics (MD) is a family of techniques that make this theoretically inspired model of molecules more computationally tractable than pure QM by performing a Newtonian approximation to the underlying Schrodinger equation. There are two fundamental trade-offs in the examination of molecular systems: the trade-off between spatiotemporal resolution in experimental observations of the molecules, and the trade-off between speed and accuracy of computational methods that explicitly model all the atoms in the system. Molecular dynamics is a type of simulation that uses Newtonian mechanics to approximate the more realistic but slower-to-evaluate physical model implemented in quantum mechanics. Inaccuracies in molecular dynamics arise primarily from errors in the force field - inaccuracies in the potential energy function based on the positions of atoms in the system - and from inadequate simulation timescales.

[0179] Systems and methods according to several embodiments of the present invention provide a framework for developing more accurate estimates of energy and thus force fields for molecular systems. Processing according to multiple embodiments of the present invention deploys spatial graph convolution to reparameterize the auxiliary model building (AMBER) function form with energy refinement to more accurately represent the behavior of small organic molecules. The method according to some embodiments of the present invention can automatically generate new parameters for any given query biomolecular system, from single amino acids and bases to proteins and nucleic acids. In certain embodiments, processing can deploy graph convolution based on the AMBER function form that has dominated MD simulations in recent decades to train new force fields. In multiple embodiments, a deep neural network (DNN) can be used to learn a new function form that calculates energy based on learnable atomic types derived from fixed, simple initial characterizations and dynamic relative positions of atoms. In certain embodiments, potential energy functions can be constructed and evaluated based on spatial graph convolutions (such as those described above). A graph convolution potential (GCP) model according to certain embodiments of the present invention can be trained and evaluated in a data set consisting of advanced quantum mechanical calculations of the energy of multiple conformers of a set of small molecules. According to various embodiments of the present invention, the gradient of such an energy function calculated by back-propagation can form the basis for even more accurate molecular dynamics simulations that are not subject to the inherent limitations of the AMBER functional form.

[0180] The highly popular AMBER functional form is fitted to the AMBER and CHARMM force fields using a combination of quantum mechanical calculations, NMR, and other empirical data. The functional form is as follows:

[0181]

[0182] Scalar output V(r N ) is the potential energy, which is a function of the positions of atoms, the "types" of atoms, and parameters associated with those atoms, as well as 2-tuples (bonds, van der Waals forces, electrostatic forces), 3-tuples (angles), and 4-tuples (dihedral angles). In some embodiments, a neural network is trained. A neural network according to some embodiments of the present invention may take as input an initial feature quantity tensor (x, A, R) and calculate the parameters in (16) to most accurately calculate the potential energy, where x is N×f in matrix, where each of the N atoms is represented by a in The row vector representation of features; A is N et The adjacency matrix of the N×N×N edge types (bond types, non-covalent interactions, etc.) et tensor, and R is an N×N distance matrix.

[0183] In certain embodiments, processing can derive a dense, distinguishable feature map similar to the discrete "atom type" in the traditional force field for each atom. In some such embodiments, a data set (such as the ANI-1 data set) can be used to fit a new AMBER-like force field (FF), which applies "characterization" (charge, balanced bond length, angle, dihedral angle and its force constant) to amino acids and their small molecules based on its training on the data set. The recently released ANI-1 data set includes about 20 million total conformations distributed in approximately 60,000 small organic molecules, and the associated energy values ​​calculated by density functional theory (DFT) and quantum chemical methods. In certain embodiments, or strictly based on a complete PotentiaiNet or graph convolution potential or a force field whose parameters are derived from the AMBER function form of graph convolution can be trained with data from quantum chemical calculations (e.g., ANI-1 data set), NMR experimental data, etc. For a given small molecule, x and A will be fixed, but R will be different for each conformer. Spatial graph convolutions according to many embodiments of the present invention can learn atomic-level features, where these atomic-level features are continuous, distinguishable analogs of the "atom types" commonly seen in traditional force fields.

[0184] Phase 1:

[0185]

[0186] Phase 2:

[0187]

[0188] Mapping atomic feature maps to parameters

[0189]

[0190] in is a neural network that maps each row vector of atomic-level features to a single scalar representing a point charge, is a neural network that maps concatenated 2-tuples of atomic-level features to equilibrium bond lengths and force constants, A neural network that maps a cascade of 3-tuples of atomic-level features to equilibrium angles and force constants, a neural network that maps a cascade of 4-tuples of atomic-level features to equilibrium dihedral angles, frequencies and / or amplitudes, and a neural network that maps a cascade of 2-tuples of atomic-level features to van der Waals forces. Essentially, each parameter in a new AMBER-like force field can be learned with a neural network based on the atomic features learned in the middle layers (or equivalently, the last layer) of the spatial graph convolution. In various embodiments, discrete edge types can be used with spatial graph convolution.

[0191] For new molecules not in the training set, processing according to several embodiments of the present invention can input fixed characterizations x, A in a single forward pass and output the point charges and other parameters necessary for MD simulations in molecular dynamics simulation packages such as (but not limited to) QpenMM, AMBER, DESMOND and GROMACS. An additional computational step can be the automatic generation of force field files, which is a relatively straightforward exercise.

[0192] Full-image convolution potential

[0193] In many embodiments, the systems and methods utilize a novel potential function form trained end-to-end as a deep neural network predicted on a graph convolutional architecture. In graph theory literature, "node-level outputs" that represent scalars or vectors for each node are distinguished from "graph-level outputs" (which are single scalars or vectors that are functions of all nodes and edges in the graph). In some embodiments, graph-level outputs can be achieved through a "graph aggregation" layer that is invariant to atom / node ordering:

[0194]

[0195] The final weight matrix With the final output h (out put) The number of rows is the same as the desired number of dimensions.

[0196] If the final output is a scalar value and the spatial graph convolutional model (or PotentialNet) is trained with an input molecular system connected to an energy value as a label, then the model according to some embodiments of the present invention can be viewed as a potential energy function:

[0197] φ(x,A,R)=PotentialNet(x,A,R) (21)

[0198] The calculation of the potential energy φ(x, A, R) according to some embodiments of the present invention requires that for each graph convolution layer Without the neighbor list technique used in molecular simulation, this computational cost would climb to For each of the N atoms, N must be calculated in the previous case neighbors Or in the latter case, N-1 feed-forward neural networks are computed to obtain the overall message for each graph convolutional layer.

[0199] The AMBER force field functional form has successfully elucidated the dynamics of biological macromolecules and small molecules and their interactions. In some embodiments, the calculation of the total potential energy is calculated as the sum of the AMBER functional form (with either fixed or learnable parameters, with GAFF as an initial guess) with a spatial graph convolution (or PotentialNet):

[0200] φ(x, A, R) = PotentialNet (x, A, R) + AMBER (bonds, angles, torsions) (22)

[0201] In some embodiments, a combination of the AMBER functional form and spatial graph convolutions (or AmberNet) is contemplated:

[0202] φ(x,A,R)=PotentialNet(x,A,R)+AmberNet(x,A,R) (23)

[0203] In various embodiments, the atomic features h can be derived by sharing (spatial) The middle layers are used to reduce the total number of network parameters in this mixture.

[0204] Characterizing each conformation of a molecule may require (a) atomic-level features, which may be as simple as one-hot features for each element and its hybridization type, (b) adjacency tensors for bonds, and (c) interatomic distances. In various embodiments, a neural network regressor is trained based on PotentialNet to map inputs x, A, and R to energies for each conformer. Cross-validation strategies according to several embodiments of the invention include, but are not limited to, random, temporal (if available), scaffold, and SMILES distance condensation splitting. The goal here should be to achieve state-of-the-art energy estimates that are on par with experimental or DFT errors.

[0205] In some embodiments, after training PotentialNet(x,A,R) (e.g., using ANI-1 and / or other datasets that map conformers to energy values), the parameters of this model can be fixed to produce

[0206] In a Newtonian system, the forces on the atoms can be written as:

[0207]

[0208] In some embodiments, the gradient It can be calculated using the back-propagation algorithm implemented by many deep neural network software packages. Given the input feature x, the input adjacency matrix A, and the initial position R 0 , energy can be obtained through a single forward propagation of the network Then, back propagation will produce first-order derivative information about the coordinates encoded in R. Subsequently, according to some embodiments of the present invention, The gradient information encoded in is used to move the positions of N atoms to reduce the energy While the internal atomic distance matrix R can be input as features, in many embodiments, the initial three-dimensional positions x, y, z can be used as input, and then the features and R can be easily calculated differently in the neural network. The x, y, z position of each atom can be updated using the gradient according to many embodiments of the present invention with a small step size. Subsequently, in some embodiments, the gradient can be recalculated with the updated coordinates, and the resulting distance matrix R 1 , and the position can be updated again, and the process can continue iteratively until the calculated force falls below a certain tolerance.

[0209] In a Newtonian system, the equation of motion can be written as:

[0210]

[0211] Processing according to many embodiments of the present invention may use a variety of methods including, but not limited to, the Velocity Verlet algorithm to propagate given some initial coordinates (and the associated interatomic distance matrix R 0 ) of the atoms. The initial velocity may be determined by statistical mechanics techniques such as (but not limited to) the Maxwell-Boltzmann distribution, while the initial position may be determined by a combination of crystallography and energy minimization. Energy minimization according to some embodiments of the present invention may be performed by methods such as those described above. In some embodiments, the training data set does not contain water molecules in different conformers, so it may be desirable to move to (25) Add a T'IR3R or similar potential.

[0212] In many embodiments, it may be desirable to perform a first MD simulation with GCPs in dihedral space only. Some biophysicists believe that many biologically relevant dynamics of macromolecules arise solely from dihedral motions of rotatable bonds. A dihedral angle, or torsion angle, is defined by the angle between two planes defined by two consecutive series of four bonded atoms. There are a number of ways to constrain molecular simulations to dihedral space. First, in some embodiments, the gradient of the potential can be taken directly with respect to d dihedral angles θ:

[0213]

[0214] Furthermore, processing according to some embodiments of the present invention may compute further derivatives to propagate the dihedral / intrinsic coordinate space derivatives as derivatives from the original x, y, z coordinates. Those skilled in the art will recognize that there are a variety of methods for performing MD in dihedral space.

[0215] In various embodiments, full-graph convolution energies from PotentialNet or, alternatively, AmberNet can be used as the basis for a molecular docking energy function. In many embodiments, by seeding a given conformation of a given ligand with a given orientation, either energy minimization or stochastic dynamic simulations can be used to predict the overall energy-minimum conformation or docking pose of the ligand in the receptor. As described above, different variants of PotentialNet can be used to predict the bonding free energy / bonding affinity / valency of a given ligand-protein interaction. In some embodiments, the process can bond a pure graph convolution form of PotentialNet with a potential similar to AmberNet. By calculating the bonding energy as

[0216] ΔG(x,A,R)=PotentialNet(x,A,R)+AmberNet(x,A,R) (27)

[0217] Concepts such as conformational entropy, torsional strain terms, intramolecular collisions, etc. can be combined with classical molecular docking methods, but these parameters can be derived through deep neural networks. In many embodiments, graph convolutions can be a natural functional form for calculating reaction coordinates to monitor and drive the collective dynamics of biomolecular systems.

[0218] Although the present invention has been described in certain specific aspects, many additional modifications and variations will be apparent to those skilled in the art. Therefore, it should be understood that the present invention may be practiced in a manner different from that specifically described. Therefore, the embodiments of the present invention are to be considered in all respects to be illustrative and not restrictive.

Claims

1. A method for training a spatial convolutional graph model, the method comprising: performing a first set of graph convolutions of the spatial convolutional graph model with the set of molecules, wherein the first set of graph convolutions is based on bonds between the set of molecules; performing a second set of graph convolutions with the spatial convolutional graph model, wherein the second set of graph convolutions is based at least on distances between each atom and every other atom in the set of molecules; Perform graph aggregation using spatial convolutional graph models; Computing the loss of a collection of molecules based on graph aggregation; and Update the spatial convolutional graph model based on the calculated loss.

2. The method of claim 1 further comprises training a set of one or more neural networks using layers of a spatial convolutional graph model to predict a set of one or more parameters of a force field.

3. The method of claim 2, wherein the set of parameters is associated with a set of one or more characteristics, wherein the set of characteristics comprises at least one of charge, bonding, angle, and dihedral.

4. The method of claim 2, wherein the spatial convolutional graph model is a first spatial convolutional graph model, wherein the method further comprises training a second spatial convolutional graph model to predict potential energy; and Predicting the potential energy of molecular systems based on first and second spatial convolutional graph models.

5. The method of claim 4, wherein training the second spatial convolutional graph model comprises sharing a set of one or more layers between the first and second spatial convolutional graph models.

6. The method of claim 2, wherein the force field is in the form of Assisted Model Building with Energy Refinement (AMBER) functions.

7. The method of claim 1 further comprises using the layers of the spatial convolutional graph model to train an ensemble of one or more neural networks to predict the potential energy of an input molecular system.

8. The method of claim 7, further comprising identifying a set of conformations of the set of molecules by minimizing a potential energy predicted by a spatial graph convolutional model.

9. The method of claim 1, further comprising predicting the motion of a collection of molecules.

10. A method for predicting a characteristic of a molecule, the method comprising: is obtained as having N atoms and N et A set of molecules represented by a graph of edge types; With N et The first set of graph convolutions is performed using a graph representation of a set of molecules with a subset of edge types; Perform one or more additional sets of graph convolutions, each with N et A subset of edge types; as well as Graph aggregation is performed on subgraphs of the graph and then on further neural network layers.

11. The method of claim 10, wherein the representation of the set of molecules further comprises an NxN distance matrix.

12. The method of claim 10, wherein N et The edge types include at least one of a bond type, a salt bridge, a pi stack, a distance bin, a distance basis set, and a raw distance.

13. The method of claim 10, wherein performing a first set of graph convolutions comprises directly incorporating information from an NxN distance matrix.

14. The method of claim 10, wherein performing one or more additional sets of graph convolutions is based on an NxN distance matrix. The method of claim 10 , wherein a subgraph of a graph can be one or more of the molecules.

Images