Molecular interaction prediction by deep kernel inversion (DKI)

Abstract

Prediction of binding affinity is provided. A first and second molecular descriptor are provided to a trained machine learning model. Each of the molecular descriptors represent a plurality of chemical constituents and a set of corresponding coordinates. A vectorized representation of each frame of the first and second molecular descriptors is received from the trained machine learning model. The vectorized representations have a real component and an imaginary component for each dimension thereof. A dot product of the vectorized representations is determined. Based on the dot product, an estimate is produced of a binding affinity between the two molecular descriptors.

Description

MOLECULAR INTERACTION PREDICTION BY DEEP KERNEL INVERSION (DKI)CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U. S. Provisional Patent Application No. 63 / 713,722, filed on October 30, 2024, entitled “MOLECULAR INTERACTION PREDICTION BY DEEP KERNEL INVERSION (DKI),” the disclosure of which is hereby incorporated by reference in its entirety.BACKGROUND

[0002] Embodiments of the present disclosure relate to computational biology, and more specifically, to the prediction of molecule interactions.BRIEF SUMMARY

[0003] According to embodiments of the present disclosure, methods of and computer program products for predicting binding affinity are provided. A first and second molecular descriptor are provided to a trained machine learning model. The molecular descriptors represent a plurality of chemical constituents and a set of corresponding coordinates. A vectorized representation of each frame of the first and second molecular descriptors is received from the trained machine learning model. The vectorized representation includes a real and imaginary component for each dimension. A dot product of the vectorized representations of the first and second molecular descriptors is determined. An estimate of a binding affinity between the two molecular descriptors is produced based on the dot product.

[0004] According to embodiments of the present disclosure, methods of and computer program products for training a machine learning model to encode molecular descriptors are provided. A training dataset is read. The training dataset comprises a plurality of molecular descriptors, eachof the plurality of molecular descriptors comprising a plurality of chemical constituents and corresponding coordinates. A plurality of residue pairs is generated from the plurality of molecular descriptors. The training dataset and the plurality of residue pairs are provided to the machine learning model. The machine learning model is thereby configured to output a vectorized representation of an input molecular descriptor. The vectorized representation defines an embedding having a real component and an imaginary component for each dimension thereof.

[0005] According to embodiments of the present disclosure, methods of generating a molecular interaction database are provided. A plurality of molecular descriptors is read from a molecular descriptor characteristic database. Each molecular descriptor of the the plurality of molecular descriptors comprises a plurality of chemical constituents and a plurality of corresponding coordinates. A target molecule descriptor is read. The target molecule comprises a target plurality of chemical constituents and a target set of corresponding coordinates. A binding affinity binding affinity between the target molecular descriptor and each of the plurality of molecular descriptors is predicted, and a molecular interaction database comprising the predicted binding affinities is output.

[0006] According to embodiments of the present disclosure, a system is provided that comprises a molecular descriptor characteristic database and a computing node. The computing node comprises a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor of the computing node to perform a method for predicting molecular descriptor interactions. A first and second molecular descriptor are provided to a trained machine learning model. The molecular descriptors represent a plurality of chemical constituents and a set of corresponding coordinates. A vectorized representation of each frame of the first and second molecular descriptors is received from thetrained machine learning model. The vectorized representation includes a real and imaginary component for each dimension. A dot product of the vectorized representations of the first and second molecular descriptors is determined. An estimate of a binding affinity between the two molecular descriptors is produced based on the dot product.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0007] Fig. 1A is a schematic view of a plurality of proteins and their patterns of interaction according to embodiments of the present disclosure.

[0008] Fig. 1B is a schematic view of pairs of vectorized proteins illustrating the determination of interactions according to embodiments of the present disclosure.

[0009] Fig. 1C is a schematic view of exemplary vectorized proteins according to embodiments of the present disclosure.

[0010] Fig. ID is a schematic view of an interaction chart according to embodiments of the present disclosure.

[0011] Fig.2 is a graph of precision versus recall, illustrating the performance of methods according embodiments of the present disclosure.

[0012] Fig.3 is a graph of predicted versus measured binding values according embodiments of the present disclosure.

[0013] Fig.4 is a graph of protein fraction versus interaction count, representing an exemplary distribution of interactions across a PPI network according to embodiments of the present disclosure.

[0014] Figs.5A-B are a graph and key depicting a UMAP projection of the vector space embeddings for the human Alphafold Protein Structure Database according to embodiments of the present disclosure.

[0015] Fig.6 is an architectural diagram of an exemplary neural network according to embodiments of the present disclosure.

[0016] Figs.7A-B are block diagrams illustrating a vectorization process according to embodiments of the present disclosure.

[0017] Fig.8 is a flow chart depicting an exemplary method of producing an estimate of a binding affinity according to embodiments of the present disclosure.

[0018] Fig.9 is a flow chart depicting an exemplary method of training a machine learning model according to embodiments of the present disclosure.

[0019] Fig. 10 depicts a computing node according to embodiments of the present disclosure.DETAILED DESCRIPTION

[0020] The following description sets forth exemplary aspects of the present disclosure. It should be recognized, however, that such description is not intended as a limitation on the scope of the present disclosure. Rather, the description also encompasses combinations and modifications to those exemplary aspects described herein.

[0021] All biological systems entail a huge number of molecular interactions, and the goal of any drug is to influence a subset of these interactions associated with a particular disease. As such, computational drug discovery requires a deep understanding of all interactions — both the interaction between the drug and target molecules (efficacy) and the many more potential interactions with molecules that must be avoided (specificity).

[0022] Various methods may be employed to predict the interactions between pairs of molecules. However, these methods are difficult to scale, and the number of potential interactions between molecules in human biology is immense. Focusing on an important subset of these interactions — protein vs. protein interactions (PPIs), it is estimated that there are ~105different proteins in the human proteome, which means there are ~1010possible PPIs. Various methods may be used to increase the scale of experimental methods for detecting PPIs, such as yeast two-hybrid assays and co-immunoprecipitation, but the current state of the art is capable of measuring only ~105possible interactions. Computational approaches utilizing protein structure prediction algorithms such as AlphaFold2 and RoseTTAFold may be employed. While potentially more scalable than experimental methods, such techniques rely on querying computationally intensive models for every possible protein-protein pair. As such, they cannot scale to the entire PPI network, with predicted networks built by only querying ~106potential interactions. Therefore, there exists a need for an improved method to predict interactions between proteins or other molecules base on their molecular descriptors.

[0023] Vector representations as set forth herein offer an effective solution to the problem of inferring entire PPI networks. The interaction between two proteins is a complicated, non-linear process, but this does not preclude the existence of a high-dimensional vector space where this interaction can be approximated as a dot product (a quick and low-cost calculation). If all proteins in a proteome are mapped into such a vector space, then the entire PPI network can be inferred nearly instantly by calculating the dot product between all vector representations.

[0024] In some embodiments, methods provided herein for embedding or projection are referred to as Deep Kernel Inversion (DKI). DKI projects each molecular descriptor into a highdimensional vector space using only the structure of the protein. In this vector space, the dot product between any two vectors is a highly accurate prediction of whether the vectors' corresponding proteins will interact. Each protein in a proteome needs to be embedded into a vector space only once. Once embedded, predicting interactions between a protein and any other already-embedded proteins becomes a computationally trivial task of calculating the dot productbetween vectors. In other words, this changes the task of predicting an entire PPI network from O(n2) to O(n). For the human proteome, this means a 100,000-fold reduction in computation, making it feasible for the first time to predict all possible human protein PPIs accurately.Additionally, this approach can be applied to interactions between all molecules, not just proteins, and it could potentially capture an even richer set of properties beyond static interactions.

[0025] Alternative approaches to predicting interactions between proteins have various drawbacks. Some alternative approaches use co-evolutionary data, essentially relying on the interactions between similar proteins in other species. For example, multiple sequence alignment (MSA) is used to identify proteins with similar sequences or subsequences from which properties such as structure or interactions can be inferred. The challenge with MSA- or co-evolutionary-based approaches is multifold. First, MSA is computationally expensive. Even with attempts to accelerate the computation, it remains the case that every protein pair within a proteome must be considered. This means querying a model billions of times, and none of these methods are capable of handling that scale. Another challenge with MSA-dependent methods is that they are less effective with proteins that have fewer matching sequences, and especially with novel or unseen proteins.

[0026] One use case of PPI prediction is in designing de novo proteins (novel proteins that are dissimilar to any protein found in nature). DKI can be applied to predict all interactions a newly designed protein will have with 120K different extracellular epitopes within the human proteome in order to optimize their specificity. With MSA-based methods, this would be impossible, both because no similar sequence to the designed protein has ever been observed, and because querying a model 120K times at each step in an iterative protein design process is unfeasible. Insummary, MSA-based methods are simply extrapolating previously observed patterns, and they are prohibitively slow in doing so.

[0027] With reference now to Figs. 1A-D, the utilization of vectorization to predict interactions based on molecular descriptors is illustrated. In some embodiments, a molecular descriptor may correspond to a protein. Fig. 1A illustrates multiple 3D representations of proteins corresponding to exemplary molecular descriptors 101...104, with certain binding sites 111...119 indicated. Fig. 1B further illustrates these binding sites in a series of representative vectors. In some embodiments, a deep learning model is trained to predict, for each molecular descriptor, the possible interaction patterns based on chemical structure, and embed these predictions into one or more representative vectors. Fig. 1C illustrates the determination of the dot product of two proteins (Pl and P2; Pl and P3). The dot product between the two vectors is the intersection of interaction patterns between the two molecular descriptors and the binding sites of the corresponding molecules. Fig. ID illustrates the dot products across all pairs of vectors. This is a fast and accurate prediction of which molecular descriptor pairs interact. In some embodiments, the diagonal of the matrix (which represents the molecule corresponding to a molecular descriptor’s proclivity to bind to itself, or self-dimerize), is an edge case calculated in an alternative manner.

[0028] Kernel functions are essentially similarity measures between items within a set, and they are used throughout machine learning. Various machine learning algorithms work by making a series of comparisons between items within a dataset, but these comparisons are usually made via a dot product, which leads to models only being capable of finding linear patterns. It has been shown, however, that nonlinear kernels with certain properties can be substituted for these dot products, leading to more sophisticated pattern finding. The "kernel trick" in Support VectorMachines is such an example. The interchangeability of nonlinear kernels with dot products arises from the equivalence these kernels have to dot products in a much higher-dimensional vector space. What is a nonlinear comparison in lower dimensions is equivalent to a linear comparison when projected into a much higher dimensional space. The methods presented herein begin with a nonlinear and expensive-to-calculate kernel (i.e., the likelihood that two proteins will interact), and finds a higher-dimensional vector space in which the dot product is (near) equivalent to the kernel. In doing so, each protein only needs to be projected into this vector space once, and then simply by taking the dot product between all proteins, one can infer all interactions with no additional querying of the model. Vector representations are also related to the idea of higher-dimensional encodings. Vector representations of words were heavily used in the early days of deep learning-based natural language processing because they could capture additional semantic information within the vectors themselves. This meant that by building a vector space based on word co-occurrences across text, words could be fed into neural networks with more information than a simple one-hot encoding scheme. With word encodings, however, the enormity of data and the fact that words co-occur with many other words to varying degrees allowed the vectors to be inferred directly. There was no need to “predict" each word’s vector or map a word’s properties to its vector. Here, protein interactions are highly sparse, with most proteins having zero or one interaction that has been captured in structure. Thus, deep learning is relied upon to map the structure of the protein to its appropriate projection into the vector space.

[0029] Fig.2 is a graph illustrating the performance of DKI against PEPPI. PEPPI is a machine learning pipeline that utilizes multiple models based on structural and sequential similarities, which are then used as inputs into a naive Bayesian classifier. Unlike DKI, PEPPI is reliant on MSA, and as such, exhibits variability in performance across datasets depending on the numberof matched sequences available for the Protein-Protein Interaction networks (PPIs). This model and the DKI model are applied to five different yeast PPI datasets: a PPI network inferred via yeast two-hybrid assays, a literature-curated PPI network, a PPI network inferred from affinity purification and mass spectrometry, and two additional datasets from the Yeast Interactome Project (http: / / interactome.dfci.harvard.edu / S_cerevisiae / ). These datasets had overlapping examples with the training dataset, so those examples were removed before running both models. The models were tested on a balanced dataset: for each true PPI, one pair of non-interacting proteins as a negative example was randomly sampled.DatasetPEPPI DKIY2H 0.867 0.945LC 0.931 0.941Collins 0.783 0.908CCSB-PRS 0.761 0.917CCSB-Binary-GS 0.876 0.960Table 1: Performance of DKI versus PEPPI in predicting protein-protein interactions across multiple yeast interactome datasets.

[0030] In all tested datasets, DKI outperformed PEPPI. PEPPI must be run on a traditional CPU and took days to complete the tasks, with most of the compute time being spent on MSA. DKI is able to be run on a GPU and completed all datasets in minutes.

[0031] DKI was also compared to AlphaFold2, RoseTTAFold, and coevolution of proteins across species (referred to as "Directed-Coupling Analysis" or DCA) to predict interactions among human mitochondrial proteins. DKI was applied to the same datasets as the system in Pei et al., “Human mitochondrial protein complexes revealed by large-scale coevolution analysis and deep learning -based structure modeling” (2022), hereinafter “Pei” to compare performancewith the results reported in Pei. A positive-to-negative example ratio of 1: 100 was used. The results are shown in Table 2. The DKI model outperforms the three different methods described in Pei by a wide margin. Additionally, Humphreys et. al., “Computed structures of core eukaryotic protein complexes” (2021), hereinafter Humphrey supplied similar methods to the yeast proteome, noting that the most accurate AlphaFold2-based method would take up to 1 million "GPU hours" to predict the interactions among the 4K proteins in the yeast proteome. DKI takes less than one hour on a modestly powered GPU.Model PR-AUCDKI 0.304AF2 0.132RoseTTAFold 0.113DCA 0.014

[0032] Fig.3 illustrates the DKI predicted logarithmic binding affinity versus the measured logarithmic binding affinity. Different interacting pairs have affinities (denoted as Kd) that vary from 10-3molar to 10-15molar. The affinity between two interacting molecules is a function of how quickly, how often, and for how long they remain bound to each other, and thus plays a crucial role in molecular dynamics. In some embodiments, the DKI model is configured to output a predicted log(Kd) (instead of a binary bind / no-bind probability). PDBBind (http: / / www.pdbbind.org.cn / index.php), a collection of many molecular interactions and their experimentally measured affinities, was utilized to first fine-tune and then test the model, hence ensuring that the test examples had minimal or no structural similarity to examples used for training and fine-tuning. DKI achieved a Pearson correlation coefficient of R = 0.677.Alternative methodologies rely on querying the model with every possible pair, causing compounding compute overhead and vastly increasing the amount of time and power necessary to calculate at this scale. DKI’s ability to predict binding affinities for every single proteinprotein interaction across an entire proteome is a significant advantage.

[0033] Fig.4 is a graph representing the distribution of interactions across a PPI network.

[0034] To demonstrate the scalability of DKI, a complete PPI network for the entire set of human protein structures in the AlphaFold Protein Structure Database was predicted. In total, this network includes 20,425 proteins, implying that there are 208.6 million possible interactions that could exist. According to Humphries, it would take somewhere between 90 million and 900 million GPU hours to apply AlphaFold2 to all of these possible interactions. DKI ran on 4 Nvidia A100 GPUs for 4.5 hours to complete the entire network. In total, DKI predicted 147,291 interactions across the 20,425 structures.

[0035] Referring now to Figs.5A-B, a UMAP projection of the vector space embeddings for the human AlphaFold Protein Structure Database visualizes the vector space. In the embedding of Fig. 5A, two proteins that are near each other are more likely to interact with the same other proteins but not necessarily with each other. It is not uncommon for adjacently embedded proteins to have a very low likelihood of interaction with each other. To confirm the common binding of nearby proteins, embeddings were clustered, and the resulting clusters were labeled based on which Gene Ontology term showed the most differentiated occurrence for proteins within a cluster compared to all other proteins. Fig.5B illustrates the Gene Ontology terms used to cluster the results. Many of the differentiating Gene Ontology terms represent binding to proteins and other molecules, such as “Antigen binding," “Sequence-specific double-stranded DNA binding," and “Ubiquitin protein ligase activity." This implies that many of the embeddings within these clusters are defined by binding to the same 3rd-party molecules. This proximity of the embeddings based on common binding makes DKI particularly useful in predicting the specificity of therapeutics. However, the proximity of embeddings to one another does not imply that the two bind to one another. Designing therapeutics for specificity, or thedegree to which a therapeutic molecule avoids interacting with off-target proteins, is a significantly harder problem than on-target binding affinity because it means optimizing interactions with more than 100,000 epitopes throughout human biology. Using DKI, every extracellular epitope in the human proteome was mapped and the vector embeddings were calculated. Then, when designing a new molecule targeting a specific epitope, one can use embeddings adjacent to the target epitope's embedding to identify the proteins most likely to have off-target binding. This approach, in combination with using DKI to rapidly predict interactions between designed molecules and every protein in the human proteome, means that one is able to design for specificity by minimizing every possible off-target interaction.

[0036] One challenge in building such a vector representation of proteins (and thus molecular descriptors) is that the dot product of any vector with itself will be positive. Given that the vector space must be high-dimensional to capture all the variance and dynamics between interacting proteins, the vector's self dot product will likely be higher than its dot product with most other proteins’ vectors (as briefly discussed in connection with Fig. ID). This leads to a prediction that all proteins are self-binding dimers, which is accurate for only a subset of proteins. Additionally, many protein pairs with similar or homologous structures may be incorrectly predicted to interact with high probability.

[0037] To address this, the representative vectors are complex-valued, e.g.,for all i, where D is the vector space dimension. In some embodiments, imaginary components of the vector entries capture "anti-self" binding properties — such as convex / concave shapes, hydrogenbond donating / accepting atoms, and other structural elements that interact strongly with their complement and not with themselves. In some embodiments, real components of the vectorentries capture self-interacting elements like hydrophobic surfaces and protein interfaces that match in surface area size.

[0038] The vector space needs to be high-dimensional to capture the nonlinear dynamics of molecular interactions. In some embodiments, the method will include a scaling factor in the dot product for effective training of the model. This leads to the final expression:P(i binds to j) = sigmoidwhere VHis the conjugate transpose of vector v, e.g.,

[0039] If, for example, a molecular surface has a large number of positively charged residues arranged in a concave shape, then Re(Vj) might be small in magnitude while Im(Vj) is large. When dotted with itself, the imaginary component would dominate, resulting in a low predicted probability of binding. In contrast, if the molecular surface were flat and hydrophobic, the real component would likely dominate.

[0040] Given the large variability in protein sizes, encoding an entire protein into a single vector might not be feasible. This can result in various surfaces on the moleculule that could interact with other molecules. In one version of the DKI model, this issue was overcome by including specific "binding site" residues of the protein (residues that are within 8 angstroms of the other protein upon binding) as model inputs. Predicting the binding site on one protein without knowing the other protein is challenging. In some scenarios, the primary focus is on designing small protein ligands (less than 100 residues) that bind to larger receptor proteins. This means the binding sites are of specific sizes, reducing variability and enabling the binding site predictor to achieve a per-residue PR-AUC > 0.95. With the goal of modeling all PPI, the system does not have the benefit of consistent binding site size, and an equivalent binding site classifier was only able to achieve a PR-AUC=0.79. This would mean that the binding site inputs into the generalPPI model would be highly variable. It was found to be more effective to minimize variability in the inputs and let the general model encode the variability in the resulting vector space. Thus, instead of including the entire binding site, the system includes a "seed residue" for the protein. The seed residue is a surface residue (within 1 angstrom from the protein surface) that is minimally distant from all other residues in the other protein. This is similar to "hot residues," although it is not required for the residue to be crucial for the interaction. The “seed residue” represents the "center" of where the binding site might be. In constructing a PPI network, a vector encoding is generated for every surface residue in each protein. The probability that two proteins interact is the maximum probability predicted across all of their respective seed residue encodings.

[0041] Fig. 6 is an architectural diagram of an exemplary neural network 600. In some embodiments, a deep learning model maps a molecular structure through its molecular descriptor to its vector space projection in CD. First, the mapping from structure to vector must be translationally and rotationally equivariant. This is achieved by restricting the input features 610 to the amino acid types and the Euclidean distances between the Cabackbone atoms of residues.

[0042] In some embodiments, the architecture consists of two parts: the residue pair encoder 620 and a self-attention-based "reducer" 630. The residue pair encoder 620 learns vector representations for each pair of residues / amino acids in the molecular descriptor. The pair encoder 620 comprises multiple blocks of layers, each block being a transformer that performs two self-attentions — one with the first residue constant and the second varying, and a second with the second residue constant and the first varying. In some embodiments, the pair encoder 620 may comprise up to twelve layers. Specifically, in any given layer, the attention vector for residue pair i, j isAwhere q1I-J, ' kP ll \ ' vPt P ' qkPlJ ' k^ IllpJ, ' and vP lilPj are vectors of dimension d ’h*■., ' and dh= 128. Theabove equation represents "single-headed" attention. Multi-headed attention can be employed, but the specific notation has been omitted for clarity.

[0043] The residue pair encoder 620 assigns a vector for each pair of residues that comprise the molecule. The residue pair encoder 620 is many layers deep, with each layer running an attention mechanism across the previous layer’s vectors to yield a refined vector representation for each residue pair. The top layer of the residue pair encoder 620 produces final vector representations for every residue pair. The attention reducer 630 uses an attention mechanism across all of these vectors to produce a single vector that represents all residue pair vectors.

[0044] Figs.7A-B are block diagrams illustrating the vectorization process. The output of the neural network is a single vector (e.g., the molecular descriptor’s vector representation).Therefore, on top of the L residue pair transformers, an attention reducer layer can be employed to "reduce" the N2residue pair encodings into a single vector. This is also achieved using selfattention:ReducedVector = “S / i (Zt; softmax (w j) •where H is the number of attention heads (e.g., H = 8), w^j is a scalar, and v-j G IRD, both generated by single-layer linear feed-forward networks using the residue pair ij’s encoding as inputs. D is the dimension of the vector space into which the molecular descriptors are mapped.

[0045] The ReducedVector is then passed through LayerNormalization and a multi-layer feedforward network. In some embodiments, the multi-layer feed-forward network may have twolayers. In some embodiments, the method can produce the real and imaginary components of the vector separately using independent two-layer feed-forward networks. In some embodiments, an independent feed-forward network is utilized to produce a bias vector.

[0046] While certain exemplary embodiments provided herein refer to a molecular descriptor, it will be appreciated that the present disclosure is applicable more generally to any molecular descriptor that physically characterizes a small molecule or a polymer. For example, a molecular descriptor in some embodiments comprises a plurality of chemical constituents and a set of corresponding coordinates. In the case of a small molecule, the chemical constituents may be functional groups and the corresponding coordinates may indicate the relative position of the functional groups. In the case of a protein, the chemical constituents may be residues and the corresponding coordinates are distances between pairs of residues.

[0047] Fig.8 is a flow chart illustrating an exemplary method 800 of training a machine learning model. Projecting molecular surfaces into a high-dimensional vector space enables significant improvements in PPI network inference and also introduces a more efficient training process. A naive training process might randomly sample pairs of molecular descriptors, some of which are known interactions while others are negative examples (two molecular descriptors sampled randomly with no known interaction). The high sparsity of interactions means the false negative rate of this sampling is negligible. The forward pass step in backpropagation must output predicted vectors for every sampled molecular descriptor pair. Once generated, all vectors within a training batch can be compared to each other and to the other molecular descriptors in the pair with no additional computation. Exploiting the sparsity of PPI networks, the method 800 can add these comparisons to the loss without increasing the compute requirements for the backward pass.

[0048] At 810, a training dataset is read. The training dataset comprises a plurality of molecular descriptors, each of the plurality of molecular descriptors comprising a plurality of chemical constituents and corresponding coordinates. At 820, a plurality of residue pairs is generated from the plurality of molecular descriptors. At 830, the training dataset and the plurality of residue pairs are provided to the machine learning model. The machine learning model is thereby configured to output a vectorized representation of an input molecular descriptor. The vectorized representation defines an embedding having a real component and an imaginary component for each dimension thereof.

[0049] More specifically, given a training batch size of b, let 1^ 6 {0,1} indicate whether the i-th and j-th training examples in the batch are known to interact (e.g., positive training examples). The binary cross-entropy loss is then

[0050] This expression can be computed entirely through matrix operations, so when running on GPUs, inter- vector comparisons add comparatively negligible time to the training step.

[0051] In some embodiments, a batch size of 80 molecular descriptors can be used during training, which means the traditional classifier approach to training would train on 80 / 2 = 40 molecular descriptor pair examples per batch. In some embodiments of the DKI method, each batch contains 80 * 79 / 2 = 3160 molecular descriptor pair examples, representing a significant gain in training efficiency. In some embodiments, since nearly all inter-pair comparisons are labeled as negative examples, the method samples positive pairs to populate the batch. Thus, each sampled molecular descriptor has one positively labeled comparison in the batch and 79 negative ones.

[0052] To train the vector encoder, the model training method extracted a dataset of structural interactions between different molecular descriptors from publicly available PDBs (810). In some embodiments, the dataset includes the plurality of chemical constituents and the corresponding coordinates of each extracted molecular descriptor. In some embodiments, if two chains had three or more residues in contact (Ca-Cadistance < 8 angstroms) and both chains were mapped to different UNIPROT IDs, the molecular descriptor pair can be included in the dataset. Each unique molecular descriptor pair may be included once, even if it appeared multiple times in the same or different PDBs. The result was 13,252 generated molecular descriptor pairs (820). In some embodiments, the model training method partitioned the pairs such that 80% were in the training dataset, 10% were in a validation dataset (used for early stopping during training to prevent overfitting and for hyperparameter optimization), and 10% were used in a test set. The model training method used Uniref90 clusters to ensure that all pairs with sequential similarities were not split across datasets to prevent data leakage. The machine learning model was thereby configured to output a vectorized representation of an input molecular descriptor, the vectorized representation defining an embedding having a real component and an imaginary component for each dimension thereof (830).

[0053] Since molecular interactions can be complex and nonlinear, the vector space required to encode these interactions linearly must be large. Through use of the validation set, testing determined that the optimal dimension for the vector space was D = 1024. However, larger vector spaces become more sparsely populated, and the Curse of Dimensionality becomes a factor. To address this, loosening restrictions on including duplicates and chains not mapped to UNIPROT IDs in the dataset was attempted. This increased the dataset size to nearly 100K but decreased example quality, resulting in only a minor improvement in performance.

[0054] To address the massive data need, a proprietary dataset of structural molecular interactions including 2.92 million relevant training examples of PPI was employed. In some embodiments, the model is pre-trained on this proprietary dataset (with special consideration to avoid data leakage) and then fine-tuned on the 13K examples from public data. This approach significantly improved performance, achieving a PR-AUC of 0.57 (noting that the positive to negative example ratio is 1:79).

[0055] The output of the DKI model can be modified, and the model is fine-tuned on a binding affinity dataset. Specifically, the output was modified to

[0056] where Kdis the binding affinity between molecules corresponding to molecular descriptor i and molecular descriptor j. The subset of PDBBind protein-protein interactions were used where Kdwere measured (as opposed to other measures of affinity), resulting in 1601 interactions. In training, validation, and test sets, these interactions were split, while ensuring Uniref90 partitioning was consistent with other training datasets to prevent data leakage.

[0057] In some embodiments, the method extracted from the Alphafold Protein Structure Database the 20,425 “VI" structures mapped to human UNIPROT records. In each structure, the method removed unfolded or low-confidence regions before embedding them into the vector space by removing any 4 or more consecutive residues with pLDDT<50.

[0058] Fig.9 is a flow chart illustrating an exemplary method 900 of utilizing the trained machine learning model. A first and second molecular descriptor are provided to the machine learning model, each of the first and second molecular descriptors comprising a plurality of chemical constituents and a set of corresponding coordinates (910). The machine learning model produces a vectorized representation having a real component (e.g., a representation ofhydrophobic surfaces and protein interfaces that match in surface area size) and an imaginary component (e.g., a representation of convex / concave shapes and hydrogen-bond donating / accepting atoms) for each dimension thereof (920). As discussed in Fig. 5, the real and imaginary components help in regulating predictions of self -dimerization and predictions for homologous molecular descriptors.

[0059] The dot product of the vectorized representations of the first and second molecular descriptors is determined and used to produce an estimate of the binding affinity of the two molecular descriptors. In an embodiment, an interaction between two molecular descriptors was predicted based on the max score across each molecular descriptors’ seed residue projections. In total, 57,120 seed residues were predicted to be interacting, from which 147,291 total molecular descriptor interactions were predicted.

[0060] Referring now to Fig. 10, a schematic of an example of a computing node is shown. Computing node 10 is only one example of a suitable computing node and is not intended to suggest any limitation as to the scope of use or functionality of embodiments described herein. Regardless, computing node 10 is capable of being implemented and / or performing any of the functionality set forth hereinabove.

[0061] In computing node 10 there is a computer system / server 12, which is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

[0062] Computer system / server 12 may be described in the general context of computer systemexecutable instructions, such as program modules, being executed by a computer system.Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system / server 12 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

[0063] As shown in Fig. 10, computer system / server 12 in computing node 10 is shown in the form of a general-purpose computing device. The components of computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including system memory 28 to processor 16.

[0064] Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect Express (PCIe), and Advanced Microcontroller Bus Architecture (AMBA).

[0065] Computer system / server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system / server 12, and it includes both volatile and non-volatile media, removable and non-removable media.

[0066] System memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a "hard drive"). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to bus 18 by one or more data media interfaces. As will be further depicted and described below, memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the disclosure.

[0067] Program / utility 40, having a set (at least one) of program modules 42, may be stored in memory 28 by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules 42 generally carry out the functions and / or methodologies of embodiments as described herein.

[0068] Computer system / server 12 may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with computer system / server 12; and / or any devices (e.g., network card, modem, etc.) that enable computer system / server 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interfaces 22. Still yet, computer system / server 12 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 20. As depicted, network adapter 20 communicates with the other components of computer system / server 12 via bus 18. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with computer system / server 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0069] The present disclosure may be embodied as a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0070] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0071] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0072] Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk,C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0073] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0074] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computerreadable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0075] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0076] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems thatperform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0077] The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

CLAIMSWhat is claimed is:

1. A method of predicting binding affinity, the method comprising:providing, to a trained machine learning model, a first and second molecular descriptor, each of the first and the second molecular descriptor representing a plurality of chemical constituents and a set of corresponding coordinates;receiving, from the trained machine learning model, a vectorized representation of each of the first and second molecular descriptors, the vectorized representation having a real component and an imaginary component for each dimension thereof;determining a dot product of the vectorized representations of the first and second molecular descriptors; andbased on the dot product, producing an estimate of a binding affinity between the two molecular descriptors.

2. A method of training a machine learning model to encode molecular descriptors, the method comprising:reading a training dataset, the training dataset comprising a plurality of molecular descriptors, each of the plurality of molecular descriptors representing a plurality of chemical constituents and a set of corresponding coordinates;generating a plurality of residue pairs from the plurality of molecular descriptors; andproviding the training dataset and the plurality of residue pairs to a machine learning model, thereby configuring the machine learning model to output a vectorized representation of an input molecular descriptor, the vectorized representation defining an embedding having a real component and an imaginary component for each dimension thereof.

3. The method of claim 1, wherein the trained machine learning model is trained according to the method of claim 2.

4. The method of any one of claims 1-3, wherein the chemical constituents are residues.

5. The method of any one of claims 1-4, wherein the chemical constituents are amino acids.

6. The method of any one of claims 1-5, wherein the corresponding coordinates are paired distances.

7. The method of any one of claims 1-6, wherein the dot product is scaled according to a scaling factor.

8. The method of any one of claims 1-7, wherein producing the estimate of the binding affinity further comprises determining a logarithm of the dot product.

9. The method of any one of claims 2-8, wherein the plurality of chemical constituents of each of the plurality of molecular descriptors of the training dataset is limited to binding residues within one angstrom of the corresponding coordinates.

10. The method of any of claims 2-9, wherein the embedding further comprises a bias scalar.

11. The method of any of claims 1-10, wherein the trained machine learning model is an artificial neural network.

12. The method of any of claims 1-11, wherein the trained machine learning model comprises a residue pair encoder, an attention reducer, and one or more feedforward networks.

13. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform the method of any one of claims 1-12.

14. A system comprising:a molecular descriptor characteristic database;a computing node comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor of the computing node to cause the processor to perform a method comprising:reading, from the molecular descriptor characteristic database, a first and second molecular descriptor, each of the first and second molecular descriptors representing a plurality of chemical constituents and a set of corresponding coordinates;providing the first and second molecular descriptors to a trained machine learning model;receiving, from the trained machine learning model, a vectorized representation of each frame of the first and second molecular descriptors, the vectorizedrepresentation having a real component and an imaginary component for each dimension thereof;determining a dot product of the vectorized representations of the first and second molecular descriptors; andbased on the dot product, producing an estimate of a binding affinity between the two molecular descriptors.

15. A method of generating a molecular interaction database, the method comprising:reading, from a molecular descriptor characteristic database, a plurality of molecular descriptors, each of the molecular descriptors comprising a plurality of chemical constituents and a plurality of corresponding coordinates;reading, a target molecular descriptor, the target molecular descriptor comprising a target plurality of chemical constituents and a target set of corresponding coordinates;predicting a binding affinity between the target molecular descriptor and each of the plurality of molecular descriptors by applying the method of any one of claims 1 or 2-12; andoutputting a molecular interaction database comprising the predicted binding affinities.

16. The system of claim 15, wherein the target molecular descriptor corresponds to a de novo protein.

17. A computer readable storage medium having encoded therein a database generated according to any one of claims 15-16.

Images