Training generative machine learning models on molecular dynamics simulation data

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a generative machine learning model In one aspect, a method comprises: obtaining a set of training examples for training a generative machine learning model to perform a structure prediction task; augmenting the set of training examples to include a plurality of new training examples, comprising, for each training example in the set of training examples: performing a molecular dynamics simulation of a molecular system, starting from a conformation of the molecular system that is specified by the training example, to generate a trajectory of new conformations adopted by the molecular system over a duration of time; generating one or more new training examples corresponding to the molecular system; and training the generative machine learning model to perform the structure prediction task on the augmented set of training examples.

Description

Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT ApplicationTRAINING GENERATIVE MACHINE LEARNING MODELS ON MOLECULARDYNAMICS SIMULATION DATABACKGROUND

[0001] This specification relates to training generative machine learning models on molecular dynamics simulation data.

[0002] Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model.

[0003] Some machine learning models are deep models that employ multiple layers of models to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output.SUMMARY

[0004] This specification describes a system implemented as computer programs on one or more computers in one or more locations that can train a generative machine learning model to perform a structure prediction task for molecular systems using training data generated by molecular dynamics simulations.

[0005] Throughout this specification, a “molecular system” can refer to any set of one or more molecules, e.g., including one or more of small molecules (e.g., molecules with a molecular mass <900 daltons), protein molecules, deoxyribonucleic acid (DNA) molecules, ribonucleic acid (RNA) molecules, lipid molecules, carbohydrate molecules, and so forth. In some cases, the molecule may be a drug candidate. In cases where a molecular system includes more than one molecule, the molecules in the molecular system can form a molecule complex, i.e., an assembly where the molecules are physically bound together, e.g., through interactions such as hydrogen bonds, van der Waals forces, ionic interactions, or hydrophobic effects. In general, a molecular system may have a net charge, e.g., the molecular system may be a cation or an anion, or may have no net charge, i.e., be a neutral molecular system.

[0006] A “conformation” of a molecular system can refer to a specific three-dimensional (3D) spatial arrangement of the atoms within the molecular system, e.g., resulting from rotation around single bonds and other flexible components. A conformation of a molecular system can be defined by data that specifies, for some or all of the atoms in the molecularIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application system, a respective 3D spatial position of the atom in the molecular system. In some instances, conformations of a molecular system can refer to respective 3D spatial arrangements of atoms that can interconvert by rotation of single bonds under biological conditions, e.g., intercellular or intracellular conditions.

[0007] A “molecular dynamics simulation” is a computational technique for modeling the time-dependent behavior of a molecular system. Starting from an initial conformation of the molecular system, the simulation uses a force field to calculate the forces acting on each atom based on their interactions, such as bond stretching, angle bending, and non-bonded interactions. By integrating these forces over time, the simulation predicts the movement of each atom, generating a sequence of conformations that form a trajectory. This trajectory represents the dynamic evolution of the molecular system, capturing how its structure changes over time. Molecular dynamics simulations can use force fields that are based on, e.g., AMBER (Assisted Model Building with Energy Refinement), CHARMM (Chemistry at Harvard Macromolecular Mechanics), or OPLS (Optimized Potentials for Liquid Simulations). In some cases, a molecular dynamics simulation can incorporate quantum mechanical effects in the motion of the atoms, e.g., such as through the use of path integral molecular dynamics (PIMD), ring polymer molecular dynamics (RPMD), or centroid molecular dynamics (CMD).

[0008] A "ligand" can refer to a molecule that specifically binds to a target molecule, such as a receptor or enzyme, e.g., forming a complex that can modulate the function or activity of the target molecule. Ligands can be, e.g., small organic molecules, peptides, proteins, nucleotides, nucleic acids, ions, lipids, carbohydrates, and so forth.

[0009] A “binding pocket” on a target molecule can refer to a specific three-dimensional cavity or crevice within the structure of the target molecule where a ligand can bind to the target molecule. The binding pocket can, in some cases, be understood as a "lock" that fits the shape and chemical properties of ligands that act as "keys" for the lock. In other cases, the ligand may initially not fit perfectly into the binding pocket, e.g., due to structural differences or slight mismatches in shape or chemical groups, but conformational changes during binding can cause the interaction between the ligand and the binding pocket to become more complementary and specific, e.g., as in induced-fit binding. Examples of binding pockets include, e.g., orthosteric binding pockets, allosteric binding pockets, and cryptic binding pockets.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0010] An “embedding” of an entity (e.g., an atom, or a ligand, or a protein) can refer to a representation of the entity as an ordered collection of numerical values, e.g., a vector, matrix, or other tensor of numerical values.

[0011] “Conditioning” a model (e.g., a generative model) or a neural network (e.g., a denoising neural network) or an operation (e.g., a self-attention operation) on conditioning data (e.g., an embedding representing a molecular system) can refer to providing the conditioning data (or a representation, e.g., an embedding, of the conditioning data) as an input (e.g., a side input) to the model, neural network, or operation, such that outputs generated by the model, neural network, or operation are influenced by (depend on) the conditioning data.

[0012] A “binding affinity” of a ligand for a target molecule refers to the strength or degree of attraction between the ligand and the target molecule when they interact to form a complex. Binding affinities can be determined experimentally by, for example, measuring equilibrium constants or free energies for association of the ligand and target molecule.

[0013] A 3D spatial position of an atom can be represented by a set of coordinates in an appropriate coordinate system, e.g., a 3D Cartesian coordinate system or a spherical coordinate system. The coordinate system can be defined with respect to the molecular system (e.g., a body-centered coordinate system) comprising the atom, or with respect to space-fixed coordinate system.

[0014] According to one aspect, there is provided a method performed by one or more computers, the method comprising: obtaining a set of training examples for training a generative machine learning model to perform a structure prediction task, wherein each training example corresponds to a respective molecular system of one or more molecules and comprises: (i) a training input to the generative machine learning model that identifies the one or more molecules included in the molecular system, and (ii) a target output of the generative machine learning model that defines a conformation of the molecular system; augmenting the set of training examples to include a plurality of new training examples, comprising, for each training example in the set of training examples: performing a molecular dynamics simulation of the molecular system, starting from the conformation of the molecular system that is specified by the training example, to generate a trajectory of new conformations adopted by the molecular system over a duration of time; generating one or more new training examples corresponding to the molecular system, wherein each new training example includes a target output that defines a new conformation of the molecular system generated by the molecular dynamics simulation of the molecular system; and adding the one or more new trainingIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application examples to the set of training examples; and training the generative machine learning model to perform the structure prediction task on the augmented set of training examples that includes the new training examples specifying new conformations generated by molecular dynamics simulations.

[0015] In some implementations, for each of one or more training examples in the set of training examples, generating one or more new training examples corresponding to the molecular system comprises: applying a clustering operation to the trajectory of new conformations generated by the molecular dynamics simulation of the molecular system to identify a plurality of clusters of conformations; and selecting one or more new conformations from the trajectory of new conformations, based on the plurality of clusters of conformations, for use in generating new training examples; wherein only a proper subset of new conformations from the trajectory of new conformations are selected for use in generating new training examples.

[0016] In some implementations, selecting one or more new conformations from the trajectory of new conformations, based on the plurality of clusters of conformations, for use in generating new training examples comprises: selecting, for each cluster of conformations, a conformation that defines a medoid of the cluster of conformations for use in generating a new training example.

[0017] In some implementations, the method further comprises, after training the generative machine learning model to perform the structure prediction task on the augmented set of training examples: receiving data identifying an input molecular system that comprises a ligand and a target molecule; generating, using the generative machine learning model, a plurality of conformations of molecular system that includes the ligand and the target molecule; and determining a binding affinity of the ligand for the target molecule using the plurality of conformations of the molecular system that includes the ligand and the target molecule.

[0018] In some implementations, determining the binding affinity of the ligand for the target molecule using the plurality of conformations of the molecular system that includes the ligand and the target molecule comprises: performing, for each of the plurality of conformations of the molecular system, a molecular dynamics simulation starting from the conformation to generate a trajectory of conformations; determining, for each conformation in each trajectory of conformations, whether the ligand is bound to the target molecule in the conformation; and determining the binding affinity based on a number of conformations from the trajectories of conformations where the ligand is bound to the target molecule.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0019] In some implementations, determining, for each conformation in each trajectory of conformations, whether the ligand is bound to the target molecule in the conformation comprises, for each conformation: determining a distance between the ligand and a binding pocket of the target molecule in the conformation; and determining whether the ligand is bound to the target molecule in the conformation based on the whether the distance between the ligand and the binding pocket satisfies a threshold.

[0020] In some implementations, the generative machine learning model comprises an embedding neural network and a generative neural network; wherein the embedding neural network is configured to process data identifying one or more molecules included in a molecular system to generate an embedding of the molecular system; and wherein the generative neural network is configured to generate one or more conformations of the molecular system when conditioned on the embedding of the molecular system.

[0021] In some implementations, training the generative machine learning model comprises jointly training the embedding neural network and the generative neural network by backpropagating gradients through the generative neural network and into the embedding neural network.

[0022] In some implementations, the method further comprises, after training the generative machine learning model to perform the structure prediction task: receiving data identifying an input molecular system that comprises a ligand and a target molecule; generating, using the embedding neural network, an embedding of the input molecular system that comprises the ligand and the target molecule; and processing the embedding of the input molecular system using a binding affinity machine learning model to predict a binding affinity of the ligand for the target molecule.

[0023] In some implementations, training the generative machine learning model to perform the structure prediction task comprises, for each training example in the augmented set of training examples: training the generative machine learning model, by a machine learning training technique, to reduce a discrepancy between: (i) the conformation specified by the target output of the training example, and (ii) a conformation generated by the generative machine learning model by processing the training input of the training example.

[0024] In some implementations, the generative machine learning model comprises a generative diffusion model that is configured to perform operations comprising: receive positional data defining a respective initial position of each atom in a molecular system; and denoise the positional data over a sequence of time steps using a denoising neural networkIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application over a sequence of denoising iterations; wherein after a final time step in the sequence of time steps, the positional data defines a predicted conformation of the molecular system.

[0025] In some implementations, training the generative machine learning model to perform the structure prediction task comprises, for each training example: determining a force field that defines a model of an energy of the molecular system corresponding to the training example; generating energy gradient data that defines, for each of a plurality of atoms in the molecular system, a gradient of the force field with respect to a position of the atom in the conformation of the molecular system that is specified by the target output of the training example; training the denoising neural network to optimize an objective function that depends at least in part on the energy gradient data.

[0026] In some implementations, training the denoising neural network to optimize an objective function that depends at least in part on the energy gradient data comprises: generating a noisy conformation of the molecular system by combining noise with the conformation of the molecular system that is specified by the target output of the training example; processing a network input based on the noisy conformation of the molecular system using the denoising neural network to generate a denoising output; wherein the objective function measures an error between the denoising output and an overall denoising target; wherein the overall denoising target is a combination of: (i) a first denoising target that is based on the conformation of the molecular system that is specified by the target output of the training example, or the noise combined with the conformation of the molecular system that is specified by the target output of the training example, or both; and (ii) a second denoising target that is based on the energy gradient data.

[0027] In some implementations, the overall denoising target is a linear combination of the first denoising target and the second denoising target.

[0028] In some implementations, generating the noisy conformation of the molecular system by combining noise with the conformation of the molecular system that is specified by the target output of the training example comprises: sampling a denoising iteration; and determining a noise scaling coefficient as a monotonically decreasing function of denoising iterations; wherein the linear combination of the first denoising target and the second denoising target comprises an energy scaling coefficient that scales the second denoising target that is based on the energy gradient data; and wherein the energy scaling coefficient is defined as a monotonically increasing function of denoising iterations.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0029] According to another aspect, there is provided a system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the methods described herein.

[0030] According to another aspect, there are provided one or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the methods described herein.

[0031] According to another aspect, there is provided a method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands: (a) using a generative machine learning model trained using the methods described herein to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and (b) evaluating an interaction of the candidate ligand with the target protein molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating.

[0032] In some implementations, the target protein molecule comprises a receptor or enzyme, and wherein the ligand is an agonist or antagonist of the receptor or enzyme.

[0033] In some implementations, the ligand is a drug, and the method comprises: performing steps (a) and (b) for each of a plurality of target protein molecules; and selecting one or more of the candidate ligands as the ligand to either i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

[0034] In some implementations, the ligand comprises an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, in particular a virus or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect.

[0035] In some implementations, the ligand is a polypeptide ligand, a polynucleoside ligand, or a polynucleotide ligand.

[0036] According to another aspect, there is provided a method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: using a generative machine learning model trained using the method of any one of claims 1-15 to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the targetIsomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application protein molecule; and evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating.

[0037] In some implementations, evaluating the interaction of one of the candidate ligands comprises determining an interaction score for the candidate ligand, wherein the interaction score comprises a measure of an interaction between the candidate ligand and the target molecule.

[0038] In some implementations, the method further comprises synthesizing the ligand or diagnostic antibody or aptamer marker.

[0039] In some implementations, the method further comprises testing biological activity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

[0040] According to another aspect, there is provided a method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; performing the methods described herein to determine a predicted structure of the molecule complex; using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

[0041] In some implementations, the experimental technique comprises one or more of: x-ray crystallography, nuclear magnetic resonance, and electron microscopy.

[0042] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0043] A generative machine learning model can be trained to generate data defining 3D structures of molecular systems. The training data for training a generative machine learning model may include training examples that each correspond to a unique molecular system and define a single, static structure of that molecular system that has been determined through experimental techniques such as, e.g., x-ray crystallography. Training the generative machine learning model on the training data thus encourages the generative machine learning model to reproduce the single, static 3D structure of each molecular system that is included in the training data.

[0044] However, real world molecular systems typically do not assume a static 3D structure. Rather, they transition through large numbers of conformations due to thermal motion and intrinsic flexibility. A generative machine learning model that is trained on training data thatIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application associates only a single static 3D structure (or only a small number of static 3D structures) with molecular systems may have the capacity to generate conformations from a single conformational mode of a molecular system, e.g., conformations that resemble the conformation of the molecular system when crystallized, but may fail to generate conformations that span a wide range of possible conformations that can be adopted by the molecular system, e.g., under biological conditions.

[0045] One approach to determining new conformations of a molecular system is to run molecular dynamics simulations starting from known (e.g., experimentally determined) conformations of the molecular system. However, molecular dynamics simulations are highly computationally intensive, as they require the integration of differential equations of motion across a large number of small time steps. Additionally, these simulations can become less accurate over longer time scales, as small numerical errors may accumulate and propagate throughout the simulation, potentially leading to deviations from the true behavior of the molecular system. Therefore, both for computational efficiency and to maintain accuracy, molecular dynamics simulations are usually run over short time scales, e.g., nanoseconds to microseconds. However, for certain molecular systems, conformational changes can occur over a much longer time scale, e.g., milliseconds or seconds, and molecular dynamics simulations are thus unable to capture these conformational changes.

[0046] The system described in this specification addresses these issues with generative machine learning models and molecular dynamics simulations. Given an initial set of training examples that each define a conformation of a corresponding molecular system, the system performs molecular dynamics simulations, starting from the initial conformations defined by the training examples, to generate trajectories of new conformations. The system can then generate new training examples based on the conformations generated by the molecular dynamics simulations, and use these new training examples to augment the set of training data used for training the generative machine learning model. After being augmented with the new training examples, the training data characterizes large numbers of possible conformations of molecular systems. Training the generative machine learning model on the augmented set of training examples can allow the generative machine learning model to learn to generate conformations that span a wide range of possible conformations that can be adopted by molecular systems.

[0047] Several issues can arise when generating new training examples using conformations that are generated by molecular dynamics simulations. For instance, a molecular dynamics simulation can generate a very large number of new conformations, e.g., depending on theIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application length of the timescale of the simulation and the incremental time step size used during the simulation, the number of new conformations can range from thousands to millions. Further, some of these conformations may have high levels of similarity to one another, e.g., because the molecular dynamics simulation uses small time steps, or because the simulation oscillates between a few stable conformational states. Generating a new training example for every conformation generated by a molecular dynamics simulation may therefore be computationally infeasible, and further, may generate highly duplicative training examples.

[0048] The system described in this specification can address these issues by applying a clustering operation to trajectories of conformations generated by molecular dynamics simulations. The clustering operation can identify clusters of conformations, where conformations in the same cluster tend to be more similar than conformations in different clusters. The system can then leverage the conformation clusters to select only a proper subset (i.e., fewer than all) of the new conformations generated by molecular dynamics simulations for use in generating new training examples. The system can thus reduce consumption of computational resources (e.g., memory and computing power) during training of the generative machine learning model by limiting the number of new training examples that are generated and used for training the generative machine learning model.

[0049] In some implementations, the generative machine learning model can be implemented as a generative diffusion model that generates a conformation of a molecular system by iteratively denoising positional data that defines a respective position of each atom in the molecular system using a denoising neural network. As part of training the generative diffusion model on a training example that defines a conformation of a molecular system, the system can combine noise with the conformation (i.e., add respective noise values to the coordinates of one or more or each of the atoms in the molecular system in the conformation) to generate a noisy conformation, and then train the generative diffusion model to optimize an objective function that encourages (e.g., induces or biases) the generative diffusion model to denoise the noisy conformation.

[0050] The system can enhance the training of the generative diffusion model by leveraging an additional source of information during training, in particular, energy gradient data that defines forces acting on atoms in the un-noised conformation of the molecular system (i.e., energy gradient data that defines forces acting on atoms in the conformation of the molecular system without noise having been added to the positional data of the atoms). The system can generate the energy gradient data by determining gradients of a force field that defines a model for the energy of the conformation with respect to the positions of the atoms in theIsomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application conformation. The system can incorporate the energy gradient data into the objective function used for training the generative diffusion model and thus encourage the generative diffusion model to denoise atom positions in a manner that is consistent with the energy gradient data, which can improve the accuracy of conformations generated by the generative diffusion model during inference. Further, incorporating the energy gradient data into the training of the generative diffusion model can provide a richer training signal and reduce the amount of training data required in order to train the generative diffusion model to achieve an acceptable accuracy, thus reducing consumption of computational resources (e.g., memory and computing power) during training.

[0051] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0052] FIG. 1 shows an example training system.

[0053] FIG. 2 shows an example prediction system.

[0054] FIG. 3 is a flow diagram of an example process for training a generative machine learning model to perform a structure prediction task using training examples generated by molecular dynamics simulations.

[0055] FIG. 4 is a flow diagram of an example process for determining a predicted binding affinity of a ligand for a target molecule using conformations generated by the generative machine learning model.

[0056] FIG. 5 is a flow diagram of an example process for determining a predicted binding affinity of a ligand for a target molecule using an embedding of the ligand - target molecule complex that is generated by a generative machine learning model.

[0057] FIG. 6 is a flow diagram of an example process for generating a predicted conformation of a molecular system using a generative machine learning model that is implemented as a generative diffusion model.

[0058] FIG. 7 is a flow diagram of an example process for training a generative diffusion model using an objective function that depends on energy gradient data.

[0059] Like reference numbers and designations in the various drawings indicate like elements.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT ApplicationDETAILED DESCRIPTION

[0060] FIG. 1 shows an example training system 100. The training system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

[0061] The training system 100 trains a generative machine learning model 114 to perform a structure prediction task. More specifically, the training system 100 trains the generative machine learning model 114 to process data identifying one or more molecules included in a molecular system 112 to generate data defining a predicted conformation 116 of the molecular system.

[0062] The generative machine learning model 114 can be configured to process any appropriate data identifying one or more molecules included in a molecular system. For instance, if the molecular system includes a protein molecule, then the generative machine learning model 114 can process data identifying an amino acid sequence of the protein. As another example, if the molecular system includes a nucleic acid molecule (e.g., a DNA or RNA molecule), then the generative machine learning model 114 can process data identifying a nucleic acid sequence of the nucleic acid molecule. As another example, if the molecular system includes a small molecule, then the generative machine learning model 114 can process data identifying a chemical structure of the small molecule, e.g., in the form of a SMILES (Simplified Molecular Input Line Entry System) string.

[0063] The generative machine learning model 114 can be any appropriate conditional generative model. More specifically, the generative machine learning model 114 can be any appropriate model that, by processing data identifying one or more molecules included in a molecular system, can generate samples from a distribution over a space of possible conformations of the molecular system. For instance, the generative machine learning model 114 can be implemented as a generative diffusion model, or a generative adversarial neural network (GAN) model, or a flow-based neural network model (normalizing flow model), and so forth.

[0064] Particular examples of generative machine learning models that can perform structure prediction tasks are described with reference to: Abramson, Josh, et al. "Accurate structure prediction of biomolecular interactions with AlphaFold 3." Nature (2024): 1-3; Watson, Joseph L., et al. "De novo design of protein structure and function with RF diffusion." Nature 620.7976 (2023): 1089-1100; Watson, Joseph L., et al. "Broadly applicable and accurateIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application protein design by integrating structure prediction networks and diffusion generative models." BioRxiv (2022): 2022-12.

[0065] In some implementations, the generative machine learning model 114 can be implemented as a generative diffusion model that generates a predicted conformation of a molecular system by iteratively denoising positional data that defines a respective position of each atom in the molecular system using a denoising neural network. An example process for generating a predicted conformation of a molecular system using a generative diffusion model is described with reference to FIG. 6. For example, the generative diffusion model may perform operations including: receiving positional data defining a respective initial position of each atom in a molecular system; and denoising the positional data over a sequence of time steps using a denoising neural network over a sequence of denoising iterations. After a final time step in the sequence of time steps, the positional data can define a predicted conformation of the molecular system.

[0066] For a given molecular system, the generative machine learning model 114 can be prompted an arbitrary number of times to generate a predicted conformation of the molecular system. Due to stochasticity in certain operations performed by the generative machine learning model 114, at least some (and possibly all) of the predicted conformations of a molecular system that are generated by the generative machine learning model 114 may be distinct from one another.

[0067] The training system 100 trains the generative machine learning model in a manner that encourages the generative machine learning model, when repeatedly prompted to generate predicted conformations of a molecular system, to generate conformations that span a wide range of possible conformations that can be adopted by the molecular system. Once trained in this manner, the generative machine learning model can be used, e.g., as part of a process for predicting the binding affinity of a ligand for a target molecule in a manner that accounts for a range of possible conformations of the ligand and the target molecule when bound in a complex, as will be described in more detail below.

[0068] The training system 100 receives a set of training examples 102. Each training example corresponds to a respective molecular system and includes: (i) a training input to the generative machine learning model, and (ii) a target output of the generative machine learning model. The training input identifies one or more molecules included in the molecular system. The target output defines a possible conformation of the molecular system.

[0069] For each training example, the system 100 uses a molecular dynamics simulation engine 104 to perform a molecular dynamics simulation of the corresponding molecularIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application system, starting from the conformation specified by the training example, to generate a trajectory of conformations 106 that are adopted by the molecular system over a duration of time.

[0070] The system 100 processes the trajectories of possible conformations 106 generated by the simulation engine 104 to generate new training examples 108. Each new training example 108 corresponds to a respective molecular system and includes a training input and a target output (as described above), where the target output is a conformation of the molecular system that is drawn from a trajectory of conformations generated by the simulation engine 104 by performing a molecular dynamics simulation.

[0071] The system augments the original set of training examples 102 with the new training examples 108 generated using molecular dynamics simulations to generate an augmented set of training examples. Thus, for each of multiple molecular systems, the augmented set of training examples includes training examples that characterize a range of possible conformations of the molecular system.

[0072] The system 100 uses a training engine 110 to train the generative machine learning model 114 to perform the structure prediction task on the augmented set of training examples. More specifically, for each training example, the training engine 110 trains the generative machine learning model 114, by a machine learning training technique, to reduce a discrepancy between: (i) the conformation specified by the target output of the training example, (ii) a conformation generated by the generative machine learning model 114 by processing the training input of the training example. Training the generative machine learning model 114 on the augmented set of training examples encodes information in the set of generative machine learning model parameters that enables the generative machine learning model 114 to generate conformations that span a wide range of possible conformations that can be adopted by a molecular system.

[0073] The specific operations performed by the training engine 110 to train the generative machine learning model 114 depend on the type and architecture of the generative machine learning model 114. For instance, for a generative machine learning model 114 implemented by one or more neural networks, the training engine 110 can train the generative machine learning model by iteratively adjusting the neural network parameters of the one or more neural networks, e.g., using stochastic gradient descent, to optimize an objective function.

[0074] In implementations where the generative machine learning model 114 is implemented as a generative diffusion model, the training engine 110 can train the generative machine learning model 114 in a manner that leverages additional information about a molecularIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application system that is provided by energy gradient data for the molecular system. More specifically, as part of training the generative machine learning model on a training example corresponding to a molecular system, the training engine 110 can determine an energy of the molecular system, e.g., using a force field of the molecular system. The system 100 can determine gradients of the energy of the molecular system with respect to positions of the atoms in the molecular system, i.e., when the atoms are in the conformation specified by the training example. The system 100 can then train the generative machine learning model on the training example using an objective function that depends (in part) on the energy gradient data for the conformation of the molecular system.

[0075] Conditioning the objective function to depend on the energy gradient data for the conformation can provide a richer training signal for adjusting the values of the set of generative machine learning model parameters and can improve the accuracy of predicted conformations generated by the generative machine learning model. An example process for training a generative diffusion model using an objective function that depends on energy gradient data is described in more detail with reference to FIG. 7.

[0076] Once trained, the generative machine learning model 114 can be used, e.g., to predict the binding affinity of a ligand for a target molecule in a manner that accounts for a range of possible conformations of the ligand and the target molecule when bound in a complex. An example of a prediction system that can use the generative machine learning model 114 (e.g., as trained by the training system 100) as part of a process for predicting the binding affinity of a ligand for a target molecule is described with reference to FIG. 2.

[0077] In addition or as an alternative to generating new conformations of molecular systems using molecular dynamics simulations, the system can also generate new conformations of molecular systems using a Monte Carlo simulation process. In this approach, an initial conformation of the molecular system is selected, and successive conformational modifications are proposed by randomly perturbing one or more structural degrees of freedom, such as atomic positions, bond angles, or dihedral angles. For each proposed conformation, an energy value is determined using a molecular mechanics or quantum mechanical potential energy function. The proposed conformation is then probabilistically accepted or rejected based on a comparison of its energy to that of the current conformation, for example according to a Metropolis acceptance criterion in which lower-energy conformations are preferentially accepted while higher-energy conformations are accepted with a probability that decreases exponentially with energy difference. By iteratively repeating this process, the molecular system samples a series of conformations thatIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application collectively represent the equilibrium distribution of conformational states at a specified temperature. The system can use new molecular system conformations generated by Monte Carlo simulation in a similar or identical manner to how this specification describes the system as using new molecular system conformations generated by molecular dynamics simulations.

[0078] An example technique for generating new conformations of a molecular system by Monte Carlo simulation is described in: “Sampling of the conformational landscape of small proteins with Monte Carlo methods”, Scientific Reports volume 10, Article number: 18211 (2020).

[0079] A variety of possible downstream uses of the generative machine learning model 114 (e.g., as trained by the training system 100) are described next. Various use cases of the generative machine learning model that are described next involve determining a binding affinity of a ligand for a target molecule. In any of these use cases, the prediction system described with reference to FIG. 2 can be used to determine the binding affinity of the ligand for the target molecule.

[0080] The generative machine learning model and methods described herein can be used to obtain a ligand (i.e. a ligand molecule or ligand molecule complex) such as a drug or a ligand of an industrial enzyme. In general, the drug or industrial enzyme may be a molecule that inhibits or catalyzes a chemical or biochemical process. A molecule complex may include, for example, a protein, a ribozyme (ribonucleic acid enzyme), or a deoxyribozyme (deoxyribonucleic acid enzyme). For example, a method of obtaining a ligand may include obtaining a target amino acid sequence, in particular the amino acid sequence of a target protein molecule (or target protein molecule complex), e.g. a drug target, and processing an input based on the target amino acid sequence using the generative machine learning model to determine a (tertiary) structure of the target protein molecule, e.g., a predicted structure of a complex comprising the target protein molecule and the candidate ligand. The method may then include evaluating an interaction of one or more candidate ligands with the target protein molecule. The method may further include selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating of the interaction. Predicting the structure of a complex comprising the target protein molecule and a candidate ligand may preferably account for changes in the structure of the molecule caused by binding of the candidate ligand and / or changes in the structure of the candidate ligand. Evaluating the interaction of the one or more candidate ligands with the target protein molecule may, for example,Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application comprise determining a binding energy or an equilibrium constant for the formation of the complex.

[0081] In some implementations, evaluating the interaction may include evaluating binding of the candidate ligand with the structure of the target protein molecule. For example, evaluating the interaction may include identifying a ligand that binds with sufficient affinity for a biological effect. In some other implementations, evaluating the interaction may include evaluating an association of the candidate ligand with the target protein molecule which has an effect on a function of the target protein molecule, e.g., an enzyme. The evaluating may include evaluating an affinity between the candidate ligand and the target protein molecule or complex, or evaluating a selectivity of the interaction. The candidate ligand(s) may be selected according to which have the highest affinity. Evaluating the interaction may additionally comprise simulating a dynamical behavior of the ligand and target protein molecule, such as through molecular dynamics simulations, which may allow kinetic aspects of the interaction to be taken into account.

[0082] The candidate ligand(s) may be derived from a database of candidate ligands, and / or may be derived by modifying ligands in a database of candidate ligands, e.g., by modifying a structure or amino acid sequence of a candidate ligand, and / or may be derived by stepwise or iterative assembly / optimization of a candidate ligand. For example, the collection of candidate ligands can be obtained from an existing library of ligands, such as so-called compound libraries (e.g. available commercially), libraries generated by combinatorial techniques, and other sources (e.g. databases of complexes, such as the protein data bank). The candidate ligand(s) may alternately or additionally include one or more candidate ligands generated using a generative model conditioned on (the structure of) the target protein molecule or part of the target protein molecule, e.g. a structure of a binding site or other part of the target protein molecule.

[0083] The evaluation of the interaction of a candidate ligand with the target protein molecule may be performed using a computer-aided approach in which graphical models of the candidate ligand and target protein molecule structure are displayed for usermanipulation, and / or the evaluation may be performed partially or completely automatically, for example using standard molecular (e.g. protein-ligand) docking software. In some implementations the evaluation may include determining an interaction score for the candidate ligand, where the interaction score includes a measure of an interaction between the candidate ligand and the target protein molecule. The interaction score may be dependentIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application upon a strength and / or specificity of the interaction, e.g., a score dependent on binding free energy. A candidate ligand may be selected dependent upon its score.

[0084] In some implementations the target protein molecule includes a receptor or enzyme and the ligand is an agonist or antagonist of the receptor or enzyme. In some implementations the method may be used to identify the structure of a cell surface marker. This may then be used to identify a ligand, e.g., an antibody or aptamer or a label such as a fluorescent label, which binds to the cell surface marker. This may be used to identify and / or treat cancerous cells.

[0085] In some implementations the ligand is a drug and the interaction of each of a plurality of target protein molecules with each of the candidate ligands is evaluated. Then one or more of the candidate ligands may be selected either to obtain a ligand that (functionally) interacts with each of the target protein molecules, or to obtain a ligand that (functionally) interacts with only one of the target protein molecules. For example in some implementations it may be desirable to obtain a drug that is effective against multiple drug targets. Also or instead it may be desirable to screen a drug for off-target effects. For example in agriculture it can be useful to determine that a drug designed for use with one plant species does not interact with another, different plant species and / or an animal species.

[0086] In some implementations the ligand is a drug and the predicted structure of a target protein that is a protein complex, e.g. a dimer or multimer, is determined. Evaluating the interaction of the one or more candidate ligands with the target protein may then comprise identifying a candidate ligand that interacts with the protein complex, and that might therefore be expected to affect the formation or stability of the complex. This could afterwards be confirmed by experimental screening. Thus such a process may be used to identify a drug which is able to disrupt a protein complex or inhibit formation of the complex. Some diseases, e.g. neuro degenerative diseases such as dementia, are caused by protein aggregation. The method may thus be used to identify a ligand that is a drug to treat such a disease.

[0087] In some implementations the candidate ligand(s) may include small molecule complex ligands, e.g., organic compounds with a molecular weight of <900 daltons. In some other implementations the candidate ligand(s) may include polypeptide ligands, i.e., defined by an amino acid sequence.

[0088] In another aspect there is provided a method of using the generative machine learning model to obtain a ligand, which may be a biological molecule, such as a polypeptide, polynucleotide, or polynucleotide ligand (e.g., the molecule or its amino acid or nucleotideIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application sequence). For example, the method may include obtaining data defining one or more candidate ligands, e.g. an amino acid sequence of one or more candidate polypeptide or polynucleotide ligands. The method may include selecting a target molecule to which the ligand is to bind. The method may further include, for each of the candidate ligands, using the generative machine learning model to determine (tertiary) structure of a complex comprising the candidate ligand and the target protein molecule. The method may further include obtaining a target protein structure of a target molecule, in silico and / or by physical investigation. The method may comprise evaluating an interaction between each of the one or more candidate ligands and the target protein molecule, e.g. by evaluating an interaction between the predicted structure of the candidate ligand and the structure of the target molecule, or using the predicted structure of the complex comprising the candidate ligand and the target protein molecule. The method may further include selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluation.

[0089] As before, evaluating the interaction may include evaluating binding of the candidate ligand with the structure of the target protein molecule, e.g., identifying a ligand that binds with sufficient affinity for a biological effect, and / or evaluating an association of the candidate ligand with the structure of the target protein molecule which has an effect on a function of the target protein molecule, e.g., an enzyme, and / or evaluating an affinity between the candidate ligand and the structure of the target protein molecule, or evaluating a selectivity of the interaction. In some implementations the ligand may be an aptamer. Again, the candidate ligand(s) may be selected according to which have the highest affinity.

[0090] As before, the selected ligand (e.g. selected polypeptide or polynucleotide ligand) may comprise a receptor or enzyme and the ligand may be an agonist or antagonist of the receptor or enzyme. In some implementations the ligand may comprise an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, for example a virus, in particular a virus coat protein, or a protein expressed on a cancer cell. In these implementations the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect. For example, the antibody or aptamer may bind to the target and act as an agonist for a particular receptor; alternatively, the antibody or aptamer may prevent binding of another ligand to the target, and hence prevent activation of a relevant biological pathway.

[0091] Implementations of the method may further include synthesizing the ligand, i.e., making the small molecule, polynucleotide or polypeptide ligand. The ligand may be synthesized by any conventional chemical techniques and / or may already be available, e.g.,Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application may be from a compound library or may have been synthesized using combinatorial chemistry.

[0092] The method may further include testing the ligand for biological activity in vitro and / or in vivo. For example the ligand may be tested for ADME (absorption, distribution, metabolism, excretion, e.g., by a living organism or cell culture or tissue model) and / or toxicological properties, to screen out unsuitable ligands. The testing may include, e.g., bringing the candidate small molecule, polypeptide or polynucleotide ligand into contact with the target protein molecule and measuring a change in expression or activity of the target molecule. For example, the selected ligand(s) can be screened according to a degree to which binding is accompanied by a biological (therapeutic) effect such as facilitating a biological mechanism or directly or indirectly inhibiting a biological disease mechanism (e.g. inhibiting a bacteria or virus from entering a cell), toxicity, clearance time, and so forth.

[0093] In some implementations a candidate (e.g. polypeptide or polynucleotide) ligand may include: an isolated antibody or aptamer, a fragment of an isolated antibody or aptamer, a single variable domain antibody, a bi- or multi-specific antibody, a multivalent antibody, a dual variable domain antibody, an immuno-conjugate, a fibronectin molecule, an adnectin, an DARPin, an avimer, an affibody, an anticalin, an affilin, a protein epitope mimetic or combinations thereof. A candidate (polypeptide) ligand may include an antibody with a mutated or chemically modified amino acid Fc region, e.g., which prevents or decreases ADCC (antibody-dependent cellular cytotoxicity) activity and / or increases half-life when compared with a wild type Fc region. Candidate (polypeptide or polynucleotide) ligands may include antibodies with different CDRs (Complementarity -Determining Regions).

[0094] As another example, the target protein molecule may be an enzyme comprising a CRISPR associated protein and the ligand may comprise a guide RNA molecule. The method may be performed to identify a combination of guide RNA molecule and CRISPR associated protein, in particular one that operates efficiently to edit genes. Such a method can involve determining a predicted structure of the enzyme, e.g. as described above, in particular to check that the enzyme shape and the guide RNA shape fit and work together effectively. The guide RNA may have a part with a defined 3D structure, e.g. it may be a single guide RNA (sgRNA), incorporating a guide sequence and a tracrRNA sequence.

[0095] The generative machine learning model described herein can also be used to obtain a diagnostic antibody or aptamer marker of a disease. There is also provided a method that comprises selecting a target protein molecule that is to be recognized by the antibody or aptamer marker, and for each of one or more candidate antibodies or aptamers e.g. asIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application described above, uses the generative machine learning model to determine a predicted structure of a complex comprising the target protein molecule and the candidate antibody or aptamer. The method may also involve, evaluating an interaction between each of the one or more candidate antibodies or aptamers and the target protein molecule, and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating, e.g. selecting one or more candidate antibodies or aptamers that have the highest affinity to the target protein. The method may include making the diagnostic antibody or aptamer marker. The diagnostic antibody or aptamer marker may be used to diagnose a disease by detecting whether it binds to the target protein molecule in a sample obtained from a patient, e.g. a sample of bodily fluid. As described above, a corresponding technique can be used to obtain a therapeutic antibody or aptamer (e.g. polypeptide or polynucleotide ligand).

[0096] In some other aspects a computer-implemented method as described above or herein may be used to identify active / binding / blocking sites on a target protein from its amino acid sequence.

[0097] The generative machine learning model and methods described herein can also be used to determine the structure of a molecule or molecule complex. For example, an experimental technique may be applied to a physical sample comprising a molecule complex (i.e. the physical sample of the molecule complex is “interrogated” using the experimental technique) to measure experiment signals dependent on a structure of the molecule complex. In some implementations, the experimental technique may be a scattering technique or a spectroscopic technique. The experimental technique may, for example, comprise one or more of: x-ray crystallography, nuclear magnetic resonance (NMR), and electron microscopy (e.g. cryogenic electron microscopy, cryo-EM). The generative machine learning model may be used to determine a predicted structure of the molecule complex. The experiment signals may then be compared with corresponding simulated signals generated using the predicted structure of the molecule complex. For example, the predicted structure of the molecule complex can be used to generate predicted x-ray diffraction patterns (e.g. from an electron density distribution determined using the predicted structure) that can be compared with experimentally measured x-ray diffraction patterns. For example, the experiment signals may comprise NMR signals, electron microscope images, or x-ray diffraction patterns; or signals derived therefrom.

[0098] The predicted structure (conformation) of the molecule complex may be determined by adjusting the predicted structure of the molecule complex dependent upon a result of theIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application comparison. The method may be performed iteratively, wherein for each of one or more iterations, after the predicted structure has been adjusted, the predicted signals may be generated for the adjusted structure, and the comparing and adjusting performed again to refine the predicted structure. Alternatively (or additionally), a plurality of different possible structures of the molecule complex can be predicted and the expected experiment signals from each compared with the actual experiment signals to determine a match, e.g. a best or most likely match, that can be taken as the determined structure of the molecule complex.

[0099] In general an aptamer as described above may comprise DNA or RNA. An enzyme as described above may comprise a protein or a DNA enzyme, e.g. a deoxyribozyme or “DNAzyme”, or an RNA enzyme, e.g. a ribozyme. As well as the applications described above such enzymes can also be used for biosensors of many types, e.g. DNAzymes and aptamers can be useful for detecting metal ions, and in general aptamer targets can include small molecule complexes, proteins, and cells. Aptamers have many uses including, e.g. as probes in assays, as biosensors (e.g. they can detectably change shape when binding to a target), to modulate the activity of biomolecule complexes, and to provide a controlled release mechanism. The techniques described herein can be used to design, and then make, such aptamers, sometimes referred to as chemical antibodies.

[0100] FIG. 2 shows an example prediction system 200. The prediction system 200 is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

[0101] The prediction system 200 is configured to process data identifying a ligand 202 and a target molecule 204 to generate an output that defines a predicted binding affinity 206 of the ligand 202 for the target molecule. The prediction system 200 uses the generative machine learning model 114, e.g., that has been trained by the training system 100 described with reference to FIG. 1, to predict the binding affinity of the ligand 202 for the target molecule 204 in a manner that accounts for a range of possible conformations of the ligand and the target molecule when bound in a complex.

[0102] In some implementations, to determine the predicted binding affinity 206, the prediction system 200 uses the generative machine learning model 114 to generate a collection of conformations of a complex that includes the ligand 202 and the target molecule 204. The prediction system 200 augments the collection of conformations to include additional conformations generated by performing molecular dynamics simulations starting from the conformations generated by the generative machine learning model 114. TheIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application prediction system can then determine the predicted binding affinity 206, e.g., by determining a number of generated conformations of the complex where the ligand is positioned within a binding pocket of the target molecule. An example process for determining a predicted binding affinity of a ligand for a target molecule using conformations generated by the generative machine learning model of a complex that includes the ligand and the target molecule is described in more detail with reference to FIG. 4.

[0103] In some implementations, to determine the predicted binding affinity 206, the prediction system 200 generates an embedding of a complex that includes the ligand 202 and the target molecule 204 using the generative machine learning model 114. In more detail, the generative machine learning model 114 can include an: (i) an embedding neural network, and (ii) a generative neural network. The embedding neural network can be configured to process data identifying one or more molecules included in a molecular system to generate an embedding of the molecular system. The generative neural network can be configured to generate one or more conformations of the molecular system when conditioned on the embedding of the molecular system, i.e., as generated by the embedding neural network. For instance, in an implementation where the generative machine learning model is a generative diffusion model, the generative neural network can be a denoising neural network, e.g., as described with reference to FIG. 6. As another example, in an implementation where the generative machine learning model is a generative adversarial neural network (GAN), the generative neural network can be the generator of the GAN.

[0104] The prediction system 200 can generate an embedding of the complex that include the ligand 202 and the target molecule 204 by processing data identifying the ligand 202 and the target molecule 204 using the embedding neural network of the generative machine learning model 114. The prediction system 200 can then process the embedding of the ligand - target molecule complex using a binding affinity machine learning model to generate the predicted binding affinity 206 of the ligand 202 for the target molecule 204.

[0105] As part of training the generative machine learning model 114, e.g., as described with reference to FIG. 1, the training system 100 jointly trains the embedding neural network and the generative neural network of the generative machine learning model 114. Training the generative machine learning model 114 to generate wide ranges of possible conformations of molecular systems causes the embedding neural network to generate embeddings that encode information characterizing many possible conformations of molecular systems. A binding affinity machine learning model can leverage the information content encoded in embeddings generated by the embedding neural network to more accurately predict binding affinity.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0106] An example process for determining a predicted binding affinity of a ligand for a target molecule using an embedding of the ligand - target molecule complex that is generated by the generative machine learning model is described in more detail with reference to FIG. 5.

[0107] The prediction system 200 can perform a variety of downstream tasks based on the binding affinity predictions 206 generated using the generative machine learning model 114. A few examples of downstream tasks that use the predicted binding affinities 206 are described next.

[0108] In one example, the prediction system 200 can use binding affinity predictions as part of performing drug discovery. Drug discovery can involve identifying specific molecules within the body that are involved in a disease process. These molecules are often proteins, such as enzymes, receptors, or signaling proteins, that play a key role in the disease's development or progression. A ligand, often a small molecule, peptide, or antibody, can be selected to bind specifically to an identified target protein. When a drug that includes the ligand is administered to a patient, the ligand can bind to the target protein with high affinity and in doing so contribute to achieving a therapeutic effect in the patient. For instance, if the target protein is an enzyme involved in a disease process, the ligand can inhibit its activity, thus disrupting the disease pathway. More generally, the interaction between the ligand and the target protein can activate, inhibit, or alter the function of the target protein to achieve a therapeutic effect.

[0109] Therefore, identifying ligands with high (or low) binding affinity for a protein can be a crucial step in the process of drug discovery. (Identifying ligands with low binding affinities for a protein can be desirable, e.g., when the protein is an off-target protein and the binding of the ligand to the protein may cause undesirable side effects).

[0110] Binding affinity predictions generated by the prediction system 200 can be used to determine a ranking of candidate ligands in a collection of candidate ligands based on their respective predicted binding affinities for a protein. The ranking of the candidate ligands in the collection of candidate ligands based on their respective predicted binding affinities for the protein can be used, e.g., to select a proper subset of the collection of candidate ligands for experimental validation and testing. For instance, one or more candidate ligands having the highest or lowest predicted binding affinities for the protein (i.e., according to the ranking) can be selected for experimental validation and testing, e.g., for use in a drug that achieves a therapeutic effect in patients. In particular, each selected candidate ligand can be physically synthesized and then tested for a variety of properties, e.g., absorption,Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application distribution, metabolism, and excretion. One or more of the candidate ligands from the collection of ligands can be selected for inclusion in a drug, e.g., based at least in part on results of the testing. A drug that includes one or more of the candidate ligands can be synthesized using any appropriate drug synthesis technique.

[0111] The collection of candidate ligands can include any appropriate number of ligands, e.g., 10 ligands, or 1000 ligands, or 100,000 ligands. In some cases, only a fraction of the candidate ligands in the set of candidate ligands are selected for physical synthesis, e.g., based on the ranking of the candidate ligands by their predicted binding affinities for the protein. For instance, less than 50%, or less than 10%, or less than 1%, or less than 0.1% of the candidate ligands in the collection of candidate ligands may be selected for physical synthesis.

[0112] In another example, the prediction system 200 can use binding affinity predictions for drug repurposing. In more detail, an existing drug may include a particular ligand, e.g., that is known to achieve a therapeutic effect in patients by binding to a target protein involved with a particular disease process. Drug repurposing can involve identifying new protein binding targets for the ligand, e.g., that are potentially involved in different disease processes. If the ligand has a high binding affinity for a new target protein that is involved in a disease process, then the ligand can be selected for experimental validation and potential inclusion in a drug for treating the disease. Drug repurposing can leverage the safety and efficacy data already available for a drug that includes the ligand, potentially accelerating the development process and reducing research costs. Drug repurposing can identify novel treatment options and address unmet medical needs by repurposing known ligands to treat different diseases or conditions.

[0113] To identify new target proteins for a ligand, binding affinity predictions generated by the prediction system 200 can be used to determine a ranking of candidate proteins in a collection of candidate proteins based on a respective predicted binding affinity of a particular ligand for each of the candidate proteins. The ranking of the candidate proteins in the collection of candidate proteins based on the predicted binding affinities of the ligand for the candidate proteins can be used, e.g., to select a proper subset of the collection of candidate proteins for experimental validation and testing. For instance, one or more candidate proteins for which the ligand has the highest predicted binding affinity (i.e., according to the ranking) can be selected for experimental validation and testing, e.g., for use in a drug that achieves a therapeutic effect in patients. In particular, each selected candidate protein can be physically synthesized and the binding affinity of the ligand for the candidateIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application protein can then be experimentally tested and validated. One or more of the candidate proteins from the collection of candidate proteins can be selected as binding targets for the ligand, e.g., based at least in part on results of the testing.

[0114] The collection of candidate proteins can include any appropriate number of proteins, e.g., 10 proteins, or 1000 proteins, or 100,000 proteins. In some cases, only a fraction of the candidate proteins in the set of candidate proteins are selected for physical synthesis, e.g., based on the ranking of the candidate proteins by the predicted binding affinity of the ligand for the candidate proteins. For instance, less than 50%, or less than 10%, or less than 1%, or less than 0.1% of the candidate proteins in the collection of candidate proteins may be selected for physical synthesis.

[0115] FIG. 3 is a flow diagram of an example process 300 for training a generative machine learning model to perform a structure prediction task using training examples generated by molecular dynamics simulations. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a training system, e.g., the training system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 300.

[0116] The system obtains a set of training examples for training the generative machine learning model to perform the structure prediction task (302). Each training example corresponds to a respective molecular system of one or more molecules and includes: (i) a training input to the generative machine learning model, and (ii) a target output of the generative machine learning model. The training input identifies the one or more molecules included in the molecular system. The target output defines a conformation of the molecular system.

[0117] The molecular system conformations that are included in the training examples may have been generated in any of a variety of possible ways. For instance, some of the conformations included in the training examples may have been determined through experimental techniques such as x-ray crystallography, nuclear magnetic resonance (NMR) spectroscopy, cryo-electron microscopy (Cryo-EM), and so forth. There are also many public databases that can be used to obtain training examples, such as the Protein Data Bank (wwpdb.org), and many others. In some cases, 3D structures for training the system may also be obtained using other protein structure determination systems, e.g., the system described Jumper et al., “Highly accurate protein structure prediction with AlphaFold,” Nature, Vol 596, 26 August 2021.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0118] For each training example in the set of training examples, the system performs a molecular dynamics simulation of the corresponding molecular system, starting from the conformation of the molecular system that is specified by the training example, to generate a trajectory of possible conformations adopted by the molecular system over a duration of time (304). The molecular dynamics simulation can simulate the conformational behavior of the molecular system over any appropriate duration of time, e.g., ranging from a few nanoseconds to several microseconds, and can generate any appropriate number of possible conformations, e.g., ranging from thousands of conformations to millions of conformations.

[0119] For each training example in the set of training examples, the system generates one or more new training examples corresponding to the molecular system using the conformations generated by the molecular dynamics simulations (306). More specifically, each new training example includes a target output that defines a possible conformation of the molecular system generated by the molecular dynamics simulation of the molecular system.

[0120] Several issues can arise when generating new training examples using conformations that are generated by molecular dynamics simulations. For instance, a molecular dynamics simulation can generate a very large number of new conformations, e.g., depending on the length of the timescale of the simulation and the incremental time step size used during the simulation, the number of new conformations can range from thousands to millions. Further, some of these conformations may have high levels of similarity to one another, e.g., because the molecular dynamics simulation uses small time steps, or because the simulation oscillates between a few stable conformational states. Generating a respective new training example for every conformation generated by a molecular dynamics simulation may therefore be computationally infeasible, and further, may generate highly duplicative training examples.

[0121] To address these issues, for each training example, the system can apply a clustering operation to the trajectory of conformations generated by the molecular dynamics simulation that starts from the conformation specified by the training example. The clustering operation can identify a set of clusters (groups) of conformations, where each cluster includes multiple conformations, and where conformations in the same cluster tend to be more similar than conformations in different clusters.

[0122] The clustering operation can operate on a set of feature vectors, where each feature vector represents a respective conformation. The system can represent a conformation as a feature vector in any of a variety of possible ways. For instance, to generate a feature vector representing a conformation, the system can center the conformation by translating the atomic coordinates so that the center of mass of the molecular system is at the origin, andIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application then align the conformation to a reference conformation to eliminate rotational degrees of freedom, e.g., by aligning the principal axes of the molecular system to those of the reference conformation. After centering and alignment, the system can define the feature vector as a list of 3D spatial positions of the atoms in the molecular system, where the 3D spatial positions are listed in accordance with a predefined ordering of the atoms in the molecular system. As another example, the system can represent a conformation as a feature vector defining internal coordinates of the conformation such as bond lengths, bond angles, and torsional angles.

[0123] The clustering operation can be implemented using any appropriate clustering technique, e.g., k-means clustering, expectation-maximization (EM) clustering, hierarchical clustering, and so forth. The clustering operation generates a number of clusters that is substantially less than the original number of conformations, e.g., by one or more orders of magnitude.

[0124] After applying the clustering operation to identify the clusters of conformations, the system can select a proper subset (i.e., less than all) of the conformations of the molecular system for use in generating new training examples. The manner in which the system selects which conformations to use in generating new training examples is dependent upon the clusters of conformations identified by the clustering operation. For instance, for each cluster, the system can identify a conformation that is the medoid of the cluster, and then select that conformation for use in generating a new training example. The medoid is the conformation within the cluster that has the minimum average distance to all other conformations in the cluster. As another example, for each cluster, the system can randomly sample a predefined number of conformations from the cluster for use in generating new training examples. Selecting a proper subset of the conformations from a trajectory of conformations generated by a molecular dynamics simulation for use in generating new training examples based on a clustering of the conformations can increase the likelihood that the new training examples represent a diverse range of possible conformations of the molecular system.

[0125] The system augments the set of training examples to include the new training examples (308). More specifically, the system adds the new training examples to the original set of training examples.

[0126] The system trains the generative machine learning model to perform the structure prediction task on the augmented set of training examples that includes the new training examples specifying conformations generated by molecular dynamics simulations (310). For instance, for each training example, the system can train the generative machine learningIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application model, by a machine learning training technique, to reduce a discrepancy between: (i) the conformation specified by the target output of the training example, and (ii) a conformation generated by the generative machine learning model by processing the training input of the training example. An example process for training a generative diffusion model using an objective function that depends on energy gradient data is described in more detail with reference to FIG. 7.

[0127] FIG. 4 is a flow diagram of an example process 400 for determining a predicted binding affinity of a ligand for a target molecule using conformations generated by the generative machine learning model. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a prediction system, e.g., the prediction system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 400.

[0128] The system receives data identifying a molecular system that includes a ligand and a target molecule (402). The ligand and the target molecule can be any appropriate types of molecules, e.g., small organic molecules, peptide molecules, protein molecules, nucleotide molecules, RNA molecules, DNA molecules, lipid molecules, carbohydrate molecules, and so forth. The system can receive the data identifying the molecular system, e.g., from a user, by way of a graphical user interface (GUI) or an application programming interface, and over a data communications network, e.g., the internet.

[0129] The system generates, using the generative machine learning model, a set of conformations of the molecular system that includes the ligand and the target molecule (404). The generative machine learning model can be any appropriate model that, by processing data identifying one or more molecules included in a molecular system, can generate samples from a distribution over a space of possible conformations of the molecular system. The generative machine learning model may have been trained by the training system described with reference to FIG. 1, and thus been trained in a manner that enables the generative machine learning model to generate conformations that span a wide range of conformations that can be adopted by the molecular system.

[0130] The system performs, for each of the conformations generated by the generative machine learning model, a molecular dynamics simulation starting from the conformation to generate a trajectory of conformations (406). The system thus generates a collection of conformations of the molecular system that includes: (i) the conformations generated by the generative machine learning model, and (ii) the conformations generated by performingIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application molecular dynamics simulations starting from the conformations generated by the generative machine learning model.

[0131] The system determines, for each conformation in the collection of conformations, whether the ligand is bound to the target molecule in the conformation (408). The system can determine whether the ligand is bound to the target molecule in a given conformation in any of a variety of possible ways. For instance, the system can determine a distance between the ligand and a binding pocket on the target molecule, and can determine that the ligand is bound to the target molecule only if the distance from the ligand to the binding pocket satisfies (e.g., is less than) a threshold. The system can determine the distance from the ligand to the binding pocket, e.g., as the minimal distance from any atom in the ligand to any atom in the binding pocket, or as a measure of central tendency (e.g., a mean, median, or mode) of the minimal distance of each atom in the ligand to any atom in the binding pocket.

[0132] In some implementations, the system can determine the binding affinity of the ligand for the target molecule based on a number of conformations (i.e., from the collection of conformations) where the ligand is bound to the target molecule (410). For instance, the system can determine a fraction of conformations in the collection of conformations where the ligand is bound to the target molecule, e.g., as a ratio of: (i) the number of conformations where the ligand is bound to the target molecule, and (ii) a total number of conformations in the collection of conformations. As another example, the system can determine a fraction of conformations in the collection of conformations where the ligand is bound to the target molecule, e.g., as a ratio of: (i) the number of conformations where the ligand is bound to the target molecule, and (ii) the number of conformations where the ligand is not bound to the target molecule. Intuitively, the larger the number of conformations in which the ligand is bound to the target molecule, the higher the binding affinity of the ligand for the target molecule.

[0133] The binding affinity value determined by the system, e.g., as described above, provides a numerical value that is correlated with binding affinity but that may not be expressed on the same scale as binding affinity values that are measured using experimental assays such as surface plasmon resonance (SPR), isothermal titration calorimetry (ITC), fluorescence polarization (FP), and so forth. Optionally, the system can process the initial binding affinity value using a “calibration function” to calibrate the binding affinity value by mapping it to the same scale as a binding affinity value generated by a particular experimental assay. The parameters of the calibration function can be determined based on a correlation between initial binding affinity values generated by the system (e.g., inIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application accordance with the process 400, as described above) and binding affinity values generated using experimental assays.

[0134] FIG. 5 is a flow diagram of an example process 500 for determining a predicted binding affinity of a ligand for a target molecule using an embedding of the ligand - target molecule complex that is generated by a generative machine learning model. For convenience, the process 500 will be described as being performed by a system of one or more computers located in one or more locations. For example, a prediction system, e.g., the prediction system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 500.

[0135] The system receives data identifying a molecular system that includes a ligand and a target molecule (502). The ligand and the target molecule can be any appropriate types of molecules, e.g., small organic molecules, peptide molecules, protein molecules, nucleotide molecules, RNA molecules, DNA molecules, lipid molecules, carbohydrate molecules, and so forth. The system can receive the data identifying the molecular system, e.g., from a user, by way of a graphical user interface (GUI) or an application programming interface, and over a data communications network, e.g., the internet.

[0136] The system processes data identifying the ligand and the target molecule using an embedding neural network included in the generative machine learning model to generate an embedding of the molecular system that includes the ligand and the target molecule (504). In more detail, the generative machine learning model can include an: (i) an embedding neural network, and (ii) a generative neural network. The embedding neural network can be configured to process data identifying one or more molecules included in a molecular system to generate an embedding of the molecular system. The generative neural network can be configured to generate one or more conformations of the molecular system when conditioned on the embedding of the molecular system, i.e., as generated by the embedding neural network. For instance, in an implementation where the generative machine learning model is a generative diffusion model, the generative neural network can be a denoising neural network, e.g., as described with reference to FIG. 6. As another example, in an implementation where the generative machine learning model is a generative adversarial neural network (GAN), the generative neural network can be the generator of the GAN. Generative adversarial neural networks are described in, for example, Goodfellow et al., arXiv: 1406.2661.

[0137] The embedding neural network and the generative neural network can each have any appropriate neural network architectures. For instance, each can include any appropriateIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application types of neural network layers (e.g., fully connected layers, attention layers, convolutional layers, and so forth) in any appropriate number (e.g., 5 layers, 10 layers, or 20 layers) and connected in any appropriate configuration (e.g., as a directed graph of layers).

[0138] Example architectures of particular embedding neural networks and generative neural networks are described with reference to: Abramson, Josh, et al. "Accurate structure prediction of biomolecular interactions with AlphaFold 3." Nature (2024): 1-3; Watson, Joseph L., et al. "De novo design of protein structure and function with RF diffusion." Nature 620.7976 (2023): 1089-1100; Watson, Joseph L., et al. "Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models." BioRxiv (2022): 2022-12.

[0139] The embedding neural network and the generative neural network are jointly trained during the training of the generative machine learning model. More specifically, during training, gradients of an objective function are backpropagated through the generative neural network and into the embedding neural network. Training the generative machine learning model to generate wide ranges of possible conformations of molecular systems (as described with reference to FIG. 1) causes the embedding neural network to generate embeddings that encode information characterizing many possible conformations of molecular systems. These embeddings are therefore particularly useful for predicting binding affinities because the binding affinity of a ligand for a target molecule is a property that is closely related to the conformations adopted by the ligand and the target molecule when bound in a complex.

[0140] The system processes the embedding of the molecular system using a binding affinity machine learning model to predict the binding affinity of the ligand for the target molecule (506). For example, the binding affinity machine learning model can be configured to process an input comprising the embedding of a molecular system comprising a ligand and the target molecule to generate an output comprising a predicted binding affinity of the ligand for the target molecule. The binding affinity machine learning model can be any appropriate type of machine learning model, e.g., a neural network, or a random forest, or a support vector machine, or a linear regression model, and so forth. In implementations where the binding affinity machine learning model is implemented as a neural network, the binding affinity machine learning model can include any appropriate types of neural network layers (e.g., fully connected layers, attention layers, convolutional layers, and so forth) in any appropriate number (e.g., 5 layers, 10 layers, or 20 layers) and connected in any appropriate configuration (e.g., as a directed graph of layers).Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0141] The system can train the binding affinity machine learning model on a set of training examples using a machine learning training technique. Each training example corresponds to a respective training molecular system that includes a ligand and a target molecule. Each training example includes: (i) an embedding of the training molecular system that is generated by processing data identifying the training molecular system using the embedding neural network of the generative machine learning model, and (ii) a target binding affinity of the ligand and the target molecule of the training molecular system. The target binding affinity may be generated, e.g., using an appropriate experimental assay, e.g., surface plasmon resonance (SPR), isothermal titration calorimetry (ITC), or fluorescence polarization (FP). Training examples are available from publicly or commercially available databases, such as ChEMBL(https: / / www.ebi. ac.uk / chembl / ), which is a manually curated database of bioactive molecules with drug-like properties. Protocols for measuring binding affinities are described in, for example, Zambaldi et al., “De novo design of high-affinity protein binders with AlphaProteo” arXiv: 2409.08022.

[0142] For each training example, training the binding affinity machine learning model on the training example can include training the binding affinity machine learning model to reduce a discrepancy between: (i) the target binding affinity specified by the training example, and (ii) a predicted binding affinity generated by processing the molecular system embedding included in the training example using the binding affinity machine learning model. The system can measure the discrepancy between the target binding affinity and the predicted binding affinity, e.g., using an absolute error, or a squared error, or in any other appropriate way. The specific operations of the machine learning training technique depend on the type of machine learning model that is used to implement the binding affinity machine learning model. For instance, if the binding affinity machine learning model is implemented as a neural network, then the machine learning training technique can be a stochastic gradient descent training technique.

[0143] FIG. 6 is a flow diagram of an example process 600 for generating a predicted conformation of a molecular system using a generative machine learning model that is implemented as a generative diffusion model. For convenience, the process 600 will be described as being performed by a system of one or more computers located in one or more locations. For example, a prediction system, e.g., the prediction system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 600.Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0144] The system generates a respective initial 3D spatial position for each atom in the molecular system (602). For instance, for each atom in the molecular system, the system can sample a 3D spatial position of the atom from a probability distribution over 3D space, e.g., a standard Normal distribution over 3D space.

[0145] The system performs steps 604 - 610, which are described next, over a sequence of iterations that may be referred to as “denoising iterations”. The description of steps 604 - 610 which follows will reference a “current” denoising iteration for convenience; the current denoising iteration can be any denoising iteration in the sequence of denoising iterations. The system can perform the steps 604 - 610 over any appropriate number of denoising iterations, e.g., 3 denoising iterations, 10 denoising iterations, or 100 denoising iterations. The number of denoising iterations can be a predetermined number of denoising iterations.

[0146] The system processes a network input that defines a current 3D spatial position of each atom in the molecular system using a denoising neural network to generate a denoising output (604). If the current denoising iteration is the first denoising iteration, then the current 3D spatial position of each atom may be the initial 3D spatial positions generated at step 602. If the current denoising iteration is after the first denoising iteration, then the current 3D spatial position of each atom may have been generated by a diffusion sampler at the preceding denoising iteration, e.g., as will be described with reference to step 610. The network input to the denoising neural network can additionally include: (i) an embedding of the molecular system that is generated by processing data identifying the one or more molecules included in the molecular system using an embedding neural network, and (ii) data identifying the current denoising iteration.

[0147] The denoising output can be any appropriate data that enables estimation of the “final” 3D spatial position of each atom in the molecular system. For instance, the denoising output can define, for each atom in the molecular system, a predicted error in the 3D spatial position of the atom at the current denoising iteration. As another example, the denoising output can directly define, for each atom in the molecular system, a predicted 3D spatial position of the atom. As another example, the denoising output can define, for each atom in the molecular system, both: (i) a predicted error in the 3D spatial position of the atom at the current denoising iteration, and (ii) a predicted 3D spatial position of the atom. As another example, the denoising output can define, for each atom in the molecular system, a prediction for a value that is a linear combination of: (i) an actual 3D spatial position of the atom, and (ii) an error between the 3D spatial position of the atom at the current denoising iteration and the actual 3D spatial position of the atom, e.g., as implemented by the v-parametrizationIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application described in: Tim Salimans, Jonathan Ho, “Progressive distillation for fast sampling of diffusion models,” ICLR 2022, arXiv:2202.00512v2.

[0148] The system generates an initial estimate of the 3D spatial position for each atom in the molecular system using at least the denoising output generated by the denoising neural network (606). The system can generate the initial estimate of the 3D spatial position for each atom in the molecular system in any appropriate way, depending on the form of the denoising output. A few example techniques for generating the initial estimate of the 3D spatial position for each atom in the molecular system using the denoising output are described next.

[0149] In one example, the denoising output defines, for each atom, a respective prediction for the 3D spatial position of the atom. In this example, the respective predicted 3D spatial position for each atom defines the initial estimate of the 3D spatial position for the atom.

[0150] In another example, the denoising output defines, for each atom, a predicted error in the 3D spatial position of the atom at the current denoising iteration. In this example, the system can generate the initial estimate for the 3D spatial position for each atom as a linear combination of: (i) the current 3D spatial position of the atom, and (ii) the predicted error in the 3D spatial position of the atom. Each term in the linear combination can be scaled by a respective constant value that is dependent on the denoising iteration. For instance, the system can generate the initial estimate for the 3D spatial position xt-for an atom in the molecular system as:

[0151] where t indexes the current denoising iteration, at, at, and atare constants specific to denoising iteration t, and eext, t) is the predicted error in the 3D spatial position of the atom (e.g., as generated by the denoising neural network at the denoising iteration). (In the notation of equation (1), the denoising iterations decrement, such that denoising iteration t — 1 is the “next” denoising iteration after denoising iteration t). The constants in equation (1) (at, at, and <Jt) can be selected in accordance with a predefined noise schedule.

[0152] In another example, the denoising output defines, for each atom, both: (i) a predicted 3D spatial position of the atom, and (ii) a predicted error in the 3D spatial position of the atom at the current denoising iteration. In this example, the system can generate the initial estimate for the 3D spatial position of the atom as a combination (e.g., an average) of: (i) the predicted 3D spatial position of the atom as specified by the denoising output, and (ii) a predicted 3D spatial position of the atom that is derived from the predicted error in the 3D spatial position of the atom at the current denoising iteration, e.g., using equation (1).Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0153] In another example, the denoising output is expressed using a v-parametrization, and the system generates a respective initial estimate for the 3D spatial position for each atom using the techniques described in Tim Salimans, Jonathan Ho, “Progressive distillation for fast sampling of diffusion models,” ICLR 2022, arXiv:2202.00512v2.

[0154] If the current denoising iteration is the final denoising iteration (i.e., in the sequence of denoising iterations), the system outputs the initial estimate of the 3D spatial position of each atom in the molecular system as the predicted conformation of the molecular system (612).

[0155] If the current denoising iteration is not the final denoising iteration, the system generates a respective 3D spatial position for each atom for the next denoising iteration based on the initial estimates of the 3D spatial positions of the atoms (as generated at step 606) using an appropriate diffusion sampling technique (610). A few examples of possible diffusion sampling techniques are described next.

[0156] In one example, the system can generate the 3D spatial position of each atom in the molecular system at the next denoising iteration by combining random noise with the initial estimate of the 3D spatial position of the atom. For instance, for each atom, the system can add respective random noise to the initial estimate of the 3D spatial position of the atom. The random noise can be sampled from a probability distribution over 3D space. The probability distribution over 3D space can vary based on the denoising iterations, e.g., such that the variance of the noise combined with the updated 3D spatial positions of the atoms decreases over the sequence of denoising iterations.

[0157] As another example, the system generates the 3D spatial position of each atom in the molecular system at the next denoising iteration using a deterministic diffusion sampling technique, i.e., that does not rely on random noise. An example of a deterministic diffusion sampling technique is the denoising diffusion implicit model (DDIM), e.g., as described in: Jiaming Song, Chenlin Meng, Stefano Ermon, “Denoising diffusion implicit models,” ICLR 2021, arXiv:2010.02502v4.

[0158] Optionally, the system can perform the process 600 multiple times to generate multiple predictions for the conformation of the molecular system. Each execution of the steps of the process 600 can result in the generation of a different predicted conformation of the molecular system, e.g., as a result of stochasticity in the random sampling performed to generate the initial positions of the atoms (at step 602), and in some cases, as a result of stochasticity in diffusion sampler (at step 610).Isomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application

[0159] FIG. 7 is a flow diagram of an example process 700 for training a generative diffusion model using an objective function that depends on energy gradient data. For convenience, the process 700 will be described as being performed by a system of one or more computers located in one or more locations. For example, a training system, e.g., the training system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 700.

[0160] The system obtains a training example for training the generative diffusion model (702). The training example corresponds to a molecular system and includes data that defines a conformation of the molecular system.

[0161] The system determines energy gradient data for the conformation of the molecular system (704). More specifically, the system determines a force field that defines a model for the energy of the conformation of the molecular system, e.g., based on one or more of: bond stretching (describing the energy associated with the stretching or compressing of covalent bonds between two atoms), angle bending (describing the energy associated with the bending of bond angles between three atoms connected by covalent bonds), torsions (describing the energy associated with the rotation around a bond connecting two atoms), and non-bonded interactions (e.g., Van der Waals forces and electrostatic interactions). The force field that defines the model for the energy of the conformation of the molecular system can be based on, e.g., AMBER (Assisted Model Building with Energy Refinement), CHARMM (Chemistry at Harvard Macromolecular Mechanics), or OPLS (Optimized Potentials for Liquid Simulations). The force field that defines the model for the energy of the conformation of the molecular system can be defined as a function of the 3D spatial positions of the atoms of the molecular system in the conformation. The system can determine the energy gradient data by determining, for each atom in the molecular system, a gradient of the force field (i.e., a gradient of the energy) with respect to the 3D spatial position of the atom in the conformation.

[0162] The system samples a denoising iteration from a predefined sequence of denoising iterations (706). More specifically, during inference, the generative diffusion model can be configured to perform a sequence of denoising iterations, e.g., as described with reference to steps 604 - 610 of FIG. 6. During training, the system can randomly sample a single denoising iteration from the sequence of denoising iterations, e.g., in accordance with a uniform distribution over the sequence of denoising iterations.

[0163] The system generates a noisy conformation of the molecular system by combining random noise with the respective spatial position of each atom in the conformation of theIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application molecular system as defined by the training example (708). For instance, for each atom in the molecular system, the system can generate a noisy spatial position for the atom by adding random noise to the spatial position of the atom in the conformation defined by the training example. The system can scale the random noise combined with the spatial positions of the atoms by a scaling coefficient that depends on the sampled denoising iteration, e.g., where the values of the scaling coefficients corresponding to the denoising iterations are defined by a noise schedule. In particular, the noise schedule can be defined so that the values of the noise scaling coefficients monotonically decrease with the denoising iterations, e.g., so that lesser amounts of noise are combined with the conformation of the molecular system at higher denoising iterations.

[0164] The system processes a network input that includes data defining the noisy conformation of the molecular system using the denoising neural network to generate a denoising output (710).

[0165] The system determines gradients of an objective function that measures an error between: (i) the denoising output, and (ii) an overall denoising target, and uses the gradients to update the parameter values of the denoising neural network (712). Optionally, the system also determines gradients of the objective function with respect to one or more other neural networks included in the generative diffusion model (e.g., the embedding neural network described with reference to FIG. 5), and uses the gradients to update the parameter values of the other neural networks.

[0166] The overall denoising target can be a combination of (i) a first denoising target, and (ii) a second denoising target.

[0167] The first denoising target can be based on the conformation of the molecular system as defined by the training example, or the noise combined with the conformation of the molecular system as defined by the training example, or both. For instance, the first denoising target can define an output of the denoising neural network that, if used to generate an initial estimate of the 3D spatial positions of the atoms in the molecular system (as described in step 608 of FIG. 6), would cause the initial estimate of the 3D spatial positions of the atoms to match the target (actual) 3D spatial positions of the atoms in the conformation defined by the training example training example.

[0168] The second denoising target can be based on the energy gradient data, e.g., as described with reference to step 704. For instance, the second denoising target may be given by the energy gradient data, or by an appropriate transformation of the energy gradient data.Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application

[0169] In a particular example, when the objective function is implemented as a score matching objective, the first denoising target may be a rescaled version of the noise combined with the conformation of the molecular system as defined by the training example, and the second denoising target may be a rescaled version of the energy gradient data.

[0170] In some cases, the overall denoising target may be a linear combination of the first denoising target and the second denoising target, e.g., such that the objective function L may be given by:

[0171] where 0 define the neural network parameters of the generative diffusion model (including the neural network parameters of the denoising neural network),is the first denoising target, T2is the second denoising target, a> E (0,1) is a scaling value, and se(t,Xt) is the denoising output of the denoising neural network.

[0172] The second denoising target (i.e., that is based on the energy gradient data) is particularly effective for influencing the denoising of the noisy conformation at high denoising iterations where the noisy conformation is already close to the un-noised conformation. Therefore the system can select the value of the scaling coefficient that scales the second denoising target (i.e., that is based on the energy gradient data) as a monotonically increasing function of the denoising iteration, e.g., so that the overall denoising target is increasingly dominated by the second denoising target (i.e., that is based on the energy gradient data) at higher denoising iterations.

[0173] This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0174] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as oneIsomorphic Labs LimitedF&R Ref.: 53672-0018W01 PCT Application or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0175] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0176] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0177] In this specification the term “engine” is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or moreIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0178] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0179] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0180] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0181] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from aIsomorphic Labs LimitedF&R Ref.: 53672-0018WO1 PCT Application device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0182] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and compute-intensive parts of machine learning training or production, i.e., inference, workloads.

[0183] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework.

[0184] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0185] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0186] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment.Isomorphic Labs Limited F&R Ref.: 53672-0018W01 PCT ApplicationConversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0187] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0188] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT ApplicationCLAIMS1. A method performed by one or more computers, the method comprising: obtaining a set of training examples for training a generative machine learning model to perform a structure prediction task, wherein each training example corresponds to a respective molecular system of one or more molecules and comprises:(i) a training input to the generative machine learning model that identifies the one or more molecules included in the molecular system, and(ii) a target output of the generative machine learning model that defines a conformation of the molecular system; augmenting the set of training examples to include a plurality of new training examples, comprising, for each training example in the set of training examples: performing a molecular dynamics simulation of the molecular system, starting from the conformation of the molecular system that is specified by the training example, to generate a trajectory of new conformations adopted by the molecular system over a duration of time; generating one or more new training examples corresponding to the molecular system, wherein each new training example includes a target output that defines a new conformation of the molecular system generated by the molecular dynamics simulation of the molecular system; and adding the one or more new training examples to the set of training examples; and training the generative machine learning model to perform the structure prediction task on the augmented set of training examples that includes the new training examples specifying new conformations generated by molecular dynamics simulations.

2. The method of claim 1, wherein for each of one or more training examples in the set of training examples, generating one or more new training examples corresponding to the molecular system comprises: applying a clustering operation to the trajectory of new conformations generated by the molecular dynamics simulation of the molecular system to identify a plurality of clusters of conformations; and selecting one or more new conformations from the trajectory of new conformations, based on the plurality of clusters of conformations, for use in generating new training44Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application examples; wherein only a proper subset of new conformations from the trajectory of new conformations are selected for use in generating new training examples.

3. The method of claim 2, selecting one or more new conformations from the trajectory of new conformations, based on the plurality of clusters of conformations, for use in generating new training examples comprises: selecting, for each cluster of conformations, a conformation that defines a medoid of the cluster of conformations for use in generating a new training example.

4. The method of any preceding claim, further comprising, after training the generative machine learning model to perform the structure prediction task on the augmented set of training examples: receiving data identifying an input molecular system that comprises a ligand and a target molecule; generating, using the generative machine learning model, a plurality of conformations of molecular system that includes the ligand and the target molecule; and determining a binding affinity of the ligand for the target molecule using the plurality of conformations of the molecular system that includes the ligand and the target molecule.

5. The method of claim 4, wherein determining the binding affinity of the ligand for the target molecule using the plurality of conformations of the molecular system that includes the ligand and the target molecule comprises: performing, for each of the plurality of conformations of the molecular system, a molecular dynamics simulation starting from the conformation to generate a trajectory of conformations; determining, for each conformation in each trajectory of conformations, whether the ligand is bound to the target molecule in the conformation; and determining the binding affinity based on a number of conformations from the trajectories of conformations where the ligand is bound to the target molecule.

6. The method of claim 5, wherein determining, for each conformation in each trajectory of conformations, whether the ligand is bound to the target molecule in the conformation comprises, for each conformation:45Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application determining a distance between the ligand and a binding pocket of the target molecule in the conformation; and determining whether the ligand is bound to the target molecule in the conformation based on the whether the distance between the ligand and the binding pocket satisfies a threshold.

7. The method of any preceding claim, wherein the generative machine learning model comprises an embedding neural network and a generative neural network; wherein the embedding neural network is configured to process data identifying one or more molecules included in a molecular system to generate an embedding of the molecular system; and wherein the generative neural network is configured to generate one or more conformations of the molecular system when conditioned on the embedding of the molecular system.

8. The method of claim 7, wherein training the generative machine learning model comprises jointly training the embedding neural network and the generative neural network by backpropagating gradients through the generative neural network and into the embedding neural network.

9. The method of any one of claims 7-8, further comprising, after training the generative machine learning model to perform the structure prediction task: receiving data identifying an input molecular system that comprises a ligand and a target molecule; generating, using the embedding neural network, an embedding of the input molecular system that comprises the ligand and the target molecule; and processing the embedding of the input molecular system using a binding affinity machine learning model to predict a binding affinity of the ligand for the target molecule.

10. The method of any preceding claim, wherein training the generative machine learning model to perform the structure prediction task comprises, for each training example in the augmented set of training examples: training the generative machine learning model, by a machine learning training technique, to reduce a discrepancy between: (i) the conformation specified by the target46Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application output of the training example, and (ii) a conformation generated by the generative machine learning model by processing the training input of the training example.

11. The method of any preceding claim, wherein the generative machine learning model comprises a generative diffusion model that is configured to perform operations comprising: receive positional data defining a respective initial position of each atom in a molecular system; and denoise the positional data over a sequence of time steps using a denoising neural network over a sequence of denoising iterations; wherein after a final time step in the sequence of time steps, the positional data defines a predicted conformation of the molecular system.

12. The method of claim 11, wherein training the generative machine learning model to perform the structure prediction task comprises, for each training example: determining a force field that defines a model of an energy of the molecular system corresponding to the training example; generating energy gradient data that defines, for each of a plurality of atoms in the molecular system, a gradient of the force field with respect to a position of the atom in the conformation of the molecular system that is specified by the target output of the training example; training the denoising neural network to optimize an objective function that depends at least in part on the energy gradient data.

13. The method of claim 12, wherein training the denoising neural network to optimize an objective function that depends at least in part on the energy gradient data comprises: generating a noisy conformation of the molecular system by combining noise with the conformation of the molecular system that is specified by the target output of the training example; processing a network input based on the noisy conformation of the molecular system using the denoising neural network to generate a denoising output; wherein the objective function measures an error between the denoising output and an overall denoising target; wherein the overall denoising target is a combination of:(i) a first denoising target that is based on the conformation of the molecularIsomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application system that is specified by the target output of the training example, or the noise combined with the conformation of the molecular system that is specified by the target output of the training example, or both; and(ii) a second denoising target that is based on the energy gradient data.

14. The method of claim 13, wherein the overall denoising target is a linear combination of the first denoising target and the second denoising target.

15. The method of claim 14, wherein generating the noisy conformation of the molecular system by combining noise with the conformation of the molecular system that is specified by the target output of the training example comprises: sampling a denoising iteration; and determining a noise scaling coefficient as a monotonically decreasing function of denoising iterations; wherein the linear combination of the first denoising target and the second denoising target comprises an energy scaling coefficient that scales the second denoising target that is based on the energy gradient data; and wherein the energy scaling coefficient is defined as a monotonically increasing function of denoising iterations.

16. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of claims 1-15.

17. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of claims 1-15.

18. A method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands:Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application(a) using a generative machine learning model trained using the method of any one of claims 1-3 or 7-8 or 10-15 to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and(b) evaluating an interaction of the candidate ligand with the target protein molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating.

19. A method as claimed in claim 18, wherein the target protein molecule comprises a receptor or enzyme, and wherein the ligand is an agonist or antagonist of the receptor or enzyme.

20. A method as claimed in claim 18 or 19 wherein the ligand is a drug, and the method comprises: performing steps (a) and (b) for each of a plurality of target protein molecules; and selecting one or more of the candidate ligands as the ligand to either i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

21. The method of claim 18, wherein the ligand comprises an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, in particular a virus or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect.

22. The method of any one of claims 18-21, wherein the ligand is a polypeptide ligand, a polynucleoside ligand, or a polynucleotide ligand.

23. A method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: using a generative machine learning model trained using the method of any one of claims 1-15 to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and49Isomorphic Labs Limited F&R Ref.: 53672-0018WO1 PCT Application evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating.

24. A method as claimed in any one of claims 18-23 wherein the evaluating the interaction of one of the candidate ligands comprises determining an interaction score for the candidate ligand, wherein the interaction score comprises a measure of an interaction between the candidate ligand and the target molecule.

25. A method as claimed in any one of claims 18-24 further comprising synthesizing the ligand or diagnostic antibody or aptamer marker.

26. A method as claimed in claim 25 further comprising testing biological activity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

27. A method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; performing the method of any one of claims 1-3 or 7-8 or 10-15 to determine a predicted structure of the molecule complex; using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

28. A method as claimed in claim 27, wherein the experimental technique comprises one or more of: x-ray crystallography, nuclear magnetic resonance, and electron microscopy.50

Images