Scoring predicted three-dimensional (3D) structures of molecular systems

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a quality score for a 3D structure of a molecular system In one aspect, a method comprises: receiving data identifying one or more molecules included in a molecular system; processing a model input that identifies the one or more molecules included in the molecular system using a generative machine learning model to generate, as model outputs of the generative machine learning model, a plurality of predicted 3D structures of the molecular system; generating, for each of the plurality of predicted 3D structures of the molecular system, a quality score; and outputting the plurality of predicted 3D structures of the molecular system and the quality scores for the plurality of predicted 3D structures of the molecular system.

Description

[0001] Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0002] SCORING PREDICTED THREE-DIMENSIONAL (3D) STRUCTURES OF MOLECULAR SYSTEMS

[0003] CROSS-REFERENCE TO RELATED APPLICATIONS

[0004] This application claims priority to U.S. Provisional Application No. 63 / 725,575 filed on November 27, 2024. The disclosure of the prior application is considered part of and is incorporated by reference in the disclosure of this application.

[0005] BACKGROUND

[0006] This specification relates to generating a quality score for a predicted three- dimensional (3D) structure of a molecular system, for example for applications in drug discovery.

[0007] Predictions can be made using machine learning models. Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model. Some machine learning models are deep models that employ multiple layers of models to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output.

[0008] SUMMARY

[0009] This specification describes a system implemented as computer programs on one or more computers in one or more locations that can predict a quality score for a 3D structure of a molecular system. The quality score can estimate a similarity between the 3D structure and an actual 3D structure of the molecular system. The molecular system can include one or more molecules such as a protein, ligand, or nucleic acid.

[0010] There are also described efficient methods for training a neural network to generate a quality score, in particular to reduce a number of actual 3D structures needed for the training; and efficient methods for training a generative machine learning model using the quality score, that can also reduce a number of actual 3D structures needed.

[0011] A “protein” can be understood to refer to any biological molecule that is specified by one or more sequences (or “chains”) of amino acids. For example, the term protein can refer to a protein domain, e.g., a portion of an amino acid chain of a protein that can undergo Isomorphic Labs Limited

[0012] F&R Ref: 53672-0024W01 PCT Application protein folding nearly independently of the rest of the protein. As another example, the term protein can refer to a protein complex, i.e., that includes multiple amino acid chains that jointly fold into a protein structure.

[0013] A “ligand” can refer to a molecule or compound that binds to a target molecule, e.g., a protein. Ligands can include, e.g., small organic molecules (e.g. with a molecular weight of <900 daltons), complex organic molecules, proteins, biomolecules, and so forth.

[0014] A “multiple sequence alignment” (MSA) for an amino acid chain in a protein specifies a sequence alignment of the amino acid chain with multiple additional amino acid chains, e.g., from other proteins, e.g., homologous proteins. More specifically, the MSA can define a correspondence between the positions in the amino acid chain and corresponding positions in multiple additional amino acid chains. A MSA for an amino acid chain can be generated, e.g., by processing a database of amino acid chains using any appropriate computational sequence alignment technique, e.g., progressive alignment construction. The amino acid chains in the MSA can be understood as having an evolutionary relationship, e.g., where each amino acid chain in the MSA may share a common ancestor. The correlations between the amino acids in the amino acid chains in a MSA for an amino acid chain can encode information that is relevant to predicting the structure of the amino acid chain.

[0015] “Conditioning” a model (e.g., a generative machine learning model) or a neural network (e.g., a scoring neural network) or an operation on conditioning data (e.g., a model input) can refer to providing the conditioning data as an input to the model, neural network, or operation, such that outputs generated by the model, neural network, or operation are influenced by (depend on) the conditioning data.

[0016] A 3D spatial position of an atom can be represented by a set of coordinates in an appropriate coordinate system, e.g., a 3D Cartesian coordinate system or a spherical coordinate system.

[0017] In one aspect there is described a method performed by one or more computers, the method comprising: receiving data identifying one or more molecules included in a molecular system; processing a model input that identifies the one or more molecules included in the molecular system using a generative machine learning model to generate, as model outputs of the generative machine learning model, a plurality of predicted 3D structures of the molecular system; generating, for each of the plurality of predicted 3D structures of the molecular system, a quality score that estimates a similarity between: (i) the predicted 3D structure of the molecular system, and (ii) an actual 3D structure of the molecular system, comprising, for each of the plurality of predicted 3D structures of the molecular system: processing a scoring Isomorphic Labs Limited

[0018] F&R Ref: 53672-0024W01 PCT Application model input that comprises: (i) data identifying the one or more molecules included in the molecular system, and (ii) the predicted 3D structure of the molecular system, using a scoring neural network to generate the quality score for the predicted 3D structure of the molecular system; and outputting the plurality of predicted 3D structures of the molecular system and the quality scores for the plurality of predicted 3D structures of the molecular system.

[0019] In some implementations, the scoring neural network has been trained on a set of training examples that each correspond to a respective training molecular system and include: (a) a training input that comprises data that identifies one or more molecules included in the training molecular system and a predicted 3D structure of the training molecular system that is generated by the generative machine learning model; and (b) a target quality score that measures a similarity between the predicted 3D structure of the training molecular system and an actual 3D structure of the training molecular system.

[0020] In some implementations, the method further includes, for each training example, generating the target quality score for the training example by performing operations comprising: obtaining data defining a baseline 3D structure of the training molecular system; performing a molecular dynamics simulation to generate a molecular conformational trajectory that includes: (i) the baseline 3D structure of the training molecular system, and (ii) a plurality of additional 3D structures of the molecular system; determining, for each a plurality of 3D structures that are included in the molecular conformational trajectory, a similarity measure between: (i) the predicted 3D structure, and (ii) the 3D structure that is included in the molecular conformational trajectory; and determining the target quality score based on the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory.

[0021] In some implementations, the method further includes, for each training example, generating the target quality score for the training example by performing operations comprising: obtaining data defining a baseline 3D structure of the training molecular system; performing a Monte Carlo simulation to generate a molecular conformational trajectory that includes: (i) the baseline 3D structure of the training molecular system, and (ii) a plurality of additional 3D structures of the molecular system; determining, for each a plurality of 3D structures that are included in the molecular conformational trajectory, a similarity measure between: (i) the predicted 3D structure, and (ii) the 3D structure that is included in the molecular conformational trajectory; and determining the target quality score based on the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory. Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0022] In some implementations, the target quality score based on the similarity measures comprises: determining the target quality score as a minimum of the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory.

[0023] In some implementations, training the scoring neural network on the set of training examples comprises training the scoring neural network, by a machine learning training technique, to optimize an objective function that, for each training example, measures an error between: (i) the target quality score specified by the training example, and (ii) a predicted quality score generated by processing the training input of the training example using the scoring neural network.

[0024] In some implementations, the method further comprises continuously generating predicted 3D structures of the molecular system using the generative machine learning model until a termination criterion is satisfied; wherein the termination criterion is based at least in part on quality scores generated by the scoring neural network for the predicted 3D structures generated by the generative machine learning model.

[0025] In some implementations, the termination criterion is that the generative machine learning model has generated at least a threshold number of predicted 3D structures that each have a respective quality score that satisfies a quality score threshold.

[0026] In some implementations, the threshold number of predicted 3D structures is one.

[0027] In some implementations, the method further comprises training the generative machine learning model to optimize an objective function that is based on quality scores generated by the scoring neural network for predicted 3D structures generated by the generative machine learning model.

[0028] In some implementations, training the generative machine learning model to optimize an objective function that is based on quality scores generated by the scoring neural network comprises, for one or more of the plurality of predicted 3D structures of the molecular system: determining gradients, with respect to a set of generative machine learning model parameters of the generative machine learning model, of an objective function that is based on the quality score generated by the scoring neural network for the predicted 3D structure of the molecular system; and updating current values of the set of generative machine learning model parameters using the gradients.

[0029] In some implementations, the generative machine learning model and the scoring neural network have been trained by operations comprising: obtaining training data that, for each of a plurality of training molecular systems, identifies an actual 3D structure of the Isomorphic Labs Limited

[0030] F&R Ref: 53672-0024W01 PCT Application molecular system; partitioning the plurality of training molecular systems characterized by the training data into: (i) a first set of training data for training the generative machine learning model, and (ii) a second set of training data for training the scoring neural network; training the generative machine learning model on the first set of training data; after training the generative machine learning model on the first set of training data, training the scoring neural network on the second set of training data, comprising: generating predicted 3D structures of the training molecular systems of the second set of training data using the trained generative machine learning model; and the scoring neural network using the predicted 3D structures generated by the trained generative machine learning model of the training molecular systems of the second set of training data.

[0031] In some implementations, there is no overlap between the first set of training data for training the generative machine learning model and the second set of training data for training the scoring neural network.

[0032] In some implementations, generative machine learning model comprises a generative diffusion model.

[0033] In some implementations, the scoring neural network is implemented as a graph neural network that includes a plurality of message passing neural network layers.

[0034] In some implementations, the method further comprises determining a ranking of the plurality of predicted 3D structures of the molecular system based on the quality scores.

[0035] In some implementations, the method further comprises selecting one of the plurality of predicted 3D structures based at least in part on the ranking of the plurality of predicted 3D structures according to the quality scores.

[0036] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0037] In another aspect there is described a method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands: (a) selecting one of the plurality of predicted 3D structures based at least in part on the ranking of the plurality of predicted 3D structures according to the quality scores as described above, to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and (b) evaluating an interaction of the candidate ligand with the target protein molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating. Isomorphic Labs Limited

[0038] F&R Ref: 53672-0024WO1 PCT Application

[0039] In some implementations, the target protein molecule comprises a receptor or enzyme, and wherein the ligand is an agonist or antagonist of the receptor or enzyme.

[0040] In some implementations, the ligand is a drug, and the method comprises: performing steps (a) and (b) for each of a plurality of target protein molecules; and selecting one or more of the candidate ligands as the ligand to either i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

[0041] In some implementations, the ligand comprises an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, in particular a virus or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect.

[0042] In some implementations, the ligand is a polypeptide ligand, a polynucleotide ligand, or a polynucleotide ligand.

[0043] In another aspect there is described a method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands: (a) using a generative machine learning model trained as described above, to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and (b) evaluating an interaction of the candidate ligand with the target protein molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating.

[0044] In another aspect there is described a method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: selecting one of the plurality of predicted 3D structures based at least in part on the ranking of the plurality of predicted 3D structures according to the quality scores as described above, to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating.

[0045] In some implementations, the evaluating the interaction of one of the candidate ligands comprises determining an interaction score for the candidate ligand, wherein the interaction score comprises a measure of an interaction between the candidate ligand and the target molecule. Isomorphic Labs Limited

[0046] F&R Ref: 53672-0024WO1 PCT Application

[0047] In some implementations, the method further comprises synthesizing the ligand or diagnostic antibody or aptamer marker.

[0048] In some implementations, the method further comprises testing biological activity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

[0049] In another aspect there is described a method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: using a generative machine learning model trained as described above, to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating.

[0050] In another aspect there is described a method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; selecting one of the plurality of predicted 3D structures based at least in part on the ranking of the plurality of predicted 3D structures according to the quality scores as described above, to determine a predicted structure of the molecule complex; and using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

[0051] In some implementations, the experimental technique comprises one or more of: x-ray crystallography, nuclear magnetic resonance, and electron microscopy.

[0052] In another aspect there is described a method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; using a generative machine learning model trained as described above, to determine a predicted structure of the molecule complex; and using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

[0053] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0054] A generative machine learning model can be trained to perform a structure prediction task, in particular, trained to process data identifying a molecular system to generate a Isomorphic Labs Limited

[0055] F&R Ref: 53672-0024W01 PCT Application predicted 3D structure of the molecular system. Upon prompting, such a generative machine learning model can generate large numbers of unique predicted 3D structures for the molecular system. The diversity in predicted 3D structures generated by the generative machine learning model can account for, e.g., various conformations that the molecular system adopts over time due to rotations of flexible bonds, and also can arise from inherent uncertainties in the generative machine learning model itself. In particular, certain predicted 3D structures generated by the generative machine learning model may be accurate, i.e., may have a high similarity to an actual 3D structure of the molecular system, while others may be inaccurate, i.e., may have a low similarity to the actual 3D structure of the molecular system. An issue that arises when using a generative machine learning model to perform a structure prediction task is that the generative machine learning model may generate some inaccurate predicted 3D structures of the molecular system, but distinguishing accurate predicted 3D structures from inaccurate predicted 3D structures can be challenging.

[0056] The system described in this specification addresses this issue. More specifically, the system can generate a quality score for a predicted 3D structure of a molecular system using a scoring neural network. The quality score estimates a similarity between the predicted 3D structure and an actual 3D structure of the molecular system; that is, the quality score characterizes the accuracy of the predicted 3D structure of the molecular system. Example processes for training a scoring neural network to generate such quality scores will be described in detail in this specification.

[0057] The system can use the quality scores generated by the scoring neural network in a variety of possible ways, several of which are described next.

[0058] For example, the system can use quality scores generated by the scoring neural network for a set of predicted 3D structures generated by the generative machine learning model for a molecular system to rank the set of predicted 3D structures and select one or more of the predicted 3D structures for further processing. For instance, the system can provide the selected 3D structures for further computational validation which can include, e.g., using a molecular mechanics algorithm, e.g., based on force fields such as AMBER or CHARMM, to relax the structure and minimize its potential energy, thus optimizing bond lengths, bond angles, and non-bonded interactions; or performing a molecular dynamics or Monte Carlo simulation to simulate the evolution of the predicted 3D structure over time; or performing a density functional theory (DFT) or other quantum mechanical calculation on key regions (e.g., active sites or binding pockets) to ensure the accuracy of bond lengths, bond angles, and so forth. Typically, performing further computational validation of a Isomorphic Labs Limited

[0059] F&R Ref: 53672-0024W01 PCT Application predicted 3D structure can be computationally intensive, and performing further computational validation of all the predicted 3D structures generated by the generative machine learning model may be computationally infeasible. The system can use the quality scores generated by the scoring neural network to prioritize the set of predicted 3D structures and select only a fraction of the set of predicted 3D structures (e.g., <50%, or <10%, or <1%) of the predicted 3D structures for further computational validation, thus reducing consumption of computational resources such as memory and computing power.

[0060] As another example, the system can use the quality scores to determine when to stop generating predicted 3D structures of a molecular system using the generative machine learning model. For example, the system can continuously generate predicted 3D structures of the molecular system using the generative machine learning model until the termination criterion is satisfied. The termination criterion can be based at least in part on quality scores. The system can thus cease using the generative machine learning model to generate further predicted 3D structures once the termination criterion is satisfied, reducing the amount of computational resources that would be consumed in generating further predicted 3D structures.

[0061] The system described in this specification can also use quality scores for training the generative machine learning model. Conventionally, training a generative machine learning model to generate 3D structures of molecular systems requires training data that includes a large number of ground truth 3D structures of molecular systems. Obtaining ground truth 3D structures of molecular systems, e.g., through physical experiments (e.g., x-ray crystallography or nuclear magnetic resonance spectroscopy) can be expensive and timeconsuming, and generally there is a scarcity of ground truth 3D structures of molecular systems that are available for use in training.

[0062] The system can train the generative machine learning model to optimize an objective function that is based on quality scores for the training molecular systems. For example, the system can generate quality scores for the predicted 3D structures of the training molecular systems using the scoring neural network. The scoring neural network directly maps from a 3D structure of a molecular structure to a predicted quality score, without requiring knowledge of the actual 3D structure of the molecular structure. Thus the system can train the generative machine learning model on training data for training molecular systems without requiring the ground truth 3D structures for the training molecular systems, increasing the amount of training data available for the generative machine learning model. The system can thus use the scoring neural network to provide a technical solution to a technical problem that Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application arises when training a generative machine learning model to perform a structure prediction task for molecular systems. In particular, the system can use quality scores generated by the scoring neural network for predicted 3D structures generated by the generative machine learning model to provide a training signal for training the generative machine learning model for any molecular system, including for molecular systems where the ground truth 3D structure of the molecular system is unknown.

[0063] In some implementations the system described in this specification can train the generative machine learning model and the scoring neural network on different sets of training data. For example, the system can partition training data that identifies an actual 3D structure of multiple training molecular systems into a first set of training data and a second set of training data. Thus the system can train the scoring neural network using training data that was not used to train the generative machine learning model, which can result in a scoring neural network and a generative machine learning model that are more robust and generalizable. Such an approach can also reduce the number of actual 3D structures that would otherwise be needed.

[0064] In some cases, a molecular system can have multiple possible actual structures. For example, an actual structure of a molecular system can have a range of possible conformations over time. In some implementations, the system can train the scoring model on training data that includes, for each of one or more training molecular systems, a target quality score that accounts for the multiple possible actual structures of the training molecular system. For example, the system can determine the target quality score based on similarity measures between the predicted 3D structure and multiple 3D structures included in a molecular conformational trajectory for the training molecular system. Thus the system can train the scoring model to generate quality scores that more accurately reflect the estimated similarity between a predicted 3D structure of a molecular system and any of multiple possible actual 3D structures of the molecular system.

[0065] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0066] BRIEF DESCRIPTION OF THE DRAWINGS

[0067] FIG. 1 shows an example structure scoring system. Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0068] FIG. 2 is a flow diagram of an example process for generating quality scores for predicted 3D structures.

[0069] FIG. 3 is a flow diagram of an example process for continuously generating predicted 3D structures of a molecular system.

[0070] FIG. 4 is a flow diagram of an example process for determining a target quality score for training a scoring neural network.

[0071] FIG. 5 is a flow diagram of an example process for training a scoring neural network.

[0072] FIG. 6 is a flow diagram of an example process for training a generative machine learning model.

[0073] FIG. 7 is a flow diagram of an example process for training a scoring neural network and a generative machine learning model on training data.

[0074] Like reference numbers and designations in the various drawings indicate like elements.

[0075] DETAILED DESCRIPTION

[0076] FIG. 1 shows an example structure scoring system 100. The structure scoring system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

[0077] The structure scoring system 100 is configured to generate multiple predicted 3D structures 122a-n of a molecular system that includes one or more molecules. For example, as described below, the structure scoring system 100 can generate the predicted 3D structures 122a-n of the molecular system given data identifying the one or more molecules in the molecular system using a generative machine learning model 120.

[0078] Each predicted 3D structure 122a-n defines a structure parameter of one or more atoms in the molecular system that collectively define a structure of the molecular system. For example, a predicted 3D structure can define a spatial arrangement, e.g., a respective predicted 3D spatial location, of each of one or more atoms in the molecular system. As a particular example, a predicted 3D structure can define a respective position of particular atoms in structural units of a molecule included in the molecular system, e.g., alpha carbon atoms in amino acids in a protein included in the molecular system.

[0079] As another example, a predicted 3D structure can define torsion angles of bonds between atoms in amino acids in a protein included in the molecular system. For example, the predicted 3D structure can define backbone torsion angles in the protein. Each torsion Isomorphic Labs Limited

[0080] F&R Ref: 53672-0024W01 PCT Application angle may be represented, e.g., as a 2-D vector. In general the other 3D structures referred to herein can similarly be represented by 3D atom locations, or bond torsion angles.

[0081] The structure scoring system 100 is configured to generate a quality score 142a-n for each predicted 3D structure 122a-n of the molecular system. Each of the quality scores 142a- n estimates a similarity between a predicted 3D structure 122a-n and an actual (e.g. physically stable) 3D structure of the molecular system. Generally, the similarity between a predicted 3D structure 122a-n and an actual 3D structure of the molecular system can be measured, e.g., by a similarity measure that characterizes how closely the structure parameter of one or more atoms in the predicted 3D structure aligns with the actual structure parameter of the one or more atoms in the actual 3D structure. For example, the similarity measure can characterize how closely the position of an atom in the predicted 3D structure conforms with the actual position of the atom in the actual 3D structure. As another example, the similarity measure can characterize how closely the torsion angle of a bond between atoms in the predicted 3D structure aligns with the actual torsion angle of the bond between the atoms in the actual 3D structure.

[0082] To generate the quality score 142a-n for each of the predicted 3D structures 122a-n of the molecular system, the structure scoring system 100 receives data identifying one or more molecules 102 included in the molecular system. The data identifying one or more molecules 102 can include any appropriate data characterizing the one or more molecules in the molecular system. For example, the molecules can include one or more proteins, small molecule ligands, ribonucleic acid (RNA) molecules, or deoxyribonucleic acid (DNA) molecules. In some cases, the molecular system can include multiple molecules, e.g., at least 2 molecules, or at least 5 molecules, or at least 10 molecules.

[0083] Data identifying one or more proteins can include, for each protein, any appropriate data characterizing the protein, e.g., data defining one or more amino acid sequences of the protein, or data defining an MSA (Multiple Sequence Alignment) for the protein, or data characterizing a respective structure of each of one or more “template” proteins, or a combination thereof. A template protein can refer to a protein that is “similar” to the protein, e.g., such that the value of a similarity measure between the template protein and the protein satisfies (e.g., exceeds) a threshold (e.g., 0.8, or 0.9, or 0.99, or any other appropriate threshold). Similarity between a first protein and a second protein can be measured using any appropriate similarity measure, e.g., a sequence identity or percent identity similarity measure between the respective amino acid sequence(s) of the first protein and the second protein. The structure of a template protein can be represented in any appropriate manner, e.g., by a Isomorphic Labs Limited

[0084] F&R Ref: 53672-0024W01 PCT Application contact map, or by data defining a respective 3D spatial position of each atom in the template protein. Optionally, the data can exclude any data that directly defines the 3D structure of the protein, e.g., the 3D spatial locations of the atoms or amino acid residues in a 3D conformation of the protein.

[0085] Data identifying one or more ligands can include any appropriate data characterizing each ligand. For instance, the data can include, for each ligand, a respective textual representation of one or more of a chemical structure of the ligand (e.g., the arrangement of atoms and bonds in the ligand), the atom types in the ligand and their connectivity, the chirality of the bonds in the ligand, or any functional groups (e.g., hydroxyl groups, amino groups, carboxyl groups, and so forth) included in the ligand. The respective textual representation of each ligand can include, e.g., a simplified molecular-input line-entry system (SMILES) string characterizing the ligand. As another example, the data can include, for each ligand, a respective representation of the ligand by way of graph data representing a graph, e.g., where the nodes in the graph represent atoms in the ligand and the edges in the graph represent bonds between atoms in the ligand. Optionally, the data can exclude any data that directly defines the 3D structure of each ligand, e.g., the 3D spatial locations of the atoms in a 3D conformation of the ligand.

[0086] In some examples, the molecules can include one or more nucleic acids such as DNA or RNA. The data identifying the one or more nucleic acids can include a nucleic acid sequence for each nucleic acid.

[0087] The structure scoring system 100 processes a model input 112 that identifies the one or more molecules, identified in the data 102, included in a molecular system using the generative machine learning model 120 to generate multiple predicted 3D structures 122a-n of the molecular system. For example, the structure scoring system 100 can sample from the generative machine learning model 120 n times, when the generative machine learning model 120 is conditioned on the model input 112, to generate the multiple predicted 3D structures 122a-n. In some implementations the model input 112 can be the data identifying one or more molecules 102.

[0088] As an example, the molecular system can include a protein and one or more ligands. Each predicted 3D structure for the molecular system can define a structure of the molecular system where one or more of the ligands are bound to respective binding sites on the protein.

[0089] In some examples, the system 100 can generate the model input 112. For example, for each molecule identified in the data 102, the system 100 can include the data identifying the molecule in the model input 112. As a particular example, when the data 102 identifies Isomorphic Labs Limited

[0090] F&R Ref: 53672-0024WO1 PCT Application one or more proteins, the model input 112 can include a respective amino acid sequence of each of the one or more proteins. When the data 102 identifies one or more nucleic acids, the model input 112 can include a respective nucleic acid sequence of each of the one or more nucleic acids. When the data 102 identifies one or more ligands, the model input 112 can define a respective chemical structure of each of the one or more ligands, e.g., by respective SMILES strings.

[0091] The generative machine learning model 120 is configured to process the model input 112 that identifies the one or more molecules included in a molecular system to generate a predicted 3D structure of the molecular system.

[0092] The generative machine learning model 120 can be implemented as any appropriate type of machine learning model that can perform a 3D structure prediction task, e.g., by processing a model input that identifies one or more molecules included in a molecular system to generate a predicted 3D structure of the molecular system. For instance, the generative machine learning model 120 can be implemented as a neural network model, generative diffusion model, a generative adversarial model, a normalizing flow, or an autoregressive generative model, and so forth. In implementations where generative machine learning model 120 is implemented as a neural network, the generative machine learning model 120 can include any appropriate types of neural network layers (e.g., fully connected layers, message passing layers, convolutional layers, attention layers, recurrent layers, pooling layers, and so forth), in any appropriate number (e.g., 5 layers, or 10 layers, or 50 layers), and connected in any appropriate configuration (e.g., as a directed graph of layers).

[0093] As a particular example, the generative machine learning model 120 can be based on the AlphaFold3 model, as described in Abramson, Josh, et al. "Accurate structure prediction of biomolecular interactions with AlphaFold 3." Nature (2024): 630, 493-500.

[0094] As another particular example, the generative machine learning model 120 can be based on the RF diffusion model, as described in Watson, Joseph L., et al. "De novo design of protein structure and function with RF diffusion." Nature 620.7976 (2023): 1089-1100. As a further particular example, the generative machine learning model 120 can be based on the Boltz-1 model, as described in Wohlwend et al., “Boltz-1: Democratizing Biomolecular Interaction Modeling”, bioRxiv, 2024.

[0095] For each of the predicted 3D structures 122a-n, the structure scoring system 100 processes a scoring model input that includes the predicted 3D structure of the molecular system and data identifying the one or more molecules 102 included in the molecular system Isomorphic Labs Limited

[0096] F&R Ref: 53672-0024WO1 PCT Application using a scoring neural network 140 to generate a corresponding quality score for the predicted 3D structure.

[0097] In some examples, the system 100 can generate scoring model inputs 132a-n. For example, for each scoring model input 132a-n, the system 100 can include data identifying the one or more molecules 102 included in the molecular system and the corresponding predicted 3D structure 122a-n of the molecular system.

[0098] The scoring neural network 140 is configured to process a scoring model input 132a-n that includes data identifying one or more molecules included in a molecular system and a predicted 3D structure for the molecular system to generate a quality score 142a-n for the predicted 3D structure.

[0099] The scoring neural network 140 can have any appropriate neural network architecture that enables the scoring neural network 140 to perform its described functions. In particular, the scoring neural network 140 can include any appropriate types of neural network layers (e.g., fully connected layers, message passing layers, convolutional layers, attention layers, recurrent layers, pooling layers, and so forth), in any appropriate number (e.g., 5 layers, or 10 layers, or 50 layers), and connected in any appropriate configuration (e.g., as a directed graph of layers).

[0100] As a particular example, the scoring neural network 140 can be implemented as a graph neural network that includes multiple message passing neural network layers. More specifically, the scoring neural network can process a graph representation of the predicted structure of the molecular system. Each node in the graph can represent a corresponding atom in the molecular system. Each edge can connect a respective pair of nodes in the graph and can represent, e.g., that the pair of atoms represented by the pair of nodes are separated by less than a threshold distance (e.g., 8 Angstroms) or that the pair of atoms represented by the pair of nodes are chemically bonded. Each node in the graph can be associated with a set of features that represents at least the predicted 3D spatial position of the atom, i.e., according to the predicted 3D structure generated by the generative machine learning model.

[0101] The scoring neural network can include an encoding subnetwork, one or more message passing neural network layers, and a decoding subnetwork. The encoding subnetwork can generate a respective embedding associated with each node in the graph by processing the set of features associated with the node (some example features are given below). The message passing layers can iteratively update the embeddings associated with the nodes in the graph using message passing operations that share information between the nodes. The decoding subnetwork can process the embeddings associated with the nodes to Isomorphic Labs Limited

[0102] F&R Ref: 53672-0024W01 PCT Application generate the output quality score. For instance, the decoding subnetwork can pool (e.g., average pool or max pool) the embeddings associated with the nodes to generate a combined embedding, and then process the combined embedding using a fully connected layer to generate the quality score for the predicted structure.

[0103] In another example, the scoring neural network can be implemented as a neural network that includes an encoding subnetwork, one or more self-attention layers, and a decoding subnetwork. In this example, the network input to the scoring neural network can include a respective set of features associated with each atom in the molecular system. The set of features associated with an atom includes at least the predicted 3D spatial position of the atom, i.e., according to the predicted 3D structure generated by the generative machine learning model. The set of features associated with an atom can optionally include other features that identify, e.g., the elemental type of the atom, bonds associated with the atom, and so forth. The encoding subnetwork of the scoring neural network can generate a respective embedding associated with each atom in the molecular system by processing the set of features associated with the atom using one or more neural network layers. The selfattention layers can update the embeddings of the atoms using self-attention operations. The decoding subnetwork can process the embeddings associated with the atoms to generate the output quality score. For instance, the decoding subnetwork can pool (e.g., average pool or max pool) the embeddings associated with the atoms to generate a combined embedding, and then process the combined embedding using a fully connected layer to generate the quality score for the predicted structure.

[0104] The structure scoring system 100 can receive the data identifying one or more molecules 102 of a molecular system from any appropriate source, e.g., from a user or from another system, by way of an appropriate interface, e.g., an application programming interface (API) or a user interface (e.g., a graphical user interface). As an example, the other system can be an upstream system that generates data identifying candidate molecular systems, e.g., in a drug discovery workflow.

[0105] After generating the predicted 3D structures 122a-n of the molecular system and the quality scores 142a-n for the predicted 3D structures of the molecular system, the structure scoring system 100 can, e.g., store data defining the predicted 3D structures 122a-n of the molecular system, the quality scores 142a-n for the predicted 3D structures of the molecular system, or both, in a memory. As another example, the structure scoring system 100 can transmit data defining the predicted 3D structures 122a-n of the molecular system, the quality scores 142a-n for the predicted 3D structures of the molecular system, or both, over a data Isomorphic Labs Limited

[0106] F&R Ref: 53672-0024WO1 PCT Application communication network. As another example, the structure scoring system 100 can provide data defining the predicted 3D structures 122a-n of the molecular system, the quality scores 142a-n for the predicted 3D structures of the molecular system, or both, directly to a system that performs downstream processing based on the predicted 3D structures 122a-n of the molecular system, the quality scores 142a-n for the predicted 3D structures of the molecular system, or both.

[0107] The quality scores 142a-n generated by the structure scoring system 100 can be used in any of a variety of possible downstream applications. A few examples of downstream applications that process quality scores 142a-n generated by the structure scoring system 100 are described next.

[0108] In one example, the system 100 can continuously generate predicted 3D structures 132a-n for a molecular system until a termination criterion is satisfied. Continuously generating predicted 3D structures until a termination criterion is satisfied is described below in further detail with reference to FIG. 3.

[0109] In another example, a training system of the system 100 or another training system can train the generative machine learning model 120 to optimize an objective function that is based on quality scores generated for predicted 3D structures generated by the generative machine learning model 120. Training the generative machine learning model 120 is described in further detail below with reference to FIG. 6.

[0110] In another example, the system 100 or another system can determine a ranking of the predicted 3D structures 132a-n based on the quality scores 142a-n. The system can select one of the predicted 3D structures 132a-n based at least in part on the ranking.

[0111] The generative machine learning model and methods described herein can be used to obtain a ligand (i.e. a ligand molecule or ligand molecule complex) such as a drug or a ligand of an industrial enzyme. In general, the drug or industrial enzyme may be a molecule that inhibits or catalyzes a chemical or biochemical process. The molecule complex may include, for example, a protein, a ribozyme (ribonucleic acid enzyme), or a deoxyribozyme (deoxyribonucleic acid enzyme). For example, a method of obtaining a ligand may include obtaining a target amino acid sequence, in particular the amino acid sequence of a target protein molecule (or target protein molecule complex), e.g. a drug target, and processing an input based on the target amino acid sequence using the generative machine learning model to determine a (tertiary) structure of the target protein molecule, e.g., a predicted structure of a complex comprising the target protein molecule and the candidate ligand. For example, the system can determine the predicted structure by selecting one of the predicted 3D structures Isomorphic Labs Limited

[0112] F&R Ref: 53672-0024W01 PCT Application

[0113] 132a-n based at least in part on the ranking. The method may then include evaluating an interaction of one or more candidate ligands with the target protein molecule. The method may further include selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating of the interaction. Predicting the structure of a complex comprising the target protein molecule and a candidate ligand may preferably account for changes in the structure of molecule caused by binding of the candidate ligand and / or changes in the structure of the candidate ligand. Evaluating the interaction of the one or more candidate ligands with the target protein molecule may, for example, comprise determining a binding energy or an equilibrium constant for the formation of the complex.

[0114] In some implementations, evaluating the interaction may include evaluating binding of the candidate ligand with the structure of the target protein molecule. For example, evaluating the interaction may include identifying a ligand that binds with sufficient affinity for a biological effect. In some other implementations, evaluating the interaction may include evaluating an association of the candidate ligand with the target protein molecule which has an effect on a function of the target protein molecule, e.g., an enzyme. The evaluating may include evaluating an affinity between the candidate ligand and the target protein molecule or complex, or evaluating a selectivity of the interaction. The candidate ligand(s) may be selected according to which have the highest affinity. Evaluating the interaction may additionally comprise simulating a dynamical behavior of the ligand and target protein molecule, such as through molecular dynamics simulations, which may allow kinetic aspects of the interaction to be taken into account.

[0115] The candidate ligand(s) may be derived from a database of candidate ligands, and / or may be derived by modifying ligands in a database of candidate ligands, e.g., by modifying a structure or amino acid sequence of a candidate ligand, and / or may be derived by stepwise or iterative assembly / optimization of a candidate ligand. The candidate ligand(s) may alternately or additionally include one or more candidate ligands generated using a generative model conditioned on (the structure of) the target protein molecule or part of the target protein molecule, e.g. a structure of a binding site or other part of the target protein molecule.

[0116] The evaluation of the interaction of a candidate ligand with the target protein molecule may be performed using a computer-aided approach in which graphical models of the candidate ligand and target protein molecule structure are displayed for usermanipulation, and / or the evaluation may be performed partially or completely automatically, for example using standard molecular (e.g. protein-ligand) docking software. In some implementations the evaluation may include determining an interaction score for the Isomorphic Labs Limited

[0117] F&R Ref: 53672-0024WO1 PCT Application candidate ligand, where the interaction score includes a measure of an interaction between the candidate ligand and the target protein molecule. The interaction score may be dependent upon a strength and / or specificity of the interaction, e.g., a score dependent on binding free energy. A candidate ligand may be selected dependent upon its score.

[0118] In some implementations the target protein molecule includes a receptor or enzyme and the ligand is an agonist or antagonist of the receptor or enzyme. In some implementations the method may be used to identify the structure of a cell surface marker. This may then be used to identify a ligand, e.g., an antibody or aptamer or a label such as a fluorescent label, which binds to the cell surface marker. This may be used to identify and / or treat cancerous cells.

[0119] In some implementations the ligand is a drug and the interaction of each of a plurality of target protein molecules with each of the candidate ligands is evaluated. Then one or more of the candidate ligands may be selected either to obtain a ligand that (functionally) interacts with each of the target protein molecules, or to obtain a ligand that (functionally) interacts with only one of the target protein molecules. For example in some implementations it may be desirable to obtain a drug that is effective against multiple drug targets. Also or instead it may be desirable to screen a drug for off-target effects. For example in agriculture it can be useful to determine that a drug designed for use with one plant species does not interact with another, different plant species and / or an animal species.

[0120] In some implementations the ligand is a drug and the predicted structure of a target protein that is a protein complex, e.g. a dimer or multimer, is determined. Evaluating the interaction of the one or more candidate ligands with the target protein may then comprise identifying a candidate ligand that interacts with the protein complex, and that might therefore be expected to affect the formation or stability of the complex. This could afterwards be confirmed by experimental screening. Thus such a process may be used to identify a drug which is able to disrupt a protein complex or inhibit formation of the complex. Some diseases, e.g. neuro degenerative diseases such as dementia, are caused by protein aggregation. The method may thus be used to identify a ligand that is a drug to treat such a disease.

[0121] In some implementations the candidate ligand(s) may include small molecule complex ligands (i.e. small molecule ligands bound to a partner in a complex), e.g., organic compounds with a molecular weight of <900 daltons. In some other implementations the candidate ligand(s) may include polypeptide ligands, i.e., defined by an amino acid sequence. Isomorphic Labs Limited

[0122] F&R Ref: 53672-0024W01 PCT Application

[0123] In another aspect there is provided a method of using the generative machine learning model to obtain a ligand, which may be a biological molecule, such as a polypeptide, polynucleotide, or polynucleoside ligand (e.g., the molecule or its amino acid or nucleotide sequence). For example, the method may include obtaining data defining one or more candidate ligands, e.g. an amino acid sequence of one or more candidate polypeptide or polynucleotide ligands. The method may include selecting a target molecule to which the ligand is to bind. The method may further include, for each of the candidate ligands, using the generative machine learning model to determine (tertiary) structure of a complex comprising the candidate ligand and the target protein molecule. The method may further include obtaining a target protein structure of a target molecule, in silico and / or by physical investigation. The method may comprise evaluating an interaction between each of the one or more candidate ligands and the target protein molecule, e.g. by evaluating an interaction between the predicted structure of the candidate ligand and the structure of the target molecule, or using the predicted structure of the complex comprising the candidate ligand and the target protein molecule. The method may further include selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluation.

[0124] As before, evaluating the interaction may include evaluating binding of the candidate ligand with the structure of the target protein molecule, e.g., identifying a ligand that binds with sufficient affinity for a biological effect, and / or evaluating an association of the candidate ligand with the structure of the target protein molecule which has an effect on a function of the target protein molecule, e.g., an enzyme, and / or evaluating an affinity between the candidate ligand and the structure of the target protein molecule, or evaluating a selectivity of the interaction. In some implementations the ligand may be an aptamer. Again the candidate ligand(s) may be selected according to which have the highest affinity.

[0125] As before, the selected ligand (e.g. selected polypeptide or polynucleotide ligand) may comprise a receptor or enzyme and the ligand may be an agonist or antagonist of the receptor or enzyme. In some implementations the ligand may comprises an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, for example a virus, in particular a virus coat protein, or a protein expressed on a cancer cell. In these implementations the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect. For example, the antibody or aptamer may bind to the target and act as an agonist for a particular receptor; alternatively, the antibody or aptamer may prevent binding of another ligand to the target, and hence prevent activation of a relevant biological pathway. Isomorphic Labs Limited

[0126] F&R Ref: 53672-0024W01 PCT Application

[0127] Implementations of the method may further include synthesizing the ligand, i.e., making, the small molecule, polynucleotide or polypeptide ligand. The ligand may be synthesized by any conventional chemical techniques and / or may already be available, e.g., may be from a compound library or may have been synthesized using combinatorial chemistry.

[0128] The method may further include testing the ligand for biological activity in vitro and / or in vivo. For example the ligand may be tested for ADME (absorption, distribution, metabolism, excretion) and / or toxicological properties, to screen out unsuitable ligands. The testing may include, e.g., bringing the candidate small molecule, polypeptide or polynucleotide ligand into contact with the target protein molecule and measuring a change in expression or activity of the target molecule.

[0129] In some implementations a candidate (e.g. polypeptide or polynucleotide) ligand may include: an isolated antibody or aptamer, a fragment of an isolated antibody or aptamer, a single variable domain antibody, a bi- or multi-specific antibody, a multivalent antibody, a dual variable domain antibody, an immuno-conjugate, a fibronectin molecule, an adnectin, an DARPin, an avimer, an affibody, an anticalin, an affilin, a protein epitope mimetic or combinations thereof. A candidate (polypeptide) ligand may include an antibody with a mutated or chemically modified amino acid Fc region, e.g., which prevents or decreases ADCC (antibody-dependent cellular cytotoxicity) activity and / or increases half-life when compared with a wild type Fc region. Candidate (polypeptide or polynucleotide) ligands may include antibodies with different CDRs (Complementarity -Determining Regions).

[0130] As another example, the target protein molecule may be an enzyme comprising a CRISPR associated protein and the ligand may comprise a guide RNA molecule. The method may be performed to identify a combination of guide RNA molecule and CRISPR associated protein, in particular one that operates efficiently to edit genes. Such a method can involve determining a predicted structure of the enzyme, e.g. as described above, in particular to check that the enzyme shape and the guide RNA shape fit and work together effectively. The guide RNA may have a part with a defined 3D structure, e.g. it may be a single guide RNA (sgRNA), incorporating a guide sequence and a tracrRNA sequence.

[0131] The generative machine learning model described herein can also be used to obtain a diagnostic antibody or aptamer marker of a disease. There is also provided a method that comprises selecting a target protein molecule that is to be recognized by the antibody or aptamer marker, and for each of one or more candidate antibodies or aptamers e.g. as described above, uses the generative machine learning model to determine a predicted Isomorphic Labs Limited

[0132] F&R Ref: 53672-0024WO1 PCT Application structure of a complex comprising the target protein molecule and the candidate antibody or aptamer. The method may also involve, evaluating an interaction between each of the one or more candidate antibodies or aptamers and the target protein molecule, and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating, e.g. selecting one or more candidate antibodies or aptamers that have the highest affinity to the target protein. The method may include making the diagnostic antibody or aptamer marker. The diagnostic antibody or aptamer marker may be used to diagnose a disease by detecting whether it binds to the target protein molecule in a sample obtained from a patient, e.g. a sample of bodily fluid. As described above, a corresponding technique can be used to obtain a therapeutic antibody or aptamer (e.g. polypeptide or polynucleotide ligand).

[0133] In some other aspects a computer-implemented method as described above or herein may be used to identify active / binding / blocking sites on a target protein from its amino acid sequence.

[0134] The generative machine learning model and methods described herein can also be used to determine the structure of a molecule or molecule complex. For example, an experimental technique may be applied to a physical sample comprising a molecule complex (i.e. the physical sample of the molecule complex is “interrogated” using the experimental technique) to measure experiment signals dependent on a structure of the molecule complex. In some implementations, the experimental technique may be a scattering technique or a spectroscopic technique. The experimental technique may, for example, comprise one or more of: x-ray crystallography, nuclear magnetic resonance (NMR), and electron microscopy (e.g. cryogenic electron microscopy, cryo-EM). A generative machine learning model may be used to determine a predicted structure of the molecule complex. The experiment signals may then be compared with corresponding simulated signals generated using the predicted structure of the molecule complex. For example, the predicted structure of the molecule complex can be used to generate predicted x-ray diffraction patterns (e.g. from an electron density distribution determined using the predicted structure) that can be compared with experimentally measured x-ray diffraction patterns. For example, the experiment signals may comprise NMR signals, electron microscope images, or x-ray diffraction patterns; or signals derived therefrom.

[0135] The predicted structure of the molecule complex may be determined by adjusting the predicted structure of the molecule complex dependent upon a result of the comparison. The method may be performed iteratively, wherein for each of one or more iterations, after the Isomorphic Labs Limited

[0136] F&R Ref: 53672-0024WO1 PCT Application predicted structure has been adjusted, the predicted signals may be generated for the adjusted structure, and the comparing and adjusting performed again to refine the predicted structure. Alternatively (or additionally), a plurality of different possible structures of the molecule complex can be predicted and the expected experiment signals from each compared with the actual experiment signals to determine a match, e.g. a best or most likely match, that can be taken as the determined structure of the molecule complex.

[0137] In general an aptamer as described above may comprise DNA or RNA. An enzyme as described above may comprise a protein or a DNA enzyme, e.g. a deoxyribozyme or “DNAzyme”, or an RNA enzyme, e.g. a ribozyme. As well as the applications described above such enzymes can also be used for biosensors of many types, e.g. DNAzymes and aptamers can be useful for detecting metal ions, and in general aptamer targets can include small molecule complexes, proteins, and cells. Aptamers have many uses including, e.g. as probes in assays, as biosensors (e.g. they can detectably change shape when binding to a target), to modulate the activity of biomolecule complexes, and to provide a controlled release mechanism. The techniques described herein can be used to design, and then make, such aptamers, sometimes referred to as chemical antibodies.

[0138] FIG. 2 is a flow diagram of an example process 200 for generating quality scores for predicted 3D structures. For convenience, the process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a structure scoring system, e.g., the structure scoring system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 200.

[0139] The system receives data identifying one or more molecules included in a molecular system (202). The one or more molecules can include, for example proteins, small molecule ligands, or nucleic acids. In some examples, the system can receive the data identifying the one or more molecules included in the molecular system from any appropriate source, e.g., from a user or from an upstream system that generates data identifying candidate molecular systems, e.g., in a drug discovery workflow. The system can receive the data identifying the one or more molecules included in the molecular system by way of an appropriate interface, e.g., an application programming interface (API) or a user interface (e.g., a graphical user interface).

[0140] The system processes a model input that identifies the one or more molecules to generate multiple predicted 3D structures of the molecular system (step 204). The multiple predicted 3D structures can include, e.g., at least 3 predicted 3D structures, or at least 100 Isomorphic Labs Limited

[0141] F&R Ref: 53672-0024WO1 PCT Application predicted 3D structures, or at least 1000 predicted 3D structures. At least some or all of the multiple predicted 3D structures can be unique, i.e., different from one another.

[0142] For example, the system processes the model input using a generative machine learning model to generate, as model outputs of the generative machine learning model, the multiple predicted 3D structures of the molecular system. The system can sample from the generative machine learning model when conditioned on the model input to generate the model outputs. An example generative machine learning model is described above with reference to FIG. 1. Training the generative machine learning model is described below with reference to FIG. 6 and FIG. 7.

[0143] The system generates, for each of the multiple predicted 3D structures of the molecular system, a quality score (step 206). For each predicted 3D structure, the quality score estimates a similarity between (i) the predicted 3D structure of the molecular system, and (ii) an actual 3D structure of the molecular system.

[0144] For example, for each of the multiple 3D structures, the system can process a scoring model input that includes (i) data identifying the one or more molecules included in the molecular system, and (ii) the predicted 3D structure of the molecular system, using a scoring neural network to generate the quality score for predicted 3D structure of the molecular system. An example scoring neural network is described above with reference to FIG. 1. Training the scoring neural network is described in further detail below with reference to FIG. 4, FIG. 5, and FIG. 7.

[0145] The system outputs the multiple predicted 3D structures of the molecular system and the quality scores for the multiple predicted 3D structures of the molecular system (step 208). For example, the system can store data defining the predicted 3D structures and the quality scores in a memory. As another example, the system can provide data defining the predicted 3D structures and the quality scores to a system that performs downstream processing based on the predicted 3D structures, the quality scores, or both. As another example, the system can render the predicted 3D structures of the molecular system. The system can present the rendered predicted 3D structures and the quality scores for the predicted 3D structures for display to a user, e.g., by way of a user interface.

[0146] In some implementations, the system continuously generates predicted 3D structures of the molecular system until a termination criterion is satisfied. For example, as described below with reference to FIG. 3, the termination criterion can be based at least in part on quality scores, and the system can perform steps 204-206 until the termination criterion is satisfied. Isomorphic Labs Limited

[0147] F&R Ref: 53672-0024WO1 PCT Application

[0148] FIG. 3 is a flow diagram of an example process 300 for continuously generating predicted 3D structures of a molecular system. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a structure scoring system, e.g., the structure scoring system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 300.

[0149] The system can perform the steps 302-304 continuously for a molecular system until a termination criterion is satisfied. For example, the termination criterion can be based at least in part on quality scores generated by the scoring neural network for predicted 3D structures generated by the generative machine learning model.

[0150] The system generates one or more predicted 3D structures of the molecular system (step 302). For example, the system can generate the one or more predicted 3D structures of the molecular system by sampling from the generative machine learning model when conditioned on a model input that identifies the one or more molecules included in the molecular system to generate a model output, as described above with reference to step 204 of FIG. 2.

[0151] The system generates one or more quality scores for the one or more predicted 3D structures (step 304). For example, the system can generate the quality score for each of the one or more predicted 3D structures using the scoring neural network as described above with reference to step 206 of FIG. 2.

[0152] The system determines whether the termination criterion is satisfied (step 306). In some examples, the termination criterion can be that the generative machine learning model has generated at least a threshold number of predicted 3D structures that each have a respective quality score that satisfies a quality score threshold. As a particular example, the threshold number of predicted 3D structures can be one.

[0153] If the system determines that the termination criterion is not satisfied, the system can return to step 302 to generate one or more additional predicted 3D structures of the molecular system.

[0154] If the system determines that the termination criterion is satisfied, the system can cease generating additional predicted 3D structures of the molecular system. In some examples, the system can output predicted 3D structures of the molecular system and quality scores for the predicted 3D structures (step 308). As an example, the system can output the predicted 3D structures of the molecular system that each have a respective quality score that Isomorphic Labs Limited

[0155] F&R Ref: 53672-0024WO1 PCT Application satisfies the quality score threshold, and the quality score for each of the predicted 3D structures that each have a respective quality score that satisfies the quality score threshold.

[0156] Thus, in response to determining that the generative machine learning model has generated the threshold number of predicted 3D structures that each have a quality score that satisfies a quality score threshold, the system ceases generating predicted 3D structures of the molecular system, reducing the amount of computational resources that would be consumed in continuing to generate predicted 3D structures.

[0157] FIG. 4 is a flow diagram of an example process 400 for determining a target quality score for training a scoring neural network. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a structure scoring system, e.g., the structure scoring system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 400.

[0158] The system can train the scoring neural network on a set of training examples that each correspond to a respective training molecular system. Each training example can include (a) a training input that includes data that identifies one or more molecules included in the training molecular system and a predicted 3D structure of the training molecular system that is generated by the generative machine learning model, and (b) a target quality score that measures a similarity between the predicted 3D structure of the training molecular system and an actual 3D structure of the training molecular system. In some examples, the system can obtain the predicted 3D structure of the training molecular system by processing the data that identifies the one or more molecules included in the training molecular system using the generative machine learning model. In some examples, an actual 3D structure can be used to obtain the input for the generative machine learning model that generates the predicted 3D structure.

[0159] The system can perform steps 402-408 for each of the training examples to generate the target quality score for the training example.

[0160] The system obtains data defining a baseline (initial) 3D structure of the training molecular system (step 402). For example, the data defining the baseline 3D structure of the training molecular system can have been derived through a physical experiment (e.g., x-ray crystallography or nuclear magnetic resonance spectroscopy or cryo-EM) involving the one or more molecules of the training molecular system, or from the output of another computational system or technique (such as AlphaFold3, RF diffusion, or Boltz-1), or from a Isomorphic Labs Limited

[0161] F&R Ref: 53672-0024W01 PCT Application database (examples include ChEMBL-3D, The Protein Data Bank, wwpdb.org, the Cambridge Structural Database; there are many other examples).

[0162] The system performs a molecular dynamics or Monte Carlo simulation to generate a molecular conformational trajectory (step 404). For example, the molecular dynamics simulation models the movement of atoms and molecules in the molecular system over a period of time (the molecular conformational trajectory can be a time series of 3D structures of the molecular system that represents a dynamic behavior of the molecular system). For example, the molecular dynamics or Monte Carlo simulation predicts trajectories, e.g., position and movement, starting from the baseline 3D structure, of the atoms and molecules based on the forces acting on each atom and molecule, and interactions between atoms and molecules. The molecular conformational trajectory generated by performing the molecular dynamics or Monte Carlo simulation represents the arrangement of the atoms and molecules included in the molecular system at different points in time over the period of time of the molecular dynamics simulation. For example, the molecular conformational trajectory includes (i) the baseline 3D structure of the training molecular system, and (ii) multiple additional 3D structures of the molecular system. Each additional 3D structure can be an actual 3D structure of the molecular system. There are many suitable systems for performing such a simulation; a few examples are AMBER (Assisted Model Building with Energy Refinement), CHARMM (Chemistry at Harvard Macromolecular Mechanics), or OPLS (Optimized Potentials for Liquid Simulations); some can incorporate quantum mechanical effects, such as through the use of path integral molecular dynamics (PIMD), ring polymer molecular dynamics (RPMD), or centroid molecular dynamics (CMD).

[0163] The system determines, for each of the multiple 3D structures included in the molecular conformational trajectory, a similarity measure (step 406). For example, for each of the multiple 3D structures that are included in the molecular conformational trajectory, the system can determine the similarity measure between: (i) the predicted 3D structure, and (ii) the 3D structure that is included in the molecular conformational trajectory. The similarity measure can be, e.g., a root-mean-square deviation (RMSD) measure, or a global distance test (GDT) measure, or a template modeling score (TM-score), and so forth.

[0164] The system determines the target quality score based on the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory (step 408). For example, the system can determine the target quality score as a minimum of the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory. As a particular Isomorphic Labs Limited

[0165] F&R Ref: 53672-0024W01 PCT Application example, the target quality score can represent an estimated similarity between the predicted 3D structure and the 3D structure included in the molecular conformation trajectory that has the lowest similarity measure, i.e., is most similar to, the predicted 3D structure.

[0166] The target quality score for a predicted 3D structure can indicate a high similarity between the predicted 3D structure and any 3D structure included in the molecular conformational trajectory, rather than a single 3D structure for the molecular system, e.g., the baseline 3D structure. Thus, the system can generate target quality scores for the training examples that accounts for the multiple 3D structures included in the molecular conformational trajectory.

[0167] The system can then train the scoring neural network, by a machine learning training technique, to optimize an objective function that, for each training example, measures an error between: (i) the target quality score specified by the training example, and (ii) a predicted quality score generated by processing the training input of the training example using the scoring neural network. An example process for training the scoring neural network is described below with reference to FIG. 5.

[0168] FIG. 5 is a flow diagram of an example process 500 for training a scoring neural network. For convenience, the process 500 will be described as being performed by a system of one or more computers located in one or more locations. For example, a structure scoring system, e.g., the structure scoring system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 500.

[0169] The system can train the scoring neural network over a sequence of training iterations.

[0170] At each training iteration, the system can receive one or more training examples for the training iteration (step 502). For example, as described with reference to FIG. 4, each training example can include (a) a training input that includes data that identifies one or more molecules included in a training molecular system and a predicted 3D structure of the training molecular system that is generated by the generative machine learning model, and (b) a target quality score that measures a similarity between the predicted 3D structure of the training molecular system and an actual 3D structure of the training molecular system.

[0171] The system can generate predicted quality scores for the training iteration (step 504). For example, for each training example for the training iteration, the system can process the training input using the scoring neural network to generate a predicted quality score for the training example.

[0172] The system can update parameters of the scoring neural network (step 506). For example, the system can determine gradients of an objective function that depends on the Isomorphic Labs Limited

[0173] F&R Ref: 53672-0024WO1 PCT Application predicted quality scores and can use the gradients to update parameter values of the scoring neural network. As a particular example, the objective function can, for each training example, measure an error (e.g., an absolute error, or a squared error) between: (i) the target quality score specified by the training example, and (ii) a predicted quality score generated in step 504 for the training example.

[0174] The system can determine whether training is complete (step 508). If the system determines that training is not complete, the system can continue to a next training iteration (e.g., return to step 502). The system can determine whether training is complete using any of a variety of criteria. For example, the system can determine that training is complete after a pre-determined number of training iterations. As another example, the system can determine that training is complete when a value of the objective function for the training iteration falls below a pre-determined threshold. As another example, the system can determine that training is complete when a difference between the value of the objective function for the current training iteration and a value of the objective function for a previous training iteration falls below a pre-determined threshold.

[0175] When the system determines that training is complete, the system can return the trained scoring neural network (step 510).

[0176] FIG. 6 is a flow diagram of an example process 600 for training a generative machine learning model. For convenience, the process 600 will be described as being performed by a system of one or more computers located in one or more locations. For example, a structure scoring system, e.g., the structure scoring system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 600.

[0177] The system can train, or further train, the generative machine learning model to optimize an objective function that is based on (e.g. a monotonic function of) quality scores (step 602), e.g. any loss function that decreases as quality score increases (where a higher quality score is better). The system can generate the quality scores using the scoring neural network for predicted 3D structures generated by the generative machine learning model.

[0178] The system can train the generative machine learning model over a sequence of training iterations.

[0179] For example, for the training iteration, the system can generate predicted 3D structures by processing training model inputs that each identify one or more molecules included in a training molecular system. The system can generate, for each of the predicted 3D structures, a quality score. For example, for each predicted 3D structure, the system can process the predicted 3D structure for the training molecular system and data identifying the Isomorphic Labs Limited

[0180] F&R Ref: 53672-0024W01 PCT Application one or more molecules included in the training molecular system using the scoring neural network to generate the quality score for the predicted 3D structure.

[0181] For one or more of the predicted 3D structures generated by the generative machine learning model, the system can determine gradients, with respect to a set of generative machine learning model parameters of the generative machine learning model, of an objective function (step 604). For example, the system can determine the gradients using backpropagation. The objective function can be based on the quality score generated by the scoring neural network for the predicted 3D structure of the molecular system.

[0182] For one or more of the predicted 3D structures generated by the generative machine learning model, the system can update current values of the set of generative machine learning model parameters using the gradients (step 606).

[0183] The system can determine whether training is complete. If the system determines that training is not complete, the system can continue to a next training iteration (e.g., return to step 604 for one or more other predicted 3D structures generated by the generative machine learning model). The system can determine whether training is complete using any of a variety of criteria. For example, the system can determine that training is complete after a pre-determined number of training iterations. As another example, the system can determine that training is complete when a value of the objective function for the training iteration falls below a pre-determined threshold. As another example, the system can determine that training is complete when a difference between the value of the objective function for the current training iteration and a value of the objective function for a previous training iteration falls below a pre-determined threshold.

[0184] When the system determines that training is complete, the system can return the trained generative machine learning model.

[0185] Thus the system can train the generative machine learning model to generate 3D structures for molecular systems that have quality scores that indicate a higher similarity to actual 3D structures for the molecular systems, e.g., that are predicted to be more similar to actual 3D structures for the molecular systems. The system can train the generative machine learning model using training molecular systems without requiring ground truth 3D structures of the training molecular systems, which can be expensive or time-consuming to obtain. The system thus increases the amount of training data available for the generative machine learning model, improving the performance of the generative machine learning model.

[0186] FIG. 7 is a flow diagram of an example process 700 for training a scoring neural network and a generative machine learning model on training data. For convenience, the Isomorphic Labs Limited

[0187] F&R Ref: 53672-0024W01 PCT Application process 700 will be described as being performed by a system of one or more computers located in one or more locations. For example, a structure scoring system, e.g., the structure scoring system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 700.

[0188] The system obtains training data that, for each of multiple training molecular systems, identifies an actual 3D structure of the molecular system (step 702). For example, the training data can include, for each training molecular system, data identifying the one or more molecules of the molecular system, and data defining an actual 3D structure of the molecular system.

[0189] The system partitions the multiple training molecular systems into a first set of training data and a second set of training data (step 704). The first set of training data can be for training the generative machine learning model, and the second set of training data can be for training the scoring neural network. For example, the system can partition the training molecular systems so that there is no overlap between the first set of training data for training the generative machine learning model and the second set of training data for training the scoring neural network.

[0190] The system trains the generative machine learning model on the first set of training data (step 706). For example, the system can train the generative machine learning model using training examples derived from the first set of training data, where each training example includes a training input that includes identifying one or more molecules of a training molecular system, and a training output that includes data defining the actual 3D structure of the training molecular system.

[0191] After training the generative machine learning model on the first set of training data, the system trains the scoring neural network on the second set of training data (step 708). For example, the system can train the scoring neural network using training examples derived from the second set of training data.

[0192] For example, the system can generate predicted 3D structures of the training molecular systems of the second set of training data using the trained generative machine learning model. The system can train the scoring neural network using the predicted 3D structures generated by the trained generative machine learning model of the training molecular systems of the second set of training data. For example, for each training molecular system of the second set of training data, the system can generate a training example that includes a training input that includes data that identifies the one or more molecules included in the training molecular system and a predicted 3D structure of the Isomorphic Labs Limited

[0193] F&R Ref: 53672-0024W01 PCT Application training molecular system, and a target quality score for the predicted 3D structure of the training molecular system. An example process for generating the target quality score is described above with reference to FIG. 4.

[0194] The system can thus train the generative machine learning model and scoring neural network using different sets of training data, resulting in a more robust and generalizable scoring neural network and generative machine learning model at inference. For example, by training the scoring neural network using training data that was not used during training of the generative machine learning model, the scoring neural network can be more robust and generalizable at inference.

[0195] This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0196] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0197] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA Isomorphic Labs Limited

[0198] F&R Ref: 53672-0024W01 PCT Application

[0199] (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0200] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0201] In this specification the term “engine” is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0202] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0203] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented Isomorphic Labs Limited

[0204] F&R Ref: 53672-0024WO1 PCT Application by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0205] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0206] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0207] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and computeintensive parts of machine learning training or production, i.e., inference, workloads.

[0208] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework.

[0209] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end Isomorphic Labs Limited

[0210] F&R Ref: 53672-0024W01 PCT Application component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0211] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0212] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0213] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0214] This specification also provides the subject-matter of the following numbered clauses:

[0215] 1. A method performed by one or more computers, the method comprising: receiving data identifying one or more molecules included in a molecular system; processing a model input that identifies the one or more molecules included in the molecular system using a generative machine learning model to generate, as model outputs of the generative machine learning model, a plurality of predicted 3D structures of the molecular system; generating, for each of the plurality of predicted 3D structures of the molecular system, a quality score that estimates a similarity between: (i) the predicted 3D structure of the molecular system, and (ii) an actual 3D structure of the molecular system, comprising, for each of the plurality of predicted 3D structures of the molecular system: processing a scoring model input that comprises: (i) data identifying the one or more molecules included in the molecular system, and (ii) the predicted 3D structure of the molecular system, using a scoring neural network to generate the quality score for the predicted 3D structure of the molecular system; and outputting the plurality of predicted 3D structures of the molecular system and the quality scores for the plurality of predicted 3D structures of the molecular system.

[0216] 2. The method of clause 1, wherein the scoring neural network has been trained on a set of training examples that each correspond to a respective training molecular system and include:

[0217] (a) a training input that comprises data that identifies one or more molecules included in the training molecular system and a predicted 3D structure of the training molecular system that is generated by the generative machine learning model; and

[0218] (b) a target quality score that measures a similarity between the predicted 3D structure of the training molecular system and an actual 3D structure of the training molecular system.

[0219] 3. The method of clause 2, further comprising, for each training example, generating the target quality score for the training example by performing operations comprising: obtaining data defining a baseline 3D structure of the training molecular system; Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application performing a molecular dynamics or Monte Carlo simulation to generate a molecular conformational trajectory that includes: (i) the baseline 3D structure of the training molecular system, and (ii) a plurality of additional 3D structures of the molecular system; determining, for each a plurality of 3D structures that are included in the molecular conformational trajectory, a similarity measure between: (i) the predicted 3D structure, and (ii) the 3D structure that is included in the molecular conformational trajectory; and determining the target quality score based on the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory.

[0220] 4. The method of clause 3, wherein determining the target quality score based on the similarity measures comprises: determining the target quality score as a minimum of the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory.

[0221] 5. The method of any one of clauses 2-4, wherein training the scoring neural network on the set of training examples comprises training the scoring neural network, by a machine learning training technique, to optimize an objective function that, for each training example, measures an error between: (i) the target quality score specified by the training example, and (ii) a predicted quality score generated by processing the training input of the training example using the scoring neural network.

[0222] 6. The method of any preceding clause, further comprising continuously generating predicted 3D structures of the molecular system using the generative machine learning model until a termination criterion is satisfied; wherein the termination criterion is based at least in part on quality scores generated by the scoring neural network for the predicted 3D structures generated by the generative machine learning model.

[0223] 7. The method of clause 6, wherein the termination criterion is that the generative machine learning model has generated at least a threshold number of predicted 3D structures that each have a respective quality score that satisfies a quality score threshold. Isomorphic Labs Limited F&R Ref: 53672-0024W01 PCT Application

[0224] 8. The method of clause 7, wherein the threshold number of predicted 3D structures is one.

[0225] 9. The method of any preceding clause, further comprising training the generative machine learning model to optimize an objective function that is based on quality scores generated by the scoring neural network for predicted 3D structures generated by the generative machine learning model.

[0226] 10. The method of clause 9, wherein training the generative machine learning model to optimize an objective function that is based on quality scores generated by the scoring neural network comprises, for one or more of the plurality of predicted 3D structures of the molecular system: determining gradients, with respect to a set of generative machine learning model parameters of the generative machine learning model, of an objective function that is based on the quality score generated by the scoring neural network for the predicted 3D structure of the molecular system; and updating current values of the set of generative machine learning model parameters using the gradients.

[0227] 11. The method of any preceding clause, wherein the generative machine learning model and the scoring neural network have been trained by operations comprising: obtaining training data that, for each of a plurality of training molecular systems, identifies an actual 3D structure of the molecular system; partitioning the plurality of training molecular systems characterized by the training data into: (i) a first set of training data for training the generative machine learning model, and (ii) a second set of training data for training the scoring neural network; training the generative machine learning model on the first set of training data; after training the generative machine learning model on the first set of training data, training the scoring neural network on the second set of training data, comprising: generating predicted 3D structures of the training molecular systems of the second set of training data using the trained generative machine learning model; and training the scoring neural network using the predicted 3D structures generated by the trained generative machine learning model of the training molecular systems of the second set of training data. Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0228] 12. The method of clause 11, wherein there is no overlap between the first set of training data for training the generative machine learning model and the second set of training data for training the scoring neural network.

[0229] 13. The method of any preceding clause, wherein the generative machine learning model comprises a generative diffusion model.

[0230] 14. The method of any preceding clause, wherein the scoring neural network is implemented as a graph neural network that includes a plurality of message passing neural network layers.

[0231] 15. The method of any preceding clause, further comprising determining a ranking of the plurality of predicted 3D structures of the molecular system based on the quality scores.

[0232] 16. The method of clause 15, further comprising selecting one of the plurality of predicted 3D structures based at least in part on the ranking of the plurality of predicted 3D structures according to the quality scores.

[0233] 17. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of clauses 1-16.

[0234] 18. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of clauses 1-16.

[0235] 19. A method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands: Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0236] (a) performing the method of clause 16, to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and

[0237] (b) evaluating an interaction of the candidate ligand with the target protein molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating.

[0238] 20. A method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands:

[0239] (a) using a generative machine learning model trained using the method of any one of clauses 9-12, to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and

[0240] (b) evaluating an interaction of the candidate ligand with the target protein molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating.

[0241] 21. A method as claimed in clause 19 or 20, wherein the target protein molecule comprises a receptor or enzyme, and wherein the ligand is an agonist or antagonist of the receptor or enzyme.

[0242] 22. A method as claimed in any one of clauses 19-21, wherein the ligand is a drug, and the method comprises: performing steps (a) and (b) for each of a plurality of target protein molecules; and selecting one or more of the candidate ligands as the ligand to either i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

[0243] 23. The method of clause 19 or clause 20, wherein the ligand comprises an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, in particular a virus or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect. Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0244] 24. The method of any one of clauses 19-23, wherein the ligand is a polypeptide ligand, a polynucleotide ligand, or a polynucleotide ligand.

[0245] 25. A method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: performing the method of clause 16, to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating.

[0246] 26. A method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: using a generative machine learning model trained using the method of any one of clauses 9-12, to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as the diagnostic antibody or aptamer marker dependent on a result of the evaluating.

[0247] 27. A method as claimed in any one of clauses 19-26 wherein the evaluating the interaction of one of the candidate ligands comprises determining an interaction score for the candidate ligand, wherein the interaction score comprises a measure of an interaction between the candidate ligand and the target molecule.

[0248] 28. A method as claimed in any one of clauses 19-27 further comprising synthesizing the ligand or diagnostic antibody or aptamer marker. Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application

[0249] 29. A method as claimed in clause 28, further comprising testing biological activity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

[0250] 30. A method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; performing the method of clause 16, to determine a predicted structure of the molecule complex; and using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

[0251] 31. A method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; using a generative machine learning model trained using the method of any one of clauses 9-12, to determine a predicted structure of the molecule complex; and using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

[0252] 32. A method as claimed in clause 30 or 31, wherein the experimental technique comprises one or more of: x-ray crystallography, nuclear magnetic resonance, and electron microscopy.

[0253] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT ApplicationCLAIMS1. A method performed by one or more computers, the method comprising: receiving data identifying one or more molecules included in a molecular system; processing a model input that identifies the one or more molecules included in the molecular system using a generative machine learning model to generate, as model outputs of the generative machine learning model, a plurality of predicted 3D structures of the molecular system; generating, for each of the plurality of predicted 3D structures of the molecular system, a quality score that estimates a similarity between: (i) the predicted 3D structure of the molecular system, and (ii) an actual 3D structure of the molecular system, comprising, for each of the plurality of predicted 3D structures of the molecular system: processing a scoring model input that comprises: (i) data identifying the one or more molecules included in the molecular system, and (ii) the predicted 3D structure of the molecular system, using a scoring neural network to generate the quality score for the predicted 3D structure of the molecular system; and outputting the plurality of predicted 3D structures of the molecular system and the quality scores for the plurality of predicted 3D structures of the molecular system.

2. The method of claim 1, wherein the scoring neural network has been trained on a set of training examples that each correspond to a respective training molecular system and include:(a) a training input that comprises data that identifies one or more molecules included in the training molecular system and a predicted 3D structure of the training molecular system that is generated by the generative machine learning model; and(b) a target quality score that measures a similarity between the predicted 3D structure of the training molecular system and an actual 3D structure of the training molecular system.

3. The method of claim 2, further comprising, for each training example, generating the target quality score for the training example by performing operations comprising: obtaining data defining a baseline 3D structure of the training molecular system; performing a molecular dynamics or Monte Carlo simulation to generate a molecular conformational trajectory that includes: (i) the baseline 3D structure of the training molecular system, and (ii) a plurality of additional 3D structures of the molecular system;43Isomorphic Labs LimitedF&R Ref: 53672-0024WO1 PCT Application determining, for each a plurality of 3D structures that are included in the molecular conformational trajectory, a similarity measure between: (i) the predicted 3D structure, and (ii) the 3D structure that is included in the molecular conformational trajectory; and determining the target quality score based on the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory.

4. The method of claim 3, wherein determining the target quality score based on the similarity measures comprises: determining the target quality score as a minimum of the similarity measures between the predicted 3D structure and the 3D structures included in the molecular conformational trajectory.

5. The method of any one of claims 2-4, wherein training the scoring neural network on the set of training examples comprises training the scoring neural network, by a machine learning training technique, to optimize an objective function that, for each training example, measures an error between: (i) the target quality score specified by the training example, and (ii) a predicted quality score generated by processing the training input of the training example using the scoring neural network.

6. The method of any preceding claim, further comprising continuously generating predicted 3D structures of the molecular system using the generative machine learning model until a termination criterion is satisfied; wherein the termination criterion is based at least in part on quality scores generated by the scoring neural network for the predicted 3D structures generated by the generative machine learning model.

7. The method of claim 6, wherein the termination criterion is that the generative machine learning model has generated at least a threshold number of predicted 3D structures that each have a respective quality score that satisfies a quality score threshold.

8. The method of claim 7, wherein the threshold number of predicted 3D structures is one.44Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application9. The method of any preceding claim, further comprising training the generative machine learning model to optimize an objective function that is based on quality scores generated by the scoring neural network for predicted 3D structures generated by the generative machine learning model.

10. The method of claim 9, wherein training the generative machine learning model to optimize an objective function that is based on quality scores generated by the scoring neural network comprises, for one or more of the plurality of predicted 3D structures of the molecular system: determining gradients, with respect to a set of generative machine learning model parameters of the generative machine learning model, of an objective function that is based on the quality score generated by the scoring neural network for the predicted 3D structure of the molecular system; and updating current values of the set of generative machine learning model parameters using the gradients.

11. The method of any preceding claim, wherein the generative machine learning model and the scoring neural network have been trained by operations comprising: obtaining training data that, for each of a plurality of training molecular systems, identifies an actual 3D structure of the molecular system; partitioning the plurality of training molecular systems characterized by the training data into: (i) a first set of training data for training the generative machine learning model, and (ii) a second set of training data for training the scoring neural network; training the generative machine learning model on the first set of training data; after training the generative machine learning model on the first set of training data, training the scoring neural network on the second set of training data, comprising: generating predicted 3D structures of the training molecular systems of the second set of training data using the trained generative machine learning model; and training the scoring neural network using the predicted 3D structures generated by the trained generative machine learning model of the training molecular systems of the second set of training data.

12. The method of claim 11, wherein there is no overlap between the first set of training data for training the generative machine learning model and the second set of training data for45Isomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application training the scoring neural network.

13. The method of any preceding claim, wherein the generative machine learning model comprises a generative diffusion model.

14. The method of any preceding claim, wherein the scoring neural network is implemented as a graph neural network that includes a plurality of message passing neural network layers.

15. The method of any preceding claim, further comprising determining a ranking of the plurality of predicted 3D structures of the molecular system based on the quality scores.

16. The method of claim 15, further comprising selecting one of the plurality of predicted 3D structures based at least in part on the ranking of the plurality of predicted 3D structures according to the quality scores.

17. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of claims 1-16.

18. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of claims 1-16.

19. A method of obtaining a ligand, wherein the ligand is a drug or a ligand of an industrial enzyme, the method comprising: for each of one or more candidate ligands:(a) performing the method of claim 16 or using a generative machine learning model trained using the method of any one of claims 9-12, to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and(b) evaluating an interaction of the candidate ligand with the target proteinIsomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application molecule dependent on the predicted structure; and selecting one or more of the candidate ligands as the ligand dependent on a result of the evaluating.

20. A method as claimed in claim 19, wherein the target protein molecule comprises a receptor or enzyme, and wherein the ligand is an agonist or antagonist of the receptor or enzyme.

21. A method as claimed in claim 19 or 20 wherein the ligand is a drug, and the method comprises: performing steps (a) and (b) for each of a plurality of target protein molecules; and selecting one or more of the candidate ligands as the ligand to either i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

22. The method of claim 19, wherein the ligand comprises an antibody or aptamer and the target protein molecule comprises an antibody or aptamer target, in particular a virus or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect.

23. The method of any one of claims 19-22, wherein the ligand is a polypeptide ligand, a polynucleotide ligand, or a polynucleotide ligand.

24. A method of obtaining a diagnostic antibody or aptamer marker of a disease, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: performing the method of claim 16 or using a generative machine learning model trained using the method of any one of claims 9-12, to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and evaluating an interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more of the candidate antibodies or aptamers as theIsomorphic Labs Limited F&R Ref: 53672-0024WO1 PCT Application diagnostic antibody or aptamer marker dependent on a result of the evaluating.

25. A method as claimed in any one of claims 19-24 wherein the evaluating the interaction of one of the candidate ligands comprises determining an interaction score for the candidate ligand, wherein the interaction score comprises a measure of an interaction between the candidate ligand and the target molecule.

26. A method as claimed in any one of claims 19-25 further comprising synthesizing the ligand or diagnostic antibody or aptamer marker.

27. A method as claimed in claim 26 further comprising testing biological activity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

28. A method of determining the structure of a molecule complex comprising a protein and one or more ligands, comprising: applying an experimental technique to a physical sample comprising the molecule complex to measure experiment signals dependent on a structure of the molecule complex; performing the method of claim 16 or using a generative machine learning model trained using the method of any one of claims 9-12, to determine a predicted structure of the molecule complex; and using the experiment signals and the predicted structure of the molecule complex to determine the structure of the molecule complex.

29. A method as claimed in claim 28, wherein the experimental technique comprises one or more of: x-ray crystallography, nuclear magnetic resonance, and electron microscopy.48

Images