Protein binder selection using structure prediction machine learning models

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting protein binders, e.g., for physical synthesis. In one aspect, a method comprises: determining, for each candidate protein binder in a set of candidate protein binders for a target molecule, one or more quality scores for the candidate protein binder; and selecting a proper subset of the set of candidate protein binders based at least in part on the quality scores.

Description

[0001] DeepMind Technologies Limited

[0002] F&R Ref.: 45288-0503WO1 PCT Application

[0003] PROTEIN BINDER SELECTION USING STRUCTURE PREDICTION MACHINE LEARNING MODELS

[0004] CROSS-REFERENCE TO RELATED APPLICATIONS

[0005] This application claims priority to U.S. Provisional Application No. 63 / 689,547, filed on August 30, 2024. The disclosure of the prior application is considered part of and is incorporated by reference in the disclosure of this application.

[0006] BACKGROUND

[0007] This specification relates to methods for obtaining a protein binder for a target molecule.

[0008] A protein includes a sequence of amino acids. An amino acid is an organic compound which includes an amino functional group and a carboxyl functional group, as well as a side-chain (i.e., group of atoms) that is specific to the amino acid.

[0009] Protein folding refers to a physical process by which a sequence of amino acids folds into a three-dimensional configuration. The structure of a protein defines the three- dimensional (3D) configuration of the atoms in the amino acid sequence of the protein after the protein undergoes protein folding. When in a sequence linked by peptide bonds, the amino acids may be referred to as amino acid residues.

[0010] Predictions can be made using machine learning models. Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model. Some machine learning models are deep models that employ multiple layers of models to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output.

[0011] SUMMARY

[0012] This specification generally describes a system implemented as computer programs on one or more computers in one or more locations for obtaining a protein binder for a target molecule, e.g., a polypeptide capable of binding to the target molecule.

[0013] The system described in this specification can select protein binders, e.g., proteins that bind to a specified target molecule, from a set of candidate protein binders. The candidate protein binders can be selected for physical synthesis and, optionally, for DeepMind Technologies Limited

[0014] F&R Ref.: 45288-0503WO1 PCT Application experimental testing, such as in vitro or in vivo testing. The system can include a structure prediction machine learning model that is configured to process a model input that includes an amino acid sequence of a protein to predict a 3D structure of the protein or a 3D structure of a complex that includes the protein.

[0015] According to one aspect there is provided a method of obtaining a protein binder for a target molecule, the method performed by one or more computers, the method comprising: determining, for each candidate protein binder in a set of candidate protein binders for a target molecule, one or more quality scores for the candidate protein binder, comprising: processing a model input that comprises at least an amino acid sequence of the candidate protein binder using a structure prediction machine learning model to generate a model output that comprises: (i) a predicted 3D structure of a complex that comprises the candidate protein binder and the target molecule, and (ii) a respective confidence score for each of a plurality of atoms in the complex, wherein a confidence score for an atom characterizes a confidence of the structure prediction machine learning model in a position of the atom in the predicted 3D structure of the complex; and determining a quality score of the one or more quality scores for the candidate protein binder based on the respective confidence scores for the atoms in the complex; and selecting a proper subset of the set of candidate protein binders (such as for physical synthesis or for some other purpose, such as further (in silico) characterization) based at least in part on the quality scores.

[0016] In some implementations, selecting the proper subset of the set of candidate protein binders, such as for physical synthesis or further characterization, comprises selecting only candidate protein binders that are predicted to bind successfully to the target molecule under real -world experimental conditions. For example, each quality score can provide a measure of likelihood of binding under real -world experimental conditions. As another example, each quality score can be indicative of a value of a physical property, or a combination of values of physical properties, that is associated (e.g., correlated) with successful binding of the candidate protein binder to the target molecule under the real-world experimental conditions. For example, each quality score can be indicative of a binding affinity (e.g., an equilibrium constant or free energy change) for the candidate protein binder and the target molecule under the real-world experimental conditions. As described below, in some implementations, the quality scores can be calibrated against binding affinities for protein binders and molecules that have been measured under the real-world experimental conditions. Thus, values of the quality scores can be correlated with values of experimental scores characterizing binding affinities for binder proteins and one or more target molecules, DeepMind Technologies Limited

[0017] F&R Ref.: 45288-0503WO1 PCT Application e.g., such that the quality scores can be used to predict whether the candidate protein binders will bind to the target molecule under the real-world experimental conditions. The real-world experimental conditions can, for example, comprise in vitro (such as in a cell culture or tissue model) or in vivo experimental conditions, or any other appropriate set of physical conditions.

[0018] In some implementations, the candidate protein binders are polypeptides, e.g., with respective molecular masses between 5 and 15 kDa.

[0019] In some implementations, determining a quality score for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: processing the predicted 3D structure of the complex to identify a set of atoms that are included in an interface between the candidate protein binder and the target molecule; and determining a quality score for the candidate protein binder based only on confidence scores for the set of atoms that are included in the interface, i.e., without using confidence scores of atoms that are not in the set of atoms included in the interface.

[0020] In some implementations, determining a quality score for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: determining a quality score for the candidate protein binder based only on confidence scores for atoms that are included in the candidate protein binder, e.g., without using confidence scores for atoms that are included in the target molecule.

[0021] In some implementations, determining a quality score for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: determining a quality score for the candidate protein binder based only on confidence scores for atoms that are included in the target molecule, e.g., without using confidence scores for atoms that are included in the candidate protein binder.

[0022] In some implementations, determining a quality score for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: determining a quality score for the candidate protein binder based on confidence scores of both: (i) atoms that are included in the candidate protein binder, and (ii) atoms that are included in the target molecule.

[0023] In some implementations, the set of candidate protein binders have been identified using a protein design generative model by performing operations comprising: generating, using the protein design generative model and while the protein design generative model is conditioned on data characterizing the target molecule, a plurality of predicted 3D protein DeepMind Technologies Limited

[0024] F&R Ref.: 45288-0503WO1 PCT Application structures, wherein each predicted 3D protein structure defines a respective predicted 3D structure of a candidate protein binder that is predicted to bind to the target molecule.

[0025] In some implementations, the method further comprises, for each candidate protein binder, determining an amino acid sequence of the candidate protein binder based on the predicted 3D structure of the candidate protein binder that is generated by the protein design generative model.

[0026] In some implementations, the method further comprises, for each candidate protein binder in the set of candidate protein binders: determining a second quality score of the one or more quality scores for the candidate protein binder based on a measure of similarity between: (i) the predicted 3D structure of the candidate protein binder as generated by the protein design generative model when conditioned on data characterizing the target molecule, and (ii) the predicted 3D structure of the candidate protein binder as generated by the structure prediction machine learning model by processing a model input that comprises at least the amino acid sequence of the candidate protein binder.

[0027] In some implementations, the method further comprises: generating a set of training examples for training the protein design generative model based on the proper subset of the set of candidate protein binders, comprising, for each selected candidate protein binder: generating a training example that comprises: (i) a conditioning input that specifies the target molecule, and (ii) a target output that specifies a 3D structure of the candidate protein binder; and training the protein design generative model on the set of training examples.

[0028] In some implementations, for each training example in the set of training examples, training the protein design generative model on the training example comprises: training the protein design generative model to reduce a discrepancy between: (i) 3D protein structures generated by the protein design generative model when conditioned on the conditioning input of the training example, and (ii) the 3D structure of the candidate protein binder specified by the target output of the training example.

[0029] In some implementations, the protein design generative model has been trained on a set of training examples that comprises a plurality of unconditional training examples, wherein: each unconditional training example comprises a target output that specifies a 3D structure of a molecule or molecule complex; and training the protein design generative model on an unconditional training example comprises training the protein design generative model to generate a predicted output that matches the target output of the unconditional training example. DeepMind Technologies Limited

[0030] F&R Ref.: 45288-0503WO1 PCT Application

[0031] In some implementations, the protein design generative model has been trained on a set of training examples that comprises a plurality of conditional training examples, wherein: each conditional training example comprises: (i) a conditioning input that specifies a portion of a 3D structure of a molecule or molecule complex, and (ii) a target output that specifies all of the 3D structure of the molecule or molecule complex; and training the protein design generative model on a conditional training example comprises training the protein design generative model to generate a predicted output that matches the target output of the training example when conditioned on the conditioning input of the conditional training example.

[0032] In some implementations, the method further comprises, for each candidate protein binder in the set of candidate protein binders: determining a predicted stability of the candidate protein binder; and determining a third (or “further”) quality score of the one or more quality scores for the candidate protein binder based at least in part on the predicted stability of the candidate protein binder.

[0033] In some implementations, selecting a proper subset of the set of candidate protein binders, e.g., for physical synthesis, based at least in part on the quality scores comprises, for one or more of the quality scores: determining, for each candidate protein binder in the set of candidate protein binders, that the candidate protein binder is eligible to be selected (such as for physical synthesis) only if a value of the quality score for the candidate protein binder satisfies a threshold that is specific to the quality score.

[0034] In some implementations, for one or more of the quality scores, the threshold that is specific to the quality score has been determined by performing operations comprising: determining, for each a plurality of example proteins: (i) a value of the quality score for the example protein, and (ii) a value of an experimental score for the example protein, wherein the experimental score is evaluated by physically synthesizing the example protein and performing an experiment to measure a value of a property of the example protein; and determining the threshold that is specific to the quality score based on the values of the quality score and the experimental score for the plurality of example proteins.

[0035] In some implementations, for each of the plurality of example proteins, the value of the experimental score for the example protein characterizes a binding affinity of the example protein for an example target molecule. Binding affinities can be determined experimentally by, for example, measuring equilibrium constants or free energies for association of the example target molecule and the example protein.

[0036] In some implementations, determining the threshold that is specific to the quality score based on the values of the quality score and the experimental score for the plurality of DeepMind Technologies Limited

[0037] F&R Ref.: 45288-0503WO1 PCT Application example proteins comprises: determining the threshold based on a correlation between the values of the quality score and the values of the experimental score for the plurality of example proteins.

[0038] In some implementations, the target molecule comprises a protein, or a small molecule (e.g., an organic compound with a molecular weight of <1000 daltons), or a ribonucleic acid (RNA), or a deoxyribonucleic acid (DNA) molecule.

[0039] In some implementations, the set of candidate protein binders comprises at least 100,000 candidate protein binders.

[0040] In some implementations, selecting the proper subset of the set of candidate protein binders (such as for physical synthesis) based at least in part on the quality scores comprises: selecting fewer than 1% of the candidate protein binders in the set of candidate protein binders.

[0041] In some implementations, the method further comprises physically synthesizing some or all of the candidate protein binders in the proper subset of candidate protein binders.

[0042] In some implementations, the method further comprises, for each candidate protein binder selected for the proper subset: performing physical experiments to assess one or more properties of the candidate protein binder, in particular in vitro or in vivo.

[0043] In some implementations, the method further comprises, selecting the proper subset of the set of candidate protein binders for physical synthesis based at least in part on the quality scores and for each candidate protein binder selected for physical synthesis: performing physical experiments to assess one or more properties of the candidate protein binder.

[0044] In some implementations, for each candidate protein binder selected for physical synthesis, performing physical experiments to assess one or more properties of the candidate protein binder comprises: performing physical experiments to assess one or more of: a binding affinity of the candidate protein binder for the target molecule; or a stability of the candidate protein binder; or absorption properties of the candidate protein binder; or distribution properties of the candidate protein binder; or metabolism properties of the candidate protein binder; or excretion properties of the candidate protein binder; or toxicity properties of the candidate protein binder. The physical experiments can be performed in vitro (such as in a cell culture or tissue model), or in vivo, in some cases. For example the candidate protein binder(s) can be screened according to a degree to which binding to the target molecule is accompanied by a biological (therapeutic) effect such as facilitating a biological mechanism or directly or indirectly inhibiting a biological disease mechanism DeepMind Technologies Limited

[0045] F&R Ref.: 45288-0503WO1 PCT Application

[0046] (e.g. inhibiting a bacteria or virus from entering a cell), toxicity, clearance time, and so forth. The physical experiments may include, e.g., bringing the candidate protein binder into contact with the target molecule and measuring a change in expression or activity of the target molecule.

[0047] In some implementations, the structure prediction machine learning model has been trained to process a model input that includes an amino acid sequence of a protein to predict a 3D structure of the protein or a 3D structure of a complex that includes the protein.

[0048] In some implementations, the target molecule may comprise a receptor or enzyme and the selected candidate protein binder(s) may be an agonist or antagonist of the receptor or enzyme. In some implementations, the candidate protein binder(s) may comprise an antibody and the target molecule comprises an antibody target, for example a virus, in particular a virus coat protein, or a protein expressed on a cancer cell. In these implementations, the antibody binds to the antibody target to provide a therapeutic effect. For example, the antibody may bind to the target and act as an agonist for a particular receptor; alternatively, the antibody may prevent binding of another binder to the target, and hence prevent activation of a relevant biological pathway.

[0049] According to another aspect there are provided one or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computer to implement the methods described herein.

[0050] According to another aspect there is provided a system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the respective operations of the methods described herein.

[0051] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0052] The system described in this specification allows for the creation of high-quality, e.g., small, thermostable, or high-affinity, binders with minimal experimental effort for a wide variety of applications. The system described in this specification performs computational design of candidate protein binders that can target molecules of a wide variety of structural and biochemical properties. For example, the system can use a protein design generative model to generate a set of candidate protein binders. The protein design generative model can be configured to generate a predicted 3D protein structure conditioned on data characterizing a target molecule. DeepMind Technologies Limited

[0053] F&R Ref.: 45288-0503WO1 PCT Application

[0054] Protein design generative models can generate a large number, e.g., hundreds of thousands, of candidate protein binders for binding to a target molecule. However, synthesizing a large number of candidate protein binders to determine binding qualities of the candidate protein binders is often inefficient or infeasible.

[0055] The system described in this specification can select protein binders that are likely to perform well (e.g., by having a high affinity for the target molecule) from a set of protein binders without performing laboratory experiments for all of the protein binders in the set, saving time and laboratory resources. The protein binders can be selected for physical synthesis, for example. The system described in this specification can also select protein binders for downstream functional efficacy, for example, protein binders that are predicted to bind successfully to the target molecule under real-world experimental conditions.

[0056] Some conventional techniques for selecting protein binders such as experimental selection are often too laborious for routine research applications. The system described in this specification can select the most promising protein binders from the large number of candidate protein binders, reducing the number of candidate protein binders to synthesize. The system described in this specification identifies the most promising protein binders out of the set of candidate protein binders based on quality scores. For example, the system determines, for each candidate protein binder in a set of candidate protein binders for a target molecule, one or more quality scores for the candidate protein binder. The system can determine the one or more quality scores for the candidate protein binder based on the output of a structure prediction machine learning model. The system selects a proper subset of the set of candidate protein binders based at least in part on the quality scores.

[0057] The system can be used for identifying a drug, i.e. a protein binder with a therapeutic effect (e.g. an agonist or antagonist of a receptor or enzyme). This can involve selecting the candidate protein binders as putative drugs based on the quality scores. Optionally the selected candidate protein binders(s) can then be further screened in silico, or after physical synthesis, in vitro (e.g. in a cell culture or tissue model), or in vivo. For example the selected candidate protein binders can be screened for further useful properties, e.g. according to a degree to which binding is accompanied by a biological (therapeutic) effect such as facilitating a biological mechanism or directly or indirectly inhibiting a biological disease mechanism (e.g. inhibiting a bacteria or virus from entering a cell); toxicity; clearance time; and so forth. When a drug that includes the protein binder is administered to a patient, the protein binder can bind to the target protein with high affinity and in doing so contribute to achieving a therapeutic effect in the patient. For instance, if the target molecule DeepMind Technologies Limited

[0058] F&R Ref.: 45288-0503WO1 PCT Application is an enzyme involved in a disease process, the ligand can inhibit its activity, thus disrupting the disease pathway. More generally, the interaction between the protein binder and the target molecule (e.g., protein) can activate, inhibit, or alter the function of the target molecule to achieve a therapeutic effect.

[0059] In some implementations the target molecule includes a receptor or enzyme and the candidate protein binder is an agonist or antagonist of the receptor or enzyme. In some implementations the method may be used to predict whether the candidate protein binders can bind to a cell surface marker. The candidate protein binders can then act as, e.g., an antibody or a label such as a fluorescent label, which binds to the cell surface marker. This may be used to identify and / or treat cancerous cells.

[0060] In some implementations a candidate protein binder may include: an isolated antibody, a fragment of an isolated antibody, a single variable domain antibody, a bi- or multi-specific antibody, a multivalent antibody, a dual variable domain antibody, an immuno-conjugate, a fibronectin molecule, an adnectin, an DARPin, an avimer, an affibody, an anticalin, an affilin, a protein epitope mimetic or combinations thereof. A candidate (e.g. polypeptide) protein binder may include an antibody with a mutated or chemically modified amino acid Fc region, e.g., which prevents or decreases ADCC (antibody-dependent cellular cytotoxicity) activity and / or increases half-life when compared with a wild type Fc region. Candidate (polypeptide or polynucleotide) protein binders may include antibodies with different CDRs (Complementarity-Determining Regions). In some implementations, (i) one or more of the selected candidate protein binders comprises the CDR or Fc region of an antibody or (ii) one or more of the selected candidate protein binders comprises a scaffold structure supporting a protein region or protein domain for binding or catalysis, or (iii) one or more of the selected candidate protein binders is an agonist or antagonist of a receptor or enzyme or an antibody or monobody configured to bind to an antibody or monobody target.

[0061] The system described herein can also be used to obtain a diagnostic antibody marker of a disease. The diagnostic antibody marker may be used to diagnose a disease by detecting whether it binds to the target protein molecule in a sample obtained from a patient, e.g. a sample of bodily fluid.

[0062] In some examples, the protein design generative model can be trained to generate realistic and structurally diverse predicted 3D protein structures. For example, the protein design generative model can have been trained by a training system of the system described in this specification, or another training system, on a set of training examples that includes training examples for a mixture of tasks. Training the protein design generative model DeepMind Technologies Limited

[0063] F&R Ref.: 45288-0503WO1 PCT Application allows the protein design generative model to generalize better to previously unseen inputs. As another example, the protein design generative model can have been trained on a filtered set of training examples. For example, the filtered set of training examples can include training examples that have a target output with a molecule or molecule complex that meets one or more threshold filter scores. Each filter score represents a predicted measure of quality. The filtered set of training examples can thus include high-quality training examples and the protein design generative model can be trained on the high-quality training examples. As another example, the system can further train the protein design generative model on a set of training examples based on the proper subset of the set of candidate protein binders that are selected. For example, for each candidate protein binder selected for the proper subset, the system can generate a training example that includes: (i) a conditioning input that specifies the target molecule, and (ii) a target output that specifies a 3D structure of the candidate protein binder. Because the candidate protein binders selected for the proper subset are the most promising out of the set of candidate protein binders, by training the protein design generative model on the set of training examples, the system can train the protein design generative model on high-quality training examples.

[0064] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0065] BRIEF DESCRIPTION OF THE DRAWINGS

[0066] FIG. l is a block diagram of an example protein selection system.

[0067] FIG. 2 is a flow diagram of an example process for selecting candidate protein binders based on quality scores.

[0068] FIG. 3 is a flow diagram of an example process for determining a threshold for a quality score.

[0069] FIG. 4 is a flow diagram of an example process for training a protein design generative model on a set of training examples.

[0070] FIG. 5 is a flow diagram of an example process for training a protein design generative model to generate a predicted 3D protein structure.

[0071] Like reference numbers and designations in the various drawings indicate like elements. DeepMind Technologies Limited

[0072] F&R Ref.: 45288-0503WO1 PCT Application

[0073] DETAILED DESCRIPTION

[0074] FIG. 1 shows an example protein selection system 100. The protein selection system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

[0075] The protein selection system 100 can select a subset of candidate protein binders 142 from a set of candidate protein binders 102 for a target molecule. The set of candidate protein binders 102 includes multiple candidate protein binders (e.g., polypeptide ligands) for the target molecule. The target molecule can include, for example, a protein, or a small molecule, or a ribonucleic acid (RNA), or a deoxyribonucleic acid (DNA) molecule. In some examples where the target molecule is a small molecule (e.g., having a molecular mass of less than 1000 daltons), the set of candidate protein binders 102 includes enzymes predicted to bind to the small molecule.

[0076] The protein selection system 100 can obtain data representing the set of candidate protein binders 102. In some examples, the data representing the set of candidate protein binders 102 can include 3D protein structures. Alternatively or in addition, the data representing the set of candidate protein binders 102 can include the amino acid sequences of each of the 3D protein structures.

[0077] In some examples, the system can identify the set of candidate protein binders 102. In these examples, the system 100 can include a protein design generative model 150. For example, the system 100 can sample from the protein design generative model 150, when the protein design generative model 150 is conditioned on data characterizing the target molecule, to generate respective predicted 3D protein structures of multiple candidate protein binders that are predicted to bind to the target molecule. The system 100 can include the predicted 3D protein structures in the data representing the set of candidate protein binders 102. In examples where the data representing the set of candidate protein binders 102 includes amino acid sequences, the system 100 can use the input system 110 described below to generate the amino acid sequences from the predicted 3D protein structures.

[0078] The protein design generative model 150 can generate a predicted 3D protein structure when conditioned on data characterizing a target molecule. The predicted 3D protein structure can define the predicted 3D structure of a candidate protein binder that is predicted to bind to the target molecule. For example, the 3D protein structure can define, for each position in the amino acid sequence for the candidate protein binder, a spatial DeepMind Technologies Limited

[0079] F&R Ref.: 45288-0503WO1 PCT Application position for each atom in a predefined set of atoms that can be included in the amino acid at the position.

[0080] As a particular example, the protein design generative model 150 can include a diffusion model. The diffusion model can include a denoising neural network that, at each denoising iteration in a sequence of denoising iterations, processes a noisy version of a protein to predict a denoised version of the protein. In particular, the diffusion model can be conditioned on structural information from target molecules, such as the atomic coordinates of a target molecule. Training the protein design generative model 150 is described in further detail below with reference to FIG. 4 and FIG. 5.

[0081] Examples of a protein design generative model are described in PCT Application Serial Number PCT / EP2024 / 063990 and in Zambaldi et al., “De novo design of high-affinity protein binders with AlphaProteo,” arXiv: 2409.08022.

[0082] Another example of a protein design generative model is described in Watson, Joseph L., et al., “De novo design of protein structure and function with RFdiffusion.” Nature 620.7976 (2023): 1089-1100.

[0083] The subset of candidate protein binders 142 can include a proper subset of the set of candidate protein binders 102 (i.e., one or more but not all of the candidate protein binders). To select the subset of candidate protein binders 142, the system 100 can process data representing each candidate protein binder in the set of candidate protein binders 102 to determine one or more quality scores for the candidate protein binder.

[0084] As a particular example, the set of candidate protein binders 102 can include a large number, e.g., at least 100,000, of candidate protein binders. The system 100 can select a small number, e.g., 50 or 100, or a small proportion, e.g., fewer than 1%, of the candidate protein binders in the set of candidate protein binders 102.

[0085] The system 100 can include an input system 110. The input system 110 can generate a model input 112 for a candidate protein binder.

[0086] For example, the input system 110 can receive data representing the set of candidate protein binders 102 and generate a model input 112 for each candidate protein binder. The model input 112 for a candidate protein binder includes at least an amino acid sequence of the candidate protein binder.

[0087] In examples where the data representing the set of candidate protein binders 102 includes 3D protein structures of the candidate protein binders, the input system 110 can determine an amino acid sequence of each candidate protein binder based on the 3D DeepMind Technologies Limited

[0088] F&R Ref.: 45288-0503WO1 PCT Application structure of the candidate protein binder. In some examples, the 3D structure can have been predicted by the protein design generative model 150.

[0089] The input system 110 can process the 3D structure of the candidate protein binder to determine the identity of an amino acid at each of multiple positions in the amino acid sequence. For example, the input system 110, for each of the multiple positions in the amino acid sequence of the protein, can identify a proper subset of the predefined set of possible atoms as being included in the amino acid at the position in the amino acid sequence. As an example, the input system 110 can use a predefined mapping between sets of atoms and amino acids to determine the identities of the amino acids in the amino acid sequence of the candidate protein binder.

[0090] As another example, the input system 110 can process the 3D structure of the candidate protein binder using a sequencing machine learning model to determine an amino acid sequence. An example sequencing machine learning model is described by Dauparas et al. in “Robust Deep Learning-Based Protein Sequence Design Using ProteinMPNN”, Science 6615 (378), p49-56 (2022).

[0091] In examples where the data representing the set of candidate protein binders 102 includes amino acid sequences of the candidate protein binders, the input system 110 can include the amino acid sequence for each candidate protein binder in a model input 112 for the candidate protein binder.

[0092] In some examples, the model input 112 for a candidate protein binder includes data representing the target molecule. For example, the data representing the target molecule can represent one or more chains of the target molecule or at least part of the 3D structure of the target molecule.

[0093] The system 100 can include a structure prediction machine learning model 120. The structure prediction machine learning model 120 can process a model input that includes data characterizing a molecule or a molecule complex to generate a predicted 3D structure of the molecule or molecule complex. The predicted 3D structure of a molecule or molecule complex can define a respective predicted 3D spatial position of each atom in the molecule or molecule complex. When the molecule or molecule complex includes one or more proteins, the model input to the structure prediction machine learning model can include a respective amino acid sequence of each of the one or more proteins. When the molecule or molecule complex includes one or more nucleic acids, the model input to the structure prediction machine learning model can include a respective nucleic acid sequence of each of the one or more nucleic acids. When the molecule or molecule complex includes one or DeepMind Technologies Limited

[0094] F&R Ref.: 45288-0503WO1 PCT Application more small molecules, the model input to the structure prediction machine learning model can define a respective chemical structure of each of the one or more small molecules, e.g., by respective SMILES strings.

[0095] The structure prediction machine learning model 120 is a different machine learning model than the protein design generative model 150. The structure prediction machine learning model 120 has a different model architecture and different model weights, and is trained to perform a different task, than the protein design generative model 150. In particular, the protein design generative model is configured and trained to perform a protein design task, e.g., generating data defining proteins that are predicted to bind to a target molecule when conditioned on data characterizing the target molecule. The structure prediction machine learning model is configured and trained to perform a structure prediction task, e.g., processing a model input that characterizes a molecule or molecule complex (e.g.: data defining an amino acid sequence of a protein and data characterizing a chemical structure of a separate target molecule) to generate a predicted 3D structure of the molecule or molecule complex.

[0096] The structure prediction machine learning model 120 can be implemented in any of a variety of possible ways. For instance, the structure prediction machine learning model 120 can be based on the AlphaFold2 model, as described in Jumper, John, et al., “Highly accurate protein structure prediction with AlphaFold.” Nature 596.7873 (2021):583-589. As another example, the structure prediction machine learning model 120 can be based on the AlphaFold3 model, as described in Abramson, Josh, et al., “Accurate structure prediction of biomolecular interactions with AlphaFold 3.” Nature (2024): 1-3. As another example, the structure prediction machine learning model 120 can be based on the RoseTTAFold model, as described in Baek, Minkyung, et al., “Accurate prediction of protein structures and interactions using a three-track neural network.” Science 373.6557 (2021):871-876.

[0097] For example, the system 100 can process the model input 112 for a candidate protein binder using the structure prediction machine learning model 120 to generate a model output 122 for the candidate protein binder. The model output 122 includes a predicted 3D structure of a complex 124 that includes the candidate protein binder and the target molecule. The model output 122 also includes confidence scores 126.

[0098] The confidence scores 126 include a respective confidence score for each of multiple atoms in the complex 124. A confidence score for an atom characterizes a confidence of the structure prediction machine learning model 120 in a position of the atom in the predicted 3D structure of the complex 124. Example techniques for determining confidence scores for DeepMind Technologies Limited

[0099] F&R Ref.: 45288-0503WO1 PCT Application atoms are described in: Jumper, John, et al., “Highly accurate protein structure prediction with AlphaFold.” Nature 596.7873 (2021):583-589; and Abramson, Josh, et al., “Accurate structure prediction of biomolecular interactions with AlphaFold 3.” Nature (2024): 1-3. As one example, the confidence score for each atom can be derived, at least implicitly, from a probability distribution for distances (or displacements) between the atom and other atoms in the complex 124 predicted by the structure prediction machine learning model. For example, the confidence score may be indicative of a characteristic length scale or width for the probability distribution. In some implementations, each confidence score may comprise one or more of a predicted local distance difference test (pLDDT) score, or a predicted distance error (PDE), or variants thereof, as described in the supplementary information accompanying Abramson, Josh, et al., see e.g., section 4.3 “Model Confidence Prediction”.

[0100] The system 100 includes a quality score generation system 130. The quality score generation system 130 determines one or more quality scores 132 for the candidate protein binder. For example, the quality score generation system 130 can determine one or more quality scores 132 for the candidate protein binder based on the confidence scores 126. Determining quality scores for the candidate protein binder is described in further detail below with reference to FIG. 2.

[0101] The system 100 includes a quality score processing system 140. The quality score processing system 140 can select the subset of candidate protein binders 142 based at least in part on the quality scores 132 for each of the candidate protein binders in the set of candidate protein binders 102. Selecting candidate protein binders based at least in part on quality scores is described in further detail below with reference to FIG. 3.

[0102] An example process for selecting candidate protein binders (e.g., for physical synthesis) based on quality scores is described in more detail below with reference to FIG. 2.

[0103] After selecting the subset of candidate protein binders 142, the system 100 can output data characterizing the subset of candidate protein binders 142 to a synthesis system in order to synthesize some or all of the candidate protein binders of the subset. As an example, the data characterizing the subset of candidate protein binder 142 can include the 3D structures for the candidate protein binders.

[0104] As another example, the data characterizing the subset of candidate protein binders 142 can specify the amino acid sequences of the candidate protein binders, and the synthesis system can synthesize proteins having the specified amino acid sequences. As a further example, the data characterizing the subset of candidate protein binders 142 can include instructions for synthesizing the candidate protein binders (e.g., required conditions and / or DeepMind Technologies Limited

[0105] F&R Ref.: 45288-0503WO1 PCT Application reagents for synthesizing the candidate protein binders) and the synthesis system can synthesize the candidate protein binders as instructed.

[0106] In some examples, the synthesis system can perform, for each candidate protein binder selected (e.g., for physical synthesis), physical experiments to assess one or more properties of the candidate protein binder. For example, the properties can include one or more of: a binding affinity of the candidate protein binder for the target molecule, a stability of the candidate protein binder, absorption properties of the candidate protein binder, distribution properties of the candidate protein binder, metabolism properties of the candidate protein binder, excretion properties of the candidate protein binder, or toxicity properties of the candidate protein binder.

[0107] FIG. 2 is a flow diagram of an example process 200 for selecting candidate protein binders, in this particular example, for physical synthesis, based on quality scores. For convenience, the process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a protein selection system, e.g., the protein selection system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 200.

[0108] In some implementations, the system can identify a set of candidate protein binders using a protein design machine learning model (step 202). For example, the system can use a protein design generative model to generate 3D protein structures conditioned on data characterizing a target molecule for inclusion in the set of candidate protein binders. In some implementations, the system can receive (e.g., from a user), data characterizing the target molecule.

[0109] In some examples, the system can use the protein design generative model to generate 3D protein structures conditioned on data characterizing a target molecule and a set of residues representing the desired epitope. In some examples, the data characterizing the target molecule characterizes a portion of the target molecule.

[0110] The system can determine, for each candidate protein binder in the set of candidate protein binders for the target molecule, one or more quality scores (step 204). The system can perform steps 206-208 to determine one or more quality scores for each candidate protein binder in the set of candidate protein binders.

[0111] The system processes a model input using a structure prediction machine learning model to generate a model output (step 206). The model input includes at least an amino acid sequence of the candidate protein binder and data characterizing the target molecule, e.g., an amino acid sequence of the target molecule, or a nucleic acid sequence of the target DeepMind Technologies Limited

[0112] F&R Ref.: 45288-0503WO1 PCT Application molecule, or a SMILES string defining a chemical structure of the target molecule. The model output includes a predicted 3D structure of a complex that includes the candidate protein binder and the target molecule, and a respective confidence score for each of multiple atoms in the complex. A confidence score for an atom characterizes a confidence (i.e., lack of uncertainty) of the structure prediction machine learning model in a (spatial) position of the atom in the predicted 3D structure of the complex.

[0113] The system determines one or more quality scores for the candidate protein binder (step 208). Each quality score measures the quality, e.g., likelihood of binding or stability, of the candidate protein binder, e.g., stability of the candidate protein binder separate from the target molecule, or stability of a complex comprising the candidate protein binder and target molecule. In some examples, each quality score measures the quality of the candidate protein binder under real-world experimental conditions, i.e., the suitability of the candidate protein binder for binding to the target molecule under the real-world experimental conditions.

[0114] As an example, the system can determine one or more quality scores based on the confidence scores for the atoms in the complex. Higher confidence scores for atoms in the complex can indicate a higher likelihood of binding between the candidate protein binder and the target molecule. If the candidate protein is not likely to bind strongly or successfully to the target molecule, the structure prediction machine learning model will be less confident, i.e., output lower confidence scores, in predicting positions of the atoms in the complex. More specifically, the structure prediction machine learning model can be trained on training examples that represent stable, real-world molecule complexes. Thus, when conditioned on a model input that does not represent a molecule complex with strong or successful binding, the structure prediction machine learning model will generate a model output with lower confidence scores for the atoms in the complex of the model output, compared to the confidence scores generated by the structure prediction machine learning model when conditioned on a model input that represents a molecule complex with strong or successful binding.

[0115] In some examples, the system processes the predicted structure of the complex to identify a set of atoms that are included in an interface between the candidate protein binder and the target molecule. The system determines a quality score for the candidate protein binder based only on confidence scores for the set of atoms that are included in the interface. DeepMind Technologies Limited

[0116] F&R Ref.: 45288-0503WO1 PCT Application

[0117] In some examples, the system determines a quality score for the candidate protein binder based only on confidence scores for atoms that are included in the candidate protein binder.

[0118] In some examples, the system determines a quality score for the candidate protein binder based only on confidence scores for atoms that are included in the target molecule.

[0119] In some examples, the system determines a quality score for the candidate protein binder based on confidence scores for atoms that are included in the candidate protein binder and for atoms that are included in the target molecule.

[0120] In some examples where the system identifies the set of candidate protein binders using a protein design generative model, the system determines a quality score for the candidate protein binder based on a measure of similarity between (i) the predicted 3D structure of the candidate protein binder as generated by the protein design generative model when conditioned on data characterizing the target molecule, and (ii) the predicted 3D structure of the candidate protein binder as generated by the structure prediction machine learning model by processing a model input that includes at least the amino acid sequence of the candidate protein binder. The measure of similarity can be, e.g., a root-mean-square deviation (RMSD) measure, or a global distance test (GDT) measure, see e.g., Zemla, "LGA: A method for finding 3D similarities in protein structures", Nucleic Acids Research, 31 (13), 3370-3374, 2003), or a template modeling score (TM-score), and so forth.

[0121] The protein design generative model, when conditioned on data characterizing the target molecule, generates a predicted 3D structure of a candidate protein binder for the target molecule. The structure prediction machine learning model, when processing a model input that includes at least the amino acid sequence of the candidate protein binder, generates a predicted 3D structure of the candidate protein binder. The amino acid sequence of the candidate protein binder can be the most probable amino acid sequence for the predicted 3D structure of the candidate protein binder generated by the protein design generative model, generated, for example, using a predefined mapping or a sequencing machine learning model. If the structure generated by the structure prediction machine learning model for the most probable amino acid sequence and the structure generated by the protein design generative model have a measure of similarity that indicates high similarity, this can indicate that the structure generated by the structure prediction machine learning model for the most probable amino acid sequence is the most probable structure for the most probable amino acid sequence. Thus the candidate protein binder is more likely to have the predicted 3D structure generated by the protein design generative model, indicating a higher likelihood of DeepMind Technologies Limited

[0122] F&R Ref.: 45288-0503WO1 PCT Application binding between the candidate protein binder and the target molecule. In some examples, the system determines a predicted stability of the candidate protein binder. The system can determine a quality score for the candidate protein binder based at least in part on the predicted stability of the candidate protein binder. A higher stability can indicate a higher likelihood of the candidate protein binder maintaining its shape and function in different environments, e.g., in vitro or in vivo environments. In some examples, the predicted stability can be based on the predicted free energies of the candidate protein binder in different states. Example techniques for predicting protein stability are described in: Blaabjerg, Lasse M., et al., “Rapid protein stability prediction using deep learning representations.” Elife 12 (2023):e82593.

[0123] In some implementations, the system can determine to cease performing the step 204 for the candidate protein binder. For example, the system can determine one or more quality scores for the candidate protein binder, such as a quality score based on the predicted stability, prior to performing step 206 or step 208. In response to determining that one of the quality scores does not meet a threshold for the quality score, the system can determine to cease performing the step 204 for the candidate protein binder. The system can perform step 204 for another candidate protein binder in the set of candidate protein binders, or proceed to step 210. The system can thus efficiently select the subset of candidate protein binders by ceasing performance of the step 204 based on the quality scores.

[0124] The system selects a proper subset of the set of candidate protein binders for physical synthesis (step 210). For example, the system selects the proper subset based at least in part on the quality scores. Selecting the subset is described in further detail below with reference to FIG. 3. In some examples, the system selects only candidate protein binders that are predicted to bind successfully to the target molecule under real-world experimental conditions. For example, the system can use one or more of the quality scores as a measure of likelihood of binding under certain experimental conditions.

[0125] In some examples, the system can output data characterizing one or more of the proper subset of candidate protein binders (e.g., the amino acid sequence of the candidate protein binder, the structure of the candidate protein binder, synthesis instructions for the candidate protein binder, etc.) to a protein synthesis system, and the protein synthesis system can synthesize a protein having the amino acid sequence of the candidate protein binders.

[0126] In some implementations, the system determines a ranking for each candidate protein binder in the subset. The ranking characterizes a prediction by the system for the relative success of each candidate protein binder binding to the target molecule. In some examples, DeepMind Technologies Limited

[0127] F&R Ref.: 45288-0503WO1 PCT Application the system can determine the ranking based on one or more scores. For example, the system can rank the candidate protein binders in order of descending scores. As another example, the system can rank the candidate protein binders in order of descending combined scores. As an example, the combined score for a candidate protein binder can be an average of multiple scores. In some examples, the average can be a weighted average.

[0128] Some example scores include the quality score based on the confidence scores for the set of atoms that are included in the interface between each candidate protein binder and the target molecule, the quality score based only on confidence scores for atoms that are included in the candidate protein binder, the quality score based only on confidence scores for atoms that are included in the target molecule, the quality score based on confidence scores of both: (i) atoms that are included in the candidate protein binder, and (ii) atoms that are included in the target molecule, and the quality score based on a measure of similarity between: (i) the predicted 3D structure of the candidate protein binder as generated by the protein design generative model when conditioned on data characterizing the target molecule, and (ii) the predicted 3D structure of the candidate protein binder as generated by the structure prediction machine learning model by processing a model input that includes at least the amino acid sequence of the candidate protein binder.

[0129] In some of these implementations, the system can output data characterizing one or more of the proper subset of candidate protein binders and data characterizing the ranking for each of the one or more candidate protein binders. In some of these implementations, the system can output data characterizing one or more of the proper subset of candidate protein binders according to the rankings. For example, the system can output data characterizing the candidate protein binders of the subset that have the top-A highest rankings, where k is an integer greater than or equal to one. The system can thus provide the protein synthesis system with data characterizing candidate protein binders that the system determines has the highest quality or likelihood of binding.

[0130] FIG. 3 is a flow diagram of an example process 300 for determining a threshold for a quality score. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a protein selection system, e.g., the protein selection system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 300.

[0131] The system can perform the steps 302 and 304 for one or more of the quality scores to determine a threshold for the quality score. The system can determine, for each of DeepMind Technologies Limited

[0132] F&R Ref.: 45288-0503WO1 PCT Application multiple example proteins, (i) a value of the quality score for the example protein, and (ii) a value of an experimental score for the example protein (step 302).

[0133] The experimental score is evaluated by physically synthesizing the example protein and performing an experiment to measure a value of a property of the example protein. As an example, the value of the experimental score for the example protein can characterize a binding affinity of the example protein for an example target molecule, e.g., under the experimental conditions of the experiment.

[0134] The system determines the threshold that is specific to the quality score based on the values of the quality scores and the experimental scores for the multiple proteins (step 304). For example, the system can determine the threshold based on a correlation between the values of the quality score and the values of the experimental score for the multiple example proteins.

[0135] In some examples, the system can determine the threshold to be predictive of binding. For example, a positive correlation between the values of the quality score and the values of the experimental score that characterizes a binding affinity for the example proteins can indicate a higher probability that the example protein will bind to the example target molecule when the value of the quality score is higher compared to when the value of the quality score is lower.

[0136] In some examples, the threshold is a binary threshold. For example, the system can determine the threshold to be a value within the range of the values of the quality score, where for values less than the threshold, the probability that the example proteins bind to the example target molecule was lower than a threshold probability such as 50%, and for values greater than or equal to the threshold, the probability that the example proteins bind to the example target molecule was greater than or equal to the threshold probability.

[0137] The system can select a proper subset of the set of candidate protein binders (e.g., for physical synthesis) based at least in part on a threshold specific to a quality score for one or more of the quality scores. For example, the system can perform steps 306-308 as part of step 210 of FIG. 2.

[0138] The system can perform steps 306-308 for each candidate protein binder in the set of candidate protein binders and for one or more of the quality scores, e.g., one or more of the quality scores that have a threshold determined as described with reference to steps 302-304 above. The system determines whether the value of the quality score for the candidate protein binder satisfies a threshold that is specific to the quality score (step 306). As an example, the system can determine that the value of the quality score for the candidate DeepMind Technologies Limited

[0139] F&R Ref.: 45288-0503WO1 PCT Application protein binder satisfies the threshold that is specific to the quality score if the value of the quality score for the candidate protein binder is greater than or equal to the threshold.

[0140] If the system determines that the value of the quality score for the candidate protein binder satisfies the threshold, the system can determine that the candidate protein binder is eligible to be selected for physical synthesis (step 308). In some implementations, the system can include the candidate protein binder in the subset of candidate protein binders. In some of these implementations, the system can then return to step 306 for a next candidate protein binder in the set of candidate protein binders.

[0141] If the system determines that the value of the quality score for the candidate protein binder does not satisfy the threshold, the system can return to step 306 for a next quality score.

[0142] If there are no other quality scores for the candidate protein binder, the system can determine that the candidate protein binder is not eligible to be selected (e.g., for physical synthesis). That is, the system does not include the candidate protein binder in the subset of candidate protein binders. The system can then return to step 306 for a next candidate protein binder in the set of candidate protein binders.

[0143] In some implementations, the system includes the candidate protein binder in the subset of candidate protein binders only if the values of the quality scores for the candidate protein binder satisfy all of the thresholds specific to the quality scores.

[0144] In some examples, the system can perform the steps 302 and 304 for each of the one or more quality scores to determine thresholds prior to performing steps 306-308 for the candidate protein binders.

[0145] In some examples, the system can perform the steps 302 and 304 at multiple iterations to update the thresholds specific to the quality scores. For example, for a first iteration, the system can determine initial thresholds specific to the quality scores. After selecting a subset of candidate protein binders as described with reference to step 210 of FIG. 2, the system can synthesize some or all of the candidate protein binders in the subset. The system can perform physical experiments to assess one or more properties of the candidate protein binders that are relevant to binding. For example, the system can determine an experimental score that characterizes the binding affinity for the candidate protein binders. For the second iteration, the system can include the synthesized candidate protein binders in the example proteins described above with reference to steps 302 and 304 to determine updated thresholds specific to the quality scores. The system can thus update the thresholds in order to make the thresholds more predictive, which allows for the DeepMind Technologies Limited

[0146] F&R Ref.: 45288-0503WO1 PCT Application subsequent selection of subsets of candidate protein binders that are more likely to bind to the target molecule.

[0147] FIG. 4 is a flow diagram of an example process 400 for training a protein design generative model on a set of training examples. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a protein selection system, e.g., the protein selection system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 400.

[0148] The system obtains a set of training examples for training the protein design generative model (step 402).

[0149] In some examples, the system can generate the set of training examples based on the proper subset of the set of candidate protein binders, e.g., that are selected for physical synthesis, as described with reference to step 210 of FIG. 2. For each candidate protein binder selected, the system can generate a synthetic training example that includes (i) a conditioning input that specifies the target molecule, and (ii) a target output that specifies a 3D structure of the candidate protein binder. For example, the target output that specifies the 3D structure of the candidate protein binder can have been generated by the structure prediction machine learning model as described with respect to step 206 of FIG. 2.

[0150] In some examples, the system can obtain the set of training examples from an experimental database or from outputs generated by a structure prediction machine learning model. For example, the target output that specifies a 3D structure of the candidate protein binder can have been determined, e.g., through physical experiments. Public databases with 3D structures of candidate protein binders that have been determined e.g., by x-ray diffraction or electron microscopy are available, such as the Protein Data Bank (wwpdb.org), and many others. Examples of training examples for different tasks are described in more detail below.

[0151] The system trains the protein design generative model on the set of training examples (step 404). In some examples, the system can further train, e.g., fine-tune, the protein design generative model on the set of training examples.

[0152] For example, for each training example in the set of synthetic training examples described above, the system can train the protein design generative model to reduce a discrepancy between: (i) 3D protein structures generated by the protein design generative model when conditioned on the conditioning input of the training example, and (ii) the 3D structure of the candidate protein binder specified by the target output of the training DeepMind Technologies Limited

[0153] F&R Ref.: 45288-0503WO1 PCT Application example. The system can thus train the protein design generative model on diverse and high-quality synthetic training examples.

[0154] In some implementations, the system can train the protein design generative model to perform different protein design tasks. As an example, the system can train the protein design generative model to perform an unconditional generation task. The system can obtain a set of training examples that includes multiple unconditional training examples. For example, each unconditional training example can include a target output that specifies a 3D structure of a molecule or molecule complex. In these examples, the system can train the protein design generative model on an unconditional training example by training the protein design generative model to generate a predicted output that matches the target output of the unconditional training example.

[0155] As another example, the system can train the protein design generative model to perform conditional generation tasks such as structure conditioning, binding, and ligand binding. The system can obtain a set of training examples that include multiple conditional training examples. For example, each conditional training example can include (i) a conditioning input that specifies a portion of a 3D structure of a molecule or molecule complex, and (ii) a target output that specifies all of the 3D structure of the molecule or molecule complex. In these examples, the system can train the protein design generative model on a conditional training example by training the protein design generative model to generate a predicted output that matches the target output of the training example when conditioned on the conditioning input of the conditional training example.

[0156] As an example, the conditioning input can specify one or more chains of the molecule or the molecule complex of the target output. For example, the conditioning input can include the molecule or the molecule complex of the target output, with one or more chains masked. In some examples, the one or more chains can be contiguous. In some examples, the one or more chains can be randomly selected. The target output can specify all of the chains of the molecule or the molecule complex. As a particular example, the conditioning input can specify the secondary structure of the molecule of the target output.

[0157] As another example, the conditioning input can identify the structure of a target molecule and a set of residues representing the desired epitope. The target output can include a binder that interacts with the target molecule at the residues. In some examples, the target output can specify a molecule complex that includes the target molecule and the binder. DeepMind Technologies Limited

[0158] F&R Ref.: 45288-0503WO1 PCT Application

[0159] As another example, the conditioning input can specify one or more functional elements of the molecule or the molecule complex of the target output. For example, the conditioning input can include the molecule or the molecule complex of the target output, with one or more elements that are important to aspects such as stability but are not important to function, masked. The target output can specify all elements of the molecule or the molecule complex.

[0160] As another example, the conditioning input can specify a ligand (e.g., a small molecule ligand or a protein ligand). The target output can specify a protein that binds with the ligand. In some examples, the target output can specify a molecule complex that includes the ligand and the protein that binds with the ligand.

[0161] In some implementations, the set of training examples can include a mixture of different types of training examples, such as synthetic training examples, and training examples for unconditional and conditional tasks. In some implementations, the mixture can include numbers of training examples for particular types according to different weights. For example, for each training iteration, the system can obtain the batch for the training iteration by sampling from the training examples for each task according to a respective probability for the type.

[0162] In some implementations, the system can update the weights for particular types of training examples during training. For example, the system can determine the performance of the protein design generative model during training. In response to determining that the performance does not meet a threshold performance, the system can update the probability of sampling from the training examples for each type of training example according to the updated weights for one or more subsequent training iterations.

[0163] In some implementations, the system filters the set of training examples. For example, the system can exclude training examples from the set if one or more filter scores for the molecule or molecule complex of the target output do not meet one or more threshold filter scores. Each filter score represents a predicted measure of quality for the target output. In examples where the target output specifies a 3D structure of a candidate protein binder that has been generated by the structure prediction machine learning model, the filter score can be based on confidence scores of the structure prediction machine learning model. As another example, the filter score can represent a property of the 3D structure, such as the secondary structure of the candidate protein binder, a stability of the candidate protein binder, or a fraction of residues in different regions of the candidate protein binder such as the core of the candidate protein binder, the surface of the candidate protein binder, or in DeepMind Technologies Limited

[0164] F&R Ref.: 45288-0503WO1 PCT Application between the core and the surface. Thus the system can filter the set of training examples to include high-quality training examples that can be used to train the protein design generative model to generate high-quality outputs.

[0165] FIG. 5 is a flow diagram of an example process 500 for training a protein design generative model to generate a predicted 3D protein structure. For convenience, the process 500 will be described as being performed by a system of one or more computers located in one or more locations. For example, a protein selection system, e.g., the protein selection system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 500.

[0166] The system can train the protein design generative model over a sequence of training iterations.

[0167] At each training iteration, the system can receive one or more training examples for the training iteration (step 502). Each training example can be a conditional training example or an unconditional training example as described above with reference to FIG. 4.

[0168] The system can generate a predicted output for the training iteration (step 504). For example, the system can provide the training example as input to the protein design generative model to generate the predicted output. As an example, the predicted output can specify the structure of a molecule or molecule complex.

[0169] The system can update the parameter values of the protein design generative model (step 506). For example, the system can determine gradients of an objective function (e.g., using backpropagation) and can use the gradients to update the parameter values. For each training example, the objective function can measure an error between: (i) the predicted output generated by the protein design generative model, and (ii) a target output of the training example.

[0170] In some implementations, the system uses Bayesian optimization to optimize hyperparameter values for improved performance. In some implementations, the system tunes the model architecture for improved performance. In some implementations, such as implementations where the protein design generative model includes a diffusion model, the system can use a dynamic uncertainty -weighted loss.

[0171] The system can determine whether training is complete (step 508). If the system determines that training is not complete, the system can continue to a next training iteration (e.g., return to step 502). The system can determine whether training is complete using any of a variety of criteria. For example, the system can determine that training is complete after a pre-determined number of training iterations. As another example, the system can DeepMind Technologies Limited

[0172] F&R Ref.: 45288-0503WO1 PCT Application determine that training is complete when a value of the objective function for the training iteration falls below a pre-determined threshold. As another example, the system can determine that training is complete when a difference between the value of the objective function for the current training iteration and a value of the objective function for a previous training iteration falls below a pre-determined threshold.

[0173] When the system determines that training is complete, the system can return the trained protein design generative model (step 510).

[0174] This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0175] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0176] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated DeepMind Technologies Limited

[0177] F&R Ref.: 45288-0503WO1 PCT Application circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0178] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0179] In this specification the term “engine” is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0180] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0181] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer DeepMind Technologies Limited

[0182] F&R Ref.: 45288-0503WO1 PCT Application will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0183] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0184] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0185] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and compute-intensive parts of machine learning training or production, i.e., inference, workloads.

[0186] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework.

[0187] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data DeepMind Technologies Limited

[0188] F&R Ref.: 45288-0503WO1 PCT Application server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0189] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0190] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0191] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described DeepMind Technologies Limited F&R Ref.: 45288-0503W01 PCT Application program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0192] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. DeepMind Technologies Limited F&R Ref.: 45288-0503WO1 PCT ApplicationCLAIMS1. A method of obtaining a protein binder for a target molecule, the method performed by one or more computers, the method comprising: determining, for each candidate protein binder in a set of candidate protein binders for a target molecule, one or more quality scores for the candidate protein binder, comprising: processing a model input that comprises at least an amino acid sequence of the candidate protein binder using a structure prediction machine learning model to generate a model output that comprises: (i) a predicted 3D structure of a complex that comprises the candidate protein binder and the target molecule, and (ii) a respective confidence score for each of a plurality of atoms in the complex, wherein a confidence score for an atom characterizes a confidence of the structure prediction machine learning model in a position of the atom in the predicted 3D structure of the complex; and determining a quality score of the one or more quality scores for the candidate protein binder based on the respective confidence scores for the atoms in the complex; and selecting a proper subset of the set of candidate protein binders based at least in part on the quality scores.

2. The method of claim 1, wherein selecting the proper subset of the set of candidate protein binders comprises selecting only candidate protein binders that are predicted to bind successfully to the target molecule under real-world experimental conditions.

3. The method of any preceding claim, wherein determining a quality score for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: processing the predicted 3D structure of the complex to identify a set of atoms that are included in an interface between the candidate protein binder and the target molecule; and determining a quality score for the candidate protein binder based only on confidence scores for the set of atoms that are included in the interface.32DeepMind Technologies Limited F&R Ref.: 45288-0503WO1 PCT Application4. The method of any preceding claim, wherein determining a quality score of the one or more quality scores for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: determining a quality score for the candidate protein binder based only on confidence scores for atoms that are included in the candidate protein binder.

5. The method of any preceding claim, wherein determining a quality score of the one or more quality scores for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: determining a quality score for the candidate protein binder based only on confidence scores for atoms that are included in the target molecule.

6. The method of any preceding claim, wherein determining a quality score of the one or more quality scores for the candidate protein binder based on the respective confidence scores for the atoms in the complex comprises: determining a quality score for the candidate protein binder based on confidence scores of both: (i) atoms that are included in the candidate protein binder, and (ii) atoms that are included in the target molecule.

7. The method of any preceding claim, wherein the set of candidate protein binders have been identified using a protein design generative model by performing operations comprising: generating, using the protein design generative model and while the protein design generative model is conditioned on data characterizing the target molecule, a plurality of predicted 3D protein structures, wherein each predicted 3D protein structure defines a respective predicted 3D structure of a candidate protein binder that is predicted to bind to the target molecule.

8. The method of claim 7, further comprising, for each candidate protein binder, determining an amino acid sequence of the candidate protein binder based on the predicted 3D structure of the candidate protein binder that is generated by the protein design generative model.33DeepMind Technologies Limited F&R Ref.: 45288-0503WO1 PCT Application9. The method of any one of claims 7-8, further comprising, for each candidate protein binder in the set of candidate protein binders: determining a second quality score of the one or more quality scores for the candidate protein binder based on a measure of similarity between:(i) the predicted 3D structure of the candidate protein binder as generated by the protein design generative model when conditioned on data characterizing the target molecule, and(ii) the predicted 3D structure of the candidate protein binder as generated by the structure prediction machine learning model by processing a model input that comprises at least the amino acid sequence of the candidate protein binder.

10. The method of any one of claims 7-9, further comprising: generating a set of training examples for training the protein design generative model based on the proper subset of the set of candidate protein binders, comprising, for each candidate protein binder in the proper subset: generating a training example that comprises: (i) a conditioning input that specifies the target molecule, and (ii) a target output that specifies a 3D structure of the candidate protein binder; and training the protein design generative model on the set of training examples.

11. The method of claim 10, wherein for each training example in the set of training examples, training the protein design generative model on the training example comprises: training the protein design generative model to reduce a discrepancy between: (i) 3D protein structures generated by the protein design generative model when conditioned on the conditioning input of the training example, and (ii) the 3D structure of the candidate protein binder specified by the target output of the training example.

12. The method of any one of claims 7-11, wherein the protein design generative model has been trained on a set of training examples that comprises a plurality of unconditional training examples, wherein: each unconditional training example comprises a target output that specifies a 3D structure of a molecule or molecule complex; and training the protein design generative model on an unconditional training example comprises training the protein design generative model to generate a predicted output that matches the target output of the unconditional training example.DeepMind Technologies Limited F&R Ref.: 45288-0503WO1 PCT Application13. The method of any one of claims 7-12, wherein the protein design generative model has been trained on a set of training examples that comprises a plurality of conditional training examples, wherein: each conditional training example comprises: (i) a conditioning input that specifies a portion of a 3D structure of a molecule or molecule complex, and (ii) a target output that specifies all of the 3D structure of the molecule or molecule complex; and training the protein design generative model on a conditional training example comprises training the protein design generative model to generate a predicted output that matches the target output of the training example when conditioned on the conditioning input of the conditional training example.

14. The method of any preceding claim, further comprising, for each candidate protein binder in the set of candidate protein binders: determining a predicted stability of the candidate protein binder; and determining a third quality score of the one or more quality scores for the candidate protein binder based at least in part on the predicted stability of the candidate protein binder.

15. The method of any preceding claim, wherein selecting a proper subset of the set of candidate protein binders based at least in part on the quality scores comprises, for one or more of the quality scores: determining, for each candidate protein binder in the set of candidate protein binders, that the candidate protein binder is eligible to be selected for the proper subset only if a value of the quality score for the candidate protein binder satisfies a threshold that is specific to the quality score.

16. The method of claim 15, wherein for one or more of the quality scores, the threshold that is specific to the quality score has been determined by performing operations comprising: determining, for each a plurality of example proteins: (i) a value of the quality score for the example protein, and (ii) a value of an experimental score for the example protein, wherein the experimental score is evaluated by physically synthesizing the example protein and performing an experiment to measure a value of a property of the example protein; and determining the threshold that is specific to the quality score based on the values of the quality score and the experimental score for the plurality of example proteins.DeepMind Technologies Limited F&R Ref.: 45288-0503W01 PCT Application17. The method of claim 16, wherein for each of the plurality of example proteins, the value of the experimental score for the example protein characterizes a binding affinity of the example protein for an example target molecule.

18. The method of any one of claims 16-17, wherein determining the threshold that is specific to the quality score based on the values of the quality score and the experimental score for the plurality of example proteins comprises: determining the threshold based on a correlation between the values of the quality score and the values of the experimental score for the plurality of example proteins.

19. The method of any preceding claim, wherein the target molecule comprises a protein, or a small molecule, or a ribonucleic acid (RNA), or a deoxyribonucleic acid (DNA) molecule.

20. The method of any preceding claim, wherein the set of candidate protein binders comprises at least 100,000 candidate protein binders.

21. The method of any preceding claim, wherein selecting the proper subset of the set of candidate protein binders based at least in part on the quality scores comprises: selecting fewer than 1% of the candidate protein binders in the set of candidate protein binders for the proper subset.

22. The method of any preceding claim, wherein the structure prediction machine learning model has been trained to process a model input that includes an amino acid sequence of a protein to predict a 3D structure of the protein or a 3D structure of a complex that includes the protein.

23. The method of any preceding claim, wherein the target molecule comprises a receptor or enzyme, in particular an industrial enzyme , and wherein each quality score is indicative of whether the candidate protein binder is an agonist or an antagonist of the receptor or enzyme.

24. The method of any preceding claim, wherein the target molecule comprises an antibody target, in particular a virus or cancer cell protein, and wherein each quality score predicts whether the candidate protein binder binds to the antibody target to provide a therapeutic effect.36DeepMind Technologies Limited F&R Ref.: 45288-0503WO1 PCT Application25. The method of any preceding claim, wherein the method is for obtaining a protein binder that is a diagnostic antibody marker of a disease.

26. The method of any preceding claim, further comprising physically synthesizing some or all of the candidate protein binders selected for the proper subset.

27. The method of any preceding claim, further comprising, for each candidate protein binder selected for the proper subset: performing physical experiments to assess one or more properties of the candidate protein binder, in particular in vitro or in vivo.

28. The method of claim 27, wherein for each candidate protein binder selected for the proper subset, performing physical experiments to assess one or more properties of the candidate protein binder comprises: performing physical experiments, in particular in vitro or in vivo, to assess one or more of: a binding affinity of the candidate protein binder for the target molecule; or a stability of the candidate protein binder; or absorption properties of the candidate protein binder; or distribution properties of the candidate protein binder; or metabolism properties of the candidate protein binder; or excretion properties of the candidate protein binder; or toxicity properties of the candidate protein binder.

29. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of claims 1-25.

30. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of claims 1-25.37

Images