Predicting the joint three-dimensional (3D) structure of a protein and a ligand by co-folding

Abstract

Methods, systems, and apparatus for generating predicted joint 3D structures of proteins and one or more ligands, comprising computer programs encoded on a computer storage medium. In one aspect, a method includes: obtaining network input characterizing the protein and the one or more ligands; processing the network input using an embedded neural network to generate protein-ligand embeddings of the protein and the one or more ligands; and using a generative model and, when the generative model is conditioned on the protein-ligand embeddings, generating the predicted joint three-dimensional (3D) structure of the protein and the one or more ligands.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 546,444, filed October 30, 2023, and U.S. Provisional Application No. 63 / 611,638, filed December 18, 2023. The disclosure of the earlier applications is considered part of the disclosure of this application and is incorporated herein by reference. Background Technology

[0003] This specification relates to predicting the combined three-dimensional (3D) structure of a protein and one or more ligands.

[0004] Machine learning models can be used for prediction. A machine learning model takes input and generates an output, such as a predicted output, based on the received input. Some machine learning models are parametric models and generate an output based on the received input and the values ​​of the model parameters. Some machine learning models are deep models, which employ multiple layers to generate an output for the received input. For example, a deep neural network is a deep machine learning model that contains an output layer and one or more hidden layers, each of which applies a non-linear transformation to the received input to generate an output. Summary of the Invention

[0005] This specification describes a system implemented as a computer program on one or more computers at one or more locations, capable of predicting the combined 3D structure of a protein and one or more ligands.

[0006] The term "protein" can be understood as any biomolecule specified by a sequence (or "chain") of one or more amino acids. For example, the term protein can refer to a protein domain, such as a portion of a protein's amino acid chain, which can fold almost independently of the rest of the protein. As another example, the term protein can refer to a protein complex, which consists of multiple chains of amino acids that fold together to form a protein structure.

[0007] A "ligand" can refer to a molecule or compound that binds to a target molecule (such as a protein). Ligands can include, for example, small organic molecules, complex organic molecules, proteins, biomolecules, and so on.

[0008] Multiple sequence alignment (MSA) of an amino acid chain in a protein refers to aligning the sequence of the stated amino acid chain with multiple other amino acid chains (e.g., amino acid chains from other proteins, such as homologous proteins). More specifically, an MSA can define the correspondence between positions in an amino acid chain and corresponding positions in multiple other amino acid chains. For example, an MSA of an amino acid chain can be generated by processing an amino acid chain database using any suitable computational sequence alignment technique (e.g., progressive alignment construction). The amino acid chains in an MSA can be understood as having evolutionary relationships; for example, each amino acid chain in the MSA may share a common prototype. In an MSA of an amino acid chain, the correlations between amino acids in the amino acid chain can encode information related to predicting the structure of the amino acid chain.

[0009] A "binding pocket" on a protein can refer to a specific three-dimensional cavity or gap in the protein structure where a ligand can bind. In some cases, a binding pocket can be understood as a "lock," whose shape and chemical properties match the ligand, which acts as the "key." In other cases, such as due to structural differences or slight mismatches in shape or chemical groups, the ligand may initially not fit perfectly into the binding pocket, but conformational changes during binding can make the interaction between the ligand and the binding pocket more complementary and specific, for example, in induced fit binding. Examples of binding pockets include ortho-binding pockets, allosteric binding pockets, and cryptic binding pockets.

[0010] If the first neural network is contained within the second neural network, then the first neural network can be referred to as a "subnetwork" of the second neural network.

[0011] In neural networks, a “block” (e.g., a “self-attention block”) can refer to a group of one or more neural network layers.

[0012] "Embedding" of entities (such as atoms, ligands, or proteins) can refer to representing entities as ordered sets of numerical values, such as vectors, matrices, or other numerical tensors.

[0013] "Conditioning" a model (e.g., a generative model), neural network (e.g., a denoising neural network), or operation (e.g., a self-attention operation) with conditional data (e.g., an embedding of a protein and one or more ligands) can mean providing the conditional data as input (e.g., auxiliary input) to the model, neural network, or operation such that the output generated by the model, neural network, or operation is influenced by the conditional data (depending on the conditional data).

[0014] The "binding affinity" of a ligand to a protein refers to the strength or degree of attraction between the two proteins when they interact to form a complex. Binding affinity can be determined experimentally, for example, by measuring the equilibrium constant or free energy of ligand-protein association.

[0015] The 3D spatial position of an atom can be represented by a set of coordinates in an appropriate coordinate system, such as a 3D Cartesian coordinate system or a spherical coordinate system.

[0016] According to a first aspect, a method executed by one or more computers is provided, the method comprising: obtaining network input characterizing a protein and one or more ligands; processing the network input characterizing the protein and the one or more ligands using an embedded neural network to generate a protein-ligand embedding of the protein and the one or more ligands; and generating a predicted joint three-dimensional (3D) structure of the protein and the one or more ligands using a generative model and, when the generative model is conditioned on the protein-ligand embedding, wherein the predicted joint 3D structure of the protein and the one or more ligands defines a corresponding predicted three-dimensional spatial location for each atom in the protein and for each atom in each of the one or more ligands.

[0017] In some implementations, the embedding neural network includes a protein embedding neural network and a ligand embedding neural network, and wherein processing the network input characterizing the protein and the one or more ligands using the embedding neural network to generate the protein-ligand embedding includes: processing data characterizing the protein using the protein embedding neural network to generate a protein embedding of the protein; processing data characterizing the one or more ligands using the ligand embedding neural network to generate ligand embeddings of the one or more ligands; and processing the protein embedding and the ligand embedding to generate the protein-ligand embedding.

[0018] In some implementations, the protein embedding includes the corresponding amino acid embedding of each amino acid in the protein; wherein the ligand embedding includes the corresponding atomic embedding of each atom in each of the one or more ligands; and processing the protein embedding and the ligand embedding to generate the protein-ligand embedding includes generating data defining 1D amino acid embedding and atomic embedding sequences by concatenating the following: (i) the amino acid embedding of the protein embedding, and (ii) the atomic embedding of the ligand embedding; wherein the protein-ligand embedding is derived from the 1D amino acid embedding and atomic embedding sequences.

[0019] In some implementations, processing the protein embedding and the ligand embedding to generate the protein-ligand embedding further includes converting the 1D amino acid embedding and atom embedding sequences into a two-dimensional (2D) embedding array; wherein the protein-ligand embedding is derived from the 2D embedding array.

[0020] In some implementations, the 2D embedding array includes multiple atom-atom embeddings, each atom-atom embedding derived from a corresponding pair of atom embeddings of the ligand embedding.

[0021] In some implementations, the 2D embedding array includes multiple amino acid-amino acid embeddings, each amino acid-amino acid embedding derived from a corresponding pair of amino acid embeddings of the protein embedding.

[0022] In some implementations, the 2D embedding array includes multiple amino acid-atom embeddings, each amino acid-atom embedding derived from: (i) the corresponding atom embedding of the ligand embedding, and (ii) the corresponding amino acid embedding of the protein embedding.

[0023] In some implementations, converting the 1D amino acid embedding and atom embedding sequences into the 2D embedding array includes: applying an outer product operation to the 1D amino acid embedding and atom embedding sequences; or applying a 2D linking operation to the 1D amino acid embedding and atom embedding sequences.

[0024] In some implementations, the embedding neural network further includes a fusion neural network; and processing the protein embedding and the ligand embedding to generate the protein-ligand embedding further includes: processing the 2D embedding array using the fusion neural network to generate an updated 2D embedding array; wherein the updated 2D embedding array defines the protein-ligand embedding.

[0025] In some implementations, the fusion neural network includes a sequence of self-attention blocks, each configured to perform an operation including applying one or more self-attention operations to an input 2D embedding array to update the input 2D embedding array.

[0026] In some implementations, for one or more of the self-attention blocks, the self-attention operation includes one or more line-by-line self-attention operations.

[0027] In some implementations, for one or more of the self-attention blocks, the self-attention operation includes one or more column-by-column self-attention operations.

[0028] In some implementations, for one or more of the self-attention blocks, the self-attention operation includes one or more triangular self-attention operations.

[0029] In some implementations, the network input includes data defining one or more of the following: the amino acid sequence of the protein; multiple sequence alignment (MSA) of the protein; the corresponding structure of each of the one or more template proteins; and a representation of the corresponding chemical structure of each of the one or more ligands.

[0030] In some implementations, the generative model is a generative diffusion model that includes a denoising neural network.

[0031] In some implementations, using the generative model and generating the predicted joint 3D structure of the protein and the one or more ligands, conditioned on the protein-ligand embedding, includes: generating positional data defining the corresponding initial positions of each atom in the complex comprising the protein and the one or more ligands; using the denoising neural network and denoising the positional data within a time-step sequence, conditioned on the protein-ligand embedding; wherein the predicted joint 3D structure of the protein and the one or more ligands is defined by the positional data following the last time step in the time-step sequence.

[0032] In some implementations, generating positional data that defines the corresponding initial position of each atom in the complex includes sampling the corresponding initial position of each atom in the complex from a probability distribution in 3D space.

[0033] In some implementations, denoising the positional data within the time step sequence using the denoising neural network and when the denoising neural network is conditioned on the protein-ligand embedding includes, at each of one or more time steps in the time step sequence: receiving current positional data defining the corresponding current position of each atom in the complex at that time step; generating a denoised output using the denoising neural network and when the denoising neural network is conditioned on the protein-ligand embedding; and generating positional data defining the corresponding position of each atom in the complex at the next time step using the denoised output.

[0034] In some implementations, the denoising output includes the corresponding prediction error for each atom in the complex at the current position of the time step.

[0035] In some implementations, using the denoising neural network and generating the denoised output conditioned on the protein-ligand embeddings includes: generating a set of atomic embeddings using an encoder block of the denoising neural network, wherein each atomic embedding represents one or more atoms in the complex and is based at least in part on the corresponding current spatial position of the one or more atoms at the time step; processing the set of atomic embeddings using an update block of the denoising neural network to generate an updated set of atomic embeddings; and processing the updated set of atomic embeddings to generate the denoised output.

[0036] In some implementations, the set of atomic embeddings includes a corresponding atomic embedding representing each atom contained in each of the one or more ligands.

[0037] In some implementations, the atom embedding set contains a corresponding atom embedding representing each atom contained in each amino acid of the protein.

[0038] In some implementations, for each amino acid in the protein, the set of atomic embeddings includes a corresponding atomic embedding that jointly represents all the atoms contained in the amino acid.

[0039] In some implementations, each atomic embedding in the set of atomic embeddings is based on: (i) the current position of the atom at the time step; and (ii) the corresponding conditional embedding of the atom selected from the set of embeddings included in the protein-ligand embeddings.

[0040] In some implementations, for each atom embedding representing an atom in the complex contained in the ligand, the conditional embedding of the atom includes an atom-to-atom embedding corresponding to the atom in the protein-ligand embedding.

[0041] In some implementations, for each atom embedding of an atom in the complex representing an amino acid of the protein, the conditional embedding of the atom includes an amino acid-amino acid embedding corresponding to the amino acid in the protein-ligand embedding.

[0042] In some implementations, the update block of the denoising neural network includes a sequence of self-attention blocks; wherein each self-attention block is configured to apply one or more self-attention operations to the current set of atom embeddings to update the current set of atom embeddings; wherein each of the one or more self-attention operations is conditioned on the protein-ligand embedding.

[0043] In some implementations, applying a self-attention operation to the current set of atomic embeddings to update the current set of atomic embeddings includes: generating a corresponding intermediate attention score for each pair of current atomic embeddings in the current set of atomic embeddings based on the current set of atomic embeddings; generating a corresponding attention score deviation for each pair of current atomic embeddings in the current set of atomic embeddings based on the protein-ligand embeddings; generating a corresponding final attention score for each pair of current atomic embeddings in the current set of atomic embeddings based on the intermediate attention score and the attention score deviation; and updating the current set of atomic embeddings using the final attention score.

[0044] In some implementations, generating the attention score deviation for each pair of current atomic embeddings in the current atomic embedding set includes adding the intermediate attention score for the pair of current atomic embeddings to the attention score deviation for the pair of current atomic embeddings.

[0045] In some implementations, generating the attention score bias for each pair of current atomic embeddings in the current atomic embedding set includes processing a corresponding conditional embedding selected from the set of embeddings included in the protein-ligand embeddings using a projective neural network to generate the attention score bias.

[0046] In some implementations, for each pair of current atom embeddings comprising (i) a first atom embedding representing a first atom contained in a ligand and (ii) a second atom embedding representing a second atom contained in a ligand, the selected conditional embedding comprises an atom-to-atom embedding corresponding to the first atom and the second atom in the protein-ligand embedding.

[0047] In some implementations, for each pair of current atom embeddings comprising (i) a first atom embedding representing a first atom contained in an amino acid and (ii) a second atom embedding representing a second atom contained in a ligand, the selected conditional embedding comprises an amino acid-atom embedding corresponding to the amino acid containing the first atom and (ii) the second atom in the protein-ligand embedding.

[0048] In some implementations, for each pair of current atom embeddings comprising (i) a first atom embedding representing a first atom contained in a first amino acid and (ii) a second atom embedding representing a second atom contained in a second amino acid, the selected conditional embedding comprises an amino acid-amino acid embedding corresponding to (i) the first amino acid and (ii) the second amino acid in the protein-ligand embedding.

[0049] In some implementations, for each pair of current atom embeddings comprising (i) a first atom embedding jointly representing all atoms in a first amino acid and (ii) a second atom embedding representing all atoms in a second amino acid, the selected conditional embedding comprises an amino acid-amino acid embedding corresponding to (i) the first amino acid and (ii) the second amino acid in the protein-ligand embedding.

[0050] In some implementations, for each pair of current atom embeddings comprising (i) a first atom embedding jointly representing all atoms in an amino acid and (ii) a second atom embedding representing atoms contained in a ligand, the selected conditional embedding comprises an amino acid-atom embedding corresponding to the amino acid and (ii) atoms in the protein-ligand embedding.

[0051] In some implementations, the method further includes: selecting one or more ligands to be physically synthesized based at least in part on the predicted joint 3D structure of the protein and the one or more ligands; and physically synthesizing the selected ligands.

[0052] In some implementations, the method further includes: selecting the protein to be physically synthesized based at least in part on the predicted joint 3D structure of the protein and the one or more ligands; and physically synthesizing the protein.

[0053] According to another aspect, a method is provided, comprising: for each ligand in a set of ligands, generating a corresponding predicted 3D structure of the ligand and a protein using the method described herein; for each ligand in the set of ligands, determining a corresponding predicted binding affinity of the ligand to the protein based on the predicted 3D structure of the ligand and the protein; and selecting one or more ligands from the set of ligands for physical synthesis, at least in part based on the predicted binding affinity.

[0054] In some implementations, the method further includes physically synthesizing one or more selected ligands.

[0055] According to another aspect, a method is provided, comprising: for each protein in a protein set, generating a corresponding predicted joint 3D structure of the protein and a ligand using the method described herein; for each protein in the protein set, determining a corresponding predicted binding affinity of the ligand to the protein based on the predicted joint 3D structure of the ligand and the protein; and selecting one or more proteins from the protein set for physical synthesis based at least in part on the predicted binding affinity.

[0056] In some implementations, the method further includes physically synthesizing one or more selected proteins.

[0057] According to another aspect, a system is provided, comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations of the methods described herein.

[0058] According to another aspect, a non-transient computer storage medium is provided that stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the methods described herein.

[0059] According to another aspect, a method for obtaining a ligand is provided, wherein the ligand is a ligand for a drug or an industrial enzyme, the method comprising: for each of one or more candidate ligands: (a) performing the method of any one of claims 1 to 35 to determine a predicted structure of a complex comprising a target protein molecule and the candidate ligand; and (b) evaluating the interaction between the candidate ligand and the target protein molecule based on the predicted structure; and selecting one or more candidate ligands from the candidate ligands as the ligand based on the result of the evaluation.

[0060] In some implementations, the target protein molecule includes a receptor or an enzyme, and the ligand therein is an agonist or antagonist of the receptor or enzyme.

[0061] In some implementations, the ligand is a drug, and the method includes: performing steps (a) and (b) on each of a plurality of target protein molecules; and selecting one or more candidate ligands from the candidate ligands as the ligand to i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

[0062] In some implementations, the ligand comprises an antibody or aptamer, and the target protein molecule comprises an antibody or aptamer target, particularly a viral or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect.

[0063] In some implementations, the ligand is a polypeptide ligand, a polynucleotide ligand, or a polynucleotide ligand.

[0064] According to another aspect, a method for obtaining a diagnostic antibody or aptamer marker for a disease is provided, the method comprising: selecting a target protein molecule; for each of one or more candidate antibodies or aptamers: performing the method of any one of claims 1 to 35 to determine a predicted structure of a complex comprising the candidate antibody or aptamer and the target protein molecule; and evaluating the interaction between the candidate antibody or aptamer and the target protein molecule; and selecting one of the one or more candidate antibodies or aptamers as the diagnostic antibody or aptamer marker based on the result of the evaluation.

[0065] In some implementations, evaluating the interaction of one of the candidate ligands includes determining an interaction score for the candidate ligand, wherein the interaction score includes a measure of the interaction between the candidate ligand and the target molecule.

[0066] In some implementations, the method further includes synthesizing the ligand or diagnostic antibody or aptamer marker.

[0067] In some implementations, the method further includes testing the bioactivity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

[0068] According to another aspect, a method is provided for determining the structure of a molecular complex comprising a protein and one or more ligands, comprising: applying experimental techniques to a physical sample comprising the molecular complex to measure an experimental signal based on the structure of the molecular complex; performing the method of any one of claims 1 to 35 to determine a predicted structure of the molecular complex; and using the experimental signal and the predicted structure of the molecular complex to determine the structure of the molecular complex.

[0069] In some implementations, the experimental techniques include one or more of the following: X-ray crystallography, nuclear magnetic resonance, and electron microscopy.

[0070] According to another approach, a ligand is provided that is synthesized by performing the method described herein.

[0071] According to another aspect, one or more non-transient computer storage media are provided for storing ligand data defining ligands, wherein the ligands are selected from a set of candidate ligands by performing operations including: for each candidate ligand in the set of candidate ligands, generating a corresponding predicted joint 3D structure of the candidate ligand and a protein using the methods described herein; for each candidate ligand in the set of candidate ligands, determining a corresponding predicted binding affinity of the candidate ligand to the protein based on the predicted joint 3D structure of the candidate ligand and the protein; and selecting one or more candidate ligands from the set of ligands for physical synthesis, at least in part based on the predicted binding affinity.

[0072] According to another aspect, a protein is provided that is synthesized by performing the methods described herein.

[0073] According to another aspect, one or more non-transient computer storage media are provided that store ligand data defining proteins, wherein the proteins are selected from a set of proteins by performing operations including: for each protein in the set of proteins, generating a corresponding predicted joint 3D structure of the protein and the ligand using the methods described herein; for each protein in the set of proteins, determining a corresponding predicted binding affinity of the ligand to the protein based on the predicted joint 3D structure of the ligand and the protein; and selecting one or more proteins from the set of proteins for physical synthesis, at least in part based on the predicted binding affinity.

[0074] Specific implementations of the subject matter described in this specification may be carried out to achieve one or more of the following advantages.

[0075] Traditionally, molecular docking of a protein with one or more ligands involves obtaining data defining the corresponding 3D structures of the protein and one or more ligands, and searching within the possible orientation space (i.e., spatial configurations) of the protein and one or more ligands to optimize a scoring function. The scoring function measures, for example, the energy of each joint conformation of the protein and one or more ligands. Traditional molecular docking can be computationally expensive, for example, because optimizing the scoring function requires searching a large possible orientation space of the protein and one or more ligands. Furthermore, traditional molecular docking requires prior knowledge of the corresponding 3D structures of the protein and one or more ligands, as well as the binding sites on the protein. Moreover, even if the 3D structure of the protein is known (e.g., through crystallographic methods), the 3D structure of the protein may be deformed through conformational changes when the ligand interacts with the protein (e.g., by binding to the binding sites on the protein). However, traditional molecular docking does not account for the conformational changes that may occur due to protein-ligand interactions, which can lead to inaccurate results.

[0076] The cofolding system described in this specification addresses these problems by directly mapping data to predicted joint 3D structures of proteins and one or more ligands: (i) data defining the protein, such as its amino acid sequence and MSA; and (ii) data defining one or more ligands, such as their chemical structures. The cofolding system does not require prior knowledge of the 3D structure of the protein or one or more ligands, nor does it require prior knowledge of the binding sites on the protein. To generate predicted joint 3D structures of proteins and one or more ligands, the cofolding system can generate protein-ligand embeddings of the protein and one or more ligands. The system can then use the protein-ligand embeddings to modulate a generative model that can generate one or more predicted joint 3D structures of proteins and one or more ligands by sampling from a distribution of possible 3D structure spaces.

[0077] Compared to traditional methods such as molecular docking, cofolding systems are computationally more efficient (specifically, they consume fewer computational resources, such as memory and computing power). More specifically, performing molecular docking requires optimizing a scoring function by searching a large possible pose space of the protein and one or more ligands. In contrast, cofolding systems directly map protein and ligand data to a predicted joint 3D structure through a single forward propagation of a neural network, completely avoiding the iterative search process required for molecular docking.

[0078] Embedding neural networks can generate protein-ligand embeddings by generating protein embeddings (containing the corresponding amino acid embeddings for each amino acid in the protein) and ligand embeddings (containing the corresponding atom embeddings for each atom in each of one or more ligands). The embedding neural network can then generate protein-ligand embeddings, for example, by converting the amino acid embeddings and atom embeddings into a two-dimensional (2D) embedding array, which captures the pairwise relationships between the amino acids of the protein and the atoms of the ligands. The embedding neural network can further refine the protein-ligand embeddings, for example, by performing a sequence of self-attention operations on the 2D embedding array to enrich each embedding with information encoded in other embeddings. Thus, the embedding neural network generates protein-ligand embeddings in a way that leverages useful inductive biases for the current problem, particularly since the pairwise relationships between ligand atoms and amino acids encode information that helps predict their joint structure. Furthermore, protein-ligand embeddings originate from information encoded at multiple resolution levels: information characterizing the protein is expressed at a lower resolution of amino acids, while information characterizing one or more ligands is expressed at a higher resolution of atoms. Representing proteins at a lower resolution level of amino acids (rather than at a higher resolution level of atoms) can reduce computational resource consumption, for example, by reducing the number of self-attention operations computed when generating protein-ligand embeddings.

[0079] When conditioned on protein-ligand embedding, generative models can generate any desired number of predicted 3D structures of a protein and one or more ligands by repeatedly sampling from a distribution of possible (joint) 3D structural spaces. Furthermore, generative models can be configured through training to learn how to interpret conformational changes in the protein during binding, in contrast to traditional molecular docking methods that assume the protein and ligand maintain static, predefined structures. That is, generative models can be trained using training examples, each corresponding to a protein and a corresponding complex of one or more ligands bound to the protein, for example, where one or more ligands each bind to a corresponding binding site on the protein.

[0080] The co-folding system can be used for a wide variety of drug discovery and drug reuse tasks, as will be described in detail in this manual.

[0081] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the detailed description, the drawings, and the claims. Attached Figure Description

[0082] Figure 1 An example co-folding system is shown.

[0083] Figure 2 An example embedded neural network is shown.

[0084] Figure 3A This is a flowchart of an example process that uses a fusion neural network to process the protein embedding of a protein and the ligand embedding of one or more ligands to generate a protein-ligand embedding that jointly represents the protein and one or more ligands.

[0085] Figure 3B The operation performed by the cofolding system to generate protein-ligand intercalation is shown.

[0086] Figure 4 This is a flowchart of an example process for generating a predicted joint 3D structure of a protein and one or more ligands using a generative diffusion model that includes a denoising neural network.

[0087] Figure 5 This is a flowchart of an example process for predicting the corresponding error in the current 3D spatial position of each atom in a complex using a denoised neural network conditioned on protein-ligand embedding.

[0088] Figure 6This is a flowchart of an example process for updating the current set of atom embeddings using a self-attention operation, which is implemented by a self-attention block of a denoising neural network and conditioned on protein-ligand embeddings.

[0089] Figure 7 This is a flowchart illustrating an example process of jointly training a generative model of an embedded neural network and a co-folding system.

[0090] Figure 8 This is a flowchart illustrating an example process for jointly training an embedded neural network and generating a diffusion model on a training example.

[0091] Figure 9 An example of the output generated by the cofolding system is shown, demonstrating the predicted 3D structure of the protein and two ligands.

[0092] The same reference numerals and symbols in the various figures denote the same elements. Detailed Implementation

[0093] Figure 1 An example co-folding system 100 is shown. The co-folding system 100 is an example of a system implemented as a computer program on one or more computers at one or more locations, in which the systems, components and techniques described below are implemented.

[0094] The cofolding system 100 is configured to process data 102 characterizing proteins (“protein data”) and data 104 characterizing one or more ligands (“ligand data”) to generate a predicted joint 3D structure 110 of a “complex” (i.e., a molecular system) comprising the protein and one or more ligands. The predicted joint 3D structure 110 defines a corresponding predicted 3D spatial position for each atom in the complex (i.e., each atom in the protein and each atom in each of the one or more ligands). The predicted joint 3D structure can define the structure of the complex in which one or more ligands bind to corresponding binding sites on the protein.

[0095] Protein data 102 may contain any suitable data characterizing the protein, such as data defining one or more amino acid sequences of the protein, or data defining the MSA of the protein, or data characterizing the corresponding structure of each of one or more "template" proteins, or combinations thereof. A template protein may refer to a protein "similar" to protein 102, for example, such that a similarity metric between the template protein and protein 102 satisfies (e.g., exceeds) a certain threshold (e.g., 0.8, 0.9, 0.99, or any other suitable threshold). Any suitable similarity metric may be used to measure the similarity between the first protein and the second protein, such as a sequence identity or percentage identity similarity metric between the corresponding amino acid sequences of the first and second proteins. The structure of the template protein may be represented in any suitable manner, for example, by contact diagrams, or by data defining the corresponding 3D spatial positions of each atom in the template protein. Optionally, protein data 102 may exclude any data that directly defines the 3D structure of the protein, such as the 3D spatial positions of atoms or amino acid residues in a protein 3D conformation.

[0096] The ligand data 104 may contain any appropriate data characterizing each ligand in a set of one or more ligands. For example, for each ligand, the ligand data 104 may contain a corresponding textual representation of one or more of the following: the chemical structure of the ligand (e.g., the arrangement of atoms and bonds in the ligand), the types of atoms in the ligand and how they are connected, the chirality of the bonds in the ligand, or any functional groups contained in the ligand (e.g., hydroxyl, amino, carboxyl, etc.). The corresponding textual representation of each ligand may contain, for example, a simplified molecular linear input specification (SMILES) string characterizing the ligand. As another example, for each ligand, the ligand data 104 may contain a corresponding representation of the ligand by means of graph data representing a graph, for example, nodes in the graph representing atoms in the ligand, and edges in the graph representing bonds between atoms in the ligand. Optionally, the ligand data 104 may exclude any data that directly defines the 3D structure of each ligand, such as the 3D spatial positions of atoms in the ligand's 3D conformation.

[0097] The ligand data 104 may contain data characterizing any appropriate number of ligands. In some cases, the ligand data 104 characterizes one ligand. In other cases, the ligand data characterizes more than one ligand, such as two, three, four, or five ligands.

[0098] The cofolding system 100 uses an embedded neural network 200 and a generative model 108 to process protein data 102 and ligand data 104 to generate a predicted joint 3D structure 110 of the complex. The embedded neural network 200 and the generative model 108 will be described in more detail below (and throughout this specification).

[0099] The embedded neural network 200 is configured to process protein data 102 and ligand data 104 to generate protein-ligand embeddings 106 representing proteins and one or more ligands.

[0100] The embedded neural network 200 can have any suitable neural network architecture, enabling it to perform the functions described herein. Specifically, the embedded neural network 200 can contain any suitable type of neural network layers (e.g., fully connected layers, convolutional layers, attention layers, etc.) with any suitable number of layers (e.g., 5, 10, or 20 layers) and connected in any suitable configuration (e.g., directed graphs as layers). Reference Figure 2 Specific examples of possible architectures for embedding neural networks 200 are described in more detail.

[0101] Generative model 108 is configured to generate one or more predicted joint 3D structures 110 of the complex when conditioned on protein-ligand embedding 106. Generative model 108 can be any suitable conditional generative model. More specifically, generative model 108 can be any suitable model that can generate samples from the distribution of possible joint 3D structure spaces of the complex when conditioned on protein-ligand embedding 106. For example, generative model 108 can be implemented as a generative diffusion model, a generative adversarial neural network (GAN) model, a flow-based neural network model (normalized flow model), etc. (References) Figures 4 to 6 A sample process for generating predictive joint 3D structures using a generative diffusion model is described in detail.

[0102] In some implementations, the generative model 108 can be implemented as an equivariant diffusion model, for example, configured to perform operations that preserve symmetry (such as rotation and translation invariance). An example of an equivariant diffusion model is described with reference to "Equivariant diffusion for molecule generation in 3d" by Hoogeboom, Emil, et al. (International Conference on Machine Learning, PMLR, 2022).

[0103] Optionally, the generative model 108 can generate multiple different predicted joint 3D structures of the complex. Specifically, the generative model 108 can generate multiple samples from the spatial distribution of possible joint 3D structures of the complex. The differences between the predicted joint 3D structures generated by the generative model 108 can reflect both the uncertainty of the predicted structure and the various structural patterns (e.g., conformations) of the complex containing the protein and one or more ligands.

[0104] The co-folding system 100 can be trained using appropriate machine learning training techniques, jointly training the embedding neural network 200 and the generative model 108 on a training dataset. The training data may contain a set of training examples, where each training example corresponds to a complex of a protein and one or more ligands, for example, where one or more ligands each bind to a corresponding binding site on the protein. (Reference) Figure 7 The example process of jointly training the embedded neural network 200 and the generative model 108 is described in more detail.

[0105] The cofolding system 100 can receive protein data 102 and ligand data 104 from any suitable source (e.g., a user or another system) through an appropriate interface (e.g., an application programming interface (API) or a user interface (e.g., a graphical user interface)). After generating the predicted co-3D structure 110, the cofolding system 100 can, for example, store the data defining the predicted co-3D structure 110 in memory, transmit the data defining the predicted co-3D structure 110 via a data communication network, or directly provide the data defining the predicted co-3D structure 110 to a system performing downstream processing based on the predicted co-3D structure 110.

[0106] The predicted joint 3D structure 110 generated by the co-folding system 100 can be used for any and all possible downstream applications. Several examples of downstream applications that process the predicted joint 3D structure 110 generated by the co-folding system 100 will be described below.

[0107] For example, the predicted 3D joint structure 110 (or features derived from the predicted 3D joint structure 110) can be processed by a scoring function to generate predicted binding affinity of one or more ligands to the protein. In some cases, the scoring function can generate predicted binding affinity based on factors such as electrostatic interaction factors, van der Waals force factors, hydrophobic interaction factors, lipophilic interaction factors, etc., all derived from the predicted 3D joint structure. In some cases, the scoring function can be a machine learning model configured to process the predicted 3D joint structure (or features derived from the predicted 3D joint structure) according to the values ​​of the machine learning model parameter set to generate predicted binding affinity.

[0108] In some cases, the predicted conjugate 3D structure 110 generated by the cofolding system 100 can be used for drug discovery. Drug discovery may involve identifying specific molecules in the body involved in disease processes in humans or animals. These molecules are typically proteins, such as enzymes, receptors, or signaling proteins, which play a crucial role in the development or progression of disease. Ligands (typically small molecules, such as those with a molecular weight equal to or less than 900 Daltons), peptides, or antibodies can be selected to specifically bind to the identified target protein. When a drug containing the ligand is administered to a patient, the ligand can bind to the target protein with high affinity, thereby contributing to a therapeutic effect in the patient. For example, if the target protein is an enzyme involved in the disease process, the ligand can inhibit its activity, thereby disrupting the disease pathway. More generally, the interaction between the ligand and the target protein can activate, inhibit, or alter the function of the target protein, thereby achieving a therapeutic effect. For example, the ligand can be an agonist or antagonist of the target protein receptor.

[0109] Therefore, identifying (e.g., characterizing or defining) ligands with high (or low) binding affinity to proteins can be a critical step in the drug discovery process. (For example, when the protein is an off-target protein and the binding of the ligand to the protein may cause adverse side effects, it may be necessary to identify ligands with low binding affinity to the protein.) However, determining the binding affinity of a ligand to a protein, for example through computational simulations or physical experiments, can be both expensive and time-consuming.

[0110] To address these issues, a predicted joint 3D structure 110 generated by the cofolding system 100 can be used to determine the ranking of candidate ligands based on their corresponding predicted binding affinity to the protein. More specifically, for each candidate ligand, the cofolding system 100 can generate a corresponding predicted joint 3D structure 110 of the protein and the candidate ligand. For each candidate ligand, a scoring function can process the predicted joint 3D structure 110 of the protein and the candidate ligand to generate the predicted binding affinity of the candidate ligand to the protein. The candidate ligand set can then be ranked based on the corresponding predicted binding affinity of each candidate ligand to the protein. The candidate ligand set can be obtained from existing ligand libraries, such as so-called compound libraries (e.g., available from commercial sources), libraries generated through combinatorial techniques, and other sources (e.g., complex databases, such as protein databases).

[0111] Candidate ligands from a candidate ligand set can be used, for example, to select an appropriate subset of the candidate ligand set for experimental validation and testing, based on their respective predicted binding affinity to proteins. For example, one or more candidate ligands with the highest or lowest predicted binding affinity to proteins (i.e., according to ranking) can be selected for experimental validation and testing, such as in drugs intended to achieve therapeutic effects in patients. For example, each selected candidate ligand can be physically synthesized and then tested, for example, in vitro (e.g., in cell culture or tissue models) or in vivo, testing various properties such as absorption, distribution, metabolism, and / or excretion, for example, by testing in vivo organisms, cell cultures, or tissue models. For example, selected ligands can be screened based on the degree of biological (therapeutic) effect associated with binding (e.g., promoting biological mechanisms or directly or indirectly inhibiting biological disease mechanisms (e.g., inhibiting bacterial or viral entry into cells)), toxicity, clearance time, etc. One or more candidate ligands can be selected from the ligand set and added to a drug, for example, at least in part based on test results. Drugs containing one or more candidate ligands can be synthesized using any suitable drug synthesis technique.

[0112] The candidate ligand set can contain any suitable number of ligands, such as 10 ligands, 1000 ligands, or 100,000 ligands. In some cases, such as candidate ligand ranking based on the predicted binding affinity of candidate ligands to proteins, only a subset of the candidate ligands in the set may be selected for physical synthesis. For example, less than 50%, less than 10%, less than 1%, or less than 0.1% of the candidate ligands in the set may be selected for physical synthesis.

[0113] In some cases, the predicted co-folded 3D structure 110 generated by the co-folding system 100 can be used for drug reuse. More specifically, existing drugs may contain specific ligands, for example, ligands known to achieve therapeutic effects in patients by binding to target proteins involved in specific disease processes. Drug reuse may involve identifying novel protein-binding targets of the ligand, for example, targets that may be involved in different disease processes. If the ligand has a high binding affinity to a novel target protein involved in the disease process, the ligand can be selected for experimental validation and may potentially be incorporated into a drug for treating the disease. Drug reuse can leverage existing safety and efficacy data of drugs containing ligands, potentially accelerating the development process and reducing R&D costs. Drug reuse can identify new treatment options and meet unmet medical needs by reusing known ligands to treat different diseases or conditions.

[0114] To identify novel target proteins for ligands, a predicted joint 3D structure 110 generated by the cofolding system 100 can be used to determine the ordering of candidate proteins in the candidate protein set based on the predicted binding affinity of a specific ligand for each candidate protein. More specifically, for each candidate protein, the cofolding system 100 can generate a corresponding predicted joint 3D structure 110 for the ligand and the candidate protein. For each candidate protein, a scoring function can process the predicted joint 3D structure 110 for the ligand and the candidate protein to generate the predicted binding affinity of the ligand for the candidate protein. The candidate protein set can then be ordered based on the corresponding predicted binding affinity of the ligand for each candidate protein.

[0115] Candidate proteins in a set of candidate proteins can be ranked based on the predicted binding affinity of the ligand to the candidate protein, for example, to select an appropriate subset of the candidate ligand set for experimental validation and testing. For example, one or more candidate proteins for which the ligand has the highest predicted binding affinity (i.e., according to the ranking) can be selected for experimental validation and testing, such as drugs intended to achieve therapeutic effects in patients. Specifically, each selected candidate protein can be physically synthesized, and then the binding affinity of the ligand to the candidate protein can be experimentally tested and validated. One or more candidate proteins can be selected from the candidate protein set as binding targets for the ligand, for example, at least in part based on the test results.

[0116] The candidate protein set can contain any suitable number of proteins, such as 10 proteins, 1000 proteins, or 100,000 proteins. In some cases, such as candidate protein ranking based on the predicted binding affinity of ligands to candidate proteins, only a subset of candidate proteins in the candidate protein set may be selected for physical synthesis. For example, less than 50%, less than 10%, less than 1%, or less than 0.1% of the candidate proteins in the candidate protein set may be selected for physical synthesis.

[0117] The cofolding systems and methods described herein can be used to obtain ligands (i.e., ligand molecules or ligand-molecule complexes), such as ligands for drugs or industrial enzymes. Generally, drugs or industrial enzymes may be molecules that inhibit or catalyze chemical or biochemical processes. Molecular complexes may comprise, for example, proteins, ribozymes (ribonucleases), or deoxyribonucleases (deoxyribonucleases). For example, methods for obtaining ligands may include obtaining the target amino acid sequence, particularly the amino acid sequence of a target protein molecule (or target protein-molecule complex), such as a drug target; and using a cofolding system to process the input based on the target amino acid sequence to determine the (tertiary) structure of the target protein molecule, such as the predicted structure of a complex comprising the target protein molecule and a candidate ligand. The method may then include evaluating the interaction between one or more candidate ligands and the target protein molecule. The method may further include selecting one or more candidate ligands from the candidate ligands as ligands based on the results of the evaluation of the interaction. Preferably, predicting the structure of a complex comprising the target protein molecule and the candidate ligand can take into account changes in molecular structure caused by the binding of the candidate ligand and / or changes in the structure of the candidate ligand. For example, evaluating the interaction between one or more candidate ligands and a target protein molecule may include determining the binding energy or equilibrium constant for complex formation.

[0118] In some implementations, evaluating interactions can involve assessing the binding of candidate ligands to the target protein molecule. For example, evaluating interactions can involve identifying ligands with sufficient affinity to bind and achieve a biological effect. In other implementations, evaluating interactions can involve assessing the association between candidate ligands and the target protein molecule, which affects the function of the target protein molecule (e.g., an enzyme). Evaluation can include assessing the affinity between the candidate ligand and the target protein molecule or complex, or assessing the selectivity of the interaction. Candidate ligands can be selected based on the highest affinity. Evaluating interactions can also include, for example, simulating the dynamic behavior of the ligand and target protein molecule through molecular dynamics simulations, which can take into account the dynamic aspects of the interaction.

[0119] Candidate ligands can be obtained from a candidate ligand database, and / or can be obtained by modifying ligands in the database (e.g., by modifying the structure or amino acid sequence of the candidate ligand), and / or by stepwise or iterative assembly / optimization of the candidate ligands. Candidate ligands may alternatively or additionally include one or more candidate ligands generated using a generative model conditioned on the target protein molecule or a portion thereof (the structure of the target protein molecule), such as the structure of the binding site or other parts of the target protein molecule.

[0120] Computer-aided methods can be used to evaluate the interaction between candidate ligands and target protein molecules, where graphical models displaying the structures of the candidate ligand and target protein molecules are available for user manipulation, and / or the evaluation can be performed partially or fully automatically, for example, using standard molecule (e.g., protein-ligand) docking software. In some implementations, the evaluation may include determining the interaction score of the candidate ligand, where the interaction score comprises a measure of the interaction between the candidate ligand and the target protein molecule. The interaction score may depend on the strength and / or specificity of the interaction, for example, on the fraction of the binding free energy. Candidate ligands can be selected based on their scores.

[0121] In some implementations, the target protein molecule contains a receptor or enzyme, and the ligand is an agonist or antagonist of the receptor or enzyme. In some implementations, the method can be used to identify the structure of a cell surface marker. This method can then be used to identify ligands (e.g., antibodies or aptamers) or markers (e.g., fluorescent markers) that can bind to the cell surface marker. This can be used to identify and / or treat cancer cells.

[0122] In some implementations, the ligand is the drug, and the interaction between each of a plurality of target protein molecules and each of the candidate ligands is evaluated. One or more candidate ligands can then be selected to obtain a ligand that interacts (functionally) with each of the target protein molecules, or to obtain a ligand that interacts (functionally) with only one of the target protein molecules. For example, in some implementations, it may be necessary to obtain a drug that is effective against multiple drug targets. Additionally or alternatively, it may be necessary to screen the drug for off-target effects. For example, in agriculture, it may be useful to determine that a drug designed specifically for a particular plant species will not interact with other different plant and / or animal species.

[0123] In some implementations, the ligand is a drug, and the predicted structure of the target protein (i.e., a protein complex, such as a dimer or multimer) is determined. Evaluating the interaction of one or more candidate ligands with the target protein can then include identifying candidate ligands that interact with the protein complex and are therefore expected to affect the formation or stability of the complex. This can then be confirmed through experimental screening. Therefore, this process can be used to identify drugs capable of disrupting protein complexes or inhibiting their formation. Certain diseases, such as neurodegenerative diseases like dementia, are caused by protein aggregation. Therefore, the method described can be used to identify ligands as drugs for treating such diseases.

[0124] In some implementations, the candidate ligand can comprise a small molecule complex ligand, such as an organic compound with a molecular weight <900 Daltons. In other implementations, the candidate ligand can comprise a peptide ligand, i.e., a ligand defined by an amino acid sequence.

[0125] On the other hand, a method is provided for obtaining ligands using a cofolding system, wherein the ligands may be biomolecules, such as peptides, polynucleotides, or polynucleotide ligands (e.g., the molecule or its amino acid or nucleotide sequence). For example, the method may include obtaining data defining one or more candidate ligands, such as the amino acid sequences of one or more candidate peptide or polynucleotide ligands. The method may include selecting a target molecule to which the ligand will bind. The method may further include, for each candidate ligand, using the cofolding system to determine the (tertiary) structure of the complex comprising the candidate ligand and the target protein molecule. The method may further include obtaining the target protein structure of the target molecule through computer simulation and / or physical studies. The method may include evaluating the interaction between each of the one or more candidate ligands and the target protein molecule, for example, by evaluating the interaction between the predicted structure of the candidate ligand and the structure of the target molecule, or using the predicted structure of the complex comprising the candidate ligand and the target protein molecule. The method may further include selecting one or more candidate ligands as the ligand based on the evaluation results.

[0126] As previously described, evaluating interactions can include assessing the binding of candidate ligands to target protein molecular structures, for example, identifying ligands with sufficient affinity to bind and achieve a biological effect; and / or assessing the association between candidate ligands and target protein molecular structures that would affect the function of the target protein molecule (e.g., an enzyme); and / or assessing the affinity between candidate ligands and target protein molecular structures, or assessing the selectivity of the interaction. In some implementations, the ligand may be an aptamer. Again, candidate ligands can be selected based on the highest affinity for them.

[0127] As previously described, the selected ligand (e.g., a selected polypeptide or polynucleotide ligand) may comprise a receptor or enzyme, and the ligand may be an agonist or antagonist of the receptor or enzyme. In some implementations, the ligand may comprise an antibody or aptamer, and the target protein molecule may comprise an antibody or aptamer target, such as a virus, particularly a viral capsid protein, or a protein expressed on a cancer cell. In these implementations, the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect. For example, the antibody or aptamer may bind to the target and act as an agonist of a specific receptor; alternatively, the antibody or aptamer may prevent another ligand from binding to the target and thus prevent activation of the associated biological pathway.

[0128] The method may be further implemented by synthesizing ligands, i.e., preparing small molecule, polynucleotide, or peptide ligands. The ligands can be synthesized using any conventional chemical technique, or are already available, for example, from a compound library, or can be synthesized using combinatorial chemistry.

[0129] The method may further include testing the bioactivity of the ligand in vitro and / or in vivo. For example, the ADME (absorption, distribution, metabolism, excretion) and / or toxicological properties of the ligand may be tested to screen out unsuitable ligands. For example, the test may involve contacting a candidate small molecule, peptide, or polynucleotide ligand with a target protein molecule and measuring changes in the expression or activity of the target molecule.

[0130] In some implementations, candidate (e.g., peptide or polynucleotide) ligands may comprise: isolated antibodies or aptamers, fragments of isolated antibodies or aptamers, monovariable domain antibodies, bispecific or multispecific antibodies, multivalent antibodies, bivariable domain antibodies, immunoconjugates, fibronectin molecules, adnectin, DARPin, avimer, affibody, antiicalin, affilin, protein epitope mimics, or combinations thereof. Candidate (peptide) ligands may comprise antibodies with mutated or chemically modified amino acid Fc regions, for example, antibodies that inhibit or reduce ADCC (antibody-dependent cytotoxicity) activity and / or increase half-life compared to wild-type Fc regions. Candidate (peptide or polynucleotide) ligands may comprise antibodies with different CDRs (complementarity-determining regions).

[0131] As another example, the target protein molecule can be an enzyme that includes a CRISPR-related protein, and the ligand can include a guide RNA molecule. The described method can be performed to identify combinations of guide RNA molecules and CRISPR-related proteins, particularly combinations capable of effectively editing genes. This method can involve determining the predicted structure of the enzyme, for example as described above, specifically to check whether the shape of the enzyme and the shape of the guide RNA match and work effectively together. The guide RNA can have a portion with a defined 3D structure; for example, it can be a single guide RNA (sgRNA) containing a guide sequence and a tracrRNA sequence.

[0132] The cofolding system described herein can also be used to obtain diagnostic antibodies or aptamer markers for diseases. The present invention also provides a method comprising selecting a target protein molecule to be recognized by an antibody or aptamer marker, and for each of one or more candidate antibodies or aptamers (e.g., as described above), using the cofolding system to determine a predicted structure comprising the target protein molecule and the candidate antibody or aptamer. The method may further comprise evaluating the interaction between each of one or more candidate antibodies or aptamers and the target protein molecule, and selecting one of the one or more candidate antibodies or aptamers as a diagnostic antibody or aptamer marker based on the evaluation results, for example, selecting one or more candidate antibodies or aptamers with the highest affinity for the target protein. The method may include the preparation of a diagnostic antibody or aptamer marker. Diagnostic antibodies or aptamer markers can be used to diagnose diseases by detecting whether they bind to a target protein molecule in a sample obtained from a patient (e.g., a body fluid sample). As described above, corresponding techniques can be used to obtain therapeutic antibodies or aptamers (e.g., peptides or polynucleotide ligands).

[0133] In some other respects, the active / binding / blocking sites of target proteins can be identified from their amino acid sequences using computer-implemented methods as described above or herein.

[0134] The folding systems and methods described herein can also be used to determine the structure of molecules or molecular complexes. For example, experimental techniques can be applied to physical samples containing molecular complexes (i.e., using experimental techniques to “probe” physical samples of molecular complexes) to measure experimental signals based on the structure of the molecular complex. In some implementations, the experimental techniques can be scattering techniques or spectroscopic techniques. For example, experimental techniques can include one or more of the following: X-ray crystallography, nuclear magnetic resonance (NMR), and electron microscopy (e.g., cryo-EM). The co-folding system can be used to determine the predicted structure of the molecular complex. The experimental signal can then be compared with a corresponding analog signal generated using the predicted structure of the molecular complex. For example, a predicted X-ray diffraction pattern (e.g., based on the electron density distribution determined using the predicted structure) can be generated using the predicted structure of the molecular complex and compared with an experimentally measured X-ray diffraction pattern. For example, the experimental signal can include an NMR signal, an electron microscope image, or an X-ray diffraction pattern; or a signal derived therefrom.

[0135] The predicted structure of a molecular complex can be determined by adjusting the predicted structure based on comparison results. The method can be performed iteratively, wherein in one or more iterations, a predicted signal can be generated for the adjusted structure after adjustment, and the comparison and adjustment can be repeated to improve the predicted structure. Alternatively (or additionally), multiple different possible structures of the molecular complex can be predicted, and the expected experimental signal for each structure can be compared with the actual experimental signal to determine a match that can be considered a definite structure of the molecular complex, such as the best match or the most likely match.

[0136] Generally, aptamers as described above can comprise DNA or RNA. Enzymes as described above can comprise proteins or DNA enzymes, such as deoxyribozymes or "DNAzymes," or RNases, such as ribozymes. In addition to the applications described above, these enzymes can be used in various types of biosensors; for example, DNAzymes and aptamers can be used to detect metal ions, and generally, aptamer targets can comprise small molecule complexes, proteins, and cells. Aptamers have many uses, including, for example, as probes in assays, as biosensors (e.g., they can significantly change shape upon binding to a target), modulating the activity of biomolecular complexes, and providing controlled release mechanisms. The techniques described herein can be used to design and manufacture such aptamers, sometimes referred to as chemical antibodies.

[0137] Figure 2 It is shown, for example, in the reference Figure 1 The described cofolding system includes an example embedding neural network 200. The embedding neural network 200 is configured to process: (i) protein data 102 characterizing a protein, and (ii) ligand data 104 characterizing one or more ligands, to generate a protein-ligand embedding 106 of the protein and one or more ligands.

[0138] The embedded neural network 200 includes a protein embedded neural network 202, a ligand embedded neural network 204, and a fusion neural network 210, which will be described in more detail below (and throughout this specification).

[0139] The protein embedding neural network 202 is configured to process protein data 102 characterizing proteins to generate protein embeddings 206. For example, protein embedding 206 may contain, for example, the corresponding amino acid embedding for each amino acid in each amino acid sequence of the protein.

[0140] The protein embedding neural network 202 can have any suitable neural network architecture that enables it to perform the functions described herein. Specifically, the protein embedding neural network 202 can contain any suitable type of neural network layers (e.g., fully connected layers, convolutional layers, attention layers, etc.) with any suitable number of layers (e.g., 5, 10, or 20 layers) and connected in any suitable configuration (e.g., a directed graph as layers).

[0141] A concrete example of a possible architecture for protein embedding neural networks 202 is Jumper et al. "nature" The "Evoformer" neural network is described in the journal, Volume 596, August 26, 2021, in the article "Highly accurate protein structure prediction with AlphaFold". Evoformer can process network inputs derived from (i) the amino acid sequence of a protein, (ii) the MSA of a protein, and (iii) the 3D structure of one or more template amino acid sequences, thereby generating an output containing a "single representation" that defines the corresponding embedding at each position in each amino acid sequence of the protein. (The template amino acid sequence is the amino acid sequence of an amino acid chain in a protein, wherein the folding structure of the template sequence is known, for example, through physical experiments).

[0142] The ligand embedding neural network 204 is configured to process ligand data 104 characterizing one or more ligands to generate a corresponding ligand embedding 208 for each of the one or more ligands. Specifically, the ligand embedding neural network 204 can be configured to process the ligand data 104 characterizing each ligand individually to generate a corresponding embedding for each ligand. The embedding of each ligand can include a corresponding atom embedding representing each atom in the ligand. The embeddings of the individual ligands can jointly define the overall ligand embedding 208. For example, the overall ligand embedding can be defined by concatenating the embeddings of the individual ligands. Therefore, the overall ligand embedding can include a corresponding atom embedding for each atom in each of the one or more ligands.

[0143] The ligand embedding neural network 204 can have any suitable neural network architecture that enables it to perform the functions described herein. Specifically, the ligand embedding neural network 204 can contain any suitable type of neural network layers (e.g., fully connected layers, convolutional layers, attention layers, etc.) in any suitable number of layers (e.g., 5, 10, or 20 layers) and connected in any suitable configuration (e.g., a directed graph of layers).

[0144] In a specific example, the ligand embedding neural network 204 can be configured to receive an initial set of atomic embeddings containing the corresponding initial atomic embedding for each atom in the ligand and derived from data characterizing the ligand, such as the string "SMILES" representing the ligand. The initial atomic embedding for each atom can contain data characterizing the type of the atom, other atoms to which the atom is bound, whether the atom is contained in any functional group, etc. The ligand embedding neural network can process the initial set of atomic embeddings through one or more sequences of attention neural network layers, each attention neural network layer configured to update the current set of atomic embeddings through a self-attention operation to generate the embeddings of the ligand, for example, as the set of atomic embeddings output by the last attention layer in the sequence of attention layers.

[0145] The fusion neural network 210 is configured to process protein embeddings 206 and ligand embeddings 208 to generate a protein-ligand embedding 106 that jointly represents a protein and one or more ligands. The fusion neural network 210 can have any suitable neural network architecture that enables it to perform the functions described herein. Specifically, the fusion neural network 210 can contain any suitable type of neural network layers (e.g., fully connected layers, convolutional layers, attention layers, etc.) with any suitable number of layers (e.g., 5, 10, or 20 layers) and connected in any suitable configuration (e.g., a directed graph as layers). Reference Figure 3A An example of processing protein embedding 206 and ligand embedding 208 to generate protein-ligand embedding 106 is described in more detail.

[0146] Figure 3A This is a flowchart of an example process 300 that uses a fusion neural network to process the protein embedding of a protein and the ligand embedding of one or more ligands to generate a joint representation of the protein and the protein-ligand embedding of one or more ligands. For convenience, process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a co-folding system appropriately programmed according to this specification (e.g., Figure 1 The co-folding system 100 in the middle can execute process 300.

[0147] The system receives protein embeddings and ligand embeddings of one or more ligands (302). Protein embeddings can be obtained via a reference. Figure 2 The protein embedding described is generated by a neural network and can contain the corresponding amino acid embedding for each position in each amino acid sequence of the protein. Ligand embedding can be referenced... Figure 2 The described ligand embeddings are generated by a neural network and may include the corresponding atom embeddings of each atom in each of one or more ligands.

[0148] The system connects protein embeddings and ligand embeddings to generate a one-dimensional (1D) embedding sequence (304). The 1D embedding sequence contains amino acid embeddings of the protein embedding and atomic embeddings of the ligand embedding. The embeddings contained in the 1D embedding sequence can be ordered in any suitable manner; for example, the 1D embedding sequence can be ordered with amino acid embeddings followed by atomic embeddings, or with atomic embeddings followed by amino acid embeddings. The length of the 1D embedding sequence can be the sum of (i) the number of amino acid embeddings in the protein embedding and (ii) the number of atomic embeddings in the ligand embedding. The 1D embedding sequence can be generated from sequences with dimensionality... The data indicates that, among them The sum of the following is given: (i) the number of amino acid insertions in the protein insertion, and (ii) the number of atomic insertions in the ligand insertion, and It is a positive integer value that defines the number of channel dimensions in each amino acid embedding and atom embedding.

[0149] The system processes 1D amino acid and atom embedding sequences to generate data (306) defining a two-dimensional (2D) embedding array. The 2D embedding array can be generated from sequences with dimensions... The data represents, where (as described above) The sum of the following is given: (i) the number of amino acid insertions in the protein insertion, and (ii) the number of atomic insertions in the ligand insertion, and ' is a positive integer value that defines the number of channel dimensions in each embedding ( It can be equal to (i.e., the number of channel dimensions in each amino acid embedding and atom embedding). The system can generate a 2D embedding array from a 1D embedding sequence in several ways. For example, the system can generate a 2D embedding array as the result of an element-wise outer product of the 1D embedding sequence with itself. As another example, the system can generate a 2D array through appropriate 2D join operations, for example, where each position in the 2D embedding array... The embedding is achieved by connecting the 1D amino acid embedding and the atomic embedding in position (i) of the sequence. Embedding and (ii) Position Embedding generation (where index) , ,in (This is the length of the 1D embedding sequence).

[0150] Each embedding in the 2D embedding array can be, for example: (i) an atom-to-atom embedding, or (ii) an amino acid-to-amino acid embedding, or (iii) an amino acid-to-atom embedding. Each atom-to-atom embedding originates from a pair of atom embeddings representing atoms in one or more ligands. Each amino acid-to-amino acid embedding originates from a pair of embeddings containing an amino acid embedding representing an amino acid in a protein and an atom embedding representing an atom in a ligand.

[0151] The system processes the 2D embedding array by fusing a set of neural network layers to generate an updated 2D embedding array (308) that defines the protein-ligand embedding. The updated 2D embedding array can have the same dimensions as the original 2D embedding array, for example... (in and (The definition of ' is as described above). To generate an updated 2D embedding array, the fusion neural network can process the 2D embedding array using a sequence of one or more self-attention blocks. Each self-attention block can be configured to receive the current 2D embedding array as input, update the current 2D embedding array through one or more self-attention operations (e.g., single-head or multi-head query key-value (QKV) self-attention operations), and feed the updated 2D embedding array to subsequent neural network layers (e.g., to another self-attention block, or to the output layer of the fusion neural network).

[0152] Self-attention blocks in fused neural networks can implement any appropriate self-attention operation. Several examples of self-attention operations that can be implemented using self-attention blocks in fused neural networks are described below.

[0153] In some implementations, one or more self-attention blocks of a fused neural network perform "row-by-row" or "column-by-column" self-attention on the current 2D embedding array (i.e., provided as input to the self-attention block). In a row-by-row self-attention operation, the self-attention layer uses self-attention only on embeddings that are in the same row as a given embedding in the 2D embedding array to update each given embedding in the 2D embedding array. In a column-by-column self-attention operation, the self-attention block uses self-attention only on embeddings that are in the same column as a given embedding in the 2D embedding array to update each given embedding in the 2D embedding array.

[0154] In some implementations, one or more self-attention blocks of a fused neural network perform a triangular self-attention operation. Jumper et al. in... "nature"An example implementation of the triangular self-attention operation is described in the journal, Volume 596, August 26, 2021, titled "Highly accurate protein structure prediction with AlphaFold".

[0155] In some implementations, one or more self-attention blocks of a fused neural network perform a full self-attention operation on the current 2D embedding array, for example, by updating each embedding in the 2D embedding array by paying attention to the entire 2D embedding array.

[0156] Fusion neural networks can contain other neural network layers, meaning that in addition to the sequence of self-attention blocks, they can also contain other neural network layers (such as fully connected layers or normalized layers) interspersed between self-attention blocks. Fusion neural networks can also include features such as skip connections, for example, to implement residual blocks in fusion neural networks.

[0157] The system outputs a 2D embedding array generated by the fusion neural network as protein-ligand embeddings (310). The system can provide protein-ligand embeddings generated by the fusion neural network, for example, for tuning the generative model, as will be referred to below. Figures 4 to 6 A more detailed description.

[0158] Figure 3B The operation performed by a cofolding system to generate protein-ligand embeddings is illustrated. The system generates a protein embedding containing an amino acid embedding sequence 316 (one embedding for each amino acid in the protein's amino acid sequence) and a ligand embedding containing an atom embedding sequence 318 (one embedding for each atom in one or more ligands). The cofolding system concatenates the amino acid embedding sequence and the atom embedding sequence into a 1D embedding sequence, and then converts the 1D embedding sequence (e.g., through an outer product operation) into a 2D embedding array. The 2D embedding array may contain: (i) atom-atom embeddings 330, (ii) amino acid-amino acid embeddings 324, and (iii) amino acid-atom embeddings 326, 328. Each atom-atom embedding originates from a pair of atom embeddings representing atoms in one or more ligands. Each amino acid-amino acid embedding originates from a pair of amino acid embeddings representing amino acids in the protein. Each amino acid-atom embedding originates from a pair of embeddings containing an amino acid embedding representing an amino acid in the protein and an atom embedding representing an atom in the ligand. The cofolding system can use a sequence of one or more self-attention blocks (e.g., implementing row-wise attention, column-wise attention, triangular self-attention, or full self-attention) to process the 2D embedding array 322 to generate protein-ligand embeddings.

[0159] Figure 4 This is a flowchart of an example process 400 for generating a predicted joint 3D structure of a protein and one or more ligands using a generative diffusion model incorporating a denoising neural network. For convenience, process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a co-folding system appropriately programmed according to this specification (e.g., Figure 1 The co-folding system 100 in the middle can execute process 400.

[0160] The system generates a corresponding initial 3D spatial position (402) for each atom in a complex comprising a protein and one or more ligands. For example, for each atom in the complex, the system can sample the 3D spatial position of the atom from a probability distribution in 3D space (e.g., a standard normal distribution in 3D space).

[0161] The system executes steps 404-406, described below, within an iterative sequence that may be referred to as a "time step." For convenience, the description of steps 404-406 (and related steps 408 and 412) will refer to the "current" time step; the current time step can be any time step in the time step sequence. The system can execute steps 404-406 within any appropriate number of time steps (e.g., 3 time steps, 10 time steps, or 100 time steps). The number of time steps can be a pre-defined number.

[0162] The system uses a denoised neural network conditioned on protein-ligand embedding to generate a denoised output (404). The denoised output can be any appropriate data capable of estimating the “final” 3D spatial position of each atom in the complex. For example, for each atom in the complex, the denoised output can define the prediction error of the atom’s 3D spatial position at the current time step. As another example, for each atom in the complex, the denoised output can directly define the predicted 3D spatial position of the atom. As yet another example, for each atom in the complex, the denoised output can define both: (i) the prediction error of the atom’s 3D spatial position at the current time step, and (ii) the predicted 3D spatial position of the atom. As another example, for each atom in the complex, the denoising output can be defined as a prediction of the value of a linear combination of the following two: (i) the actual 3D spatial position of the atom, and (ii) the error between the atom's 3D spatial position at the current time step and the atom's actual 3D spatial position, implemented, for example, by the v-parameterization described in the following literature: Tim Salimans, Jonathan Ho, “Progressive distillation for fastsampling of diffusion models,” ICLR 2022, arXiv:2202.00512v2. Reference Figures 5 to 6 A more detailed example process for generating denoised output using a denoising neural network is described.

[0163] The system uses the denoised output generated by the denoising neural network to generate an initial estimate of the 3D spatial location of each atom in the complex (406). The system can generate the initial estimate of the 3D spatial location of each atom in the complex in any suitable manner, depending on the form of the denoised output. Several example techniques for generating initial estimates of the 3D spatial location of each atom in the complex using the denoised output are described below.

[0164] For example, for each atom, the denoising output defines a corresponding prediction of the atom's 3D spatial location. In this example, the corresponding predicted 3D spatial location for each atom defines an initial estimate of the atom's 3D spatial location.

[0165] In another example, for each atom, the denoising output defines the prediction error of the atom's 3D spatial position at the current time step. In this example, the system can generate an initial estimate of the 3D spatial position of each atom as a linear combination of (i) the atom's current 3D spatial position and (ii) the prediction error of the atom's 3D spatial position. Each term in the linear combination can be scaled by a corresponding constant value depending on the time step. For example, the system can generate the 3D spatial positions of atoms in a complex. The initial estimate is:

[0166] in Indicates the current time step. , and It is specific to time step The constant, and This is the prediction error of the 3D spatial location of the atom (e.g., generated by the denoising neural network at this time step). (In the notes to equation (1), the time steps decrease, making the number of time steps...) It is a time step The "next" time step after that). The constant in equation (1) , and It can be selected based on predefined noise scheduling.

[0167] In another example, for each atom, the denoising output defines both: (i) the predicted 3D spatial position of the atom, and (ii) the prediction error of the atom's 3D spatial position at the current time step. In this example, the system can generate an initial estimate of the atom's 3D spatial position as a combination (e.g., an average) of: (i) the predicted 3D spatial position of the atom specified by the denoising output, and (ii) the predicted 3D spatial position of the atom derived, for example, from the prediction error of the atom's 3D spatial position at the current time step using equation (1).

[0168] In another example, the denoised output is represented using v-parameterization, and the system generates corresponding initial estimates of the 3D spatial location of each atom using the techniques described in the following literature: Tim Salimans, Jonathan Ho, “Progressive distillation for fast sampling of diffusion models”, ICLR 2022, arXiv:2202.00512v2.

[0169] Optionally, the system can generate a corresponding confidence metric for the initial estimate of the corresponding 3D spatial location of each atom in the complex. For example, as part of generating the denoised output, the denoising neural network can generate the corresponding atom embedding for each atom in the complex, for example, as the output of the update block of the denoising neural network, as referenced. Figure 5 As described in step 506. The system may process each atom embedding using one or more neural network layers (e.g., a combination of one or more of the following: fully connected layers, attention layers, or pooling layers) to generate a corresponding confidence estimate of the initial estimate of the 3D spatial location of the atom represented by the atom embedding. The confidence metric of the initial estimate of the 3D spatial location of the atom can characterize the prediction error of the initial estimate of the 3D spatial location of the atom.

[0170] Optionally, the system can generate corresponding confidence measures for the initial estimates of the 3D spatial positions of paired atoms in the complex. For example, for a first atom and a second atom in the complex, the system can use one or more neural network layers (e.g., a combination of fully connected layers, attention layers, or pooling layers) to process the atom embeddings representing the first atom and the atom embeddings representing the second atom (e.g., generated by an update block of a denoising neural network) to generate confidence estimates of the initial estimates of the 3D spatial positions of the first and second atoms. The confidence measure can characterize, for example, the prediction error of the relative 3D displacement of the first and second atoms.

[0171] Alternatively, the system can generate a confidence metric of the protein structure, such as that defined by an initial estimate of the 3D spatial location of the atoms in the protein, for example by combining (e.g., summing or averaging) confidence metrics of the individual atoms contained in the protein.

[0172] Optionally, for each ligand, the system can generate a confidence metric of the ligand structure, such as by combining (e.g., summing or averaging) confidence metrics of the individual atoms contained in the ligand, as defined by an initial estimate of the 3D spatial locations of the atoms in the ligand.

[0173] Optionally, for each ligand, the system can generate a confidence measure of the interface structure, for example, by combining (e.g., summing or averaging) confidence measures of the pairs of atoms contained in the interface between the protein and the ligand. For example, a pair of atoms may be considered contained in the interface if: (i) the pair of atoms contains a first atom contained in the ligand, and (ii) the pair of atoms contains a second atom contained in the protein, wherein the relative displacement between the 3D spatial positions of the atoms is less than a threshold, such as 2 Å, 3 Å, or 8 Å. An example of generating a confidence measure of a pair of atoms is described below.

[0174] If the current time step is the last time step (i.e. the last time step in the denoised time step sequence), the system outputs the initial estimate of the 3D spatial position of each atom in the complex as the predicted joint 3D structure of the complex (410).

[0175] If the current time step is not the last time step, the system uses an appropriate diffusion sampling technique to generate the corresponding 3D spatial position (412) of each atom in the next time step based on the initial estimate of the 3D spatial position of the atom (as generated in step 406). Several examples of possible diffusion sampling techniques are described below.

[0176] For example, the system can generate the 3D spatial position of each atom in the complex at the next time step by combining random noise with an initial estimate of the atom's 3D spatial position. For example, for each atom, the system can add corresponding random noise to the initial estimate of the atom's 3D spatial position. The random noise can be sampled from a probability distribution in 3D space. The probability distribution in 3D space can vary based on the time step, for example, such that the variance of the noise combined with the updated 3D spatial positions of the atoms decreases over the time step sequence.

[0177] As another example, the system uses deterministic diffusion sampling techniques (i.e., independent of random noise) to generate the 3D spatial location of each atom in the complex at the next time step. Examples of deterministic diffusion sampling techniques include denoising diffusion implicit models (DDIM), as described in, for example, in: Jiaming Song, Chenlin Meng, and Stefano Ermon, “Denoising diffusion implicit models,” ICLR 2021, arXiv:2010.02502v4.

[0178] Optionally, the system may execute process 400 multiple times to generate multiple predictions of the joint 3D structure of the complex. Each execution of the steps of process 400 may result in different predicted joint 3D structures of the complex, for example, due to the randomness of performing random sampling to generate the initial positions of the atoms (in step 402), and in some cases due to the randomness of the diffusion sampler (in step 412).

[0179] By directly predicting atomic coordinates—that is, determining the 3D spatial position of each atom in the complex—the system can reduce or eliminate the need for further processing of the complex's 3D structure, such as enforcing stereochemical constraints. For example, the system can generate chemically plausible and well-defined local structures, such as the configuration of amino acid side chains, without carefully adjusting stereochemical losses or special treatment of binding modes, and without computationally expensive molecular dynamics simulations.

[0180] In some implementations, the system executes the operation of process 400 multiple times to generate a set of multiple (distinct) predicted 3D structures of the complex. The system can generate any appropriate number of predicted 3D structures of the complex, such as at least 5, at least 10, at least 1000, or at least 10,000 predicted 3D structures. The system can also rank the predicted 3D structures based on a corresponding confidence metric for each structure. Example techniques for determining the confidence metric of the predicted 3D structures have been described above. Optionally, the system can discard any predicted 3D structures with a confidence metric below a confidence metric threshold.

[0181] Figure 5 This is a flowchart of an example process 500 for generating denoised output using a denoising neural network conditioned on protein-ligand embeddings. For convenience, process 500 will be described as being executed by a system of one or more computers located in one or more locations. For example, a co-folding system appropriately programmed according to this specification (e.g., Figure 1 The co-folding system 100 in the middle can execute process 500.

[0182] The system receives: (i) data defining the corresponding current 3D spatial position of each atom in the complex, and (ii) protein-ligand embeddings (502). The protein-ligand embeddings may include: (i) atom-to-atom embeddings, (ii) amino acid-to-amino acid embeddings, and (iii) amino acid-to-atom embeddings, as described in more detail above with reference to Figure 3. Optionally, the system may receive additional inputs, such as inputs defining the current time step in the diffusion process implemented by the generative diffusion model, as described above with reference to... Figure 4 As described.

[0183] The system uses an encoder block of a generative neural network to generate a corresponding atomic embedding for each atom in the complex, at least in part, based on the current 3D spatial position of the atom (504). Optionally, for example, in addition to the current 3D spatial position of the atom, the system may also generate the atomic embedding of the atom at least in part based on the protein-ligand embedding (and optionally the current time step in the diffusion process). For example, for each atom, the system may generate the atomic embedding based on (i) the current 3D spatial position of the atom, and (ii) a corresponding conditional embedding selected from the set of embeddings included in the protein-ligand embedding. For an atom contained in an amino acid in the complex, the conditional embedding of the atom may be an amino acid-amino acid embedding (i.e., from the protein-ligand embedding) (i.e., an amino acid-amino acid embedding corresponding to a pair of amino acids containing two copies of the amino acid). For an atom contained in a ligand in the complex, the conditional embedding of the atom may be an atom-atom embedding (i.e., from the protein-ligand embedding) (i.e., an atom-atom embedding corresponding to a pair of atoms containing two copies of the atom).

[0184] The system can generate atomic embeddings of atoms in any of the various possible ways based on the current 3D spatial location of the atom (and optionally, the conditional embedding of the atom). For example, the system can generate atomic embeddings of atoms by processing the 3D spatial location of the atom using an encoder block of a denoising neural network. As another example, the system can generate atomic embeddings of atoms by processing both (i) the 3D spatial location of the atom and (ii) the conditional embedding of the atom using an encoder subnetwork of a denoising neural network. As yet another example, the system can generate atomic embeddings of atoms by concatenating both (i) the output generated by the denoising neural network by processing the 3D spatial location of the atom and (ii) the conditional embedding of the atom.

[0185] The encoder block of a denoising neural network can contain any suitable type of neural network layers (e.g., fully connected layers, convolutional layers, etc.) in any suitable number of layers (e.g., 1, 3, or 5 layers) and connected in any suitable configuration (e.g., a directed graph of layers). In a particular example, the encoder block contains a sequence of fully connected neural network layers and is configured to process data defining the 3D spatial location of the atom and the conditional embedding of the atom for each atom to generate an atom embedding.

[0186] In some implementations, for each amino acid in a protein, the system can generate atomic embeddings that jointly represent all atoms in the amino acid. Therefore, in these implementations, the number of atomic embeddings may be equal to the sum of (i) the number of atoms in one or more ligands, and (ii) the number of amino acids in the protein. The system can generate atomic embeddings that jointly represent all atoms in an amino acid in any suitable manner. For example, the system can generate a corresponding atomic embedding for each atom in the amino acid (as described above), and then combine the atomic embeddings of the atoms in the amino acid, for example, using pooling operations (e.g., max pooling or summation pooling), or by processing the atomic embeddings of the atoms in the amino acid using one or more neural network layers (e.g., fully connected layers or self-attention layers), to generate an embedding that jointly represents all atoms in the amino acid. Generating atomic embeddings that jointly represent all atoms in an amino acid can significantly reduce the total number of atomic embeddings, thereby reducing the computational resources (e.g., memory and computing power) consumed by the update block execution operations of the denoising neural network, as described below.

[0187] The system uses update blocks of a denoising neural network to process the atomic embeddings of atoms in the complex, generating a corresponding updated atomic embedding for each atom in the complex (506). The update blocks of the denoising neural network layer may contain a sequence of self-attention blocks. Each self-attention block may be configured to receive the corresponding current atomic embedding for each atom in the complex, apply a self-attention operation to the current atomic embedding of the atoms in the complex, and provide updated atomic embeddings, for example, for processing by subsequent neural network layers.

[0188] Each self-attention block contained in the update block can apply any appropriate self-attention operation to the current atom embedding of atoms in the complex, such as single-head or multi-head query key-value (QKV) self-attention operations. Optionally, the system can control the self-attention operation of one or more self-attention blocks based on the protein-ligand embedding. (Reference) Figure 6 An example process for implementing self-attention operations conditioned on protein-ligand embedding is described in more detail.

[0189] The update block of the denoising neural network can contain any appropriate number of self-attention blocks (e.g., 1 self-attention block, 10 self-attention blocks, or 50 self-attention blocks) and can optionally contain any appropriate type of neural network layer (e.g., fully connected layer, convolutional layer, etc.) in any appropriate number of layers (e.g., 1 layer, 3 layers, or 5 layers) and connected in any appropriate configuration (e.g., interleaved with self-attention blocks).

[0190] The system uses the decoder block of the denoising neural network to process the updated atomic embeddings (i.e., generated by the update block of the denoising neural network) to generate a denoised output (508). Reference Figure 4An example of a denoised output is described. The decoder block of the denoised neural network can contain any suitable type of neural network layers (e.g., fully connected layers, convolutional layers, etc.) in any suitable number of layers (e.g., 1 layer, 3 layers, or 5 layers) and connected in any suitable configuration (e.g., a directed graph as layers).

[0191] In a specific example, the decoder block may contain a sequence of fully connected neural network layers configured to operate on each updated atom embedding to generate either a prediction error of the corresponding atom's 3D spatial location at the current time step, or a predicted 3D spatial location of the corresponding atom, or both.

[0192] In an implementation where an atomic embedding jointly represents all atoms in an amino acid (as described above with reference to step 504), the decoder block can process the (updated) atomic embedding to generate a corresponding denoised output for all atoms in the amino acid. For example, the decoder block can process the updated atomic embedding, which jointly represents all atoms in the amino acid, to generate a corresponding prediction error for the 3D spatial position of each atom in the amino acid, or to generate the corresponding 3D spatial position of each atom in the amino acid, or both.

[0193] Figure 6 This is a flowchart of an example process 600 for updating the current set of atom embeddings using a self-attention operation, which is implemented by a self-attention block of a denoising neural network and conditional on protein-ligand embeddings. For convenience, process 600 will be described as being performed by a system of one or more computers located in one or more locations. For example, a co-folding system appropriately programmed according to this specification (e.g., Figure 1 The co-folding system 100 in the middle can execute process 600.

[0194] The system receives: (i) the current set of atom embeddings, and (ii) the protein-ligand embedding (602). The current set of atom embeddings contains the corresponding atom embedding for each atom in the complex. (In some cases, for each amino acid in the protein, all atoms in the amino acid are represented jointly by a (corresponding) atom embedding, as referenced above.) Figure 5 (As described above). The current set of atomic embeddings can be generated from, for example, a previous self-attention layer in an encoder block of a denoising neural network or an update block of a denoising neural network, as referenced above. Figure 5 As described above, protein-ligand embeddings can be generated by embedding neural networks, as referenced above. Figures 2 to 3A As described. Protein-ligand intercalation can include: (i) atom-atom intercalation, (ii) amino acid-amino acid intercalation and (iii) amino acid-atom intercalation.

[0195] The system generates an intermediate attention score set (604) based on the current set of atomic embeddings. The intermediate attention score set contains the corresponding attention score for each pair of current atomic embeddings in the current set of atomic embeddings.

[0196] The system can generate intermediate attention scores in several possible ways. For example, the system can process the current atomic embedding using a query neural network to generate a corresponding query embedding for each current atomic embedding, such as:

[0197] in Each column (or row) defines a matrix containing a corresponding query embedding. It is a parameter value matrix (defined in this example as a query neural network), and Each column (or row) defines a matrix corresponding to the current atom embedding. Furthermore, the system can process the current atom embedding using a key-value neural network to generate a corresponding key-value embedding for each current atom embedding, for example:

[0198] in Each column (or row) defines a matrix with corresponding key-value embeddings. It is a parameter value matrix (defined in this example as a key-value neural network), and Each column (or row) defines a matrix corresponding to the current atomic embedding. The system can generate intermediate attention scores based on query embeddings and key-value embeddings, for example:

[0199] in It is the intermediate attention score matrix. It queries the embedding matrix, and It is a key-value embedding matrix.

[0200] The system generates an attention score bias set (606) based on protein-ligand embeddings. The attention score bias set contains the corresponding attention score bias for each pair of current atom embeddings in the current atom embedding set.

[0201] The system can generate the attention score bias set in a variety of possible ways. For example, for each pair of current atom embeddings, the system can generate the attention score bias for the pair of current atom embeddings by processing the corresponding conditional embeddings selected from the set of embeddings included in the protein-ligand embeddings using a projective neural network.

[0202] For each pair of current atom embeddings containing (i) a first current atom embedding representing a first atom contained in a ligand and (ii) a second current atom embedding representing a second atom contained in a ligand, the conditional embedding can be an atom-to-atom embedding of the first atom and the second atom in a protein-ligand embedding.

[0203] For each pair of current atom embeddings containing (i) a first current atom embedding representing an atom contained in an amino acid and (ii) a second current atom embedding representing an atom contained in a ligand, the conditional embedding can be an amino acid-atom embedding corresponding to the amino acid containing the first atom and (ii) the second atom in the protein-ligand embedding.

[0204] For each pair of current atom embeddings containing (i) a first current atom embedding representing an atom contained in the first amino acid and (ii) a second current atom embedding representing an atom contained in the second amino acid, the conditional embedding can be an amino acid-amino acid embedding corresponding to (i) the first amino acid and (ii) the second amino acid in the protein-ligand embedding.

[0205] For each pair of current atom embeddings containing (i) a first current atom embedding jointly representing an atom in the first amino acid and (ii) a second current atom embedding jointly representing an atom in the second amino acid, the conditional embedding can be an amino acid-amino acid embedding corresponding to (i) the first amino acid and (ii) the second amino acid in the protein-ligand embedding.

[0206] For each pair of current atom embeddings containing (i) a first current atom embedding representing an atom in an amino acid and (ii) a second current atom embedding representing an atom contained in a ligand, the conditional embedding can be an amino acid-atom embedding corresponding to the (i) amino acid and (ii) atom in the protein-ligand embedding.

[0207] Projective neural networks can have any suitable neural network architecture that enables them to perform the functions they describe, such as processing conditional vectors to generate attention score biases. Specifically, a projective neural network can contain any suitable number of neural network layers (e.g., 1, 3, or 5 layers) and connected in any suitable configuration (e.g., a directed graph of layers).

[0208] The system generates a final attention score set (608) by combining (i) intermediate attention scores and (ii) attention score biases. The final attention score set contains the corresponding final attention score for each pair of current atomic embeddings in the current atomic embedding set. The system can generate a pair of final attention scores for a pair of current atomic embeddings by combining (e.g., summing) the following: (i) the intermediate attention scores of a pair of current atomic embeddings, and (ii) the attention score biases of a pair of current atomic embeddings. Combining the intermediate attention scores and attention score biases allows the protein-ligand embeddings to control (i.e., regulate) the information flow in the current atomic embeddings. Optionally, the system can apply further processing operations to the final attention score set, such as applying a softmax operation to some or all of the final attention scores.

[0209] The system uses (i) the current set of atomic embeddings and (ii) the final set of attention scores to generate an updated set of atomic embeddings (610). For example, to generate the updated set of atomic embeddings, the system can process the current atomic embeddings using a value neural network to generate a corresponding value embedding for each current atomic embedding in the current set of atomic embeddings, for example:

[0210] in Each column (or row) defines a matrix with a corresponding value embedding. It is a parameter value matrix (defined in this example as a value neural network), and Each column (or row) defines a matrix corresponding to the current atom embedding. The system can then generate an updated set of atom embeddings, for example:

[0211] in Each column (or row) defines a corresponding updated atomic embedding. Each column (or row) defines a corresponding value embedding, and This represents the final set of attention scores arranged in a matrix.

[0212] In the implementation of multi-head attention operations using self-attention blocks, each head of the attention operation can independently execute the steps of process 600, and the updated atomic embeddings generated by each attention head can be combined (e.g., concatenated) to define the overall output of the multi-head attention operation. Each attention head can have a corresponding set of neural network parameters, the values ​​of which are specific for each attention head and are used to generate intermediate attention scores and attention score biases.

[0213] Figure 7This is a flowchart of an example process 700 for jointly training a generative model of an embedded neural network and a co-folding system. In example process 700, the generative model is a model that implements a differentiable generative process. For example, the generative model could be a generative diffusion model implemented using a denoising neural network, as referenced above. Figures 4 to 6 As described. For convenience, process 700 will be described as being performed by a system of one or more computers located in one or more locations. For example, a co-folding system appropriately programmed according to this specification (e.g., Figure 1 The co-folding system 100 in the middle can execute process 700.

[0214] The system receives data characterizing a set of protein-ligand complexes (702). Each protein-ligand complex defines the joint 3D structure of a protein and one or more ligands, for example, where each of the one or more ligands binds to a corresponding binding site on the protein. For example, the joint 3D structure of the received data can be determined through physical experiments. Furthermore, many public databases are available, such as the Protein Data Bank (wwpdb.org), etc. In some cases, other protein structure determination systems can also be used to obtain the joint 3D structure for training the system, such as those developed by Jumper et al. "nature" The system described in the journal, Volume 596, August 26, 2021, "Highly accurate protein structure prediction with AlphaFold".

[0215] The system generates a training example set (704). Each training example corresponds to a specific protein-ligand complex and contains data defined as follows: (i) the training input to the cofolded system, and (ii) the target output of the cofolded system. The training input to the cofolded system contains protein data (proteins characterizing the protein-ligand complex) and ligand data (one or more ligands characterizing the protein-ligand complex). The target output of the cofolded system may be based on the joint 3D structure of the protein-ligand complex.

[0216] The system jointly trains an embedded neural network and a generative model on a training example set using machine learning training techniques. More specifically, for each training sample, the system can process the training input of the training example using the embedded neural network and the generative model to generate a predicted output of the generative model. The system can evaluate an objective function that measures the error (e.g., root mean square deviation (RMSD), mean absolute error (MAE), or mean squared error (MSE)) between (i) the predicted output of the generative model and (ii) the target output of the generative model. The system can determine the gradient of the objective function with respect to the parameters of the embedded neural network and the generative model, for example, using backpropagation. (The parameters of the generative model may include, for example, the set of neural network parameters implemented by the neural network of the generative model). The system can then update the current values ​​of the parameters of the embedded neural network and the generative model using the gradient, for example, through the update rules of an appropriate gradient descent optimization algorithm (e.g., RMSprop or Adam).

[0217] The specific aspects of training (e.g., the operations and objective function used to generate the model during training) can depend on how the generative model is implemented. See below for further details. Figure 8 This section describes in more detail the example process of training a generative diffusion model containing a denoising neural network on training examples. Figure 8 In the example process, the operations that modify the generative diffusion model during training (e.g., compared to the operations of the generative diffusion model during inference, for example, as referenced) Figure 4 (As described below), will be described in more detail below.

[0218] In some implementations, the generative model includes one or more "confidence estimation" neural network layers to process the atomic embeddings generated by the generative model to generate a confidence metric for the estimated positions of atoms or pairs of atoms in the complex (as referenced above). Figure 4 (As described). The system can jointly train a confidence estimation neural network layer, as well as an embedded neural network and a generative model. For example, for each atom in the complex, the system can generate a confidence metric defining the prediction error of the atom's 3D spatial position, and the objective function can include a term measuring the difference between (i) the prediction error of the atom's 3D spatial position and (ii) the actual error of the atom's 3D spatial position. As another example, for each pair of atoms in the complex, the system can generate a confidence metric defining the prediction error of the relative 3D displacement of the pair of atoms, and the objective function can include a term measuring the difference between (i) the predicted relative 3D displacement of the pair of atoms and (ii) the actual relative 3D displacement of the pair of atoms. The system can backpropagate gradients to the confidence estimation neural network layer and optionally to the generative model and / or the embedded neural network.

[0219] Figure 8This is a flowchart of an example process 800 for jointly training an embedded neural network and generating a diffusion model on training examples. For convenience, process 800 will be described as being performed by a system of one or more computers located in one or more locations. For example, a co-folding system appropriately programmed according to this specification (e.g., Figure 1 The co-folding system 100 in the middle can execute process 800.

[0220] The system uses an embedded neural network to generate protein-ligand embeddings of protein-ligand complexes corresponding to training examples (802).

[0221] The system samples one time step (804) from the denoised time step sequence. More specifically, during inference, the generative diffusion model can be configured to perform a denoised time step sequence, for example, as referenced. Figure 4 Steps 404, 406, 408, and 412 are described. During training, the system can randomly sample a denoised time step from the denoised time step sequence, for example, based on a uniform distribution on the denoised time step sequence.

[0222] The system generates a corresponding noise spatial position for each atom in the complex by combining random noise with the target spatial position of the atom (i.e., the target 3D structure of the complex). (806) For example, for each atom in the complex, the system can generate the noise spatial position of the atom by adding random noise to the target spatial position of the atom. The system can scale the random noise combined with the target spatial position of the atom by a constant that depends on the sampling time step, for example, where the value of the constant corresponding to the denoising time step is defined by the noise schedule.

[0223] The system uses a denoising neural network that generates denoised output conditioned on protein-ligand embedding (808). Reference Figure 5 A detailed example process for generating denoised output is described. Figure 5 In step 502, the current position of each atom in the complex can be defined as the noise spatial position of each atom in the complex.

[0224] The system determines the gradient of the objective function that depends on the denoised output and uses said gradient to update the parameter values ​​of the denoising neural network and the embedded neural network (810). The objective function can measure the error between (i) the denoised output of the denoising neural network and (ii) the target output of the denoising neural network. The target output of the denoising neural network can define the output of the denoising neural network if used to generate initial estimates of the 3D spatial positions of atoms in the complex (e.g., ...). Figure 4 As described in step 406, this will match the initial estimate of the 3D spatial location of the atom with the target (actual) 3D spatial location of the atom in the protein-ligand complex of the training example.

[0225] Figure 9 An example of the output generated by the cofolding system is shown, illustrating the predicted 3D structure of the protein and two ligands. In this example, the first ligand binds to a binding pocket on the protein, while the second ligand is located outside any binding pocket of the ligand. This suggests, for example, that the second ligand may not bind to any binding pocket on the protein, or that the second ligand has a lower binding affinity to the protein than the first ligand.

[0226] The term "configuration" is used in this specification to describe systems and computer program components. A system configuration of one or more computers to perform a specific operation or action means that the system has its software, firmware, hardware, or a combination thereof installed, which, when run, causes the system to perform the operation or action. A computer program configured to perform a specific operation or action means that one or more programs contain instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0227] The embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their equivalents), or in combinations thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transient storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination thereof. Alternatively, program instructions may be encoded onto artificially generated propagation signals, such as machine-generated electrical signals, optical signals, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiving device for execution by the data processing device.

[0228] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, apparatuses, and machines used for processing data, including, for example, programmable processors, computers, or multiple processors or computers. The device may also be or further include a dedicated logic circuit system, such as a FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit). In addition to hardware, the device may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0229] A computer program (which may also be referred to or described as a program, software, software application, app, module, software module, script, or code) can be written in any programming language (including compiled or interpreted languages, declarative or procedural languages) and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program may be stored as a part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code portions). A computer program may be deployed to execute on a single computer or on multiple computers (located in one place or distributed across multiple locations and interconnected via a data communication network).

[0230] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Typically, an engine is implemented as one or more software modules or components installed on one or more computers at one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in others, multiple engines may be installed and run on the same one or more computers.

[0231] The processes and logic flows described in this specification can be executed by one or more programmable computers, which execute one or more computer programs to perform functions by manipulating input data and generating output. These processes and logic flows can also be executed by a dedicated logic circuit system (e.g., an FPGA or ASIC), or by a combination of a dedicated logic circuit system and one or more programmed computers.

[0232] A computer suitable for executing computer programs can be based on a general-purpose microprocessor, a special-purpose microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or integrated into a special-purpose logic circuit system. Generally, a computer will also include or be operatively coupled to one or more mass storage devices (e.g., disks, magneto-optical disks, or optical disks) for receiving data from or transferring data to, or both. However, a computer does not necessarily need to have such devices. Furthermore, computers can be embedded in other devices such as mobile phones, personal digital assistants (PDAs), mobile audio or video players, game consoles, GPS receivers, or portable storage devices (e.g., Universal Serial Bus (USB) flash drives), and so on.

[0233] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0234] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's device in response to a request received from a web browser. Additionally, the computer can also interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running an instant messaging application) and receiving reply messages from the user.

[0235] Data processing devices used to implement machine learning models may also include, for example, dedicated hardware accelerator units for processing computationally intensive parts (i.e., inference, workloads) common in machine learning training or production.

[0236] Machine learning models can be implemented and deployed using machine learning frameworks such as TensorFlow or Jax.

[0237] Implementations of the subjects described in this specification can be carried out in computing systems, which may include back-end components (e.g., data servers), middleware components (e.g., application servers), or front-end components (e.g., client computers with graphical user interfaces, web browsers, or apps through which users can interact with implementations of the subjects described in this specification), or any combination of one or more such back-end components, middleware components, or front-end components. Components of the system may be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0238] A computing system may include clients and servers. Clients and servers are typically geographically distant and usually interact via a communication network. The client-server relationship is established by computer programs running on their respective computers that establish a client-server relationship between them. In some embodiments, the server transmits data (e.g., HTML web pages) to a user device, for example, to display data to a user interacting with the device acting as a client and to receive user input from said user. Data generated on the user device (e.g., due to user interaction) may be received from said device at the server.

[0239] While this specification contains numerous specific implementation details, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features are described above as acting in certain combinations, and even initially claimed in this way, one or more features from the claimed combination may, in some cases, be detached from said combination, and the claimed combination may be for sub-combinations or variations thereof.

[0240] Similarly, although the operations are depicted in a specific order in the drawings and referenced in a specific order in the claims, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all of the shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system modules and components in the embodiments described above should not be construed as requiring such separation in all embodiments, and should be understood as meaning that the described program components and systems can generally be integrated together in a single software product or packaged in multiple software products.

[0241] Specific embodiments of this subject matter have been described. Other embodiments are also within the scope of the appended claims. For example, the actions set forth in the claims can be performed in different orders and still achieve the desired result. For example, the processes depicted in the figures do not necessarily need to be performed in the specific order shown or in sequence to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method executed by one or more computers, the method comprising: Obtain network inputs characterizing proteins and one or more ligands; The network input characterizing the protein and the one or more ligands is processed using an embedded neural network to generate protein-ligand embeddings of the protein and the one or more ligands; as well as A generative model is used, and a predicted joint three-dimensional (3D) structure of the protein and one or more ligands is generated when the generative model is conditioned on the protein-ligand embedding. The predicted joint 3D structure of the protein and the one or more ligands defines the corresponding predicted three-dimensional spatial position of each atom in the protein and each atom in each of the one or more ligands.

2. The method of claim 1, wherein the embedding neural network comprises a protein embedding neural network and a ligand embedding neural network, and wherein processing the network input characterizing the protein and the one or more ligands using the embedding neural network to generate the protein-ligand embedding comprises: The protein embedding neural network is used to process the data characterizing the protein to generate the protein embedding of the protein; The ligand embedding neural network is used to process data characterizing the one or more ligands to generate ligand embeddings for the one or more ligands; as well as The protein embedding and the ligand embedding are processed to generate the protein-ligand embedding.

3. The method of claim 2, wherein the protein embedding comprises the corresponding amino acid embedding of each amino acid in the protein; The ligand embedding includes the corresponding atom embedding of each atom in each of the one or more ligands; and The processing of the protein embedding and the ligand embedding to generate the protein-ligand embedding includes: Data defining 1D amino acid embedding and atom embedding sequences is generated by connecting the following: (i) the amino acid embedding of the protein embedding, and (ii) the atom embedding of the ligand embedding; The protein-ligand embeddings described therein are derived from the 1D amino acid embeddings and atom embedding sequences.

4. The method of claim 3, wherein processing the protein embedding and the ligand embedding to generate the protein-ligand embedding further comprises: The 1D amino acid embedding and atom embedding sequences are converted into a two-dimensional (2D) embedding array; The protein-ligand embeddings described therein are derived from the 2D embedding array.

5. The method of claim 4, wherein the 2D embedding array comprises a plurality of atom-atom embeddings, each atom-atom embedding originating from a corresponding pair of atom embeddings of the ligand embedding.

6. The method of any one of claims 4 to 5, wherein the 2D embedding array comprises a plurality of amino acid-amino acid embeddings, each amino acid-amino acid embedding being derived from a corresponding pair of amino acid embeddings of the protein embedding.

7. The method of any one of claims 4 to 6, wherein the 2D embedding array comprises a plurality of amino acid-atom embeddings, each amino acid-atom embedding originating from: (i) a corresponding atom embedding of the ligand embedding, and (ii) a corresponding amino acid embedding of the protein embedding.

8. The method of any one of claims 4 to 7, wherein converting the 1D amino acid embedding and atom embedding sequences into the 2D embedding array comprises: Apply an outer product operation to the 1D amino acid embedding and atom embedding sequences; or A 2D ligation operation is applied to the 1D amino acid embedding and atom embedding sequences.

9. The method of any one of claims 4 to 8, wherein the embedded neural network further comprises a fusion neural network; and The processing of the protein embedding and the ligand embedding to generate the protein-ligand embedding further includes: The fusion neural network is used to process the 2D embedding array to generate an updated 2D embedding array; The updated 2D embedding array defines the protein-ligand embedding.

10. The method of claim 9, wherein the fusion neural network comprises a sequence of self-attention blocks, wherein each self-attention block is configured to perform operations including: One or more self-attention operations are applied to the input 2D embedding array to update the input 2D embedding array.

11. The method of claim 10, wherein for one or more of the self-attention blocks, the self-attention operation comprises one or more line-by-line self-attention operations.

12. The method of any one of claims 10 to 11, wherein for one or more of the self-attention blocks, the self-attention operation comprises one or more column-by-column self-attention operations.

13. The method of any one of claims 10 to 12, wherein for one or more of the self-attention blocks, the self-attention operation comprises one or more triangular self-attention operations.

14. The method of any preceding claim, wherein the network input comprises data defining one or more of the following: the amino acid sequence of the protein; multiple sequence alignment (MSA) of the protein; the corresponding structure of each of the one or more template proteins; and a representation of the corresponding chemical structure of each of the one or more ligands.

15. The method of any of the preceding claims, wherein the generative model is a generative diffusion model comprising a denoising neural network.

16. The method of claim 15, wherein using the generative model and generating the predicted joint 3D structure of the protein and the one or more ligands conditioned on the protein-ligand embedding comprises: Generate positional data defining the corresponding initial positions of each atom in the complex of the protein and one or more ligands; The location data is denoised within the time step sequence using the denoising neural network and when the denoising neural network is conditioned on the protein-ligand embedding; The predicted joint 3D structure of the protein and the one or more ligands is defined by the positional data following the last time step in the time step sequence.

17. The method of claim 16, wherein generating position data defining the corresponding initial positions of each atom in the complex comprises: The corresponding initial position of each atom in the complex is sampled from a probability distribution in 3D space.

18. The method of any one of claims 16 to 17, wherein using the denoising neural network and, when the denoising neural network is conditioned on the protein-ligand embedding, denoising the location data within the time step sequence comprises each of one or more time steps in the time step sequence: Receive current position data defining the corresponding current position of each atom in the complex at the time step; Using the denoising neural network and generating a denoised output when the denoising neural network is conditioned on the protein-ligand embedding; and The denoised output is used to generate position data that defines the position of each atom in the complex at the next time step.

19. The method of claim 18, wherein the denoising output includes a corresponding prediction error for each atom in the complex at the current position of the time step.

20. The method of any one of claims 18 to 19, wherein using the denoising neural network and generating the denoised output when the denoising neural network is conditioned on the protein-ligand embedding comprises: The encoder block of the denoising neural network generates a set of atomic embeddings, wherein each atomic embedding represents one or more atoms in the complex and is based at least in part on the corresponding current spatial position of the one or more atoms at the time step. The atomic embedding set is processed using the update block of the denoising neural network to generate an updated atomic embedding set; as well as The updated set of atomic embeddings is processed to generate the denoised output.

21. The method of claim 20, wherein the atom embedding set comprises a corresponding atom embedding representing each atom contained in each of the one or more ligands.

22. The method of any one of claims 20 to 21, wherein the atom embedding set comprises a corresponding atom embedding representing each atom contained in each amino acid of the protein.

23. The method of claims 20 to 21, wherein for each amino acid in the protein, the set of atomic embeddings comprises a corresponding atomic embedding that jointly represents all the atoms contained in the amino acid.

24. The method of any one of claims 20 to 23, wherein each atomic embedding in the set of atomic embeddings is based on: (i) the current position of the atom at the time step; and (ii) a corresponding conditional embedding of the atom selected from the set of embeddings included in the protein-ligand embeddings.

25. The method of claim 24, wherein for each atom embedding representing an atom in the complex contained in the ligand, the conditional embedding of the atom includes an atom-to-atom embedding corresponding to the atom in the protein-ligand embedding.

26. The method of any one of claims 24 to 25, wherein for each atom embedding of an atom in the complex representing an amino acid of the protein, the conditional embedding of the atom comprises an amino acid-amino acid embedding corresponding to the amino acid in the protein-ligand embedding.

27. The method of any one of claims 20 to 26, wherein the update block of the denoising neural network comprises a sequence of self-attention blocks; Each of the self-attention blocks is configured to apply one or more self-attention operations to the current set of atomic embeddings to update the current set of atomic embeddings; Each of the one or more self-attention operations is conditioned on the protein-ligand embedding.

28. The method of claim 27, wherein applying a self-attention operation to the current set of atomic embeddings to update the current set of atomic embeddings comprises: Based on the current set of atomic embeddings, generate a corresponding intermediate attention score for each pair of current atomic embeddings in the current set of atomic embeddings; Based on the protein-ligand embedding, a corresponding attention score bias is generated for each pair of current atom embeddings in the current atom embedding set; Based on the intermediate attention score and the attention score deviation, a corresponding final attention score is generated for each pair of current atomic embeddings in the current atomic embedding set; as well as Update the current set of atomic embeddings using the final attention score.

29. The method of claim 28, wherein generating the attention score bias for each pair of current atomic embeddings in the current atomic embedding set comprises: Add the intermediate attention score for the current atom embedding to the attention score deviation for the current atom embedding.

30. The method of any one of claims 28 to 29, wherein generating the attention score bias for each pair of current atomic embeddings in the current atomic embedding set comprises: The attention score bias is generated by processing the appropriate conditional embeddings selected from the set of embeddings included in the protein-ligand embeddings using a projection neural network.

31. The method of claim 30, wherein for each pair of current atom embeddings comprising (i) a first atom embedding representing a first atom contained in a ligand and (ii) a second atom embedding representing a second atom contained in a ligand, the selected conditional embedding comprises an atom-to-atom embedding corresponding to the first atom and the second atom in the protein-ligand embedding.

32. The method of any one of claims 30 to 31, wherein for each pair of current atom embeddings comprising (i) a first atom embedding representing a first atom contained in an amino acid and (ii) a second atom embedding representing a second atom contained in a ligand, the selected conditional embedding comprises an amino acid-atom embedding corresponding to the amino acid containing the first atom and (ii) the second atom in the protein-ligand embedding.

33. The method of any one of claims 30 to 32, wherein for each pair of current atom embeddings comprising (i) a first atom embedding representing a first atom contained in a first amino acid and (ii) a second atom embedding representing a second atom contained in a second amino acid, the selected conditional embedding comprises an amino acid-amino acid embedding corresponding to (i) the first amino acid and (ii) the second amino acid in the protein-ligand embedding.

34. The method of any one of claims 30 to 33, wherein for each pair of current atom embeddings comprising (i) a first atom embedding jointly representing all atoms in a first amino acid and (ii) a second atom embedding representing all atoms in a second amino acid, the selected conditional embedding comprises an amino acid-amino acid embedding corresponding to (i) the first amino acid and (ii) the second amino acid in the protein-ligand embedding.

35. The method of any one of claims 30 to 34, wherein for each pair of current atom embeddings comprising (i) a first atom embedding jointly representing all atoms in an amino acid and (ii) a second atom embedding representing atoms contained in a ligand, the selected conditional embedding comprises an amino acid-atom embedding corresponding to the (i) amino acid and (ii) atom in the protein-ligand embedding.

36. The method of any of the preceding claims, further comprising: One or more ligands to be physically synthesized are selected based at least in part on the predicted joint 3D structure of the protein and the one or more ligands. as well as The selected ligands are synthesized physically.

37. The method of any of the preceding claims, further comprising: The protein to be physically synthesized is selected based at least in part on the predicted joint 3D structure of the protein and one or more ligands; as well as The protein is synthesized physically.

38. A method comprising: For each ligand in the ligand set, the corresponding predicted joint 3D structure of the ligand and protein is generated using the method described in any one of claims 1 to 35; For each ligand in the ligand set, the corresponding predicted binding affinity of the ligand to the protein is determined based on the predicted joint 3D structure of the ligand and the protein. as well as One or more ligands are selected from the set of ligands for physical synthesis, based at least in part on the predicted binding affinity.

39. The method of claim 38, further comprising physically synthesizing one or more selected ligands.

40. A method comprising: For each protein in the protein set, the corresponding predicted joint 3D structure of the protein and ligand is generated using the method described in any one of claims 1 to 35; For each protein in the protein set, the predicted binding affinity of the ligand to the protein is determined based on the predicted joint 3D structure of the ligand and the protein. as well as One or more proteins are selected from the protein set for physical synthesis, based at least in part on the predicted binding affinity.

41. The method of claim 40, further comprising physically synthesizing one or more selected proteins.

42. A system comprising: One or more computers; as well as One or more storage devices are communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operation of the corresponding method as described in any one of claims 1 to 35.

43. A non-transient computer storage medium that stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operation of the corresponding method as claimed in any one of claims 1 to 35.

44. A method for obtaining a ligand, wherein the ligand is a ligand for a drug or an industrial enzyme, the method comprising: For each of one or more candidate ligands: (a) Performing the method of any one of claims 1 to 35 to determine the predicted structure of a complex comprising the target protein molecule and the candidate ligand; and (b) Evaluate the interaction between the candidate ligand and the target protein molecule based on the predicted structure; as well as Based on the evaluation results, one or more candidate ligands are selected as the ligands.

45. The method of claim 44, wherein the target protein molecule comprises a receptor or an enzyme, and wherein the ligand is an agonist or antagonist of the receptor or enzyme.

46. ​​The method of claim 44 or 45, wherein the ligand is a drug, and the method comprises: Perform steps (a) and (b) for each of the multiple target protein molecules; as well as Select one or more of the candidate ligands as the ligand to i) obtain a ligand that interacts with each of the target protein molecules, or ii) obtain a ligand that interacts with only one of the target protein molecules.

47. The method of claim 44, wherein the ligand comprises an antibody or aptamer, and the target protein molecule comprises an antibody or aptamer target, particularly a viral or cancer cell protein, and wherein the antibody or aptamer binds to the antibody or aptamer target to provide a therapeutic effect.

48. The method of any one of claims 44 to 47, wherein the ligand is a polypeptide ligand, a polynucleotide ligand, or a polynucleotide ligand.

49. A method for obtaining a diagnostic antibody or aptamer marker for a disease, the method comprising: Select target protein molecules; For each of one or more candidate antibodies or aptamers: Perform the method as described in any one of claims 1 to 35 to determine the predicted structure of the complex comprising the candidate antibody or aptamer and the target protein molecule; and Evaluate the interaction between the candidate antibody or aptamer and the target protein molecule; as well as Based on the results of the evaluation, one of the one or more candidate antibodies or aptamers is selected as the diagnostic antibody or aptamer marker.

50. The method of any one of claims 44 to 49, wherein evaluating the interaction of one of the candidate ligands comprises determining an interaction score of the candidate ligand, wherein the interaction score comprises a measure of the interaction between the candidate ligand and the target molecule.

51. The method of any one of claims 44 to 50, further comprising synthesizing the ligand or diagnostic antibody or aptamer marker.

52. The method of claim 51, further comprising testing the bioactivity of the ligand or diagnostic antibody or aptamer marker in vitro and in vivo.

53. A method for determining the structure of a molecular complex comprising a protein and one or more ligands, comprising: Experimental techniques are applied to physical samples containing the molecular complex in order to measure experimental signals based on the structure of the molecular complex; Perform the method as described in any one of claims 1 to 35 to determine the predicted structure of the molecular complex; The structure of the molecular complex is determined using the experimental signal and the predicted structure of the molecular complex.

54. The method of claim 53, wherein the experimental technique comprises one or more of the following: X-ray crystallography, nuclear magnetic resonance, and electron microscopy.

55. A ligand synthesized by performing the method of claim 39.

56. One or more non-transient computer storage media storing ligand data defining ligands, wherein the ligands are selected from a set of candidate ligands by performing an operation comprising: For each candidate ligand in the candidate ligand set, the corresponding predicted joint 3D structure of the candidate ligand and protein is generated using the method described in any one of claims 1 to 35; For each candidate ligand in the candidate ligand set, the predicted binding affinity of the candidate ligand to the protein is determined based on the predicted joint 3D structure of the candidate ligand and the protein. as well as Based at least in part on the predicted binding affinity, one or more candidate ligands are selected from the ligand set for physical synthesis.

57. A protein synthesized by performing the method of claim 41.

58. One or more non-transient computer storage media storing ligand data defining proteins, wherein the proteins are selected from a protein set by performing an operation, the operation comprising: For each protein in the protein set, the corresponding predicted joint 3D structure of the protein and ligand is generated using the method described in any one of claims 1 to 35; For each protein in the protein set, the predicted binding affinity of the ligand to the protein is determined based on the predicted joint 3D structure of the ligand and the protein. as well as One or more proteins are selected from the protein set for physical synthesis, based at least in part on the predicted binding affinity.

Images