Protein design using diffusion models operating in whole-atom representation

Abstract

A method, system, and apparatus for designing proteins, including a computer program encoded on a computer storage medium. In one embodiment, the method includes generating noise-containing molecular structure data sampled from a noise distribution that defines a corresponding initial spatial position for each atom in a given set of possible atoms for each position in the amino acid sequence of a protein, and processing the noise-containing molecular structure data using a diffusion model including a denoising neural network to generate denoised molecular structure data that defines a denoised version of the noise-containing molecular structure data.

Description

Background Art

[0001] This specification relates to protein design.

[0002] A protein contains an amino acid sequence. An amino acid is an organic compound containing an amino functional group and a carboxyl functional group, as well as a side chain (i.e., atomic group) specific to the amino acid.

[0003] Protein folding refers to the physical process by which an amino acid sequence folds into a three-dimensional structure. The structure of a protein defines the three-dimensional (3D) structure of the atoms within the amino acid sequence of the protein after the protein has undergone protein folding. When in a sequence linked by peptide bonds, an amino acid can be called an amino acid residue.

[0004] Predictions can be made using a machine learning model. A machine learning model receives an input and generates an output, such as a predicted output, based on the received input. Some machine learning models are parametric models and generate an output based on the received input and the values of the model's parameters. Some machine learning models are deep models that use multiple layers of the model to generate an output for the received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers, and each layer applies a non-linear transformation to the received input to generate an output.

Summary of the Invention

Problems to be Solved by the Invention

[0005] This specification outlines a system for performing protein design implemented as a computer program on one or more computers at one or more locations.

Means for Solving the Problems

[0006] The systems described herein can be used to design novel proteins, such as proteins that bind to specific ligands, e.g., small molecule ligands or protein ligands. The systems may include a diffusion model in which, at each denoised repeat in a sequence of denoised repeats, a denoised neural network is used to process the "noisy" version of the protein in order to predict the denoised version of the protein.

[0007] Protein design involves determining the amino acid sequence of the protein, that is, determining each amino acid at each position in the amino acid sequence of the protein. Each amino acid is composed of atoms in its respective position. The system can switch between or smoothly interpolate between sets of atoms being dealt with, because the types of amino acids in the amino acid sequence change during the diffusion process (and therefore the atoms making up the amino acid sequence at each position change).

[0008] Certain embodiments of the subject matter described herein may be implemented to achieve one or more of the following advantages:

[0009] The protein design system described herein can implement a diffusion model that can control all possible atoms for each possible protein residue. The system can then infer the amino acid sequence of the protein directly from the designed coordinates. Thus, the system can simultaneously sample novel structures and corresponding amino acid sequences. Rather than having to go through the stage of "introducing and not introducing" unused atoms, the system trains the diffusion model to "hide" unused atoms at disposable spatial locations of the corresponding residue (e.g., alpha-carbon positions), that is, spatial locations where the atom is not part of the designed protein. The system can then automatically discard unused atoms at disposable spatial locations.

[0010] A protein design system can condition the structural information from a target molecule with a noise reduction process. This conditional setting is a process that derives a diffusion model in additional context, in this case, for example, the atomic coordinates of the target molecule. Thus, the protein design system can not only re-establish the scaffold but also design novel functional protein components that act as binders or catalysts (enzymes) for the target molecule.

[0011] A protein design system can jointly determine, for example, the (i) structure and (ii) amino acid sequence of a protein that is predicted to bind to a target molecule. In contrast, conventional systems generally handle these tasks separately; for example, one conventional system may be configured to process the amino acid sequence of a protein to determine the protein structure, while another conventional system may be configured to process the protein structure to determine the amino acid sequence of a protein that realizes that structure. By simultaneously predicting the protein structure and amino acid sequence in a single forward pass, for example, using a diffusion-generative machine learning model, the system can reduce the consumption of computational resources, such as memory and computing power, compared to previous systems that performed these tasks separately.

[0012] Details of one or more embodiments of the subject matter of this specification are described in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from this specification, the drawings, and the claims. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram of an exemplary protein design system. [Figure 2] This is an illustrative process flowchart for designing proteins using a diffusion model. [Figure 3] This is a block diagram of an exemplary diffusion model. [Figure 4] An example of an exemplary embedding scheme for specifying the position of atoms within amino acids in a protein is shown. [Figure 5] An example of an exemplary embedding scheme for specifying the positions of heavy atoms within the amino acids of a protein is shown. [Figure 6] An example of an exemplary embedding scheme for specifying the positions of groups of atoms within the amino acids of a protein is shown. [Figure 7] FIG. is a flowchart of an exemplary process for generating protein molecular structure data using a noise removal neural network. [Figure 8] FIG. is a flowchart of an exemplary process for training a noise removal neural network to generate protein molecular structure data. [Figure 9] FIG. is a flowchart of an exemplary process for processing noise-removed molecular structure data to identify atoms contained within the amino acids of a protein. [Figure 10] Examples of the disposable spatial positions of amino acids are shown. [Figure 11] An exemplary protein designed to bind to a target protein is shown. [Figure 12] An exemplary protein designed to include a specified amino acid sequence is shown.

MODE FOR CARRYING OUT THE INVENTION

[0014] Like reference numerals and names in the various drawings indicate like elements.

[0015] FIG. 1 shows an exemplary protein design system 100. Protein design system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations where the systems, components, and techniques described below are implemented.

[0016] The protein design system 100 can generate protein design data 102 that characterizes the designed protein using the diffusion model 104. As an example, the protein design data 102 can specify the structure of the designed protein. As another example, the protein design data 102 can identify the amino acid sequence of the designed protein. As another example, the protein design data 102 can identify both (i) the structure of the designed protein and (ii) the amino acid sequence of the designed protein.

[0017] The system 100 can design a protein for a specific function using the diffusion model 104. For example, the system 100 can design a protein that binds to a specific ligand (e.g., a small molecule ligand or a protein ligand). As a further example, the system 100 can receive data characterizing a specified ligand and generate protein design data 102 for the designed protein based on the received data characterizing the specified ligand. As another example, the system 100 can design a protein that binds to a specified target protein.

[0018] To design a protein, the system 100 can use the diffusion model 104 to process the molecular structure data 106 containing noise to generate the denoised molecular structure data 108 that characterizes the designed protein. The molecular structure data 106 containing noise and the denoised molecular structure data 108 can include corresponding amino acid embeddings for each amino acid in the amino acid sequence of the designed protein. Each amino acid embedding can include a sequence of atom embeddings for each of a set (i.e., collection) of predetermined atoms that can be included within the corresponding amino acid of the protein. The set of predetermined atoms can include multiple instances of the same type of atom (i.e., element). Each atom embedding can characterize the spatial position of the corresponding atom.

[0019] The noisy molecular structure data 106 specifies the initial spatial position for each atom that may be contained within the amino acids of the designed protein. The diffusion model 104 processes the noisy molecular structure data 106 following a denoising process to generate denoised molecular structure data 108 that characterizes the designed protein. In some embodiments, the denoising process is an iterative process, and the diffusion model 104 can generate denoised molecular structure data 108 by iteratively denoising the noisy molecular structure data 106 over a sequence of denoising iterations. Following the denoising process, the diffusion model 104 can determine, for each amino acid of the designed protein, both (i) the positions of atoms that may be contained within the amino acid, and (ii) which atoms should be contained within the amino acid.

[0020] A specific amino acid within a designed protein may not contain all atoms from a given set of atoms that can be present within that amino acid. The diffusion model 104 can determine which atoms are included within the amino acids of the designed protein and can encode which atoms are included within each amino acid of the designed protein by generating denoised molecular structure data 108. In particular, the diffusion model 104 can encode which atoms are included within the designed protein using the generated spatial positions of the atoms. For example, each amino acid in a designed protein may be associated with a "disposable" location (spatial position) for that amino acid, and the diffusion model 104 can encode a given atom being excluded from the designed protein by placing the atom in that amino acid's "disposable" location.

[0021] The diffusion model is explained in more detail below with reference to Figure 3.

[0022] The protein design system 100 may include an initialization system 110. The initialization system 110 can generate noisy molecular structure data 106. In particular, the initialization system 110 can initialize the noisy molecular structure data 106 by sampling some or all of the initial spatial positions of atoms that may be contained in the designed protein from a noise distribution (e.g., Gaussian distribution, uniform distribution, etc.).

[0023] The initialization system 110 can receive conditioning data 112 and generate molecular structure data 106 containing noise based on the conditioning data 112.

[0024] For example, conditioning data 112 can specify a predetermined spatial position for one or more atoms of the designed protein, and the initialization system 110 can generate noisy molecular structure data 106 to specify that atom is a static atom with a spatial position not modified by the diffusion model 104. This allows the protein design system 100 to design a protein to include a static atom. For example, a static atom may be part of a specified protein residue (e.g., an atom of the protein associated with a protein binding site), and the system 100 can design a protein to include the specified protein residue. An example of designing a protein to include a target sequence of amino acids is shown below with reference to Figure 12.

[0025] In another example, the conditioning data 112 can characterize the target molecule (e.g., for a designed protein, the target small molecule ligand, target protein ligand, target protein, etc.), and the initialization system 110 can generate noisy molecular structure data 106 to define the structure of the target molecule. Generally, the noisy molecular structure data 106 can specify the initial spatial position of each atom of the target molecule. In particular, the noisy molecular data 106 may include one or more embeddings that characterize the spatial positions of the atoms of the target molecule. As another example, the conditioning data 112 can characterize the electrostatic potential or surface charge distribution that the designed protein should have or create.

[0026] If the noisy molecular data 106 includes an embedding of the target molecule, the embedding of the target molecule can share the same dimensions as the amino acid embedding of the designed protein. For example, the embedding of the target molecule could be an atomic embedding from a predetermined set of atoms that may be present in the designed protein.

[0027] When the initialization system 110 generates noisy molecular structure data 106 to define the structure of the target molecule, the initialization system 110 may also generate noisy molecular structure data 106 to specify one or more atoms in the target molecule as static atoms having predetermined spatial positions that are not modified by the diffusion model 104. For example, the initialization system 110 may generate noisy molecular structure data 106 to specify all atoms of the target molecule as static atoms having predetermined spatial positions that are not modified by the diffusion model 104. This enables the protein design system 100 to design a protein that binds to the target molecule. An example of designing a protein that binds to a target molecule is shown below with reference to Figure 11.

[0028] The protein design system 100 includes a structure processing system 114. The structure processing system 114 can process the denoised molecular structure data 108 generated by the diffusion model 104 to generate protein design data 102.

[0029] As described above, the denoised molecular structure data 108 can encode which atoms are included in each amino acid of the designed protein. The structure processing system 114 can process the denoised molecular structure data 108 to remove atoms that are excluded from the designed protein. For example, if the denoised molecular structure data 108 encodes which atoms are included in the designed protein using the spatial positions of the generated atoms, the structure processing system 114 can remove atoms based on their generated spatial positions. An exemplary process for removing atoms based on their generated spatial positions is described in more detail below with reference to Figure 9.

[0030] The structure processing system 114 can perform various operations to process the denoised molecular structure data 108 to determine the types of amino acids in the amino acid sequence of the designed protein.

[0031] For example, the structure processing system 114 can determine the types of amino acids in the amino acid sequence of a designed protein using a predetermined mapping between sets of atoms and amino acids. As a further example, if the structure processing system 114 removes atoms from a designed protein, it can use the predetermined mapping to determine the types of each amino acid in the designed protein based on the remaining atoms of the amino acid.

[0032] As another example, the structure processing system 114 may include a sequencing machine learning model, which can be used to process the denoised molecular structure data 108 to determine the amino acid sequence. For example, the diffusion model 104 may incorrectly exclude or include atoms in the denoised molecular structure data 108, and the structure processing system 114 can use the sequencing machine learning model to determine the corrected amino acid sequence of the designed protein. As yet another example, the diffusion model 104 can generate a rough spatial structure of the designed protein, and the structure processing system 114 can use the sequencing machine learning model to generate the final amino acid sequence of the designed protein based on the rough spatial structure generated by the diffusion model 104. An example of a sequencing machine learning model is described in Dauparas et al. in “Robust Deep Learning- Based Protein Sequence Design Use ProteinMPNN”, Science 6615(378), p49-56(2022).

[0033] An exemplary process for designing a protein using diffusion model 104 is described in more detail below with reference to Figure 2.

[0034] After designing a protein, the protein design system 100 can output protein design data 102 to an external system for synthesizing the designed protein. For example, the protein design data 102 can specify the amino acid sequence of the designed protein, and the external system can synthesize a protein having the specified amino acid sequence. As a further example, the protein design data 102 can include instructions for synthesizing the designed protein (e.g., necessary conditions for synthesizing the designed protein), and the external system can synthesize the designed protein according to the instructions.

[0035] Figure 2 is a flowchart illustrating an exemplary process for designing a protein using a diffusion model. For convenience, the process 200 is described as being performed by one or more computer systems located in one or more locations. For example, a protein design system, e.g., the protein design system 100 of Figure 1, appropriately programmed according to this specification, can perform the process 200.

[0036] In some embodiments, the system can receive conditioning data (e.g., from a user) for designing a protein (step 202). Based on the conditioning data, the system can generate the designed protein. For example, the conditioning data can specify a predetermined spatial position of one or more atoms (e.g., atoms of a specified protein residue) in the designed protein, and the system can design a protein to contain the specified atoms. In another embodiment, the conditioning data can characterize a target molecule (e.g., a target small molecule ligand of the designed protein, a target protein ligand, a target protein, etc.), and the system can design a protein that binds to the target molecule.

[0037] The system can generate noisy molecular structure data that defines the initial spatial positions of atoms that may be present in the designed protein (step 204). In particular, for each amino acid in the designed protein, the noisy molecular structure data specifies the spatial position of each of a given set of atoms that may be present within the amino acid.

[0038] A given set of atoms that may be contained within an amino acid can include each atom present in a given set of amino acids (or amino acid residues) (e.g., skeletal atoms, side chain atoms, etc.). That is, each atom in a given set of atoms can correspond to each of at least one atom of the amino acids in the given set of amino acids. Thus, a given set of atoms can contain multiple instances of the same type of atom (i.e., element). For example, if a given set of amino acids consists of a glycine residue and an alanine residue, the given set of atoms may include three carbon atoms, five hydrogen atoms, one nitrogen atom, and one oxygen atom, and the given set of atoms can be used to represent either the glycine residue or the alanine residue. As a specific example, a given set of atoms may consist of more than 30 atoms (e.g., 37 atoms, 39 atoms, 41 atoms, etc.) and may be found within the following sets of amino acids, which are alanine, cysteine, aspartic acid, glutamic acid, phenylalanine, glycine, histidine, isoleucine, lysine, leucine, methionine, asparagine, pyrrolicine, proline, glutamine, arginine, serine, threonine, selenocysteine, valine, tryptophan, and tyrosine.

[0039] In general, a given set of atoms can contain more atoms than are present in any individual amino acid of a protein. After denoising noisy molecular structure data, the system can determine which atoms are excluded from each amino acid of a protein, and can determine the type of amino acid based on an appropriate subset of the given set of atoms contained within the amino acid. An exemplary process for determining which atoms are excluded from the amino acids of a protein is described in more detail below with reference to Figure 9.

[0040] Noisy molecular structure data can designate one or more atoms as static atoms with a given spatial position. For example, if the system receives conditioning data that designates a given spatial position for one or more atoms of a designed protein, the noisy molecular data can include the designated spatial position and designate the corresponding atom as a static atom of the protein. As another example, if the system receives conditioning data that characterizes a target molecule (e.g., for a designed protein, a target small molecule ligand, a target protein ligand, a target protein, etc.), the noisy molecular data can characterize the designated spatial position for an atom of the target molecule and designate the corresponding atom of the molecule as a static atom.

[0041] A molecular structure containing noise can include amino acid embeddings for each amino acid within a designed protein. Each amino acid embedding can include corresponding atomic embeddings for each of a given set of atoms. Each atomic embedding can be a set of numerical values ​​characterizing the spatial position of the corresponding atom.

[0042] When noisy molecular structure data characterizes the atomic positions of a target molecule for a designed protein, the noisy molecular structure data can include atomic embeddings for each atom of the target molecule. Atomic embeddings for atoms of the target molecule can share the same dimensions as atomic embeddings for atoms of the designed protein. In particular, noisy molecular structure data can characterize a target molecule using a given set of atoms, and a common atomic embedding scheme can be used to represent both the atoms of the target molecule and the atoms of the designed protein.

[0043] In some embodiments, the system can generate noisy molecular structure data by sampling the initial spatial positions of some or all atoms of the designed protein from a noise distribution of atomic embeddings (e.g., Gaussian distribution, uniform distribution, etc.). In particular, the system can sample atomic embedding values ​​within the noisy molecular structure data. If the noisy molecular structure data designates a particular atom as a static atom having a predetermined spatial position, the system can use the predetermined spatial position of the static atom rather than sampling the initial spatial position of the static atom.

[0044] The system can process noisy molecular structure data using a diffusion model to generate denoised molecular structure data (step 206). The denoised molecular structure data can define a denoised version of the noisy molecular structure data and specify the final positions of atoms within the designed protein. An exemplary diffusion model is described in more detail below with reference to Figure 3.

[0045] In some embodiments, when noisy molecular structure data designates a particular atom as a static atom having a predetermined spatial position, the system can fix and maintain the spatial position of the static atom when generating denoised molecular structure data. In particular, the system can ensure that the denoised molecular structure data designates the static atom as having the same predetermined spatial position as specified by the noisy molecular structure data.

[0046] In some embodiments, the diffusion model may include a denoising neural network, and the system may use the denoising neural network to iteratively denoise noisy molecular structure data over a sequence of denoising iterations to generate denoised molecular structure data. An exemplary process of iteratively denoising noisy molecular structure data using a denoising neural network is described below with reference to Figure 7.

[0047] In some embodiments, the system can process denoised molecular structure data to determine the atoms contained within the designed protein (step 208). In particular, for each amino acid of the designed protein, the system can identify a suitable subset of a given set of atoms that are contained within the amino acid, based on the denoised molecular structure data. An exemplary process for processing denoised molecular structure data to identify the atoms contained within the amino acids of the designed protein is described in more detail below with reference to Figure 9.

[0048] In some embodiments, the system can process denoised molecular structure data to determine the amino acid sequence of the designed protein (step 210). Generally, the system can process denoised molecular structure data to determine the type of each amino acid in the amino acid sequence of the protein. For example, the system can use a sequencing machine learning model to process some or all of the denoised molecular structure to generate data that defines the type of each amino acid in the amino acid sequence of the protein.

[0049] As another example, the system can store a mapping from a suitable subset of a given set of atoms to the amino acid types of a given set of amino acids. When the system identifies that a suitable subset of a given set of atoms is present in each amino acid of a designed protein, the system can use the stored mapping to determine the type of amino acid. For example, if the amino acids of a designed protein contain the same set of atoms as a particular amino acid from a given set of amino acids, the system can identify the amino acid of the designed protein as that particular amino acid from the given set of amino acids. As a further example, for each amino acid of a designed protein, the system can identify the amino acid as that particular amino acid from a given set of amino acids that maximizes the degree of similarity (Jacquard coefficient) between the set of atoms in the amino acid of the designed protein and the set of atoms in the particular amino acid.

[0050] The system can ultimately output data characterizing the designed protein (step 212). Specifically, the system outputs data characterizing the designed protein (e.g., amino acid sequence of the designed protein, structure of the designed protein, synthesis instructions for the designed protein, etc.) to the protein synthesis system, which can then synthesize a protein having the amino acid sequence of the designed protein.

[0051] Figure 3 is a block diagram of an exemplary diffusion model 104. Diffusion model 104 is an example of a system that runs as a computer program on one or more computers in one or more locations where the systems, components, and techniques described below are performed.

[0052] As described above, the diffusion model 104 can process the noisy molecular structure data 106 to generate denoised molecular structure data 108 that characterizes the designed protein. The noisy molecular structure data 106 and the denoised molecular structure data 108 may include embeddings that characterize the positions of atoms that may be present in the designed protein. Examples of embedding schemes for the noisy molecular structure data 106 and the denoised molecular structure data 108 are described in more detail below with reference to Figures 4 to 6.

[0053] The diffusion model 104 can generate denoised molecular structure data 108 using an iterative denoising process. In particular, the diffusion model 104 may include a denoising neural network 302 configured to denoise and update the noisy molecular structure data 106 over a sequence of denoising iterations.

[0054] In each denoising iteration, the denoising neural network 302 can process the noisy molecular structure data 106 for the iteration and generate (at least partially) denoised molecular structure data 108 for the iteration. The denoising neural network 302 can use the denoised molecular structure data 108 for the iteration as input to the next denoising iteration (e.g., as the noisy molecular structure data 106). In some embodiments, the denoising neural network 302 can generate the noisy molecular structure data 106 for the next iteration by combining random noise (e.g., sampled from a Gaussian distribution, a uniform distribution, etc.) with the denoised molecular structure data 108 for the denoising iteration.

[0055] An exemplary process for generating denoised molecular structure data of proteins using a denoising neural network is described in more detail below with reference to Figure 7.

[0056] The denoising neural network 302 can have any of a variety of neural network architectures. For example, the denoising neural network 302 may include one or more self-attention layers capable of processing the noisy molecular structure data 106 according to a self-attention mechanism between atomic embeddings in the noisy molecular structure data 106. As another example, the denoising neural network 302 may be a graph neural network, which can process an input graph representing the noisy molecular structure data 106 to generate denoised molecular structure data 108. Other architectures for the denoising neural network 302 may also be used, such as a multilayer perceptron architecture, a recurrent neural network architecture, etc.

[0057] Figure 4 shows an exemplary embedding scheme that specifies the positions of atoms within amino acids in a protein.

[0058] As illustrated, the protein is represented by a sequence of amino acid embeddings 400, which includes amino acid embeddings 402-X, 402-Y, and 402-Z. Each amino acid embedding in sequence 400 contains information that identifies the composition and spatial structure of the corresponding amino acid from the protein. For example, as illustrated, amino acid embedding 402-Y identifies the composition and spatial structure of amino acid 404-Y of the protein. For illustrative purposes, amino acid 404-Y is illustrated as alanine, but the amino acid embedding may feature any of the various amino acids in the protein.

[0059] As illustrated, each amino acid embedding in sequence 400 includes multiple atomic embeddings representing a predetermined set of atoms that may be contained within the corresponding amino acid of the protein. For example, amino acid embedding 402-Y includes atomic embeddings 406-A to 406-N. Each of atomic embeddings 406-A to 406-N may specify the spatial location of an atom relative to amino acid 404-Y.

[0060] For example, in the following embedding scheme shown in Figure 4, each of the atomic embeddings 406-A to 406-N can specify the spatial position of an atom that may be contained within amino acid 404-Y. The type and spatial structure of amino acid 404-Y can be determined by which atoms are contained within amino acid 404-Y and by the spatial positions of the contained atoms represented by atomic embeddings 406-A to 406-N. As shown in the figure, amino acid 404-Y contains atoms 408-A to 408-J, and their spatial positions and elemental types are specified by the corresponding atomic embeddings 406-A to 406-J.

[0061] A given set of atoms in each amino acid of a protein can include each atom present in a given set of amino acids or amino acid residues. For example, atomic embeddings 406-A to 406-N may each represent an atom present in a given set of amino acids. As in the specific example, each atomic embedding 406-A to 406-N can represent one of the 37 atoms present in the following set of amino acids: alanine, cysteine, aspartic acid, glutamic acid, phenylalanine, glycine, histidine, isoleucine, lysine, leucine, methionine, asparagine, pyrrolicine, proline, glutamine, arginine, serine, threonine, selenocysteine, valine, tryptophan, and tyrosine.

[0062] In general, a given set of atoms can contain more atoms than are present in any individual amino acid of a protein. For example, amino acid 404-Y contains 10 of the 37 exemplary atoms represented by atomic embeddings 406-A to 406-N. The system can determine which atoms are excluded from each amino acid of a protein. An exemplary process for determining which atoms are excluded from the amino acids of a protein is described in more detail below with reference to Figure 9. As a further example, the system can use the exemplary process in Figure 9 to determine that the remaining 27 atoms represented by atomic embeddings 406-A to 406-N are excluded from amino acid 404-Y.

[0063] Figure 5 shows an example sequence of embeddings that specify the position of heavy atoms within the amino acids of a protein.

[0064] As illustrated, the protein is represented by a sequence of amino acid embeddings 500, which includes amino acid embeddings 502-X, 502-Y, and 502-Z. Each amino acid embedding in sequence 500 contains information that identifies the composition and spatial structure of the corresponding amino acid from the protein. For example, as illustrated, amino acid embedding 502-Y identifies the composition and spatial structure of amino acid 504-Y of the protein. For illustrative purposes, amino acid 504-Y is illustrated as alanine, but the amino acid embedding may feature any of the various amino acids in the protein.

[0065] As illustrated, each amino acid embedding in sequence 500 includes multiple atomic embeddings that represent a predetermined set of atoms that may be contained within the corresponding amino acid of the protein. For example, amino acid embedding 502-Y includes atomic embeddings 506-A to 506-N.

[0066] In some use cases, it may not be necessary to specify the positions of light atoms (e.g., hydrogen) to design a protein. In these use cases, the system described can use an embedding scheme that specifies only the positions of heavy atoms (e.g., atoms heavier than hydrogen) within the amino acids of the protein. For example, following the embedding scheme shown in Figure 5, each of the atomic embeddings 506-A to 506-N can specify the spatial position of an atom that may be contained within amino acid 504-Y. The type and spatial structure of amino acid 504-Y may be determined by which atoms are contained within amino acid 504-Y and by the spatial positions of the contained atoms represented by atomic embeddings 506-A to 506-N. As shown in the figure, amino acid 504-Y contains atoms 508-A to 508-E, and their spatial positions and elemental types are specified by the corresponding atomic embeddings 506-A to 506-E.

[0067] Figure 6 shows an exemplary embedding sequence that specifies the positions of groups of atoms within the amino acids of a protein.

[0068] As illustrated, the protein is represented by a sequence of amino acid embeddings 600, which includes amino acid embeddings 602-X, 602-Y, and 602-Z. Each amino acid embedding in sequence 600 contains information specifying the composition and spatial structure of the corresponding amino acid from the protein. For example, as illustrated, amino acid embedding 602-Y specifies the composition and spatial structure of amino acid 604-Y of the protein. For illustrative purposes, amino acid 604-Y is illustrated as alanine, but the amino acid embedding may feature any of the various amino acids in the protein.

[0069] In some embodiments, each amino acid embedding can specify the position of an amino acid component (e.g., an atom or group of atoms) that may be contained within the amino acid. As shown in the figure, each amino acid embedding in sequence 600 includes multiple component embeddings that represent a predetermined set of amino acid components that may be contained within the corresponding amino acid of the protein. For example, amino acid embedding 602-Y includes component embeddings 606-A to 606-N.

[0070] Following the embedding scheme shown in Figure 6, each of the component embeddings 606-A to 606-N can specify the spatial position and spatial orientation of each group of atoms (e.g., moieties, functional groups, side chains, etc.) that may be contained within amino acid 604-Y. The type and spatial structure of amino acid 604-Y can be determined by which groups of atoms are contained within amino acid 604-Y, and by the spatial position and spatial orientation of the contained groups of atoms represented by component embeddings 606-A to 606-N. As shown in the figure, amino acid 604-Y contains groups of atoms 608-A to 608-D, and their spatial positions, spatial orientations, and elemental types are specified by the corresponding component embeddings 606-A to 606-D.

[0071] Figure 7 is a flowchart illustrating an exemplary process for generating molecular structure data of a protein using a denoising neural network. For convenience, the process 700 is described as being performed by a system of one or more computers located in one or more locations. For example, a protein design system, e.g., the protein design system 100 of Figure 1, appropriately programmed according to this specification, can perform the process 700.

[0072] The system can receive molecular structure data containing protein noise (step 702). As described above, the noisy molecular structure data may include sequences of amino acid embeddings. Each amino acid embedding can characterize the structure and composition of the corresponding amino acid within the protein.

[0073] In particular, each amino acid embedding in noisy molecular structure data may include atomic embeddings for each of a given set of atoms. Each atomic embedding may include numerical values ​​that characterize the spatial position of atoms that may be contained within the corresponding amino acid of a protein.

[0074] In some embodiments, the molecular structure data, including noise, may include atomic embeddings representing atoms of the target molecule. The target molecule may be, for example, a target ligand for a protein (e.g., a protein ligand or small molecule ligand). In another example, the target molecule may be a target protein.

[0075] In some embodiments, noisy molecular structure data may have one or more atoms designated as static atoms. When the system denoises the noisy molecular structure data, the system preserves the spatial positions of the static atoms. For example, to generate denoised molecular structure data of a protein that is expected to bind to a target molecule, the noisy molecular structure data may include atomic embeddings of atoms of the target molecule designated as static embeddings. As another example, to generate denoised molecular structure data of a complete protein that includes a specific substructure (e.g., a specific set of amino acid residues of a protein), the noisy molecular structure data may include atomic embeddings of atoms of that specific substructure designated as static embeddings.

[0076] The system can process noisy molecular structure data using a denoising neural network to generate denoised molecular structure data (step 704). The denoising neural network may include any of various processing layers that can generate denoised molecular structure data. For example, the denoising neural network may include feedforward processing layers (e.g., linear layers, nonlinear layers, etc.).

[0077] As another example, a denoising neural network may include one or more self-attention layers, which can generate noisy molecular structure data by applying a self-attention mechanism to the noisy molecular structure data. Each self-attention layer can update the embeddings in the noisy molecular structure data based on attention weights calculated for each pair of embeddings. As an example, a self-attention layer updates the embedding of the i-th atom in the noisy molecular structure data.

number

number

[0078] In the formula, Q, K, and V are the learning matrices of the self-attention layer.

number

number

number

[0079] Attention bias term b i,j This can be calculated based on the distance between the i-th and j-th atoms. For example, attention bias term b i,j This can be calculated based on the spatial distance (e.g., L2 distance) between the i-th and j-th atoms (e.g., represented by the embedding of the i-th and j-th atoms in molecular structure data containing noise). Another example is the attention bias term b. i,j This may be calculated based on the array distance between the i-th atom and the j-th atom (e.g., the difference between the indices for the i-th and j-th atoms). In a further example, attention bias term b i,j This can be calculated based on the spatial distance and alignment distance between the i-th and j-th atoms.

[0080] Attention bias term b i,j For example, d is the value of the distance between the i-th and j-th atoms. i,j It may have the attention bias term b. Another example is the attention bias term b. i,j This can be calculated based on the cutoff threshold τ as follows:

number

[0081] The system can determine whether denoising is complete (step 706). For example, the system may determine that denoising is complete based on the fact that a predetermined number of denoising iterations have been performed. If the system determines that denoising is not complete, the system may continue to the next denoising iteration (for example, returning to step 702). If the system continues to the next denoising iteration, the system may generate noisy molecular structure data for the next denoising iteration. As an example, the system may use the denoised molecular structure data generated during the current denoising iteration as noisy molecular structure data for the next denoising iteration. As another example, the system may generate noisy molecular structure data for the next iteration by combining random noise with the denoised molecular structure data generated during the current denoising iteration (e.g., sampled from a Gaussian distribution, a uniform distribution, etc.).

[0082] Once the system determines that denoising is complete, it can return the denoised molecular structure data (step 708).

[0083] Figure 8 is a flowchart illustrating an exemplary process for training a denoising neural network to generate molecular structure data of a protein. For convenience, process 800 is described as being performed by one or more computer systems located in one or more locations. For example, a protein design system, e.g., the protein design system 100 of Figure 1, appropriately programmed according to this specification, can perform process 800.

[0084] The system can train a denoising neural network over a sequence of training iterations.

[0085] In each training iteration, the system may receive one or more training examples of the training iteration (step 802). Each training example may contain data specifying the target 3D spatial positions of atoms for each protein.

[0086] The system can generate molecular structure data including noise for training iterations (step 804). For each training example, the system can generate a sequence of initial amino acid embeddings for the training example protein that characterize the target 3D spatial positions of atoms within the protein. As described above with reference to Figures 4-7, each amino acid embedding for a protein may contain multiple atomic embeddings representing a predetermined set of atoms that may be contained within the corresponding amino acid of the protein. For each amino acid in the training example protein, the initial amino acid embedding can be initialized to characterize (i) the target 3D spatial positions of atoms contained within the amino acid of the training example protein, and (ii) predetermined disposable positions of the amino acid as the 3D spatial positions of atoms not contained within the amino acid of the training example protein. Examples of disposable amino acid positions are described in detail below with reference to Figure 10.

[0087] The system generates noise-infused molecular structure data for each training example by combining random noise with the initial amino acid embeddings of the training examples. For example, for each initial amino acid embedding, the system can generate a corresponding noise-infused amino acid embedding by adding random noise to this initial amino acid embedding.

[0088] A denoising neural network can be configured to execute a sequence of denoising iterations, for example, as described with reference to steps 702, 704, and 706 in Figure 7. For each training example, the system can determine the denoising iterations of the training example. For example, for each training example, the system can randomly sample denoising iterations of the training example from the sequence of denoising iterations according to a uniform distribution across the sequence of denoising iterations. The system can scale the random noise combined with the initial amino acid embeddings of the training example by a constant that depends on the sampled denoising iterations of the training iteration, for example, the value of the constant corresponding to the denoising iteration is defined by the noise schedule.

[0089] The system can process noisy molecular structure data using a denoising neural network to generate denoised molecular structure data for training iterations (step 806). An exemplary process for generating denoised molecular structure data using a denoising neural network is described in more detail below with reference to Figure 7.

[0090] The system can determine the gradient of the objective function, which depends on the denoised molecular structure data, and use that gradient to update the parameter values ​​of the denoised neural network (step 808). For each training example, the objective function can measure the error between (i) the denoised molecular structure data generated by the denoised neural network and (ii) the target output of the denoised neural network. The target output of the denoised neural network for the training example can define the output of the denoised neural network, and if used to generate initial estimates of the 3D spatial locations of atoms in a protein (as shown in step 406 of Figure 4), the initial estimates of the 3D spatial locations of atoms will match the target 3D spatial locations of atoms in the protein of the training example.

[0091] The system can determine whether training is complete (step 810). If the system determines that training is not complete, it can proceed to the next training iteration (for example, return to step 802). The system can use any of several criteria to determine whether training is complete. For example, the system can determine that training is complete after a predetermined number of training iterations. As another example, the system can determine that training is complete when the value of the objective function for a training iteration falls below a predetermined threshold. As yet another example, the system can determine that training is complete when the difference between the value of the objective function for the current training iteration and the value of the objective function for a previous training iteration falls below a predetermined threshold.

[0092] Once the system determines that training is complete, it can return the trained denoising neural network (step 812).

[0093] Figure 9 is a flowchart illustrating an exemplary process for processing denoised molecular structure data to identify atoms contained within the amino acids of a protein. For convenience, process 900 is described as being performed by one or more computer systems located in one or more locations. For example, a protein design system, e.g., the protein design system 100 of Figure 1, appropriately programmed according to this specification, can perform process 900.

[0094] As described above, denoised molecular structure data may include sequences of amino acid embeddings that specify the final spatial position of each atom within the corresponding amino acid. In particular, each amino acid embedding may include multiple atom embeddings, each specifying the final spatial position of each atom from a predetermined set of atoms that can be contained within the amino acid. Based on the final spatial positions of the atoms within the amino acids of the protein, the denoised molecular structure data can identify which atoms are contained within the amino acid. For each amino acid in the protein, the system can process the corresponding amino acid embeddings from the denoised molecular structure data (for example, following steps 902, 904, and 906 described below) to determine which atoms are contained within the amino acid.

[0095] In some embodiments, the system can determine a disposable spatial position associated with an amino acid (step 902). For example, the disposable spatial position may be a predetermined spatial position or region of the amino acid. Alternatively, the system may determine the disposable spatial position based on denoised molecular structure data. For example, the system may process the atomic embedding of an amino acid and determine the disposable spatial position associated with the amino acid based on the final spatial position of a specified atom of the amino acid. Examples of disposable spatial positions of amino acids are described in detail below with reference to Figure 10.

[0096] When the system determines the disposable spatial positions of amino acids, the system can compare the final spatial position of each atom with the disposable spatial position (step 904). In particular, for each atom, the system can determine whether the atom's final spatial position is within a threshold distance of the disposable spatial position.

[0097] The system can ultimately determine, based on the atom's final spatial position, whether the atom is included in the amino acid sequence within the amino acid (step 906). For example, if the system determines that the final spatial position of a particular atom is within a threshold distance of the disposable spatial position of the amino acid, the system can determine that the particular atom is not included in the amino acid.

[0098] Figure 10 shows an example of a disposable spatial location for an amino acid. As described above, the system can determine the disposable spatial location of an amino acid based on the final location of a designated atom of the amino acid. In general, the atom designated as the disposable location for an amino acid can be an atom present in each of a given set of amino acids. In particular, the atom designated as the disposable location for an amino acid can be a skeleton atom of the amino acid. For example, the designated atom could be the alpha carbon skeleton atom 1002. As another example, the designated atom could be the nitrogen skeleton atom 1004. As yet another example, the designated atom could be the carboxyl group skeleton atom 1006. As yet another example, the designated atom could be the oxygen skeleton atom 1008.

[0099] Figure 11 shows an exemplary generated protein 1102 (darker area in Figure 11) designed to bind to target protein 1104 (brighter area in Figure 11). Protein 1102 is generated by an embodiment of the system described above that denoises molecular structural data specifying the spatial positions of atoms in both protein 1102 and target protein 1104. In particular, protein 1102 is generated by denoising the spatial positions of atoms in protein 1102 while the retention positions of atoms from target protein 1104 are fixed.

[0100] Figure 12 shows an exemplary generated protein 1202 (darker area in Figure 12) designed to contain the target sequence of amino acid 1204 (brighter area in Figure 12). Protein 1202 is generated by an embodiment of the system described above that denoises molecular structural data specifying the spatial positions of atoms in protein 1202, including atoms of the target sequence of amino acid 1204. In particular, protein 1202 is generated by denoising the spatial positions of atoms in protein 1202 while the retention positions of atoms from the target sequence of amino acid 1204 are fixed.

[0101] Some further applications of this system are described below.

[0102] System 100 can be used to obtain polypeptide ligands such as drug or diagnostic antibody markers for diseases, or ligands for industrial enzymes. For example, molecular structure data 106 containing noise may include data defining the structure of the target molecule (e.g., atomic positions), and the system then uses this to generate data such as protein design data 102 that defines the amino acid sequence of the polypeptide ligand.

[0103] In some embodiments, the target molecule comprises a receptor or enzyme, and the polypeptide ligand is an agonist or antagonist of the receptor or enzyme. In some embodiments, the polypeptide ligand comprises an antibody, and the target molecule comprises an antibody target, in particular a viral or cancer cell protein, and the antibody binds to the antibody target to provide a therapeutic effect. In some embodiments, the polypeptide ligand may be designed to bind to a cell surface marker. This may be used to identify and / or treat cancer cells. In some embodiments, the target molecule may be a small molecule ligand, e.g., an organic compound with a molecular weight of less than 900 daltons. In some other embodiments, the target molecule may be the polypeptide ligand itself, i.e., defined by its amino acid sequence.

[0104] Polypeptide ligands can be synthesized, and their biological activity can be tested in vitro and / or in vivo. For example, polypeptide ligands can be tested for ADME (absorption, distribution, metabolism, excretion) and / or toxicological properties, and unsuitable ligands can be screened. Testing may include, for example, contacting the polypeptide ligand with a target molecule and measuring changes in protein expression or activity. In some embodiments, polypeptide ligands and / or target molecules may include isolated antibodies, fragments of isolated antibodies, monovariable domain antibodies, bispecific or multispecific antibodies, multivalent antibodies, bivariable domain antibodies, immune complexes, fibronectin molecules, adnectin, DARPin, avimers, aphibodies, antikalin, affilin, protein epitope mimes, or combinations thereof. Target molecules may include antibodies with mutated or chemically modified amino acid Fc regions, which, for example, prevent or reduce ADCC (antibody-dependent cell-mediated cytotoxicity) activity and / or extend half-life compared to a wild-type Fc region.

[0105] System 100 can also be used to obtain the amino acid sequence of a protein using data characterizing the protein structure obtained by experiments using experimental techniques, including, for example, one or more of X-ray crystallography, nuclear magnetic resonance, and electron microscopy. For example, data characterizing the protein structure can be provided as conditioning data 112, which is then processed by the initialization system 110 to generate molecular structure data 106 containing noise, including the spatial positions of a subset of atoms in the protein. The denoised molecular structure data 108 can then provide a more complete or more accurate structure of the protein, which can then be processed by the structure processing system 114 to obtain data defining the amino acid sequence of the protein.

[0106] This specification uses the term “configured” in relation to systems and computer program components. When one or more computer systems are configured to perform a particular operation or action, it means that, while in operation, software, firmware, hardware, or a combination thereof is installed on the system that causes the system to perform that operation or action. When one or more computer programs are configured to perform a particular operation or action, it means that one or more programs, when executed by a data processing device, contain instructions that cause the device to perform that operation or action.

[0107] The subject matter and functional embodiments described herein may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a tangible non-temporary storage medium, which are executed by or control the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random-access memory device or a serial-access memory device, or one or more combinations thereof. Alternatively or additionally, the program instructions may be encoded in artificially generated propagating signals, such as machine-generated electrical signals, optical signals or electromagnetic signals, which are generated to encode information to be transmitted to a suitable receiving device for execution by a data processing device.

[0108] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, machines, and equipment for processing data, including, for example, programmable processors, computers, or multiple processors or multiple computers. A device may be, or further may be, a special-purpose logic circuit, such as an FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit). Optionally, in addition to hardware, a device may include code that constructs the execution environment for computer programs, such as processor firmware, protocol stacks, database management systems, operating systems, or one or more combinations thereof.

[0109] Computer programs, which may also be called or described as programs, software, software applications, apps, modules, software modules, scripts, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A program may, but may not, correspond to a file in a file system. A program may be stored in a single file dedicated to the program in question, in a part of a file that holds one or more scripts stored in other programs or data, such as a markup language document, or in a series of collaborative files, such as a file that holds one or more modules, subprograms, or parts of code. A computer program may be deployed to run on one computer, or on multiple computers located in one place, or distributed across multiple locations and interconnected by a data communication network.

[0110] In this specification, the term “engine” is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Typically, an engine is implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers are dedicated to a particular engine, while in other cases, multiple engines may be installed and run on the same one or more computers.

[0111] The processes and logic flows described herein can be performed by one or more programmable computers executing one or more computer programs to act on input data and produce outputs, thereby performing their functions. Alternatively, the processes and logic flows can be performed by special-purpose logic circuits, such as FPGAs or ASICs, or by a combination of special-purpose logic circuits and one or more programmed computers.

[0112] A computer suitable for running computer programs may be based on a general-purpose or dedicated microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory, random-access memory, or both. The essential elements of a computer are the central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be complemented by or integrated into dedicated logic circuits. Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is operablely coupled to them to receive data from them, transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be integrated into other devices, such as mobile phones, personal digital assistants (PDAs), mobile audio or video players, game consoles, Global Positioning System (GPS) receivers, or portable storage devices (such as Universal Serial Bus (USB) flash drives) (these are just a few examples).

[0113] Computer-readable media suitable for storing computer program instructions and data include, for example, all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0114] Embodiments of the subject matter described herein may be implemented in a computer having a display device for displaying information to a user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device that allows the user to input into the computer, such as a mouse or trackball, in order to provide user interaction. Other types of devices can also be used to interact with the user. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, voice, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from a device used by the user, for example, by sending a web page to a web browser on the user's device in response to a request received from a web browser. The computer can also interact with the user by sending text messages or other forms of messages to a personal device, such as a smartphone running a messaging application, and then receiving response messages from the user.

[0115] Data processing equipment for implementing machine learning models may include, for example, dedicated hardware accelerator units for processing the general and numerical computation portions (i.e., inference, workloads) of machine learning training or production.

[0116] Machine learning models can be implemented and deployed using machine learning frameworks, such as the TensorFlow framework or the Jax framework.

[0117] Embodiments of the subject matter described herein can be implemented in a computing system including, for example, a backend component as a data server, or a computing system including a middleware component, for example, an application server, or a frontend component, for example, a client computer having a graphical user interface, a web browser, or an application that allows a user to interact with embodiments of the subject matter described herein, or in a computing system including any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by digital data communications of any form or medium, for example, a communication network. Examples of communication networks include local area networks (LANs), wide area networks (WANs), for example, the Internet.

[0118] A computing system can include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship arises from computer programs that run on each computer and have a client-server relationship with each other. In some embodiments, the server sends data, such as an HTML page, to a user device for the purpose of displaying data to a user interacting with a device acting as a client and receiving user input from that user. Data generated on the user device, such as the results of user interactions, can be received from the device by the server.

[0119] While this specification includes details of many specific embodiments, these should not be construed as limiting the scope of any invention or claimable content, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described herein in the context of separate embodiments may also be realized in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be realized individually or in any suitable subcombination in multiple embodiments. Furthermore, features may be described above as functioning in a particular combination, and even if initially claimed as such, one or more features from the claimed combination may be removed from the combination, and the claimed combination may cover a subcombination or a variation of a subcombination.

[0120] Similarly, while operations are shown in the drawings and described in a specific order in the claims, this should not be understood as requiring that such operations be performed in a specific or sequential order shown, or that all shown operations be performed, in order to obtain the desired results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described can generally be integrated into a single software product or packaged into multiple software products.

[0121] In addition to the embodiments described above, the following embodiments are also innovative.

[0122] Embodiment 1 is a method for designing a protein performed by one or more computers, comprising: generating noise-containing molecular structure data for each position in the amino acid sequence of the protein, defining the corresponding initial spatial position of each atom within a predetermined set of possible atoms, wherein at least some of the initial spatial positions in the noise-containing molecular structure data are sampled from a noise distribution; and processing the noise-containing molecular structure data using a diffusion model including a denoising neural network to generate denoised molecular structure data that defines a denoised version of the noise-containing molecular structure data.

[0123] Embodiment 2 is the method of Embodiment 1, further comprising processing the denoised molecular structure data to determine the type of each amino acid at each of a plurality of positions in the amino acid sequence of the protein.

[0124] Embodiment 3 is the method of Embodiment 1 or Embodiment 2, wherein the predetermined set of possible atoms includes each atom present in each amino acid within the predetermined set of possible amino acids.

[0125] Embodiment 4 is the method according to Embodiment 3, wherein the predetermined set of possible atoms includes at least 30 atoms.

[0126] Embodiment 5 is the method according to Embodiment 4, wherein the predetermined set of possible atoms includes at least 37 atoms.

[0127] Embodiment 6 is the method according to any one of Embodiments 3 to 5, wherein the predetermined set of possible amino acids includes at least 20 amino acids.

[0128] Embodiment 7 is a method according to any one of Embodiments 3 to 6, comprising the operation of the method according to Embodiment 2 for determining the respective type of the amino acid at each of the plurality of positions in the amino acid sequence of the protein, wherein processing the denoised molecular structure data includes, for each of the plurality of positions in the amino acid sequence of the protein, identifying, based on the denoised molecular structure data, that a suitable subset of a predetermined set of possible atoms is contained in the amino acid at the position, and for each of the plurality of positions in the amino acid sequence of the protein, determining the type of the amino acid at the position based on the suitable subset of the set of possible atoms contained in the amino acid at the position.

[0129] Embodiment 8 is the method of Embodiment 7, wherein for each of the plurality of positions in the amino acid sequence of the protein, determining the type of the amino acid at the position based on a suitable subset of the set of possible atoms contained in the amino acid at the position includes mapping the suitable subset of the set of possible atoms contained in the amino acid to the corresponding amino acid type according to a predetermined mapping at the position.

[0130] Embodiment 9 is the method of Embodiment 7 or Embodiment 8, wherein the denoised molecular structure data defines the final spatial position of each atom in the predetermined set of atoms for each position in the amino acid sequence of the protein, and for each of the plurality of positions in the amino acid sequence of the protein, based on the denoised molecular structure data, identifies that a suitable subset of the predetermined set of possible atoms is contained in the amino acid at the position in the amino acid sequence, which includes determining, for each of the plurality of atoms in the predetermined set of atoms, whether the atom is contained in the amino acid at the position in the amino acid sequence, based on the final spatial position of the atom.

[0131] Embodiment 10 is the method of Embodiment 9, wherein for each of the plurality of atoms of the predetermined set of atoms, determining whether the atom is included in the amino acid at the position in the amino acid sequence based on the final spatial position of the atom, determines a disposable spatial position associated with the amino acid at the position in the amino acid sequence based on the denoised molecular structure data, and for each of the plurality of atoms of the predetermined set of atoms, determining whether the atom is included in the amino acid at the position based on a comparison of (i) the final spatial position of the atom and (ii) the disposable spatial position associated with the amino acid at the position in the amino acid sequence.

[0132] Embodiment 11 is the method of Embodiment 10, wherein determining the disposable spatial position associated with the amino acid at the position in the amino acid sequence based on the denoised molecular structure data is performed by determining the disposable spatial position associated with the amino acid based on the final spatial position of an atom specified from a predetermined set of possible atoms.

[0133] Embodiment 12 is the method of Embodiment 11, wherein the designated atom from the predetermined set of possible atoms is present in each amino acid within the predetermined set of possible amino acids.

[0134] Embodiment 13 is the method of Embodiment 12, wherein the designated atom from the predetermined set of possible atoms is a skeletal atom.

[0135] Embodiment 14 is the method according to Embodiment 13, wherein the designated atom from the predetermined set of possible atoms is an alpha carbon skeleton atom.

[0136] Embodiment 15 is the method of Embodiment 13, wherein the designated atom from the predetermined set of possible atoms is a nitrogen skeleton atom.

[0137] Embodiment 16 is the method of Embodiment 13, wherein the designated atom from the predetermined set of possible atoms is an oxygen skeleton atom.

[0138] Embodiment 17 is a method according to any one of Embodiments 10 to 16, wherein, for one or more atoms of the predetermined set of atoms, determining whether the atom is included in the amino acid at the position is, based on a comparison of (i) the final spatial position of the atom and (ii) the disposable spatial position associated with the amino acid at the position in the amino acid sequence, by determining that the final spatial position of the atom is within a threshold distance of the disposable spatial position associated with the amino acid at the position in the amino acid sequence, and in response, determining that the atom is not included in the amino acid at the position in the amino acid sequence.

[0139] Embodiment 18 is a method according to any one of Embodiments 1 to 17, wherein the molecular structure data including the noise designates one or more atoms in the protein as static atoms having predetermined spatial positions that are not corrected by the diffusion model.

[0140] Embodiment 19 is a method according to any one of Embodiments 1 to 18, wherein the molecular structure data including noise includes data that defines the structure of a target molecule, and the protein is predicted to bind to the target molecule.

[0141] Embodiment 20 is the method of Embodiment 19, wherein the target molecule is not a protein.

[0142] Embodiment 21 is the method according to Embodiment 19, wherein the target molecule is a protein.

[0143] Embodiment 22 is a method according to any one of Embodiments 19 to 21, wherein the molecular structure data including noise defines the initial spatial position corresponding to each atom in the target molecule.

[0144] Embodiment 23 is a method according to any one of Embodiments 19 to 22, wherein the molecular structure data including the noise designates one or more atoms in the target molecule as stationary atoms having predetermined spatial positions that are not modified by the diffusion model.

[0145] Embodiment 24 is the method according to any one of Embodiments 19 to 23, wherein the molecular structure data including the noise includes a set of embeddings, the set of embeddings includes a respective embedding corresponding to each position in the amino acid sequence of the protein, and the set of embeddings includes a respective embedding corresponding to each atom in the target molecule.

[0146] Embodiment 25 is the method of Embodiment 24, wherein the embedding corresponding to the position of the protein within the amino acid sequence has the same number of dimensions as the embedding corresponding to the atom within the target molecule.

[0147] Embodiment 26 is a method according to any one of Embodiments 1 to 25, wherein processing the noisy molecular structure data using the diffusion model to generate the denoised molecular structure data includes iteratively denoising the noisy molecular structure data over a sequence of denoising iterations using the denoising neural network.

[0148] Embodiment 27 is the method of Embodiment 26, wherein the iterative denoising of the noise-containing molecular structure data over a sequence of denoising iterations includes, in each denoising iteration, receiving the noise-containing molecular structure data for the denoising iteration, and processing the noise-containing molecular structure data for the denoising iteration using the denoising neural network to generate denoised molecular structure data for the denoising iteration.

[0149] Embodiment 28 is the method of Embodiment 27, further comprising, in each denoising iteration preceding the last denoising iteration in the sequence of denoising iterations, generating noise-containing molecular structure data for the next denoising iteration based on the denoised molecular structure data for the denoising iteration, and providing the noise-containing molecular structure data for the next denoising iteration.

[0150] Embodiment 29 is the method of Embodiment 28, wherein, in each denoising iteration prior to the last denoising iteration in the sequence of denoising iterations, generating molecular structure data including the noise for the next denoising iteration based on the denoised molecular structure data for the denoising iteration is performed by combining random noise with the denoised molecular structure data for the denoising iteration.

[0151] Embodiment 30 is a method according to any one of Embodiments 26 to 29, wherein in each denoising iteration, the molecular structure data containing the noise for the denoising iteration includes a set of embeddings, each embedding corresponding to a position in the amino acid sequence of the protein or an atom in the target molecule, and processing the molecular structure data containing the noise for the denoising iteration using the denoising neural network to generate the denoised molecular structure data for the denoising iteration includes processing the set of embeddings by one or more self-attention neural network layers of the denoising neural network.

[0152] Embodiment 31 is the method of Embodiment 30, wherein processing the set of embeddings by one or more self-attention neural networks in the denoising neural network layer includes, for each self-attention neural network layer, processing the set of embeddings to generate respective attention weights for each pair of embeddings from the set of embeddings, generating respective attention biases for each pair of embeddings from the set of embeddings, generating final attention weights for each pair of embeddings from the set of embeddings based on the attention weights and the attention biases, and updating the set of embeddings using the final attention weights.

[0153] Embodiment 32 is the method of Embodiment 31, wherein for each pair of embeddings from the set of embeddings, the attention bias for the pair of embeddings is based on the spatial distance between the pair of entities represented by the pair of embeddings, or the array distance between the pair of entities represented by the pair of embeddings, or both.

[0154] Embodiment 33 is a method according to any one of Embodiments 3 to 32, wherein processing the denoised molecular structure data to determine the respective type of amino acid at each of the plurality of positions in the amino acid sequence of the protein includes processing at least a portion of the denoised molecular structure using a sequencing machine learning model to generate data defining the respective type of amino acid at each of the plurality of positions in the amino acid sequence of the protein.

[0155] Embodiment 34 is a system comprising one or more computers and one or more storage devices communicably coupled to the one or more computers, wherein the one or more storage devices, when executed by the one or more computers, store instructions causing the one or more computers to perform each of the operations according to any one of Embodiments 1 to 33.

[0156] Embodiment 35 is a non-temporary computer storage medium which, when executed by one or more computers, stores instructions that cause the one or more computers to perform the operation of each of the methods described in any one of Embodiments 1 to 33.

[0157] Embodiment 36 is a method for producing a protein, comprising generating data defining the amino acid sequence of a protein using the method described in any one of Embodiments 1 to 33, and synthesizing a protein having the amino acid sequence.

[0158] Embodiment 37 is a method for obtaining a polypeptide ligand, wherein the polypeptide ligand is a ligand for a disease drug or diagnostic antibody marker or an industrial enzyme, and the method comprises generating data defining the amino acid sequence of the polypeptide ligand using the method of any one of Embodiments 1 to 33, wherein the noise-inclusive molecular structure data comprises data defining the structure of a target molecule to which the polypeptide ligand should bind.

[0159] Embodiment 38 is the method of Embodiment 37, wherein the target molecule comprises a receptor or enzyme, the polypeptide ligand is an agonist or antagonist of the receptor or enzyme, or the polypeptide ligand comprises an antibody, the target molecule comprises an antibody target, particularly a viral or cancer cell protein, and the antibody binds to the antibody target to provide a therapeutic effect.

[0160] Embodiment 39 is the method of Embodiment 37 or 38, further comprising testing the biological activity of the polypeptide ligand in vitro or in vivo.

[0161] Embodiment 40 is a method for obtaining the amino acid sequence of a protein, comprising receiving data characterizing the structure of the protein, the data being obtained experimentally, and performing the method according to any one of Embodiments 1 to 33 to determine the predicted amino acid sequence of the protein, wherein the molecular structure data, including noise, includes the data characterizing the structure of the protein.

[0162] Embodiment 41 is the method of Embodiment 40, wherein the data characterizing the structure of the protein includes the spatial positions of a subset of atoms within the protein.

[0163] Embodiment 42 is the method according to Embodiment 40 or 41, wherein the experimental technique includes one or more of X-ray crystallography, nuclear magnetic resonance, and electron microscopy.

[0164] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions described in the claims may be performed in a different order, and this may still yield the desired results. As an example, the process shown in the accompanying diagram does not necessarily require to be performed in the specific order or sequence shown to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. [Explanation of Symbols]

[0165] 100 Protein Design Systems 102 Protein design data 104 Diffusion Model 106, 108 Molecular structure data 110 Initialization System 112 Conditioning Data 114 Structural Processing System 302 Denoising Neural Network 400 arrays 404-Y amino acid 408-A~408-J atoms 500 arrays 504-Y amino acids 508-A to 508-E atoms 600 array 604-Y amino acid 608-A~608-D atoms 1002 Alpha carbon skeleton atoms 1004 Nitrogen skeleton atoms 1006 Carboxylate backbone atoms 1008 Oxygen skeleton atoms 1102 Protein 1104 Target Protein 1202 Protein 1204 amino acids

Claims

1. A method performed by one or more computers to design a protein, The process involves generating molecular structure data that includes noise defining the initial spatial position corresponding to each atom within a predetermined set of possible atoms for each position in the amino acid sequence of the protein, At least some of the initial spatial positions in the molecular structure data containing the noise are sampled from the noise distribution and generated. A method comprising processing the noisy molecular structure data using a diffusion model with a denoising neural network to generate denoised molecular structure data that defines a denoised version of the noisy molecular structure data.

2. The method according to claim 1, further comprising processing the denoised molecular structure data to determine the type of amino acid at each of a plurality of positions in the amino acid sequence of the protein.

3. The method according to claim 1 or 2, wherein the predetermined set of possible atoms includes each atom present in each amino acid within the predetermined set of possible amino acids.

4. The method according to claim 3, wherein the predetermined set of possible atoms includes at least 30 atoms.

5. The method according to claim 4, wherein the predetermined set of possible atoms includes 37 atoms.

6. The method according to any one of claims 3 to 5, wherein the predetermined set of possible amino acids comprises at least 20 amino acids.

7. Processing the denoised molecular structure data in order to determine the type of each of the amino acids at each of the plurality of positions in the amino acid sequence of the protein is For each of the plurality of positions in the amino acid sequence of the protein, based on the denoised molecular structure data, an appropriate subset of the predetermined possible set of atoms is identified as being contained in the amino acid at that position. For each of the plurality of positions in the amino acid sequence of the protein, the type of amino acid at that position is determined based on the appropriate subset of the possible sets of atoms contained in the amino acid at that position. A method according to any one of claims 3 to 6 dependent on claim 2, comprising:

8. For each of the plurality of positions in the amino acid sequence of the protein, determining the type of amino acid at the position based on the appropriate subset of the possible sets of atoms contained in the amino acid at that position is: The method according to claim 7, comprising mapping a suitable subset of the set of possible atoms contained in the amino acid to a corresponding type of amino acid according to a predetermined mapping at the position.

9. The noise-reduced molecular structure data defines the final spatial position of each atom within the predetermined set of atoms for each position in the amino acid sequence of the protein, For each of the plurality of positions in the amino acid sequence of the protein, to identify, based on the denoised molecular structure data, that an appropriate subset of the predetermined set of possible atoms is contained in the amino acid at that position in the amino acid sequence is: The method according to claim 7 or 8, further comprising determining, for each of a plurality of atoms in the predetermined set of atoms, whether the atom is included in the amino acid at the position in the amino acid sequence, based on the final spatial position of the atom.

10. For each of the plurality of atoms in the predetermined set of atoms, determining whether the atom is included in the amino acid at the position in the amino acid sequence, based on the final spatial position of the atom, is: Based on the noise-reduced molecular structure data, the disposable spatial position associated with the amino acid at the position in the amino acid sequence is determined, For each of the plurality of atoms in the predetermined set of atoms, (i) the final spatial position of the atom and (ii) the disposable spatial position associated with the amino acid at that position in the amino acid sequence are used to determine whether the atom is included in the amino acid at that position. The method according to claim 9, comprising:

11. Determining the disposable spatial position associated with the amino acid at the position within the amino acid sequence based on the denoised molecular structure data is: The method according to claim 10, comprising determining the disposable spatial position associated with the amino acid based on the final spatial position of an atom specified from a predetermined set of possible atoms.

12. The method according to claim 11, wherein the designated atom from the predetermined set of possible atoms is present in each amino acid within the predetermined set of possible amino acids.

13. The method according to claim 12, wherein the designated atom from the predetermined set of possible atoms is a skeletal atom.

14. The method according to claim 13, wherein the designated atom from the predetermined set of possible atoms is an alpha carbon skeleton atom.

15. The method according to claim 13, wherein the designated atom from the predetermined set of possible atoms is a nitrogen skeleton atom.

16. The method according to claim 13, wherein the designated atom from the predetermined set of possible atoms is an oxygen skeleton atom.

17. For one or more atoms in the predetermined set of atoms, determining whether an atom is included in the amino acid at a given position based on a comparison between (i) the final spatial position of the atom and (ii) the disposable spatial position associated with the amino acid at that position in the amino acid sequence is: The determination that the final spatial position of the atom is within the threshold distance of the disposable spatial position associated with the amino acid at the position in the amino acid sequence, In response, it is determined that the atom is not included in the amino acid at the position in the amino acid sequence, The method according to any one of claims 10 to 16, comprising:

18. The method according to any one of claims 1 to 17, wherein the molecular structure data containing the noise designates one or more atoms in the protein as static atoms having predetermined spatial positions that are not modified by the diffusion model.

19. The aforementioned molecular structure data, including noise, comprises data that defines the structure of the target molecule. The method according to any one of claims 1 to 18, wherein the protein is predicted to bind to the target molecule.

20. The method according to claim 19, wherein the target molecule is not a protein.

21. The method according to claim 19, wherein the target molecule is a protein.

22. The method according to any one of claims 19 to 21, wherein the molecular structure data including the noise defines the initial spatial position corresponding to each atom in the target molecule.

23. The method according to any one of claims 19 to 22, wherein the molecular structure data containing the noise designates one or more atoms in the target molecule as stationary atoms having predetermined spatial positions that are not modified by the diffusion model.

24. The aforementioned molecular structure data, including noise, comprises a set of embeddings. The set of embeddings comprises each embedding corresponding to each position in the amino acid sequence of the protein, The method according to any one of claims 19 to 23, wherein the set of embeddings comprises a set of embeddings corresponding to each atom in the target molecule.

25. The method according to claim 24, wherein the embeddings corresponding to the positions of the protein within the amino acid sequence have the same number of dimensions as the embeddings corresponding to atoms in the target molecule.

26. The method according to any one of claims 1 to 25, wherein processing the noisy molecular structure data using the diffusion model to generate the denoised molecular structure data comprises iteratively denoising the noisy molecular structure data over a sequence of denoising iterations using the denoising neural network.

27. The process of iteratively denoising the molecular structure data containing the noise over the sequence of denoising iterations is performed in each denoising iteration. Receiving molecular structure data containing noise for the aforementioned noise reduction iteration, To generate denoised molecular structure data for the denoising iteration, the denoising neural network is used to process the molecular structure data containing the noise for the denoising iteration. The method according to claim 26, comprising:

28. In each denoising iteration preceding the last denoising iteration in the sequence of denoising iterations, Based on the denoised molecular structure data for the denoising iteration, generate noise-containing molecular structure data for the next denoising iteration. To provide molecular structure data including the noise for the subsequent noise reduction iteration, The method according to claim 27, further comprising:

29. In each denoising iteration preceding the last denoising iteration in the sequence of denoising iterations, generating the noise-containing molecular structure data for the next denoising iteration based on the denoised molecular structure data for the denoising iteration is: The method according to claim 28, comprising combining random noise with the denoised molecular structure data for the denoising iteration.

30. In each noise reduction iteration, The molecular structure data containing the noise for the noise-reducing iteration comprises a set of embeddings, each embedding corresponding to a position in the amino acid sequence of the protein or an atom in the target molecule, and processing the molecular structure data containing the noise for the noise-reducing iteration using the noise-reducing neural network to generate the noise-reduced molecular structure data for the noise-reducing iteration is: The method according to any one of claims 26 to 29, further comprising processing the set of embeddings by one or more self-attention neural network layers of the noise-reducing neural network.

31. Processing the set of embeddings by one or more self-aware neural network layers of the denoising neural network is done for each self-aware neural network layer, For each pair of embeddings from the aforementioned set of embeddings, the set of embeddings is processed in order to generate their respective attention weights. For each pair of embeddings from the aforementioned set of embeddings, generate a corresponding attentional bias, Based on the aforementioned attention weights and attention biases, a final attention weight is generated for each pair of embeddings from the set of embeddings. Updating the set of embeddings using the final attention weights, The method according to claim 30, comprising:

32. The method according to claim 31, wherein for each pair of embeddings from the set of embeddings, the attention bias for the pair of embeddings is based on the spatial distance between the pair of entities represented by the pair of embeddings, or the array distance between the pair of entities represented by the pair of embeddings, or both.

33. Processing the denoised molecular structure data in order to determine the type of each of the multiple positions in the amino acid sequence of the protein is: The method according to any one of claims 3 to 32, dependent on claim 2, further comprising processing at least a portion of the denoised molecular structure using a sequencing machine learning model to generate data defining the type of each of the amino acids at each of the plurality of positions in the amino acid sequence of the protein.

34. It is a system, One or more computers, A system comprising one or more storage devices communicably coupled to one or more computers, wherein the one or more storage devices, when executed by the one or more computers, store instructions causing the one or more computers to perform each of the operations according to any one of claims 1 to 33.

35. One or more non-temporary computer storage media, which, when executed by one or more computers, store instructions causing the one or more computers to perform each of the operations according to any one of claims 1 to 33.

36. A method for producing protein, Using the method according to any one of claims 1 to 33, generate data that defines the amino acid sequence of a protein, A method comprising synthesizing a protein having the aforementioned amino acid sequence.

37. A method for obtaining a polypeptide ligand, wherein the polypeptide ligand is a ligand for a disease drug or diagnostic antibody marker or an industrial enzyme, and the method is: A method comprising generating data defining the amino acid sequence of the polypeptide ligand using the method according to any one of claims 1 to 33, wherein the molecular structure data including noise comprises data defining the structure of a target molecule to which the polypeptide ligand should bind.

38. The method according to claim 37, wherein the target molecule comprises a receptor or enzyme, the polypeptide ligand is an agonist or antagonist of the receptor or enzyme, or the polypeptide ligand comprises an antibody, the target molecule comprises an antibody target, particularly a viral or cancer cell protein, and the antibody binds to the antibody target to provide a therapeutic effect.

39. The method according to claim 37 or 38, further comprising testing the biological activity of the polypeptide ligand in vitro or in vivo.

40. A method for obtaining the aforementioned amino acid sequence of a protein, Receiving data characterizing the structure of the protein, wherein the data is obtained experimentally, and receiving A method comprising: performing the method according to any one of claims 1 to 33 to determine the predicted amino acid sequence of the protein, wherein the molecular structure data, including noise, comprises the data characterizing the structure of the protein.

41. The method according to claim 40, wherein the data characterizing the structure of the protein comprises the spatial positions of a subset of atoms within the protein.

42. The method according to claim 40 or 41, wherein the experimental technique comprises one or more of X-ray crystallography, nuclear magnetic resonance, and electron microscopy.

Images