Use of generative artificial intelligence for protein engineering

Abstract

The present disclosure provides systems and methods for engineering proteins. In one aspect, described herein is using a Transform neural network and a diffusion model to predict an active site of an enzyme and then generate a predicted structure of the enzyme having the predicted active site.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Publication Number (43) Publication Date (21) Application Number 202480057270.8 (22) Application Date 2024.06.20 (30) Priority Data 63 / 526,655 2023.07.13 US (85) PCT International Application Entering National Phase Date 2026.03.06 (86) PCT International Application Application Data PCT / US2024 / 034852 2024.06.20 (87) PCT International Application Publication Data WO2025 / 014634 EN 2025.01.16 (71) Applicant: EtherBioMachines, Inc. Address: California, USA (72) Inventors: Mirudula Mukundan, Eric John Ropreet, Stelianos Kiriacu (74) Patent Agency: Beijing Anxin Fangda Intellectual Property Agency Co., Ltd. 11262 Patent Attorney: Li Jianwu, Jingjing (51) Int.Cl. G01N 33 / 68 (2006.01) G06N 3 / 02 (2006.01) G16B 15 / 20 (2006.01) G16B 15 / 30 (2006.01) G16C 20 / 10 (2006.01) G16C 20 / 70 (2006.01) G06N 20 / 00 (2006.01) (54) Invention Title: Use of Generative Artificial Intelligence for Protein Engineering (57) Abstract: This disclosure provides systems and methods for engineering proteins. In one aspect, this document describes the use of a Transformer neural network and a diffusion model to predict the active site of an enzyme and then generate a predicted structure of the enzyme having the predicted active site. Claims 1 page, Description 14 pages, Drawings 6 pages, CN 121794577 A 2026.04.03 CN 1 21 79 45 77 A 1. A computer-implemented method comprising: receiving input data characterizing an enzyme-catalyzed reaction; generating a predicted location of an active site of an enzyme based on the input data using a Transformer neural network; generating a predicted structure of the enzyme using at least a diffusion model and based on the predicted location of the active site of the enzyme, the predicted structure of the enzyme defining a three-dimensional configuration of atoms included in the enzyme. 2. The method of claim 1, wherein the input data comprises data characterizing a substrate bound to the enzyme. 3. The method of any one of claims 1-2, wherein the Transformer neural network is configured as a Graph Transformer neural network operating on graph data defining the enzyme-catalyzed reaction.4. The method of any one of claims 1-2, wherein the Transformer neural network is configured to autoregress and generate the predicted positions of the active sites of the enzyme from a large language model neural network. 5. The method of any one of claims 1-4, wherein generating the predicted structure of the enzyme comprises: generating predicted three-dimensional positions of the atoms included in the enzyme using the diffusion model; generating an amino acid sequence based on the predicted three-dimensional positions of the atoms included in the enzyme using a sequence prediction neural network; and generating the predicted structure of the enzyme based on the amino acid sequence using a folding neural network or a protein structure prediction model. 6. The method of any one of claims 1-5, further comprising performing one or more actions on the enzyme having the predicted structure. 7. The method of claim 6, wherein the one or more actions comprise: determining whether the enzyme having the predicted structure is effective based on the three-dimensional configuration of the atoms defined by the predicted structure. 8. The method of claim 6, wherein the one or more actions comprise: performing laboratory synthesis and validation of the enzyme having the predicted structure. 9. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of a corresponding method as described in any one of claims 1-8. 10. One or more non-transitory computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations of a corresponding method as described in any one of claims 1-8. 11. An AI-driven enzyme design platform that combines a large language model and a diffusion model to design novel enzymes. 12. The AI-driven enzyme design platform of claim 11, wherein the large language model is trained on scientific literature, protein databases, and experimental datasets related to enzyme biology, biochemistry, and structural biology. 13. The AI-driven enzyme design platform of claim 11 or claim 12, wherein the large language model is used to predict potential active sites for a given enzyme target. 14. The AI-driven enzyme design platform according to any one of claims 11-13, wherein the predicted active sites from the large language model are used to prompt the diffusion model, which designs novel enzymes integrating these active sites. Claims 1 / 1 page 2 CN 121794577 A Use of Generative Artificial Intelligence for Protein Engineering Technical Field

[0001] This specification generally relates to the fields of biotechnology and artificial intelligence. More particularly, the present invention relates to the development of a computational platform for designing novel enzymes using machine learning models.Enzyme design is a key aspect of biotechnology with numerous applications, including biocatalysis, drug discovery, and bioenergy.

[0002] Machine learning models generally predict outputs based on given inputs. Some machine learning models employ a single-layer architecture and rely on statistical analysis to classify or regress a specific dataset. These include models such as linear regression, decision trees, etc. Other machine learning models have more complex architectures, such as multi-layer architectures. These include large language models (LLM) and diffusion models. Background Art

[0003] Proteins can be used to catalyze chemical reactions and as precise binding agents (e.g., for diagnosis, treatment, detection, or isolation). Enzyme catalysis can replace traditional chemical synthesis, which typically relies on the use of petrochemical-based solvents, hazardous reagents, and metal-based chemical catalysts, to achieve chemical transformations. In contrast, enzymes are harmless and non-toxic, biodegradable, operate in aqueous and non-aqueous solvents, and are fully renewable because they are produced safely and inexpensively through biological methods. Additionally, enzymes can be designed (e.g., as antibodies) to bind to targets with high selectivity, inducing the binding of proteins to desired ligands or epitopes while avoiding binding to other closely related ligands or epitopes. This can lead to improved detection accuracy, selective enrichment of therapies or products with reduced side effects. In principle, this allows for the synthesis of products that are difficult to achieve through conventional chemical synthesis, as well as the use of alternative raw materials, thereby significantly reducing infrastructure and operating costs.

[0004] Although previous advances in the engineering of protein activity compared to wild-type (i.e., variants of proteins found in nature) have resulted in many improvements to proteins, these have typically required several months to validate, achieving only modest improvements and with unpredictable results. Furthermore, previous methods generally required an initial protein with at least some activity on the desired substrate and were therefore limited to improving proteins rather than engineering novel activities. There is a need for fundamentally new systems and methods for high-throughput protein design. Summary of the Invention

[0005] The systems and methods described in this specification meet the need for new and improved systems and methods for engineered proteins. This disclosure relates to an AI-driven enzyme design platform that utilizes the comprehensive predictive power of a Transformer neural network and the conditional generation capabilities of a diffusion model to create enzymes that can promote desired reactions.

[0006] The system employs a Transformer neural network to generate conditional inputs, which serve as guidance for a generation process performed using a diffusion model. This process involves querying the Transformer neural network to predict the conformation of the active site binding to a specific ligand. Subsequently, the diffusion model uses this predicted active site conformation to design the overall structure of the enzyme around the active site, effectively integrating the proposed active site into its structure. The symbiotic functionality of these two models enables accurate and efficient enzyme design.

[0007] According to one aspect, a computer-implemented method is provided, comprising: receiving input data characterizing an enzyme-catalyzed reaction; generating a predicted position of an active site of an enzyme based on the input data using a Transformer neural network; and generating a predicted structure of an enzyme based on the predicted positions of the active sites of the enzyme using at least a diffusion model, the predicted structure of the enzyme defining a three-dimensional configuration of atoms included in the enzyme specification 1 / 14 page 3 CN 121794577 A.

[0008] The input data may include data characterizing a substrate bound to the enzyme. The Transformer neural network may be configured as a Graph Transformer neural network operating on graph data defining the enzyme-catalyzed reaction. The Transformer neural network may be configured as a large language model neural network for autoregression to generate the predicted positions of the active sites of the enzyme. Generating the predicted structure of the enzyme may include: generating the predicted three-dimensional positions of atoms included in the enzyme using a diffusion model; generating an amino acid sequence based on the predicted three-dimensional positions of atoms included in the enzyme using a sequence prediction neural network; and generating the predicted structure of the enzyme based on the amino acid sequence using a folding neural network or a protein structure prediction model. The method may also include performing one or more actions on the enzyme having the predicted structure. One or more actions may include: determining whether an enzyme having the predicted structure is effective based on the three-dimensional configuration of atoms defined by the predicted structure. One or more actions may include: performing laboratory synthesis and validation of an enzyme having the predicted structure.

[0009] According to another aspect, one or more computer-readable storage media are provided, which are encoded with instructions that, when executed by one or more computers, cause one or more computers to perform operations related to the methods described above.

[0010] According to another aspect, a system is provided comprising one or more computers and one or more storage devices storing instructions that, when executed by one or more computers, cause one or more computers to perform corresponding operations related to the methods described above.

[0011] Specific embodiments of the subject matter described herein may be implemented to achieve one or more of the following advantages. The enzyme design platform described herein can predict the structure of an enzyme based on a target reaction that the enzyme can assist in. An enzyme having a predicted structure can bind to a substrate to accelerate the reaction, allow it to occur at a lower energy level, or even assist in the detection of the substrate (e.g., detection of harmful chemicals). More specifically, enzyme design platforms can do this by leveraging: (i) the knowledge learned by a Graph Transformer neural network from scientific literature, protein databases, experimental datasets, and other training data through its large-scale pre-training to predict potential active sites and (ii) the ability to generate diffusion models conditioned on the predicted potential active sites.By understanding the intricate complexities involved in atomic-level interactions at catalytic binding sites and their overall impact on protein structure, enzyme design platforms can design novel enzymes more efficiently and accurately compared to traditional enzyme design methods that attempt to thoroughly search through a very large enzyme design space.

[0012] Existing enzyme design systems involve time-consuming and expensive trial-and-error experimental processes and can require significant time and computational costs, sometimes hundreds or thousands of human expert hours to design an enzyme for a single reaction. In contrast, the enzyme design platforms described in this specification can rapidly predict the structure of novel enzymes with high accuracy. For example, in some embodiments, generating a prediction of the enzyme structure can take no more than a few minutes or even a few seconds. Furthermore, enzyme design platforms are able to integrate the chemical rules behind the reaction with the required amino acid combinations in the protein to perform the intended function. This reduces the time required to search the vast space of proteins with different amino acid combinations. Therefore, the use of enzyme design platforms can significantly improve the efficiency of the process of designing novel enzymes, allowing human experts to test more enzyme designs for a wider range of applications, including biocatalysis, drug discovery, and bioenergy, and to iterate the design-build-test cycle more quickly.

[0013] Details of one or more embodiments of the subject matter of this specification are shown in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, figures, and claims. Brief Description of the Drawings

[0014] FIG1 is a schematic diagram of an example enzyme design platform.

[0015] FIG2 is a flowchart of an example process for generating a predicted structure of an enzyme.

[0016] FIG3 is an example illustration of perfluoroalkyl and polyfluoroalkyl substances (PFAS). Specification 2 / 14 pages 4 CN 121794577 A

[0017] FIG4 is an example illustration of partial output data that can be obtained using a Graph Transformer neural network.

[0018] FIG5 is an example illustration of an atomic-level representation of an enzyme.

[0019] FIG6 is an example illustration of an amino acid sequence in FASTA format.

[0020] Similar reference numerals and names in the various figures denote similar elements. Detailed Description

[0021] FIG1 is a schematic diagram of an example enzyme design platform 100. The enzyme design platform 100 is an example of a platform implemented as a computer program on one or more computers at one or more locations, wherein the systems, components, and techniques described below can be implemented.

[0022] The enzyme design platform 100 obtains input data 102 characterizing one or more desired properties of the enzyme, and uses the input data 102 to generate output data defining a predicted structure 132 of the enzyme having one or more desired properties characterized by the input data 102.

[0023] The input data 102 can be obtained in any of a variety of ways.For example, enzyme design platform 100 may receive input data as user input from a user of the platform via a data communication network (e.g., using an application programming interface (API) or graphical user interface (GUI) provided by the system) or a web browser through which the user can interact with the platform. As another example, enzyme design platform 100 may receive user input from a user specifying which data, maintained by the platform or by another computer system accessible to the platform, should be used as input data to characterize one or more desired properties of the enzyme.

[0024] Input data 102 may characterize the desired properties of the enzyme in a variety of ways. In some embodiments, input data 102 may characterize a target (chemical) reaction that the enzyme can assist in. Such a target reaction may involve an enzyme, a substrate, and one or more products formed after the enzyme binds to the substrate. The target reaction may be part of a pathway or process of interest. As part of the target reaction, the enzyme acts on the substrate and lowers the activation energy necessary for the chemical reaction to occur by stabilizing the transition state. This stabilization accelerates the reaction rate and causes it to occur at a physiologically significant rate. In order to act on the substrate, the enzyme binds the substrate at a key location in the enzyme structure called the active site, for example, using a lock-and-key mechanism. Once the substrate is located at the active site of the enzyme, the target reaction can occur.

[0025] Therefore, input data 102 can characterize aspects of the substrate involved in the target reaction. For example, input data 102 can define the type of substrate, for example, it can identify which of various known types of substrates should be used as the substrate to participate in the target reaction. As another example, input data 102 can define the structure of the substrate, for example, data defining the three-dimensional (3D) spatial positions of the atoms in the substrate. As yet another example, input data 102 can define the concentration of the substrate during the chemical reaction, the rate of change of the substrate concentration, etc.

[0026] Additionally or alternatively, input data 102 can characterize aspects of the product that can be formed as a result of the target reaction, i.e., after the enzyme binds to the substrate. For example, input data 102 can define the type of product. As another example, input data 102 can define the structure of the product, for example, data defining the three-dimensional (3D) spatial positions of the atoms in the product. As yet another example, input data 102 can define the reaction rate required for the formation of the product during the chemical reaction.

[0027] The enzyme design platform 100 includes a Transformer neural network 110, a diffusion model 120, and a post-processing engine 130.

[0028] The Transformer neural network 110 processes input data 102, data derived (generated) from the input data 102, or both of the inputs (i) and (ii) to generate a neural network output that defines a predicted location 112 of the active site of the enzyme.

[0029] The term "Transformer" neural network means that it has a Transformer-based architecture comprising multiple Transformer blocks, each Transformer block applying attention operations to its inputs to generate a Transformer block output. Each "block" comprises a group of one or more neural network layers, such as one or more attention layers, such as one or more self-attention layers or one or more cross-attention layers, one or more feedforward layers or both, and possibly other layers, including normalization layers and residual connection layers.

[0030] Proteins, such as enzymes or substrates, are composed of a series of amino acids (residues). An amino acid is an organic compound that includes an amino functional group and a carboxyl functional group, as well as amino acid-specific side chains (i.e., atomic groups). These amino acids are linked together by peptide bonds to form a protein. When in a sequence linked by peptide bonds, an amino acid can be referred to as an amino acid residue.

[0031] In an embodiment, the Transformer neural network 110 may be configured as a Graph Transformer neural network to perform the functions described herein.

[0032] For example, a Graph Transformer neural network can take a substrate graph as input and output a predicted location of the active site of an enzyme that should bind to this substrate. A substrate graph is a data structure that includes nodes and edges connecting the nodes. In general, nodes and edges can characterize various properties, such as the positions of atoms and bond interactions within the substrate involved in the target reaction.

[0033] In some embodiments, if the substrate is a protein or peptide, the nodes in the substrate graph can represent amino acid residues, and the edges in the substrate graph can represent the spatial relationships between amino acid residues based on their relative distances.

[0034] In some embodiments, the output of the Graph Transformer neural network can be in the form of a decoded graph structure having nodes and edges representing amino acids in the active site that binds to the substrate graph structure. Initially, each node in the substrate graph is represented by an embedding vector. These embeddings capture the features of the nodes and are used as input to the Graph Transformer neural network.

[0035] The Graph Transformer neural network also employs a self-attention mechanism to capture the relationships between nodes in the graph. By applying the self-attention mechanism, each node can notice other nodes in the graph, and the attention score represents a measure of the attention (or importance) that each node assigns to other nodes.The Graph Transformer neural network also employs a message-passing mechanism, where each node aggregates information from its neighbors based on its importance score (as determined by applying an attention mechanism) and updates its embedding accordingly.

[0036] In some embodiments, the Graph Transformer neural network typically comprises multiple layers, each performing a self-attention mechanism followed by a message-passing mechanism. This allows the model to capture increasingly complex relationships between nodes across multiple hops in the graph.

[0037] In some embodiments, the Graph Transformer neural network may also include a decoder layer that can process the graph generated by the message-passing updates to generate an output based on the processed graph representation. In this case, the predicted location of the enzyme's active site is generated as a graph structure having the location of each atom in the active site. In some embodiments, the decoder layer may receive combined node embeddings of the nodes in the graph generated by the message-passing updates (e.g., generated by pooling, averaging, cascading, etc., of the node embeddings of the nodes). For example, a Graph Transformer neural network can generate combined node embeddings based on a graph produced by message passing updates, and process the combined node embeddings using a decoder layer to generate another graph with the predicted location 112 of the enzyme's active site.

[0038] Some example implementations of the Graph Transformer neural network are described in more detail below: Yoon, M., Wu, Y., Palowitch, J., Perozzi, B. and Salakhutdinov, R., 2022. Graph generative model for benchmarking graph neural networks. arXiv preprint arXiv: 2207.04396 and Müller, L., Galkin, M., Morris, C. and Rampášek, L., 2023. Attending to graph transformers. arXiv preprint arXiv: 2302.04181. Instruction manual 4 / 14 pages 6 CN 121794577 A

[0039] As another example, a large language model can receive an input sequence consisting of tokens selected from a vocabulary and automatically regress to generate an output sequence consisting of tokens from the vocabulary.Large language models can have any of a variety of Transformer-based neural network architectures, such as encoder-only Transformer architectures, encoder-decoder-Transformer architectures, decoder-only Transformer architectures, other attention-based architectures, etc.

[0040] Example implementations of such large language models are described in more detail below: Rohan Anil, et al., “Palm 2 technical report.” arXiv preprint arXiv:2305.10403; and Hugo Touvron, et al., “Llama: Open and efficient foundation language models.” arXiv preprint arXiv:2302.13971, but others may also be used.

[0041] More specifically, the output generated by autoregression is created by generating each specific tag in an output sequence conditioned on the current input sequence, which includes any tags preceding the specific tag in the output sequence, i.e., tags that have been generated for any previous position preceding a specific position of the specific tag in the output sequence.

[0042] The vocabulary of tags may include text tags that can represent text data. For example, the vocabulary of symbols may include any of the following: characters, subwords, words, punctuation marks, symbolic symbols (e.g., #, $ and other symbols), mathematical symbols, numbers, etc., in a language (e.g., natural language, computer programming language, or markup language).

[0043] The input sequence may include any symbols selected from the vocabulary that can characterize various properties of the substrate involved in the target reaction, such as the type of substrate, the structure of the substrate, etc. The output sequence may include any symbols selected from the vocabulary that can define the predicted location 112 of the enzyme's active site.

[0044] The diffusion model 120 is a neural network that processes (i) the predicted location 112 of the enzyme's active site (already generated by the Transformer neural network 110), (ii) data derived from the predicted location 112 of the enzyme's active site, or both (i) and (ii) to generate an atomic representation of the enzyme. The atomic representation of the enzyme is a representative structure that defines the predicted three-dimensional (3D) locations 122 of the atoms included in the overall structure of the enzyme.

[0045] Specifically, the diffusion model 120 is configured to generate an atomic representation of the enzyme by performing a backdiffusion process across multiple update iterations, conditioned on the predicted location 112 of the enzyme's active site.

[0046] The diffusion model 120 can have any suitable conditional diffusion neural network architecture, which can be used to generate the diffusion model output in each of the multiple update iterations using information about the enzyme's active site.The 3D atomic positions of an enzyme can be inferred from the diffusion model output generated at the last update iteration.

[0047] For example, diffusion model 120 can be configured to process the diffusion model input for a given update iteration, which includes the current intermediate representation of the enzyme's atomic-level representation (up to a given update iteration), at any given update iteration to generate the diffusion model output for that given update iteration, from which the updated intermediate representation of the enzyme's atomic-level representation can be inferred.

[0048] In this way, enzyme design platform 100 uses information about a portion of the enzyme's structure (active site) and diffusion model 120, conditioned on these atomic positions on the active site residues, to predict the possible (stable) structure of the enzyme.

[0049] Such diffusion models can have been previously trained on a large corpus of proteins and their 3D atomic positions. Example implementations of such diffusion models are described in more detail below: Watson, J.L., Juergens, D., Bennett, NR, Trippe, BL, Yim, J., Eisenach, HE, Ahern, W., Borst, AJ, Ragotte, R.J., Milles, L.F., and Wicky, BI., 2023. De novo design of protein structure and function with RFdiffusion. Nature, 620(7976), pp. 1089-1100.

[0050] In some implementations, diffusion model 120 can be configured by training as a denoising diffusion model, i.e., it is trained to apply an incremental noise process to data on a specified protein structure. Diffusion model 120 may have been trained to learn the distribution of this denoising process as described on page 5 / 14 of the specification, CN 121794577 A.

[0051] In some embodiments, self-conditioning is employed during training, wherein the diffusion model 120 conditions its current prediction in relation to its previous outputs. This can improve the coherence and stacking effect of the generated structures.

[0052] In some embodiments, the diffusion model input at any given update iteration includes (i) an intermediate representation of the enzyme's atomic-level representation (up to the update iteration), (ii) the predicted position 112 of the enzyme's active site, (iii) data derived from the predicted position 112 of the enzyme's active site, or some combination of (i)-(iii). For example, the enzyme design platform 100 can provide the predicted atomic position 112 of the enzyme's active site as input and conditionally condition the diffusion model on the input to generate the predicted 3D positions 122 of the atoms included in the enzyme based on that input.

[0053] During this process, if a given update iteration of the denoising process is the first update iteration in the backdiffusion process, the current intermediate representation can be an initial intermediate representation, for example, an initial intermediate representation generated based on sampling the values ​​of each variable included in the atomic representation of the enzyme from a predetermined noise distribution (e.g., a Gaussian distribution). For any subsequent update iteration, the current intermediate representation is the updated intermediate representation generated in the immediately preceding update iteration.

[0054] The diffusion model output can directly define the updated intermediate representation, for example, where the diffusion model output includes predictions of the updated intermediate representation, or indirectly define it, for example, where the diffusion model output includes predictions of the noise components in the current intermediate representation. For example, the diffusion model output can define predictions of the noise that needs to be added to the atomic representation of the enzyme generated by the enzyme design platform 100 to generate the current intermediate representation, and the updated intermediate representation is generated by denoising the current intermediate representation using the predicted noise, i.e., by removing the predicted noise from the current intermediate representation.

[0055] For example, diffusion model 120 may have a Transformer neural network architecture that processes diffusion model inputs through a series of attention layers to generate diffusion model outputs, some or all of which apply attention operations on layer inputs including: (i) the predicted location 112 of the enzyme's active site, (ii) data derived from the predicted location 112 of the enzyme's active site, or both (i) and (ii). An example embodiment of such diffusion model 120 is described in more detail in the following: Jonathan Ho, et al., “Denoising diffusion probabilistic models.” Advances in neural information processing systems 33 (2020): 6840-6851, but others may also be used.

[0056] As another example, diffusion model 120 may have an equivariant neural network architecture that links 3D symmetries and effectively captures interatomic interactions.Example implementations of such diffusion models are described in the following: Igashov, I., Stärk, H., Vignac, C., Schneuing, A., Satorras, V.G., Frossard, P., Welling, M., Bronstein, M., and Correia, B., 2024. Equivariant 3D-conditional diffusion model for molecular linker design. Nature Machine Intelligence, pp. 1–11; and Hoogeboom, E., Satorras, VG, Vignac, C., and Welling, M., 2022, June. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning (pp. 8867–8887). PMLR.

[0057] As another example, diffusion model 120 may have a combination of Transformer architecture and equivariant neural network architecture, for example, in a cascaded architecture, where multiple diffusion models may operate at different atomic resolutions. An example implementation of a cascaded diffusion model is described in: Ho, J., Saharia, C., Chan, W., Fleet, D.J., Norouzi, M. and Salimans, T., 2022. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47), pp. 1–33.

[0058] Post-processing engine 130 further processes the atomic representation of the enzyme to generate a predicted structure 132 of the enzyme. In particular, the atomic-level representation of the enzyme requires further processing because it only defines the predicted three-dimensional (3D) positions 122 of the atoms included in the enzyme and lacks information about which of the multiple amino acids each atom may be specific to, i.e., which amino acid each atom in the specification (page 6 / 14, CN 121794577 A) may belong to.

[0059] The post-processing engine 130 may include one or more components that can map the atomic-level representation of the enzyme to the predicted structure 132 of the enzyme.In some embodiments, the predicted structure 132 of the enzyme can be defined by a set of structural parameters that collectively define the predicted three-dimensional (3D) configuration of atoms in the amino acids of the enzyme structure, for example, after the enzyme undergoes protein folding. Protein folding refers to the physical process by which one or more amino acid sequences fold into a three-dimensional (3D) configuration.

[0060] In one example, the set of structural parameters may include: (i) position parameters, (ii) rotation parameters, or both (i) and (ii) for each amino acid in the enzyme.

[0061] The position parameters of the amino acids can specify the predicted 3D spatial position of a particular atom in the amino acid of the enzyme structure. The particular atom can be the α-carbon atom in the amino acid, i.e., the carbon atom in the amino group, carboxyl group, and the side chain bonded to it. The position parameters of the amino acids can be represented in any suitable coordinate system, such as a three-dimensional [x, y, z] Cartesian coordinate system.

[0062] The rotation parameters of the amino acids can specify the predicted "orientation" of the amino acid in the enzyme structure. More specifically, the rotation parameter can specify a 3D spatial rotation operation, which, when applied to the coordinate system of the position parameter, results in the three main functional groups in the amino acid being in fixed positions relative to the rotation coordinate system. The three main chain atoms in an amino acid refer to the series of connections of nitrogen, α-carbon, and carbonyl carbon atoms (or, for example, α-carbon, nitrogen, and oxygen atoms) in the amino acid. The rotation parameter of an amino acid can be represented, for example, as an orthogonal 3×3 matrix with a determinant of 1.

[0063] In general, the position and rotation parameters of an amino acid define an egocentric reference system for the amino acid. In this egocentric reference system, the side chain of each amino acid can start from the origin and can be along the first bond of the side chain (i.e., the α-carbon-β-carbon bond) along a defined direction.

[0064] In another example, the structural parameters defining the predicted structure 132 of the enzyme can include a “distance map” that characterizes the estimated distance (e.g., measured in angstroms) between each pair of amino acids in the enzyme. The distance map can characterize the estimated distance between amino acid pairs, for example, by a probability distribution of the set of possible distances between amino acid pairs.

[0065] In yet another example, the structural parameters defining the predicted structure 132 of the enzyme can define the three-dimensional (3D) spatial location of each atom in each amino acid in the enzyme structure.

[0066] In some embodiments, components of the post-processing engine 130 include a sequence prediction neural network 140 and a folded neural network 150. The sequence prediction neural network 140 processes inputs including (i) an atomic-level representation of the enzyme, (ii) data derived from the atomic-level representation of the enzyme, or both (i) and (ii) to generate a neural network output defining an amino acid sequence.

[0067] The amino acid sequence can be represented in any suitable format. For example, an amino acid sequence can be represented in FASTA format. FASTA format is a text-based format in which amino acids are represented using single-letter codes.When represented in FASTA format, the amino acid sequence can begin with a greater than character (">") followed by a description of the sequence. The line immediately following the description is the sequence representation, one letter per amino acid.

[0068] As another example, the amino acid sequence can be represented as a sequence of one-hot vectors. In this example, each one-hot vector represents the corresponding amino acid in the amino acid sequence. The one-hot vectors have different components for each different amino acid (e.g., a predetermined number of amino acids). The one-hot vector representing a specific amino acid has a value of one (or some other predetermined value) in the component corresponding to that specific amino acid and a value of zero (or some other predetermined value) in other components.

[0069] In some embodiments, the sequence prediction neural network 140 can be configured as a message passing neural network (MPNN). A message passing neural network is a type of neural network that operates on graphical data that defines an enzyme graph that generates an amino acid sequence as output, as per specification page 7 / 14, page 9, CN 121794577 A. In these embodiments, each amino acid in the protein sequence is an organic compound comprising an amino functional group and a carboxyl functional group, as well as a side chain, i.e., a subset of atoms included in the atomic-level representation of the enzyme generated by the diffusion model 120, which is specific to the amino acid.

[0070] The enzyme graph operated by the sequence prediction neural network 140 is a data structure comprising nodes and edges connecting the nodes. For example, nodes in the enzyme graph may represent atoms included in the enzyme, and edges in the enzyme graph may represent the relative distances between atoms included in the enzyme. Such enzyme graphs may be generated by the enzyme design platform 100 based on the atomic-level representation of the enzyme, defined by the predicted 3D positions 122 of atoms included in the enzyme generated by the diffusion model 120.

[0071] Typically, the MPNN has a two-stage process: a message passing stage and a readout stage. In the message passing stage, the MPNN updates the hidden state of each node in the graph based on messages passed from neighboring nodes. This is done using learned message functions and vertex update functions. The message function and vertex update function are differentiable and learned during training, allowing MPNNs to flexibly capture long-range dependencies in the graph. The readout phase then uses the learned readout function to compute the feature vector for the entire graph, which is invariant to the permutation of node states. This two-stage process makes MPNNs a powerful and versatile class of neural networks that can efficiently learn representations of graphs through iterative message passing and readout operations.

[0072] An example embodiment of such a message-passing neural network 140 predicting a protein sequence from the predicted atomic positions 122 of the protein is described in more detail below: Justas Dauparas et al., “Robust deep learning–based protein sequence design using proteinMPNN.” Science, 378(6615):49–56, but others may also be used.

[0073] The folding neural network 150 is a neural network that processes inputs including (i) an amino acid sequence, (ii) data derived from the amino acid sequence, or both (i) and (ii) to generate an output that defines the predicted structure 132 of the enzyme. The predicted structure can define an estimate of the three-dimensional (3D) conformation of the atoms in the amino acid sequence of the enzyme (generated by the sequence prediction neural network 140) after the enzyme has undergone protein folding. As mentioned above, in some embodiments, the predicted structure 132 of the enzyme can be defined by a set of structural parameters that define the 3D spatial position of each amino acid in the enzyme and optionally also define 3D spatial rotations.

[0074] Example implementations of such folded neural networks 150 are described in more detail below: John Jumper et al., “Highly accurate protein structure prediction with AlphaFold.” Nature 596, 583–589 (2021) and Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S. and Rives, A., 2022. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022, p. 500902, but others may also be used.

[0075] Before using the Transformer neural network 110, diffusion model 120, sequence prediction neural network 140, and folding neural network 150 in series (as discussed above) to generate a predicted structure 132 of the enzyme from the input data 102, the enzyme design platform 100 or a separate training system trains these neural networks, along with other trainable components of the platform, on training data so that these neural networks can perform their respective functions well.

[0076] In one example, based on optimizing a self-supervised objective function, the Graph Transformer neural network can be trained on known public repositories (e.g., scientific literature repositories, protein databases, and experimental datasets) that include data defining active sites and substrate locations in 3D space. As described above, the location of the active site or substrate can be defined, for example, by its respective structural parameters, which collectively define the predicted three-dimensional (3D) conformation of atoms in the amino acids within the active site or substrate structure.

[0077] In a similar example, based on optimizing a self-supervised objective function, a large language model can be trained on known public repositories, such as those listed in the specification (page 8 / 14, 10 CN 121794577 A), which include active sites and substrate locations in 3D space. For example, an enzyme design platform 100 or a separate training system pre-trains the large language model on a language modeling task (e.g., given a current text tag sequence, the task of predicting the next tag after the current sequence in the training data) using a large dataset of text (e.g., text publicly available from the Internet or other text corpora), and then fine-tunes the pre-trained large language model using known public repositories, based on optimizing a maximum likelihood objective function, which include active sites, substrate locations, and information about the products formed due to the reaction between the enzyme and the substrate. In this way, the large language model is trained to learn the language and concepts of enzyme biology, biochemistry, structural biology, etc.

[0078] In another example, a diffusion model 120 can be trained on known public repositories, including known protein structures, using a denoised score matching objective. In one example, such a public repository could be a Protein Data Bank (PDB) repository mentioned above.

[0079] Figure 2 is a flowchart of an example process 200 for generating a predicted structure of an enzyme. For example, the operation of process 200 can be performed by the enzyme design platform 100 of Figure 1 or by another system implemented as a computer program on one or more computers in one or more locations. The operation of process 200 can also be implemented as instructions stored on a computer-readable medium, which can be non-transitory.One or more data processing devices execute instructions causing one or more data processing devices to perform the operation of process 200.

[0080] Input data characterizing one or more aspects of the target enzyme-catalyzed reaction is obtained (step 202). For example, the enzyme design platform 100 may receive input data as user input from a user of the platform via a data communication network (e.g., using an application programming interface (API) or graphical user interface (GUI) provided by the system) or a web browser through which the user can interact with the platform. As another example, the enzyme design platform 100 may receive user input from a user specifying which data, maintained by the platform or by another computer system accessible to the platform, should be used as data characterizing one or more aspects of the target enzyme-catalyzed reaction.

[0081] The target enzyme-catalyzed reaction may involve an enzyme, a substrate, and one or more products formed after the enzyme binds to the substrate. The input data may characterize aspects of the substrate involved in the target enzyme-catalyzed reaction, i.e., it may characterize the substrate used to bind to the enzyme. For example, the input data may define the type of substrate. As another example, the input data may define the structure of the substrate. As yet another example, the input data may define the concentration of the substrate during the chemical reaction, the rate of change of the substrate concentration, etc.

[0082] Alternatively or additionally, the input data can characterize aspects of the product that can be formed due to the target enzyme-catalyzed reaction, i.e., after the enzyme binds to the substrate. For example, the input data can define the type of product. As another example, the input data can define the structure of the product. As yet another example, the input data can define the reaction rate required for product formation during the chemical reaction.

[0083] As a specific example for illustration, the target enzyme-catalyzed reaction can be a chemical reaction involving the degradation of perfluoroalkyl and polyfluoroalkyl substances (PFAS) by means of an enzyme, and the input data can characterize the substances involved in the chemical reaction (i.e., PFAS). In this specific example, the input data can be generated based on known 3D atomic positions in the PFAS. For example, the input data can have a graphical structure.

[0084] Figure 3 is an example illustration 300 of perfluoroalkyl and polyfluoroalkyl substances (PFAS). As shown, PFAS comprises molecules consisting of linked chains of carbon (C) and fluorine (F) atoms. Due to the strength of the carbon-fluorine bonds, the degradation of PFAS in the environment can be slow or not at all. PFAS can be used in non-stick coatings for cookware, stain-resistant clothing and carpets, or in fire-fighting foams to make them more effective. In addition, PFAS are used in industries such as aerospace, automotive, construction, electronics, and military equipment. Currently, more than 9,000 PFAS have been identified. Because PFAS can persist in the environment for unknown periods and gradually accumulate in the human body, concerns about their public health impacts are growing.

[0085] PFAS include, but are not limited to, perfluorooctanoic acid (PFOA), perfluorooctane sulfonate (PFOS), hexafluoropropylene oxide (HFPO) dimer acid and its ammonium, sodium, and potassium salts or any combination thereof. Other examples of PFAS include N-ethylperfluorooctane sulfonamide, N-methylperfluorooctane sulfonamide, perfluorobutane sulfonic acid, perfluorodecanoic acid, perfluorododecanic acid, perfluoroheptanoic acid, perfluorohexane sulfonic acid, perfluorohexanoic acid, and perfluorononanoic acid. In some examples, PFAS include perfluorooctanoic acid (PFOA), perfluorooctane sulfonate (PFOS), or any combination thereof.

[0086] The Transformer neural network processes neural network inputs, which include (i) input data, (ii) data derived from the input data, or both (i) and (ii) to generate a neural network output that defines the predicted location of the enzyme's active site (step 204). An active site refers to a designated critical location of an enzyme, where a substrate (e.g., PFAS) binds and catalyzes the reaction as part of the target enzyme. In some embodiments, the predicted location of the enzyme's active site may include the predicted 3D spatial location of the active site within the enzyme structure.

[0087] For example, when a Transformer neural network is configured as a Graph Transformer neural network, the enzyme design platform 100 can generate graphical data defining a substrate map from input data and process the substrate map using the Graph Transformer neural network to generate the predicted location of the enzyme's active site as output.

[0088] In some embodiments, the output of the Graph Transformer neural network can be converted into a readable format with information about 3D atomic positions.

[0089] Figure 4 is an example illustration 400 of partial output data that can be obtained from further processing of the output of the Graph Transformer neural network. In the example of Figure 4, the output data defines the predicted amino acid and the 3D atomic position of the active site in a Protein Database (PDB) format. The PDB format is a text format describing the three-dimensional structure of molecules such as proteins and nucleic acids, stored in a global protein database (www.pdb.org). Therefore, the PDB format provides descriptions and annotations of protein and nucleic acid structures, including atomic coordinates, secondary structure assignments, and atomic connectivity.

[0090] The output data in the PDB format can define the structure of predicted active site residues. Specifically, the output data in the PDB format can describe the coordinates of atoms that are potential enzyme parts.For example, in Figure 4, the first ATOM line describes the α-N atom as part of the aspartic acid residue in chain L, and its residue (amino acid) ID as "1"; the first three floating-point numbers are its x, y, and z coordinates in angstroms.

[0091] As another example, when the Transformer neural network is configured as a large language model, the enzyme design platform 100 can represent the input data as an input tag sequence and process the input tag sequence using the large language model to generate an output tag sequence that defines the predicted location of the enzyme's active site. In this example, in some embodiments, the neural network output may include output data in the same format as the input data, for example, the predicted location of the enzyme's active site is also in PDB format.

[0092] The diffusion model generates an atomic-level representation of the enzyme based on the predicted location of the enzyme's active site (step 206). The atomic-level representation of the enzyme can define the predicted 3D locations of the atoms included in the enzyme. In some embodiments, the atomic-level representation of the enzyme is in the same format as the input data of the Transformer neural network, for example, the atomic-level representation of the enzyme is also in PDB format.

[0093] To generate an atomic-level representation of the entire enzyme, the diffusion model is configured to perform a back-diffusion process conditioned on the predicted location of the enzyme's active site by updating the intermediate representation of the enzyme's atomic-level representation in each of multiple update iterations. In other words, the atomic-level representation of the enzyme is an updated intermediate representation following the last iteration in multiple update iterations.

[0094] Figure 5 is an example illustration 500 of an atomic-level representation of an enzyme defining the predicted 3D locations of the atoms included in the enzyme. In Figure 5, the atomic-level representation of the enzyme is in PDB format. For example, in Figure 5, the first ATOM line describes the α-N atom as part of a glycine residue in chain L, and its residue (amino acid) ID as "1"; the first three floating-point numbers are the predicted x, y, and z coordinates of the α-N atom in angstroms; and the next two numbers are the occupancy of the α-N atom and the temperature factor, respectively.

[0095] Note that in column 5, which should correspond to an amino acid residue, all rows are represented as GLY. This is the default placeholder residue generated after a back-diffusion process using the diffusion model. Therefore, the next step of process 200 is to predict the sequence, i.e., the amino acids included in the enzyme, based on the 3D coordinates defined in the atomic representation of the enzyme specification, page 10 / 14, 12 CN 121794577 A.

[0096] The sequence prediction neural network processes the neural network input, which includes (i) the atomic representation of the enzyme, (ii) data derived from the atomic representation of the enzyme, or both (i) and (ii) to generate a neural network output that defines the amino acid sequence (step 208). The amino acid sequence can define the linear order of the amino acids that make up the enzyme. The amino acid sequence can be represented in any suitable format.

[0097] Figure 6 is an example illustration 600 of an amino acid sequence in FASTA format. In Figure 6, the amino acid sequence begins with a greater than character (">"), followed by a description of the sequence. The line immediately following the description line is the sequence representation, one letter for each amino acid. For example, in Figure 6, "A" represents the amino acid alanine, "C" represents the amino acid cysteine, "T" represents the amino acid threonine, and "G" represents the amino acid glycine, etc.

[0098] The folding neural network processes the neural network input, which includes (i) the amino acid sequence, (ii) data derived from the amino acid sequence, or both (i) and (ii) to generate a neural network output that defines the predicted structure of the enzyme (step 210). Continuing with the PFAS example mentioned above, the neural network output can define the predicted structure of an enzyme capable of catalyzing a chemical reaction involving the degradation of PFAS.

[0099] In some embodiments, the predicted structure of the enzyme can define the three-dimensional configuration of the atoms included in the enzyme. The predicted structure can define an estimate of the 3D configuration of the atoms in the amino acid sequence of the enzyme after the enzyme undergoes protein folding. As mentioned above, in some embodiments, the predicted structure of an enzyme can be defined by a set of structural parameters that define the 3D spatial position of each amino acid in the enzyme and optionally also define 3D spatial rotation.

[0100] In particular, since the backdiffusion process is conditioned on or guided by the predicted position of the enzyme's active site, the atomic-level representation of the enzyme and the corresponding predicted structure of the enzyme will integrate the geometry of these active sites. Thus, for example, the accessibility of the enzyme to the substrate during the target enzyme-catalyzed reaction can be improved.

[0101] The predicted structure of the enzyme generated by the enzyme design platform 100 can be used in any of a variety of ways. For example, the enzyme design platform 100 can provide the predicted structure of the enzyme for presentation to a user on a user's computer or store the predicted structure of the enzyme in an output data repository for later use.

[0102] In some embodiments, enzymes having predicted structures can be physically synthesized, for example using manual or automated laboratory techniques. Such physically synthesized proteins can be used in any of a variety of possible ways. For example, enzymes generated by enzyme design platform 100 can be used to catalyze industrial processes involving the decomposition of PFAS, such as chemical reactions in which the substrate is PFAS, and one or more products formed after the enzyme binds to the substrate include substances that are useful and / or less harmful.

[0103] Physical synthesis of enzymes having predicted structures generated by enzyme design platform 100 may include experimentally validating the enzyme, for example, by measuring the enzyme's stability, or by measuring the enzyme's actual structure and comparing it with the structure predicted by enzyme design platform 100, or by evaluating the products formed after the enzyme binds to the substrate based on one or more aspects of the products defined for the input data.

[0104] In some embodiments, a relatively large number of predicted enzyme structures that can be generated by repeating process 200 multiple times can be filtered according to a predetermined set of criteria, and a relatively small number of the filtered predicted enzyme structures can be synthesized in the laboratory and their performance tested for validation. For example, enzymes with these filtered predicted structures can be evaluated based on their predicted performance, structural stability, and other relevant factors. Data defining the predicted structures of the filtered enzymes can be discarded, i.e., data not included in the relatively small number of filtered predicted structures.

[0105] For example, the predetermined set of criteria may include distance criteria between predicted enzyme structures from the diffusion model and the folded neural network 150. Both models include data PDB structures, which have information about the α-carbon positions of potential amino acids. The root mean square deviation (RMSD) is measured between the diffusion model and the folded neural network for the α-carbon positions of amino acids in the same order. RMSD is a measure used to quantify the similarity or difference between two molecular structures or conformations. It calculates the average distance between atoms of two structures after optimal superposition or alignment, which is an indicator of how much the predicted structure deviates from the two models. In this example, structures with low RMSD can be selected, and a list of selected structures can then be sent to a laboratory for validation.

[0106] As another example, a predetermined set of criteria may include a Predictive Average Alignment (PAE) criterion that may depend on the output of the folded neural network. PAE measures the confidence in the relative positions and orientations of different parts of the predicted structure (e.g., domains). Low PAE values ​​between residues from different domains indicate that the folded neural network predicts well-defined relative positions and orientations of these domains. In this example, structures with low PAE values ​​can be selected, and a list of selected structures can then be sent to a laboratory for validation.

[0107] This specification uses the term "configuration" in association with system and computer program components. For a system of one or more computers configured to perform a particular operation or action, it means that the system has software, firmware, hardware, or a combination thereof installed thereon that causes the system to perform that operation or action during operation. For one or more computer programs configured to perform a specific operation or action, it means that one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0108] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuits, tangible computer software or firmware, computer hardware (including the structures disclosed in this specification and their structural equivalents), or combinations thereof.Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, such as one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium, for execution by or control of the operation of a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, program instructions can be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus.

[0109] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of devices, apparatuses, and machines for processing data, including, by way of example, programmable processors, computers, or multiprocessors or computers. The apparatus can also be or further include special-purpose logic circuitry, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit). In addition to hardware, the apparatus may optionally include code that creates an execution environment for the computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0110] A computer program, also referred to or described as a program, software, software application, application, module, software module, script, or code, may be written in any form of programming language, including compiled or interpreted languages, or declarative or programming languages; and it may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may (but not necessarily) correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data, such as one or more scripts stored in a markup language document, stored in a single file dedicated to the program in question, or stored in multiple coordination files, such as a file storing one or more modules, subroutines, or code portions. A computer program may be deployed to execute on a single computer or deployed to execute on multiple computers located at a site or distributed across multiple sites and interconnected by a data communication network.

[0111] In this specification, the term “database” is used broadly to refer to any collection of data: the data does not need to be structured in any particular way or does not require structure, and it may be stored on storage devices in one or more locations. Thus, for example, an indexed database may include multiple collections of data, each of which may be organized and accessed differently. Instruction manual 12 / 14 pages 14 CN 121794577 A

[0112] Similarly, in this specification, the term "engine" is used broadly to refer to a software-based system, subsystem or process programmed to perform one or more specific functions.Generally, the engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines may be installed and run on the same or more computers.

[0113] The processes and logic flows described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by dedicated logic circuitry (e.g., FPGA or ASIC) or a combination of dedicated logic circuitry and one or more programmable computers.

[0114] A computer suitable for executing computer programs can be based on a general-purpose or dedicated microprocessor or both, or any other type of central processing unit. Generally, the central processing unit will receive instructions and data from read-only memory or random access memory or both. The basic elements of a computer are the central processing unit for fulfilling or executing instructions and one or more storage devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into dedicated logic circuitry. In general, a computer will also include one or more mass storage devices (e.g., disks, magneto-optical disks, or optical disks) for storing data, or be operatively coupled thereto to receive data from or transfer data to or both. However, a computer does not need to have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name just a few.

[0115] Suitable computer-readable media for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, by way of example, semiconductor storage devices, such as EPROMs, EEPROMs, and flash memory devices; disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs.

[0116] To provide interaction with the user, embodiments of the subject matter described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input.Additionally, a computer can interact with a user by sending and receiving documents to and from a device used by the user; for example, by sending a webpage to a web browser on the user's device in response to a request received from a web browser. Furthermore, a computer can interact with a user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and receiving response messages returned by the user.

[0117] The data processing apparatus for implementing machine learning models may also include, for example, a dedicated hardware accelerator unit for processing the computationally intensive portions common in machine learning training or production (e.g., inference) workloads.

[0118] Machine learning models can be implemented and deployed using machine learning frameworks (e.g., the TensorFlow framework or the Jax framework).

[0119] Embodiments of the subjects described in this specification can be implemented in computing systems that include backend components (e.g., as a data server), middleware components (e.g., an application server), or frontend components (e.g., a client computer having a graphical user interface, a web browser, or an application through which a user can interact with embodiments of the subjects described in this specification), or any combination of one or more such backend components, middleware components, or frontend components. The components of the system can be interconnected through any form or medium of digital data communication (e.g., a communication network). Examples of networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0120] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact via a communication network. The relationship between clients and servers is generated by computer programs running on their respective computers, and there is a client-server relationship between them. In some embodiments, the server transmits data (e.g., HTML pages) to a user device, for example, to display data to a user interacting with the device as a client and to receive user input. Data generated at the user device, such as the result of user interaction, may be received from the device at the server.

[0121] Although this specification contains many specific details of embodiments, these should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features characteristic of a particular embodiment of a particular invention. Certain features described in this specification in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single implementation can also be implemented separately in multiple implementations or in any suitable sub-combination.Furthermore, although features may be described above as functioning in certain combinations, or even initially claimed in this way, in some cases one or more features from the claimed combination may be removed from that combination, and the claimed combination may refer to a sub-combination or a variation of the sub-combination.

[0122] Similarly, although operations are depicted in the drawings and recited in the claims in a specific order, this should not be construed as requiring such operations to be performed in the specific order or sequence shown, or requiring all illustrated operations to be performed to obtain the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0123] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in different orders and may still achieve the desired result. As an example, the processes depicted in the drawings do not necessarily require the specific order or sequence shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Instruction manual 14 / 14 pages 16 CN 121794577 A Figure 1 Instruction manual figure 1 / 6 pages 17 CN 121794577 A Figure 2 Instruction manual figure 2 / 6 pages 18 CN 121794577 A Figure 3 Instruction manual figure 3 / 6 pages 19 CN 121794577 A Figure 4 Instruction manual figure 4 / 6 pages 20 CN 121794577 A Figure 5 Instruction manual figure 5 / 6 pages 21 CN 121794577 A Figure 6 Instruction manual figure 6 / 6 pages 22 CN 121794577 A.

Claims

1. A computer-implemented method comprising: Receive input data characterizing enzyme-catalyzed reactions; The Transformer neural network is used to generate predicted locations of enzyme active sites based on the input data; A predicted structure of the enzyme is generated using at least a diffusion model and based on the predicted location of the enzyme's active site, the predicted structure of the enzyme defining the three-dimensional configuration of the atoms included in the enzyme.

2. The method of claim 1, wherein the input data comprises data characterizing the substrate bound to the enzyme.

3. The method of any one of claims 1-2, wherein the Transformer neural network is configured as a Graph Transformer neural network for manipulating graphical data defining the enzyme-catalyzed reaction.

4. The method of any one of claims 1-2, wherein the Transformer neural network is configured to automatically regress and generate a large language model neural network for predicting the location of the active site of the enzyme.

5. The method of any one of claims 1-4, wherein generating the predicted structure of the enzyme comprises: The diffusion model is used to generate the predicted three-dimensional positions of the atoms included in the enzyme; An amino acid sequence is generated using a sequence prediction neural network based on the predicted three-dimensional positions of the atoms included in the enzyme; and The predicted structure of the enzyme is generated based on the amino acid sequence using a folded neural network or a protein structure prediction model.

6. The method of any one of claims 1-5, further comprising performing one or more actions on the enzyme having the predicted structure.

7. The method of claim 6, wherein the one or more actions include: Whether an enzyme having the predicted structure is effective is determined based on the three-dimensional configuration of the atoms defined by the predicted structure.

8. The method of claim 6, wherein the one or more actions include: The enzyme having the predicted structure was synthesized and validated in the laboratory.

9. A system comprising: One or more computers; as well as One or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operation of the corresponding method as described in any one of claims 1-8.

10. One or more non-transitory computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operation of the corresponding method as described in any one of claims 1-8.

11. An AI-driven enzyme design platform that combines large language models and diffusion models to design novel enzymes.

12. The AI-driven enzyme design platform of claim 11, wherein the large language model is trained on scientific literature, protein databases, and experimental datasets related to enzyme biology, biochemistry, and structural biology.

13. The AI-driven enzyme design platform of claim 11 or claim 12, wherein the large language model is used to predict potential active sites for a given enzyme target.

14. The AI-driven enzyme design platform of any one of claims 11-13, wherein the predicted active sites from the large language model are used to prompt the diffusion model, which designs novel enzymes integrating these active sites.

Images