Discrete graph probabilistic denoising diffusion model for protein sequence generation

Abstract

Disclosed are discrete graph probabilistic denoising diffusion models for protein sequence generation, given protein backbone guides the diffusion process of the corresponding amino acid residue type. The model infers the joint distribution of amino acids, conditional on the biophysical properties of the nodes and the local environment. In addition, the present invention uses an amino acid substitution matrix for the diffusion forward process, encoding the biological significance of amino acids prior knowledge, including their spatial and sequence neighbors and themselves, thereby reducing the sampling space of the generation process. The model of the present invention achieves industry-leading performance in sequence recovery and shows great potential in generating diverse protein sequences that determine the structure of the protein backbone.

Description

Technical Field

[0001] This invention belongs to the field of protein sequence generation technology, and relates to a discrete graph probabilistic denoising diffusion model for protein sequence generation. Background Technology

[0002] Protein sequence generation, or de novo protein design, aims to predict feasible amino acid sequences that can fold into specific 3D protein structures or possess specific functions. Developing methods for protein sequence generation facilitates the design of novel proteins with desired structural and functional properties. These proteins can be applied in numerous fields, such as targeted drug delivery and enzyme design, and hold broad promise for both academic research and industrial applications.

[0003] Protein sequence generation is a challenging task because of the one-to-many mapping between structure and sequence, and function and sequence; that is, numerous amino acid sequences may fold into the same protein backbone and perform the same function. Furthermore, the vast sequence space, the complexity of protein folding, and the intricate mechanisms involved in protein function make protein sequence generation itself a highly challenging task. Besides determining protein folding states based on energy-based physical reasoning, recent advances in deep learning have made significant progress in directly learning the mapping from protein structure to amino acid sequences. For example, discriminative models, using Transformer-based models, formulate this problem as predicting the most likely sequence given a structure. However, these models typically only provide a fixed answer, and therefore are not entirely satisfactory in accurately capturing the one-to-many mapping from protein structure to non-unique amino acid sequences. Diffusion probability models, as an emerging type of generative approach, offer the potential to generate diverse sequence candidate sets that define the protein backbone. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a discrete graph probabilistic denoising diffusion model for protein sequence generation.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A discrete graph probabilistic denoising diffusion model for protein sequence generation, where the given protein backbone guides the corresponding amino acid residue types, specifically including:

[0007] (1) Discrete probability inverse diffusion process based on deep neural network training;

[0008] (2) Conditional reverse diffusion distribution generation guided by prior knowledge of proteins;

[0009] (3) Equal-variable graph neural network (EGNN) is used for the parameterized denoising process of reverse diffusion;

[0010] (4) Sampling and accelerating the denoising process based on the Denoising Diffusion Implicit Model (DDIM).

[0011] As one of the preferred technical solutions, the model uses the given graph G = {X, E}, where the node features are X and the edge features are E as conditions; among them, the node features X = [X pos , X aa , X prop contains the position X pos of the amino acid, the amino acid type X aa , and the spatial and biochemical properties X prop .

[0012] As one of the preferred technical solutions, step (1) specifically includes a diffusion process, training a denoising network, and a parameterized generation process.

[0013] As one of the further preferred technical solutions, the specific method of the diffusion process is as follows: Independently add noise to each amino acid node of the protein. For any given node, the transition probability is defined by the matrix Q t . Through a predefined transition matrix, the forward diffusion kernel is defined as: where is the transition probability matrix at time step t.

[0014] As one of the further preferred technical solutions, the specific method of training the denoising network is: Construct a denoising neural network f θ , parameterized by the parameter θ; this network accepts the noise input where X t is the concatenation of the noise amino acid type and other amino acid properties, 20 one-hot encoded amino acid types and 15 geometric properties of the protein backbone; the training objective of the denoising model is to predict the noise-free amino acid type X aa , so as to model various sequence combinations potentially corresponding to the protein structure while maintaining its internal structural constraints.

[0015] As one of the further preferred technical solutions, the specific method of the parameterized generation process is: The new amino acid sequence is generated by performing reverse diffusion iteration on the amino acid type x aa of each node at time t (1 < t < T). The corresponding generated probability distribution p θ (x t-1 |x t ) is predicted by the neural network for the probability The estimation is performed. This invention utilizes a deep neural network to predict probabilistic marginalization in order to calculate the generation distribution for each iteration step:

[0016] in Where the posterior distribution q(x) t-1 |x t x aa The transition matrix and the node features at time t are given by x. t and amino acid type x aa The result is calculated. Where x... aa Probability predicted by the denoising network Obtained by sampling.

[0017] As one of the preferred technical solutions, step (2) specifically includes two parts: Markov transition matrix and secondary structure sampling. The specific method for obtaining the Markov transition matrix is ​​as follows: first, the block substitution matrix (BLOSUM) is incorporated into the diffusion and generation process, normalized to a probability, and then the softmax function is used; then, different probability temperatures are used to adjust the normalized block substitution matrix B to control the noise scale during the diffusion process; the transition matrix Q at time t... t By Q t =B T The method for secondary structure sampling is as follows: DSSP is used to predict the secondary structure of each amino acid, and one-hot encoding is used to represent these structures; the neural network takes the one-hot encoding as input and uses it to denoise the amino acids, thereby incorporating the protein secondary structure as a guiding condition in the denoising sampling process.

[0018] As one of the preferred technical solutions, the specific method of step (3) is as follows:

[0019] For a protein graph with n amino acids, at layer l, an isovariant graph convolution (EGC) yields the node expression. Recessive vector representations of amino acid types and geometric properties, and edge representations. This represents the hidden features on connected nodes i and j, and the three-dimensional coordinates of the nodes. The modified EGC layer can be used to update the hidden layer representation H of the (l+1)th layer node. (l+1) Hidden layer representation M of edges (l+1) In short, the present invention has

[0020] H (l+1) M (l+1) =EGC[H (l) X pos M (l) ].

[0021] As one of the further preferred technical solutions, the EGC layer defines the following operations:

[0022] -For edge embedding:

[0023]

[0024] -For node coordinates:

[0025]

[0026] - For hidden representations of nodes:

[0027]

[0028] Where, φ e φ h and φ x It is an edge and node propagation operation, φ x It is an additional operation that embeds vector edges into m. ij Project onto a scalar. The modified EGC layer preserves the node coordinates X. pos It exhibits isomorphism in rotation and translation, and maintains invariance in arrangement on the node set.

[0029] As one of the preferred technical solutions, the specific method of step (4) is as follows:

[0030]

[0031] Temperature T is used to adjust the degree of determinism or randomness in the estimated probability distribution. The posterior distribution after multi-step denoising is:

[0032]

[0033] The beneficial effects of this invention are as follows:

[0034] This invention proposes a novel graph-based denoising diffusion model for protein sequence generation, where a given protein backbone guides the diffusion process for corresponding amino acid residue types. For a target protein, the model infers the joint distribution of the amino acid sequence based on the biophysical characteristics of the amino acid nodes and their local environment. Furthermore, this invention utilizes an amino acid substitution matrix for the forward diffusion process, encoding prior knowledge of the biological significance of the amino acids, including their spatial and sequence neighbors and their own characteristics, thereby reducing the sampling space of the generation process. The model of this invention achieves industry-leading performance in sequence recovery and demonstrates great potential in generating diverse protein sequences with defined protein backbone structures.

[0035] Diffusion probability models are gaining increasing attention due to their powerful learning capabilities. Under inherent stochastic conditions, they can generate a wide variety of molecular outputs from a fixed set of conditions. For example, TorsionDiffusion learns the torsion angle distribution of heavy atoms to simulate the conformation of small molecules. Meanwhile, SMCDIFF learns the stable backbone distribution supporting the target motif to accomplish protein folding tasks. Similarly, DIFFDOCK employs a generative approach in protein ligand docking, creating a series of possible ligand binding poses for the target pocket structure.

[0036] Despite the widespread use of diffusion models, their full potential remains relatively unexplored in the context of protein backfolding. Current sequence design approaches are primarily based on language models, including masked language models and autoregressive generative models. Masked language models treat sequence generation as a masking enhancement process by tokenizing amino acids. These models typically mask the sequence by selecting tokens from a given context and then learning to predict these masked tokens. When trained with a parameterized objective, this process can be viewed as a discrete diffusion probability denoising model. In contrast, autoregressive models can be viewed as a deterministic diffusion process. It introduces a conditional distribution for each token, but the overall dependencies across the entire amino acid sequence are reconstructed through independently executed diffusion processes.

[0037] Conversely, diffusion probability models employ an iterative prediction approach, generating samples with less noise and demonstrating potential in capturing the inherent diversity of real-world data distributions. This unique characteristic further underscores the promising role diffusion models may play in advancing protein sequence design. To bridge this gap, the applicant has for the first time attempted protein sequence generation using a discrete diffusion probability model. This invention models the sequence generation problem as a denoising problem, namely, recovering the wild-type of randomly assigned amino acids in a protein (backbone) graph. The protein graph, containing all amino acid spatial and biochemical information, is represented by an isovariant graph neural network, with the diffusion process occurring at the graph nodes. In real-world sequence generation tasks, the proposed model achieves a recovery rate of up to 70%, particularly for biologically significant conserved regions. Furthermore, the generated sequences, predicted using AlphaFold2, exhibit high pLDDT confidence and show minimal difference from the native protein structure compared to experimental measurement errors (3 Å), indicating structural consistency.

[0038] To maintain the desired functionality, this invention innovatively conditions the model to secondary and tertiary structures, presenting them as residue graphs and corresponding node features. The main contributions of this invention are threefold. First, it proposes GRADE-IF, an inverse folding diffusion model supported by rotation-translation equivariant graph neural networks. Compared to other models, it can generate a diverse set of sequence candidates. Second, unlike the uniform noise in traditional discrete diffusion models, this invention encodes prior knowledge of amino acid responses to evolutionary pressures by using a block substitution matrix as a translation kernel. Furthermore, to accelerate the sampling process, this invention transforms the Denoising Diffusion Implicit Model (DDIM) from its original continuous form into a sampling method suitable for discrete cases, thereby accelerating model training and inference speed. Attached Figure Description

[0039] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following drawings are provided for illustration.

[0040] Figure 1 This is a diagram of the model structure.

[0041] Figure 2 This is an amino acid diagram generated based on the protein backbone.

[0042] Figure 3 Examples of amino acid probability distributions generated based on a random probability distribution (left) and a block substitution matrix (right).

[0043] Figure 4 An isovariant graph neural network is used to fit the amino acid probability distribution during the denoising process.

[0044] Figure 5 GRADE-IF has a higher recovery capacity in internally conserved amino acids.

[0045] Figure 6 Compared to PIFOLD and PROTEINMPNN, GRADE-IF can simultaneously ensure both the quality and diversity of generated proteins.

[0046] Figure 7 DDIM can significantly improve model speed with minimal loss of model performance.

[0047] Figure 8 Comparison of the predicted structures of protein sequences generated by different GRADE-IFs after AlphaFold2 folding and the structures of the original protein (PDB ID: 3FKF). Detailed Implementation

[0048] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0049] 1.1 Amino acid diagram of protein backbone formation

[0050] The residue diagram is represented as Used to describe the geometry of proteins (e.g.) Figure 2 (As shown). Specifically, each node represents an amino acid in a protein. Accordingly, each node is assigned a carefully selected set of node attributes X to reflect its biophysical and topological properties. The local environment of a given node is defined by its spatial neighbors, determined by the k-nearest neighbor (kNN) algorithm. Therefore, each amino acid node is connected to at most k other nodes in the graph, particularly with… The node with the smallest Euclidean distance among all nodes within the contact region. The properties of an edge are denoted as E∈R. 93 This describes the relationships between connected nodes. These relationships are determined by parameters such as inter-atomic distance, local NC position, and sequence position encoding scheme.

[0051] 1.2 Protein sequence generation is defined as a noise reduction problem.

[0052] The goal of protein sequence generation is to design sequences that can fold into a pre-specified desired structure. This invention utilizes the coordinates of Cα atoms to represent the three-dimensional positions of amino acids in Euclidean space, thus reflecting the protein backbone. Based on naturally occurring protein structures, the model of this invention is constructed to generate native protein sequences based on their backbone atomic coordinates. Formally, this invention treats this problem as learning a conditional distribution p(X). aa |X pos Given a protein of length n and a sequence of spatial coordinates representing each Cα atom in the backbone structure. The goal is to predict This refers to the native sequence of amino acids. This density model is modeled along with other amino acids throughout the chain. The model of this invention is trained by minimizing the negative log-likelihood between the generated amino acid sequence and the native wild-type sequence. Sequences can then be designed by sampling or identifying sequences that maximize the conditional probabilities given the desired secondary and tertiary structures.

[0053] 1.3 Discrete Denoising Diffusion Probability Model

[0054] Diffusion models belong to the category of generative models, and their training phase includes diffusion and denoising processes. The diffusion process involves T steps. The original data x0~q(x) are transformed into a series of latent variables {x1, ..., x}. T Each latent variable carries progressively increasing noise. Conversely, the denoising process... The noise in these latent variables is gradually reduced, guiding them back to the original data distribution. This iterative denoising process is driven by differentiable operators, such as trainable neural networks.

[0055] Theoretically, q(x) t |x t-1 There is no strict form, but for effective sampling, p θ Several conditions need to be met:

[0056] (i) Diffusion nucleus q(x) t |x0) needs to have a closed form to sample noisy data for different time steps in parallel training;

[0057] (ii) The kernel should have a simplified form for calculating the posterior q(x) t |x0), therefore the posterior probability distribution p θ (x t-1 |x t )=∫q(x t-1 |x t ,x0)dp θ (x0|x t ) can be used as the target of a trainable neural network, where θ represents the parameters in the neural network. In this way, the original input x0 can be used as the target.

[0058] (iii) Marginal distribution q(x) T ) should be independent of x0, which allows the present invention to use q(x) in reasoning. T ) is used as a prior distribution.

[0059] The aforementioned criteria are crucial for developing suitable noise-adding modules and training processes. To meet these prerequisites, this invention follows the settings of previous research. For x t For K-class classification data ∈{1,...,K} (in a protein sequence generation task, K=20, representing 20 amino acids), the transition probabilities are calculated from the matrix [Q]. t ] ij =q(x t =j|x t-1 =i). Classification features x using transition matrix and one-hot encoding t The present invention can define the transfer nucleus in the diffusion process in the following ways:

[0060]

[0061] in It is the result of superimposing multiple transition matrices. The posterior distribution, derived from Bayes' rule, can be calculated in closed-form as follows: Therefore, the model output of the transfer kernel, time t, and process state x can be used.t The generation probability is determined by this process. Through iterative sampling, the generated output x0 is finally produced.

[0062] The final output probability distribution p(x) T The transition matrix should be independent of the observed value x0. Therefore, constructing the transition matrix requires using a noisy time table. The simplest and most commonly used method is uniform transition, which can be parameterized as follows: in It is the transpose of the identity matrix I. As t approaches infinity, α gradually decays until it reaches 0. Therefore, the distribution q(x) T It asymptotically tends to a uniform distribution, and thus is essentially independent of x.

[0063] Discrete graph probabilistic denoising diffusion model for protein sequence generation (e.g.) Figure 1 The diffusion noise addition process applies noise T times to the one-hot encoding vector of each amino acid node using a transition matrix, until the probability distribution of each amino acid approximates a uniform distribution. The denoising (reverse diffusion) process uses a trainable deep neural network to denoise the amino acid types T times, ultimately obtaining a joint probability distribution of a set of amino acid sequences used to generate the final sample. Conditional constraints are added during the denoising process to guide the recovery of the probability distribution, including the protein's secondary and tertiary structures.

[0064] This model utilizes a given graph The node features are X, and the edge features are E as conditions. Specifically, the node features include the location, type, spatial and biochemical properties of amino acids, X = [X...]. pos X aa X prop The present invention relates to amino acid characteristic X. aa The diffusion process is defined and denoised on a graph structure E encoded by an equivariant neural network. Furthermore, this invention incorporates protein-specific prior knowledge, including amino acid substitution scoring matrices and protein secondary structures. This invention also introduces a novel accelerated algorithm for the discrete diffusion generation process based on a transformation matrix.

[0065] 2.1 Discrete Probability Inverse Diffusion Process Based on Deep Neural Network Training

[0066] Diffusion Process: To capture the distribution of amino acid types, this invention independently adds noise to each amino acid node of the protein. For any given node, the transition probability is determined by matrix Q. t Definition. Using a predefined transition matrix, this invention defines the forward diffusion kernel as:

[0067] and in It is the transition probability matrix when the time step is t.

[0068] Training the denoising network: The second component of the diffusion model is the denoising neural network f θ The parameter is θ. The network accepts noisy input. Where X t It is a splicing of noisy amino acid types and other amino acid properties, including 20 uniquely encoded amino acid types and 15 protein backbone-related amino acid geometric properties, such as solvent-accessible surface area (SASA), normalized surface-aware node features, main chain atom dihedral angles, and three-dimensional positions. Its goal is to predict noise-free amino acid type X. aa This allows for the modeling of diverse amino acid sequences that potentially conform to protein structures while maintaining their intrinsic structural constraints. To train f... θ This invention optimizes the prediction probability of amino acid type for each node. The cross-entropy loss L.

[0069] Parameterized generation process: New amino acid sequences are generated iteratively through backdiffusion at each node x. Generation probability distribution. The probability of prediction is determined by a neural network. The estimation is performed. This invention marginalizes the network predictions to calculate the generation distribution for each iteration step:

[0070]

[0071]

[0072] Where the posterior distribution q(x) t-1 |x t x aa The transition matrix, the node characteristic states at time t, and the amino acid type x are used to determine the relationship between the transition matrix and the node characteristic states at time t. aa Calculated. x aa Probability prediction from denoising network The sampled samples.

[0073] 2.2 Conditional Backdiffusion Distribution Generation Guided by Prior Protein Knowledge

[0074] 2.2.1 Markov Transition Matrix

[0075] The transition matrix, serving as a guide in discrete diffusion models, plays a crucial role in providing the transition probabilities from the current time step to the next. Because it reflects the probability of transitioning from one amino acid type to another, this matrix is ​​vital in both the diffusion and generation processes. During the diffusion phase, the transition matrix is ​​repeatedly applied to the observed data, which evolves over time due to inherent noise. As the diffusion time increases, the probability of the original amino acid type gradually decays, eventually converging to a uniform distribution across all amino acid types. During the generation phase, the conditional probability p... θ (x t-1 |x t It is affected by the model prediction and characteristics of the transition matrix Q.

[0076] Given the biological specificity of amino acid substitutions, the transfer probabilities between amino acids are not uniformly distributed, making it unreasonable to define random directions during generation or sampling. Instead, the diffusion process can reflect evolutionary pressures using an amino acid substitution score matrix that preserves protein function, structure, or stability. In this study, the invention uses a Blocks Substitution Matrix (BLOSUM), which identifies conserved regions within a protein that are considered to have greater functional relevance. Based on empirical observations of protein evolution, BLOSUM provides an estimate of the probability of substitutions between different amino acids. Therefore, this invention incorporates BLOSUM into the diffusion and generation processes. First, the matrix is ​​normalized to probabilities using a softmax function. Then, the normalized matrix B is adjusted using different probability temperatures to control the noise level during the diffusion process. Therefore, the transfer matrix at time t is determined by Q. t =B T Given this matrix, by using it to improve the transition probabilities, the generation space that needs to be sampled can be effectively reduced, thus allowing the model's predictions to converge to a meaningful subspace. Figure 3 The transition matrices over time were compared for both random and BLOSUM cases.

[0077] 2.2.2 Secondary Structure

[0078] The secondary structure of a protein refers to the local spatial arrangement of amino acid residues within the protein chain. Two common types of protein secondary structures are α-helices and β-sheets, which are stabilized by hydrogen bonds between backbone atoms. Protein secondary structure acts as a crucial mediator, bridging the gap between the amino acid sequence and the overall three-dimensional conformation of the protein. In this invention, eight different types of secondary structures are incorporated as conditions during the sampling process of amino acid nodes. This strategy effectively reduces the exploration space of potential amino acid sequences. Specifically, this invention uses DSSP (Define Secondary Structure of Proteins) to predict the secondary structure of each amino acid and employs one-hot encoding to represent these structures. The neural network of this invention takes the one-hot encoding as input and utilizes it to denoise the amino acids, thereby using the secondary structure as a guiding condition for sampling the generated amino acid sequence.

[0079] By applying secondary structure conditions such as α-helices and β-sheets to the amino acid sequence search, the sampling space of potential sequences can be significantly reduced, and the resulting protein sequences can be endowed with biological meaning. By conditioning the amino acid sequences with corresponding secondary structure types during the sampling process, this invention guides the obtained protein sequences to obtain appropriate three-dimensional structures with feasible thermal stability, while maintaining their ability to perform their intended functions.

[0080] 2.3 Equivalent Graph Neural Networks Figure 4 )

[0081] Biomolecules such as proteins and compounds have three-dimensional structures. Regardless of how the input protein is localized and oriented, the model-predicted binding complexes should remain consistent in encoding robust and expressive hidden representations. This property can be guaranteed by using rotationally equivariant neural networks. A typical equivariant neural network is an equivariant graph neural network. This invention modifies its SE(3)-equivariant neural layer to update the representations of nodes and edges, thereby maintaining SO(3) rotational equivariance and E(3) translation invariance.

[0082] In the l-th layer, an isomorphic graph convolution (EGC) is input to a set of n hidden node embeddings. Used to describe AA type and geometric properties, as well as edge embedding. Regarding the connected nodes i and j, and their coordinates The goal of the modified EGC layer is to update the hidden representation H of the nodes. (l+1) The representation of edges M (l+1) In short, the present invention has H (l+1) M (l+1) =EGC[H (l) X pos M(l) ].

[0083] To achieve this goal, the EGC layer defines the following operations:

[0084] -For edge embedding:

[0085]

[0086] -For node coordinates:

[0087]

[0088] - For hidden representations of nodes:

[0089]

[0090] Here, φ e φ h and φ x It is an edge and node propagation operation, φ x It is an additional operation that embeds vector edges into m. ij Project onto a scalar. The modified EGC layer preserves the node coordinates X. pos It exhibits isomorphism in rotation and translation on the node set and maintains invariance in the arrangement of nodes, similar to other graph neural networks.

[0091] 2.4 Sampling and Acceleration of the Denoising Process Based on the Denoising Diffusion Implicit Model (DDIM)

[0092] A significant drawback of diffusion models is that the generation process is typically very slow because it involves many incremental steps. To address this issue, deterministic denoising implicit models (DDIM) are commonly used in continuous-variable diffusion generation models. DDIM is based on a non-Markovian forward diffusion process that is always conditioned on the input, not the previous step. By setting the noise variance of each step to 0, the backward generation process becomes completely deterministic, given an initial prior sample.

[0093] Similarly, since the present invention formally possesses a generation probability p θ (x t-1 |x t It is based on the predicted x aa and posterior distribution p(x) t-1 |x t x aa The calculation is obtained, and the present invention can also control p(x) to obtain p(x). aa |x t The sampling temperature makes the generative model deterministic. Therefore, this invention defines a multi-step generation process as follows:

[0094]

[0095] The temperature T controls whether it is deterministic or random, and the multi-step posterior distribution is:

[0096]

[0097] 3.1 Experimental Design

[0098] Training Setup: This invention employs the CATHv4.3.0 dataset partitioning based on GRAPHTRANS (John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph-based protein design. Advances in Neural Information Processing Systems, 32, 2019.) and GVP (Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In International Conference on Learning Representations, 2021.). Proteins are classified according to CATH topological classification, resulting in 18,024 proteins for training, 608 for validation, and 1,120 for testing. To evaluate the generation quality of different proteins, this invention tests its model on three different categories: short sequences, single-stranded proteins, and all proteins. Short sequences include proteins with a length less than 100; the single-stranded category contains proteins composed of a single strand. Furthermore, the total number of time steps for the diffusion model is configured to 500, following a cosine scheduling noise setting. For the denoising network, this invention implements six stacked EGNN blocks, each with 128 hidden dimensions. The model is trained for 200 epochs by default, using the Adam optimizer. During training, this invention uses a batch size of 64 and a learning rate of 0.0005. Additionally, to prevent overfitting, this invention introduces a dropout rate of 0.1 in the model architecture.

[0099] Evaluation Metrics: This invention measures the quality of the recovered protein sequence using perplexity and recovery rate. Perplexity measures how well the amino acid probabilities predicted by the model match the actual amino acids at each position in the sequence. Lower perplexity indicates a better fit between the model and the data. Recovery rate assesses the model's ability to recover the correct amino acid sequence from the protein's three-dimensional structure. Typically, it is calculated as the proportion of amino acids in the predicted sequence that match the original sequence. A higher recovery rate indicates a stronger ability of the model to predict the original sequence from the structure.

[0100] Table 1. Performance of GRADE-IF in restoring protein in CATH

[0101]

[0102]

[0103] Table 2. Performance of GRADE-IF in restoring protein in TS50 and TS500

[0104]

[0105] 3.2 Generation of New Sequences

[0106] Table 1 compares the performance of GRADE-IF in recovering proteins in CATH. To generate high-confidence sequences, GRADE-IF uses approximate probability... This eliminates prior uncertainties. Notably, this invention found that the recovery rates for single-chain proteins and short sequences improved by 4.2% and 5.4%, respectively. This invention was also evaluated on the TS50 and T500 datasets, and the results are shown in Table 2.

[0107] After further subdividing the recovery performance into buried and surface amino acids, this invention found that more conserved core residues exhibited higher native sequence recovery rates. Conversely, amino acids on the active surface showed lower sequence recovery rates. Figure 5 Amino acid conservation was examined by solvent-accessible surface area (SASA) (where SASA < 0.25 indicates an internal amino acid) and contact number (the number of amino acids adjacent to it in 3D space)

[10] . In all three types of protein sequences, the recovery rate of internal residues was significantly higher than that of external residues, and the recovery rate increased with the increase of contact number. The present invention also presents recovery rates for different secondary structures, achieving high recovery rates in most secondary structures, with only one rare 5-helix structure showing a low recovery rate.

[0108] This invention also includes Figure 6This paper compares the diversity of GRADE-IF with PIFOLD (Zhangyang Gao, Cheng Tan, and Stan Z. Li. Pifold: Toward effective and efficient protein inverse folding. In International Conference on Learning Representations, 2023.) and PROTEINMPNN (Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J. Ragotte, Lukas F. Milles, Basile I. M. Wicky, Alexis Courbet, Rob J. de Haas, Neville Bethel, et al. Robust deep learning–based protein sequence design using proteinmpnn. Science, 378(6615):49–56, 2022.). For a given backbone, this invention generates 100 sequences with self-similarity less than 50% and projects them into a two-dimensional space using t-SNE. At the same level of diversity, GRADE-IF includes wild-type sequences, while the other two methods fail to include wild-type sequences within their sample regions. Furthermore, for this protein with a recovery rate threshold of 45%, GRADE-IF can generate a large number of samples, while the other two methods become deterministic. This further confirms the superiority of the model of this invention in achieving sequence diversity and high recovery rates.

[0109] This invention also evaluated the speed-boosting sampling algorithm on this dataset, such as... Figure 7 As shown. Using DDIM, this invention can skip k steps in the sampling phase. This invention selected a series of step sizes and evaluated its performance based on the recovery rate and the time required to sample 1200 sequences. As the step size increases, the recovery rate decreases slightly, reaching 48.13% at a step size of 100. However, at a step size of 100, the sampling speed is 100 times faster than at a step size of 1, demonstrating a significant speedup.

[0110] 3.3 Folding prediction of generated sequences

[0111] This invention extends the research to the foldability of sequences generated at different sequence recovery rates. Figure 8The crystal structure of a native protein (PDB ID: 3FKF) was compared with the structures folded from three sequences generated by GRADE-IF, each derived from a different GRADE-IF sequence. The crystal structure resolution was [resolution missing]. This indicates that the folding structure of all generated sequences is almost identical to that of the original sequence, with a root mean square deviation of approximately [value missing] at 139 residues. The average pLDDT score was 0.835, compared to 0.91 for the native protein, indicating that their folding structure is reliable. Figure 7 The evidence presented in the present invention demonstrates superiority in generating more similar results, and the present invention is confident that GRADE-IF can generate biologically plausible new sequences for a given protein structure.

[0112] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A discrete graph probabilistic denoising diffusion model for protein sequence generation, characterized in that, The given protein backbone guides the corresponding amino acid residue types, specifically including: (1) Discrete probability inverse diffusion process based on deep neural network training; (2) Conditional reverse diffusion distribution generation guided by prior knowledge of proteins; (3) Equivalent graph neural network, used for parameterized denoising process of reverse diffusion; (4) The denoising process is sampled and accelerated based on the denoising diffusion implicit model; The model utilizes a given graph G = {X, E}, where node features are X and edge features are E, as conditions; where node features X = [X... pos X aa X prop The position X containing the amino acid pos Amino acid type X aa And spatial and biochemical properties X prop ; Step (1) specifically includes the diffusion process, training the denoising network, and the parameterization generation process; In step (1), the specific method of the diffusion process is as follows: noise is independently added to each amino acid node of the protein, and for any given node, the transition probability is given by matrix Q. t The forward diffusion kernel is defined using a predefined transition matrix as follows: in It is the transition probability matrix when the time step is t; In step (1), the specific method for training the denoising network is as follows: construct a denoising neural network f θ The parameter is θ; the network accepts noisy input. Where X t It is a concatenation of noisy amino acid types and other amino acid attributes, including 20 one-hot encoded amino acid types and 15 geometric attributes related to the protein backbone; the training objective of the denoising model is to predict the noise-free amino acid type X. aa This allows for the modeling of multiple sequence combinations that may correspond to the protein structure while maintaining its intrinsic structural constraints. In step (1), the specific method for parameterization generation is as follows: the new amino acid sequence is generated by adjusting the amino acid type x of each node at time t. aa Perform back-diffusion iteration to generate the corresponding probability distribution p. θ (x t-1 |x t The predicted probability is obtained by a neural network. The estimation is performed; a deep neural network is used to predict probabilistic marginalization to calculate the generation distribution for each iteration step: in Where the posterior distribution q(x) t-1 |x t x aa The transition matrix and the node features at time t are given by x. t and amino acid type x aa The calculation shows that x aa Probability predicted by the denoising network Obtained by sampling; Step (2) specifically includes two parts: the Markov transition matrix and secondary structure sampling. The specific method for the Markov transition matrix is ​​as follows: first, the block substitution matrix is ​​incorporated into the diffusion and generation process, normalized to a probability, and then the softmax function is used; then, different probability temperatures are used to adjust the normalized block substitution matrix B to control the noise scale during the diffusion process; the transition matrix Q at time t... t By Q t =B T The method for secondary structure sampling is as follows: DSSP is used to predict the secondary structure of each amino acid, and one-hot encoding is used to represent these structures; the neural network takes the one-hot encoding as input and uses it to denoise the amino acids, thereby incorporating the protein secondary structure as a guiding condition in the denoising sampling process; The specific method for step (3) is as follows: For a protein graph with n amino acids, at the l-th layer, an isomorphic graph convolution yields the node expression. Recessive vector representations of amino acid types and geometric properties, and edge representations. This represents the hidden features on connected nodes i and j, and the three-dimensional coordinates of the nodes. The modified EGC layer can be used to update the hidden layer representation H of the (l+1)th layer node. (l+1) Hidden layer representation M of edges (l+1) That is, H (l+1) M (l+1) =EGC[H (l) X pos M (l) ]; The specific method for step (4) is as follows: The temperature T is used to adjust the degree of determinism or randomness of the estimated probability distribution; the posterior distribution after multi-step denoising is:

Images