Method and system for protein three-dimensional structure prediction

Abstract

The application provides a protein three-dimensional structure prediction method and system, comprising the following steps: S1: obtaining a multiple sequence alignment matrix containing protein coevolution information and a protein template, and performing conversion of MSA sequence and residue pair coding to obtain MSA sequence coding and residue pair coding; S2: the MSA sequence coding and the residue pair coding are updated by a recurrent attention neural network to generate the latest MSA sequence coding and the residue pair coding; S3: based on the current residue pair coding, target sequence coding and starting main chain framework T i The current target sequence coding is updated by the invariant point attention neural network; and S4: the main chain framework is updated based on the current target sequence coding, and the torsion angle between amino acids is calculated, so that the protein main chain structure is obtained, and the side chain atom angle and the final three-dimensional structure are calculated through the residue network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of protein three-dimensional structure prediction, in particular, to a protein three-dimensional structure prediction method and system, and more particularly to a protein three-dimensional structure prediction method and system based on bidirectional optimization and invariant point attention model. BACKGROUND

[0002] Proteins are the main bearers of life activities, and many important life processes in living organisms involve proteins. The three-dimensional structure of a protein determines its function. Predicting the three-dimensional structure of a protein from its amino acid sequence is a fundamental but unsolved problem in bioinformatics. Protein structure prediction not only helps to enhance human understanding of protein folding mechanisms. Furthermore, structure prediction is of fundamental significance for new protein design--to design a new protein with a certain specific function / structure, structure prediction is undoubtedly a powerful tool to shorten the design process. Therefore, there is an urgent need for more effective methods to fill the gap in traditional protein structure prediction methods.

[0003] There are two main categories of methods for directly predicting protein three-dimensional structures from sequences: one is template-based modeling, and the other is ab initio modeling. Due to the limitations of the number and quality of available structure templates in protein structure databases, ab initio prediction has become increasingly important. Tools for ab initio prediction based on protein sequence fragments, such as Rosetta and QUARK, have achieved certain results in protein structure prediction, but their scope of application is very limited, and they still cannot accurately predict the structures of most proteins with high throughput.

[0004] In the past two years, ab initio prediction methods have benefited from the latest deep learning technology to predict protein tertiary structures, achieving unprecedented improvements. Deep learning-based protein structure prediction mainly predicts the distance and angle between amino acids in protein structures, thereby reconstructing the structure.

[0005] Currently competitive deep learning protein three-dimensional structure prediction models include:

[0006] The method and system for single sequence protein structure prediction based on deep learning proposed by Beijing Molecular Heart Technology Co., Ltd. also uses attention model to extract information between residues / protein sequences from protein sequences, and extracts turn / distance prediction three-dimensional structure sites, and the process mainly includes three steps: step 1: based on pre-trained ESM-1b, ESM-1v, ProtTrans and other large language models, obtain the initial code of protein residue coding and residue pair; step 2: use a similar model module to update the sequence coding and amino acid pair coding iteratively; step 3: extract the distance / torsion angle information of the residues from the updated sequence coding and amino acid pair coding information through the structure generation module, and predict the final three-dimensional structure; the main difference between the process in step 1 and our model is that the matrix generated by multiple sequence alignment is not used as input, so the site interaction information existing in protein evolution is missing, and if this information is obtained only through single sequence information, it depends on the protein sequence correlation contained in a training batch, which is relatively random, and it is difficult to learn the accurate relationship between protein residues. The correlation between residues is crucial to predict the three-dimensional structure of the sequence folding; at the same time, the learning iteration process of sequence coding and amino acid pair coding in step 2 of the process does not use column-based self-attention layer, and the column-based self-attention layer is conducive to learning the true action information between amino acid pairs, so it is difficult to obtain accurate residue coding in advance before the triangular update module, which will also affect the subsequent judgment of the distance and torsion angle of the residues.

[0007] Patent document CN114530194A (application number: 202210185769.X) discloses a protein structure prediction method based on multi-residue distance map constraints. The score function of the inter-residue distance is calculated by constructing a protein three-dimensional structure prediction based on a multi-model, and the structure generated by Rosetta multiple times is selected as the final conformation. The process mainly consists of three parts: step 1: three protein inter-residue distance prediction modules, trRosetta, RaptorX, and LOMETS, are used to calculate three specific protein inter-residue distance files, thereby constructing a score function based on inter-residue distance, and the 3d site is based on the protein; step 2: the Rosetta model is cycled to generate an initial population of protein conformations, and more conformations are generated for testing through cross-variation; step 3: the three-dimensional conformation that best meets the standard is found by clustering the model to reduce the population size in step 2, and the conformation is updated according to the score function in step 1 to obtain the optimal protein structure closest to the natural three-dimensional structure; the patent contains more process systems built by applying other mature algorithms. The underlying model used by the patent, such as trRosetta, also uses a self-attention model to calculate the three-dimensional structure of the protein by using multiple sequence alignment and iteratively optimizing the distance between protein residues, but as described in step 2, the generation of a large number of protein structures requires a huge amount of computing power, which is not suitable for a relatively lightweight and expandable high-throughput machine learning model. At the same time, as described in step 1, the inter-residue distance score function is based on pure mathematical deduction, which is more linear than machine learning models, making it difficult to accurately find local optimal solutions. The optimal distance obtained by the function based on multiple model distances may not satisfy the triangle inequality property after some degree of superposition and averaging, so the three-dimensional structure calculated may not accurately simulate the protein folding conformation in nature. SUMMARY

[0008] In view of the defects in the prior art, the purpose of the present application is to provide a protein three-dimensional structure prediction method and system.

[0009] According to the protein three-dimensional structure prediction method provided by the present application, the following steps are included:

[0010] Step S1: Obtain a multiple sequence alignment matrix containing protein coevolution information and a protein template, and perform MSA sequence and residue pair coding conversion to obtain MSA sequence coding and residue pair coding;

[0011] Step S2: The MSA sequence coding and residue pair coding are cross-iterated and updated by a recurrent attention neural network to generate the latest MSA sequence coding and residue pair coding;

[0012] Step S3: The current residue pair coding and the target sequence coding are updated by an invariance point attention neural network to update the current target sequence coding.

[0013] Step S4: updating the main chain framework based on the interaction information between the residues contained in the current target sequence encoding, and calculating the torsion angle between the residues and atoms, thereby predicting the protein structure;

[0014] The multiple sequence alignment matrix containing protein co-evolution information is based on the premise of evolutionary events including mutations, insertions, deletions, and recombination, aligning corresponding residues in multiple amino acid sequences and comparing them column by column, thereby constructing a data set similar to the predicted protein sequence;

[0015] The protein template is based on known structural proteins similar to the target protein in the protein structure database;

[0016] The recurrent attention neural network is updated by using a multi-head neural network to update the sequence edges and residue edges respectively and mutually, and using a triangular update mechanism to update the residue pair encoding.

[0017] Preferably, the residue pair encoding includes: one-hot encoding of the distance between the beta carbon atoms of the residues, the point positions of the alpha carbon atoms around each residue, and the residue position mask information.

[0018] Preferably, the multi-head neural network in the recurrent attention neural network adopts: using a multi-head neural network to update the sequence edges and residue edges respectively and mutually, and obtaining a residue edge embedding vector matrix and a sequence edge embedding vector matrix after updating;

[0019] The updated residue edge embedding vector matrix:

[0020] E' = f(E + W)

[0021] Where E' represents the updated residue edge embedding vector matrix; f represents the activation function; W represents the weight matrix of the multiple sequence alignment matrix M;

[0022]

[0023] Where D represents the depth of the protein multiple sequence alignment matrix; W ij represents the weight between residue i and residue j, represents the encoding vector of residue i on the dth sequence in M, represents the encoding vector of residue j on the dth sequence in M, represents the cross product of two vectors, and linear is a dimension reduction linear layer;

[0024] The updated sequence edge embedding vector matrix:

[0025] M' = a(M)

[0026] wherein M' represents the updated sequence edge embedding matrix, and a represents a weight matrix based on the residue edge embedding matrix E:

[0027] a ij = F(E ij )

[0028] wherein F represents an activation function;

[0029] The updating of the residue pair encoding by using the triangle updating mechanism adopts: updating the residue pair encoding by using the adjacent residue edge pair to the residue pair, so as to ensure that the encoding satisfies the triangle inequality characteristic of the 3D space;

[0030] The triangle updating mechanism includes a triangle product updating mechanism and a triangle self-attention updating mechanism; wherein the triangle product updating mechanism updates the residue edge by using the adjacent edge information of the triangle formed by the residue edge pair, and includes a product layer based on the outward adjacent edge and a product layer based on the inward adjacent edge;

[0031] The triangle self-attention updating mechanism updates the residue edge by using the adjacent edge information of the starting point / ending point of the residue edge pair, and includes a triangle attention layer based on the starting point and a triangle attention layer based on the ending point;

[0032] The product layer based on the outward adjacent edge and the product layer based on the inward adjacent edge respectively adopt:

[0033] E ij = g(E ij )⊙f(Σ k g(E ik )⊙g(E jk ));

[0034] E ij = g(E ij )⊙f(∑ k g(E ki )⊙g(E kj ));

[0035] wherein E ij represents the encoding of the residue edge pair (i,j), g represents a superposition function of a linear layer and an activation function, and f represents a superposition function of layer normalization and a linear layer;

[0036] The triangle attention layer based on the starting point and the triangle attention layer based on the ending point respectively adopt:

[0037] E ij = trans(att(Q ij ,K ik ,V ik ));

[0038] E ij = trans(att(Q ij ,K kj ,V kj ))

[0039] where Q ij ,K ij ,V ij are query, key and value vectors derived from E ij for self-attention computation, att is a self-attention update model, and trans is a transition layer containing regularization and activation function.

[0040] Preferably, the invariant point attention neural network in step S3 adopts: the target sequence encoding calculates the query, key and value vectors of the encoding space and the 3D space corresponding to each amino acid via a linear layer, respectively, and then calculates the attention matrix of the encoding space and the attention matrix of the 3D space based on the global invariant transformation of the starting main chain framework according to the attention mechanism, and introduces the current residue pair encoding as the bias value of the attention matrix to obtain the final attention matrix, and iteratively updates the target sequence encoding through the attention matrix.

[0041] Preferably, the iteratively updating the target sequence encoding through the attention matrix adopts:

[0042]

[0043]

[0044]

[0045]

[0046] where m i represents the encoding of the i-th residue in the target sequence encoding, A ij represents the attention between residues i and j in the attention matrix, E ij represents the encoding of the residue pair edge (i, j), v j represents the self-attention value of m i corresponding to the encoding space, represents the self-attention value of m j corresponding to the 3D space, represents the weighted output of the residue pair encoding through the self-attention, o i represents the weighted output of the encoding space value, represents the weighted output of the 3D space, T i represents the main chain framework of residue i, and Linear represents a linear layer.

[0047] Preferably, the initial main chain framework T i Employing:

[0048] The main chain framework is initialized as the identity transformation:

[0049]

[0050] All amino acids are placed on the origin of the global coordinate.

[0051] Preferably, the updating the main chain framework based on the current target sequence encoding employs:

[0052]

[0053]

[0054]

[0055]

[0056] where m i represents the encoding of the i-th residue in the updated target sequence encoding, a i , b i , c i , d i are important components for calculating the rotation matrix R i in the main chain framework T i , is the deviation coordinate in the main chain framework T i .

[0057] According to the protein three-dimensional structure prediction system provided by the application, comprising:

[0058] Module M1: obtaining a multiple sequence alignment matrix containing protein coevolution information and a protein template, and performing MSA sequence and residue pair encoding conversion to obtain MSA sequence encoding and residue pair encoding;

[0059] Module M2: the MSA sequence encoding and the residue pair encoding are cross-iteratively updated by a recurrent attention neural network to generate the latest MSA sequence encoding and the residue pair encoding;

[0060] Module M3: the current residue pair encoding and the target sequence encoding are updated by an invariance point attention neural network to the current target sequence encoding;

[0061] Module M4: updating the main chain framework based on the interaction information between the residues in the current target sequence encoding, and calculating the torsion angle between the residues and the atoms, thereby predicting the protein structure;

[0062] The multiple sequence alignment matrix containing protein co-evolution information is based on the premise of evolutionary events including mutations, insertions, deletions, and recombination, aligning corresponding residues in multiple amino acid sequences and comparing them column by column, thereby constructing a data set similar to the predicted protein sequence;

[0063] The protein template is based on known structural proteins similar to the target protein in the protein structure database;

[0064] The cyclic attention neural network uses a multi-head neural network to update sequence edges and residue edges respectively and mutually, and uses a triangular update mechanism to update residue pair encoding.

[0065] Preferably, the multi-head neural network in the cyclic attention neural network uses: a multi-head neural network to update sequence edges and residue edges respectively and mutually, and obtains a residue edge embedding vector matrix and a sequence edge embedding vector matrix after updating;

[0066] The updated residue edge embedding vector matrix:

[0067] E' = f(E + W)

[0068] Where E' represents the updated residue edge embedding vector matrix; f represents an activation function; and W represents a weight matrix of the multiple sequence alignment matrix M;

[0069]

[0070] Where D represents the depth of the protein multiple sequence alignment matrix; W ij represents the weight between residue i and residue j, represents the encoding vector of residue i on the dth sequence in M, represents the encoding vector of residue j on the dth sequence in M, represents the cross product of two vectors, and linear is a dimension reduction linear layer;

[0071] The updated sequence edge embedding vector matrix:

[0072] M' = a(M)

[0073] Where M' represents the updated sequence edge embedding vector matrix, and a represents a weight matrix based on the residue edge embedding vector matrix E:

[0074] a ij = F(E ij )

[0075] Where F represents an activation function;

[0076] The updating of the residue pair encoding by using the triangle updating mechanism adopts: updating the residue pair encoding by taking the adjacent residue edges of the residue pair as the edges of the residue pair to ensure that the encoding satisfies the triangle inequality characteristics of the 3D space;

[0077] The triangle updating mechanism includes a triangle product updating mechanism and a triangle self-attention updating mechanism; wherein the triangle product updating mechanism updates the residue edge by using the adjacent edge information of the triangle formed by the residue edge, and includes a product layer based on the outward adjacent edge and a product layer based on the inward adjacent edge;

[0078] The triangle self-attention updating mechanism updates the residue edge by using the adjacent edge information of the starting point / ending point of the residue edge, and includes a triangle attention layer based on the starting point and a triangle attention layer based on the ending point;

[0079] The product layer based on the outward adjacent edge and the product layer based on the inward adjacent edge respectively adopt:

[0080] E ij =g(E ij )⊙f(Σ k g(E ik )⊙g(E jk ));

[0081] E ij =g(E ij )⊙f(∑ k g(E ki )⊙g(E kj ));

[0082] Wherein, E ij represents the encoding of the residue edge (i,j), g represents the superposition function of the linear layer and the activation function, and f represents the superposition function of the layer normalization and the linear layer;

[0083] The triangle attention layer based on the starting point and the triangle attention layer based on the ending point respectively adopt:

[0084] E ij =trans(att(Q ij ,K ik ,V ik ));

[0085] E ij =trans(att(Q ij ,K kj ,V kj ))

[0086] Wherein, Q ij ,K ij ,V ij are E ijThe derived query, key and value vectors for self-attention calculation, att is a self-attention update model, trans is an overlayer containing regularization and activation function;

[0087] The invariant point attention neural network in the module M3 adopts: the target sequence encoding calculates the query, key and value vectors of each amino acid corresponding to the encoding space and 3D space through linear layers respectively, then calculates the attention matrix of the encoding space and the attention matrix of the 3D space based on the global invariant transformation of the starting main chain framework according to the attention mechanism, and introduces the current residue pair encoding as the bias value of the attention matrix to obtain the final attention matrix, and iteratively updates the target sequence encoding through the attention matrix;

[0088] The iteratively updating the target sequence encoding through the attention matrix adopts:

[0089]

[0090]

[0091]

[0092]

[0093] Wherein, m i represents the encoding of the i-th residue in the target sequence encoding, A ij represents the attention between residues i and j in the attention matrix, E ij represents the encoding of the residue pair edge (i, j), v j represents m i corresponding to the self-attention value of the encoding space, represents m j corresponding to the self-attention value of the 3D space, represents the weighted output of the residue pair encoding through the self-attention, o i represents the weighted output of the encoding space value, represents the weighted output of the 3D space, T i represents the main chain framework of residue i, and Linear represents a linear layer.

[0094] The starting main chain framework T i adopts:

[0095] The main chain framework is initialized as an identity transformation:

[0096]

[0097] All amino acids are placed on the origin of the global coordinate.

[0098] Preferably, the updating main chain framework based on the current target sequence encoding adopts:

[0099]

[0100]

[0101]

[0102] wherein m i represents the encoding of the i-th residue in the updated target sequence encoding, a i , b i , c i , d i are important components for calculating the rotation matrix R i in the main chain framework T i , is the deviation coordinate in the main chain framework T i .

[0103] Compared with the prior art, the present application has the following beneficial effects:

[0104] 1. The present application enables the network to more evenly extract globalized information, thereby more accurately predicting the overall three-dimensional structure of the protein;

[0105] 2. The present application solves the problem of whether the neural network can more comprehensively extract the hidden information required for predicting the distance and angle by directly inputting the multiple sequence alignment matrix containing the protein co-evolution information as input;

[0106] 3. The present application improves the accuracy of angle prediction and three-dimensional structure site by inputting the potential template with rich information searched from the known protein structure library (Protein Data Bank) according to the amino acid sequence of the target protein as the input feature of the neural network, letting the network learn various input features in the training mode, and combining with the globally invariant point attention network, and finally achieves the effect of pLDDT, RMSD, TM-Score and other parameters of the predicted structure being equal to or superior to the optimal model of the same type;

[0107] 4. The present application enables the network to more evenly extract globalized information, improves the accuracy of protein residue distance and angle prediction, and thereby more accurately predicts the overall three-dimensional structure of the protein. BRIEF DESCRIPTION OF DRAWINGS

[0108] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the following drawings:

[0109] Figure 1 A schematic diagram of a protein three-dimensional structure prediction system. DETAILED DESCRIPTION

[0110] The application will be described in detail below with specific examples. The following examples will help those skilled in the art to further understand the application, but in no way limit the application. It should be noted that for those skilled in the art, without departing from the concept of the application, a number of changes and improvements can be made. These are within the scope of the application.

[0111] The application provides a protein three-dimensional structure prediction method and system based on multi-sequence alignment and residue pair bidirectional optimization and using an invariant point attention model, which automatically and accurately generates structure prediction from protein sequence and evolutionary information, including: converting a multi-sequence alignment matrix containing protein co-evolution information and a protein template into sequence and residue pair coding, and directly inputting the coding into a model; then bidirectionally updating sequence and residue information through a recurrent attention neural network, and simulating the interaction between residues by means of a triangular update mechanism; finally, the updated protein information is extracted from the hidden global invariant inter-residue angle information by a novel structure prediction model, and a specific three-dimensional structure is generated. The application can enable the network to more evenly extract global information, improve the accuracy of protein inter-residue distance and angle prediction, and thus more accurately predict the overall three-dimensional structure of the protein.

[0112] Example 1

[0113] According to the protein three-dimensional structure prediction method provided by the application, as shown in the formula (1), the method comprises the following steps: Figure 1

[0114] Data coding input step: obtaining a multi-sequence alignment matrix containing protein co-evolution information and a protein template, and converting MSA sequence and residue pair coding, and inputting the coding into a multi-head neural network based on an attention mechanism;

[0115] Recurrent attention neural network protein coding update step: each cycle of the recurrent attention neural network first optimizes and updates the latest MSA sequence coding and residue pair coding generated on the sequence side and the residue side respectively and mutually, and then iterates the residue pair coding by means of a triangular update mechanism. The updated MSA sequence coding and residue pair coding will participate in the cycle and enter a new round of recurrent attention neural network iterative optimization;

[0116] ​Invariant point attention network calculates distance and angle matrix step: the encoding of the residue pair optimized by the evolutionary neural network and the target sequence encoding enter the invariant point attention neural network, which iterates the target sequence encoding according to the attention mechanism between the residues and the starting main chain framework in the 3D space, and ensures that the local 3D information contained is invariant to the global Euclidean transformation;

[0117] 3D structure prediction step: update the main chain framework according to the output target sequence encoding, and calculate the torsion angle between amino acids to obtain the structure of the protein main chain; then calculate the side chain atom angle and the final three-dimensional structure through a light residue network, and calculate the confidence score through a small auxiliary neural network, which can be used to evaluate the reliability of the predicted protein 3D structure.

[0118] Specifically, when constructing the MSA sequence matrix, HHblits and JackHMMER are used to perform step-by-step searches from each metagenomic protein sequence database, and the results obtained are effectively combined to obtain the deepest multiple sequence alignment matrix. Unlike the method mentioned in the background art of constructing artificial features as input according to the multiple sequence alignment matrix, the multiple sequence alignment matrix of the protein is directly encoded and converted into an embedding vector as the input of the neural network.

[0119] Specifically, the data encoding input step includes: the multiple sequence alignment matrix containing protein coevolution information includes: considering evolutionary events such as mutations, insertions, deletions, and recombination, the corresponding residues in multiple amino acid sequences are aligned as much as possible and compared column by column, thereby constructing a data set similar to the predicted protein sequence in the database;

[0120] Specifically, the data encoding input step includes: the protein template includes: a known structure protein in the protein structure database similar to the target protein, the searched protein template information such as amino acid sequence, distance matrix, etc. are all converted into embedding vectors as the input of the neural network;

[0121] Specifically, the data encoding input step includes: the residue pair encoding includes one-hot encoding of the distance between the beta carbon atoms of the residues, the point position of the alpha carbon atoms around each residue, and the residue site mask and other information;

[0122] Specifically, the cyclic attention neural network protein encoding update step includes that the cyclic attention neural network includes: 1) a multi-head neural network is used to update the sequence edges and the residue edges respectively and mutually; 2) a triangular update mechanism; wherein:

[0123] The sequence side and the residue side are updated respectively and mutually, including: in the sequence side, the network of attention mechanism can better distinguish the sequence containing sequence and non-relevant information contained in the input protein multiple sequence alignment matrix embedded vector matrix M, the updated M of attention operation will be used to update the embedded vector matrix E of the residue side:

[0124] E' = f(E + W)

[0125] Wherein, E' is the updated embedded vector matrix of the residue side, f is the activation function, and the current activation function is RELU; W is the weight matrix of the base M:

[0126]

[0127] Wherein, D is the depth of the protein multiple sequence alignment matrix; W ij represents the weight between residue i and residue j, represents the compiled vector of residue i on the dth sequence in M, represents the compiled vector of residue j on the dth sequence in M;

[0128] In the residue side, the network of attention mechanism can better extract the interaction relationship between each pair of amino acid residues, and the updated E of attention operation will be used to update the embedded vector matrix M of the sequence side:

[0129] M' = a(M)

[0130] Wherein, M' is the updated embedded vector matrix of the sequence side, and a is the weight matrix based on E:

[0131] a ij = F(E ij )

[0132] Wherein, F is the activation function; ij generally represents the corresponding numerical value / vector of the ith row and jth column of a certain matrix;

[0133] Specifically, the updating step of the recurrent attention neural network protein coding includes: the triangular updating mechanism updates the coding of the residue pair adjacent to the residue side pair, which ensures the triangular inequality property of the hidden 3D distance between residues, and the updating mechanism layer includes a product layer based on outward adjacent edges, a product layer based on inward adjacent edges, a triangular attention layer based on start site and a triangular attention layer based on end site.

[0134] Wherein, the updating methods of the product layers based on outward and inward adjacent edges are respectively:

[0135] E ij = g(E ij ) o f (Σ k g(E ik ) o g(Ejk ));

[0136] E ij =g(E ij )⊙f(∑ k g(E ki )⊙g(E kj ));

[0137] Among them, E ij Let f be the compilation vector of the edge (residue i, residue j), and g be the activation functions, respectively; the activation function g of the product layer based on the outward and inward border edges is sigmoid, and f is the superposition of LayerNorm and Linear;

[0138] The update methods for the triangular attention layer based on the initiation and termination sites are as follows:

[0139] E ij =trans(att(Q) ij ,K ik V ik ));

[0140] E ij =trans(att(Q) ij ,K kj V kj ))

[0141] Among them, Q ij ,K ij V ij E respectively ij The derived query, key, and value vectors are: att is a self-attention update model similar to that in a transformer encoder; trans is a transition layer containing regularization and activation functions; the activation function in the transition layer based on the triangular attention layer with start and end points is ReLU.

[0142] Specifically, the step of calculating the distance and angle matrix using the invariant point attention network includes: the input of the invariant attention network is the target sequence encoding (the first row of MSA sequence encoding) and the residue pair encoding E. ij , and the initial main chain framework T i The attention network encodes {m} according to the target sequence. i initial The query, key, and value vectors for each amino acid in the coding space and 3D space are calculated using a linear layer. Then, the attention matrix in the coding space is calculated based on the attention mechanism and the starting main chain framework T. i Attention matrix in 3D space for global invariant transformation, and E ijThe bias value is introduced as the attention matrix, and the final attention matrix A is obtained. Finally, the target sequence encoding is updated iteratively through A.

[0143] wherein m i initial The encoding of the i-th amino acid in the target sequence;

[0144] wherein m i The query, key, and value values corresponding to the encoding space are q i , k i , vi; m i The query, key, and value values corresponding to the 3D space are:

[0145] The way in which A iteratively updates the target sequence encoding is:

[0146]

[0147]

[0148]

[0149]

[0150] Specifically, the invariant point attention network calculates the distance and angle matrix step includes: the main chain framework is initialized as an identity transformation:

[0151]

[0152] wherein I represents the feature matrix; represents the origin vector; that is, all amino acids are placed on the origin of the global coordinates;

[0153] wherein Ti can be represented as:

[0154]

[0155] wherein the recurrent attention neural network protein encoding update step and the invariant point attention network calculate the distance and angle matrix step both use multi-head attention network (6-8 heads), and use residual connection to facilitate network optimization.

[0156] In the invariant point attention network calculates the distance and angle matrix step, a multi-head neural network is used to update attention, Nquery points=4, and a multi-layer invariant point attention neural network is used to update the target sequence encoding.

[0157] Specifically, the 3D structure prediction step includes that the way in which the main chain framework is updated is:

[0158]

[0159]

[0160]

[0161]

[0162] Specifically, the 3D structure prediction step includes that the torsion angle between the amino acids is obtained by updating the m i With m i initial Linear superposition, and multiple times into RELU-linear-RELU-linear layer update, the obtained angle contains 3 main chain atom torsion angle and 4 side chain atom torsion angle. Note that the torsion angle is the only quantity free to learn in the model, the angle and length of the chemical bond are all kept in the preset state.

[0163] Specifically, the 3D structure prediction step includes that the final three-dimensional structure is obtained by superimposing the torsion matrix transformation calculated from the torsion angle on the basis of the main chain framework.

[0164] According to the protein three-dimensional structure prediction system provided by the application, the protein three-dimensional structure prediction system comprises:

[0165] The data encoding input module acquires the multiple sequence alignment matrix containing protein coevolution information and the protein template, and performs conversion of MSA sequence and residue pair coding, and inputs the coding into the multi-head neural network based on the attention mechanism.

[0166] The cyclic attention neural network protein coding update module: each cycle of the cyclic attention neural network first generates the latest MSA sequence coding and residue pair coding by optimizing and updating the sequence edge and the residue edge respectively and mutually, and then iterates the residue pair coding by means of the triangular update mechanism. The updated MSA sequence coding and residue pair coding will participate in the cycle and enter a new round of cyclic attention neural network iteration optimization.

[0167] The invariant point attention network distance and angle matrix calculation module: the residue pair coding and the target sequence coding after the evolutionary neural network iteration optimization enter the invariant point attention neural network, the attention network iterates the target sequence coding according to the attention mechanism between the residues and the starting main chain framework in the 3D space, and ensures that the local 3D information contained is invariant to the global Euclidean transformation.

[0168] 3D structure prediction module: update the main chain framework according to the output target sequence encoding, and calculate the torsion angle between amino acids to obtain the structure of the protein main chain; then calculate the side chain atom angle and the final three-dimensional structure through the light residual network, and calculate the confidence score through the small auxiliary neural network, which can be used to evaluate the reliability of the predicted protein 3D structure.

[0169] Specifically, when constructing the MSA sequence matrix, HHblits and JackHMMER are used to perform step-by-step searches from each metagenomic protein sequence database, and the resulting results are effectively combined to obtain the deepest multiple sequence alignment matrix. Unlike the method mentioned in the background art of constructing artificial features as input according to the multiple sequence alignment matrix, the multiple sequence alignment matrix of the protein is directly encoded and converted into an embedding vector as the input of the neural network.

[0170] Specifically, the data encoding input module includes: the multiple sequence alignment matrix containing protein coevolution information includes: considering evolutionary events such as mutations, insertions, deletions, and recombination, the corresponding residues in multiple amino acid sequences are aligned as much as possible and compared column by column, thereby constructing a data set similar to the predicted protein sequence in the database;

[0171] Specifically, the data encoding input module includes: the protein template includes: a known structure protein similar to the target protein in the protein structure database, the searched protein template information such as amino acid sequence, distance matrix, etc. are all converted into embedding vectors as the input of the neural network;

[0172] Specifically, the data encoding input module includes: the residue pair encoding includes one-hot encoding of the distance between the beta carbon atoms of the residues, the point position of the alpha carbon atoms around each residue, and the residue position mask and other information;

[0173] Specifically, the cyclic attention neural network protein encoding update module includes, the cyclic attention neural network includes: 1) a multi-head neural network is used to update the sequence edge and the residue edge respectively and mutually; 2) a triangular update mechanism; wherein:

[0174] The sequence edge and the residue edge are updated respectively and mutually, including: in the sequence edge, the attention mechanism network can better distinguish the sequence containing in the input protein multiple sequence alignment matrix embedding vector matrix M and the sequence containing irrelevant information, and the M updated by the attention operation will be used to update the embedding vector matrix E of the residue edge:

[0175] E' = f(E + W)

[0176] Wherein, E' is the updated residue edge embedding vector matrix, f is the activation function, and the current activation function is RELU; W is the weight matrix of the base M:

[0177]

[0178] Wherein, D is the depth of the protein multiple sequence alignment matrix; W ij represents the weight between residue i and residue j, represents the compiled vector of residue i on the dth sequence in M, represents the compiled vector of residue j on the dth sequence in M;

[0179] In the residue edge, the network of the attention mechanism can better extract the interaction relationship between each pair of amino acid residues, and the updated E will be used to update the sequence edge embedding vector matrix M:

[0180] M' = a(M)

[0181] Wherein, M' is the updated sequence edge embedding vector matrix, and a is the weight matrix based on E:

[0182] a ij = F(E ij )

[0183] Wherein, F is the activation function; ij generally represents the corresponding numerical value / vector of the ith row and jth column of a certain matrix;

[0184] Specifically, the cyclic attention neural network protein coding update module comprises a triangular update mechanism which updates the coding of the residue pair adjacent to the residue edge to ensure the triangular inequality property of the hidden residues in 3D distance, and the update mechanism layer comprises a product layer based on outward adjacent edges, a product layer based on inward adjacent edges, a triangular attention layer based on the starting site and a triangular attention layer based on the ending site.

[0185] Wherein, the update methods of the product layers based on outward and inward adjacent edges are respectively:

[0186] E ij = g(E ij ) o f (∑ k g(E ik ) o g(E jk ));

[0187] E ij = g(E ij ) o f (∑ k g(E ki ) o g(E kj ));

[0188] Wherein, E ijThe compiled vector of the edge (residue i, residue j), f, g are activation functions respectively; the activation function g of the product layer based on the outward and inward bordering edges is sigmoid, and f is the superposition of LayerNorm and Linear;

[0189] wherein the update method of the triangular attention layer based on the start site and the end site is respectively:

[0190] E ij = trans(att(Q ij , K ik , V ik ));

[0191] E ij = trans(att(Q ij , K kj , V kj ))

[0192] wherein Q ij , K ij , V ij are the query, key and value vectors derived from E ij , att is a self-attention update model similar to the transformer encoder, and trans is an overlayer containing regularization and activation function; the activation function in the overlayer of the triangular attention layer based on the start site and the end site is RELU.

[0193] Specifically, the invariant point attention network distance and angle matrix module comprises: the input of the invariant attention network is target sequence encoding (MSA sequence encoding first row), residue pair encoding E ij , and starting main chain framework T i . The attention network calculates the query, key and value vectors of each amino acid in the encoding space and 3D space respectively according to the target sequence encoding {m i initial}, then calculates the attention matrix of the encoding space and the attention matrix of the 3D space based on the starting main chain framework T i global invariant transformation, and introduces E ij as the bias value of the attention matrix to obtain the final attention matrix A. Finally, the target sequence encoding is updated iteratively through A.

[0194] wherein m i initial is the encoding of the i-th amino acid in the target sequence;

[0195] wherein m iThe query, key, and value values corresponding to the encoding space are q i i i The query, key, and value values corresponding to the 3D space are:

[0196] Wherein the A iteratively updates the target sequence encoding in the following manner:

[0197]

[0198]

[0199]

[0200]

[0201] Specifically, the invariance point attention network distance and angle matrix calculation module includes: the main chain framework is initialized as an identity transformation:

[0202]

[0203] Wherein, I represents a feature matrix; represents an origin vector; that is, all amino acids are placed at the origin of the global coordinates;

[0204] Wherein, Ti can be represented as:

[0205]

[0206] Wherein, the cyclic attention neural network protein encoding update module and the attention network in the invariance point attention network distance and angle matrix calculation module both use a multi-head attention network (6-8 heads), and residual connection is used to facilitate network optimization.

[0207] In the invariance point attention network distance and angle matrix calculation module, a multi-head neural network is used to update attention, Nquery points = 4, and a multi-layer invariance point attention neural network is used to update the target sequence encoding.

[0208] Specifically, the 3D structure prediction module includes, and the main chain framework is updated in the following manner:

[0209]

[0210]

[0211]

[0212] ​​

[0213] Specifically, the 3D structure prediction module includes that the torsion angle between the amino acids is updated by the updated m i With m i initial Linear superposition, and multiple times into the RELU-linear-RELU-linear layer update, the angle obtained after the update contains 3 main chain atom torsion angles and 4 side chain atom torsion angles. Note that the torsion angle is the only quantity learned freely in the model, and the angles and lengths of the chemical bonds are all kept in the preset state.

[0214] Specifically, the 3D structure prediction module includes that the final three-dimensional structure is obtained by superimposing the torsion matrix transformation calculated by the torsion angle on the basis of the main chain framework.

[0215] Those skilled in the art know that in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can be realized by logically programming the method steps to make the system, device and each module thereof provided by the present application in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures within hardware components.

[0216] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for predicting the three-dimensional structure of a protein, characterized in that, include: Step S1: Obtain the multiple sequence permutation matrix containing protein co-evolution information and the protein template, and perform the conversion between MSA sequence and residue pair coding to obtain MSA sequence coding and residue pair coding; Step S2: The MSA sequence encoding and residue pair encoding are updated iteratively through a recurrent attention neural network to generate the latest MSA sequence encoding and residue pair encoding; Step S3: The current residue pair encoding and the target sequence encoding are updated using an invariant point attention neural network to update the current target sequence encoding; Step S4: Update the main chain framework based on the interaction information between residue pairs encoded by the current target sequence, and calculate the torsion angles between residues and atoms to predict the protein structure; The multiple sequence permutation matrix containing protein co-evolution information is based on evolutionary events including mutation, insertion, deletion, and recombination. It aligns corresponding residues in multiple amino acid sequences and compares them column by column to construct a dataset similar to the predicted protein sequence. The protein template is a known structural protein similar to the target protein based on a protein structure database. The recurrent attention neural network uses a multi-head neural network to optimize and update sequence edges and residue edges separately and mutually, while using a triangular update mechanism to update the encoding of residue pairs. The invariant point attention neural network in step S3 employs the following approach: the target sequence encoding is processed by linear layers to calculate the query, key, and value vectors for each amino acid in the coding space and 3D space. Then, the attention matrix of the coding space and the attention matrix of the 3D space based on the global invariance of the initial main chain framework are calculated according to the attention mechanism. The current residue pair encoding is introduced as the deviation value of the attention matrix to obtain the final attention matrix. The target sequence encoding is then iteratively updated through the attention matrix.

2. The protein three-dimensional structure prediction method according to claim 1, characterized in that, The residue pair encoding includes: one-hot encoding of the distance between β carbon atoms of the residue, the location of α carbon atoms around each residue, and residue site mask information.

3. The method for predicting the three-dimensional structure of proteins according to claim 1, characterized in that, The multi-head neural network in the recurrent attention neural network adopts the following approach: the sequence edge and residue edge are optimized and updated separately and mutually by the multi-head neural network, and the updated residue edge embedding vector matrix and sequence edge embedding vector matrix are obtained. Updated residue edge embedding vector matrix: Where E' represents the updated residue edge embedding vector matrix; f represents the activation function; and W represents the weight matrix of the multiple sequence permutation matrix M. ) Where D represents the depth of the protein multiple sequence permutation matrix; This represents the weight between residue i and residue j. This represents the compilation vector of residue i on the d-th sequence in M. This represents the compilation vector of residue j on the d-th sequence in M. This represents the cross product of two vectors, and linear represents a linear reduction layer. Updated sequence edge embedding vector matrix: Where M' represents the updated sequence edge embedding vector matrix, The weight matrix is ​​represented by the residue-edge embedding vector matrix E: Where F represents the activation function; The method of updating the residue pair encoding using the triangular update mechanism is as follows: the residue pair encoding is updated by using the residue edges of adjacent residue pairs to ensure that the encoding satisfies the triangular inequality property in 3D space. The triangular update mechanism includes a triangular product update mechanism and a triangular self-attention update mechanism; wherein, the triangular product update mechanism updates the residue edges by using the neighboring edge information that forms a triangle with the opposite edge of the residue, and includes a product layer based on outward bordering edges and a product layer based on inward bordering edges. The triangular self-attention update mechanism updates the residue edges by using the information of the neighboring edges that are adjacent to the start / end points of the residue's opposite edge. It includes a triangular attention layer based on the start point and a triangular attention layer based on the end point. The product layer based on the outward border and the product layer based on the inward border are respectively adopted as follows: ； in, This represents the encoding of the residue pair edge (i,j). This represents the superposition function of a linear layer and an activation function. The superposition function of layer normalization and linear layers; The triangular attention layer based on the start point and the triangular attention layer based on the end point respectively adopt: ； Among them, Q ij , K ij V ij E respectively ij The derived query, key, and value vectors used for self-attention computation, att represents the self-attention update model, and trans represents the transition layer containing regularization and activation functions.

4. The protein three-dimensional structure prediction method according to claim 1, characterized in that, The method of iteratively updating the target sequence encoding through the attention matrix employs: in, This represents the encoding of the i-th residue in the target sequence. This represents the attention between residues i and j in the attention matrix. This represents the encoding of the residue pair edge (i,j). express The self-attention value corresponding to the encoding space. express The corresponding self-attention value in 3D space. This indicates that the residue pairs encode a weighted output through self-attention. This represents the weighted output of the value in the encoding space. This represents the weighted output in 3D space. Represents the main chain framework of residue i. This indicates a linear layer.

5. The method for predicting the three-dimensional structure of proteins according to claim 1, characterized in that, Starting main chain framework T i use: The main chain framework is initialized with an identity transformation: All amino acids are placed at the origin of the global coordinate system.

6. The method for predicting the three-dimensional structure of proteins according to claim 1, characterized in that, The framework for updating the main chain based on the current target sequence encoding adopts the following: in, express , , Main chain framework Calculate the rotation matrix An important component Main chain framework The deviation coordinates in the diagram.

7. A protein three-dimensional structure prediction system, characterized in that, include: Module M1: Obtains the multiple sequence permutation matrix containing protein co-evolution information and the protein template, and performs the conversion between MSA sequence and residue pair coding to obtain MSA sequence coding and residue pair coding; Module M2: The MSA sequence encoding and residue pair encoding are updated iteratively through a recurrent attention neural network to generate the latest MSA sequence encoding and residue pair encoding; Module M3: The encoding of the current residue pair and the encoding of the target sequence are updated using an invariant point attention neural network; Module M4: Updates the main chain framework based on the interaction information between residue pairs encoded by the current target sequence, and calculates the torsion angles between residues and atoms, thereby predicting the protein structure; The multiple sequence permutation matrix containing protein co-evolution information is based on evolutionary events including mutation, insertion, deletion, and recombination. It aligns corresponding residues in multiple amino acid sequences and compares them column by column to construct a dataset similar to the predicted protein sequence. The protein template is a known structural protein similar to the target protein based on a protein structure database. The recurrent attention neural network uses a multi-head neural network to optimize and update sequence edges and residue edges separately and mutually, while using a triangular update mechanism to update the encoding of residue pairs. The invariant point attention neural network in module M3 employs the following approach: the target sequence encoding is processed by linear layers to calculate the query, key, and value vectors for each amino acid in the coding space and 3D space. Then, the attention matrix in the coding space and the attention matrix in the 3D space based on the global invariance of the initial main chain framework are calculated according to the attention mechanism. The current residue pair encoding is introduced as the deviation value of the attention matrix to obtain the final attention matrix. The target sequence encoding is then updated iteratively through the attention matrix.

8. The protein three-dimensional structure prediction system according to claim 7, characterized in that, The multi-head neural network in the recurrent attention neural network adopts the following approach: the sequence edge and residue edge are optimized and updated separately and mutually by the multi-head neural network, and the updated residue edge embedding vector matrix and sequence edge embedding vector matrix are obtained. Updated residue edge embedding vector matrix: Where E' represents the updated residue edge embedding vector matrix; f represents the activation function; and W represents the weight matrix of the multiple sequence permutation matrix M. ) Where D represents the depth of the protein multiple sequence permutation matrix; This represents the weight between residue i and residue j. This represents the compilation vector of residue i on the d-th sequence in M. This represents the compilation vector of residue j on the d-th sequence in M. This represents the cross product of two vectors, and linear represents a linear reduction layer. Updated sequence edge embedding vector matrix: Where M' represents the updated sequence edge embedding vector matrix, The weight matrix is ​​represented by the residue-edge embedding vector matrix E: Where F represents the activation function; The method of updating the residue pair encoding using the triangular update mechanism is as follows: the residue pair encoding is updated by using the residue edges of adjacent residue pairs to ensure that the encoding satisfies the triangular inequality property in 3D space. The triangular update mechanism includes a triangular product update mechanism and a triangular self-attention update mechanism; wherein, the triangular product update mechanism updates the residue edges by using the neighboring edge information that forms a triangle with the opposite edge of the residue, and includes a product layer based on outward bordering edges and a product layer based on inward bordering edges. The triangular self-attention update mechanism updates the residue edges by using the information of the neighboring edges that are adjacent to the start / end points of the residue's opposite edge. It includes a triangular attention layer based on the start point and a triangular attention layer based on the end point. The product layer based on the outward border and the product layer based on the inward border are respectively adopted as follows: ； in, This represents the encoding of the residue pair edge (i,j). This represents the superposition function of a linear layer and an activation function. The superposition function of layer normalization and linear layers; The triangular attention layer based on the start point and the triangular attention layer based on the end point respectively adopt: ； Among them, Q ij , K ij V ij E respectively ij The derived query, key, and value vectors used for self-attention computation, att represents the self-attention update model, and trans represents the transition layer containing regularization and activation functions; The method of iteratively updating the target sequence encoding through the attention matrix employs: in, This represents the encoding of the i-th residue in the target sequence. This represents the attention between residues i and j in the attention matrix. This represents the encoding of the residue pair edge (i,j). express The self-attention value corresponding to the encoding space. express The corresponding self-attention value in 3D space. This indicates that the residue pairs encode a weighted output through self-attention. This represents the weighted output of the value in the encoding space. This represents the weighted output in 3D space. Represents the main chain framework of residue i. Indicates a linear layer; Starting main chain framework T i use: The main chain framework is initialized with an identity transformation: All amino acids are placed at the origin of the global coordinate system.

9. The protein three-dimensional structure prediction system according to claim 7, characterized in that, The framework for updating the main chain based on the current target sequence encoding adopts the following: in, express , , Main chain framework Calculate the rotation matrix An important component Main chain framework The deviation coordinates in the diagram.

Images