Inherent disordered protein region prediction method based on multi-modal feature fusion

Abstract

The invention discloses an inherent disordered protein region prediction method based on multi-modal feature fusion. The method comprises the following steps: firstly, carrying out feature embedding and preprocessing on a protein sequence by utilizing a pre-trained TAPE model, screening and preprocessing existing 544 amino acid physicochemical features in biology, and extracting evolutionary conservative features from the protein sequence; splicing the preprocessed protein embedding feature matrix, the preprocessed amino acid physicochemical feature matrix and the preprocessed evolutionary conservative feature matrix to obtain a multi-modal fusion feature matrix; then, carrying out layer normalization on the multi-modal fusion feature matrix, and carrying out dimensionality reduction on protein embedding features in the multi-modal fusion feature matrix to obtain a multi-modal fusion feature matrix after local dimensionality reduction; performing position coding on the multi-modal fusion feature matrix after local dimension reduction; and finally, inputting the position-coded multi-modal fusion feature matrix into the prediction model, and predicting the residues. According to the method, high complementation and synergistic interaction of multi-modal features are realized, and the prediction precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of bioinformatics, and specifically is a method for predicting intrinsically disordered protein regions based on multimodal feature fusion. Background Art

[0002] Intrinsically disordered proteins and their regions are widely present in organisms. They lack a stable three-dimensional structure, but play an important role in key biological processes such as cell signal transduction, molecular recognition, and phase separation. Although traditional experimental methods (such as nuclear magnetic resonance and circular dichroism) can directly analyze the disorder of proteins, they have problems such as high detection costs and low efficiency. In recent years, machine learning-based IDR prediction algorithms have made significant progress. However, how to efficiently integrate multi-source heterogeneous features to improve prediction accuracy remains a core challenge in this field.

[0003] At present, the evolution of protein sequence prediction can be divided into three technical generations. The first generation of methods is represented by IUPred and DisEMBL, which estimate the disorder tendency of sequences based on simplified physical models. The former uses residue interactions to predict disordered regions, and the latter constructs prediction rules through amino acid flexibility parameters. Although such methods have the advantage of high computational efficiency, their preset expert features (such as charge asymmetry and hydrophobic moment) are difficult to model the complex relationships between residues. Second-generation methods such as the ANCHOR and PONDR models attempt to introduce machine learning by combining multiple types of features to train support vector machines or random forest classifiers. However, such methods are time-consuming and their generalization performance is limited by the completeness of artificial features. The third generation of methods is based on deep learning technology and has achieved a qualitative breakthrough through autonomous feature learning. For example, the LMDisorder model can learn high-order semantic representations from large-scale unlabeled data, capture long-range dependencies between residues, and greatly improve prediction accuracy. Recurrent neural networks achieve end-to-end prediction by modeling long-range dependencies in sequences, while the introduction of attention mechanisms in the pre-trained language model DeepIDP allows the model to focus on key functional sites, and the pre-trained language model can learn high-level semantic representations from large-scale unlabeled data. However, these methods still have significant limitations: (1) the evolutionary conserved features of protein sequences are not effectively integrated; (2) the hierarchical fusion of local physicochemical properties and global semantic representations has not been solved; these can lead to missed detection of disordered regions and misjudgment of transition points between ordered and disordered regions. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the problem to be solved by the present invention is to provide a method for predicting intrinsically disordered protein regions based on multimodal feature fusion.

[0005] The present invention solves the technical problem by adopting the following technical solutions:

[0006] A method for predicting intrinsically disordered protein regions based on multimodal feature fusion, characterized by comprising the following steps:

[0007] Step 1: Use the pre-trained TAPE model to embed features in protein sequences to obtain a protein embedding feature matrix; preprocess the protein embedding feature matrix to obtain a preprocessed protein embedding feature matrix; screen and preprocess the 544 existing amino acid physicochemical features in biology to obtain a preprocessed amino acid physicochemical feature matrix; extract evolutionary conserved features from protein sequences based on the PSSM matrix to obtain an evolutionary conserved feature matrix; concatenate the preprocessed protein embedding feature matrix, the amino acid physicochemical feature matrix, and the evolutionary conserved feature matrix to obtain a multimodal fusion feature matrix;

[0008] Step 2: Perform layer-by-layer normalization on the multimodal fusion feature matrix, and reduce the dimensionality of the protein embedding features in the layer-normalized multimodal fusion feature matrix to obtain the multimodal fusion feature matrix after local dimensionality reduction.

[0009] Step 3: Position-encode the multimodal fusion feature matrix after local dimensionality reduction using the following formula to obtain a position-encoded multimodal fusion feature matrix;

[0010]

[0011] Where PE(i,2j) and PE(i,2j+1) represent the elements of the position-encoded multimodal fusion feature matrix in the i-th row and 2j-th column and the i-th row and 2j+1-th column, 2j represents an even column, 2j+1 represents an odd column, and d model represents the matrix dimension, L is the length of the protein sequence;

[0012] Step 4: Input the position-encoded multimodal fusion feature matrix into the prediction model, predict the residues, and obtain the prediction probability of each residue; if the prediction probability is greater than the set threshold, it is an ordered residue, otherwise it is a disordered residue.

[0013] Furthermore, the simulated annealing algorithm was used to screen the 544 existing biological amino acid physicochemical features. That is, a variety of amino acid physicochemical features were randomly selected from the 544 amino acid physicochemical features to form an initial feature group. The distance d between the ordered residues and the disordered residues in the initial feature group was calculated according to formula (2). euc ;

[0014] d euc =||D dis -D ord ||2 (2)

[0015] Where D dis is the mean vector of disordered residues, D ordis the mean vector of ordered residues, || ||2 represents the Euclidean distance;

[0016] Randomly select a feature from the features outside the initial feature group to replace any feature in the initial feature group to form a new feature group, and calculate the distance d between the ordered residues and the disordered residues in the new feature group euc '; If d euc 'greater than d euc , then retain the new feature group; if d euc 'Less than or equal to d euc , then calculate the distance deviation p between the initial feature group and the new feature group. If the distance deviation is less than or equal to the set threshold, the initial feature group is retained, otherwise the new feature group is retained;

[0017]

[0018] Where T represents the temperature in simulated annealing;

[0019] A feature is randomly selected from the unselected features to replace any feature in the retained feature group, and the above process is repeated to obtain the amino acid physicochemical feature group.

[0020] Furthermore, a sliding window is used to preprocess the protein embedding feature matrix and the screened amino acid physicochemical feature group. The preprocessing of the protein embedding feature matrix is ​​shown in the following formula:

[0021]

[0022] Where, v i,j ' represents the element in row i and column j of the preprocessed protein embedding feature matrix, v i,j is the element in the i-th row and j-th column of the protein embedding feature matrix, and w represents the sliding window size.

[0023] Furthermore, the sizes of the sliding windows are 11 and 45.

[0024] Furthermore, the prediction model includes an encoder and a fully connected layer; the position-encoded multimodal fusion feature matrix is ​​input into the encoder for encoding to obtain encoded features; the encoded features pass through the fully connected layer and a Sigmoid activation function is used to obtain the predicted probability of each residue;

[0025] The encoding process of the encoder is expressed as:

[0026] P'=LN(P+Contact(H1,...,H m ,...,H h )W 0 ) (6)

[0027]

[0028] Where P' is the encoding feature output by the encoder, P is the multimodal fusion feature matrix after position encoding, LN(·) represents the layer normalization operation, Contact(·) represents the splicing operation, and H m represents the output of the mth attention head, h represents the number of attention heads, and W 0 represents the linear transformation weight matrix, softmax(·) represents the softmax function, W q 、W k 、W v represents the learnable weight matrix.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The physicochemical features of amino acids, screened using a simulated annealing algorithm, directly reflect the core physicochemical driving forces of protein folding, providing highly interpretable fundamental information. However, these features essentially describe local or global average properties, making it difficult to precisely capture long-range contextual dependencies and the intrinsic semantic information of complex sequence patterns within protein sequences. This limitation can be overcome by embedding features of protein sequences using the pre-trained TAPE model. By learning features from a large number of unlabeled protein sequences, the pre-trained TAPE model can model long-range interactions between residues and understand the roles of amino acids in specific contexts, thereby precisely identifying disordered regions that rely on long-range patterns rather than simply the sum of local physicochemical properties. Evolutionarily conserved features are screened from protein sequences using the PSSM matrix. The PSSM matrix reveals the functional importance of protein sequences across the entire evolutionary tree by comparing homology information. Combining the protein embedding features obtained from the pre-trained TAPE model with the evolutionarily conserved features screened using the PSSM matrix can better distinguish functional disordered regions from non-functional disordered fragments.

[0031] Multimodal feature fusion integrates protein embedding features and amino acid physicochemical characteristics obtained at different spatial scales (implemented via a sliding window), combined with evolutionarily conserved features, achieving a high degree of complementarity and synergy among multimodal features. Driven by this multidimensional information, a more robust and accurate prediction model is constructed, improving prediction accuracy. Sliding window preprocessing enhances information interaction between ordered and disordered regions, helping to improve prediction accuracy.

[0032] The introduction of a multi-head attention mechanism in the encoder can simultaneously distinguish the contribution strength of long-range and local interactions between residues, generate context-aware feature representations, suppress the interference of noise signals in disordered regions, and retain the global pattern of synergy across sequence residues. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is the overall flow chart of the present invention;

[0034] Figure 2 This is a comparison chart of the results of different models on the DM1229 dataset;

[0035] Figure 3 This is a comparison chart of the results of different models on the SL329 dataset. DETAILED DESCRIPTION

[0036] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the scope of protection of the present application.

[0037] The present invention provides a method for predicting intrinsically disordered protein regions based on multimodal feature fusion, comprising the following steps:

[0038] Step 1: Feature embedding and preprocessing of protein sequences to obtain a preprocessed protein embedding feature matrix; screening and preprocessing of amino acid physicochemical features to obtain a preprocessed amino acid physicochemical feature matrix; extracting evolutionary conserved features from protein sequences to obtain an evolutionary conserved feature matrix; and concatenating the preprocessed protein embedding feature matrix, amino acid physicochemical feature matrix, and evolutionary conserved feature matrix to obtain a multimodal fusion feature matrix.

[0039] 1-1) Input the protein sequence into the pre-trained TAPE model for feature embedding to obtain the deep semantic information of the protein sequence and obtain a protein embedding feature matrix V with a dimension of L×768; preprocess the protein embedding feature matrix, take a sliding window on the protein embedding feature matrix, accumulate the feature vectors of the residues under the sliding window, take the average value as the feature vector of the center residue under the sliding window, traverse the entire protein embedding feature matrix, and obtain the preprocessed protein embedding feature matrix, see formula (1); considering the length of the disordered region, select two sliding windows of size 11 and 45 to obtain the interaction relationship between the disordered region and the ordered region, and obtain the preprocessed protein embedding feature matrices V1 and V2;

[0040]

[0041] Where, v i,j ' represents the element in row i and column j of the preprocessed protein embedding feature matrix, v i,j is the element in the i-th row and j-th column of the protein embedding feature matrix, L is the length of the protein sequence, and w represents the sliding window size;

[0042] 1-2) Screening the physicochemical characteristics of 544 existing amino acids in biology using a simulated annealing algorithm, and then preprocessing them to obtain preprocessed amino acid physicochemical characteristic matrices X1 and X2;

[0043] Randomly select N amino acid physicochemical features from 544 kinds of amino acid features to form the initial feature group S, and calculate the distance d between the ordered residues and the disordered residues in the initial feature group according to formula (2): euc , the larger the distance, the higher the discrimination between ordered and disordered residues;

[0044] d euc =||D dis -D ord ||2 (2)

[0045] Where D dis is the mean vector of disordered residues, D ord is the mean vector of ordered residues, ||||2 represents the Euclidean distance;

[0046] Randomly select a feature from the features outside the initial feature group to replace any feature in the initial feature group to form a new feature group S new , calculate the distance d between the ordered residues and the disordered residues in the new feature group euc '; If d euc 'greater than d euc , then retain the new feature group S new If d euc 'Less than or equal to d euc , then calculate the distance deviation p between the initial feature group and the new feature group. If the distance deviation is less than or equal to the set threshold, the initial feature group is retained, otherwise the new feature group is retained;

[0047]

[0048] Where T represents the temperature in simulated annealing;

[0049] Randomly select a feature from the unselected features to replace any feature in the retained feature set, and repeat the above process to obtain an amino acid physicochemical feature set with a dimension of L×12; preprocess the amino acid feature set by sliding window processing with sizes of 11 and 45, respectively, to obtain the preprocessed amino acid physicochemical feature matrices Y1 and Y2;

[0050] 1-3) Based on the PSSM matrix, an evolutionarily conserved feature matrix with a dimension of L×20 is screened from the protein sequence;

[0051] 1-4) The pre-processed protein embedding feature matrix, amino acid physicochemical feature matrix and evolutionary conservative feature matrix are spliced ​​to achieve preliminary fusion of multimodal features and obtain a multimodal fusion feature matrix.

[0052] Step 2: Perform layer-by-layer normalization on the multimodal fusion feature matrix, and perform PCA dimensionality reduction on the protein embedding feature part of the layer-normalized multimodal fusion feature matrix to reduce the dimension to L×256. At the same time, the principal component is maintained at 80%. The feature dimension is compressed while retaining as much principal component information as possible, effectively balancing the model complexity and representation ability, and obtaining the multimodal fusion feature matrix after local dimensionality reduction.

[0053] Step 3: Position encoding is performed on the multimodal fusion feature matrix after local dimensionality reduction to capture the topological correlation information between residues, and then capture the sequence dependency characteristics of the protein sequence to obtain a position-encoded multimodal fusion feature matrix;

[0054]

[0055] Where PE(i,2j) and PE(i,2j+1) represent the elements of the position-encoded multimodal fusion feature matrix in the i-th row and 2j-th column and the i-th row and 2j+1-th column, 2j represents an even column, 2j+1 represents an odd column, and d model Represents the matrix dimension.

[0056] Step 4: Input the position-encoded multimodal fusion feature matrix into the prediction model to predict the residues and obtain the disordered regions;

[0057] The prediction model includes an encoder and a fully connected layer. The position-encoded multimodal fusion feature matrix is ​​input into the encoder for encoding to obtain encoded features. The encoded features pass through the fully connected layer and the Sigmoid activation function to obtain the predicted probability of each residue. The predicted probability is compared with a set threshold. If the predicted probability is greater than the set threshold, the residue is ordered; otherwise, it is disordered.

[0058] The encoder uses a multi-head attention mechanism to achieve dynamic interaction of multimodal features through a learnable weight matrix. When modeling cross-residue physicochemical interactions and evolutionary constraints, it can suppress high-frequency noise interference in disordered regions while maintaining global synergy. The encoder processing process is expressed as:

[0059] P'=LN(P+Contact(H1,...,H m ,...,H h )W 0 ) (6)

[0060]

[0061] Where P' is the encoding feature output by the encoder, P is the multimodal fusion feature matrix after position encoding, LN(·) represents the layer normalization operation, Contact(·) represents the splicing operation, and H m represents the output of the mth attention head, h represents the number of attention heads, and W 0 represents the linear transformation weight matrix, softmax(·) represents the softmax function, W q 、W k 、W v represents the learnable weight matrix;

[0062] The fully connected layer is represented as:

[0063] Y=Sigmoid(P'·W+b)(8)

[0064] Where Y represents the prediction probability matrix, Sigmoid(·) represents the Sigmoid activation function, W represents the weight matrix, and b represents the bias term.

[0065] Example

[0066] In order to verify the effectiveness of this application, we conducted tests on the DM1229 and SL329 datasets, and recorded the prediction model of this application as IDP-Meta. [1] , ESpritz-N [1] , ESpritz-D [1] 、Iupred2A-shor t[2] 、Iupred2A-long [2] 、Spot-disorder-Single [3] LMDisorder [4] For comparison, AUC-ROC, AUC-PR, MCC, Se, Sp and SW are used as indicators to measure model performance. The comparison results are shown in Figure 2 and 3. AUC-ROC refers to the area under the ROC curve, which is used to evaluate the model's ability to distinguish ordered residues from disordered residues at all classification thresholds; AUC-PR refers to the area under the precision curve, which is used to evaluate the prediction accuracy of disordered residues at different sensitivities; MCC refers to the Matthews correlation coefficient, which is used to comprehensively evaluate the predictive ability when the ratio of ordered residues to disordered residues is unbalanced; Se refers to sensitivity, which is used to evaluate the proportion of all real disordered residues successfully identified by the model. A high Se indicates that the model can effectively identify disordered residues and avoid missed detection; Sp refers to specificity, which is used to evaluate the proportion of all real ordered residues correctly identified by the model. A high Sp can prevent the model from misidentifying disordered residues and ensure the reliability of model predictions; SW refers to the Youden index, which comprehensively measures the model's ability to distinguish ordered residues from disordered residues to prevent extreme situations such as Sp being too high and Se being too low, or Se being too high and Sp being too low.

[0067] As can be seen from the figure, the present application has achieved good prediction results on both data sets, especially the two evaluation indicators Se and SW are significantly higher than other models. Traditional physicochemical features are prone to miss disordered regions that depend on long-range context, resulting in the Se indicator not being able to reach a high level. The present application introduces a pre-trained TAPE model, which learns long-range context dependencies and complex sequence patterns between residues by pre-training on massive protein sequences, which can significantly enhance the sensitivity to disordered regions, reduce false negatives, and greatly improve the Se indicator. Amino acid physicochemical features and evolutionary conservative features can directly reflect the core physicochemical mechanisms of disordered residues and reveal functional constraint areas. Therefore, by fusing multimodal features (protein embedding features, amino acid physicochemical features, and evolutionary conservative features), the Se indicator can be improved while the Sp indicator is also at a high level; cross-modal feature interaction is achieved through the encoder, the synergistic benefits of multimodal features are maximized, and comprehensive indicators such as SW and MCC are significantly improved.

[0068] The sources of existing prediction models are as follows:

[0069] [1]Ian Walsh,Alberto JMMartin,Tomàs Di Domenico,Silvio CETosatto,ESpritz: accurate and fast prediction of protein disorder,Bioinformatics,Volume 28,Issue 4,February 2012,Pages 503–509.

[0070] [2]Mészáros B,Erdos G,Dosztányi Z.IUPred2A:context-dependentprediction of proteindisorder as a function of redox state and proteinbinding.Nucleic Acids Res.2018Jul2:46(W1):W329-W337.

[0071] [3]Hanson J,Paliwal K,Zhou Y.Accurate Single-Sequence Prediction ofProtein IntrinsicDisorder by an Ensemble of Deep Recurrent and ConvolutionalArchitectures.J Chem Inf Model.2018Nov 26:58(11):2369-2376.

[0072] [4]Song Y,Yuan Q,Chen S,Chen K,Zhou Y,Yang Y.Fast and accurateprotein intrinsicdisorder prediction by using a pretrained language model.Brief Bioinform.2023Jul20:24(4):bbad173.

[0073] Thanks for reading the book.

Claims

1. A method for predicting intrinsically disordered protein regions based on multimodal feature fusion, characterized in that: The following steps are involved: Step 1: Use the pre-trained TAPE model to embed features of the protein sequence and obtain the protein embedding feature matrix; Preprocessing the protein embedding feature matrix to obtain a preprocessed protein embedding feature matrix; The physicochemical characteristics of 544 existing amino acids in biology were screened and preprocessed to obtain the preprocessed amino acid physicochemical characteristic matrix; Extract evolutionary conserved features from protein sequences based on the PSSM matrix to obtain an evolutionary conserved feature matrix; concatenate the preprocessed protein embedding feature matrix, amino acid physicochemical feature matrix, and evolutionary conserved feature matrix to obtain a multimodal fusion feature matrix; Step 2: Perform layer-by-layer normalization on the multimodal fusion feature matrix, and reduce the dimensionality of the protein embedding features in the layer-normalized multimodal fusion feature matrix to obtain the multimodal fusion feature matrix after local dimensionality reduction. Step 3: Position-encode the multimodal fusion feature matrix after local dimensionality reduction using the following formula to obtain a position-encoded multimodal fusion feature matrix; Where PE(i,2j) and PE(i,2j+1) represent the elements of the position-encoded multimodal fusion feature matrix in the i-th row and 2j-th column and the i-th row and 2j+1-th column, 2j represents an even column, 2j+1 represents an odd column, and d model represents the matrix dimension, L is the length of the protein sequence; Step 4: Input the position-encoded multimodal fusion feature matrix into the prediction model, predict the residues, and obtain the prediction probability of each residue; if the prediction probability is greater than the set threshold, it is an ordered residue, otherwise it is a disordered residue.

2. The method for predicting intrinsically disordered protein regions based on multimodal feature fusion according to claim 1, characterized in that: The simulated annealing algorithm was used to screen the 544 existing biological amino acid physicochemical features. That is, a variety of amino acid physicochemical features were randomly selected from the 544 amino acid physicochemical features to form an initial feature group. The distance d between the ordered residues and the disordered residues in the initial feature group was calculated according to formula (2). euc ; d euc =||D dis -D ord ||2 (2) Where D dis is the mean vector of disordered residues, D ord is the mean vector of ordered residues, || ||2 represents the Euclidean distance; Randomly select a feature from the features outside the initial feature group to replace any feature in the initial feature group to form a new feature group, and calculate the distance d between the ordered residues and the disordered residues in the new feature group euc '; If d euc 'greater than d euc , then retain the new feature group; if d euc 'Less than or equal to d euc , then calculate the distance deviation p between the initial feature group and the new feature group. If the distance deviation is less than or equal to the set threshold, the initial feature group is retained, otherwise the new feature group is retained; Where T represents the temperature in simulated annealing; A feature is randomly selected from the unselected features to replace any feature in the retained feature group, and the above process is repeated to obtain the amino acid physicochemical feature group.

3. The method for predicting intrinsically disordered protein regions based on multimodal feature fusion according to claim 1 or 2, characterized in that: A sliding window is used to preprocess the protein embedding feature matrix and the screened amino acid physicochemical feature group. The preprocessing of the protein embedding feature matrix is ​​shown in the following formula: Where, v i,j ' represents the element in row i and column j of the preprocessed protein embedding feature matrix, v i,j is the element in the i-th row and j-th column of the protein embedding feature matrix, and w represents the sliding window size.

4. The method for predicting intrinsically disordered protein regions based on multimodal feature fusion according to claim 3, characterized in that: The sizes of the sliding windows are 11 and 45.

5. The method for predicting intrinsically disordered protein regions based on multimodal feature fusion according to claim 1, wherein: The prediction model includes an encoder and a fully connected layer; the position-encoded multimodal fusion feature matrix is ​​input into the encoder for encoding to obtain encoding features; The encoded features pass through the fully connected layer and the predicted probability of each residue is obtained through the Sigmoid activation function; The encoding process of the encoder is expressed as: P'=LN(P+Contact(H1,...,H m ,...,H h )W 0 ) (6) Where P' is the encoding feature output by the encoder, P is the multimodal fusion feature matrix after position encoding, LN(·) represents the layer normalization operation, Contact(·) represents the splicing operation, and H m represents the output of the mth attention head, h represents the number of attention heads, and W 0 represents the linear transformation weight matrix, softmax(·) represents the softmax function, W q 、W k 、W v represents the learnable weight matrix.

Images