A classification method for DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-pathway neural network

Abstract

The present invention provides a method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and a dual-path neural network, which is specifically a staged classification method combining ESM-2 with a dual-path neural network, called DRBP-EDP; in the first stage of training, protein sequences are classified as nucleic acid-binding proteins or non-nucleic acid-binding proteins; in the second stage of training, nucleic acid-binding proteins are further classified as DNA-binding proteins or RNA-binding proteins; at the same time, the present invention also constructs a refined data set construction method to create a high-quality protein classification data set. Through this method, the DRBP-EDP model achieves good performance. The classification accuracy of nucleic acid-binding proteins and non-nucleic acid-binding proteins in the first stage is 90.03%, and the classification accuracy of DNA-binding proteins and RNA-binding proteins in the second stage is 89.56%. It not only proposes new insights and methods for protein classification, but also provides innovative tools for studying protein functions, thereby providing new opportunities for the development of life sciences.

Description

Technical Field

[0001] The invention belongs to the field of biotechnology and relates to a classification method of DNA binding proteins and RNA binding proteins based on ESM-2 and a dual-path neural network. Background Art

[0002] With the rapid development of new-generation biotechnologies such as gene editing, transgenics and molecular markers, in the agricultural field, DNA-binding proteins enhance the stress and disease resistance of crops by regulating the expression of specific genes; similarly, RNA-binding proteins can promote their translation, enabling crops to more efficiently utilize genetic resources, thereby optimizing growth and increasing yields; in the medical field, DNA-binding proteins are key to gene therapy, while RNA-binding proteins contribute to cancer treatment by stabilizing mRNA involved in tumor suppression, enabling more targeted therapies.

[0003] DNA-binding proteins and RNA-binding proteins have considerable similarities in sequence and structure. Classifying them can avoid unnecessary complexity and confusion, help understand their specific functions and their roles in different biological processes, and facilitate the development of research tools and experimental techniques specifically targeting these proteins.

[0004] Traditional methods for identifying nucleic acid-binding proteins mainly rely on experimental techniques, but they are usually time-consuming, costly and difficult to apply on a large scale. In addition, although tools such as AlphaFold can predict protein structures, high-quality experimental structural data are still relatively scarce. Moreover, the computational complexity of processing three-dimensional structural data is high, and training deep learning models requires high-performance computing resources. Therefore, using deep learning for sequence-based classification of DNA-binding proteins and RNA-binding proteins has become a more efficient and feasible solution.

[0005] Although previous studies have made many advances, the protein classification task still faces many challenges. First, the prediction accuracy of the model still needs to be improved, especially when dealing with imbalanced and noisy data, the model often performs poorly. Second, the ability to extract protein sequence features is limited, and existing models are difficult to fully capture the complex features in the sequence. In addition, the quality and quantity of data is also an important issue. Although protein sequence data is gradually increasing, high-quality labeled data is still relatively scarce.

[0006] This application is not intended for the diagnosis, prevention or treatment of any form of disease. Summary of the invention

[0007] To address these problems, this study proposed a phased training method that combines a protein pre-training model with a dual-pathway deep learning network.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A classification method for DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-pathway neural network, comprising:

[0010] S1: Collect protein data and perform data correction, construct data sets, and obtain protein sequences;

[0011] S2: tokenize the protein sequence as input data;

[0012] S3: The ESM-2 pre-trained model extracts high-dimensional features after receiving input data;

[0013] S4: The dual-path neural network extracts high-level features after receiving the high-dimensional features, wherein the dual-path neural network includes a first path and a second path;

[0014] S5: high-level features are fused through the fully connected layer;

[0015] S6: The fused feature vector is passed to the classification layer to generate the classification results, and the classification of DNA-binding proteins and RNA-binding proteins is completed. The staged classification method combining ESM-2 with the dual-path neural network is called DRBP-EDP;

[0016] The Optuna framework was used to optimize the hyperparameters in the classification method model, and the best hyperparameter combination was determined based on cross-validation accuracy and continuous adjustments.

[0017] Furthermore, the specific steps of S1 include:

[0018] S1-1: Collect protein data from the Swiss-Prot sub-database;

[0019] S1-2: Delete protein sequences containing special amino acid codes in the protein dataset;

[0020] S1-3: CD-HIT tool was used to perform redundancy removal on the protein dataset and remove protein sequences with similarity greater than 40%.

[0021] Furthermore, the protein data includes nucleic acid binding protein sequences and non-nucleic acid binding protein sequences, wherein the nucleic acid binding protein sequences include DNA binding protein sequences and RNA binding protein sequences.

[0022] Furthermore, the specific steps of S2 include: tokenizing the protein sequence using ‘AutoTokenizer’ and converting the protein sequence into an input format that can be processed by the ESM-2 pre-trained model.

[0023] Furthermore, the tokenization process includes an 'input_ids' tokenization process and an 'attention_mask' tokenization process, and each amino acid character in the protein sequence corresponds to a specific digital ID to generate 'input_ids';

[0024] 'attention_mask' is a binary matrix used to indicate which tags the model should pay attention to when processing the input. By masking out the padding part, the model ensures that it only pays attention to the actual input tags and ignores the padding part. The padding part is marked as 0 and the actual input part is marked as 1.

[0025] Furthermore, in S3, the output last_hidden_state of the last layer of the encoder is directly used for fine-tuning in the ESM-2 pre-trained model, and the embedding layer and the encoder layer are frozen during the model training process to keep the pre-trained weights of these layers unchanged; the output 'last_hidden_state' has a shape of '[batch_size, seq_len, embedding_dim]', which contains the semantic information and contextual dependencies of each token in the sequence.

[0026] Furthermore, in S4, the first path includes a Transformer encoder and a CNN module, and the second path includes a BiLSTM and an Attention mechanism.

[0027] Furthermore, in the first path, the Transformer encoder uses a multi-head attention mechanism to capture global dependencies, and the CNN module further extracts local features and spatial information; in the second path, the BiLSTM captures bidirectional dependencies in the input sequence, and the Attention mechanism further extracts key information in the sequence.

[0028] Furthermore, the ESM-2 pre-trained model and the dual-path neural network training phase use stratified five-fold cross-validation to divide the data set into training set, validation set, and test set. The use of stratified five-fold cross-validation can not only more accurately evaluate the actual performance of the model, but also ensure that the proportion of each category in each fold is consistent with the original data set, thereby effectively addressing the problem of data set category imbalance.

[0029] In each partition, the model is reinitialized to ensure that each training starts from the same initial conditions, thereby avoiding interference between model states and ensuring the fairness and consistency of the evaluation. In addition, in the first stage of training, the model state in each training evaluation is saved, and in the second stage of training, the best model state of the first stage of training is loaded for the next step of training and evaluation.

[0030] Furthermore, in the simulation training process of the ESM-2 pre-trained model and the dual-path neural network, a data augmentation strategy was used to introduce protein sequence variants in order to improve the generalization ability and robustness of the model, specifically including:

[0031] (1) Insert a randomly selected amino acid at a random position in the sequence;

[0032] (2) randomly select an amino acid and delete it from the sequence;

[0033] (3) Randomly select two amino acids and swap their positions;

[0034] (4) randomly selecting an amino acid and replacing it with another randomly selected amino acid;

[0035] In addition, the data augmentation strategy combines a method of dynamically adjusting the probability of data augmentation. In the early stages of training, a higher probability of data augmentation is used to quickly introduce data diversity, thereby breaking the model's prior bias towards the initial data distribution. As training progresses, the probability of data augmentation is gradually reduced, allowing the model to pay more attention to the details and internal laws of real data in the later stages, which helps to stabilize the convergence and accuracy of the model.

[0036] Beneficial effects of the present invention:

[0037] 1. The present invention creates a new protein sequence dataset dedicated to classifying DNA-binding proteins and RNA-binding proteins, and designs a new dataset construction method. In order to obtain more reliable and representative model performance evaluation results, it is necessary to construct a new benchmark dataset to ensure the latest, complete and independent data. In the dataset of the present invention, the ratio of nucleic acid-binding proteins to non-nucleic acid-binding proteins is 1:1, while the ratio of DNA-binding proteins to RNA-binding proteins is 3:2. With the continuous expansion of the protein database, the quality of the dataset will continue to improve, and the ratios of various proteins will gradually tend to be balanced.

[0038] 2. The present invention proposes a model architecture that combines the protein language model ESM-2 and the dual-path neural network. The protein language model can automatically learn high-dimensional and complex features from sequence data, avoiding the tedious work of manual feature design and possible subjective bias, while the dual-path neural network effectively improves the model's expressiveness and predictive performance by processing different feature channels in parallel and fusing multi-scale information; the present invention successfully improves the classification performance of the model on balanced and unbalanced data sets through a reasonable data partitioning strategy and an optimized loss function. Nevertheless, the model's Specificity (SP) in the second classification stage still has room for improvement; due to hardware limitations, the ESM2 model with larger parameters was not used, which may limit the depth and detail of feature representation; future research should explore multi-path neural networks, increase the number of paths to capture more diverse features, and combine traditional protein processing methods with protein language models to fully extract and utilize sequence features, thereby further improving the model's generalization ability on various tasks.

[0039] 3. The present invention proposes a phased classification method, which first classifies protein sequences into nucleic acid binding proteins or non-nucleic acid binding proteins in the first stage, and then further subdivides nucleic acid binding proteins into DNA binding proteins or RNA binding proteins in the second stage; compared with existing binary classification methods, such as directly distinguishing DNA binding proteins from non-DNA binding proteins, RNA binding proteins from non-RNA binding proteins, and DNA binding proteins from RNA binding proteins, this method reduces confusion through step-by-step screening, improves classification accuracy, simplifies the complexity of the problem, and makes the tasks of each stage clearer; however, the disadvantage of this method is that it is difficult to deal with proteins with both DNA and RNA binding functions, because it only classifies nucleic acid binding proteins as DNA binding proteins or RNA binding proteins, without considering multiple binding functions; therefore, it is recommended to introduce multi-label classification mechanisms and other solutions in subsequent studies to more accurately characterize the multifunctionality of proteins and enhance the comprehensive understanding of their functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flow chart of a classification method for DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-pathway neural network;

[0041] Figure 2 This is a schematic diagram of the ESM-2 pre-training model;

[0042] Figure 3 This is a schematic diagram of path 1 Transformer+CNN;

[0043] Figure 4 This is the schematic diagram of BiLSTM+Attention of path 2;

[0044] Figure 5 Schematic diagram of the main process of DRBP-EDP model training;

[0045] Figure 6 For data partitioning, the dataset is divided into training set, validation set and test set in combination with stratified five-fold cross validation;

[0046] Figure 7 Four ways to enhance data;

[0047] Figure 8 The performance of DRBP-EDP model at different stages on Test_data;

[0048] Fig. 9 (a) and (b) are the confusion matrices at different stages;

[0049] Fig.10 (a), (b), (c) and (d) are scatter plots of the prediction results for each type of protein;

[0050] Fig.11 (a) is the performance of different models on Train_val_data, Fig.11 (b) Performance of different models on Test_data;

[0051] Fig.12 (a) and (b) are the performances of different models on Test_data;

[0052] Fig.13 (a) is the executable version of the DRBP-EDP model. Fig.13 (b) is the network version of the DRBP-EDP model. DETAILED DESCRIPTION

[0053] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected. Example

[0054] A method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-pathway neural networks, comprising:

[0055] S1: Collect protein data and perform data correction, construct data sets, and obtain protein sequences;

[0056] S2: tokenize the protein sequence as input data;

[0057] S3: The ESM-2 pre-trained model extracts high-dimensional features after receiving input data;

[0058] S4: The dual-path neural network extracts high-level features after receiving the high-dimensional features, wherein the dual-path neural network includes a first path and a second path;

[0059] S5: high-level features are fused through the fully connected layer;

[0060] S6: Pass the fused feature vector to the classification layer to generate classification results and complete the classification of DNA-binding proteins and RNA-binding proteins; the staged classification method combining ESM-2 with the dual-path neural network is called DRBP-EDP;

[0061] The Optuna framework was used to optimize the hyperparameters in the classification method model, and the best hyperparameter combination was determined based on cross-validation accuracy and continuous adjustments.

[0062] Protein sequence to be classified:

[0063] (1) Visit the official website of UniProt, click “UniProtKB” on the navigation bar to enter the protein knowledge base, and enter “(*) AND (reviewed:true) AND (go:0003677) AND (length:[50 TO1000])” in the search box to obtain 42,508 protein sequences with GO (Gene Ontology) of “DNA binding” and sequence length between 50 and 1000.

[0064] (2) Enter “(*) AND (reviewed:true) AND (go:0003723) AND(length:[50 TO 1000])” in the search box to obtain 64,307 protein sequences with GO (Gene Ontology) of “RNA binding” and sequence length between 50 and 1000.

[0065] (3) Enter "(*) AND (reviewed:true) AND (length:[50 TO 1000])NOT (go:0003677) NOT (go:0003723) AND (existence:1) AND (structure_3d:true)" in the search box to obtain 25,195 protein sequences that are neither DNA-binding proteins nor RNA-binding proteins. The evidence level of the sequence existence is set to 1 to ensure that these proteins have experimental support and have resolved three-dimensional structures to enhance the credibility of non-nucleic acid binding protein sequences.

[0066] (4) Deleting protein sequences containing special amino acid codes in the three datasets;

[0067] (5) CD-HIT was used to perform redundancy removal on the above datasets and remove protein sequences with a similarity greater than 40%. Finally, 8882 DNA-binding protein sequences, 5670 RNA-binding protein sequences, and 14469 non-nucleic acid-binding protein sequences were obtained, which were denoted as S dbp , S rbp and S nbp ,exist Figure 5 middle, , .

[0068] At the same time, we tried to remove protein sequences with a similarity greater than 30%. We first used CD-HIT to reduce the sequence similarity to below 60%, and then used PSI-CD-HIT to further reduce it to below 30%. However, due to the imbalance in the amount of data obtained, 40% was finally selected as the threshold to remove redundant sequences.

[0069] Tokenize the protein sequences to be classified:

[0070] The protein sequence is tokenized using 'AutoTokenizer' and converted into an input format that can be processed by the ESM-2 model. Specifically, each amino acid character in the protein sequence corresponds to a specific numeric ID to generate 'input_ids'. These IDs are based on the vocabulary index of the pre-trained model. The amino acid encoding comparison table is shown in Table 1.

[0071] 'attention_mask' is a binary matrix used to indicate which tokens the model should focus on when processing the input. By masking out the padding part, the model ensures that it only focuses on the actual input tokens and ignores the padding part. The padding part is marked as 0 and the actual input part is marked as 1.

[0072] Table 1

[0073]

[0074] ESM-2 pre-trained model:

[0075] After receiving the input data, ESM-2 undergoes multiple layers of transformation and outputs 'last_hidden_state' with a shape of '[batch_size, seq_len, embedding_dim]'. This output contains the semantic information and contextual dependencies of each tag in the sequence.

[0076] The ESM-2 pre-trained model does not involve the EsmPooler layer and the EsmContactPredictionHead layer. Instead, it directly uses the output of the last layer of the encoder (last_hidden_state) for fine-tuning, and freezes the embedding layer and encoder layer during model training to keep the pre-trained weights of these layers unchanged. In this way, the rich features learned by the pre-trained model on large-scale protein sequence data are fully utilized to improve the performance on classification tasks.

[0077] Two-way neural network:

[0078] The high-dimensional feature representation is further processed and transformed through the two paths of 'Transformer + CNN' and 'BiLSTM + Attention' to extract higher-level features. These high-level features are then fused through the fully connected layer to form a more compact and information-rich feature representation. Finally, the fused feature vector is passed to the classification layer to generate the classification result.

[0079] (1) Transformer + CNN

[0080] In path 1, the Transformer encoder uses a multi-head attention mechanism to capture global dependencies, while the CNN module further extracts local features and spatial information. The combination of the two can effectively improve the model performance. The Transformer encoder is composed of M identical layers stacked together, each layer contains two sublayers, a multi-head attention mechanism and a feedforward neural network. Each sublayer is followed by a residual connection and layer normalization. In other words, the output of each sublayer is , where x is the input feature of the current layer, is the output of the current layer.

[0081] The Scaled Dot-Product Attention mechanism is used to calculate the similarity and correlation between each amino acid in the input sequence. It obtains the attention score by calculating the dot product between the query and the key, and then uses these scores to perform weighted summation of the values ​​to generate a new embedding representation. The calculation formula is as follows:

[0082]

[0083] where are the query, key and value matrices respectively, d k is the dimension of the key.

[0084] Multi-Head Attention linearly transforms the query, key, and value into h subspaces, each with a dimension of ,in d model is the total dimension of the input; ScaledDot-Product Attention is executed in parallel in each independent subspace to obtain the output of h heads. The calculation formula for each head is as follows:

[0085]

[0086] in, , and are different linear transformation matrices used to project queries, keys, and values ​​into the 'i'th subspace; the outputs of all heads are then concatenated together to form a new matrix: , the dimension after splicing is d model , and then the concatenated result is transformed through a linear matrix to generate the final output representation:

[0087]

[0088] Among them, the matrix W o It is part of the model parameters and is learned by the back propagation algorithm during training. It linearly transforms the concatenated multi-head attention output back to the original dimension d model , in order to further fuse the information of different heads and improve the expressiveness of the model. This process makes the final output representation have the same dimension as the input, ensuring the consistency and effectiveness of the model in subsequent layer processing.

[0089] Before passing the output of the Transformer encoder to the CNN module, it needs to be resized to fit the CNN input requirements. Specifically, 'Permute' is used to rearrange the dimensions of the tensor so that its shape is converted from '[batch_size, seq_len, feature_dim]' to '[batch_size, feature_dim, seq_len]'.

[0090] The CNN module consists of N identical convolutional blocks. In each convolutional block, the Conv1d layer is used to extract local features of the input data. The calculation formula for its output length is as follows:

[0091]

[0092] in, represents the length of the input feature, Indicates the number of zero elements filled at the edge of the input feature, represents the size of the convolution kernel, Indicates the step size of the convolution kernel sliding; the LeakyReLU layer introduces nonlinearity and outputs a very small linear value when the input is less than zero, thus avoiding the "death" problem of ReLU, that is, the situation where the neuron output is always zero; the AdaptiveAvgPool1d layer adjusts the feature map size to the specified output size, aggregates features and reduces the amount of calculation by averaging the local area; the Dropout layer prevents overfitting through random inactivation and improves the generalization ability of the model. After the CNN module processes the data, its output is flattened into a one-dimensional vector so that it can be spliced ​​with the features of other paths.

[0093] (2) BiLSTM+Attention

[0094] In path 2, BiLSTM captures the bidirectional dependencies in the input sequence, and the Attention mechanism further extracts the key information in the sequence; specifically, assuming that the high-dimensional features output by the ESM-2 model are ,in represents the current time step or position, represents the length of the entire sequence, that is, the total number of time steps, The ESM-2 model is the first BiLSTM calculates hidden states through forward LSTM layer and backward LSTM layer respectively, first processing the sequence , calculate the hidden state of each time step from t=1 to t=T :

[0095]

[0096] Then process the sequence , calculate the hidden state of each time step from t=T to t=1 :

[0097]

[0098] in, is the hidden state of the forward LSTM at time step t, is the forward LSTM at time step The hidden state of is the hidden state of the backward LSTM at time step t, is the backward LSTM at time step The hidden states of ; they are then concatenated into a bidirectional output representation, i.e. .

[0099] The Attention mechanism further calculates the weights of each position in the input data. The calculation formula for the attention weight is as follows:

[0100]

[0101] Here and are all time step indices, ranging from arrive . is the query vector No. elements, corresponding to the time steps; is the key vector No. elements, corresponding to the time steps; then use softmax to normalize the attention weights, and Perform weighted summation to obtain a new representation:

[0102]

[0103] in, is a value vector No. elements, corresponding to the time steps; finally, through average pooling and fully connected layers, a fixed-size feature representation is obtained.

[0104] This study uses the Optuna framework to optimize the hyperparameters of the model. Optuna is a flexible and efficient automated hyperparameter optimization software framework. By defining the objective function and optimization algorithm, it automatically searches for the best hyperparameter combination to improve model performance. Finally, based on the cross-validation accuracy and continuous adjustment, the best hyperparameter combination is determined as shown in Table 2. In order to fully utilize the advantages of the pre-trained model, a lower learning rate and appropriate weight decay are used for fine-tuning. In particular, the weight decay of the parameters including "bias" and "LayerNorm.weight" in the pre-trained model is set to 0 to avoid unnecessary regularization; the customized dual-path network structure uses a higher learning rate and weight decay to balance the needs of pre-trained model fine-tuning and new feature learning, and prevent conflicts and instability during training; in addition, by introducing a learning rate scheduler, the learning rate is automatically adjusted according to the validation loss, and the early stopping mechanism is combined to prevent overfitting and save computing resources.

[0105] Table 2

[0106]

[0107] The accuracy ACC measures the overall correct classification ability of the model; the sensitivity SN evaluates the model's ability to correctly identify positive samples, that is, the proportion of actual positive samples that are correctly identified; the specificity SP evaluates the model's ability to correctly identify negative samples, that is, the proportion of actual negative samples that are correctly identified; the Matthews correlation coefficient MCC is a comprehensive evaluation indicator that takes into account the number of true positive examples True Positive, true negative examples True Negative, false positive examples False Positive, and false negative examples False Negative, and is suitable for datasets with imbalanced categories.

[0108]

[0109]

[0110]

[0111]

[0112] Among them, TP represents the number of correctly predicted positive samples; TN represents the number of correctly predicted negative samples; FP represents the number of negative samples that are incorrectly predicted as positive samples; and FN represents the number of positive samples that are incorrectly predicted as negative samples.

[0113] Training evaluation results at different classification stages

[0114] The experimental results in this invention are all average values ​​of stratified five-fold cross validation; Figure 8As shown in the figure, in the first stage, the ACC of the model for classifying nucleic acid binding proteins and non-nucleic acid binding proteins reached 90.03%, and the MCC was 80.18%; in the second stage, the ACC of the model for classifying DNA binding proteins and RNA binding proteins reached 89.56%, and the MCC was 78.51%. At the same time, for the first stage, the AUC and AUPR were 96.39% and 96.78%, respectively; for the second stage, the AUC and AUPR were 95.77% and 93.90%, respectively. In addition, in order to reflect the comprehensive performance of multiple folds and reduce the deviation that may be caused by a single fold, this study uses the average of the confusion matrices of all folds to calculate the average confusion matrix, as shown in Fig. 9 (a) and (b). On this basis, Fig.10 (a) Fig.10 (b) Fig.10 (c) and Fig.10 (d) The prediction performance of each protein sequence in the test set is shown in the form of a scatter plot.

[0115] Ablation experiment

[0116] In order to evaluate the impact of each component in the model on the final performance, the present invention designs a series of ablation experiments. The specific method is as follows:

[0117] (1) ESM-2: Only features are extracted using ESM-2 and directly input into the classifier.

[0118] (2) E2-TC (ESM-2-Transformer-CNN): Based on ESM-2, combined with path 1 for classification.

[0119] (3) E2-BLA (ESM-2-BiLSTM-Attention): Based on ESM-2, combined with path 2 for classification.

[0120] (4) DRBP-EDP: A complete model including ESM-2, Path 1 and Path 2, integrating features extracted from different paths through feature fusion, aiming to exploit the complementary characteristics of each module.

[0121] Each method was trained on the training validation set (Train_val_data) to obtain the best model status in the two stages, and finally evaluated on the independent test set (Test_data). As shown in Table 3, in the first stage, DRBP-EDP achieved the best performance in all indicators. Compared with ESM-2, ACC increased by 2.52% and MCC increased by 5.10%, proving the effectiveness of the dual-path feature extraction and fusion strategy. In the second stage, the ACC of E2-TC was 0.86% higher than that of E2-BLA, and SP was 2.14% higher than that of DRBP-EDP. Although the two single-path neural network combination methods of E2-TC and E2-BLA achieved good results, their AUC and AUPR were not as good as DRBP-EDP, indicating that the combination of ESM-2 and dual-path neural network has certain advantages in dealing with complex protein classification tasks.

[0122] Table 3

[0123]

[0124] like Fig.13 As shown in the figure, in order to improve accessibility and usability, DRBP-EDP has developed both an executable version and a network version. Fig.13 (a), the executable version allows users to run the application locally to ensure faster large-scale classification performance, while Fig.13 The web-based version in (b) offers the convenience of cloud computing and can be accessed through any modern browser without the need for installation.

[0125] Comparative Example

[0126] In order to evaluate the performance difference of DRBP-EDP in the protein classification task; DRBP-EDP was compared with the following three models: (1) ESM-1b. This model is similar to ESM-2 used in this study, including about 650 million parameters, 33 Transformer layers and 1280 embedding dimensions; (2) E2-BGA (ESM-2-BiGRU-Attention): Based on ESM-2, the BiLSTM in path 2 is replaced by BiGRU. BiGRU has fewer parameters and lower computational complexity than BiLSTM; (3) E1b-TC-BGA (ESM-1b-Transformer-CNN-BiGRU-Attention): Based on ESM-1b, the BiLSTM in path 2 is first replaced by BiGRU, and then combined with path 1 to form a new dual-path structure for classification.

[0127] In each stage, each model is trained on the training validation set (Train_val_data) under the same conditions, and then evaluated on Train_val_data and Test_data respectively; Fig.11 As can be seen from the figures (a) and (b), each model performs well on Train_val_data, with ACC above 86%, indicating that it can fit the training data well. At the same time, the ACC difference between Train_val_data and Test_data is within 7%, indicating that no model has overfitting problems; Fig.12 (a) and (b) further demonstrate the performance of different models on Test_data, facilitating comparative analysis among multiple indicators. It can be seen that except that the SP of DRBP-EDP in the second stage is 0.74% lower than that of E2-BGA, the other performance indicators all reach the optimal values.

Claims

1. A method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network, characterized in that: include: S1: Collect protein data and perform data correction, construct data sets, obtain protein sequences, and divide the data sets into training sets, validation sets, and test sets; S2: tokenize the protein sequence as input data; S3: The ESM-2 pre-trained model extracts high-dimensional features after receiving input data; S4: The dual-path neural network extracts high-level features after receiving the high-dimensional features, wherein the dual-path neural network includes a first path and a second path; S5: high-level features are fused through the fully connected layer; S6: Pass the fused feature vector to the classification layer to generate classification results and complete the classification of DNA-binding proteins and RNA-binding proteins; the staged classification method combining ESM-2 with the dual-path neural network is called DRBP-EDP; The classification method is staged, with the first stage classifying protein sequences as either nucleic acid-binding proteins or non-nucleic acid-binding proteins, and then in the second stage further subdividing nucleic acid-binding proteins into DNA-binding proteins or RNA-binding proteins; In the S1 dataset, the ratio of nucleic acid-binding proteins to non-nucleic acid-binding proteins is 1:1, and the ratio of DNA-binding proteins to RNA-binding proteins is 3:2; In S3, the output of the last layer of the encoder is directly used for fine-tuning in the ESM-2 pre-trained model, and the embedding layer and encoder layer are frozen during model training; In S4, the first path includes the Transformer encoder and the CNN module, and the second path includes the BiLSTM and the Attention mechanism. The first path and the second path are channels for processing different features in parallel; In the first path, the Transformer encoder uses a multi-head attention mechanism to capture global dependencies. The Transformer encoder is composed of M identical layers stacked together, each of which contains two sublayers: a multi-head attention mechanism and a feedforward neural network. Each sublayer is followed by a residual connection and layer normalization. The CNN module further extracts local features and spatial information. In the second path, the BiLSTM captures bidirectional dependencies in the input sequence, and the Attention mechanism further extracts key information in the sequence. The Optuna framework was used to optimize the hyperparameters in the classification method model, and the best hyperparameter combination was determined based on cross-validation accuracy and continuous adjustments.

2. A method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network according to claim 1, characterized in that: The specific steps of S1 include: S1-1: Collect protein data from the Swiss-Prot sub-database; S1-2: Delete protein sequences containing special amino acid codes in the protein dataset; S1-3: CD-HIT tool was used to perform redundancy removal on the protein dataset and remove protein sequences with similarity greater than 40%.

3. The method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network according to claim 2, characterized in that: The protein data includes nucleic acid binding protein sequences and non-nucleic acid binding protein sequences, wherein the nucleic acid binding protein sequences include DNA binding protein sequences and RNA binding protein sequences.

4. A method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network according to claim 1, characterized in that: The specific steps of S2 include: tokenizing the protein sequence using ‘AutoTokenizer’ and converting the protein sequence into an input format that can be processed by the ESM-2 pre-trained model.

5. The method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network according to claim 4, characterized in that: The tokenization process includes an 'input_ids' tokenization process and an 'attention_mask' tokenization process.

6. The method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network according to claim 1, characterized in that: Stratified five-fold cross validation was used in the training phase of the ESM-2 pre-trained model and the dual-path neural network.

7. The method for classifying DNA-binding proteins and RNA-binding proteins based on ESM-2 and dual-path neural network according to claim 1, characterized in that: During the simulation training of the ESM-2 pre-trained model and the dual-path neural network, a data augmentation strategy was used to introduce protein sequence variants, including: (1) Insert a randomly selected amino acid at a random position in the sequence; (2) randomly select an amino acid and delete it from the sequence; (3) Randomly select two amino acids and swap their positions; (4) randomly selecting an amino acid and replacing it with another randomly selected amino acid; In addition, the data augmentation strategy incorporates a method to dynamically adjust the probability of data augmentation.

Images