Efficient deep learning method and system based on squeeze-excitation-network and ConNet network
By employing deep learning methods combining Squeeze-Excitation-network and ConNet networks, this approach addresses the issues of insufficient feature extraction and inadequate consideration of global information in existing technologies, achieving more efficient phosphorylation site prediction and improving model performance and interpretability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI UNIV
- Filing Date
- 2023-08-11
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies cannot effectively extract sufficient feature information and do not fully consider global information and the correlation between global and local information, resulting in insufficient phosphorylation site prediction performance.
We employ a deep learning approach based on Squeeze-Excitation-network and ConNet network. By combining adaptive embedding encoding and multiple attention mechanisms with convolutional neural networks and global residual networks, we design the DeepNet deep framework to process global and local protein sequences, extract key information, and calculate prediction probabilities.
It improves the performance and interpretability of phosphorylation site prediction, better handles ultra-long protein sequences, reduces the negative impact of imbalanced datasets, and alleviates the black box effect and gradient explosion problem.
Smart Images
Figure CN117012287B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of protein function research and experimental design technology, specifically to an efficient deep learning method and system based on Squeeze-Excitation-network and ConNet network. Background Technology
[0002] Post-translational modifications (PTMs) are reversible or irreversible covalent treatment events in the later stages of protein biosynthesis, altering protein properties through proteolysis and the addition of modifying groups. PTMs play a crucial role in many biological processes, including cell cycle regulation, DNA repair, gene activation, gene regulation, and signal transduction. With the development of modern proteomics technologies, over 400 different types of PTMs have been discovered. These include the addition of small or complex chemical groups, such as phosphorylation, ubiquitination, crotonylation, acetylation, benzoylation, or succinylation, which can occur on single or multiple amino acid residues. Protein phosphorylation is one of the most important PTMs, regulating a range of key biological processes such as cell cycle, neural activity, muscle contraction, and tumorigenesis.
[0003] Identification of protein phosphorylation sites can provide new insights into drug design. For example, phosphorylated linker regions have been found to modulate the interaction of certain drugs with human multidrug transporters, particularly at low concentrations. Historically, wet or dry methods have been largely used to predict phosphorylation sites. Edman degradation and P-labeling are typical wet methods for identifying phosphorylation sites. These traditional methods require significant manpower and resources, especially when dealing with large datasets.
[0004] Several computational tools for predicting phosphorylation sites have been developed previously. These methods are mainly based on machine learning and deep learning. For example, Chenwei et al. proposed a typical machine learning-based method called GPS 5.0. GPS 5.0 uses position weights and a scoring matrix to improve the effectiveness of predicting kinase-specific phosphorylation sites. Another example is the existing invention patent application document CN110033822A entitled "Protein Coding Method and Method and System for Predicting Protein Post-translational Modification Sites". This existing method contains the following steps: (1) Collecting modification site information: First, collect the modification site information of the target type of protein after translation; take the corresponding site of the target type of modification site on the protein as the positive site, and take other amino acid sites on the protein that are the same as the positive site as the negative site; cut the primary sequence of the protein into n amino acids upstream of the positive site or the negative site as the center, and n amino acids downstream of the center. The sequence consists of 2n+1 amino acids; n is greater than or equal to 1; all amino acid sequences containing the positive sites constitute the positive dataset, and all amino acid sequences containing the negative sites constitute the negative dataset; (2) Position weight training: The formula for scoring the similarity between each peptide in the positive dataset and the negative dataset based on position weight and amino acid substitution score in step (1) is: where: L is the length of each peptide in the positive dataset (2n+1); N is the number of peptides in the positive dataset; Tij is the amino acid at position j of peptide Ti in the positive dataset. The value of i ranges from 1 to 1 ≤ i ≤ N; Pj is the amino acid at position j of the peptide; M[Pj, Tij] is the score of amino acids Pj and Tij in the BLOSUM62 amino acid substitution matrix; Wj is the weight at position j of the peptide; each peptide in the positive and negative datasets is scored sequentially against each peptide in the positive dataset, where the peptide is not scored against itself. The initial position weight Wj is 1, and the scores of the other 2n positions in the peptide, excluding the center position, are obtained; then, the scores of these 2n positions are used to perform cross-validation using penalized logistic regression, so that the weight vector with the largest AUC value is... The weights Wj at each position in the peptide segment are composed of: (3) Encoding of the peptide segment to be encoded: The average similarity Q of the amino acid pairs between the peptide segment to be encoded and the positive dataset is: where: l is the length of the peptide segment to be encoded, j is the position of the amino acid, Cj is the number of times any amino acid pair between the peptide segment to be encoded and the positive dataset appears at position j, M is the score of the amino acid pair in the BLOSUM62 amino acid substitution matrix, and Wj is the weight of the peptide segment to be encoded at position j obtained by training in step (2); The similarity scores of all amino acid pairs between the peptide segment to be encoded and the positive dataset constitute the digital vector feature of the peptide segment to be encoded.While the aforementioned existing techniques have yielded significant improvements, like machine learning, this largely depends on the extraction of sufficiently deep features from the sample, but with limited understanding of proteins in the biological environment. Therefore, further improvements are difficult to achieve.
[0005] Furthermore, the existing technologies such as MusiteDeep, CapsNet, and DeepPSP are all based on deep learning methods. MusiteDeep is the first deep learning method for predicting phosphorylation sites, and it uses multiple convolutional neural networks for prediction. Then, CapsNet uses a capsule network and predicts phosphorylation sites based on convolutional neural networks. For example, the existing invention patent application document CN113539364A, entitled "A Method for Predicting Protein Phosphorylation Using a Deep Neural Network Framework", includes the following steps: (1) building an integrated deep neural network framework: a deep neural network framework is obtained by integrating ACNet and multi-scale CapsNet; (2) selecting a dataset: a phosphorylation dataset PhosphoData1 training set and multiple independent test sets; (3) using fusion features optimized by the information gain method as network input features; (4) using the integrated deep neural network framework built in step (1) to predict the model on the dataset; (5) setting model parameters; (6) inputting the protein sequence to be tested into the model to predict whether the protein has phosphorylation sites and their locations. DeepPSP, mentioned earlier, was the first deep learning model to consider utilizing global information, and SENet was proposed to process sequence information separately. While the aforementioned existing deep learning methods have shown relatively good performance in phosphorylation site prediction, most of these methods do not consider the importance of global information or the correlation between global and local information.
[0006] In summary, existing technologies suffer from technical problems such as the inability to extract sufficient feature information, insufficient consideration of global information, and the relationship between global and local information. Summary of the Invention
[0007] The technical problem to be solved by this invention is: how to solve the technical problems in the prior art that cannot extract sufficient feature information, do not fully consider global information, and the relationship between global information and local information.
[0008] This invention solves the above-mentioned technical problems by employing the following technical solution: an efficient deep learning method based on Squeeze-Excitation-network and ConNet network includes:
[0009] S1. Obtain the global and local protein sequences as a sample set and set the model parameters of the DeepNet deep framework.
[0010] S2. Divide the sample set according to a preset ratio to obtain the training set and validation set. Set the model architecture of the DeepNet deep framework, including: local sequence processing branch, global sequence processing branch, convolutional neural network, global residual network and ConNet architecture, to process the global protein sequence and local protein sequence to extract effective feature information. Combine negative and positive samples in the global protein sequence and local protein sequence to feed into the DeepNet deep framework for training and to fine-tune the model parameters.
[0011] S3. Adaptive word embedding encoding is performed on the global and local sequences of the protein to obtain sequence vector position data;
[0012] S4. Design local sequence processing branches and global sequence processing branches to utilize convolutional networks of different scales to extract network structure features and key information between long and short sequences from sequence vector position data, and calculate the final prediction probability accordingly.
[0013] The deep framework proposed in this invention, called DeepNet, is the first deep learning framework to use Transformer for predicting kinase proteins. It utilizes adaptive embeddings and is based on convolutional neural networks, global residual networks, and the ConNet architecture. This alternative model for predicting kinase protein phosphorylation sites offers better performance and interpretability.
[0014] In a more specific technical solution, in step S1, annotated serine S and threonine T in the global and local sequences of the protein are set as positive sites, and the remaining phosphorylation sites are set as negative sites.
[0015] In a more specific technical solution, in step S1, the length of the global protein sequence is set according to a preset length.
[0016] This invention specifies the protein sequence length as 2000 for the global input sequence, and proteins with a length less than 2000 are represented by "*", thus preserving the global information of the protein sequence as much as possible.
[0017] In a more specific technical solution, step S2 includes:
[0018] S21. Through nonlinear transformation, the feature information of the global protein sequence and the local protein sequence is mapped to a preset high-dimensional space to extract effective feature information.
[0019] S22. Divide the training set and validation set according to the DeepPSP model parameters, and adjust the hyperparameters of the DeepNet deep framework based on its performance on the pre-set validation set.
[0020] In a more specific technical solution, step S22 includes:
[0021] S221. Divide negative samples from the global protein sequence and the local protein sequence into no less than two pre-defined equal parts;
[0022] S222. Perform a combination operation on the equal portions and positive samples to obtain combined input data;
[0023] S223. Feed the combined input data into the DeepNet deep framework and repeat the training to adjust the hyperparameters based on the performance of the DeepNet deep framework on the pre-defined validation set.
[0024] This invention divides the negative samples of the input data into equal parts, with each part containing the same number of positive samples. These parts are then combined and fed sequentially into the model for repeated training. This ensures that the model is not biased towards negative samples, reducing the negative impact of imbalanced datasets. Furthermore, this invention adjusts the model's hyperparameters based on the performance on the validation set, enabling the model to fit the data correctly.
[0025] In a more specific technical solution, in step S221, each preset equal part is the same as the number of positive samples.
[0026] In a more specific technical solution, step S3 includes:
[0027] S31. Adaptive word embedding encoding of the global protein sequence and the local protein sequence to obtain the encoded sequence, wherein the sequence vector position data of the encoded sequence includes: protein word vectors and position information;
[0028] S32. During backpropagation in the DeepNet deep framework, update the parameters of each word vector.
[0029] In a more specific technical solution, step S4 includes:
[0030] S41. Utilize the convolution of the DeepNet deep framework to perform convolution calculations on the original dimensions of the data to obtain convolutional features;
[0031] S42. Based on the length of the global protein sequence and the local protein sequence, set the number of filters and the size of the convolution kernel;
[0032] S43. Add a residual network to the convolution to obtain the SENet-R module, calculate the weighting matrix of each convolution feature, and obtain the network structure features accordingly.
[0033] S44. Add a fully connected layer after convolution to obtain a ConNet module, which is used to extract the classification feature matrix and obtain key information between the global protein sequence and the local protein sequence.
[0034] This invention proposes a ConNet block that utilizes a multiple attention mechanism. When processing local features, this module determines the importance of local features at a given location by performing matrix weighting on prior information of the global sequence; similarly, when processing global features, it also weights the features of the local sequence to determine the importance of global features at that location.
[0035] This invention also utilizes CNN, ResNet, BLSTM, and SENet-R modules to better capture the potential information of protein sequences. SENet can alleviate the black box effect by helping to reveal the positions in the sequence that the module focuses on during the training step, while SENet-R integrates a residual network on this basis; ResNet can effectively alleviate the gradient explosion and overfitting problems caused by the redundancy of network parameters in deep networks.
[0036] In a more specific technical solution, step S4 also includes:
[0037] S441. Using the ConNet module, a multi-head attention mechanism is adopted to integrate the internal features of the global protein sequence and the local protein sequence.
[0038] S442. Extracting deep features using convolution;
[0039] S443. Utilize residual networks to reduce parameter redundancy in the DeepNet deep framework;
[0040] S444. Using the standard cross-entropy as the loss function, calculate the error between the predicted value and the standard value, and then determine the final prediction probability.
[0041] The module proposed in this invention connects a residual network between the data input and output, which can minimize the redundancy of the module and make it easier to process ultra-long protein sequences. The invention adds a fully connected layer after most convolution operations, which can extract various feature matrices for classification, thereby extracting sequence features more effectively.
[0042] In more specific technical solutions, efficient deep learning systems based on Squeeze-Excitation-network and ConNet networks include:
[0043] The model parameter setting module is used to obtain global and local protein sequences as a sample set to set the model parameters of the DeepNet deep framework.
[0044] The model building and parameter tuning module is used to divide the sample set according to a preset ratio to obtain the training set and validation set, set the model architecture of the DeepNet deep framework, including local sequence processing branch, global sequence processing branch, convolutional neural network, global residual network and ConNet architecture, and process the global protein sequence and local protein sequence to extract effective feature information, combine negative and positive samples in the global protein sequence and local protein sequence, and feed them into the DeepNet deep framework for training, thereby tuning the model parameters. The model building and parameter tuning module is connected to the model parameter setting module.
[0045] The embedding and encoding data module is used to perform adaptive word embedding encoding on the global and local sequences of proteins to obtain sequence vector position data. The embedding and encoding data module is connected to the model building and parameter tuning module.
[0046] The prediction result acquisition module is used to design local sequence processing branches and global sequence processing branches to utilize convolutional networks of different scales to extract network structure features and key information between long and short sequences from sequence vector position data, and to calculate the final prediction probability. The prediction result acquisition module is connected to the embedded encoding data module.
[0047] Compared to existing technologies, this invention offers the following advantages: The proposed deep framework, DeepNet, is the first deep learning framework to use Transformer for predicting kinase proteins. It utilizes adaptive embeddings and is based on convolutional neural networks, global residual networks, and the ConNet architecture. This alternative model for predicting kinase protein phosphorylation sites exhibits better performance and interpretability.
[0048] This invention specifies the protein sequence length as 2000 for the global input sequence, and proteins with a length less than 2000 are represented by "*", thus preserving the global information of the protein sequence as much as possible.
[0049] This invention divides the negative samples of the input data into equal parts, with each part containing the same number of positive samples. These parts are then combined and fed sequentially into the model for repeated training. This ensures that the model is not biased towards negative samples, reducing the negative impact of imbalanced datasets. Furthermore, this invention adjusts the model's hyperparameters based on the performance on the validation set, enabling the model to fit the data correctly.
[0050] This invention proposes a ConNet block that utilizes a multiple attention mechanism. When processing local features, this module determines the importance of local features at a given location by performing matrix weighting on prior information of the global sequence; similarly, when processing global features, it also weights the features of the local sequence to determine the importance of global features at that location.
[0051] This invention also utilizes CNN, ResNet, BLSTM, and SENet-R modules to better capture the potential information of protein sequences. SENet can alleviate the black box effect by helping to reveal the positions in the sequence that the module focuses on during the training step, while SENet-R integrates a residual network on this basis; ResNet can effectively alleviate the gradient explosion and overfitting problems caused by the redundancy of network parameters in deep networks.
[0052] This invention connects a residual network between the data input and output of the module, which can minimize the redundancy of the module and make it easier to process ultra-long protein sequences. This invention adds a fully connected layer after most convolutional operations, which can extract various feature matrices for classification, thereby extracting sequence features more effectively.
[0053] This invention solves the technical problems in the prior art, such as the inability to extract sufficient feature information, insufficient consideration of global information, and the relationship between global and local information. Attached Figure Description
[0054] Figure 1 This is a schematic diagram illustrating the basic steps of the efficient deep learning method based on Squeeze-Excitation-network and ConNet network in Embodiment 1 of the present invention.
[0055] Figure 2 This is a schematic diagram of the DeepNet network model framework of Embodiment 1 of the present invention;
[0056] Figure 3 This is a schematic diagram of the data flow processing of the SENet-R network model in Embodiment 1 of the present invention;
[0057] Figure 4 This is a schematic diagram of the data flow processing of the ResNet network model in Embodiment 1 of the present invention. Detailed Implementation
[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0059] Example 1
[0060] like Figure 1As shown, the efficient deep learning method based on Squeeze-Excitation-network and ConNet network provided by this invention includes the following basic steps:
[0061] S1. Set the model parameters for the efficient deep framework of Squeeze-Excitation-network and ConNet network;
[0062] In this embodiment, commonly used protein phosphorylation site data typically come from sources including but not limited to: SWISSPROT, dbPTM, phosphoELM, and PhosphoSitePLUS.
[0063] In this embodiment, the kinase dataset is from DeepPSP by Lei et al. The DeepPSP database includes two overlapping datasets: one containing eight phosphorylation sites for different kinases, and the other containing a dataset of all phosphorylation sites for 12,238 protein sequences. In this embodiment, because the dataset containing all site information is too large, for better identification, this embodiment begins by studying each kinase-specific dataset separately.
[0064] In this embodiment, to test the model's ability to identify individual kinase phosphorylation sites, one model was prepared for each kinase-specific site dataset. For fairness, the same partitioning as DeepPSP was used, i.e., the training and test sets were split in a 9:1 ratio. In this embodiment, annotated serine (S) and threonine (T) residues in the protein sequence were considered positive sites, otherwise they were considered negative sites. The model used two protein sequences as inputs to each of the two branches, including a global protein sequence and a local sequence. For the global input sequence, to preserve the global information of the protein sequence as much as possible, the protein sequence length was specified as 2000, and proteins shorter than 2000 were marked with "*". The local sequence length was specified as 51, with a radius of 25 peptide bonds above and below the phosphorylation site. Phosphorylation information is shown in Table I:
[0065] Table I summarizes the phosphorylation data used in this paper.
[0066]
[0067]
[0068] S2. Set up the model architecture, process local and global sequences, extract effective feature information, combine negative samples with positive samples and feed them into the model for training to ensure that the model can fit normally.
[0069] In this embodiment, the model is divided into two branches to process local and global sequences respectively. A series of nonlinear transformations are used to map the features to a high-dimensional space, thereby extracting effective feature information.
[0070] In most deep learning models, the strategy is to use the original dataset, divide it into training and validation sets in a certain ratio, and adjust the hyperparameters based on the model's performance on the validation set to prevent overfitting. For fairness, this embodiment uses the same ratio as DeepPSP, namely 9:1.
[0071] In this embodiment, in order to reduce the negative impact of imbalanced datasets, the negative samples of the input data are divided into a certain number of equal parts, each of which has the same number of positive samples. These parts are then combined and fed into the model in sequence for repeated training, thereby ensuring that the model is not biased towards negative samples. Based on this, the hyperparameters of the model are adjusted according to the performance of the validation set so that the model can fit the data normally.
[0072] S3. Adaptive word embedding encoding for two protein sequences;
[0073] Adaptive word embedding encoding is performed on two protein sequences. The encoded sequences will contain the word vectors and positional information of the proteins, and the parameters of each word vector will be updated during backpropagation of the neural network.
[0074] S4. Design two branches of the DeepNet network model;
[0075] In this embodiment, the proposed DeepNet consists of two branches, which are used to process local protein sequences at phosphorylation sites and corresponding global protein sequences, respectively. In this embodiment, the modules for global protein sequences and local protein sequences have the same composition.
[0076] S5. Use convolutional networks of different scales to extract network structure features and key information between long and short sequences, and calculate the final prediction probability.
[0077] like Figure 3 As shown, in this embodiment, convolutional networks of different scales are used to extract initial features more effectively. First, convolution calculations are performed on the original dimensions of the data, with a filter set to 100. To better extract features from sequences of different lengths, the kernel size is set to 10 and 1 for long sequences and local sequences, respectively. Then, to facilitate subsequent processing, a residual network is added to the convolutions to calculate the weighting matrix of each convolutional feature, thereby obtaining more accurate feature information. SENet-R is based on SENet and incorporates a residual network (such as...). Figure 3The first SENet-R is used to optimize the convolutional feature map, and then the second SENet-R is used to receive the transposed feature map to calculate the contribution value of peptide bonds near each potential site, which can better extract network structure features. In this embodiment, BLSTM is used to learn sequence dependencies, and ConNet layers are used to extract key information between long and short sequences. At the same time, the SENet-R in the branches can effectively alleviate the black box effect of the neural network. The Fc layer is used to calculate the final prediction probability. A linear rectified function is used as the activation function of the module, as shown below:
[0078]
[0079] In the above formula, x is the input signal.
[0080] In this embodiment, the ConNet module, adapted from Conformer, is used. This is a module for processing long sequences of speech signals, which combines convolutional neural networks and transformers to efficiently process an audio sequence with a large number of parameters.
[0081] like Figure 4 As shown, in this embodiment, the internal structure of the ConNet module is modified: a residual network is connected between the data input and output to minimize the module's redundancy, making it easier to handle ultra-long protein sequences; a fully connected layer is added after the convolution operation to extract various feature matrices for classification, thus more effectively extracting sequence features. This module is named ConNet. It primarily employs a multi-head attention mechanism, first fusing the internal features of long and short sequences, then using a convolutional module to extract deeper features, while simultaneously utilizing a residual network to reduce parameter redundancy. The error between the predicted value and the standard value is calculated using standard cross-entropy as the loss function, as shown below:
[0082]
[0083] In the above formula, p represents the probability that the model identifies the current protein site as a positive site, and y refers to the label of the sample. When p is closer to 1, the prediction result is closer to the true label, that is, the prediction loss of the model is smaller.
[0084] Example 2
[0085] In this embodiment, the area under the reception operation feature curve and the area under the precision-recall curve are used as metrics to evaluate the model's performance. Four metrics, including precision (Pre), recall (Re), and Matthews correlation coefficient (MCC), are used to evaluate the method's performance, defined as follows:
[0086]
[0087]
[0088]
[0089]
[0090] Among them, true positives (TP) are the number of correctly predicted protein phosphorylated sequences, true negatives (TN) are the number of correctly predicted non-phosphorylated sequences, false negatives (FN) are the number of phosphorylated sequences that were incorrectly predicted as non-phosphorylated, and false positives (FP) are the number of non-phosphorylated sequences that were incorrectly predicted as phosphorylated.
[0091] This invention proposes a novel neural network model, DeepNet, for predicting multiple phosphorylation sites. The model consists of two identical main branches, one for processing the global protein sequence and the other for processing local sequences. Each branch utilizes CNN, RNN, BLSTM, ConNet, and fully connected layers. ConNet, proposed for more efficient extraction of fusion features from the global sequence and short local sequences, is the first Transformer network module specifically designed for extracting phosphorylation sites. Results show that the proposed model achieves the best performance in predicting phosphorylation sites.
[0092] In this embodiment, a learning strategy combining bootstrapping and ensemble learning is used for imbalanced datasets. This method can not only effectively mitigate the impact of imbalanced datasets to a certain extent, but also further improve the prediction performance of phosphorylation sites through a simple logistic regression model.
[0093] This invention also discusses the impact of different encoding methods on the model, and the results show that adaptive embedding encoding is the most suitable for phosphorylation site prediction.
[0094] Furthermore, this invention investigates the impact of global features on prediction performance by not using global features and instead having ConNet fuse only local sequence features. The results show that global sequence features help the model better learn the prior distribution of phosphorylation sites, thereby improving the model's prediction performance.
[0095] The DeepNet neural network model provided by this invention performs exceptionally well in predicting phosphorylation sites, effectively helping medical laboratory researchers discover and identify new phosphorylation sites, and providing valuable assistance for future medical research and development.
[0096] In summary, the deep framework proposed in this invention, called DeepNet, is the first deep learning framework to use Transformer for predicting kinase proteins. It utilizes adaptive embeddings and is based on convolutional neural networks, residual networks, and the ConNet architecture. This alternative model for predicting kinase protein phosphorylation sites offers better performance and interpretability.
[0097] This invention specifies the protein sequence length as 2000 for the global input sequence, and proteins with a length less than 2000 are represented by "*", thus preserving the global information of the protein sequence as much as possible.
[0098] This invention divides the negative samples of the input data into equal parts, with each part containing the same number of positive samples. These parts are then combined and fed sequentially into the model for repeated training. This ensures that the model is not biased towards negative samples, reducing the negative impact of imbalanced datasets. Furthermore, this invention adjusts the model's hyperparameters based on the performance on the validation set, enabling the model to fit the data correctly.
[0099] This invention proposes a ConNet block that utilizes a multiple attention mechanism. When processing local features, this module determines the importance of local features at a given location by performing matrix weighting on prior information of the global sequence; similarly, when processing global features, it also weights the features of the local sequence to determine the importance of global features at that location.
[0100] This invention also utilizes CNN, ResNet, BLSTM, and SENet-R modules to better capture the potential information of protein sequences. SENet can alleviate the black box effect by helping to reveal the positions in the sequence that the module focuses on during the training step, while SENet-R integrates a residual network on this basis; ResNet can effectively alleviate the gradient explosion and overfitting problems caused by the redundancy of network parameters in deep networks.
[0101] This invention connects a residual network between the data input and output of the ConNet module, which can minimize the redundancy of the module and make it easier to process ultra-long protein sequences. This invention adds a fully connected layer after the convolution operation, which can extract various feature matrices for classification, thereby extracting sequence features more effectively.
[0102] This invention solves the technical problems in the prior art, such as the inability to extract sufficient feature information, insufficient consideration of global information, and the relationship between global and local information.
[0103] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An efficient deep learning method based on Squeeze-Excitation-network and ConNet network, characterized in that, The method includes: S1. Obtain the global and local protein sequences as a sample set and set the model parameters of the DeepNet deep framework. S2. Divide the sample set according to a preset ratio to obtain the training set and validation set. Set the model architecture of the DeepNet deep framework, including: local sequence processing branch, global sequence processing branch, convolutional neural network, SENet-R network and ConNet network, to process the global protein sequence and local protein sequence to extract effective feature information. Combine negative samples and positive samples in the global protein sequence and local protein sequence to feed into the DeepNet deep framework for training and to fine-tune the model parameters. S3. Adaptive word embedding encoding is performed on the global and local sequences of the protein to obtain sequence vector position data; S4. Design local sequence processing branches and global sequence processing branches to utilize convolutional networks of different scales to extract network structure features and key information between long and short sequences from sequence vector position data, and calculate the final phosphorylation site prediction probability accordingly. Convolutional computations are performed on the original dimensions of the data, with a filter set to 100. For long sequences and local sequences, the kernel size is set to 10 and 1, respectively. A residual network is added to the convolutions to calculate the weighted matrix of each convolutional feature and obtain feature information. The SENet-R network, based on SENet, integrates a residual network. The first SENet-R network optimizes the convolutionally processed feature map, and the second SENet-R network receives the transposed feature map, calculating the contribution value of peptide bonds near each potential site and extracting network structure features. BLSTM is used to learn sequence dependencies, and the ConNet network extracts key information between long and short sequences. The Fc layer is used to calculate the final prediction probability. The ConNet network is adapted from Conformer. The ConNet network adopts a multi-head attention mechanism to integrate the internal features of the global sequence and local sequence of the protein. A residual network is connected between the data input and output. The residual network is used to reduce the parameter redundancy of the DeepNet deep framework. A fully connected layer is added after the convolution operation, and the error between the predicted value and the standard value is calculated using the standard cross-entropy as the loss function.
2. The efficient deep learning method based on Squeeze-Excitation-network and ConNet network according to claim 1, characterized in that, In step S1, the annotated serine S and threonine T sites in the global sequence and local sequence of the protein are set as positive sites, and the remaining phosphorylation sites are set as negative sites.
3. The efficient deep learning method based on Squeeze-Excitation-network and ConNet network according to claim 1, characterized in that, In step S1, the length of the global protein sequence is set according to a preset length.
4. The efficient deep learning method based on Squeeze-Excitation-network and ConNet network according to claim 1, characterized in that, Step S2 includes: S21. Through nonlinear transformation, the feature information of the global sequence of the protein and the local sequence of the protein is mapped to a preset high-dimensional space to extract the effective feature information. S22. According to the DeepPSP model parameters, divide the training set and the validation set, and adjust the hyperparameters of the DeepNet deep framework based on the performance of the DeepNet deep framework in the preset validation set.
5. The efficient deep learning method based on Squeeze-Excitation-network and ConNet network according to claim 4, characterized in that, Step S22 includes: S221. Divide the negative samples in the global protein sequence and the local protein sequence into no less than two pre-set equal parts; S222. Perform a combination operation on the equal portions and the positive samples to obtain combined input data; S223. The combined input data is fed into the DeepNet deep framework for repeated training, so as to adjust the hyperparameters according to the performance of the DeepNet deep framework on the preset validation set.
6. The efficient deep learning method based on Squeeze-Excitation-network and ConNet network according to claim 5, characterized in that, In step S221, each of the preset equal parts is the same as the number of positive samples.
7. The efficient deep learning method based on Squeeze-Excitation-network and ConNet network according to claim 1, characterized in that, Step S3 includes: S31. Adaptive word embedding encoding is performed on the global sequence of the protein and the local sequence of the protein to obtain the encoded sequence, wherein the sequence vector position data of the encoded sequence includes: protein word vectors and position information; S32. During backpropagation of the DeepNet deep framework, update the parameters of each word vector.
8. A high-efficiency deep learning system based on Squeeze-Excitation-network and ConNet network, used to execute the high-efficiency deep learning method based on Squeeze-Excitation-network and ConNet network as described in any one of claims 1 to 7, characterized in that, The system includes: The model parameter setting module is used to obtain global and local protein sequences as a sample set to set the model parameters of the DeepNet deep framework. The model construction and parameter tuning module is used to divide the sample set according to a preset ratio to obtain a training set and a validation set, set the model architecture of the DeepNet deep framework, including a local sequence processing branch, a global sequence processing branch, a convolutional neural network, a SENet-R network, and a ConNet network, and process the global protein sequence and the local protein sequence to extract effective feature information, combine negative samples and positive samples in the global protein sequence and the local protein sequence, and feed them into the DeepNet deep framework for training to tune the model parameters. The model construction and parameter tuning module is connected to the model parameter setting module. An embedding encoding data module is used to perform adaptive word embedding encoding on the global sequence and local sequence of the protein to obtain sequence vector position data. The embedding encoding data module is connected to the model construction and parameter tuning module. The prediction result acquisition module is used to design the local sequence processing branch and the global sequence processing branch, so as to use convolutional networks of different scales to extract network structure features and key information between long and short sequences from the sequence vector position data, and calculate the final phosphorylation site prediction probability accordingly. The prediction result acquisition module is connected to the embedding encoding data module.