Protein expression prediction method based on convolutional neural network and residual attention mechanism

Abstract

The invention provides a protein expression prediction method based on a convolutional neural network and a residual attention mechanism, and belongs to the field of bioinformatics and artificial intelligence. The method comprises the following steps: performing data cleaning and binary one-hot coding on a DNA sequence, inputting the coded DNA sequence into a feature extraction layer, and extracting local features of DNA; local features are extracted through a convolutional neural network, global features are obtained through a multi-head attention mechanism, and residual connection is combined; and finally, inputting the global features into a full connection layer to generate a protein expression prediction result. Furthermore, an SHAP interpretability method is utilized to interpret the output result of the prediction model, and the contribution of the basic group at each position in the DNA sequence to the predicted value is quantified. According to the method, deep learning and interpretability technologies are combined, the precision and credibility of protein expression prediction are remarkably improved, and the method is suitable for predicting protein expression.

Description

Technical Field

[0001] The present invention relates to the fields of bioinformatics and artificial intelligence, and in particular to a method for predicting the fluorescence value of a protein expressed by a DNA sequence based on a deep learning method. Background Art

[0002] In the field of bioengineering, especially in microbial production systems, protein expression level is one of the key factors that determine production efficiency and yield. Usually, the prediction of protein expression relies on experimental methods, such as gene cloning, protein expression detection (such as fluorescence measurement, Western blot, etc.). Although these methods are widely used, they face problems such as high cost, time consumption, and experimental errors. In addition, protein expression is also affected by many factors, such as the structural characteristics of DNA sequences, the efficiency of transcription and translation, and the adaptability of the intracellular environment. These factors make the prediction of protein expression complicated and difficult to grasp.

[0003] In order to improve the efficiency of protein expression prediction, researchers have begun to explore computational methods, especially the application of machine learning technology. Machine learning methods can predict protein expression by analyzing a large amount of gene sequence and phenotypic data and mining its inherent laws. However, existing machine learning models rely on a large amount of high-quality training data and carefully designed feature engineering, but in gene expression research, due to the scarcity and complexity of protein expression data, existing methods still face many challenges.

[0004] Many deep learning models, especially convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have achieved remarkable success in gene sequence and protein expression prediction. However, these models often rely on a large amount of high-quality training data. Previous studies have shown that deep learning models usually require tens of thousands to hundreds of thousands of samples for effective training, but in the field of protein expression prediction, it is often not feasible to obtain high-quality annotated data of this scale. The high cost of experiments and the difficulty of data collection make the scarcity of training data for certain genes or phenotypes the main factor restricting model performance. Therefore, how to design efficient training schemes and improve data utilization efficiency under the condition of insufficient data has become an urgent problem to be solved in current technology.

[0005] Existing studies have shown that the accuracy of gene expression prediction is closely related to the diversity of training data. If the data coverage is not wide, the model is prone to overfitting or only adapting to certain specific gene sequences. On the other hand, too extensive sequence coverage may lead to sparse data and affect the learning ability of the model. For protein expression prediction, it is a challenge to design a training data set that can effectively cover the sequence space and ensure data quality. Especially in high-throughput screening, how to balance the breadth and depth of the data to ensure that the data is representative and does not lose generalization ability due to overfitting is the key to improving model accuracy and stability.

[0006] In the process of protein expression prediction, obtaining effective phenotypic data (such as protein fluorescence value, yield, etc.) usually requires expensive experimental costs and time. Most existing methods rely on experimental screening to obtain a large amount of phenotypic data through high-throughput sequencing and deep mutation scanning. However, the collection of experimental data is not only time-consuming and costly, but the screening process can be very slow, especially when the amount of data is large and the phenotypic changes are small. Therefore, how to improve prediction accuracy and reduce the number and cost of experiments through computational methods has become an urgent need in the field of bioengineering.

[0007] Current deep learning models usually perform well on large data sets, but when the amount of training data is limited, the performance of the model is often severely affected. For application scenarios such as protein expression prediction, especially the optimization of gene expression systems, the amount of data is usually small, which makes model training more difficult. Existing deep learning methods lack optimization strategies for small data sets, resulting in the accuracy and stability of the model being far lower than expected when data is insufficient.

[0008] In summary, the common shortcomings of existing protein expression prediction methods are as follows: 1. Data demand and data shortage issues; 2. High experimental and computational costs; 3. The model performs poorly on small datasets. Summary of the invention

[0009] The present invention provides a protein expression prediction method based on convolutional neural network and residual attention mechanism. The method can accurately predict the fluorescence value of the protein corresponding to the gene sequence through deep learning analysis of the DNA sequence, thereby providing an effective prediction tool for optimizing the protein expression level. Compared with traditional experimental methods and classical machine learning models, the present invention significantly improves the accuracy of protein expression prediction by combining convolutional neural network (CNN) and attention mechanism in deep learning, and solves the problems of large data demand, weak model generalization ability, and difficult feature extraction in the prior art.

[0010] To solve the above problems, this application provides the following solutions: S1. Generate a coding model to encode the bases in the DNA sequence. The encoding method can convert the DNA sequence into a numerical format that can be processed by the machine learning model, and retain the order information of the bases; S2. Obtain a protein fluorescence expression value prediction model; the model includes a feature extraction layer, a main neural network training model, and a fully connected layer, wherein the main neural network training model includes a stacked convolutional neural network and a multi-head attention mechanism network; S3, using the original data as the input of the binary one-hot encoding function to obtain the encoded DNA sequence; S4, using the encoded DNA sequence as the input of the feature extraction layer to obtain the pre-trained features of DNA; S5, using the pre-trained features of the protein as the input of the main neural network training model to obtain the local features of the DNA pre-trained matrix; S6. After obtaining the global features of the DNA sequence, these features are input into the fully connected layer and a linear transformation is performed to generate the final output, which is the protein fluorescence value prediction, indicating the protein expression level; S7. Based on the trained weight network combined with the SHAP (SHapley Additive exPlanations) interpretability method, the protein expression value prediction model calculates the SHAP average value of the protein fluorescence expression value with which each feature has a causal relationship, and constructs a SHAP feature value ranking in the same order as the sample DNA sequence to show the contribution value of each position and each base to the prediction result.

[0011] Further, a preferred implementation method is provided, wherein step S1 comprises: S1.1. The collected DNA sequences need to be cleaned, including removing sequences with inconsistent lengths and sequences containing non-base characters to ensure the accuracy of the encoding process; the bases include adenine A, thymine T, cytosine C, and guanine G; S1.2. Define the encoding rules for bases: Each base in the DNA sequence is encoded using a 4-dimensional vector, and each vector corresponds to one base. In this vector, only the position corresponding to the base is "1", and the other positions are "0". The specific rules are as follows: Each base will then be converted into a length of If the length of the DNA sequence is , then after encoding, each sequence will consist of a A matrix representation of where each row corresponds to a base in the DNA sequence.

[0012] The step S4 is specifically as follows: The encoded DNA sequence input by S4.1 first undergoes a convolution operation, and then batch normalization is used to standardize the output of the convolution layer so that its mean is 0 and its variance is 1, thereby reducing the deviation of the data distribution and improving the training speed and stability of the model. The ReLU activation function is used to increase nonlinearity, thereby improving the expressiveness of the model. Finally, the size of the feature map is reduced through maximum pooling, reducing the computational complexity while retaining important local features. Its calculation expression is: Where X is the output of the feature extraction layer, is the DNA sequence that codes for , They are the weights and biases of the convolutional structure of the feature extraction layer, represents a 3×3 convolution operation, represents batch normalization, is the ReLU activation function, Represents a max pooling operation.

[0013] The step S5 is specifically as follows: The main neural network model of S5.1 includes a 2-layer convolutional structure, a 1-layer multi-head attention mechanism network and the residual connection formed by them; The calculation expression of the 2-layer convolution structure is: in is the output of the convolutional layer. is the feature extraction weight of DNA output in step S4, are the weights and biases of the first convolutional layer, are the weights and biases of the second convolutional structure, Represents a 3×3 convolution operation represents batch normalization, is the ReLU activation function. The calculation expression of multi-head attention and its residual connection is: in It is the result of adding the input and output residuals of the multi-head attention layer. Represents a multi-head attention operation.

[0014] The step S7 is specifically as follows: The interpretable method is to use the SHAP method to interpret the output of the protein expression value prediction model. Through SHAP analysis, the contribution of the base pair prediction value at each position in the DNA sequence can be quantified to provide the interpretability of the model. The first 10 DNA sequence data in the test set are selected for SHAP analysis. For the model, its input is the feature vector , the goal of SHAP is to calculate each feature Contribution to the model prediction value. The calculation formula is: in It is a feature The SHAP value of represents the marginal contribution of the feature to the model prediction value; It is a feature subset, which means the feature set that has been included in the model in the current calculation; is the total number of features; It contains features A subset of features Contribution to the model's predicted value; Does not contain features A subset of features Contribution to the model's predicted value; is a weighting factor that ensures that all possible feature orders are considered.

[0015] The beneficial effects of the present invention are: 1. Optimization of deep learning model: By combining convolutional neural network and multi-head attention mechanism, the present invention can extract local features and global features in DNA sequence, and then predict the fluorescence expression value of protein. The multi-head attention mechanism can capture the long-distance dependency in DNA sequence, effectively improving the prediction accuracy. 2. Model interpretability: Through the model interpretability provided by the SHAP method, this invention provides users and researchers with an effective tool to help interpret the internal mechanism of deep learning models and closely integrate theoretical research with practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1In one embodiment of the present invention, a processing flow chart for obtaining protein fluorescence expression prediction values ​​from DNA sequences through a feature extraction layer and a training network combining a convolutional neural network and multi-head attention.

[0018] Figure 2 In one embodiment of the present invention, a SHAP feature value ranking in the same order as the sample DNA sequence obtained by SHAP shows the contribution value of each position and each base to the prediction result. Specific implementation methods

[0019] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0020] like Figure 1 As shown, an embodiment of a protein expression prediction method based on deep learning in the embodiment of the present application, the specific steps are as follows: S1. Generate a coding model to encode the bases in the DNA sequence. The encoding method can convert the DNA sequence into a numerical format that can be processed by the machine learning model, and retain the order information of the bases. S2. Obtain a protein fluorescence expression value prediction model; the model includes a feature extraction layer, a main neural network training model, and a fully connected layer, wherein the main neural network training model includes a stacked convolutional neural network and a multi-head attention mechanism network; S3, using the original data as the input of the binary one-hot encoding function to obtain the encoded DNA sequence; S4, using the encoded DNA sequence as the input of the feature extraction layer to obtain the pre-trained features of DNA; S5, using the pre-trained features of the protein as the input of the main neural network training model to obtain the local features of the DNA pre-trained matrix; S6. After obtaining the global features of the DNA sequence, these features are input into the fully connected layer and a linear transformation is performed to generate the final output, which is the protein fluorescence value prediction, indicating the protein expression level.

[0021] Furthermore, this embodiment provides a preferred implementation method, wherein step S1 comprises: S1.1. The collected DNA sequences need to be cleaned, including removing sequences with inconsistent lengths and sequences containing non-base characters to ensure the accuracy of the encoding process; the bases include adenine A, thymine T, cytosine C, and guanine G; S1.2. Define the encoding rules for bases: Each base in the DNA sequence is encoded using a 4-dimensional vector, and each vector corresponds to one base. In this vector, only the position corresponding to the base is "1", and the other positions are "0". The specific rules are as follows: Each base will then be converted into a length of If the length of the DNA sequence is , then after encoding, each sequence will consist of a A matrix representation of where each row corresponds to a base in the DNA sequence.

[0022] The step S4 is specifically as follows: The encoded DNA sequence input by S4.1 first undergoes a convolution operation, and then batch normalization is used to standardize the output of the convolution layer so that its mean is 0 and its variance is 1, thereby reducing the deviation of the data distribution and improving the training speed and stability of the model. The ReLU activation function is used to increase nonlinearity, thereby improving the expressiveness of the model. Finally, the size of the feature map is reduced through maximum pooling, reducing the computational complexity while retaining important local features. Its calculation expression is: Where X is the output of the feature extraction layer, is the DNA sequence after coding, , They are the weights and biases of the convolutional structure of the feature extraction layer, represents a 3×3 convolution operation, represents batch normalization, is the ReLU activation function, Represents a max pooling operation.

[0023] The step S5 is specifically as follows: The main neural network model of S5.1 includes a 2-layer convolutional structure, a 1-layer multi-head attention mechanism network and the residual connection formed by them; The calculation expression of the 2-layer convolution structure is: in is the output of the convolutional layer. is the feature extraction weight of DNA output in step S4, are the weights and biases of the first convolutional layer, are the weights and biases of the second convolutional structure, Represents a 3×3 convolution operation represents batch normalization, is the ReLU activation function; The calculation expression of multi-head attention and its residual connection is: in It is the result of adding the input and output residuals of the multi-head attention layer. Represents a multi-head attention operation.

[0024] The step S7 is specifically as follows: The SHAP method is used to interpret the output of the protein expression value prediction model. Through SHAP analysis, the contribution of base pairs at each position in the DNA sequence to the predicted value can be quantified, providing the interpretability of the model. The first 10 DNA sequence data in the test set are selected for SHAP analysis; For the model, its input is the feature vector , the goal of SHAP is to calculate each feature Contribution to the model prediction value. The calculation formula is: in It is a feature The SHAP value of represents the marginal contribution of the feature to the model prediction value; It is a feature subset, which means the feature set that has been included in the model in the current calculation; is the total number of features; Contains features A subset of features Contribution to the model's predicted value; Does not contain features A subset of features Contribution to the model's predicted value; is a weighting factor that ensures that all possible feature orders are considered; In this implementation method, Figure 2 As shown, a SHAP feature value ranking obtained by SHAP in the same order as the sample DNA sequence shows the contribution value of each position and each base to the prediction result.

[0025] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. For ordinary technicians in this technical field, after knowing the contents recorded in the present invention, they can make several equivalent changes and substitutions without departing from the principle of the present invention. These equivalent changes and substitutions should also be regarded as belonging to the protection scope of the present invention.

Claims

1. A protein expression prediction method based on convolutional neural network and residual attention mechanism, characterized in that: The method comprises: S1. Generate a coding model to encode the bases in the DNA sequence. The encoding method can convert the DNA sequence into a numerical format that can be processed by the machine learning model, and retain the order information of the bases; S2. Obtain a protein fluorescence expression value prediction model; the model includes a feature extraction layer, a main neural network training model, and a fully connected layer, wherein the main neural network training model includes a stacked convolutional neural network and a multi-head attention mechanism network; S3, using the original data as the input of the binary one-hot encoding function to obtain the encoded DNA sequence; S4, using the encoded DNA sequence as the input of the feature extraction layer to obtain the pre-trained features of DNA; S5, using the pre-trained features of the protein as the input of the main neural network training model to obtain the local features of the DNA pre-trained matrix; S6. After obtaining the global features of the DNA sequence, these features are input into the fully connected layer and a linear transformation is performed to generate the final output, which is the protein fluorescence value prediction, indicating the protein expression level; S7. Based on the trained weight network combined with the SHAP (SHapley Additive exPlanations) interpretability method, the positions and bases of the DNA sequences that have a causal relationship with the final predicted structure are screened out, and efficient protein fluorescence expression is achieved based on the optimal feature subset.

2. The protein expression prediction method based on convolutional neural network and residual attention mechanism according to claim 1, characterized in that: The feature extraction network is a convolutional neural network; the feature extraction network includes a convolution layer, a batch normalization layer, a ReLU layer and a pooling layer; The step S4 is specifically as follows: The input encoded DNA sequence first undergoes a convolution operation, and then batch normalization is used to standardize the output of the convolution layer so that its mean is 0 and its variance is 1, thereby reducing the deviation of the data distribution and improving the training speed and stability of the model; Use the ReLU activation function to increase nonlinearity, thereby improving the expressiveness of the model; Finally, the maximum pooling is used to reduce the size of the feature map, reduce the computational complexity, and retain important local features. The calculation expression is: Where X is the output of the feature extraction layer, is the DNA sequence that codes for , They are the weights and biases of the convolutional structure of the feature extraction layer, represents a 3×3 convolution operation, represents batch normalization, is the ReLU activation function, Indicates that the maximum pooling operation is the ReLU activation function, Represents a max pooling operation.

3. The protein expression prediction method based on convolutional neural network and residual attention mechanism according to claim 1, characterized in that: The main neural network training model is a convolutional neural network and a multi-head attention mechanism network. The neural network includes a convolution layer, a pooling layer, a batch normalization layer, a ReLU layer, a multi-head attention layer and a residual connection; it specifically includes a 2-layer convolution structure, a 1-layer multi-head attention mechanism network and a residual connection constituted by them; The calculation expression of the 2-layer convolution structure is: in is the output of the convolutional layer. is the feature extraction weight of DNA output in step S4, are the weights and biases of the first convolutional layer, are the weights and biases of the second convolutional structure, Represents a 3×3 convolution operation represents batch normalization, is the ReLU activation function; The calculation expression of multi-head attention and its residual connection is: in It is the result of adding the input and output residuals of the multi-head attention layer. Represents a multi-head attention operation.

4. The protein expression prediction method based on convolutional neural network and residual attention mechanism according to claim 1, characterized in that: The explainable method is to use the SHAP method to calculate the SHAP average value of the protein fluorescence expression value with which each feature of the protein expression value prediction model has a causal relationship, and construct a SHAP feature value ranking in the same order as the sample DNA sequence to show the contribution value of each position and each base to the prediction result.

Images