Protein expression prediction method based on convolutional neural network and residual attention mechanism

Abstract

The present invention provides a protein expression prediction method based on a convolutional neural network and a residual attention mechanism, which belongs to the fields of bioinformatics and artificial intelligence. The method comprises the following steps: performing data cleaning and binary one-hot encoding on a DNA sequence, inputting the encoded DNA sequence into a feature extraction layer to extract local features of the DNA; extracting local features through a convolutional neural network, obtaining global features through a multi-head attention mechanism, and combining residual connections; finally, inputting the global features into a fully connected layer to generate protein expression prediction results. Furthermore, the SHAP interpretability method is used to interpret the output results of the prediction model and quantify the contribution of base pairs at each position in the DNA sequence to the predicted value. The present invention combines deep learning with interpretability technology to significantly improve the accuracy and credibility of protein expression prediction and is suitable for predicting protein expression.

Description

Technical Field

[0001] The present invention relates to the fields of bioinformatics and artificial intelligence, and in particular to a method for predicting the fluorescence value of a protein expressed by a DNA sequence based on a deep learning method. Background Art

[0002] In the field of bioengineering, especially in microbial production systems, protein expression levels are one of the key factors determining production efficiency and yield. Traditionally, protein expression prediction relies on experimental methods such as gene cloning and protein expression assays (e.g., fluorescence assays, Western blots, etc.). Although widely used, these methods face challenges such as high cost, time consumption, and experimental error. Furthermore, protein expression is influenced by a variety of factors, such as the structural characteristics of the DNA sequence, the efficiency of transcription and translation, and the adaptability of the intracellular environment. These factors make protein expression prediction complex and difficult to grasp.

[0003] To improve the efficiency of protein expression prediction, researchers have begun exploring computational methods, particularly machine learning. Machine learning methods analyze large amounts of genetic sequence and phenotypic data, uncovering underlying patterns and predicting protein expression. However, existing machine learning models rely on large amounts of high-quality training data and carefully designed feature engineering. However, in gene expression research, existing methods still face numerous challenges due to the scarcity and complexity of protein expression data.

[0004] Many deep learning models, particularly convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have achieved remarkable success in gene sequence and protein expression prediction. However, these models often rely on large amounts of high-quality training data. Previous studies have shown that deep learning models typically require tens of thousands to hundreds of thousands of samples for effective training. However, in the field of protein expression prediction, obtaining such large amounts of high-quality annotated data is often infeasible. High experimental costs and difficulties in data collection make the scarcity of training data for certain genes or phenotypes a major factor limiting model performance. Therefore, how to design efficient training schemes and improve data utilization efficiency under data-scarce conditions has become a pressing issue in current technology.

[0005] Existing research indicates that the accuracy of gene expression prediction is closely related to the diversity of training data. If data coverage is limited, the model is prone to overfitting or becoming limited to specific gene sequences. On the other hand, overly extensive sequence coverage can lead to data sparsity, which can impair the model's learning ability. For protein expression prediction, designing training datasets that effectively cover the sequence space while ensuring data quality is a challenge. In high-throughput screening, in particular, balancing data breadth and depth—ensuring that the data is representative and does not lose generalization ability due to overfitting—is key to improving model accuracy and stability. In protein expression prediction, obtaining valid phenotypic data (such as protein fluorescence values ​​and yield) often requires expensive experimental costs and time. Existing methods often rely on experimental screening, acquiring large amounts of phenotypic data through high-throughput sequencing and deep mutation scanning. However, experimental data acquisition is time-consuming and costly, and the screening process can be very slow, especially when the data volume is large and the phenotypic variation is small. Therefore, improving prediction accuracy and reducing the number and cost of experiments through computational methods have become urgent needs in the field of bioengineering.

[0006] Current deep learning models typically perform well on large datasets, but performance is often severely impacted when the amount of training data is limited. For protein expression prediction applications, particularly optimization of gene expression systems, data volumes are often limited, making model training more challenging. Existing deep learning methods lack optimization strategies for small datasets, resulting in model accuracy and stability far below expectations when data is insufficient.

[0007] In summary, the common shortcomings of existing protein expression prediction methods are as follows:

[0008] 1. Data demand and data shortage issues;

[0009] 2. High experimental and computational costs;

[0010] 3. The model performs poorly on small datasets. Summary of the Invention

[0011] This paper provides a protein expression prediction method based on a convolutional neural network (CNN) and a residual attention mechanism. This method uses deep learning analysis of DNA sequences to accurately predict the fluorescence value of proteins corresponding to gene sequences, thereby providing an effective prediction tool for optimizing protein expression levels. Compared with traditional experimental methods and classical machine learning models, this paper significantly improves the accuracy of protein expression prediction by combining the convolutional neural network (CNN) and the attention mechanism in deep learning. It also addresses the problems of existing technologies such as large data requirements, weak model generalization ability, and difficulty in feature extraction.

[0012] To solve the above problems, this application provides the following solutions:

[0013] S1. Encode the bases in the DNA sequence based on a DNA encoder. This encoding method can convert the DNA sequence into a numerical format that can be processed by the machine learning model, while retaining the order information of the bases;

[0014] S2. Obtaining a protein fluorescence expression value prediction model; the model includes a feature extraction layer, a main neural network training model, and a fully connected layer, wherein the main neural network training model includes a stacked convolutional structure and a multi-head attention mechanism network;

[0015] S3, using the original data as the input of the DNA encoder to obtain the encoded DNA sequence matrix;

[0016] S4. Using the encoded DNA sequence matrix as the input of the feature extraction layer to obtain the pre-trained features of the DNA sequence;

[0017] S5. Using the pre-trained features of the DNA sequence as input to the main neural network training model to obtain global features of the pre-trained features of the DNA sequence;

[0018] S6. After obtaining the global features of the pre-trained features of the DNA sequence, these features are input into the fully connected layer and a linear transformation is performed to generate the final output, which is the protein fluorescence value prediction, indicating the expression level of the protein.

[0019] S7. Based on the trained weight network and the SHAP (SHapley Additive exPlanations) interpretability method, the protein expression value prediction model calculates the SHAP average value of the protein fluorescence expression value with which each feature has a causal relationship. A SHAP feature value ranking with the same order as the sample DNA sequence is constructed to show the contribution value of each position and each base to the prediction result.

[0020] Furthermore, a preferred implementation method is provided, wherein step S1 comprises:

[0021] S1.1. The collected DNA sequences need to be cleaned, including removing sequences of inconsistent lengths and sequences containing non-base characters to ensure the accuracy of the encoding process. The bases include adenine A, thymine T, cytosine C, and guanine G.

[0022] S1.2. Define the encoding rules for bases: Each base in the DNA sequence is encoded using a 4-dimensional vector, with each vector corresponding to one base. In this vector, only the position corresponding to the base is "1", and the other positions are "0". The specific rules are as follows:

[0023] A→[1,0,0,0], T→[0,1,0,0], C→[0,0,1,0], G→[0,0,0,1], thus each base will be converted into a vector of length 4. If the length of the DNA sequence is n, then after encoding, each sequence will be represented by a matrix of size n×4, where each row corresponds to a base in the DNA sequence.

[0024] The step S4 is specifically as follows:

[0025] The encoded DNA sequence matrix input in S4.1 first undergoes a convolution operation. Batch normalization is then used to standardize the output of the convolutional layer to a mean of 0 and a variance of 1, thereby reducing the deviation of the data distribution and improving the training speed and stability of the model. The ReLU activation function is used to increase nonlinearity, thereby improving the expressiveness of the model. Finally, max pooling is used to reduce the size of the feature map, reducing computational complexity while retaining important local features. The calculation expression is:

[0026] X=MaxPool(ReLU(BN(Conv3×3(X DNA ,ω feature ,b feature ))))

[0027] Where X is the output of the feature extraction layer, X DNA is the encoded DNA sequence matrix, ω feature ,b feature are the weights and biases of the convolutional structure of the feature extraction layer, Conv3×3(·) represents a 3×3 convolution operation, BN(·) represents batch normalization, ReLU(·) is the ReLU activation function, and MaxPool(·) represents the maximum pooling operation.

[0028] The step S5 is specifically as follows:

[0029] The main neural network training model of S5.1 includes a 2-layer convolutional structure, a 1-layer multi-head attention mechanism network and the residual connection formed by them;

[0030] The calculation expression of the 2-layer convolution structure is:

[0031] S conv =BN(Conv3×3(ReLU(BN(Conv3×3(X,ω1,b1))),ω2,b2))

[0032] Among them S convis the result of the convolutional layer output, X is the pre-trained feature of DNA output in step S4, ω1, b1 are the weight and bias of the first convolutional layer, ω2, b2 are the weight and bias of the second convolutional layer, Conv3×3(·) represents a 3×3 convolution operation, BN(·) represents batch normalization, and ReLU(·) is the ReLU activation function.

[0033] The calculation expression of multi-head attention and its residual connection is:

[0034] S residual =S conv +Attn(S conv )

[0035] Among them S residual It is the result of adding the input and output residuals of the multi-head attention layer, and Attn(·) represents the multi-head attention operation.

[0036] The step S7 is specifically as follows:

[0037] The interpretability method described above uses the SHAP method to interpret the output of the protein expression prediction model. SHAP analysis quantifies the contribution of each base pair in the DNA sequence to the predicted value, improving the interpretability of the model. The first 10 DNA sequences in the test set were selected for SHAP analysis.

[0038] For the model, its input is the feature vector x=[x1,x2,…,x n ], the goal of SHAP is to calculate for each feature x i Contribution to the model prediction value. Its calculation formula is:

[0039]

[0040] where Φ i is feature x i The SHAP value of the feature represents the marginal contribution of the feature to the model prediction value; S is the feature subset, which represents the feature set that has been included in the model in the current calculation; M is the total number of features; f(S∪{i}) is the number of features including feature x i The contribution of the feature subset S to the model prediction value; f(S) is the contribution of the feature subset S that does not contain feature x i The contribution of the feature subset S to the model prediction value; is a weighting factor that ensures that all possible feature orders are considered.

[0041] The beneficial effects of the present invention are:

[0042] 1. Optimization of Deep Learning Models: By combining convolutional neural networks with a multi-head attention mechanism, this paper can extract both local and global features from DNA sequences to predict protein fluorescence expression values. The multi-head attention mechanism can capture long-range dependencies in DNA sequences, effectively improving prediction accuracy.

[0043] 2. Model interpretability: Through the model interpretability provided by the SHAP method, this invention provides users and researchers with an effective tool to help interpret the internal mechanisms of deep learning models and closely integrate theoretical research with practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0045] Figure 1 In one embodiment of the present invention, a flowchart of a process for obtaining protein fluorescence expression prediction values ​​from DNA sequences by using a feature extraction layer and a training network combining a convolutional neural network and multi-head attention;

[0046] Figure 2 In one embodiment of the present invention, a SHAP feature value ranking obtained by SHAP in the same order as the sample DNA sequence shows the contribution value of each position and each base to the prediction result.

[0047] Specific implementation method The specific implementation method of the present invention is described below to facilitate technical personnel in this technical field to understand the present invention, but it should be clear that the present invention is not limited to the scope of the specific implementation method. For ordinary technical personnel in this technical field, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

[0048] like Figure 1 As shown in the embodiment of the present application, an embodiment of a protein expression prediction method based on deep learning is shown in the embodiment of the present application, and the specific steps are as follows

[0049] S1. Encode the bases in the DNA sequence using a DNA encoder. This encoding method can convert the DNA sequence into a numerical format that can be processed by the machine learning model while preserving the order of the bases.

[0050] S2. Obtaining a protein fluorescence expression value prediction model; the model includes a feature extraction layer, a main neural network training model, and a fully connected layer, wherein the main neural network training model includes a stacked convolutional structure and a multi-head attention mechanism network;

[0051] S3, using the original data as the input of the DNA encoder to obtain the encoded DNA sequence matrix;

[0052] S4. Using the encoded DNA sequence matrix as the input of the feature extraction layer to obtain the pre-trained features of the DNA sequence;

[0053] S5. Using the pre-trained features of the DNA sequence as input to the main neural network training model to obtain global features of the pre-trained features of the DNA sequence;

[0054] S6. After obtaining the global features of the pre-trained features of the DNA sequence, these features are input into the fully connected layer and a linear transformation is performed to generate the final output, which is the protein fluorescence value prediction, indicating the expression level of the protein.

[0055] S7. Based on the trained weight network and the SHAP (SHapley Additive exPlanations) interpretability method, the positions and bases of the DNA sequences that have a causal relationship with the final predicted structure are screened out, and protein fluorescence expression is achieved based on the optimal feature subset.

[0056] Furthermore, this embodiment provides a preferred implementation method, wherein step S1 includes:

[0057] S1.1. The collected DNA sequences need to be cleaned, including removing sequences of inconsistent lengths and sequences containing non-base characters to ensure the accuracy of the encoding process. The bases include adenine A, thymine T, cytosine C, and guanine G.

[0058] S1.2. Define the encoding rules for bases: Each base in the DNA sequence is encoded using a 4-dimensional vector, with each vector corresponding to one base. In this vector, only the position corresponding to the base is "1", and the other positions are "0". The specific rules are as follows:

[0059] A→[1,0,0,0], T→[0,1,0,0], C→[0,0,1,0], G→[0,0,0,1], thus each base will be converted into a vector of length 4. If the length of the DNA sequence is n, then after encoding, each sequence will be represented by a matrix of size n×4, where each row corresponds to a base in the DNA sequence.

[0060] The step S4 is specifically as follows:

[0061] The encoded DNA sequence matrix input by S4.1 first undergoes a convolution operation, and then batch normalization is used to standardize the output of the convolution layer to a mean of 0 and a variance of 1, thereby reducing the deviation of the data distribution and improving the training speed and stability of the model. The ReLU activation function is used to increase nonlinearity, thereby improving the expressiveness of the model. Finally, the size of the feature map is reduced through maximum pooling, reducing computational complexity while retaining important local features. Its calculation expression is

[0062] X=MaxPool(ReLU(BN(Conv3×3(X DNA ,ω feature ,b feature ))))

[0063] Where X is the output of the feature extraction layer, X DNA is the encoded DNA sequence matrix, ω feature ,b feature They are the weights and biases of the convolutional structure of the feature extraction layer, Conv3×3(·) represents a 3×3 convolution operation, BN(·) represents batch normalization, ReLU(·) is the ReLU activation function, and MaxPool(·) represents the maximum pooling operation.

[0064] The step S5 is specifically as follows:

[0065] The main neural network training model of S5.1 includes a 2-layer convolutional structure, a 1-layer multi-head attention mechanism network and the residual connection formed by them;

[0066] The calculation expression of the 2-layer convolution structure is:

[0067] S conv =BN(Conv3×3(ReLU(BN(Conv3×3(X,ω1,b1))),ω2,b2))

[0068] Among them S conv is the result of the convolutional layer output, X is the pre-trained feature of DNA output in step S4, ω1, b1 are the weight and bias of the first convolutional structure, ω2, b2 are the weight and bias of the second convolutional structure, Conv3×3(·) represents the 3×3 convolution operation, BN(·) represents batch normalization, and ReLU(·) is the ReLU activation function

[0069] The calculation expression of multi-head attention and its residual connection is:

[0070] S residual =S conv +Attn(S conv )

[0071] Among them S residual It is the result of adding the input and output residuals of the multi-head attention layer, and Attn(·) represents the multi-head attention operation.

[0072] The step S7 is specifically as follows:

[0073] The SHAP method was used to interpret the output of the protein expression prediction model. SHAP analysis quantifies the contribution of each base position in the DNA sequence to the predicted value, improving the interpretability of the model. The first 10 DNA sequences in the test set were selected for SHAP analysis.

[0074] For the model, its input is the feature vector x=[x1,x2,…,x n ], the goal of SHAP is to calculate for each feature x i Contribution to the model prediction value. Its calculation formula is:

[0075]

[0076] where Φ i is feature x i The SHAP value of the feature represents the marginal contribution of the feature to the model prediction value; S is the feature subset, which represents the feature set that has been included in the model in the current calculation; M is the total number of features; f(S∪{i}) is the number of features including feature x i The contribution of the feature subset S to the model prediction value; f(S) is the contribution of the feature subset S that does not contain feature x i The contribution of the feature subset S to the model prediction value; is a weighting factor that ensures that all possible feature orders are considered.

[0077] In this implementation method, Figure 2 As shown in FIG, a SHAP feature value ranking obtained by SHAP in the same order as the sample DNA sequence shows the contribution value of each position and each base to the prediction result.

[0078] The above describes the embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. After knowing the contents described in the present invention, ordinary technicians in this technical field can make several equivalent changes and substitutions without departing from the principles of the present invention. These equivalent changes and substitutions should also be regarded as falling within the scope of protection of the present invention.

Claims

1. A protein expression prediction method based on convolutional neural network and residual attention mechanism, characterized by: The method comprises: S1. Encode the bases in the DNA sequence based on the DNA encoder. The encoding method can convert the DNA sequence into a numerical format that can be processed by the machine learning model, and retain the order information of the bases; S2. Obtaining a protein fluorescence expression value prediction model; the model includes a feature extraction layer, a main neural network training model, and a fully connected layer, wherein the main neural network training model includes a stacked convolutional structure and a multi-head attention mechanism network; S3, using the original data as the input of the DNA encoder to obtain the encoded DNA sequence matrix; S4. Using the encoded DNA sequence matrix as the input of the feature extraction layer to obtain the pre-trained features of the DNA sequence; S5. Using the pre-trained features of the DNA sequence as input to the main neural network training model to obtain global features of the pre-trained features of the DNA sequence; S6. After obtaining the global features of the pre-trained features of the DNA sequence, these features are input into the fully connected layer and a linear transformation is performed to generate the final output, which is the protein fluorescence value prediction, indicating the protein expression level; S7. Based on the trained weight network and the SHAP (SHapley Additive exPlanations) interpretability method, the positions and bases of the DNA sequences that have a causal relationship with the final predicted structure are screened out, and protein fluorescence expression is achieved based on the optimal feature subset.

2. The protein expression prediction method based on convolutional neural network and residual attention mechanism according to claim 1, characterized in that The feature extraction network is a convolutional neural network; the feature extraction network includes a convolution layer, a batch normalization layer, a ReLU layer and a pooling layer; The step S4 is specifically as follows: The input encoded DNA sequence matrix first undergoes a convolution operation, and then batch normalization is used to standardize the output of the convolution layer to have a mean of 0 and a variance of 1, thereby reducing the deviation of the data distribution and improving the training speed and stability of the model; Use the ReLU activation function to increase nonlinearity, thereby improving the expressiveness of the model; Finally, the maximum pooling is used to reduce the size of the feature map, reduce the computational complexity, and retain important local features. The calculation expression is: X=MaxPool(ReLU(BN(Conv3×3(X DNA ,ω feature ,b feature )))) Where X is the output of the feature extraction layer, X DNA is the encoded DNA sequence matrix, ω feature ,b feature are the weights and biases of the convolutional structure of the feature extraction layer, respectively. Conv3×3(·) represents a 3×3 convolution operation, BN(·) represents batch normalization, ReLU(·) is the ReLU activation function, and MaxPool(·) represents the maximum pooling operation.

3. The protein expression prediction method based on convolutional neural network and residual attention mechanism according to claim 1, characterized in that The main neural network training model includes a stacked convolutional structure and a multi-head attention mechanism network. The neural network includes a convolutional layer, a pooling layer, a batch normalization layer, a ReLU layer, a multi-head attention layer, and a residual connection. The main neural network training model specifically includes a 2-layer convolutional structure, a 1-layer multi-head attention mechanism network, and the residual connection formed by them. The calculation expression of the 2-layer convolution structure is: <h2 style=";text-align:left;direction:ltr">S<h2 style=";text-align:left;direction:ltr"> conv <h2 style=";text-align:left;direction:ltr"> (BN(Conv3×3(ReLU(BN(Conv3×3(X,ω1,b1))),ω2,b2)) Among them S conv is the result of the convolutional layer output, X is the pre-trained feature of DNA output in step S4, ω1, b1 are the weight and bias of the first convolutional layer, ω2, b2 are the weight and bias of the second convolutional layer, Conv3×3(·) represents a 3×3 convolution operation, BN(·) represents batch normalization, and ReLU(·) is the ReLU activation function; The calculation expression of multi-head attention and its residual connection is: S residual =S conv +Attn(S conv ) Among them S residual It is the result of adding the input and output residuals of the multi-head attention layer, and Attn(·) represents the multi-head attention operation.

4. The protein expression prediction method based on convolutional neural network and residual attention mechanism according to claim 1, characterized in that The interpretability method is to use the SHAP method to calculate the SHAP average value of the protein fluorescence expression value with which each feature has a causal relationship in the protein expression value prediction model, and construct a SHAP feature value ranking in the same order as the sample DNA sequence to show the contribution value of each position and each base to the prediction result.

Images