Method for predicting formation tendency of DNA G-quadruplex based on Transform model

Abstract

The invention discloses a method for predicting the formation tendency of a DNA G-quadruplex based on a Transform model, and the method comprises the steps: extracting nucleic acid sequence information and G-quadruplex position point information, and constructing a data set; a Transform deep learning model used for predicting the formation tendency of the DNA G-quadruplex is constructed; training the Transform deep learning model through parallel computing based on the data set to obtain a trained Transform deep learning model, and verifying the trained Transform deep learning model to obtain a target deep learning model; and performing DNA G-quadruplex formation tendency prediction based on the target deep learning model to obtain a prediction result. According to the method, not only can the G-quadruplex formation tendency prediction be well performed on the DNA sequence, but also the potential G-quadruplex motif can be identified.

Description

Technical Field

[0001] The invention belongs to the technical field of biochemical science and computational chemistry, and in particular relates to a method for predicting the tendency of DNA G-quadruplex formation based on a Transformer model. Background Art

[0002] G-quadruplex structures are formed in guanine-rich DNA or RNA sequences and play an important role in organisms and in various life activities, such as DNA replication, RNA transcription, and telomere maintenance. The prediction of G-quadruplex formation tendency is of great significance for basic scientific research, cancer disease treatment, and drug discovery. In view of the important biological functions of G-quadruplexes, a variety of experimental techniques have been developed in recent years to detect them in a high-throughput manner, including G4-seq, rG4-seq, and G4-ChIP-seq. G4-seq technology recognizes G-quadruplexes in vitro, while rG4-seq and G4-ChIP-seq are specifically used to detect and locate G-quadruplexes in vivo. Compared with time-consuming and labor-intensive experimental methods, computational methods can efficiently locate these structures in sequences, thereby significantly reducing the time and cost of detection. Computational methods for G-quadruplex detection are mainly divided into rule-based methods, machine learning (ML)-based methods, and deep learning (DL)-based methods.

[0003] Rule-based methods mainly evaluate the tendency of G-quadruplex formation through scoring rules. Continuous G bases or continuous C bases in DNA sequences can obtain higher scores. These methods can quickly and efficiently identify G-quadruplexes, but they are based on expert experience rather than experimental data, which may limit the accuracy of G-quadruplex structure prediction.

[0004] Machine learning-based methods mainly use methods such as artificial neural networks and gradient boosting decision trees to achieve the purpose of identifying G-quadruplex structures. These methods can achieve certain results when the data set is small, but they cannot adapt to larger data sets and cannot accurately extract the position information of DNA sequences.

[0005] Deep learning-based methods mainly use convolutional neural networks to extract features of DNA sequences to predict G-quadruplex structures. Although these methods have achieved certain improvements in prediction performance compared to machine learning, convolutional neural networks also have limitations when processing sequence data. Due to the size of the convolution kernel, it is difficult to capture long-range information in DNA sequences. In addition, convolutional neural networks have a "black box" nature and lack interpretability. Summary of the invention

[0006] To solve the above technical problems, the present invention proposes a method for predicting the tendency of DNA G-quadruplex formation based on the Transformer model. The self-attention mechanism in the Transformer architecture is undoubtedly a powerful feature extraction structure that can effectively capture long-distance dependencies and rich contextual information within the sequence.

[0007] To achieve the above object, the present invention provides a method for predicting the tendency of DNA G-quadruplex formation based on the Transformer model, comprising:

[0008] Extract nucleic acid sequence information and G-quadruplex site information and construct a data set;

[0009] Construct a Transformer deep learning model for predicting DNA G-quadruplex formation tendency;

[0010] Based on the data set, the Transformer deep learning model is trained by parallel computing to obtain a trained Transformer deep learning model, and the trained Transformer deep learning model is verified to obtain a target deep learning model;

[0011] The DNA G-quadruplex formation tendency is predicted based on the target deep learning model to obtain a prediction result.

[0012] According to the method for predicting the tendency of DNA G-quadruplex formation based on the Transformer model provided by the present invention, the process of constructing a data set includes: obtaining nucleic acid sequence information based on gene starting coordinates and genome sequence, characterizing the DNA sequence through one-hot encoding, and completing the construction of the data set.

[0013] According to the method for predicting DNA G-quadruplex formation tendency based on the Transformer model provided by the present invention, the Transformer deep learning model for predicting DNA G-quadruplex formation tendency includes a semantic feature encoder, a temporal feature encoder and a scorer;

[0014] The semantic feature encoder is used to extract local features and global features of DNA through one-dimensional convolution and Transformer-encoder to obtain high-dimensional embedding features of DNA;

[0015] The temporal feature encoder is used to extract the position information and context information of the DNA sequence through a bidirectional gated recurrent unit;

[0016] The scorer is used to convert the DNA features into a prediction result of the G-quadruplex formation tendency through a multi-layer perceptron.

[0017] According to the method for predicting the DNA G-quadruplex formation tendency based on the Transformer model provided by the present invention, the method for extracting the local features and global features of DNA through one-dimensional convolution and Transformer-encoder to obtain the high-dimensional embedding features of DNA includes:

[0018] The DNA sequence is represented by one-hot encoding and input into a one-dimensional convolutional model for model training to obtain local features closely related to the formation of G-quadruplexes.

[0019] Based on Transformer-encoder, global dependency modeling is performed on the local features, and the interactive information between any positions in the nucleic acid sequence is extracted through the self-attention mechanism. A sequence representation that integrates local and global features is constructed to obtain high-dimensional embedded features of DNA.

[0020] According to the method for predicting the DNA G-quadruplex formation tendency based on the Transformer model provided by the present invention, the method for extracting the position information and context information of the DNA sequence through the bidirectional gated recurrent unit includes:

[0021] Based on the gating mechanism of the gated recurrent unit, the nucleotide information at the current position and the hidden state at the previous moment are comprehensively considered, historical information is selectively retained, the hidden state at the current moment is updated, and the short-term and long-term dependencies in the sequence are extracted;

[0022] The bidirectional propagation mechanism based on the gated recurrent unit traverses the DNA sequence forward and backward to extract the dependency between the nucleotide at each position and the previous and next sequences.

[0023] According to the method for predicting DNA G-quadruplex formation tendency based on the Transformer model provided by the present invention, the method for converting DNA features into prediction results of G-quadruplex formation tendency through a multi-layer perceptron includes:

[0024] The encoded features are linearly combined through the fully connected layer and mapped to the new feature space;

[0025] By embedding the activation function in the fully connected layer, nonlinear transformation is introduced to enhance the expression ability of the model;

[0026] The prediction results of G-quadruplex formation tendency are obtained through the output layer.

[0027] According to the method for predicting the DNA G-quadruplex formation tendency based on the Transformer model provided by the present invention, the calculation of the high-dimensional embedding feature of DNA is obtained as follows:

[0028] X attention=LayerNorm(X+X attention )

[0029] X hidden =LayerNorm(FFN(X attention )+X attention )

[0030] Among them, X attention represents the output of multi-head attention, LayerNorm represents the normalization layer, which converts the input of each layer of neurons into a layer with consistent mean and variance, X represents the input of the previous layer, and X attention represents the attention output of the previous layer, X hidden represents the encoded hidden layer representation, FFN() represents the output of the feedforward network, and the plus sign represents a residual connection.

[0031] According to the method for predicting DNA G-quadruplex formation tendency based on the Transformer model provided by the present invention, the prediction result of the G-quadruplex formation tendency is calculated as follows:

[0032] y=Linear(ReLU(Linear(X hidden )))

[0033] Among them, Linear() represents the linear transformation layer, ReLU represents the activation function of the first layer, and X hidden represents the encoded hidden layer representation, and y represents the predicted result.

[0034] Technical effect of the present invention: The present invention discloses a method for predicting the tendency of DNA G-quadruplex formation based on the Transformer model. The G-quadruplex formation tendency prediction model constructed based on the Transformer architecture can well solve the problem of the model losing long-range information or local information during the training process. The verification results show that the G-quadruplex formation tendency prediction model based on the Transformer architecture can achieve good results on public data sets. In addition, the present invention uses deep learning technology to predict the mismatch score of DNA, saving expensive biological experiments and providing researchers in the biological field with efficient and convenient research tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 Schematic diagram of a method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to an embodiment of the present invention;

[0037] Figure 2 This is a model architecture diagram of an embodiment of the present invention;

[0038] Figure 3 For the embodiment of the present invention, K + Dataset and K + +Schematic diagram of the performance of DNA mismatch score prediction on the PDS dataset;

[0039] Figure 4 It is a schematic diagram of the performance of an embodiment of the present invention for predicting the propensity of G-quadruplex formation on a DNA microarray data set;

[0040] Figure 5 For the embodiment of the present invention, K + Schematic diagram of the performance of the DNA G-quadruplex classification task on the dataset;

[0041] Figure 6 For the embodiment of the present invention, K + +Schematic diagram of the performance of the DNA G-quadruplex classification task on the PDS dataset. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] like Figure 1 As shown, this embodiment provides a method for predicting the tendency of DNA G-quadruplex formation based on the Transformer model, comprising:

[0045] Step 1: Use the Gene Expression Omnibus (GEO) accession number GSE110582 to log in to GEO to download the human genome mismatch score data measured by G4-seq technology, and then download the human genome sequencing file and its annotation data from the UCSC Genome Browser. The DNA sequence of each chromosome was downloaded separately in fasta format.

[0046] Step 2: Divide the G4-seq data into training, validation, and test sets by the chromosome holdout method. The DNA sequence is obtained by mapping the gene coordinates in G4-seq to the hg19 human reference genome, which is downloaded from the UCSC Genome Browser. Before being input into the model, the DNA sequence is converted into a 215×4 matrix by one-hot encoding, where each column corresponds to a DNA base (A, T, C, or G).

[0047] Step 3: Figure 2 As shown in the figure, a Transformer-based deep learning model is constructed, including a semantic feature encoder, a temporal feature encoder, and a scorer.

[0048] Among them, the semantic feature encoder includes:

[0049] (1) Use one-dimensional convolution to extract local features of the sequence and enhance the expressiveness of the model:

[0050]

[0051] Among them, W is the weight, b is the bias, and X is the input.

[0052] (2) Use the Encoder module in the Transformer model, use 4 attention heads, and then pass the encoded result through a fully connected layer and a Softmax function to obtain the feature representation of each base position in the sequence. The entire execution process is summarized as follows:

[0053] The self-attention mechanism models the dependency between two positions of a task in a DNA sequence, as follows:

[0054]

[0055] head i =attention(QW i Q ,KW i K ,VW i V );

[0056] Among them, Q represents the query vector (query), K represents the key vector (key), and V represents the value vector (value). These three vectors are obtained from the original input vector through linear transformation. The formula calculates the inner product of each row vector of the matrix Q and K. In order to prevent the inner product from being too large, it is divided by The linear transformation matrix W that maps q, k, v to Q, K, V respectively Q , Q K, W V

[0057] The results of all attention heads are concatenated and passed through a final W O The linear transformation gets the output, the formula is as follows:

[0058] multiHead(Q,K,V)=concat(head1,...,head h )W O ;

[0059] After the residual connection of the self-attention mechanism, the layer is normalized and input into the feedforward neural network. The feedforward neural network also performs residual connection and layer normalization, and finally obtains the features encoded by the Transformer-encoder. The calculation formula is as follows:

[0060] X attention =LayerNorm(X+X attention ),

[0061] X hidden =LayerNorm(FFN(X attention )+X attention )

[0062] Among them, X attention represents the output of multi-head attention, LayerNorm represents the normalization layer, which converts the input of each layer of neurons into a layer with consistent mean and variance, X represents the input of the previous layer, and X attention represents the attention output of the previous layer, X hidden represents the encoded hidden layer representation, FFN() represents the output of the feedforward network, and the plus sign represents a residual connection.

[0063] The temporal feature encoder includes a bidirectional gated recurrent unit, which contains an update gate, a reset gate, a candidate hidden state, and a new hidden state. It can comprehensively consider the information of the current position and the hidden state of the previous moment, and extract the position information and context information of the DNA sequence. The entire execution process is summarized as follows:

[0064] Reset gate calculation:

[0065] R t =σ(X t W xr +H t-1 W hr +b r ),

[0066] Among them, X t represents the input at the current time t, H t-1 W represents the previous time step hidden state. xrRepresents the weight matrix from input to hidden state, mapping the input data to the space of hidden state; W hr represents the weight matrix from hidden state to hidden state, which is responsible for transmitting information between hidden states so that the network can update the internal state. σ is the sigmoid function, which is the activation function. Its function is to introduce nonlinearity to increase the expressive power of the model, so that it can capture and learn complex patterns and relationships in the data. r It's a deviation.

[0067] Update gate calculation:

[0068] Z t =σ(X t W xz +H t-1 W hz +b z ),

[0069] Among them, X t represents the input at the current time t, H t-1 W represents the previous time step hidden state. xz , W hz represents the weight parameter, σ is the sigmoid function, b z It's a deviation.

[0070] Candidate hidden state calculation:

[0071]

[0072] Among them, W xh , W hh represents the weight parameter, tanh is the activation function, b h is the deviation, and ⊙ represents the element-wise multiplication of the output of the reset gate of the current time step and the hidden state of the previous time step. As can be seen from the above formula, the reset gate controls how the hidden state of the previous time step flows into the candidate hidden state of the current time step. The hidden state of the previous time step may contain all the historical information of the time series up to the previous time step. Therefore, the reset gate can be used to discard historical information that is not relevant to the prediction.

[0073] New hidden state calculation:

[0074]

[0075] Finally, the output of the bidirectional gated recurrent unit at a time step is obtained by concatenating the forward and reverse hidden states, which not only takes into account past information, but also future information, improving the model's ability to extract contextual information:

[0076]

[0077] The scorer consists of a feedforward neural network, including two linear layers with activation functions, which are used to map the features obtained in the previous steps to a new feature space and obtain the prediction results of the G-quadruplex formation tendency through the output layer. The formula is as follows:

[0078] y=Linear(ReLU(Linear(X hidden )))

[0079] Among them, Linear() represents the linear transformation layer, ReLU represents the activation function of the first layer, and X hidden represents the encoded hidden layer representation, and y represents the predicted result.

[0080] Step 4: Based on the divided data set, the model is trained through parallel computing, and the trained model is verified to obtain the target deep learning model:

[0081] The training process uses the mean square error loss function as the optimization target. In deep learning, the mean square error is a commonly used loss function suitable for regression tasks. It measures the average of the sum of squares of the differences between the model's predicted value and the true value. The mismatch score of the G-quadruplex is a continuous value, and the prediction of the G-quadruplex mismatch score can be regarded as a regression task. During the training process, the parameters of the model are optimized by minimizing the mean square error loss function so that the model's prediction results are as close to the true mismatch score as possible. The formula of the mean square error loss function is expressed as follows:

[0082]

[0083] Among them, y i , Represent the predicted value and the true value respectively, and N represents the number of samples.

[0084] Determine the prediction model evaluation indicators to verify the prediction effect of the model, including mean absolute error (MAE), root mean square error (RMSE), determination coefficient (R2) and Pearson correlation coefficient (R). The calculation formula is as follows:

[0085]

[0086] The evaluation indicators used for classification tasks include precision, recall and Jaccard coefficient:

[0087]

[0088] Among them, TP (True Positive) means that the positive class is correctly predicted as the positive class, that is, the actual positive class is predicted as the positive class by the model. FP (False Positive) means that the negative class is incorrectly predicted as the positive class, that is, the actual negative class is predicted as the positive class by the model. FN (False Negative) means that the positive class is incorrectly predicted as the negative class, that is, the actual positive class is predicted as the negative class by the model.

[0089] The fifth step is to compare the patented model TransG4 and G4mismatch of the present invention. Figure 3 As shown in Figure 2, the mismatch score predicted by TransG4 is more accurate, with a K + The R value was 3.6% higher on the K + A 3.3% higher R value was obtained on the +PDS dataset.

[0090] Considering that the mismatch scores of most sequences in the G4-seq experimental data set are close to 0, this example selects a DNA microarray data set to verify the generalization performance of the model. The data set includes 15,671 sequences. All probe sequences will be padded with 100 nucleotides on both sides to construct a complete sequence with a length of 260 nt. The maximum value of the subsequence mismatch score will be used as the G-quadruplex formation tendency of the original sequence. Figure 4 As shown, the results show that the prediction results of the TransG4 model showed a significant positive correlation with the PDS binding strength measured in the microarray experiment, and the highest R value obtained reached 0.91. In comparison, the R value of G4mismatch was significantly lower than that of TransG4.

[0091] G4-seq experiments will K + The mismatch score is higher than 25 and K + DNA sequences with mismatch scores higher than 35 in the +PDS dataset are defined as G-quadruplexes. Based on this, this example compares the performance of TransG4 with G4mismatch, DeepG4, PENGUINN(p), PENGUINN(s), Quadron, G4Hunter, pqsfinder, and g4predict in the G-quadruplex classification task.

[0092] like Figure 5 , Figure 6 As shown in the figure, in the G-quadruplex classification task, the model of this embodiment outperforms other methods in K + and K +The Jaccard scores on the +PDS dataset are 5.4% and 9% higher than the runner-up method, respectively, which shows that the TransG4 model achieves the best balance between Precision and Recall.

[0093] The present invention discloses a method for predicting the formation tendency of DNA G-quadruplex based on the Transformer model. The constructed G-quadruplex formation tendency prediction model based on the Transformer architecture can well solve the problem of losing long-range information or local information during the training process of the model. The verification results show that the G-quadruplex formation tendency prediction model based on the Transformer architecture can achieve good results on public data sets. In addition, the present invention uses deep learning technology to predict the mismatch score of DNA, saving expensive biological experiments and providing researchers in the biological field with efficient and convenient research tools.

[0094] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for predicting the tendency of DNA G-quadruplex formation based on the Transformer model, characterized in that: The following steps are involved: Extract nucleic acid sequence information and G-quadruplex site information and construct a data set; Construct a Transformer deep learning model for predicting DNA G-quadruplex formation tendency; Based on the data set, the Transformer deep learning model is trained by parallel computing to obtain a trained Transformer deep learning model, and the trained Transformer deep learning model is verified to obtain a target deep learning model; The DNA G-quadruplex formation tendency is predicted based on the target deep learning model to obtain a prediction result.

2. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 1, characterized in that: The process of constructing the data set includes: obtaining nucleic acid sequence information based on the gene starting coordinates and genome sequence, characterizing the DNA sequence through one-hot encoding, and completing the construction of the data set.

3. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 1, characterized in that: The Transformer deep learning model used to predict the propensity of DNA G-quadruplex formation includes a semantic feature encoder, a temporal feature encoder, and a scorer; The semantic feature encoder is used to extract local features and global features of DNA through one-dimensional convolution and Transformer-encoder to obtain high-dimensional embedding features of DNA; The temporal feature encoder is used to extract the position information and context information of the DNA sequence through a bidirectional gated recurrent unit; The scorer is used to convert the DNA features into a prediction result of the G-quadruplex formation tendency through a multi-layer perceptron.

4. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 3, characterized in that: Methods for extracting local and global features of DNA through one-dimensional convolution and Transformer-encoder to obtain high-dimensional embedding features of DNA include: The DNA sequence is represented using one-hot encoding and input into a one-dimensional convolutional model for model training to obtain local features closely related to G-quadruplex formation; Based on Transformer-encoder, global dependency modeling is performed on the local features, and the interactive information between any positions in the nucleic acid sequence is extracted through the self-attention mechanism. A sequence representation that integrates local and global features is constructed to obtain high-dimensional embedded features of DNA.

5. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 3, characterized in that: The method for extracting the position information and context information of a DNA sequence by a bidirectional gated recurrent unit includes: Based on the gating mechanism of the gated recurrent unit, the nucleotide information at the current position and the hidden state at the previous moment are comprehensively considered, historical information is selectively retained, the hidden state at the current moment is updated, and the short-term and long-term dependencies in the sequence are extracted; The bidirectional propagation mechanism based on the gated recurrent unit traverses the DNA sequence forward and backward to extract the dependency between the nucleotide at each position and the previous and next sequences.

6. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 3, characterized in that: Methods for converting DNA features into prediction results of G-quadruplex formation tendency through multi-layer perceptron include: The encoded features are linearly combined through the fully connected layer and mapped to the new feature space; By embedding the activation function in the fully connected layer, nonlinear transformation is introduced to enhance the expression ability of the model; The prediction results of G-quadruplex formation tendency are obtained through the output layer.

7. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 4, characterized in that: The calculation to obtain the high-dimensional embedding features of DNA is as follows: X attention =LayerNorm(X+X attention ) X hidden =LayerNorm(FFN(X attention )+X attention ) Among them, X attention represents the output of multi-head attention, LayerNorm represents the normalization layer, which converts the input of each layer of neurons into a layer with consistent mean and variance, X represents the input of the previous layer, and X attention represents the attention output of the previous layer, X hidden represents the encoded hidden layer representation, FFN() represents the output of the feedforward network, and the plus sign represents a residual connection.

8. The method for predicting DNA G-quadruplex formation tendency based on the Transformer model according to claim 6, characterized in that: The predicted results of the G-quadruplex formation tendency were calculated as follows: and=Linear(ReLU(Linear(X hidden ))) Among them, Linear() represents the linear transformation layer, ReLU represents the activation function of the first layer, and X hidden represents the encoded hidden layer representation, and y represents the predicted result.

Images