Multi-stream fusion gene sequence prediction method based on mamba and double coding strategy

Abstract

The application provides a multi-flow fusion gene sequence prediction method based on Mamba and a double coding strategy, aims to improve the semantic understanding and modeling ability of DNA sequences, and thus improves the accuracy and generalization performance of protein expression value prediction. The method synchronously processes the forward chain and the reverse complementary chain of the DNA sequence: the reverse complementary chain is segmented and vectorized by BPE coding to capture semantic fragments; the forward chain is encoded by one-hot and linearly mapped to a dense representation. After extracting deep features by Mamba Block, the two features are spliced and fused, and then input into LSTM to model the context dependence, and finally output the expression value through the full connection layer. The method effectively solves the problem that One-Hot coding is difficult to capture biological semantic fragments and BPE coding may split the functional domain. Based on the Mamba architecture, the local structure and global semantics are effectively combined, the long-distance dependence is captured while keeping low computational complexity, and the method is suitable for tasks such as gene design and protein expression regulation.

Description

Technical Field

[0001] This invention relates to the fields of computer science, big data genomics, and other technologies, and specifically to a multi-stream fusion gene sequence prediction method based on Mamba and a dual-coding strategy. Background Technology

[0002] In the field of bioengineering, protein expression levels are one of the key factors determining production efficiency and yield. Machine learning methods can analyze massive amounts of gene sequence and phenotypic data to uncover underlying patterns and predict protein expression. In recent years, many deep learning models have achieved significant results in predicting protein expression based on gene sequences. Before inputting sequence data into a deep learning model, a tokenization step is usually required, which converts the DNA sequence data into a standard format that the model can process.

[0003] One-hot encoding and byte-pair encoding (BPE) are two mainstream methods for converting DNA sequences into machine learning-friendly formats. One-hot encoding perfectly preserves the positional structure of the sequence, making it highly suitable for neural networks to extract local patterns. However, its drawback lies in its inability to effectively capture the biological meaning inherent in multi-base combinations. In contrast, BPE can automatically learn meaningful biological functional fragments, improving the model's generalization ability, but its segmentation strategy may lead to the fragmentation of continuous biological functional domains, thus affecting the model's understanding of the overall function of the sequence. To overcome the limitations of the above methods, this invention proposes a multi-stream feature-driven DNA sequence prediction method based on the fusion of one-hot encoding and byte-pair encoding. This model aims to enhance the model's semantic understanding of DNA sequences, significantly improving prediction accuracy and generalization ability. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-stream fusion gene sequence prediction method based on Mamba and dual coding strategies, which can enhance the model's semantic understanding of DNA sequences and significantly improve the accuracy and generalization ability of prediction.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A multi-stream fusion gene sequence prediction method based on Mamba and a dual-coding strategy includes the following steps:

[0007] R1: Generate the reverse strand of the DNA sequence from the original dataset and train the BPE dictionary.

[0008] R2: Establish a multi-stream deep learning regression model, which mainly consists of two parts: multimodal feature embedding and multimodal fusion regression. The multimodal fusion regression mainly consists of several steps: stacking MambaBlocks of different modalities, feature concatenation and LSTM perception, and outputting the sequence representation value of the linear layer.

[0009] R3: Training and prediction of multi-stream deep learning regression models.

[0010] Specifically, step R1 is as follows:

[0011] R1-1: Generate the reverse strand of the DNA sequence from the original dataset.

[0012] R1-2: The initialization phase establishes an initial vocabulary consisting of base units A, T, C, and G, with a cardinality of 4. Assuming the training reverse-strand genome is AACGCACTATATA, the initial vocabulary is first initialized as the basic nucleotide set {A, T, C, G}, with a size of 4. Then, in each loop of the BPE algorithm, the following operations are performed: The frequency of all consecutive pairwise combinations in the current vocabulary is counted (e.g., in the first loop, combinations such as AA appearing once, AC appearing twice, CG appearing once, GC appearing once, CA appearing once, CT appearing once, and TA appearing three times are counted). Next, the combination with the highest frequency is selected from these frequency statistics (in the first loop, the highest frequency combination is TA). The vocabulary is then updated by adding this selected high-frequency combination as a new symbol (therefore, after the first loop, the vocabulary is updated to {A, T, C, G, TA}). The BPE algorithm will continuously repeat this cycle of frequency statistics, selection of the highest-frequency combination, and vocabulary expansion until the vocabulary size reaches the preset target vocabulary size V, at which point training stops. Each word will be assigned an integer.

[0013] Specifically, step R2 is as follows:

[0014] R2-1: Let the DNA length be L, the dataset size be B, the embedding feature dimension be E, and the vocabulary size be V. First, encode the DNA forward sequence into a one-hot feature representation with dimension (B, 4, L). Then, perform an embedding mapping on it through a linear transformation layer to obtain embedding feature 1 with dimension (B, E, L).

[0015] R2-2: The reverse DNA sequence is segmented and encoded using the BPE dictionary trained in step R1 to generate BPE features. Since the length of the BPE-encoded sequence will not exceed L, padding is applied to obtain a discrete feature representation of dimension (B, 1, L), which is then converted into an integer vector. Word vector mapping is then performed through the Embedding layer in PyTorch to obtain embedding feature 2 of dimension (B, E, L).

[0016] R2-3: Feed embedding feature 1 and embedding feature 2 into three stacked Mamba Blocks respectively, and the output feature dimension remains (B, E, L). "Mamba Block" is one of the core components in the Mamba model, representing a basic building block in the Mamba structure. Mamba is an efficient sequence modeling architecture.

[0017] Mamba Block is a high-efficiency sequence modeling module inspired by State-Space Models (SSMs), aiming to significantly reduce computational complexity while maintaining the expressive power of the Transformer. Mamba's core structure is based on a linear State-Space Model (SSM), and its main computational process can be briefly represented as follows:

[0018] y(t)=D·x(t)+∫0 t K(t-τ)·u(τ)dτ

[0019] Where x(t) is the system state, u(t) is the input, y(t) is the output, D is the direct mapping matrix from input to output, and K(t) is the kernel function from input to state output.

[0020] In discrete time, this process is parameterized as a learnable first-order recurrence relation and efficiently implemented using convolution operations, simplified to the following form:

[0021]

[0022] in For state variables, u t For the current input, y t For the output, A, B, C, and D are learnable parameters. In Mamba, this process is implemented through an efficient stateful sequence convolution, the so-called "selective SSM," whose core formula can be further expressed as:

[0023] y = SSM(u; A, B, C) + D·u

[0024] u is the current input, and the SSM operation essentially uses a specially structured convolutional kernel to dynamically weight the input sequence. This mechanism allows the model to retain the ability to model long-distance dependencies while maintaining linear time and space complexity.

[0025] In addition, to enhance modeling flexibility, Mamba introduces gating and nonlinearity in the feedforward path and selectively activates state updates using an input-dependent gating mechanism, thereby improving expressive power.

[0026] R2-4: The fused feature X is obtained by concatenating features according to the second dimension. fused The dimensions are (B, 2E, L).

[0027] R2-5: Fuse feature X fused The input is fed into an LSTM network to extract contextual dependency information from the sequence. The feature dimension of the LSTM output is maintained at (B, 2E, L). Subsequently, this feature is processed through three cascaded fully connected layers, and the final output is the expression value of the corresponding DNA sequence.

[0028] Specifically, step R3 is as follows:

[0029] R3-1: Divide the data into three parts: training set, validation set, and test set.

[0030] R3-2: Use the training set data to train the model and obtain the trained model.

[0031] R3-3: Plot a scatter plot using the model-predicted expression values ​​as the Y-axis and the actual expression values ​​as the X-axis, and calculate the Pearson coefficient to obtain the prediction results:

[0032]

[0033] in, and These are the sample means of the two variables. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating the specific implementation process of the multi-stream fusion gene sequence prediction method based on Mamba and dual coding strategy in this embodiment of the invention.

[0035] Figure 2 This is a scatter plot comparing the predicted and measured values ​​of DNA sequence expression levels obtained using a trained prediction model, as shown in this embodiment of the invention. Detailed Implementation

[0036] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0037] This embodiment uses the generated regulatory sequences and the protein expression levels of the genes they control as an example, and combines them with the appendix. Figures 1-2 The present invention will be further analyzed and explained.

[0038] The multi-stream fusion gene sequence prediction method based on Mamba and dual-coding strategy includes the following steps:

[0039] Step 1: Generate 100,000 random gene sequences with a length of 110, based on the fixed first 13 and last 17 base pairs and the 80 random base pairs in the middle.

[0040] Step 2: Divide the dataset generated in Step 1 into three parts: training set, validation set, and test set.

[0041] Step 3: Train the BPE dictionary using the training set generated in Step 2:

[0042] 3.1 Generate the reverse strand of the DNA sequence for the training set.

[0043] 3.2 The vocabulary size of the BPE dictionary is set to 4096, and special words N and [padding] are added. N represents unknown bases in the sequence, and [padding] is used to lengthen shorter sequences.

[0044] 3.2 Initialize the BPE vocabulary from the basic bases {A, T, C, G}. In a loop, it counts the frequency of all consecutive two-character combinations in the reverse chain generated in 3.1, selecting the highest frequency combination as a new symbol to add to the vocabulary. This process is repeated until the vocabulary size reaches the preset target of 4096, at which point training stops. Finally, each word is assigned a unique integer.

[0045] Step 4: Build a multi-stream deep learning regression model, such as... Figure 1 As shown:

[0046] 4.1 The DNA length is 110, the dataset size is 100,000, the embedding feature dimension is 512, and the vocabulary size is 4096. First, the forward DNA sequence is encoded into a one-hot feature representation with dimensions (100,000, 4, 110). Then, it is embedded and mapped through a linear transformation layer to obtain embedding feature 1 with dimensions (100,000, 512, 110).

[0047] 4.2 The reverse DNA sequence is segmented and encoded using the BPE dictionary trained in step 3 to generate BPE features. Since the length of the BPE-encoded sequence will not exceed 110, padding is applied to obtain a discrete feature representation with dimensions (100000, 1, 110). Then, word vector mapping is performed through the Embedding layer in PyTorch to obtain embedding feature 2 with dimensions (100000, 512, 110).

[0048] 4.3 Embedded feature 1 and embedded feature 2 are fed into three stacked Mamba Blocks respectively. The output feature dimensions remain (100000, 512, 110).

[0049] 4.4 The fused feature X is obtained by concatenating features according to the second dimension. fused The dimensions are (100000, 1024, 110).

[0050] 4.5 Fusing Feature X fused The input is fed into an LSTM network to extract contextual dependency information from the sequence. The feature dimensions of the LSTM output are maintained at (100000, 1024, 110). Subsequently, this feature is processed through three cascaded fully connected layers, ultimately outputting the expression value of the corresponding DNA sequence.

[0051] Step 5: Train the multi-stream deep learning regression model and make predictions:

[0052] 5.1 Input the training set data into the model for training to obtain the trained model.

[0053] 5.2 Plot a scatter plot using the model-predicted expression values ​​as the Y-axis and the actual expression values ​​as the X-axis, and calculate the Pearson coefficient as 0.9794. Figure 2 As shown.

[0054] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. For those skilled in the art, after learning the contents described in the present invention, several equivalent changes and substitutions can be made without departing from the principle of the present invention. These equivalent changes and substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. A multi-stream fusion gene sequence prediction method based on Mamba and a dual-coding strategy, characterized in that, Includes the following steps: R1: Generate the reverse strand of the DNA sequences from the original dataset and train the BPE dictionary; R2: Establish a multi-stream deep learning regression model. The model mainly consists of two parts: multimodal feature embedding and multimodal fusion regression. The multimodal fusion regression mainly consists of several steps: stacking Mamba Blocks of different modalities, feature concatenation and LSTM perception, and outputting the sequence expression value of the linear layer. R3: Training and prediction of multi-stream deep learning regression models; Step R2 is as follows: R2-1: Let the DNA length be L, the dataset size be B, the embedding feature dimension be E, and the vocabulary size be V. First, encode the DNA forward sequence into a one-hot feature representation with dimension (B, 4, L). Then, perform an embedding mapping on it through a linear transformation layer to obtain embedding feature 1 with dimension (B, E, L). R2-2: The reverse DNA sequence is segmented and encoded using the BPE dictionary trained in step R1 to generate BPE features, which are then converted into integer vectors. Since the length of the BPE-encoded sequence will not exceed L, padding is applied to it to obtain a discrete feature representation with dimension (B, 1, L). Then, word vector mapping is performed through the Embedding layer in PyTorch to obtain embedding feature 2 with dimension (B, E, L). R2-3: Feed embedding feature 1 and embedding feature 2 into three stacked Mamba Blocks respectively; The output feature dimension remains (B, E, L); R2-4: The fused feature X is obtained by concatenating features according to the second dimension. fused The dimensions are (B, 2E, L); R2-5: Fuse feature X fused The input is fed into an LSTM network to extract contextual dependency information from the sequence; the feature dimension of the LSTM output is maintained at (B, 2E, L); then, the feature is processed through three tandem fully connected layers to finally output the expression value of the corresponding DNA sequence.

2. The multi-stream fusion gene sequence prediction method based on Mamba and dual-coding strategy according to claim 1, characterized in that, Step R1 is as follows: R1-1: The reverse strand dataset of the DNA sequences that generated the original dataset; R1-2: Initialize the byte pair encoding vocabulary, i.e., the BPE vocabulary, from the basic bases {A,T,C,G}; In the loop, it counts the frequency of consecutive two-character combinations in all backchain datasets and selects the most frequent combination as a new symbol to add to the vocabulary. This process is repeated until the vocabulary size reaches the preset target V, at which point training stops. Ultimately, each word is assigned a unique integer.

3. The multi-stream fusion gene sequence prediction method based on Mamba and dual-coding strategy according to claim 1, characterized in that, Step R3 is as follows: R3-1: Divide the training dataset into three parts: training set, validation set, and test set; R3-2: Use the training set data to train the model and obtain a trained model; R3-3: Plot a scatter plot using the model-predicted expression values ​​as Y and the actual expression values ​​as X, and calculate the Pearson coefficient to obtain the prediction results: in, and These are the sample means of the two variables.

Images