DNA sequence reconstruction method and system based on multi-scale attention and contrast learning

Abstract

The invention discloses a DNA sequence reconstruction method and system based on multi-scale attention and contrast learning, and relates to the technical field of DNA storage data reconstruction. Comprising the following steps: collecting a plurality of DNA sequence copies, screening out abnormal length sequences, and constructing a standardized clustering data set; performing one-hot coding and filling processing on the DNA sequence; extracting context dependent features and cross-sequence variation features; an Inter-Sequence multi-head attention mechanism is constructed to calculate the similarity between the sequences, and a weighted sequence tensor is generated; a global dependency relationship in the sequence is extracted through an Intra-Sequence multi-head attention mechanism; local offset features caused by insertion and deletion errors are extracted through a multi-size convolutional network; inputting a double-layer long-short-term memory network for sequence-level modeling, and outputting base reconstruction probability distribution; and constructing positive and negative sample pairs, calculating comparison loss, combining cross entropy loss to form a joint loss function, and outputting a high-precision DNA sequence reconstruction result. The method has high accuracy and robustness under the conditions of complex noise and multiple types of errors.

Description

Technical Field

[0001] This invention relates to the field of DNA storage data reconstruction technology, specifically to a DNA sequence reconstruction method and system based on multi-scale attention and contrastive learning. Background Technology

[0002] With the continuous advancement of DNA storage technology and genomics research, high-precision reconstruction of DNA sequence sequencing results under high error rate conditions has become a key research direction in this field. Current mainstream DNA sequence reconstruction methods mainly rely on multiple sequence alignment, deBruijn graph assembly, or error correction mechanisms based on probabilistic models. However, these methods have significant limitations in handling various types of sequencing errors, such as substitutions, insertions, and deletions.

[0003] Traditional multiple sequence alignment methods are prone to alignment failures or mismatch accumulation in scenarios with high error rates or significant shift perturbations between sequences, directly leading to a significant decrease in the overall consistency of the reconstructed sequences. Assembly strategies based on deBruijn graphs often require setting high coverage thresholds to cope with sequencing noise. This operation can easily cause low-frequency real fragments to be misjudged as noise and ignored, thus affecting the integrity of the reconstruction results. Probabilistic model-based methods are insufficient in modeling complex variation patterns in high-dimensional feature spaces, making it difficult to simultaneously and accurately capture both global dependencies between sequences and fine-grained structural changes within sequences.

[0004] Furthermore, existing methods generally lack effective mechanisms for fusing global and local features, often resulting in the loss of crucial information during sequence similarity modeling and characterization of intra-sequence base dependencies. During model training and optimization, traditional single supervisory signals cannot simultaneously ensure base prediction accuracy and feature space separability, leading to poor generalization performance when faced with sequencing data from different sources and of varying quality. These issues collectively make it difficult for existing DNA sequence reconstruction technologies to guarantee the accuracy, stability, and robustness of reconstruction results under conditions of high error rates, complex variations, and strong noise. Summary of the Invention

[0005] The purpose of this invention is to propose a DNA sequence reconstruction method and system based on multi-scale attention and contrastive learning, which has high accuracy and robustness under complex noise and multi-type error conditions.

[0006] According to a first aspect of the embodiments of this disclosure, a DNA sequence reconstruction method based on multi-scale attention and contrastive learning is provided, comprising the following steps: Multiple DNA sequence copies were collected and length-filtered to remove abnormal sequences outside the set range, thus constructing a standardized DNA sequence clustering dataset. The DNA sequences in the DNA sequence clustering dataset are one-hot encoded and padded to generate a base vector matrix of uniform length. At the same time, a learnable position encoding matrix is ​​added to form an input representation tensor. The input representation tensor is input into the MSATransformer structure, and attention mechanism operations are performed in the row and column directions of the base dimension to extract context-dependent features and cross-sequence variation features, thereby obtaining a multi-dimensional fusion representation tensor. An Inter-Sequence multi-head attention mechanism is constructed based on the DNA sequence clustering dataset. The input of this mechanism is a multi-dimensional fusion representation tensor. Attention weights are assigned based on the similarity scores between sequences to generate a weighted sequence tensor. The Intra-Sequence multi-head attention mechanism is applied to the weighted sequence tensor to extract the global dependencies between bases within the sequence and generate a feature-enhanced tensor. The feature enhancement tensor is input into a multi-size convolutional network to extract local offset features caused by insertion and deletion errors. The convolutional features output by the multi-size convolutional network are input into a two-layer long short-term memory network for sequence-level modeling, and the reconstruction probability distribution of each base position is output. Based on the sequences within the DNA sequence clustering dataset, construct contrastive learning sample pairs to generate positive sample pairs between the reference sequence and the reconstructed sequence, as well as negative sample pairs between the perturbed sequence and the reconstructed sequence. Within the same sequence cluster, the contrastive loss is calculated based on the contrastive learning sample pairs, and the distribution structure of the feature space is optimized based on the positive attraction and negative repulsion mechanism; A joint loss function is constructed, which combines the cross-entropy loss corresponding to the reconstruction probability distribution of the two-layer long short-term memory network output with the contrast loss in a weighted manner, and updates the network parameters through backpropagation to output the reconstructed DNA sequence set.

[0007] In one embodiment, the input representation tensor is obtained as follows: Perform one-hot encoding on each DNA sequence to map base characters to a set of four-dimensional one-hot vectors, and construct a base vector matrix; The base vector matrix is ​​padded to a uniform length by adding a zero vector to the end of the sequence to fill it to the preset length limit, thus obtaining a coding matrix of uniform dimension. Construct a positional encoding matrix, which is a learnable vector matrix whose vector dimension is consistent with that of the base vector matrix; The position encoding matrix and the encoded base vector matrix are added element by element according to their position indices to form an input representation tensor with position information; The input representation tensors of all DNA sequences are stacked along the sample dimension to generate the set of input feature tensors for the DNA sequence clustering dataset.

[0008] In one embodiment, the multi-dimensional fusion representation tensor is obtained as follows: The input representation tensor is input into the MSATransformer structure, and the column direction attention mechanism operation is performed to calculate the feature correlation between the same base position in different DNA sequences, generating a column dimension attention output tensor. Perform average pooling operation along the sample dimension on the column-dimensional attention output tensor to generate a cross-sequence position feature representation tensor; The input representation tensor is input again into the MSATransformer structure to perform row direction attention mechanism operation, calculate the dependency between different base positions in a single DNA sequence, and generate a row dimension attention output tensor. The cross-sequence position feature representation tensor and the row-dimensional attention output tensor are concatenated in the channel dimension to form a multi-dimensional fusion representation tensor.

[0009] In one embodiment, the multi-size convolutional network execution process includes: Receive the feature-enhanced tensor and duplicate the tensor multiple times as multi-branch inputs while maintaining consistency in the channel dimension; Set convolution kernel windows of different sizes on each branch; One-dimensional convolution operations are performed in each branch to extract local base shift features within different ranges; Perform batch normalization and activation function transformation operations on the output of each branch to generate a set of standardized feature subtensors; All feature sub-tensors are concatenated along the channel dimension to generate a multi-size convolutional feature fusion tensor, which is the convolutional feature.

[0010] In one embodiment, the execution process of the two-layer long short-term memory network includes: The convolutional features are used as input sequence features and fed into the first layer of the bidirectional long short-term memory network. Information is propagated along the forward and reverse time steps respectively, and the forward and reverse state sequences are output. The forward state sequence and the reverse state sequence are concatenated along the feature dimension to form the first layer output state tensor. The first layer output state tensor is used as input and passed into the second layer bidirectional long short-term memory network. The forward and backward state propagation operations are performed again to generate the second layer state output tensor. The second layer state output tensor is input into the linear transformation layer, where the state vector is mapped to a set of output probability vectors with a dimension of 4. Normalization is performed on the set of output probability vectors to obtain the reconstruction probability distribution of A, T, C, and G at each base position.

[0011] In one embodiment, the comparison loss is calculated as follows: Within each DNA sequence cluster, the feature representation vector of each sequence in the positive and negative sample pairs is first extracted. Based on the feature representation vector, the cosine similarity between positive sample pairs and the cosine similarity between negative sample pairs are calculated respectively. The similarity scores of positive sample pairs and negative sample pairs are input into the contrastive loss function; A temperature scaling factor is introduced into the contrast loss function to adjust the relative distribution difference between sample pairs through an exponential mapping method. The distance between positive sample pairs is compressed while the distance between negative sample pairs is kept separate, and the difference optimization process is completed by minimizing the loss function value; The contrast loss in each batch is summarized and calculated, and the total contrast loss value within the cluster is output.

[0012] In one embodiment, the joint loss function construction and training process is as follows: In each training batch, the probability distribution of DNA sequence reconstruction and the corresponding base tag vector are extracted, the cross-entropy loss at each base position is calculated, and the batch-level base reconstruction loss term is obtained by averaging over all positions. Extract the feature similarity difference between all positive and negative sample pairs in the current batch cluster, and obtain the batch-level contrast loss term according to the contrast loss calculation method; Set weight coefficients α and β for cross-entropy loss and contrastive loss, and perform a weighted summation operation on the two loss terms to form a joint loss function; The joint loss function is input into the backpropagation calculation module, and chain partial derivative expansion is performed on all trainable parameters to obtain the gradient information corresponding to each parameter. The gradient information is input into the Adam optimizer to perform parameter update operations and iteratively adjust the parameter weights. During the training rounds, the joint loss optimization process is repeated, and the final output is the converged set of network parameters and the reconstructed set of DNA sequences.

[0013] According to a second aspect of the present disclosure, a DNA sequence reconstruction system based on multi-scale attention and contrastive learning is provided, comprising: The dataset standardization module collects multiple DNA sequence copies and performs length filtering to remove abnormal sequences that are outside the set range, thus constructing a standardized DNA sequence clustering dataset. The input tensor construction module performs one-hot encoding and padding on the DNA sequences in the DNA sequence clustering dataset to generate a base vector matrix of uniform length. At the same time, it adds a learnable position encoding matrix to form an input representation tensor. The multi-dimensional feature extraction module inputs the input representation tensor into the MSATransformer structure and performs attention mechanism operations in the row and column directions of the base dimension to extract context-dependent features and cross-sequence variation features, thereby obtaining a multi-dimensional fusion representation tensor. The inter-sequence attention modeling module constructs an Inter-Sequence multi-head attention mechanism based on the DNA sequence clustering dataset. The input of this mechanism is a multi-dimensional fusion representation tensor, and attention weights are assigned through the similarity scores between sequences to generate a weighted sequence tensor. The intra-sequence attention modeling module performs Intra-Sequence multi-head attention mechanism operations on the weighted sequence tensor to extract the global dependencies between bases within the sequence and generate a feature enhancement tensor. The local offset feature extraction module inputs the feature enhancement tensor into a multi-size convolutional network to extract local offset features caused by insertion and deletion errors. The sequence-level probabilistic modeling module inputs the convolutional features output by the multi-size convolutional network into a two-layer long short-term memory network to perform sequence-level modeling and output the reconstruction probability distribution of each base position. The comparison sample construction module constructs comparison learning sample pairs based on the sequences within the DNA sequence clustering dataset, generating positive sample pairs between the reference sequence and the reconstructed sequence, as well as negative sample pairs between the perturbed sequence and the reconstructed sequence; The contrastive loss calculation module performs contrastive loss calculation based on the contrastive learning sample pairs within the same sequence cluster, and optimizes the distribution structure of the feature space based on the positive attraction and negative repulsion mechanism. The joint optimization and output module constructs a joint loss function, which combines the cross-entropy loss corresponding to the reconstruction probability distribution of the two-layer long short-term memory network with the contrast loss in a weighted manner, and updates the network parameters through backpropagation to output the reconstructed DNA sequence set.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the memory, wherein the processor executes the program to implement the aforementioned DNA sequence reconstruction method based on multi-scale attention and contrastive learning.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the aforementioned DNA sequence reconstruction method based on multi-scale attention and contrastive learning.

[0016] The advantages of the above technical solutions adopted in this invention compared with the prior art are as follows: 1. To address the base deletion and mismatch issues in DNA sequencing data, a unified input representation tensor is constructed by combining clustering normalization and positional encoding. Simultaneously, by incorporating the MSATransformer structure and introducing multi-head attention mechanisms in the row and column directions, context-dependent features within the sequence and variation patterns of the same positional information between different sequences are extracted, thereby solving the problems of incomplete base information expression and local structural distortion, ultimately improving the contextual consistency and positional stability of sequence representation.

[0017] 2. Construct an Inter-Sequence and Intra-Sequence multi-head attention mechanism, and establish the similarity relationship between sequences and the global dependency path within sequences by leveraging the dual attention mapping relationship of sequence dimension and base dimension. On this basis, extract the displacement shift features caused by insertion and deletion errors through multi-size convolutional networks, and carry out sequence-level probabilistic modeling by combining a two-layer long short-term memory network, thereby improving the robust representation ability of local perturbations under high mismatch rate.

[0018] 3. A positive and negative sample construction strategy is introduced, and a contrastive loss function is constructed by combining base perturbation and cosine distance calculation methods. This contrastive loss function is then weighted and fused with the base reconstruction cross-entropy loss to form a joint loss function. Subsequently, temperature scaling and gradient backpropagation mechanisms are used to dynamically optimize the model parameters, so as to achieve stable reconstruction output of DNA base sequences under complex background perturbations, giving the model the performance advantages of strong robustness, high accuracy and good generalization ability. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0020] Figure 1 This is a flowchart of a DNA sequence reconstruction method based on multi-scale attention and contrastive learning. Detailed Implementation

[0021] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0024] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0025] Example 1: This embodiment provides a DNA sequence reconstruction method based on multi-scale attention and contrastive learning, including the following steps: Step 1: Collect multiple DNA sequence copies and perform length filtering to remove abnormal sequences that are outside the set range, and construct a standardized DNA sequence clustering dataset; Specifically, for target DNA fragments (such as specific genomic regions or stored data fragments), multiple sequence copies are obtained using high-throughput sequencing technologies (such as Illumina) (usually 10-50 copies per fragment) to reduce the impact of random errors in a single sequencing run; based on a preset length range for the target fragment (such as 200-500 bp, with a ±10% fluctuation margin), abnormal sequences (overly long contaminant fragments) that exceed the length range are removed. Using sequence similarity algorithms (such as CD-HIT), the filtered sequences are divided into clusters based on a similarity threshold of 90%-95%, ensuring that sequences in the same cluster correspond to the same target fragment. This results in a standardized DNA sequence cluster dataset that is "length compliant and clearly grouped," providing a unified input basis for subsequent modeling.

[0026] Step 2: One-hot encoding and padding are performed on the DNA sequences in the DNA sequence clustering dataset to generate a base vector matrix of uniform length. At the same time, a learnable position encoding matrix is ​​added to form an input representation tensor. Specifically, one-hot encoding is performed on each DNA sequence: the base characters A, T, C, and G are mapped to four-dimensional one-hot vectors, and a base vector matrix of dimension L×4 is constructed (L is the original sequence length). The base vector matrix is ​​padded with a uniform length by adding a zero vector to the end of the sequence to pad it to the set maximum length Lmax, resulting in an encoding matrix with dimension Lmax×4. Construct a position encoding matrix, which is a set of Lmax×4-dimensional learnable vectors, initialized with a standard normal distribution, and continuously optimized through parameter updates during the training phase; The position encoding matrix is ​​added element-wise to the encoded base vector matrix according to the position index to generate a position information input representation tensor with dimension Lmax×4; Stack the input representation tensors of all DNA sequences in the sample dimension to form a set of input feature tensors with dimension N×Lmax×4 (N is the number of sequences). The set of input feature tensors serves as the input to the MSATransformer structure, completing sequence normalization, position modeling, and structure alignment operations during the encoding phase.

[0027] Step 3: Input the input representation tensor into the MSATransformer structure, and perform attention mechanism operations in the row and column directions of the base dimension to extract context-dependent features and cross-sequence variation features to obtain a multi-dimensional fusion representation tensor; Specifically, the input representation tensor is input into the MSATransformer structure, an attention weight matrix is ​​constructed along the column direction, the attention distribution relationship of each base position between different DNA sequences is calculated, and after obtaining the attention weights in the column direction, the weight matrix is ​​normalized by the softmax function to generate the column-dimensional attention output tensor. The attention output tensor along the sample dimension is subjected to average pooling. During pooling, the representation vectors of all sequences are averaged at each base position to obtain a cross-sequence position feature representation tensor of dimension Lmax×d (d is the feature dimension). The input representation tensor is fed back into the MSATransformer structure to construct an attention graph along the row direction. The contextual relevance between different base positions in a single DNA sequence is calculated. After obtaining the row direction attention weights, the row dimension attention output tensor is generated by softmax normalization. The cross-sequence position feature representation tensor is copied and extended along the sample dimension to make it consistent with the row dimension tensor dimension. Then, it is fused in the channel dimension through tensor-level concatenation operation to form a multi-dimensional fused representation tensor containing cross-sequence and context information. The multi-dimensional fusion representation tensor is input into the Inter-Sequence multi-head attention module as the input feature tensor, where each channel corresponds to a feature representation at a different scale.

[0028] Step 4: Construct an Inter-Sequence multi-head attention mechanism based on the DNA sequence clustering dataset. The input of this mechanism is a multi-dimensional fusion representation tensor. Attention weights are assigned based on the similarity scores between sequences to generate a weighted sequence tensor. Specifically, it receives a multi-dimensional fusion representation tensor as input features. The dimensions of this input tensor are N×Lmax×d (N is the number of DNA sequences, Lmax is the uniform sequence length, and d is the number of channel dimension features). At the sample level, a sequence-level representation vector is constructed for each DNA sequence—generated by performing an average pooling operation on the feature vectors of all base positions, preserving the global representation features of the sequence; Based on the sequence-level representation vectors of all DNA sequences, cosine similarity is calculated pairwise (obtained by L2 norm normalization after dot product of feature vectors), and feature similarity score matrix is ​​constructed. Softmax normalization is performed on the similarity score matrix to generate an attention assignment weight matrix, which assigns a probability to the similarity of the remaining sequences on each target sequence. Perform a weighting operation on the input feature tensor along the sample dimension: the new representation vector of each DNA sequence is the weighted sum of the feature vectors of the remaining sequences (the weights are taken from the attention assignment weight matrix), generating a weighted feature tensor; In the weighted feature tensor, the representation of each DNA sequence incorporates the feature distribution of the remaining sequences in the cluster, and the feature dimension remains consistent with the original input. The weighted feature tensor is passed into the Intra-Sequence multi-head attention mechanism module for subsequent base-level dependency modeling.

[0029] Step 5: Perform Intra-Sequence multi-head attention mechanism operation on the weighted sequence tensor to extract the global dependencies between bases within the sequence and generate a feature enhancement tensor; Specifically, the received dimension is a weighted feature tensor of N×Lmax×d (N is the number of sequences, Lmax is the number of base positions, and d is the feature dimension of each position). Extract the complete set of position representation vectors along the base dimension in each DNA sequence. Each sequence contains Lmax position representation vectors. Each positional representation vector is mapped to a query vector, a key vector, and a value vector through a linear transformation layer. The transformation parameters are determined through training and optimization. The query vector, key vector, and value vector are copied to H attention heads respectively, and each attention head independently maintains a set of transformation parameters; In each attention head, a dot product operation is performed on the query vector and the key vector between all base positions, and the base dependency score matrix of Lmax×Lmax dimension is obtained by the inner product operation of the feature dimension. Perform softmax normalization on the dependency score matrix to generate an inter-base attention allocation coefficient matrix (each row represents the attention weight of the current base to all other bases). The set of value vectors is weighted and summed using the attention allocation coefficient matrix to generate an updated representation vector set for each attention head (keeping the position dimension unchanged, and the representation of each position after the update is integrated with the global dependency). Concatenate the set of updated representation vectors of all attention head outputs along the feature dimension to generate a fully connected attention output tensor with dimension Lmax×(H×d') (d' is the output dimension of a single attention head). The fully connected attention output tensor is input into a multi-size convolutional network as input features for local offset feature extraction.

[0030] Step 6: Input the feature enhancement tensor into a multi-size convolutional network to extract local offset features caused by insertion and deletion errors; The fully connected attention output tensor receives the output of the Intra-Sequence multi-head attention mechanism. The tensor has a dimension of Lmax×C (Lmax is the number of base positions in the DNA sequence, and C is the number of channel dimension features). While keeping the channel dimension unchanged, the tensor is copied into multiple branch inputs, each branch corresponding to a convolution kernel receptive field size. The set convolution kernel window sizes include 1×3, 1×5 and 1×7. Each branch is configured with a convolution kernel of the corresponding size and performs a one-dimensional convolution operation: the convolution kernel slides along the base dimension to extract local structural change patterns of bases in different proportions, and the edge padding uses symmetrical zero padding to keep the output length consistent with the input. After each branch's convolution output, batch normalization (the normalization parameters are dynamically learned during the training phase) and ReLU activation function transformation are performed sequentially. The standardized feature tensors output by each branch are concatenated along the channel dimension (and merged along the channel axis after tensor dimension expansion) to generate a multi-size convolutional feature fusion tensor with dimension Lmax×(C1+C2+C3) (C1, C2, and C3 are the number of output channels of each branch, respectively). The multi-size convolutional feature fusion tensor is used as the input feature of the next stage long short-term memory network to participate in the context state modeling of the sequence prediction stage.

[0031] Step 7: Input the convolutional features output by the multi-size convolutional network into a two-layer long short-term memory network to perform sequence-level modeling and output the reconstruction probability distribution of each base position; Specifically, the input sequence features are received as a multi-size convolutional feature fusion tensor, and the dimension of this input tensor is Lmax×C (Lmax is the number of base positions in the DNA sequence, and C is the channel dimension after fusion). The input sequence features are fed into the first layer of the bidirectional long short-term memory network, and state propagation is performed along the forward and reverse time steps respectively: forward propagation is from the 1st base to the Lmaxth base, and reverse propagation is from the Lmaxth base to the 1st base. The output dimensions are both forward and reverse state sequences with dimensions of Lmax×h (h is the state dimension in each direction). Perform a concatenation operation on the feature dimension of the forward state sequence and the reverse state sequence to form a first-layer output state tensor with dimension Lmax×2h; The output state tensor of the first layer is passed into the second layer bidirectional long short-term memory network to perform the second round of forward and backward state propagation, and the output state tensor of the second layer with the same dimension is output. The second-layer state tensor is input into the linear transformation layer (the transformation weights are obtained through training and fitting), and the state vector at each base position is mapped to an output vector of dimension 4 (corresponding to the prediction scores of the four bases A, T, C, and G respectively). Softmax normalization is performed on the output vector to generate the probability distribution of the four bases at each base position. The prediction results of all positions together constitute the probability representation matrix of the final reconstructed sequence.

[0032] Step 8: Based on the sequences in the DNA sequence clustering dataset, construct contrast learning sample pairs to generate positive sample pairs between the reference sequence and the reconstructed sequence, and negative sample pairs between the perturbed sequence and the reconstructed sequence; Specifically, a set of high-quality reference sequences is selected within each DNA sequence cluster. These reference sequences are derived from high-confidence sequences retained after length filtering in the original input. For each DNA sequence cluster, a reference sequence from the same source within the cluster is extracted, and a one-to-one mapping relationship is constructed to form a set of positive sample pairs between the reference sequence and the reconstructed sequence. Within each DNA sequence cluster, base perturbation is performed on the existing reference sequence set: the perturbation types include transition perturbation (substitution between purines or pyrimidines) and transversion perturbation (substitution between purines and pyrimidines), and the perturbation sites are sampled evenly distributed according to the preset error rate; Each perturbated sequence is paired with its corresponding reconstructed sequence to generate a set of negative sample pairs consisting of perturbated and reconstructed sequences. All pairing relationships maintain the same structure as positive sample pairs. The positive sample pairs are labeled as similar pairs and the negative sample pairs are labeled as dissimilar pairs. The label information is stored in the comparison training sample index to form a labeled input sample structure. All positive and negative sample pairs are merged into a unified format of contrastive training input batch data (each batch contains multiple positive and negative pairings), which is then input into the contrastive loss calculation module as the basic data for contrastive feature optimization during the training phase.

[0033] Step 9: Within the same sequence cluster, perform contrastive loss calculation based on the contrastive learning sample pairs, and optimize the distribution structure of the feature space based on the positive attraction and negative repulsion mechanism; Specifically, within each DNA sequence cluster, feature representation vectors of each sequence in the positive and negative sample pairs are extracted. These vectors are obtained by average pooling of the reconstructed state tensor output by the two-layer long short-term memory network, and are fixed-length vectors with dimensions set to d. For the two feature representation vectors in the positive sample pair, perform unit L2 norm normalization, and then calculate the cosine similarity by dividing the vector inner product by the norm product (the value range is [-1,1]). Perform the same operation on the two feature representation vectors in the negative sample pair, calculate the cosine similarity, and all similarity values ​​together constitute the positive and negative sample similarity set; The similarity set is input into the contrastive loss function of the NT-Xent structure: a unique matching pair is set for each positive sample pair, and the remaining negative sample pairs are used as the contrast denominator to construct a softmax contrastive probability structure. An adjustable hyperparameter temperature scaling factor τ (determined through validation set performance fitting) is introduced into the contrastive loss function to control the distribution range of similarity values; the final contrast probability is calculated using the exponential mapping function exp(sim / τ). Using each reconstructed sequence as an anchor point, the batch loss term is constructed by minimizing its distance from positive samples and maximizing its distance from negative samples, with the cross-entropy loss of sample pairs as the training objective. The contrast loss of all positive and negative sample pairs in each batch is summed and averaged to generate the total contrast loss value within the cluster. The total contrastive loss value within the cluster is output and passed to the joint loss function, and then weighted and combined with the base cross-entropy loss.

[0034] Step 10: Construct a joint loss function by weighting the cross-entropy loss corresponding to the reconstruction probability distribution of the two-layer long short-term memory network output with the contrast loss, and update the network parameters through backpropagation to output the reconstructed DNA sequence set.

[0035] Specifically, in each training batch, a base probability distribution matrix with dimension Lmax×4 is extracted (corresponding to the prediction scores of the four base classes A, T, C, and G at each base position in each DNA sequence). For each base position, extract the corresponding one-hot base tag vector (which corresponds one-to-one with the predicted probability), calculate the cross-entropy loss at that position, and then average the losses at all positions to obtain the batch-level base reconstruction loss term. Simultaneously extract the feature representation vectors of all positive and negative sample pairs in the current batch cluster, and obtain the batch-level contrast loss term according to the aforementioned contrast loss calculation process; Set the weighting coefficients α and β for cross-entropy loss and contrast loss (the coefficient values ​​are determined by performance tuning fitting), and construct the joint loss function according to the method of "joint loss L=α×L_CE+β×L_CL" (L_CE is cross-entropy loss and L_CL is contrast loss); The joint loss function is used as the input to the backpropagation calculation module for the training objective. The chain partial derivative expansion is performed on all trainable parameters to calculate the gradient information corresponding to each parameter. The gradient information is input into the parameter optimizer, and the Adam optimizer is used to update the parameters (hyperparameters such as learning rate and momentum are set in the initialization phase and dynamically adjusted during the training process). In each training epoch, the joint optimization process of cross-entropy and contrastive loss is repeated to progressively update all weight parameters in the Transformer, attention module, convolutional network, and long short-term memory network. During training, an early stopping strategy and a validation set performance evaluation mechanism are set. The final output is the set of neural network parameters in the converged state, as well as DNA sequence prediction results with stable reconstruction capabilities.

[0036] Example 2: This embodiment provides a DNA sequence reconstruction system based on multi-scale attention and contrastive learning, including: The dataset standardization module collects multiple DNA sequence copies and performs length filtering to remove abnormal sequences that are outside the set range, thus constructing a standardized DNA sequence clustering dataset. The input tensor construction module performs one-hot encoding and padding on the DNA sequences in the DNA sequence clustering dataset to generate a base vector matrix of uniform length. At the same time, it adds a learnable position encoding matrix to form an input representation tensor. The multi-dimensional feature extraction module inputs the input representation tensor into the MSATransformer structure and performs attention mechanism operations in the row and column directions of the base dimension to extract context-dependent features and cross-sequence variation features, thereby obtaining a multi-dimensional fusion representation tensor. The inter-sequence attention modeling module constructs an Inter-Sequence multi-head attention mechanism based on the DNA sequence clustering dataset. The input of this mechanism is a multi-dimensional fusion representation tensor, and attention weights are assigned through the similarity scores between sequences to generate a weighted sequence tensor. The intra-sequence attention modeling module performs Intra-Sequence multi-head attention mechanism operations on the weighted sequence tensor to extract the global dependencies between bases within the sequence and generate a feature enhancement tensor. The local offset feature extraction module inputs the feature enhancement tensor into a multi-size convolutional network to extract local offset features caused by insertion and deletion errors. The sequence-level probabilistic modeling module inputs the convolutional features output by the multi-size convolutional network into a two-layer long short-term memory network to perform sequence-level modeling and output the reconstruction probability distribution of each base position. The comparison sample construction module constructs comparison learning sample pairs based on the sequences within the DNA sequence clustering dataset, generating positive sample pairs between the reference sequence and the reconstructed sequence, as well as negative sample pairs between the perturbed sequence and the reconstructed sequence; The contrastive loss calculation module performs contrastive loss calculation based on the contrastive learning sample pairs within the same sequence cluster, and optimizes the distribution structure of the feature space based on the positive attraction and negative repulsion mechanism. The joint optimization and output module constructs a joint loss function, which combines the cross-entropy loss corresponding to the reconstruction probability distribution of the two-layer long short-term memory network with the contrast loss in a weighted manner, and updates the network parameters through backpropagation to output the reconstructed DNA sequence set.

[0037] The above modules can be deployed on the same device or distributed devices; the division of modules is only a functional logic description and does not limit the specific physical boundaries or implementation order.

[0038] Example 3: An electronic device is provided for running the aforementioned "DNA sequence reconstruction method based on multi-scale attention and contrastive learning". The electronic device includes a processor, a memory, and optional communication interfaces / display devices / input devices, etc.; the memory stores a computer program that can run on the processor, and when the processor executes the program, it implements steps one through ten of the method described in Embodiment 1.

[0039] The electronic device hardware can be one of a server, personal computer, workstation, industrial controller, edge computing device, or mobile terminal; the processor can be a general-purpose CPU, GPU, NPU, FPGA, or a combination thereof; the memory can be RAM, ROM, flash memory, or disk array. The device can interact with local / remote data storage (acquiring observation data and outputting inversion results) through a communication interface. The above hardware configuration does not constitute a limitation of the present invention.

[0040] Example 4: A computer-readable storage medium storing a computer program, which, when run on a processor of an electronic device, causes the program to perform steps one through ten of the method described in Embodiment 1; the storage medium may be a disk, optical disk, flash memory, solid-state drive, read-only memory, random access memory, or any combination of the above media.

[0041] Those skilled in the art will understand that the modules or steps described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, which can then be stored in a storage device for execution by a computer device. Alternatively, they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. This disclosure is not limited to any particular combination of hardware and software.

[0042] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A DNA sequence reconstruction method based on multi-scale attention and contrastive learning, characterized in that, Includes the following steps: Multiple DNA sequence copies were collected and length-filtered to remove abnormal sequences outside the set range, thus constructing a standardized DNA sequence clustering dataset. The DNA sequences in the DNA sequence clustering dataset are one-hot encoded and padded to generate a base vector matrix of uniform length. At the same time, a learnable position encoding matrix is ​​added to form an input representation tensor. The input representation tensor is input into the MSATransformer structure, and attention mechanism operations are performed in the row and column directions of the base dimension to extract context-dependent features and cross-sequence variation features, thereby obtaining a multi-dimensional fusion representation tensor. An Inter-Sequence multi-head attention mechanism is constructed based on the DNA sequence clustering dataset. The input of this mechanism is a multi-dimensional fusion representation tensor. Attention weights are assigned based on the similarity scores between sequences to generate a weighted sequence tensor. The Intra-Sequence multi-head attention mechanism is applied to the weighted sequence tensor to extract the global dependencies between bases within the sequence and generate a feature-enhanced tensor. The feature enhancement tensor is input into a multi-size convolutional network to extract local offset features caused by insertion and deletion errors. The convolutional features output by the multi-size convolutional network are input into a two-layer long short-term memory network for sequence-level modeling, and the reconstruction probability distribution of each base position is output. Based on the sequences within the DNA sequence clustering dataset, construct contrastive learning sample pairs to generate positive sample pairs between the reference sequence and the reconstructed sequence, as well as negative sample pairs between the perturbed sequence and the reconstructed sequence. Within the same sequence cluster, the contrastive loss is calculated based on the contrastive learning sample pairs, and the distribution structure of the feature space is optimized based on the positive attraction and negative repulsion mechanism; A joint loss function is constructed, which combines the cross-entropy loss corresponding to the reconstruction probability distribution of the two-layer long short-term memory network output with the contrast loss in a weighted manner, and updates the network parameters through backpropagation to output the reconstructed DNA sequence set.

2. The DNA sequence reconstruction method based on multi-scale attention and contrastive learning according to claim 1, characterized in that, The input represents the tensor obtained in the following way: Perform one-hot encoding on each DNA sequence to map base characters to a set of four-dimensional one-hot vectors, and construct a base vector matrix; The base vector matrix is ​​padded to a uniform length by adding a zero vector to the end of the sequence to fill it to the preset length limit, thus obtaining a coding matrix of uniform dimension. Construct a positional encoding matrix, which is a learnable vector matrix whose vector dimension is consistent with that of the base vector matrix; The position encoding matrix and the encoded base vector matrix are added element by element according to their position indices to form an input representation tensor with position information; The input representation tensors of all DNA sequences are stacked along the sample dimension to generate the set of input feature tensors for the DNA sequence clustering dataset.

3. The DNA sequence reconstruction method based on multi-scale attention and contrastive learning according to claim 1, characterized in that, The method for obtaining the multi-dimensional fusion representation tensor is as follows: The input representation tensor is input into the MSATransformer structure, and the column direction attention mechanism operation is performed to calculate the feature correlation between the same base position in different DNA sequences, generating a column dimension attention output tensor. Perform average pooling operation along the sample dimension on the column-dimensional attention output tensor to generate a cross-sequence position feature representation tensor; The input representation tensor is input again into the MSATransformer structure to perform row direction attention mechanism operation, calculate the dependency between different base positions in a single DNA sequence, and generate a row dimension attention output tensor. The cross-sequence position feature representation tensor and the row-dimensional attention output tensor are concatenated in the channel dimension to form a multi-dimensional fusion representation tensor.

4. The DNA sequence reconstruction method based on multi-scale attention and contrastive learning according to claim 1, characterized in that, The execution process of the multi-size convolutional network includes: Receive the feature-enhanced tensor and duplicate the tensor multiple times as multi-branch inputs while maintaining consistency in the channel dimension; Set convolution kernel windows of different sizes on each branch; One-dimensional convolution operations are performed in each branch to extract local base shift features within different ranges; Perform batch normalization and activation function transformation operations on the output of each branch to generate a set of standardized feature subtensors; All feature sub-tensors are concatenated along the channel dimension to generate a multi-size convolutional feature fusion tensor, which is the convolutional feature.

5. The DNA sequence reconstruction method based on multi-scale attention and contrastive learning according to claim 1, characterized in that, The execution process of the two-layer long short-term memory network includes: The convolutional features are used as input sequence features and fed into the first layer of the bidirectional long short-term memory network. Information is propagated along the forward and reverse time steps respectively, and the forward and reverse state sequences are output. The forward state sequence and the reverse state sequence are concatenated along the feature dimension to form the first layer output state tensor. The first layer output state tensor is used as input and passed into the second layer bidirectional long short-term memory network. The forward and backward state propagation operations are performed again to generate the second layer state output tensor. The second layer state output tensor is input into the linear transformation layer, where the state vector is mapped to a set of output probability vectors with a dimension of 4. Normalization is performed on the set of output probability vectors to obtain the reconstruction probability distribution of A, T, C, and G at each base position.

6. The DNA sequence reconstruction method based on multi-scale attention and contrastive learning according to claim 1, characterized in that, The comparative loss is calculated as follows: Within each DNA sequence cluster, the feature representation vector of each sequence in the positive and negative sample pairs is first extracted. Based on the feature representation vector, the cosine similarity between positive sample pairs and the cosine similarity between negative sample pairs are calculated respectively. The similarity scores of positive sample pairs and negative sample pairs are input into the contrastive loss function; A temperature scaling factor is introduced into the contrast loss function to adjust the relative distribution difference between sample pairs through an exponential mapping method. The distance between positive sample pairs is compressed while the distance between negative sample pairs is kept separate, and the difference optimization process is completed by minimizing the loss function value; The contrast loss in each batch is summarized and calculated, and the total contrast loss value within the cluster is output.

7. The DNA sequence reconstruction method based on multi-scale attention and contrastive learning according to claim 1, characterized in that, The process of constructing and training the joint loss function is as follows: In each training batch, the probability distribution of DNA sequence reconstruction and the corresponding base tag vector are extracted, the cross-entropy loss at each base position is calculated, and the batch-level base reconstruction loss term is obtained by averaging over all positions. Extract the feature similarity difference between all positive and negative sample pairs in the current batch cluster, and obtain the batch-level contrast loss term according to the contrast loss calculation method; Set weight coefficients α and β for cross-entropy loss and contrastive loss, and perform a weighted summation operation on the two loss terms to form a joint loss function; The joint loss function is input into the backpropagation calculation module, and chain partial derivative expansion is performed on all trainable parameters to obtain the gradient information corresponding to each parameter. The gradient information is input into the Adam optimizer to perform parameter update operations and iteratively adjust the parameter weights. During the training rounds, the joint loss optimization process is repeated, and the final output is the converged set of network parameters and the reconstructed set of DNA sequences.

8. A DNA sequence reconstruction system based on multi-scale attention and contrastive learning, characterized in that, include: The dataset standardization module collects multiple DNA sequence copies and performs length filtering to remove abnormal sequences that are outside the set range, thus constructing a standardized DNA sequence clustering dataset. The input tensor construction module performs one-hot encoding and padding on the DNA sequences in the DNA sequence clustering dataset to generate a base vector matrix of uniform length. At the same time, it adds a learnable position encoding matrix to form an input representation tensor. The multi-dimensional feature extraction module inputs the input representation tensor into the MSATransformer structure and performs attention mechanism operations in the row and column directions of the base dimension to extract context-dependent features and cross-sequence variation features, thereby obtaining a multi-dimensional fusion representation tensor. The inter-sequence attention modeling module constructs an Inter-Sequence multi-head attention mechanism based on the DNA sequence clustering dataset. The input of this mechanism is a multi-dimensional fusion representation tensor, and attention weights are assigned through the similarity scores between sequences to generate a weighted sequence tensor. The intra-sequence attention modeling module performs Intra-Sequence multi-head attention mechanism operations on the weighted sequence tensor to extract the global dependencies between bases within the sequence and generate a feature enhancement tensor. The local offset feature extraction module inputs the feature enhancement tensor into a multi-size convolutional network to extract local offset features caused by insertion and deletion errors. The sequence-level probabilistic modeling module inputs the convolutional features output by the multi-size convolutional network into a two-layer long short-term memory network to perform sequence-level modeling and output the reconstruction probability distribution of each base position. The comparison sample construction module constructs comparison learning sample pairs based on the sequences within the DNA sequence clustering dataset, generating positive sample pairs between the reference sequence and the reconstructed sequence, as well as negative sample pairs between the perturbed sequence and the reconstructed sequence; The contrastive loss calculation module performs contrastive loss calculation based on the contrastive learning sample pairs within the same sequence cluster, and optimizes the distribution structure of the feature space based on the positive attraction and negative repulsion mechanism. The joint optimization and output module constructs a joint loss function, which combines the cross-entropy loss corresponding to the reconstruction probability distribution of the two-layer long short-term memory network with the contrast loss in a weighted manner, and updates the network parameters through backpropagation to output the reconstructed DNA sequence set.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and running thereon, characterized in that, When the processor executes the program, it implements the DNA sequence reconstruction method based on multi-scale attention and contrastive learning as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements a DNA sequence reconstruction method based on multi-scale attention and contrastive learning as described in any one of claims 1-7.

Images