Deep learning system for identifying disease-related biomarkers from DNA sequences

By using a deep learning system for DNA sequence preprocessing and embedding, combined with multi-scale attention analysis and causal inference modules, the problem of insufficient identification capability of unknown functional variants in non-coding regions in traditional methods is solved, achieving high-precision biomarker identification and causal verification.

CN123120045APending Publication Date: 2026-11-17BEIJING XIANYUN QIYUAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611436452.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional DNA sequence biomarker identification methods are insufficient in identifying unknown functional variant sites in non-coding regions, making it difficult to capture long-range dependencies and complex regulatory logics. They also have a high false positive rate and lack multi-scale visualization tools, making it difficult to achieve causal inference.

Method used

A deep learning system is used for sequence preprocessing and embedding. Combined with multi-scale attention analysis and causal inference modules, the deep learning model internalizes the genomic functional grammar, generates a multi-scale attention weight distribution map, and calculates causal effect scores through virtual base editing to screen out pathogenic mutation sites or functional variant regions.

Benefits of technology

It significantly improves the accuracy of non-coding region functional decoding, reduces the false positive rate, provides multi-scale visualization tools to support intuitive interpretation, realizes full-scale feature analysis and causal verification, and improves the accuracy and practicality of biomarker identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123120045A_ABST
    Figure CN123120045A_ABST
Patent Text Reader

Abstract

This invention discloses a deep learning system for identifying disease-related biomarkers from DNA sequences, specifically relating to the interdisciplinary fields of bioinformatics and deep learning. The system converts FASTA format DNA sequences into numerical matrices through quality control, standardization, and adaptive k-mer segmentation. An improved 24-layer Transformer architecture genome model is input, and after multi-task pre-training, a 512-dimensional functional feature vector of the non-coding region is decoded. Site contributions are quantified through multi-scale attention analysis of single nucleotides, functional motifs, and long-distance interactions. Causal effect scores are calculated through virtual base editing and functional difference analysis. A three-level screening strategy is employed to fuse multi-dimensional indicators, outputting categorized biomarkers and a visual report. The system includes data input, model processing, visualization interaction, and storage units, achieving accurate identification, causal verification, and interactive analysis, thus improving the accuracy and practicality of biomarker identification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of bioinformatics and deep learning, and more specifically, to a deep learning system for identifying disease-related biomarkers from DNA sequences. Background Technology

[0002] Biomarkers serve as the core basis for disease diagnosis, prognostic assessment, and screening of therapeutic targets. Their precise identification from DNA sequences is of great significance for precision medicine. Approximately 98% of the human genome consists of non-coding regions. Although these regions do not directly encode proteins, they participate in the occurrence and development of various diseases by regulating gene expression and chromatin conformation, and are an important source of disease-related biomarkers. However, traditional DNA sequence biomarker identification methods have significant limitations: First, they rely on prior knowledge of known functional sites and have weak ability to identify variant sites with unknown functions in non-coding regions, making it difficult to discover novel biomarkers; second, traditional machine learning models (such as support vector machines and random forests) are unable to capture long-range dependencies and complex regulatory logic in DNA sequences, resulting in insufficient comprehensiveness and accuracy in feature extraction; third, they lack causal effect verification of candidate sites, and biomarkers screened solely through correlation analysis suffer from high false positive rates; fourth, the visualization methods are limited and cannot intuitively present sequence functional characteristics and site interactions at multiple scales, which is not conducive to subsequent verification by researchers. Deep learning technology has a natural advantage in processing massive sequence data, but its application in DNA sequence biomarker identification still faces bottlenecks: existing deep learning-based methods have not fully internalized the inherent functional grammar of the genome, resulting in limited accuracy in decoding the functions of non-coding regions; the lack of a combination of multi-scale attention mechanisms and in vivo computational perturbation analysis makes it difficult to achieve the leap from "association analysis" to "causal inference"; at the same time, the lack of interactive visualization tools makes it difficult to effectively interpret and apply the model output results. Therefore, developing a DNA sequence biomarker identification scheme that can accurately decode the functions of non-coding regions, verify causal effects, and support interactive analysis has become an urgent technical problem to be solved in this field. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a deep learning system for identifying disease-related biomarkers from DNA sequences, thereby addressing the problems raised in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: Sequence preprocessing and embedding module: Obtain the DNA sequence to be tested, perform k-mer segmentation on the DNA sequence, and convert it into a numerical matrix as model input; Feature extraction and non-coding region decoding module: Input the numerical matrix into the trained deep learning model. The deep learning model is based on the genome basic model architecture. Through unsupervised pre-training on massive genome data, it internalizes the inherent functional syntax and regulatory logic of DNA sequence and outputs the functional feature vector of non-coding region in the DNA sequence. Multi-scale attention analysis module: Through the multi-scale attention mechanism integrated within the deep learning model, the functional feature vector is analyzed to generate attention weight distribution maps at different biological scales, including the single nucleotide scale, the functional motif scale, and the long-distance chromatin interaction scale. The perturbation and causal inference module performs virtual base editing on candidate sites in the DNA sequence, inputs the perturbated sequence into the deep learning model, and calculates the causal effect score of the candidate sites by comparing the differences in the model output before and after perturbation. Biomarker output module: Based on the causal effect score and attention weight distribution, pathogenic mutation sites or functional variation regions are screened out, and disease-related biomarker prediction reports are generated.

[0005] Preferably, in the sequence preprocessing and embedding module, the raw DNA sequence data to be tested is acquired, and the data format is preferably the FASTA standard bioinformatics file format; for the input sequence data, a basic quality control process is first executed to automatically filter low-quality sequence fragments containing more than 5% N bases, remove abnormal sequences with a length less than 100bp or greater than 10000bp, and remove redundant sequences through sequence homology detection (based on the BLAST algorithm, with a similarity threshold set to 95%). During the preprocessing stage, sequence standardization is performed: all qualified sequences undergo uniform orientation calibration (based on promoter region features or homologous sequence alignment, inverted complementary sequences are adjusted to forward sequences), and non-base characters in the sequences are removed to generate pure base sequence strings; for very long sequences (length exceeding 5000bp), a sliding window segmentation strategy is used for segmentation, while recording the original sequence location information of each segment; for short sequences (length less than 500bp), a padded end strategy (padded characters are pseudo-bases composed of random bases) is used to unify the sequence length to 500bp; In the k-mer segmentation scheme, for capturing local features and short-range dependencies of the sequence, an adaptive k-value selection strategy is adopted (providing four optional parameters k=3, 4, 5, and 6, automatically recommending the optimal k value based on the sequence length: k=3 or 4 is recommended for short sequences, and k=5 or 6 is recommended for long sequences); the sequence is segmented according to a sliding window (window size k, step size 1) to generate k-mer fragments (e.g., the sequence "ATCGG" generates "ATC", "TCG", and "CGG" when k=3); then, each k-mer fragment is mapped to a fixed-dimensional dense vector using a pre-constructed k-mer dictionary (based on the frequency statistics of all k-mer fragments in the training dataset, retaining the top 99.9% of the k-mers with the highest frequency, and classifying the rest as "unseen" and mapping them to a unified vector). The final generated vector has a dimension of [missing information - likely a specific dimension]. Numerical matrices.

[0006] Preferably, in the feature extraction and non-coding region decoding module, the deep learning model used is based on an improved Transformer architecture to build a genome basic model. The overall architecture includes a feature input layer, a multi-layer encoder stack layer, a cross-scale feature fusion layer, and a functional decoding output layer. The encoder stack layer consists of 24 bidirectional Transformer encoders. Each encoder layer integrates 32 multi-head self-attention mechanisms and feedforward neural networks. The head dimension of the multi-head self-attention mechanism is set to 64. The feedforward neural network includes two linear transformation layers (hidden layer dimension is 2048, output layer dimension is 1024) and a GELU activation function. At the same time, layer normalization and residual connections are introduced between each encoder layer. The unsupervised pre-training process of the deep learning model is based on a large-scale genomic dataset. Training data includes complete reference genome sequences, non-coding region functional annotation data from the ENCODE database, epigenetic modification maps from Roadmap Epigenomics, and gene expression data from GTEx. A multi-task collaborative training mechanism is designed for the pre-training phase. Tasks include base masking recovery (randomly masking 15% of bases, with 80% replaced by mask identifiers, 10% replaced by random bases, and 10% retaining the original bases; the model needs to predict the original base types), regulatory element boundary prediction, chromatin open state inference, and transcription factor binding site prediction (predicting regions in the sequence that bind to known transcription factors). An adaptive learning rate strategy is used during training, with an initial learning rate of 1e-4, decreasing to 0.9 times the current value every 200,000 iterations. Mixed-precision training and gradient accumulation techniques are combined to improve training efficiency, resulting in a total of 120 training iterations. Tens of thousands of times, the early stopping mechanism ensures the model's generalization ability, ultimately enabling the deep learning model to internalize the inherent functional syntax of DNA sequences, including the conserved patterns of base arrangement, the combination rules of functional motifs (such as the cooperative distribution logic of enhancer and transcription factor binding sites), the sequence features of long-distance chromatin interactions, and the association logic between epigenetic modifications and sequence features, thus constructing a deep mapping relationship from sequence to function.

[0007] Preferably, in the multi-scale attention analysis module, for attention analysis at the single nucleotide scale, the focus is on base-level contribution evaluation. Mathematical modeling eliminates weight differences in multi-layer and multi-head attention, achieving accurate quantification of single-base contribution: First, the multi-head self-attention weight matrix of the bottom-level encoder (layers 1-8) is called. Assuming the model contains 8 bottom-level encoders, each containing 32 attention heads, for the i-th nucleotide in the input sequence, the original attention value output by the h-th attention head in layer l is... The original attention value directly reflects the contribution of a single base to the attributes of each dimension in the functional feature vector. To eliminate the weight differences between different attention heads and encoder layers, the original score is first normalized using Min-Max, mapping the score of each base to the 0-1 interval. The normalization formula is as follows:

[0008] in, This represents the normalized raw attention value, where N represents the total number of nucleotides in the input sequence. This represents the minimum value among all N original attention values ​​of the input sequence under the h-th attention head in the l-th layer; This represents the maximum value among all N original attention values ​​of the input sequence under the h-th attention head in the l-th layer; Subsequently, the normalized scores of each layer and each attention head are weighted and summed according to the feature importance weights of each encoder layer to obtain the comprehensive attention weight of the i-th nucleotide. The specific calculation method is as follows:

[0009] in, Let the total attention weight be represented as the sum of the attention weights for the i-th nucleotide. Let H represent the weights of the l-th layer, and let H represent the number of attention heads. Based on this, a single nucleotide-scale attention weight distribution map is generated, with sequence position as the x-axis and comprehensive attention weights. The vertical axis is used as the plotting axis, presented in the form of a line graph or heatmap; where, The nucleotide markers are designated as "high contribution sites". The nucleotides are labeled as "low contribution sites"; this visually demonstrates the differences in the impact of a single base on the function of the non-coding region, providing a direct basis for the pathogenicity analysis of single nucleotide variants.

[0010] Preferably, the computational perturbation and causal inference module integrates a candidate site screening engine, a virtual base editing toolset, a functional difference analysis unit, and a causal effect scoring unit to achieve fully automated processing from site selection to causal determination. It also supports user-defined editing parameters to improve the flexibility and targeting of the analysis. The candidate site screening process relies heavily on the output of the multi-scale attention analysis module, combined with sequence analysis results to accurately identify high-priority targets: firstly, the comprehensive attention weights at the single nucleotide scale are extracted. High-contribution sites, core motif regions with importance scores greater than 0.7 at the functional motif scale, and correlation coefficients at the long-distance chromatin interaction scale. The regulatory-related sites were identified to construct an initial candidate site set. Subsequently, redundant sites were removed (adjacent sites with a distance of less than 5 bp were eliminated) and functional association screening was performed (sites with a correlation ≥ 0.7 with the core functional attributes of non-coding regions—such as regulatory activity and transcription factor binding affinity) to form a final, simplified candidate site list. The list includes the sequence position of each site, its biological scale, attention-related parameters, and functional annotation information. The virtual base editing toolset supports multiple editing modes to simulate in vivo base editing experiments: users can choose from four editing modes: single-base mutation, multi-base insertion, multi-base deletion, and motif replacement; during the editing process, the location information and format of the original sequence are automatically preserved, and the edited sequence undergoes the same preprocessing and embedding process as the original sequence to ensure that the input format of the perturbation sequence and the original sequence are consistent, avoiding functional prediction deviations caused by differences in data processing; for long-distance chromatin interaction-related sites, the corresponding sites of the target site and the distant associated region R are edited simultaneously to simulate the cooperative perturbation effect of the regulatory network; The functional difference analysis unit is responsible for quantifying the magnitude of change in the model output before and after perturbation, providing a data basis for causal effect assessment: the virtually edited perturbation sequence is input into the trained deep learning model (using the same model parameters as the original sequence analysis), and the output non-coding region functional feature vector is compared with the original sequence functional feature vector dimension by dimension.

[0011] Preferably, the biomarker output module constructs a complete output system of multi-dimensional indicator fusion, hierarchical screening, and report visualization. The module integrates an indicator fusion engine, hierarchical screening unit, report generation toolset, and result visualization unit to realize fully automated processing from data integration to report output. At the same time, it supports user-defined screening thresholds and report display formats to meet the personalized needs of different application scenarios. In the indicator fusion stage, two core data sources are first integrated: one is the causal effect score and grading results output by the perturbation and causal inference module, and the other is the attention weight data at various biological scales generated by the multi-scale attention analysis module. On this basis, external authoritative database resources are further linked to supplement additional biological annotation information of candidate sites, including matching with known disease variant databases, whether they belong to reported regulatory element regions, related transcription factor binding information, epigenetic modification data, and inter-species conservation analysis results, to form a biomarker evaluation dataset. The stratified screening unit employs a three-tiered screening strategy—initial screening, refined screening, and final review—to ensure the high reliability and practicality of the output biomarkers. In the initial screening stage, causal effect scoring is the core criterion, prioritizing the retention of "strong causal sites" while incorporating some "moderate causal sites" based on user needs, and directly eliminating "weak causal sites," thus initially narrowing down the candidate biomarker pool. In the refined screening stage, multi-scale attention weight data is used for secondary validation. Candidate sites retained in the initial screening are required to meet preset standards in attention metrics at the corresponding biological scale; that is, single nucleotide core sites must meet the comprehensive attention weight, and functional motif-related sites must meet the relevant standards. To meet the importance score, long-distance regulatory sites must meet the correlation coefficient and have a significant perturbation response in the functional dimension. The reliability of biomarkers is improved through cross-validation of attention indicators and causal effects. In the final review stage, biological rationality verification is introduced. Combined with external database annotation information, candidate sites with biological contradictions are eliminated (such as sites that have been confirmed as benign variations by authoritative databases, sites located in non-functional regions and without conservation). At the same time, the screening results are classified and integrated according to the site type, and divided into three categories: "single nucleotide pathogenic mutations", "functional motif variation regions" and "long-distance regulatory variation sites". Within each category, they are sorted from high to low according to the causal effect score to form the final list of disease-related biomarker candidates. The report generation toolset is responsible for integrating the selected biomarker information into a predictive report. The report is divided into three main sections: core results, detailed annotations, and analysis descriptions. The core results section lists the biomarker candidate list, including basic information for each region (sequence location, chromosome location, variant type), core evaluation indicators (causal effect score and level, attention weight key parameters), screening level (priority is divided into first-level, second-level, and third-level from high to low), and priority suggestions for subsequent validation. The detailed annotation section displays multi-dimensional biological information for each biomarker, including database matching details (whether it is a known disease-related variant, matching database name and variant ID), functional element attribution (whether it belongs to regulatory regions such as enhancers and promoters), transcription factor binding associations (the names of transcription factors that may be affected and changes in binding strength), epigenetic modification characteristics (whether there are epigenetic modification sites and the trend of changes after perturbation), interspecies conservation analysis results (cross-species sequence similarity and conserved region annotation), and speculation on potential pathogenic mechanisms.

[0012] The technical effects and advantages of this invention are as follows: This invention transforms DNA sequences into numerical matrices through sequence preprocessing and embedding modules, and then parses them using feature extraction and non-coding region decoding modules to obtain 512-dimensional functional feature vectors. A multi-scale attention analysis module generates weight distribution maps from three scales: single bases, functional motifs, and long-range interactions. A perturbation and causal inference calculation module calculates and quantifies causal effect scores through virtual base editing and before-and-after comparison. Finally, a biomarker output module integrates multi-source indicators and outputs a categorized and ranked list of biomarker candidates and a structured report after three levels of screening. This invention utilizes a deep learning model pre-trained unsupervised on massive genomic data to internalize the functional syntax and regulatory logic of DNA sequences, significantly improving the accuracy of non-coding region functional decoding and solving the problem of insufficient ability of traditional methods to identify unknown functional variations. By integrating a multi-scale attention mechanism, it achieves full-scale feature analysis from single nucleotides to long-distance chromatin interactions, comprehensively capturing the functional associations of DNA sequences and providing multi-dimensional evidence for biomarker screening. The invention introduces in vivo computational perturbation and causal inference modules, using virtual base editing to verify the causal effects of candidate sites, effectively reducing the false positive rate of biomarkers. It provides multi-scale, interactive visualization tools, allowing users to intuitively interpret model output results and track long-distance interaction regions, providing clear guidance for subsequent experimental verification and enhancing the practicality of the technology. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the module connection of the present invention.

[0014] Figure 2 This is a schematic diagram of the multi-scale attention analysis process of the present invention.

[0015] Figure 3 This is a schematic diagram of the computational perturbation and causal inference process of the present invention. Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 1 As shown, the present invention provides a deep learning system for identifying disease-related biomarkers from DNA sequences, comprising: Sequence preprocessing and embedding module: Obtain the DNA sequence to be tested, perform k-mer segmentation on the DNA sequence, and convert it into a numerical matrix as model input; In the sequence preprocessing and embedding module, the raw DNA sequence data to be tested is acquired, preferably in the FASTA standard bioinformatics file format. For the input sequence data, a basic quality control process is first executed to automatically filter out low-quality sequence fragments containing more than 5% N bases, remove abnormal sequences with a length less than 100bp or greater than 10000bp, and remove redundant sequences through sequence homology detection (based on the BLAST algorithm, with a similarity threshold set to 95%). During the preprocessing stage, sequence standardization is performed: all qualified sequences undergo uniform orientation calibration (based on promoter region features or homologous sequence alignment, inverted complementary sequences are adjusted to forward sequences), and non-base characters in the sequences are removed to generate pure base sequence strings; for very long sequences (length exceeding 5000bp), a sliding window segmentation strategy is used for segmentation, while recording the original sequence location information of each segment; for short sequences (length less than 500bp), a padded end strategy (padded characters are pseudo-bases composed of random bases) is used to unify the sequence length to 500bp; In the k-mer segmentation scheme, for capturing local features and short-range dependencies of the sequence, an adaptive k-value selection strategy is adopted (providing four optional parameters k=3, 4, 5, and 6, automatically recommending the optimal k value based on the sequence length: k=3 or 4 is recommended for short sequences, and k=5 or 6 is recommended for long sequences); the sequence is segmented according to a sliding window (window size k, step size 1) to generate k-mer fragments (e.g., the sequence "ATCGG" generates "ATC", "TCG", and "CGG" when k=3); then, each k-mer fragment is mapped to a fixed-dimensional dense vector using a pre-constructed k-mer dictionary (based on the frequency statistics of all k-mer fragments in the training dataset, retaining the top 99.9% of the k-mers with the highest frequency, and classifying the rest as "unseen" and mapping them to a unified vector). The final generated vector has a dimension of [missing information - likely a specific dimension]. Numerical matrices; To optimize the quality of the model input, the embedded numerical matrix needs further processing: Z-score standardization (mean 0, variance 1) eliminates the dimensional differences between different feature dimensions, improving the convergence speed of model training; for the dense matrix generated by k-mer segmentation, principal component analysis is used to reduce the dimensionality and decrease the computational cost of the model; finally, the processed numerical matrix will be used as the input to the genome basic model sequence parsing architecture, while preserving the index mapping relationship between the original sequence and the embedding matrix.

[0017] Feature extraction and non-coding region decoding module: Input the numerical matrix into the trained deep learning model. The deep learning model is based on the genome basic model architecture. Through unsupervised pre-training on massive genome data, it internalizes the inherent functional syntax and regulatory logic of DNA sequence and outputs the functional feature vector of non-coding region in the DNA sequence. In the feature extraction and non-coding region decoding module, the deep learning model used is based on an improved Transformer architecture to build a genome basic model. The overall architecture includes a feature input layer, a multi-layer encoder stack layer, a cross-scale feature fusion layer, and a functional decoding output layer. The encoder stack layer consists of 24 bidirectional Transformer encoders. Each encoder layer integrates 32 multi-head self-attention mechanisms and feedforward neural networks. The head dimension of the multi-head self-attention mechanism is set to 64. The feedforward neural network includes two linear transformation layers (hidden layer dimension is 2048, output layer dimension is 1024) and a GELU activation function. At the same time, layer normalization and residual connections are introduced between each encoder layer. The unsupervised pre-training process of the deep learning model is based on a large-scale genomic dataset. Training data includes complete reference genome sequences, non-coding region functional annotation data from the ENCODE database, epigenetic modification maps from Roadmap Epigenomics, and gene expression data from GTEx. A multi-task collaborative training mechanism is designed for the pre-training phase. Tasks include base masking recovery (randomly masking 15% of bases, with 80% replaced by mask identifiers, 10% replaced by random bases, and 10% retaining the original bases; the model needs to predict the original base types), regulatory element boundary prediction, chromatin open state inference, and transcription factor binding site prediction (predicting regions in the sequence that bind to known transcription factors). An adaptive learning rate strategy is used during training, with an initial learning rate of 1e-4, decreasing to 0.9 times the current value every 200,000 iterations. Mixed-precision training and gradient accumulation techniques are combined to improve training efficiency, resulting in a total of 120 training iterations. Tens of thousands of times, the early stopping mechanism ensures the model's generalization ability, and ultimately enables the deep learning model to internalize the inherent functional syntax of DNA sequences, including the conserved patterns of base arrangement, the combination rules of functional motifs (such as the cooperative distribution logic of enhancer and transcription factor binding sites), the sequence features of long-distance chromatin interactions, and the association logic between epigenetic modifications and sequence features, thus constructing a deep mapping relationship from sequence to function. The numerical matrix output from the sequence preprocessing and embedding module is converted into a uniform-dimensional embedding vector by the feature input layer and then input into the multi-layer encoder stack for hierarchical feature extraction: The bottom encoder (layers 1-8) focuses on capturing local features of the sequence, mining the adjacency dependencies of individual bases or short fragment k-mers through a multi-head self-attention mechanism, and extracting basic features such as base composition preferences and local sequence patterns; the middle encoder (layers 9-16) focuses on the identification and integration of functional motif features, capturing conserved functional motifs and their combination patterns in the sequence by expanding the attention window range, and generating motif-level feature vectors; the top encoder (layers 17-24) focuses on the fusion of long-distance dependencies and global functional features, obtaining sequence features related to long-distance chromatin interactions in non-coding regions (such as the sequence complementarity features between distal enhancers and promoters) and regulatory network features between functional elements through attention calculation across sequence positions, while integrating latent features related to epigenetic modifications to form a multi-dimensional, multi-level global feature representation; A cross-scale feature fusion layer is set after the encoder stacking layer. Through an attention-weighted fusion mechanism, features of different scales extracted from the bottom, middle, and top layers are dynamically integrated, and the weights of features at each level are adaptively allocated according to the biological characteristics of the sequence. For example, for sequence fragments containing clear transcription factor binding sites, the weight of motif features in the middle layer is enhanced, and for long-distance regulatory sequences, the contribution of long-distance dependent features in the upper layer is increased. The fused feature vector is input to the functional decoding output layer, which consists of two fully connected layers and one normalization layer. The feature vector is mapped to a non-coding region functional feature vector through the Softmax activation function. The vector dimension is 512, where each dimension corresponds to a specific non-coding region functional attribute, including regulatory activity intensity, epigenetic modification sensitivity, transcription factor binding affinity, chromatin interaction potential, and disease association risk. The value of each dimension of the functional feature vector is normalized to the range of 0-1, with higher values ​​indicating more significant functional attributes. For DNA sequences of different species, the module automatically calls the pre-trained parameters of the corresponding species to fine-tune the feature extraction weights; for special types of non-coding regions, it improves the targeting and accuracy of functional decoding by activating dedicated feature extraction branches (such as the repeating unit feature extraction sub-network of microsatellite sequences); at the same time, the module retains the intermediate results in the feature extraction process (feature maps and attention weight matrices output by each encoder layer).

[0018] Multi-scale attention analysis module: Through the multi-scale attention mechanism integrated within the deep learning model, the functional feature vector is analyzed to generate attention weight distribution maps at different biological scales, including the single nucleotide scale, the functional motif scale, and the long-distance chromatin interaction scale. In the multi-scale attention analysis module, for attention analysis at the single nucleotide scale, the focus is on base-level contribution evaluation. Mathematical modeling eliminates weight differences in multi-layer and multi-head attention, achieving precise quantification of single-base contribution. First, the multi-head self-attention weight matrix of the bottom-level encoder (layers 1-8) is called. Assuming the model contains 8 bottom-level encoders, each layer containing 32 attention heads, for the i-th nucleotide in the input sequence, the raw attention value output by the h-th attention head in layer l is... The original attention value directly reflects the contribution of a single base to the attributes of each dimension in the functional feature vector. To eliminate the weight differences between different attention heads and encoder layers, the original score is first normalized using Min-Max, mapping the score of each base to the 0-1 interval. The normalization formula is as follows:

[0019] in, This represents the normalized raw attention value, where N represents the total number of nucleotides in the input sequence. This represents the minimum value among all N original attention values ​​of the input sequence under the h-th attention head in the l-th layer; This represents the maximum value among all N original attention values ​​of the input sequence under the h-th attention head in the l-th layer; Subsequently, the normalized scores of each layer and each attention head are weighted and summed according to the feature importance weights of each encoder layer to obtain the comprehensive attention weight of the i-th nucleotide. The specific calculation method is as follows:

[0020] in, Let the total attention weight be represented as the sum of the attention weights for the i-th nucleotide. Let H represent the weights of the l-th layer, and let H represent the number of attention heads. Based on this, a single nucleotide-scale attention weight distribution map is generated, with sequence position as the x-axis and comprehensive attention weights. The vertical axis is used as the plotting axis, presented in the form of a line graph or heatmap; where, The nucleotide markers are designated as "high contribution sites". The nucleotides are labeled as "low contribution sites"; this visually demonstrates the differences in the impact of a single base on the function of the non-coding region, providing a direct basis for the pathogenicity analysis of single nucleotide variants. In attention analysis at the functional motif scale, precise identification and importance assessment of functional motifs are achieved by quantifying clustering degree and similarity: First, based on the attention weight matrix of the mid-level encoder (layers 9-16), continuous segments with high attention clustering in the sequence are identified, and segment attention clustering degree is defined. , This represents the average attention weight of all nucleotides within fragment s; where fragment s consists of k consecutive nucleotides. The corresponding typical functional motif length; the specific method for calculating segment attention concentration is as follows:

[0021] in, This is represented as the segment attention concentration. when At that time, the fragment was determined to be a high-attention-focused fragment; subsequently, sequence alignment was performed using known functional motifs in the database, and the Smith-Waterman algorithm was used to calculate the similarity Q between the fragment and the known motifs. The similarity formula is calculated based on base matching score, mismatch penalty, and gap penalty. The specific calculation method is as follows:

[0022] Where Q represents the similarity score, M represents the number of matching bases, X represents the number of mismatched bases, G represents the number of open gaps, and E represents the gap extension length. Represented as the length of a known motif; A single base match scores 2 points, a mismatch penalty scores 1 point, a gap opening penalty scores 2 points, and a gap extension penalty scores 0.5 points. when At that time, it is determined that the fragment is homologous to the corresponding known motif, and its functional motif type is clarified; then, the overall attention weight of each functional motif is calculated. Combining the regulatory activity score corresponding to the motif in the functional feature vector A sequence-level importance score is constructed, and the specific calculation method is as follows:

[0023] in, Represented as a sequence-level importance score; The generated functional motif scale attention weight distribution map adopts a modular display method. Different colors are used to mark various functional motifs in the sequence visualization map. The color intensity of the motif region corresponds to the importance score. The motif name, length and corresponding functional attributes are also labeled. For attention analysis at the long-range chromatin interaction scale, the focus is on resolving cross-regional regulatory relationships. Accurate identification of core long-range regulatory sites is achieved by quantifying attention association coefficients: based on the long-range attention weight matrix of the upper-level encoder, sites with high attention associations to other genomic regions are extracted from the sequence, defining target site b and distant regions. Attention correlation coefficient The Pearson correlation coefficient was used for calculation, and the specific calculation method is as follows:

[0024] in, Represented as target site b and distal region The attention correlation coefficient is given by r, where r represents the site index in the distal region, and K represents the total number of sites in the distal region R. This is represented as the overall attention weight at site b. This is expressed as the average attention weight at site b. Let the summation of attention weights be the summation weights of the r-th site in the distant region R. This is expressed as the average attention weight in the far region R; Combining the chromatin interaction map from the Hi-C data, when Furthermore, Hi-C data showed significant physical interactions in this region, indicating a significant regulatory association between target site b and the distal region R. The generated long-range chromatin interaction-scale attention weight distribution map adopts a combination of network and linear maps. The linear map marks the core associated sites of the target sequence and their corresponding comprehensive attention weights. In the network graph, nodes represent the target site and its associated distant regions, with node size being equal to or smaller than the target site's size. Positive correlation, edge thickness and correlation coefficient Positive correlation reveals a regulatory network in non-coding regions through long-distance chromatin loops; Supports simultaneous display and switching of distribution maps at three scales. When a user clicks on a high-weight region at a certain scale, the system can automatically locate and highlight the corresponding position in other scale distribution maps (within the motif at the single nucleotide scale). High contribution sites and long-range scales related to this motif (The remote area); at the same time, it provides a weight details query function, and users can view the specific value of any site or motif by hovering the mouse; it also provides information such as functional contribution dimension, database comparison results, etc., and supports exporting the distribution map to SVG, PNG or PDF format, while retaining the original file of the weight data.

[0025] The perturbation and causal inference module performs virtual base editing on candidate sites in the DNA sequence, inputs the perturbated sequence into the deep learning model, and calculates the causal effect score of the candidate sites by comparing the differences in the model output before and after perturbation. The computational perturbation and causal inference module integrates a candidate site screening engine, a virtual base editing toolset, a functional difference analysis unit, and a causal effect scoring unit, enabling fully automated processing from site selection to causal determination. It also supports user-defined editing parameters, enhancing the flexibility and relevance of the analysis. The candidate site screening process relies heavily on the output of the multi-scale attention analysis module, combined with sequence analysis results to accurately identify high-priority targets: firstly, the comprehensive attention weights at the single nucleotide scale are extracted. High-contribution sites, core motif regions with importance scores greater than 0.7 at the functional motif scale, and correlation coefficients at the long-distance chromatin interaction scale. The regulatory-related sites were identified to construct an initial candidate site set. Subsequently, redundant sites were removed (adjacent sites with a distance of less than 5 bp were eliminated) and functional association screening was performed (sites with a correlation ≥ 0.7 with the core functional attributes of non-coding regions—such as regulatory activity and transcription factor binding affinity) to form a final, simplified candidate site list. The list includes the sequence position of each site, its biological scale, attention-related parameters, and functional annotation information. The virtual base editing toolset supports multiple editing modes to simulate in vivo base editing experiments: users can choose from four editing modes: single-base mutation, multi-base insertion, multi-base deletion, and motif replacement; during the editing process, the location information and format of the original sequence are automatically preserved, and the edited sequence undergoes the same preprocessing and embedding process as the original sequence to ensure that the input format of the perturbation sequence and the original sequence are consistent, avoiding functional prediction deviations caused by differences in data processing; for long-distance chromatin interaction-related sites, the corresponding sites of the target site and the distant associated region R are edited simultaneously to simulate the cooperative perturbation effect of the regulatory network; The functional difference analysis unit is responsible for quantifying the magnitude of change in model output before and after perturbation, providing a data foundation for causal effect assessment: The virtually edited perturbation sequence is input into a trained deep learning model (using the same model parameters as the original sequence analysis), and the output non-coding region functional feature vector is compared dimension-by-dimensionally with the original sequence's functional feature vector; the functional difference vector is defined, and the specific method for calculating the difference value in each dimension is as follows:

[0026] in, Represented as dimensional difference values, This represents the score in the m-th functional dimension after perturbation. This is represented as the score of the original sequence in the m-th functional dimension; In addition, for functional motif-related sites, the motif function retention rate is calculated, which is the ratio of the importance score of the perturbated motif to the original score. The specific calculation method is as follows:

[0027] in, This is represented by the importance score of the motif after perturbation. This is represented as the raw score; For long-distance regulatory sites, calculate the retention rate of regulatory network associations. , that is, the ratio of the perturbed correlation coefficient to the original correlation coefficient, is calculated as follows:

[0028] in, Represented as the correlation coefficient after perturbation, Represented as the original correlation coefficient; The causal effect scoring unit generates a quantified causal effect score for candidate loci by fusing multi-dimensional parameters. To achieve accurate determination of causal relationships: The scoring model comprehensively considers three core indicators: weighted functional difference value D, core functional dimension difference ratio E, and sequence / regulatory network retention rate P. Among them, the core functional dimension difference ratio is defined as the percentage of differences in the core dimensions. The proportion of the number of dimensions to the total number of core dimensions; the specific method for calculating the causal effect score is as follows:

[0029] in, Represented as the Sigmoid function, The value ranges from 0 to 1, with higher values ​​indicating a stronger causal regulatory effect of the candidate site on sequence function; the module has three preset scoring levels: A strong causal site, 0.4 For intermediate causal sites, The results are for weak causal sites; the final causal effect analysis report includes the editing mode for each point, details of functional differences, causal effect score and level determination, and outputs a comparison chart of functional feature vectors before and after the perturbation and a heat map of core dimension differences, which intuitively shows the specific manifestation of the causal effect.

[0030] Biomarker output module: Based on the causal effect score and attention weight distribution, pathogenic mutation sites or functional variant regions are screened out, and disease-related biomarker prediction reports are generated. The biomarker output module constructs a complete output system of multi-dimensional indicator fusion, hierarchical screening, and report visualization. The module integrates an indicator fusion engine, hierarchical screening unit, report generation toolset, and result visualization unit to achieve fully automated processing from data integration to report output. It also supports user-defined screening thresholds and report display formats to meet the personalized needs of different application scenarios. In the indicator fusion stage, two core data sources are first integrated: one is the causal effect score and grading results output by the perturbation and causal inference module, and the other is the attention weight data at various biological scales generated by the multi-scale attention analysis module. On this basis, external authoritative database resources are further linked to supplement additional biological annotation information of candidate sites, including matching with known disease variant databases, whether they belong to reported regulatory element regions, related transcription factor binding information, epigenetic modification data, and inter-species conservation analysis results, to form a biomarker evaluation dataset. The stratified screening unit employs a three-tiered screening strategy—initial screening, refined screening, and final review—to ensure the high reliability and practicality of the output biomarkers. In the initial screening stage, causal effect scoring is the core criterion, prioritizing the retention of "strong causal sites" while incorporating some "moderate causal sites" based on user needs, and directly eliminating "weak causal sites," thus initially narrowing down the candidate biomarker pool. In the refined screening stage, multi-scale attention weight data is used for secondary validation. Candidate sites retained in the initial screening are required to meet preset standards in attention metrics at the corresponding biological scale; that is, single nucleotide core sites must meet the comprehensive attention weight, and functional motif-related sites must meet the relevant standards. To meet the importance score, long-distance regulatory sites must meet the correlation coefficient and have a significant perturbation response in the functional dimension. The reliability of biomarkers is improved through cross-validation of attention indicators and causal effects. In the final review stage, biological rationality verification is introduced. Combined with external database annotation information, candidate sites with biological contradictions are eliminated (such as sites that have been confirmed as benign variations by authoritative databases, sites located in non-functional regions and without conservation). At the same time, the screening results are classified and integrated according to the site type, and divided into three categories: "single nucleotide pathogenic mutations", "functional motif variation regions" and "long-distance regulatory variation sites". Within each category, they are sorted from high to low according to the causal effect score to form the final list of disease-related biomarker candidates. The report generation toolset is responsible for integrating the selected biomarker information into a predictive report. The report is divided into three main sections: core results, detailed annotations, and analysis descriptions. The core results section lists the biomarker candidate list, including basic information for each region (sequence location, chromosome location, variant type), core evaluation indicators (causal effect score and level, attention weight key parameters), screening level (priority is divided into first-level, second-level, and third-level from high to low), and priority suggestions for subsequent validation. The detailed annotation section displays multi-dimensional biological information for each biomarker, including database matching details (whether it is a known disease-related variant, matching database name and variant ID), functional element attribution (whether it belongs to regulatory regions such as enhancers and promoters), transcription factor binding association (the names of transcription factors that may be affected and changes in binding strength), epigenetic modification characteristics (whether there are epigenetic modification sites and the trend of changes after perturbation), interspecies conservation analysis results (cross-species sequence similarity and conserved region annotation), and speculation on potential pathogenic mechanisms. The analysis and explanation section outlines the technical details of the entire screening process, the basis for setting the screening thresholds at each stage, the limitations of the system, and suggestions for experimental verification. The results visualization unit presents key information about biomarkers in a visually appealing way through charts and graphs, enhancing the readability and usability of the report: For single nucleotide pathogenic mutations, it generates a sequence location distribution map, clearly marking the specific location of the mutation site in the DNA sequence and its relative relationship with functional elements; for functional motif variant regions, it provides a motif sequence alignment map, showing the changes in the motif sequence before and after the mutation and the homology comparison with known functional motifs; for long-distance regulatory variant sites, it draws a regulatory network association map, representing the regulatory relationship between the site and distant associated regions and the network changes after perturbation; at the same time, it generates a heatmap summarizing the core indicators of biomarkers, comparing key indicators such as causal effect scores and attention weights of different biomarkers.

[0031] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning system for identifying disease-related biomarkers from DNA sequences, characterized in that: Sequence preprocessing and embedding module: Obtain the DNA sequence to be tested, perform k-mer segmentation on the DNA sequence, and convert it into a numerical matrix as model input; Feature extraction and non-coding region decoding module: Input the numerical matrix into the trained deep learning model. The deep learning model is based on the genome basic model architecture. Through unsupervised pre-training on massive genome data, it internalizes the inherent functional syntax and regulatory logic of DNA sequence and outputs the functional feature vector of non-coding region in the DNA sequence. Multi-scale attention analysis module: Through the multi-scale attention mechanism integrated within the deep learning model, the functional feature vector is analyzed to generate attention weight distribution maps at different biological scales, including the single nucleotide scale, the functional motif scale, and the long-distance chromatin interaction scale. The perturbation and causal inference module performs virtual base editing on candidate sites in the DNA sequence, inputs the perturbated sequence into the deep learning model, and calculates the causal effect score of the candidate sites by comparing the differences in the model output before and after perturbation. Biomarker output module: Based on the causal effect score and attention weight distribution, pathogenic mutation sites or functional variation regions are screened out, and disease-related biomarker prediction reports are generated. 2.The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: In the sequence preprocessing and embedding module, the raw DNA sequence data to be tested is obtained in the FASTA standard bioinformatics file format. For the input sequence data, a basic quality control process is first executed to automatically filter out low-quality sequence fragments containing more than 5% N bases, remove abnormal sequences, and remove redundant sequences through sequence homology detection. In the preprocessing stage, sequence standardization is performed: all qualified sequences are uniformly oriented and non-base characters in the sequence are removed to generate pure base sequence strings; for very long sequences, a sliding window segmentation strategy is used for segmentation, and the original sequence location information of each segment is recorded; for short sequences, a padded strategy is used to unify the sequence length to 500bp. 3.The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 2, wherein: In the k-mer tokenization processing scheme, for capturing local features and short-range dependencies of the sequence, an adaptive k-value selection strategy is adopted; the sequence is tokenized according to a sliding window to generate k-mer fragments; and then each k-mer fragment is mapped to a fixed-dimension dense vector through a pre-constructed k-mer dictionary, to finally generate a numerical matrix with a dimension of . 4.The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: In the feature extraction and non-coding region decoding module, the deep learning model used is based on an improved Transformer architecture to build a genome basic model. The overall architecture includes a feature input layer, a multi-layer encoder stack layer, a cross-scale feature fusion layer, and a functional decoding output layer. The encoder stack layer consists of 24 bidirectional Transformer encoders. Each encoder layer integrates 32 multi-head self-attention mechanisms and feedforward neural networks. The head dimension of the multi-head self-attention mechanism is set to 64. The feedforward neural network includes two linear transformation layers and a GELU activation function. At the same time, layer normalization and residual connections are introduced between each encoder layer. The unsupervised pre-training process of deep learning models is based on large-scale genomic datasets. The training data includes complete reference genome sequences, non-coding region functional annotation data from the ENCODE database, epigenetic modification map data from Roadmap Epigenomics, and gene expression data from GTEx. 5.The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: The numerical matrix output from the sequence preprocessing and embedding module is converted into a uniform-dimensional embedding vector by the feature input layer and then input into the multi-layer encoder stack for hierarchical feature extraction: the bottom encoder focuses on capturing local features of the sequence, mining the adjacent dependencies of single bases or short fragment k-mers through a multi-head self-attention mechanism, and extracting basic features of base composition preferences and local sequence patterns; the middle encoder focuses on the identification and integration of functional motif features, capturing functional motifs and their combination patterns in the sequence by expanding the attention window range, and generating motif-level feature vectors; the top encoder focuses on the fusion of long-distance dependencies and global functional features, obtaining sequence features related to long-distance chromatin interactions in non-coding regions and regulatory network features between functional elements through attention calculation across sequence positions, while integrating latent features related to epigenetic modifications to form a global feature representation; A cross-scale feature fusion layer is set after the encoder stacking layer. Through the attention weighted fusion mechanism, the features of different scales extracted from the bottom, middle and top layers are dynamically integrated, and the weights of each level of features are adaptively allocated according to the biological characteristics of the sequence. 6.The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: At the single nucleotide scale, the module uses a three-layer quantization method to eliminate weight differences between multiple heads in multiple layers through the attention weight matrix of the bottom encoder (layers 1-8): first, the original score of each head is normalized by Min-Max; second, the scores of 32 heads in the same layer are weighted and averaged according to the importance of the encoder layer; and finally, the weighted fusion across 8 layers generates the comprehensive attention weight for each base. Nucleotides with a weight greater than 0.8 were marked as "high contribution sites"; At the functional motif scale, high attention-focused continuous segments are identified based on the weights of the mid-level encoder; the motif type and functional attributes are confirmed by calculating the average weight within the segment and performing a Smith-Waterman comparison with a known motif database; finally, a motif-level importance score combining attention weights and regulatory activity scores is constructed, and various functional motifs and their relative importance are displayed in a color-coded modular map. At the long-distance interaction scale, cross-regional association patterns are extracted using the upper-level encoder; by calculating the Pearson correlation coefficient between the target site and the distant region and verifying it with Hi-C experimental data, chromatin loop structures with significant regulatory associations are identified; the visualization adopts a dual-view mode of network map and linear map, with node size positively correlated with attention weight and edge thickness positively correlated with association coefficient, clearly showing the regulatory logic at the three-dimensional genome level.

7. The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 6, wherein: The similarity formula is calculated based on base matching score, mismatch penalty, and gap penalty. The specific calculation method is as follows: wherein Q represents a similarity, M represents a number of matched bases, X represents a number of mismatched bases, G represents a number of gap openings, and E represents a length of gap extension, represents a length of the known motif; The specific method for calculating the Pearson correlation coefficient is as follows: wherein, represents the attention correlation coefficient of the target site b with the distal region r represents the site index in the distal region, and K represents the total number of sites in the distal region R, represents the integrated attention weight of the site b, represents the average attention weight of the site b, represents the integrated attention weight of the rth site in the distal region R, represents the average attention weight of the distal region R.

8. The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: The computational perturbation and causal inference module integrates a candidate site screening engine, a virtual base editing toolset, a functional difference analysis unit, and a causal effect scoring unit to achieve fully automated processing from site selection to causal determination. The candidate site screening process relies on the output of the multi-scale attention analysis module, combined with sequence analysis results, to identify high-priority targets: firstly, the comprehensive attention weights at the single nucleotide scale are extracted. High-contribution sites, core motif regions with importance scores greater than 0.7 at the functional motif scale, and correlation coefficients at the long-distance chromatin interaction scale. Regulatory-associated sites were identified, and an initial candidate site set was constructed. Subsequently, through redundancy removal and functional correlation screening, a simplified candidate site list was finally formed, which includes the sequence position of each site, its biological scale, attention-related parameters, and functional annotation information. 9.The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: The functional difference analysis unit is responsible for quantifying the magnitude of change in the model output before and after perturbation, providing a data basis for causal effect assessment: the perturbation sequence after virtual editing is input into the trained deep learning model, and the functional feature vector of the non-coding region after perturbation is compared with the functional feature vector of the original sequence in a dimension-by-dimensional comparison. The causal effect scoring unit generates a quantified causal effect score for candidate loci by fusing multi-dimensional parameters. To accurately determine causal relationships: the scoring model comprehensively considers three core indicators: weighted functional difference value D, core functional dimension difference ratio E, and sequence / regulatory network retention rate P. The core functional dimension difference ratio is defined as the percentage of differences in the core dimensions within each dimension. The proportion of the number of dimensions to the total number of core dimensions; the specific method for calculating the causal effect score is as follows: wherein, is represented as a Sigmoid function, The value range of is 0-1, the higher the value, the stronger the causal regulation effect of the candidate site on the sequence function; three preset scoring levels are: strong causal site, 0.4 moderate causal site, weak causal site; a causal effect analysis report is finally generated, including the editing mode of each site, the functional difference details, the causal effect score and the level determination, and the function characteristic vector comparison graph before and after perturbation and the core dimension difference heat map are output.

10. The deep learning system for identifying disease-associated biomarkers from DNA sequences of claim 1, wherein: In the biomarker output module, the indicator fusion engine integrates two sets of data: the scoring and ranking results of the perturbation and causal inference module, and multi-scale attention weight data. At the same time, it associates with external authoritative databases to supplement disease associations, regulatory elements, transcription factors and biological annotations to form a comprehensive evaluation dataset. The stratified screening unit adopts a three-level strategy of initial screening, fine screening and final review: the initial screening focuses on causal effect scoring and prioritizes the retention of strong causal sites; The screening process requires loci to meet attention metrics at the corresponding biological scale, and performs dual verification of causal effects and attention mechanisms. The final review combines external databases to verify biological rationality, eliminates known benign variants and contradictory loci, and finally classifies candidate biomarkers into three categories and ranks them: "single nucleotide pathogenic mutations," "functional motif variant regions," and "long-distance regulatory variant sites." The report generation toolset outputs structured prediction reports, including a list of core results, detailed biological annotations, and technical descriptions. The visualization unit provides targeted charts: sequence position distribution maps show single nucleotide mutation sites, motif alignment maps present functional motif variants, regulatory network maps analyze long-distance interactions, and a heatmap summarizing core metrics is generated for multi-marker comparison.