A 5hmC prediction method and system based on multi-branch deep learning and adaptive feature screening

Abstract

The present invention discloses a 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, including: S1. constructing positive and negative sample sets; S2. respectively performing sequence-based feature encoding, physicochemical property feature encoding, and statistical feature encoding on the positive and negative samples to generate a multi-dimensional feature set; S3. respectively inputting features of different encoding types into corresponding neural network branches for processing; S4. screening several feature subsets that contribute the most to 5hmC prediction; S5. training a prediction model including multiple neural network branches, a self-attention mechanism, and a classifier for predicting 5hmC modification sites. This solution comprehensively depicts information at multiple levels such as local patterns, global patterns, physicochemical properties, and base encoding of DNA sequences by combining different types of features such as sequence-based feature encoding, physicochemical property feature encoding, and statistical feature encoding. Through multi-feature fusion, it avoids the single-sided description of sequence information in existing methods, thereby greatly improving the model's ability to identify 5hmC modification sites.

Description

Technical Field

[0001] The present invention belongs to the field of computer bioinformatics, and particularly relates to a 5hmC prediction method and system based on multi-branch deep learning and adaptive feature screening. Background Art

[0002] 5-Hydroxymethylcytosine (5hmC) is an important form of DNA methylation and plays a key role in gene expression regulation, cell differentiation, and diseases such as cancer and neurodegenerative diseases. The change in the modification level of 5hmC is closely related to various pathological states, so its research occupies an important position in the field of epigenetics. Accurately identifying and analyzing the distribution and regulatory mechanism of 5hmC can not only provide a new perspective on the genomic regulatory network but also provide a theoretical basis for disease diagnosis and treatment. However, due to the weak signal of 5hmC and its similarity to other DNA modifications (such as 5mC), the detection of 5hmC still faces great challenges.

[0003] The current technologies for 5hmC detection are divided into two categories: experimental methods and computational methods. Experimental methods mainly include high-throughput sequencing technologies (such as TAB-Seq, oxBS-Seq) and biomarker methods. High-throughput sequencing efficiently distinguishes 5hmC from other modifications through the combination of chemical modification and sequencing, but has problems such as high cost, cumbersome steps, and high requirements for sample quality, which limit large-scale applications. The biomarker method relies on enzymatic reactions and antibody labeling technologies and also faces challenges such as high cost and low throughput, making it difficult to meet the needs of large-scale data analysis.

[0004] To overcome the limitations of experimental methods, computational methods have gradually become a research hotspot, especially driven by machine learning and deep learning. These methods do not rely on expensive experimental equipment and can show obvious advantages in large-scale screening. Traditional machine learning methods, such as support vector machines (SVM) and XGBoost, combine the features of DNA sequences for prediction, with relatively high computational efficiency and can meet certain research needs. However, these methods are difficult to capture complex patterns in sequences and have limited ability to process high-dimensional features, resulting in limited improvement in model performance.

[0005] Deep learning methods, through technologies such as convolutional neural networks (CNN) and recurrent neural networks (RNN), have significantly improved the ability to extract sequence features, especially outstanding in capturing local patterns and long-range dependencies. However, these methods are prone to overfitting on small sample datasets and have weak ability to fuse multi-source features, still facing certain challenges.

[0006] Despite the advantages and disadvantages of experimental and computational methods, there are still some key problems in current 5hmC detection methods: experimental methods are unable to achieve large-scale detection due to high costs and low throughput; traditional computational methods are insufficient in sequence complexity modeling and difficult to adapt to the increasing data requirements; while deep learning methods have problems of overfitting and insufficient feature fusion. Therefore, developing an efficient and robust 5hmC prediction method that can fully integrate multi-source features and improve the generalization ability of the model is a technical challenge that urgently needs to be solved currently. Summary of the Invention

[0007] The purpose of the present invention is to propose a 5hmC prediction method and system based on multi-branch deep learning and adaptive feature screening for the problems of the existing technology.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions:

[0009] A 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, the method comprising:

[0010] S1. Screening common 5hmC modification sites from various sequencing data to construct positive and negative sample sets;

[0011] S2. Respectively performing sequence-type feature encoding, physicochemical property feature encoding, and statistical-type feature encoding on the positive and negative samples to generate a multi-dimensional feature set;

[0012] S3. Inputting features of different encoding types into corresponding neural network branches for processing;

[0013] S4. Dynamically weighting and fusing the features output by each neural network branch through the self-attention mechanism, and combining dynamic SHAP value calculation and multi-stage LASSO regression to screen several feature subsets that contribute the most to 5hmC prediction;

[0014] S5. Adopting an adaptive regularization and adaptive resampling strategy to train a prediction model including multiple neural network branches, a self-attention mechanism, and a classifier through a class imbalance loss function for predicting 5hmC modification sites.

[0015] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, the sequence-type feature encoding includes any one or more of One-Hot encoding, Binary encoding, Kmer encoding, PSTNPsS encoding, Mismatch encoding, and ENAC encoding;

[0016] The physicochemical property feature encoding includes any one or more of PCP encoding and SCPseKNC encoding;

[0017] The above-mentioned statistical feature encodings include any one or more of DAC encoding, ANF encoding, and ASDC encoding;

[0018] The above-mentioned neural network branches include CNN branches, LSTM branches, and DNN branches.

[0019] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, in step S2, Kmer frequency encoding, PSTNPsS encoding, One-Hot encoding, Binary encoding, SCPseKNC encoding, and DAC encoding are respectively performed on the positive and negative samples to generate a multi-dimensional feature set;

[0020] The above-mentioned neural network branches include that two CNN branches respectively process Kmer frequency encoding features and PSTNPsS encoding features, two LSTM branches respectively process SCPseKNC encoding features and Binary encoding features, and two DNN branches respectively process One-Hot encoding features and DAC encoding features.

[0021] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, the CNN branch includes multiple layers of convolutional kernels for extracting local sequence-type features of Kmer frequency encoding features and PSTNPsS encoding;

[0022] The LSTM branch adopts a bidirectional structure for processing the physicochemical property time series features encoded by SCPseKNC and the binary sequence dependence of Binary encoding;

[0023] The DNN branch includes a fully connected layer and a non-linear activation function for extracting the global non-linear associations of One-Hot encoding features and DAC encoding.

[0024] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, in step S3, each neural network branch outputs a feature vector with a fixed and same dimension, and the features output by each neural network branch are dynamically weighted and fused by the self-attention mechanism to capture complementary information between different features.

[0025] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, step S4 specifically includes:

[0026] Stack the outputs of each neural network branch into a tensor with a specific shape;

[0027] Calculate the importance of different features through the multi-head attention mechanism and perform weight normalization;

[0028] After calculating the weighted features, residual connections and layer normalization are introduced to keep the model stable during deep training;

[0029] Average pooling is performed on the fused features to form a candidate feature vector of fixed length;

[0030] Through SHAP value calculation and multi-stage LASSO regression, the final feature vector is selected from the candidate feature vectors as the input of the classifier.

[0031] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, in step S4, the SHAP value calculation adopts a dynamic update mechanism. After each training round, SHAP is recalculated. Through multiple rounds of calculation, the change of SHAP value with training is monitored, and the time decay weight is used to smooth the SHAP change. Finally, several features with the highest SHAP values are selected, such as the top 50%.

[0032] After several features are selected through SHAP value calculation and enter the multi-stage LASSO regression screening stage, finally, a part of the features with the greatest impact on 5hmC prediction are retained from the selected several features, such as the top 50%.

[0033] In the above 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, in step S4, the process of screening features by SHAP value also includes multi-model cross-comparison:

[0034] XGBoost, LightGBM, and random forest training models are used, and the SHAP values are calculated respectively;

[0035] Select a part of the features that rank in the top set percentage, such as 50%, in multiple models and whose importance consistency coefficient is higher than the set value.

[0036] A 5hmC prediction system based on multi-branch deep learning and adaptive feature screening includes:

[0037] A data preprocessing module, which is used to screen the common 5hmC modification sites from multiple sequencing data and construct positive and negative sample sets according to the relevant feature parameters of the modification sites;

[0038] A multi-source feature encoding module, which is used to perform sequence-type feature encoding, physicochemical property feature encoding, and statistical-type feature encoding on the positive and negative sample sets to generate a multi-dimensional feature set;

[0039] A multi-branch deep learning module, including multiple neural network branches and a self-attention fusion unit; the multiple neural network branches are used to process the features of the corresponding encoding types respectively, and the self-attention fusion unit is used to dynamically weight and fuse the features output by each neural network branch;

[0040] The dynamic feature screening module, including SHAP calculation and LASSO regression, is used to combine dynamic SHAP value calculation and multi-stage LASSO regression to screen several feature subsets with the greatest contribution to 5hmC prediction from the features output by the self-attention fusion unit, such as screening the top fifty percent or top thirty percent of the feature subsets with the greatest contribution.

[0041] The training module uses an adaptive regularization and adaptive resampling strategy to train a prediction model including multiple neural network branches, a self-attention mechanism, and a classifier through a class imbalance loss function for predicting 5hmC modification sites.

[0042] A 5hmC prediction system based on multi-branch deep learning and adaptive feature screening, comprising:

[0043] The interaction module is used to receive the DNA context sequence to be predicted and output the prediction result;

[0044] The multi-source feature encoding module is used to perform sequence-type feature encoding, physicochemical property feature encoding, and statistical-type feature encoding on the DNA context sequence to be predicted to generate a multi-dimensional feature set;

[0045] The multi-branch deep learning module includes multiple neural network branches and a self-attention fusion unit; the multiple neural network branches are used to process the features of the corresponding encoding types respectively, and the self-attention fusion unit dynamically weights and fuses the features output by each neural network branch;

[0046] The feature screening module pre-stores the feature types screened through dynamic SHAP value calculation and multi-stage LASSO regression during the training process, and is used to extract the pre-stored feature types from the features output by the neural network branches as feature subsets for output;

[0047] The classification module is used to output the prediction result based on the feature subset output by the feature screening module.

[0048] The advantages of the present invention are as follows:

[0049] 1. This solution comprehensively characterizes the information at multiple levels such as the local pattern, global pattern, physicochemical properties, and base encoding of the DNA sequence by combining different types of features such as Kmer, One-Hot, PSTNPsS, and PCP. Through multi-feature fusion, it avoids the single-sided description of sequence information by existing methods, thereby greatly improving the model's ability to identify 5hmC modification sites;

[0050] 2. This solution uses different deep learning network branches (CNN, LSTM, DNN) for modeling different characteristic forms (such as sequential features, physicochemical property features, numerical features, etc.), giving full play to the advantages of various features, enhancing the model's ability to capture different features. In addition, a self-attention fusion module is introduced, which can dynamically weight the outputs of each branch, improve the efficiency of feature fusion, and further enhance the model performance;

[0051] 3. In the feature screening stage of this solution, dynamic SHAP calculation, time decay, multi-model cross-comparison, and multi-stage LASSO screening strategies are adopted to ensure the stability and interpretability of feature screening, avoid the overfitting problem caused by redundant high-dimensional features, and thus improve the generalization ability of the model;

[0052] 4. This solution has a more flexible and intelligent feature screening method compared with existing site prediction methods, and adopts a branch strategy for different forms of features, having higher efficiency and effect in dealing with complex features compared with the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is the method flow chart of the 5hmC prediction method based on multi-branch deep learning and adaptive feature screening of the present invention;

[0054] Figure 2 is the model architecture diagram of the 5hmC prediction method based on multi-branch deep learning and adaptive feature screening of the present invention;

[0055] Figure 3 is the accuracy test result diagram of 12 encoding methods based on the traditional machine learning model XGBoost in the 5hmC recognition task;

[0056] Figure 4 is the model performance comparison diagram of the embodiments of the present invention;

[0057] Figure 5 is the performance of the accuracy performance index under different encoded feature data;

[0058] Figure 6 is the performance of the area under the ROC curve performance index under different encoded feature data;

[0059] Figure 7 is the performance of the precision performance index under different encoded feature data;

[0060] Figure 8 is the performance of the recall performance index under different encoded feature data;

[0061] Figure 9 is the performance of the Matthews correlation coefficient performance index under different encoded feature data;

[0062] Figure 10 is the performance of the F1-score performance metric under different encoded feature data;

[0063] Figure 11 is the system composition in the model construction stage of the 5hmC prediction method based on multi-branch deep learning and adaptive feature screening of the present invention;

[0064] Figure 12 is the system composition in the model application / test stage of the 5hmC prediction method based on multi-branch deep learning and adaptive feature screening of the present invention.

[0065] Reference numerals: interaction module 1; multi-source feature encoding module 2; multi-branch deep learning module 3; feature screening module 4; classifier 5; data preprocessing module 6; dynamic feature screening module 7; training module 8. Detailed implementation manners

[0066] This embodiment provides a 5hmC prediction method based on multi-branch deep learning and adaptive feature screening. As Figure 1 shown, the overall process of the prediction method is divided into the following main steps: data acquisition and cleaning, feature extraction and fusion, multi-branch deep learning modeling, feature fusion, feature screening and optimization, and final prediction, specifically as follows:

[0067] S1. Data acquisition and cleaning

[0068] First, four types of 5hmC-related sequencing data are acquired, including WGBS, TAPS, TAPSbeta, and CAPS. These data respectively detect the positions of 5hmC through different modification methods. Specifically:

[0069] WGBS: Unmodified cytosine is converted to T, and 5mC and 5hmC remain as C.

[0070] TAPS: Unmodified cytosine remains as C, and 5mC and 5hmC are converted to T.

[0071] TAPSbeta: Only 5mC (converted to T) is detected.

[0072] CAPS: Only 5hmC (converted to T) is detected.

[0073] The fields included in the data file are information such as chromosome, start position, end position, modification level, unmodified read count, modified read count, etc. By converting these fields into numerical data and cleaning, the integrity of the data is ensured, especially handling missing values of the modification level and reads count.

[0074] Furthermore, the CpG positions, modification levels, and related metrics (such as unmodified and modified read counts) of 5hmC modification sites were extracted from the four sequencing files, and all data were ensured to be formatted with complete fields and filled missing values. Next, the 5hmC sites common to the four sequencing methods were screened out through intersection operations to ensure data consistency.

[0075] Based on this, according to the distribution of modification levels, negative and positive samples were selected:

[0076] Negative sample selection: The modification inflection point was determined through histogram analysis, and sites with lower modification levels were selected as negative samples. Specifically, we selected samples with lower modification according to the rate of change of modification levels. At the same time, special attention was also paid to the sites of "close to no modification but with signal" to ensure the diversity of negative samples.

[0077] Positive sample selection: By selecting sites with higher modification levels, it was ensured that the selected positive samples had higher modification levels and could accurately reflect the actual distribution of 5hmC.

[0078] Since 5hmC sites are relatively rare, the downsampling technique (SMOTE) was used to balance the sample ratio and solve the problem of data imbalance.

[0079] Finally, in step S1, the context sequence of each CpG site was extracted from the reference genome. Centered on each site, a 41bp window was intercepted (including 20bp prefix, the current site, and 20bp suffix). Sequences containing "N" were filtered, and the processed context sequences were saved in FASTA format and labeled with sample tags ("pos" for positive samples and "neg" for negative samples) for subsequent feature extraction.

[0080] S2. Feature Extraction and Fusion

[0081] As Figure 2 shown, in the feature extraction stage, multiple encoding methods were used to extract different types of features from the context sequences to ensure that the local patterns, global features, physicochemical properties, and statistical information of the DNA sequences could be comprehensively captured. The encoding methods used include:

[0082] Sequence-based encoding features: such as One-Hot, Binary, Kmer, PSTNPsS, etc., which aim to describe the local or global patterns of the base sequences or consider the regularity of position distribution.

[0083] Physicochemical property features: such as PCP, SCPseKNC, etc., which mainly reflect the physicochemical properties of DNA such as melting temperature, polarizability, hydration, and energy stacking.

[0084] Statistical / correlation features: such as DAC, ANF, ASDC, etc., are mainly used to characterize the correlation relationships and statistical distributions between dinucleotides or higher-order nucleotides.

[0085] In this embodiment, 12 different feature encoding methods are used. The following is a brief description of several of these feature encoding methods:

[0086] Kmer frequency encoding: 5hmC modification is related to specific K-mer combinations. Specific short sequence fragment patterns may determine the occurrence probability of 5hmC modification. The frequency of nucleotide substrings with a fixed length of k (such as k = 3) is statistically calculated to provide statistical pattern information and generate local pattern features of the sequence. The calculation formula for the feature value is:

[0087]

[0088] Mismatch encoding: Calculate the occurrence of k-length adjacent nucleic acids with at most m mismatches (m < k). Based on the Kmer frequency encoding, a certain matching tolerance is introduced to capture sequence types with minor mutations.

[0089]

[0090] L is the total length of the input sequence; m is the maximum number of allowed mismatches, satisfying m < k; represents the reference k-mer When it appears, represents the number of times that the k-mer with a Hamming distance of exactly j from it appears in the sequence; is the i-th type of k-mer; represents the Hamming distance between k-mer u and v, that is, the number of nucleotides that do not match at the corresponding positions of the two k-mers; is an indicator function that takes 1 when the condition in the parentheses holds, and 0 otherwise.

[0091] PSTNPsS (Position-Specific Trinucleotide Propensity) encoding: Analyze the trinucleotides and their position-specific propensities in the sequence. By calculating all possible trinucleotide combinations and using a feature extraction method with position information, capture the spatial distribution pattern of nucleotides and reveal possible functional or structural patterns in the sequence.

[0092]

[0093] Where: and respectively represent the frequencies of the trinucleotide at position j in the positive and negative samples.

[0094] SCPseKNC encoding: By extracting k-mers through sequence sliding window and combining them with physicochemical properties such as entropy value, the squared difference nucleotide composition is calculated, which can capture more subtle nucleotide association changes in the sequence and has a certain robustness to mutations or sequencing noise.

[0095] One-Hot encoding: Each base (A, T, C, G) in the sequence is represented as a one-hot vector, which is used to reflect the local structure of the base sequence and completely retains the original sequence information.

[0096] Binary encoding: Each base is represented as a binary vector, which is simple and efficient and can express the type distribution of the sequence in low dimensions.

[0097] DAC (Dinucleotide Auto-Covariance) encoding: Evaluates the association between two dinucleotides in the sequence and considers their physicochemical properties.

[0098]

[0099] is the physicochemical property value of the dinucleotide and is the average value of this property.

[0100] ENAC (Enhanced Nucleic Acid Composition): Counts the frequencies of each base appearing in a given window in a sliding window manner, so as to capture local base distribution information.

[0101] ANF (Accumulated Nucleotide Frequency) encoding: Encodes the nucleotide distribution of the entire RNA sequence. It provides a density value for each nucleotide according to the occurrence of each nucleotide at each position in the sequence.

[0102]

[0103] where f(si) is the nucleotide frequency at the i-th position and l is the length of the sequence.

[0104] ASDC (Adaptive Skip Dipeptide Composition) encoding: Considers the relationship between adjacent and spaced bases in the sequence, goes beyond simple dipeptide composition, and models by considering the maximum skip in the sequence as the distance of nucleotides:

[0105]

[0106] represents nucleotides, and captures their occurrence frequencies in the sequence.

[0107] PCP (Physical-Chemical Property) Encoding: Physical and chemical properties of DNA such as melting temperature, polarizability, and hydration have important effects on 5hmc formation and can be used to globally analyze the influence of DNA structural features on modification patterns. A feature matrix is constructed based on the physicochemical properties of dinucleotides (such as stacking energy and melting temperature) to quantify the physical and chemical properties of the sequence. The mean value of all dinucleotides in the sequence is calculated to obtain an overall feature representation, and autocovariance is used to calculate the property changes between different dinucleotides.

[0108] ACGT2Vec Encoding: Drawing on the ideas of natural language processing methods such as Word2Vec, DNA / RNA sequences are converted into vectors. Embedding technology is used to transform each k-mer in the sequence into a low-dimensional vector to obtain a higher-level feature representation. The embedding vectors are trained based on the context of the bases in the DNA sequence, which can better represent the structural information in the DNA sequence.

[0109] The above 12 feature encoding methods can comprehensively characterize the local patterns, physicochemical properties, statistical distributions, and other potential information of DNA sequences.

[0110] In the initial stage, pre-experiments are conducted on the above 12 encoding methods respectively to obtain a candidate set with better performance and feature dimensions. Subsequently, as Figure 3 shown, 6 most valuable feature types (Kmer, PSTNPsS, One-Hot, Binary, SCPseKNC, DAC) are roughly screened from the 12 encoding methods as input features for subsequent modeling to further reduce the high-dimensional redundancy problem.

[0111] S3. Multi-Branch Deep Learning Modeling

[0112] To improve the accuracy and robustness of the model, this solution designs a multi-branch deep learning network that processes different types of features using different network branches. Each feature is extracted through an independent network branch to capture the key pattern information of each feature. This embodiment mainly provides the following three networks for different features:

[0113] CNN Branch: It includes multiple convolutional kernels and pooling layers. The size of the convolutional kernel is 3×1 or 5×1, which is used to capture the local patterns of adjacent bases or short sequences.

[0114] LSTM Branch: It adopts a bidirectional structure with a hidden layer dimension of 64 - 128, which is used to process features with temporal dependencies.

[0115] DNN Branch: It includes fully connected layers with a hidden layer dimension of 128 - 256, which is used to process features without obvious temporal dependencies and capture the non-linear relationships between features.

[0116] For each specific feature encoding, different network branches are used as follows:

[0117] SCPseKNC encoding: The SCPseKNC encoding describes the spatial distribution of nucleotides in a sequence. In this embodiment, an LSTM branch is used for processing, and the LSTM processes this temporal dependence to capture long-range dependencies in the sequence.

[0118] Binary encoding: The Binary encoding represents each base in binary. In this embodiment, an LSTM branch is used for processing to capture the temporal dependence in this sequence feature.

[0119] PSTNPsS encoding: The PSTNPsS encoding reflects the combined characteristics of trinucleotides (short sequence fragments), and extracts features by analyzing the position specificity of trinucleotides in a longer DNA sequence. In this embodiment, a CNN branch is used for processing to extract local features from short sequences.

[0120] One-Hot encoding: The One-Hot encoding represents the local structure of a base sequence through a one-hot vector. In this embodiment, a DNN branch is used to process it to capture the non-linear relationships in long sequences.

[0121] DAC encoding: The DAC encoding evaluates the correlation of dinucleotides. In this embodiment, DNN is used to process this type of global features and extract their complex relationships.

[0122] Kmer encoding: The Kmer encoding generates encoded features by counting the frequencies of nucleotide substrings of a fixed length k (k = 3). In this embodiment, CNN is used for processing to extract local sequence-type information.

[0123] S4. Feature Fusion

[0124] Since there may be information redundancy or complementary relationships among different types of features, simple feature concatenation may lead to too high feature dimensions, thus affecting the learning ability of the model. Therefore, in this solution, a multi-layer self-attention mechanism is introduced in the feature fusion stage. By dynamically calculating the weights of features from different branches, the model can automatically select the most informative feature components.

[0125] Each branch in S3 will finally output a feature vector of a fixed dimension. In the feature fusion stage, a self-attention mechanism is adopted to further adjust the weights of the outputs of each branch to capture the complementary information between different features, ensuring that the model can better adapt to the special requirements of the 5hmC prediction task. The fusion process of the self-attention mechanism:

[0126] Feature stacking: Concatenate the outputs of each branch (referred to as cat for Concatenation) into a tensor of shape (B, 6, 64), where B is the batch size, 6 represents 6 branches, and each branch outputs a 64-dimensional feature vector.

[0127] Multi-head self-attention calculation: Calculate the importance of different features through the multi-head attention mechanism and perform weight normalization:

[0128]

[0129] where Q, K, and V are the query, key, and value matrices respectively, and d k is the scaling factor.

[0130] Residual connection and layer normalization: After calculating the weighted features, introduce residual connection and layer normalization to keep the model stable during deep training.

[0131] Global pooling: Perform average pooling on the fused features to form a final feature vector of fixed length.

[0132] The fused feature vector will be used as the input for SHAP calculation, LASSO screening, and final classification.

[0133] S5. Feature screening and optimization

[0134] After feature fusion, this solution further introduces SHAP value calculation, multi-model cross-comparison, and multi-stage LASSO screening to ensure that the finally used features have the optimal contribution to 5hmC prediction.

[0135] Dynamic SHAP calculation: The SHAP value is an important method for evaluating feature importance. The core idea is to calculate the marginal contribution of each feature to the model prediction result. During the model training process, the SHAP value will be calculated dynamically to reflect the importance of features in real time. In this embodiment, through multiple rounds of calculation, monitor the change of the SHAP value during training, and use time-decaying weights to smooth the noise to ensure that the stable feature contribution is focused on throughout the training process. Its calculation formula is:

[0136]

[0137] where f(s) is the model prediction value using any feature set S, and N is the set of all features.

[0138] The SHAP calculation of this solution adopts a dynamic update mechanism: After each epoch, recalculate the SHAP value to ensure that the feature importance is continuously adjusted as the training progresses. Use time-decaying weighted smoothing of SHAP changes to avoid noise affecting feature selection.

[0139] Furthermore, since the calculation of SHAP value depends on the model structure, different models may give different evaluations of feature importance. Therefore, in order to improve the stability of feature screening, this scheme introduces multi-model cross comparison, namely:

[0140] 1. Use XGBoost, LightGBM, and Random Forest to train the model and calculate the SHAP value respectively.

[0141] 2. Select the features that rank in the top 50% in multiple models and have high consistency in importance, and eliminate the features that fluctuate greatly between different models. In this embodiment, the standard deviation is used as the consistency coefficient.

[0142] 3. Through cross-comparison, ensure that the final selected features have a high contribution under different modeling methods, and improve the robustness of feature screening.

[0143] Only features that have stable contributions in multiple models will enter the LASSO screening stage.

[0144] Multi-stage LASSO screening: LASSO is a feature selection method based on L1 regularization, which can automatically remove features with small weights. This embodiment uses multi-stage LASSO regression for feature screening based on dynamic SHAP sorting. Through multiple rounds of LASSO regression, redundant features are further eliminated, and finally a set of streamlined and efficient feature subsets is formed. The optimization objectives of LASSO are as follows:

[0145]

[0146] Among them, λ is the regularization coefficient, w is the feature weight vector, y is the label, and X is the feature matrix. This scheme uses multi-stage LASSO screening:

[0147] First-stage LASSO: LASSO screening is performed based on the initial feature set to remove the least important 20% features.

[0148] Second stage LASSO (based on SHAP ranking adjustment): further screening, only retaining features with high SHAP importance, and performing LASSO dimensionality reduction again.

[0149] Final feature set: Only the 50% fused feature vectors with the greatest impact on 5hmC prediction are retained to improve the generalization ability of the model and reduce the computational overhead caused by redundant features.

[0150] S4. Final prediction and training optimization

[0151] After feature screening, the obtained final feature set is input into the fully connected layer FC to predict 5hmc sites. The fully connected layer FC is the classifier used in this model. During training, BCEWithLogitsLoss (binary cross-entropy loss) or FocalLoss (focal loss) is used to handle imbalanced data. The AdamW optimizer is adopted and combined with CosineAnnealingLR (cosine annealing learning rate adjustment) to dynamically adjust the learning rate. To avoid overfitting, this solution adds regularization means such as Dropout (dropout layer) and BatchNormalization (batch normalization), and at the same time uses an early stopping mechanism (patience) to monitor the validation set loss. When the loss does not improve in multiple iterations, the training is stopped in advance. In addition, through an adaptive regularization and adaptive resampling strategy, the robustness of the model is further improved.

[0152] To verify the performance and robustness of this solution, in this embodiment, indicators such as ROC AUC (area under the ROC curve), Precision (precision), SN (sensitivity / true positive rate), SP (specificity), MCC (Matthews correlation coefficient), F1_score (F1 score), Recall (recall rate), and Accuracy (accuracy rate) are used to compare and evaluate the classification effects of the model implemented by this solution and traditional methods. Figure 4 It is a radar performance comparison chart of the model of this solution and the traditional model iRhm5CNN regarding indicators such as ROC AUC, Precision, MCC, F1_score, Recall, and Accuracy. It can be seen that the model of this solution is superior to the traditional model in any indicator.

[0153] In addition, this embodiment also compares the performance of different encoded feature data in the 5hmc recognition task, such as Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 、 Figure 10 As shown, in the figure, our_features are the fusion features obtained in this application, and the rest are the corresponding features. It can be seen that the fusion features are basically superior to the single features in various indicators. For recall, although the fusion features are slightly lower than kmer, combined with other aspects such as Precision, the comprehensive performance is obvious, which can verify the effectiveness of this solution in multi-source data integration.

[0154] As Figure 11 shown, this solution also proposes a 5hmC prediction system based on multi-branch deep learning and adaptive feature screening to implement the above steps to construct a prediction model for 5hmC modification site prediction, including:

[0155] A data preprocessing module 6, which is used to clean the sequencing data, extract the context sequences, screen out the common modification sites, and construct positive and negative sample sets;

[0156] A multi-source feature encoding module 2, which is used to perform Kmer frequency encoding, PSTNPsS encoding, One-Hot encoding, Binary encoding, SCPseKNC encoding, and DAC encoding on the positive and negative sample sets to generate a multi-dimensional feature set;

[0157] A multi-branch deep learning module 3, including a CNN branch, an LSTM branch, a DNN branch, and a self-attention fusion unit; the CNN branch processes the Kmer frequency encoding features and PSTNPsS encoding features, the LSTM branch processes the SCPseKNC encoding features and Binary encoding features, and the DNN branch processes the One-Hot encoding features and DAC encoding features respectively; the self-attention fusion unit dynamically weights and fuses the features output by each neural network branch;

[0158] A dynamic feature screening module 7, including SHAP calculation and LASSO regression, which is used to combine dynamic SHAP value calculation and multi-stage LASSO regression to screen out several feature subsets that contribute the most to 5hmC prediction from the features output by the self-attention fusion unit;

[0159] A training module 8, which uses an adaptive regularization and adaptive resampling strategy to train a prediction model including multiple neural network branches, a self-attention mechanism, and a classifier 5 through a class imbalance loss function for predicting 5hmC modification sites.

[0160] Furthermore, as Figure 12 shown, the present solution also proposes a 5hmC prediction system in which a prediction model can be used for predicting 5hmC modification sites after training, including an interaction module 1, which is used to receive the DNA context sequence to be predicted and output a prediction result;

[0161] A multi-source feature encoding module 2, which is used to perform Kmer frequency encoding, PSTNPsS encoding, One-Hot encoding, Binary encoding, SCPseKNC encoding, and DAC encoding on the DNA context sequence to be predicted to obtain a multi-dimensional feature set;

[0162] The multi-branch deep learning module 3 includes a CNN branch, an LSTM branch, a DNN branch, and a self-attention fusion unit; the CNN branch processes Kmer frequency encoding features and PSTNPsS encoding features, the LSTM branch processes SCPseKNC encoding features and Binary encoding features, and the DNN branch processes One-Hot encoding features and DAC encoding features respectively; the self-attention fusion unit weights and fuses the features output by each neural network branch;

[0163] The feature screening module 4 pre-stores the feature types screened during the training process, and is used to extract the pre-stored feature types from the features output by the neural network branches as a feature subset for output;

[0164] The classifier 5 is used to output a prediction result based on the feature subset output by the feature screening module 4.

[0165] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the described specific embodiments or use similar ways for substitution, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A 5hmC prediction method based on multi-branch deep learning and adaptive feature screening, characterized in that, The method includes: S1. Screening common 5hmC modification sites from multiple sequencing data to construct positive and negative sample sets; S2. Performing sequence-type feature encoding, physicochemical property feature encoding, and statistical-type feature encoding on the positive and negative samples respectively to generate a multi-dimensional feature set; S3. Inputting the features of different encoding types into the corresponding neural network branches for processing respectively; S4. Dynamically weighting and fusing the features output by each neural network branch through a self-attention mechanism, and combining dynamic SHAP value calculation and multi-stage LASSO regression to screen several feature subsets that contribute the most to 5hmC prediction; S5. Adopting an adaptive regularization and adaptive resampling strategy to train a prediction model including multiple neural network branches, a self-attention mechanism, and a classifier through a class imbalance loss function for predicting 5hmC modification sites; The sequence-type feature encoding includes any one or more of One-Hot encoding, Binary encoding, Kmer encoding, PSTNPsS encoding, Mismatch encoding, and ENAC encoding; The physicochemical property feature encoding includes any one or more of PCP encoding and SCPseKNC encoding; The statistical-type feature encoding includes any one or more of DAC encoding, ANF encoding, and ASDC encoding; The neural network branches include a CNN branch, an LSTM branch, and a DNN branch; The CNN branch includes multiple convolutional kernels for extracting Kmer frequency encoding features and local sequence-type features of PSTNPsS encoding; The LSTM branch adopts a bidirectional structure for processing the physicochemical property time series features of SCPseKNC encoding and the binary sequence dependence of Binary encoding; The DNN branch includes a fully connected layer and a non-linear activation function for extracting One-Hot encoding features and global non-linear associations of DAC encoding; 2. The 5hmC prediction method based on multi-branch deep learning and adaptive feature screening according to claim 1, wherein In step S2, Kmer frequency encoding, PSTNPsS encoding, One-Hot encoding, Binary encoding, SCPseKNC encoding, and DAC encoding are performed on the positive and negative samples respectively to generate a multi-dimensional feature set; The neural network branches include two CNN branches for processing Kmer frequency encoding features and PSTNPsS encoding features respectively, two LSTM branches for processing SCPseKNC encoding features and Binary encoding features respectively, and two DNN branches for processing One-Hot encoding features and DAC encoding features respectively.

3. The 5hmC prediction method based on multi-branch deep learning and adaptive feature screening according to claim 1, wherein In step S3, each neural network branch outputs a feature vector with a fixed and same dimension, and the features output by each neural network branch are dynamically weighted and fused by a self-attention mechanism to capture complementary information between different features.

4. The 5hmC prediction method based on multi-branch deep learning and adaptive feature screening according to claim 3, wherein Step S4 specifically includes: Stacking the outputs of each neural network branch into a tensor with a specific shape; Calculating the importance of different features through a multi-head attention mechanism and normalizing the weights; After calculating the weighted features, introducing a residual connection and layer normalization; Performing average pooling on the fused features to form a candidate feature vector with a fixed length; The final feature vector is selected from the candidate feature vectors through SHAP value calculation and multi-stage LASSO regression and used as the input of the classifier.

5. The 5hmC prediction method based on multi-branch deep learning and adaptive feature screening according to claim 4, wherein In step S4, the SHAP value calculation adopts a dynamic update mechanism. After each training round, SHAP is recalculated. Through multiple rounds of calculation, the change of SHAP value with training is monitored, and the time decay weight is used to smooth the SHAP change. Finally, several features with the highest SHAP values are selected. After several features are selected through SHAP value calculation, they enter the multi-stage LASSO regression screening stage, and finally some features with the greatest impact on 5hmC prediction are retained from the selected features.

6. The 5hmC prediction method based on multi-branch deep learning and adaptive feature screening according to claim 5, wherein In step S4, the process of screening features by SHAP value also includes multi-model cross-comparison: Use XGBoost, LightGBM, and random forest to train models and calculate SHAP values respectively; Select some features that rank among the top set percentages in multiple models and whose importance consistency coefficients are higher than the set value.

7. A 5hmC prediction system based on multi-branch deep learning and adaptive feature screening, characterized in that, Including: A data preprocessing module (6) for screening common 5hmC modification sites from various sequencing data to construct positive and negative sample sets; A multi-source feature encoding module (2) for performing sequence-type feature encoding, physicochemical property feature encoding, and statistical-type feature encoding on the positive and negative sample sets to generate a multi-dimensional feature set; The sequence-type feature encoding includes any one or more of One-Hot encoding, Binary encoding, Kmer encoding, PSTNPsS encoding, Mismatch encoding, and ENAC encoding; The physicochemical property feature encoding includes any one or more of PCP encoding and SCPseKNC encoding; The statistical-type feature encoding includes any one or more of DAC encoding, ANF encoding, and ASDC encoding; A multi-branch deep learning module (3) includes multiple neural network branches and a self-attention fusion unit; The multiple neural network branches are used to process the features of the corresponding encoding types respectively, and the self-attention fusion unit is used to dynamically weight and fuse the features output by each neural network branch; The neural network branches include a CNN branch, an LSTM branch, and a DNN branch; The CNN branch includes multiple convolutional kernels for extracting Kmer frequency encoding features and local sequence-type features of PSTNPsS encoding; The LSTM branch adopts a bidirectional structure for processing the physicochemical property time series features of SCPseKNC encoding and the binary sequence dependence of Binary encoding; The DNN branch includes a fully connected layer and a non-linear activation function for extracting One-Hot encoding features and global non-linear associations of DAC encoding; A dynamic feature screening module (7) for screening several feature subsets with the greatest contribution to 5hmC prediction from the features output by the self-attention fusion unit by combining dynamic SHAP value calculation and multi-stage LASSO regression; A training module (8) that uses an adaptive regularization and adaptive resampling strategy to train a prediction model including multiple neural network branches, a self-attention mechanism, and a classifier (5) through a class imbalance loss function for predicting 5hmC modification sites.

8. A 5hmC prediction system based on multi-branch deep learning and adaptive feature screening, characterized in that, Including: An interaction module (1) for receiving a DNA context sequence to be predicted and outputting a prediction result; A multi-source feature encoding module (2) for performing sequence-type feature encoding, physicochemical property feature encoding, and statistical feature encoding on the DNA context sequence to be predicted to generate a multi-dimensional feature set; The sequence-type feature encoding includes any one or more of One-Hot encoding, Binary encoding, Kmer encoding, PSTNPsS encoding, Mismatch encoding, and ENAC encoding; The physicochemical property feature encoding includes any one or more of PCP encoding and SCPseKNC encoding; The statistical feature encoding includes any one or more of DAC encoding, ANF encoding, and ASDC encoding; A multi-branch deep learning module (3) including multiple neural network branches and a self-attention fusion unit; the multiple neural network branches are used to process features of corresponding encoding types respectively, and the self-attention fusion unit dynamically weights and fuses the features output by each neural network branch; The neural network branches include a CNN branch, an LSTM branch, and a DNN branch; The CNN branch includes multiple convolutional kernels for extracting Kmer frequency encoding features and local sequence-type features of PSTNPsS encoding; The LSTM branch adopts a bidirectional structure for processing the physicochemical property time series features of SCPseKNC encoding and the binary sequence dependence of Binary encoding; The DNN branch includes a fully connected layer and a non-linear activation function for extracting One-Hot encoding features and global non-linear associations of DAC encoding; A feature screening module (4) pre-stores the feature types obtained by dynamic SHAP value calculation and multi-stage LASSO regression screening during the training process, and is used to extract the pre-stored feature types from the features output by the neural network branches as a feature subset for output; A classifier (5) for outputting a prediction result based on the feature subset output by the feature screening module (4).

Images