A deep learning-based method and system for filtering genomic variants

Abstract

The application discloses a kind of based on deep learning's genomic variation filtering method and system, in pre-processing stage, five different feature engineering methods are used to generate hundreds of candidate features, based on the existing features of genomic data, enhance the learning and generalization ability of model.Design machine learning model LightGBM filters the set of original features and candidate features, and select valuable features based on the importance score of feature, to improve model training efficiency and prediction performance, reduce the influence of noise on model performance.After that, by calculating the correlation matrix between features, and combining hierarchical clustering, features are grouped and sorted, which can effectively identify and remove redundant features, reduce the influence of multicollinearity.This method optimizes the feature set, so that the model maintains high prediction ability while having better stability and generalization performance.Finally, the input layer is adjusted from a matrix to a single-channel grayscale image, and the model is generated by comparing various indicators of the convolutional neural network model in transfer learning (ResNet34).Through a more complex model, better filtering performance is obtained, thereby effectively improving the accuracy of genomic variation filtering.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology in artificial intelligence, specifically to a method and system for filtering genomic variations based on deep learning. Background Technology

[0002] With the rapid development of Next-Generation Sequencing (NGS) technology and the continuous decline in sequencing costs, the accumulation of large-scale, high-throughput genomic data has greatly promoted the advancement of variant detection technology. Currently, comprehensive detection of genomic variants, including single nucleotide polymorphisms (SNPs), insertion / deletion polymorphisms (INDELs), structural variations (such as inversions, translocations, deletions, and duplications), and copy number variants (CNVs), has become a crucial foundation for understanding gene function, elucidating disease mechanisms, and developing personalized medicine. Genomic variant detection based on NGS technology not only provides key clues for understanding the genetic diversity of biological traits but also lays an important foundation for revealing the molecular pathological mechanisms of diseases and exploring the laws of species evolution. However, NGS inevitably introduces false-positive variants during sequencing and variant detection. These false positives may originate from sequencing errors, genomic region characteristics (such as GC content bias), or limitations of existing detection algorithms. These errors not only affect subsequent genomic analysis but may also lead to erroneous association findings, hindering the development of precision medicine.

[0003] To address this issue, various variant filtering methods have been proposed in recent years, mainly falling into two categories: traditional filtering methods and machine learning-based filtering methods. Hard filtering is a simple and early-applied variant filtering method where users can set fixed thresholds based on metrics such as quality score and sequencing depth for screening. The VQSR method utilizes a Gaussian mixture model for variant quality calibration and is widely used in large-scale genomic data. TVC is designed specifically for the IonTorrent sequencing platform and can efficiently detect SNPs and INDELs. Furthermore, frequency filtering methods primarily assess variant confidence based on variant allele frequency (VAF) or allele depth (AD). While traditional methods perform well in specific applications, their flexibility and accuracy are often limited when handling complex, high-dimensional data. In contrast, deep learning and machine learning methods can automatically learn data features, thereby improving the flexibility and accuracy of variant filtering. GARFIELD-NGS is a variant identification tool based on a multilayer perceptron (MLP) for quality control GATK and is suitable for standardizing high-quality sequencing data. FVC, based on the adaptive boosting model XGBoost, automatically adjusts parameters to adapt to data obtained from different variant detectors through comprehensive analysis of multiple features. VEF utilizes ensemble learning methods, integrating multiple decision tree models to achieve efficient filtering, enhancing the robustness and flexibility of the filtering process. ForestQC, based on the random forest algorithm, combines sequencing depth, genotype quality, and other multidimensional features, fusing traditional filtering with machine learning strategies to effectively remove false positives. It is suitable for diverse sequencing data, especially showing superior performance in high-GC-content regions and repetitive regions.

[0004] Despite significant progress in false positive filtering techniques based on deep learning and machine learning, existing methods still suffer from two main problems: (1) They neglect complex relationships in data features (e.g., nonlinear relationships, interaction relationships, and asymmetry). For example, the VEF tool relies on the original features of whole genome variant identification data, and while the FVC tool integrates all possible original features from different variant identifiers, it is still limited to the original features themselves. The original features cannot capture the complex relationships that may exist in the data features, and the utilization of the feature space is still insufficient, making it difficult to capture subtle differences and trends in the data. (2) The data processing methods are simplistic and the model generalization ability is limited. The ForestQC tool applies a feature matrix to the machine learning model, but the matrix data cannot preserve the spatial structure of the data, limiting the learning of complex patterns. Although the GARFIELD-NGS tool applies an MLP model, it also relies on a feature matrix to train the model. The matrix data cannot preserve the spatial structure of the data, and compared with other complex deep learning models, machine learning and MLP models still have certain gaps in feature extraction ability, spatial information capture, model performance and accuracy, training efficiency, and generalization ability.

[0005] Therefore, this invention proposes a deep learning-based genomic variant filtering method to improve the accuracy of false positive variant filtering and the generalization ability of the model. Summary of the Invention

[0006] This invention relates to a deep learning-based genomic variation filtering method, aiming to address the problems of neglecting complex relationships in data features, insufficient utilization of feature space, and weak model generalization ability in existing technologies. This invention designs a feature engineering module that combines machine learning methods, feature correlation matrix analysis, and hierarchical clustering to expand feature dimensions and capture potential complex nonlinear and interactive relationships in the data. Specifically, firstly, the original features and newly generated candidate features are integrated, and feature filtering is performed based on the LightGBM model to extract a set of high-value information features. Subsequently, the correlation matrix between features is calculated, and hierarchical clustering is applied to further remove redundant features and reduce the adverse effects of multicollinearity on model training. Based on this, this invention constructs a convolutional neural network model based on transfer learning. By encoding the original features and the filtered feature set into grayscale image data, the spatial structure information of the samples is fully preserved to enhance the convolutional neural network's ability to extract spatial patterns. After testing and evaluating various CNN architectures, ResNet-34 was finally selected as the backbone network, combined with residual connections and batch normalization design to improve deep feature extraction capabilities, avoid gradient vanishing, and enhance the model's generalization performance. Through the above design, the present invention can effectively improve the accuracy and robustness of identifying true and false genomic variations in complex sequencing data, and is applicable to genomic data of various qualities and sources.

[0007] Therefore, the technical solution adopted in this invention is a genomic variation filtering method based on deep learning, comprising the following steps:

[0008] Machine learning models were used to predict missing values ​​in variant identification files, and then candidate features were generated using five different feature engineering methods based on the original features of the genomic data.

[0009] Based on the input original features and candidate features, the LightGBM model is used to select valuable features according to their importance scores.

[0010] By calculating the correlation matrix between features and combining hierarchical clustering to group and rank the features, candidate features for screening and ranking are obtained.

[0011] The original features and the selected candidate features are used to construct a grayscale image, which is then input into a convolutional neural network to output the true and false positive categories of gene mutations.

[0012] Furthermore, the original features include base quality rank sum, read position rank sum, depth, chain deviation Fisher test value, alignment quality, alignment quality rank sum, quality-depth ratio, chain ratio, quality value, and genotype quality; the feature engineering methods include mathematical transformation, binning operation, interactive feature generation, clustering feature generation, and nonlinear transformation.

[0013] Furthermore, the clustering feature generation employs the K-means clustering algorithm, including the following steps:

[0014] First, three centroids are randomly initialized as cluster centers. The distance from each data point to a centroid is calculated, and the data point is assigned to the nearest cluster. Then, the centroids of each cluster are recalculated, and the assignment and update process is repeated until the centroids stabilize or the maximum number of iterations is reached. The core mathematical formula revolves around its objective function, which is to minimize the intra-cluster squared error and SSE, expressed as:

[0015]

[0016] Where K is the number of clusters, and C is... k Let μ represent the dataset of the k-th cluster. k It is the centroid of the cluster, x i Here, ||·|| represents the data point, and ||·|| represents the Euclidean distance. The algorithm optimizes SSE through two iterative steps: The first allocation step assigns each data point to the cluster of the nearest centroid, i.e.

[0017]

[0018] The second update step: Recalculate the mean of the points within each cluster, which is the centroid of the cluster.

[0019]

[0020] Iterate until the centroid stops changing or the maximum number of iterations is reached.

[0021] Furthermore, the LightGBM model is a binary classifier based on Gradient Boosting Decision Tree (GBDT), trained efficiently using a leaf-wise growth strategy and histogram optimization algorithm. The model is configured with 31 maximum leaf nodes to control the complexity of a single tree, and iterates for 100 rounds with a learning rate of 0.05, randomly selecting 90% of features in each round to enhance generalization ability. By calculating the cumulative gain of features during tree node splits, the normalized feature importance is output, and features with importance greater than 0.001 are selected for subsequent analysis. The model uses log loss as the optimization objective and classification error rate as the evaluation metric.

[0022] Furthermore, the correlation matrix is ​​constructed using the Spearman correlation coefficient to create a nonlinear correlation matrix between features, where R... x R y The rank of the eigenvalues ​​is given by the formula:

[0023]

[0024] Where Cov represents the rank covariance, σ is the standard deviation of the rank, and ρ is the Spearman rank correlation coefficient. The subsequent distance transformation converts the correlation matrix into a distance matrix to eliminate the directional differences between positive and negative correlations. The formula is as follows:

[0025] Distance = 1 - |ρ|

[0026] Where Distance represents the elimination of positive and negative correlation differences through absolute value, with a result range of [0,1], so that negative correlations are not mistakenly interpreted as "far" distances. The hierarchical clustering uses the average linking method to generate a cluster tree, and extracts the leaf node order to rearrange the feature columns, as shown in the following formula:

[0027]

[0028] Where |A| and |B| are the number of samples in clusters A and B, d(a,b) is the distance between sample a and sample b, and D(A,B) represents the distance between clusters A and B.

[0029] Furthermore, when constructing the grayscale image, each pixel represents a feature of a row of data. First, the numerical features are standardized. Then, the one-dimensional feature vector of each sample is sequentially filled into a 24×24 pixel matrix (padding with zeros for insufficient parts and truncating excessively long parts), forming a 1×40×40 tensor structure. The numerical range is mapped to the [-2.5, 2.5] interval after Z-score standardization. Finally, a linear transformation from numerical values ​​to grayscale is achieved through grayscale mapping, transforming the one-dimensional features of the original table into a two-dimensional image representation with spatial locality, adapting to the ResNet-34 convolutional neural network architecture (the first layer convolutional kernel is adjusted to 5×5 to accommodate small input sizes).

[0030] This invention also provides a deep learning-based genomic variation filtering system, comprising:

[0031] The candidate feature building module is used to apply machine learning models to predict missing values ​​in variant identification files. Then, based on the original features of the genomic data, five different feature engineering methods are used to generate candidate features.

[0032] The valuable feature building module is used to select valuable features based on the importance scores of the original and candidate features input using the LightGBM model.

[0033] The candidate feature selection and ranking module is used to obtain candidate features for selection and ranking by calculating the correlation matrix between features and combining hierarchical clustering to group and rank the features.

[0034] The convolutional neural network module is used to input a grayscale image constructed from the original features and the selected and sorted candidate features into the convolutional neural network, and output the true and false positive categories of gene mutations.

[0035] The beneficial effects of this invention are:

[0036] 1. This invention expands the feature dimension through feature engineering methods, making full use of the possible complex relationships between data features and capturing the potential complex nonlinear relationships and interactions in the data.

[0037] 2. This invention converts traditional data matrices into images, enabling the use of CNNs to filter out true and false mutations, thereby improving the model's performance and generalization ability under limited computing resources. Attached Figure Description

[0038] Figure 1 Here are (a) an overall flowchart and (b) a model structure diagram of the present invention;

[0039] Figure 2 This demonstrates the performance of the present invention on ILM INDEL data;

[0040] Figure 3 This demonstrates the performance of the present invention on ION INDEL data;

[0041] Figure 4 This demonstrates the performance of the present invention on ILM SNP data.

[0042] Figure 5 This is the performance of the present invention on ION SNP data. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] A deep learning-based method for filtering genomic variants includes the following steps:

[0045] Step 1: Apply a machine learning model (LightGBM) to predict missing values ​​in the VCF file. Then, based on the existing features of the genomic data, use five different feature engineering methods to generate a large number of candidate features to enhance the model's learning and generalization capabilities.

[0046] Step 2: Use the LightGBM model to filter newly generated features and select valuable features (greater than the threshold of 0.001) based on the importance score of the features to improve model training efficiency and prediction performance and reduce the impact of noise on model performance.

[0047] Step 3: By calculating the correlation matrix between features and grouping and ranking the features using hierarchical clustering, redundant features can be effectively identified and removed, reducing the impact of multicollinearity. This method optimizes the feature set, enabling the model to maintain efficient predictive capabilities while possessing better stability and generalization performance.

[0048] Step 4: The model is generated through comparison of CNN models in transfer learning (ResNet34 has the best overall performance), which can capture spatial information between features and enhance feature extraction. The input layer is adjusted from a matrix to a single-channel grayscale image to match the original features of the mutation identification data and the selected candidate features, thereby enhancing the model's performance.

[0049] Step 5: Based on the research objectives and data characteristics, select evaluation metrics such as Area Under the ROC Curve (AUC), Matthews Correlation Coefficient (MCC), F1 score (F1), and Accuracy (ACC) to comprehensively evaluate the model's performance and practicality.

[0050] The specific operations of step 1 are as follows: For the few missing values ​​of the three features (BaseQRankSum, ReadPosRankSum, and Alignment Quality RankSum) in the variant identification file (VCF format file), prediction based on the LGBM machine learning model is performed. Then, based on 10 original features (BaseQRankSum, ReadPosRankSum, Depth, Chain Bias Fisher Test Value, Alignment Quality, Alignment Quality RankSum, Quality-Depth Ratio, Chain Ratio Ratio, Quality Value, and Genotype Quality), five feature engineering methods (mathematical transformation, binning, interactive feature generation, clustering feature generation, and nonlinear transformation) are used to generate hundreds of candidate features. The K-means clustering algorithm (K-Means) is a classic unsupervised clustering method that divides the data into K clusters through iterative optimization. In this experiment, three centroids are randomly initialized as cluster centers. The distance from each data point to the centroid is calculated, and the data point is assigned to the nearest cluster. Then, the centroids of each cluster (i.e., the mean of points within the cluster) are recalculated. This assignment and update process is repeated until the centroids stabilize or the maximum number of iterations is reached. The core mathematical formula revolves around its objective function, which is to minimize the sum of squared errors (SSE) within the cluster. The formula is expressed as:

[0051]

[0052] Where K is the number of clusters, and C is... kLet μ represent the dataset of the k-th cluster. k x is the centroid (mean vector) of the cluster. i Here, ||·|| represents the data point, and ||·|| represents the Euclidean distance. The algorithm optimizes SSE through two iterative steps: The first allocation step: assign each data point to the cluster of the nearest centroid, i.e.

[0053]

[0054] The second update step: Recalculate the mean of the points within each cluster, which is the centroid of the cluster.

[0055]

[0056] Iterate until the centroid no longer changes or the maximum number of iterations is reached. When processing discrete categorical features, one-hot encoding is used to convert the categorical features into binary vectors to eliminate numerical ambiguity. Specifically, if a feature has N mutually exclusive classes, an N-dimensional new feature is generated, with each dimension corresponding to a class. The sample is assigned a value of 1 at the class-corresponding position in the original feature, and 0 for the rest.

[0057] The specific operations of step 2 are as follows: The 10 original features and hundreds of candidate features generated by feature engineering are integrated together. LightGBM is applied to filter the integrated features (selected after comparison with Random Forest). The most valuable features are selected based on their importance scores to ensure that the model uses the maximum amount of information and minimizes noise during training. The correlation matrix between features is calculated to identify similarities and redundancy among features. Hierarchical clustering algorithm is then used to group and rank the features. Feature ranking aims to remove redundant features, reduce multicollinearity among features, and avoid the negative impact of information redundancy on model training.

[0058] The LightGBM model is a binary classifier based on Gradient Boosting Decision Trees (GBDT), employing a leaf-wise growth strategy and histogram optimization algorithm for efficient training. The model uses 31 maximum leaf nodes to control the complexity of a single tree, and iterates for 100 rounds with a learning rate of 0.05. In each round, 90% of the features are randomly selected to enhance generalization ability. By calculating the cumulative gain of features during tree node splits, the model outputs the normalized feature importance and selects features with importance greater than 0.001 for subsequent analysis. The model uses log loss as the optimization objective and classification error rate as the evaluation metric.

[0059] The specific steps for step 3 are as follows:

[0060] This approach involves calculating the correlation matrix between features to identify similarities and redundancies, and then grouping and ranking the features using a hierarchical clustering algorithm. Feature ranking aims to remove redundant features, reduce multicollinearity among features, and avoid the negative impact of information redundancy on model training. This helps optimize the feature set, enabling the model to maintain efficient predictive capabilities while exhibiting stronger stability and better generalization ability, making it suitable for a wider range of datasets and scenarios. The correlation calculation utilizes the Spearman correlation coefficient to construct a non-linear correlation matrix between features, where R0... x R y The rank of the eigenvalues ​​is given by the formula:

[0061]

[0062] Where Cov represents the rank covariance, σ is the standard deviation of the rank, and ρ is the Spearman rank correlation coefficient. The subsequent distance transformation converts the correlation matrix into a distance matrix to eliminate the directional differences between positive and negative correlations. The formula is as follows:

[0063] Distance = 1 - |ρ|

[0064] Where Distance represents the elimination of positive and negative correlation differences through absolute value, with a result range of [0,1], so that negative correlations are not mistakenly interpreted as "far" distances. Subsequent hierarchical clustering uses the average linking method to generate a cluster tree, and the leaf node order is extracted to rearrange the feature columns, as shown in the following formula:

[0065]

[0066] Where |A| and |B| are the number of samples in clusters A and B, d(a,b) is the distance between sample a (from cluster A) and sample b (from cluster B), obtained from the previous formula distance, and D(A,B) represents the distance between clusters A and B.

[0067] The specific operations of step 4 are as follows: First, a 24*24 grayscale image is constructed by scanning the original features and sorted candidate features of each row of the genome variant identification data file (VariantCall Format, VCF) to capture the spatial signal between the data in that row. Each pixel represents a feature of a row of data. Second, the input shape is adjusted according to the single-channel image of the input layer, and the most suitable convolution kernel size, stride, and padding are selected in the first convolutional layer to optimize the efficiency and accuracy of feature extraction. The output layer is designed as a binary classification structure to distinguish between true and false positive categories of gene variants. Then, regarding model selection, we will test all CNN models adapted to this type of input in transfer learning, including more than a dozen models such as VGG, ResNet, AlexNet, ConvNeXt, DenseNet, EfficientNet, MobileNet, NASNet, SqueezeNet, and Xception. These models have different advantages in feature extraction and performance through different architecture designs and convolution strategies.

[0068] The construction of the grayscale image specifically includes: first, standardizing the numerical features; then, sequentially filling the one-dimensional feature vector of each sample into a 24×24 pixel matrix (padding with zeros where insufficient and truncating excessively long parts) to form a 1×40×40 tensor structure, where the numerical range is mapped to the [-2.5, 2.5] interval after Z-score standardization; finally, achieving a linear conversion from numerical values ​​to grayscale through grayscale mapping, transforming the one-dimensional features of the original table into a two-dimensional image representation with spatial locality, adapting to the ResNet-34 convolutional neural network architecture (adjusting the first layer convolutional kernel to 5×5 to accommodate small input sizes).

[0069] The specific operations of step 5 are as follows: To evaluate the model's performance, we used the Area Under the ROC Curve (AUC) score, the Matthews Correlation Coefficient (MCC) coefficient, and the F1 score (F1) as the three most important evaluation metrics. AUC measures the model's performance at different thresholds, MCC is a comprehensive measure that has good adaptability to imbalanced data, and the F1 score balances the model's precision and recall. In addition, this embodiment also uses metrics such as accuracy (ACC), area under the precision-recall curve (AUPRG), false omission rate (OFO), F1-minor score, balanced accuracy (BACC), and negative predictive value (NPV) to measure the overall performance of the model. Generally, the six metrics given in the formula below are affected by thresholds. That is, samples greater than or equal to the threshold are predicted as positive samples, while samples less than the threshold are considered as negative samples. The default value of the threshold is 0.5, but it can be manually adjusted.

[0070]

[0071]

[0072] In this context, TP (True positive) represents the number of true positives, which is the number of true genomic variants correctly predicted as those that need to be retained; TN (True negative) represents the number of true negatives, which is the number of genomic variants to be filtered correctly predicted as those that need to be filtered; FP (False positive) represents the number of false positives, which is the number of genomic variants that originally needed to be filtered but were predicted as those that need to be retained; and FN (False negative) represents the number of false negatives, which is the number of genomic variants that originally needed to be retained but were predicted as those that need to be filtered.

[0073] Finally, after considering the above evaluation metrics, the ResNet34 model in transfer learning was selected as the best solution. Specifically, ResNet-34 consists of 16 residual blocks, each containing two convolutional layers. Each convolutional layer is followed by batch normalization and ReLU activation functions to ensure stable gradient flow and accelerate convergence.

[0074] In one embodiment: a deep learning-based method for filtering genomic variants, such as... Figure 1 As shown, it includes the following steps:

[0075] Step 1: Feature Engineering: In the data preprocessing stage, there are four datasets from two platforms: Illumina indels, Illumina snps, ION indels, and ION snps (derived from variant identification file format conversion). LightGBM is applied to predict the few missing values ​​in Illumina snps. Based on the data features of Illumina indels and ION indels, 1069 and 2945 candidate features are generated from 10 and 18 original features, respectively, using five feature engineering methods.

[0076] Steps 2 and 3: Feature selection: After feature engineering, a large number of candidate features are generated. LightGBM is then used to select the original features and candidate features, and the most valuable feature set (422 features for Illumina and 692 features for ION) is selected for subsequent model training. Then, the correlation matrix between features is calculated according to the characteristics of convolutional neural networks. Hierarchical clustering algorithm is used to group and sort the features with higher correlation. The ablation experiments of feature engineering and selection are shown in Table 1.

[0077] Table 1 Comparison of ablation experiments using feature engineering and transfer learning

[0078]

[0079] Step 4: Transfer Learning: The processed feature INDELs and original feature SNPs are converted into 24*24 grayscale images, row by row (1*N). Each pixel represents a feature of a row of data. These images are then fed into the convolutional neural network in the transfer learning process using a 5*5 convolutional kernel with a stride of 1 and padding of 2. The comparison is based on AUC, MCC, and F1, the three most important metrics. ResNet34 was ultimately chosen as the basic architecture. The model is based on 16 residual blocks and consists of four stages. Each residual block contains two convolutional layers, followed by batch normalization and ReLU activation functions. The specific model structure is as follows: Figure 1 As shown.

[0080] Step 5: Comparative Analysis: A comparative experiment was conducted on the four datasets and GARFIELD's MLP model. The AUC metric increased from 92.69 to 99.75 on the ILM INDEL dataset, representing an average increase of 7%. On the other three datasets, ILM SNP, ION INDEL, and ION SNP, the AUC increased by 10%, 4%, and 1% respectively, all achieving considerable performance improvements. The AUC curves for the four datasets are shown below. Figure 2 , 3 As shown in Figures 4 and 5.

Claims

1. A deep learning-based method for filtering genomic variations, characterized in that, Includes the following steps: Machine learning models are applied to predict missing values ​​in variant identification files. Then, based on the original features of the genomic data, five different feature engineering methods are used to generate candidate features. The original features include base quality rank sum, read position rank sum, depth, chain deviation Fisher test value, alignment quality, alignment quality rank sum, quality-depth ratio, chain ratio, quality value, and genotype quality. The feature engineering methods include mathematical transformations, binning operations, interactive feature generation, clustering feature generation, and nonlinear transformations. The clustering feature generation uses the K-means clustering algorithm and includes the following steps: First, three centroids are randomly initialized as cluster centers. The distance from each data point to a centroid is calculated, and the data point is assigned to the nearest cluster. Then, the centroids of each cluster are recalculated, and the assignment and update process is repeated until the centroids stabilize or the maximum number of iterations is reached. The core mathematical formula revolves around its objective function, which is to minimize the intra-cluster squared error and SSE, expressed as: Where K is the number of clusters, C k This represents the data set of the k-th cluster. It is the centroid of the cluster, x i Here, ||⋅|| represents the data point, and ||⋅|| represents the Euclidean distance. The algorithm optimizes SSE through two iterative steps: The first allocation step assigns each data point to the cluster of the nearest centroid, i.e. The second update step: Recalculate the mean of the points within each cluster, which is the centroid of the cluster. Iterate until the centroid no longer changes or the maximum number of iterations is reached; Based on the input original features and candidate features, the LightGBM model is used to select valuable features according to their importance scores. By calculating the correlation matrix between features and combining hierarchical clustering to group and rank the features, candidate features for screening and ranking are obtained. The original features and the selected candidate features are used to construct a grayscale image, which is then input into a convolutional neural network to output the true and false positive categories of gene mutations.

2. The genomic variation filtering method based on deep learning according to claim 1, characterized in that: The LightGBM model is a binary classifier based on gradient boosting decision trees. It employs a leaf-wise growth strategy and a histogram optimization algorithm. The model is configured with 31 maximum leaf nodes to control the complexity of a single tree. By calculating the cumulative gain of features during tree node splitting, it outputs the normalized feature importance and selects features with importance greater than 0.001 for subsequent analysis.

3. The genomic variation filtering method based on deep learning according to claim 1, characterized in that: The correlation matrix is ​​constructed using the Spearman correlation coefficient to create a nonlinear correlation matrix between features, where R... x R y The rank of the eigenvalues ​​is given by the formula: Where Cov represents the rank covariance, σ is the standard deviation of the rank, and ρ is the Spearman rank correlation coefficient. The subsequent distance transformation converts the correlation matrix into a distance matrix to eliminate the directional differences between positive and negative correlations. The formula is as follows: Where Distance represents the elimination of positive and negative correlation differences by absolute value, with the result ranging from [0, 1]. The hierarchical clustering uses the average linking method to generate a clustering tree, and extracts the leaf node order to rearrange the feature columns, as shown in the following formula: Where |A| and |B| are the number of samples in clusters A and B, d(a,b) is the distance between sample a and sample b, and D(A,B) represents the distance between clusters A and B.

4. The genomic variation filtering method based on deep learning according to claim 1, characterized in that: When constructing the grayscale image, each pixel represents a feature of a row of data. The one-dimensional feature vector of each sample is sequentially filled into a 24×24 pixel matrix to form a 1×40×40 tensor structure. The numerical range is mapped to the [-2.5, 2.5] interval after Z-score normalization. Finally, the linear conversion from numerical value to grayscale is achieved through grayscale mapping, so that the one-dimensional features of the original table are transformed into a two-dimensional image representation with spatial locality.

5. A deep learning-based genomic variation filtering system capable of implementing the method of any one of claims 1-4, characterized in that, include: The candidate feature building module is used to apply machine learning models to predict missing values ​​in variant identification files. Then, based on the original features of the genomic data, five different feature engineering methods are used to generate candidate features. The valuable feature building module is used to select valuable features based on the importance scores of the features from the input raw features and candidate features using the LightGBM model. The candidate feature selection and ranking module is used to obtain candidate features for selection and ranking by calculating the correlation matrix between features and combining hierarchical clustering to group and rank the features. The convolutional neural network module is used to input a grayscale image constructed from the original features and the selected and sorted candidate features into the convolutional neural network, and output the true and false positive categories of gene mutations.

6. The deep learning-based genome variation filtering system according to claim 5, characterized in that: The original features include base quality rank sum, read position rank sum, depth, chain deviation Fisher test value, alignment quality, alignment quality rank sum, quality-depth ratio, chain ratio, quality value, and genotype quality; The feature engineering module in the candidate feature construction module is used to perform mathematical transformations, binning operations, interactive feature generation, clustering feature generation, and nonlinear transformations.

7. The deep learning-based genome variation filtering system according to claim 6, characterized in that: The clustering feature generation uses the K-means clustering algorithm and includes the following steps: First, three centroids are randomly initialized as cluster centers. The distance from each data point to a centroid is calculated, and the data point is assigned to the nearest cluster. Then, the centroids of each cluster are recalculated, and the assignment and update process is repeated until the centroids stabilize or the maximum number of iterations is reached. The core mathematical formula revolves around its objective function, which is to minimize the intra-cluster squared error and SSE, expressed as: Where K is the number of clusters, C k This represents the data set of the k-th cluster. It is the centroid of the cluster, x i Here, ||⋅|| represents the data point, and ||⋅|| represents the Euclidean distance. The algorithm optimizes SSE through two iterative steps: The first allocation step assigns each data point to the cluster of the nearest centroid, i.e. The second update step: Recalculate the mean of the points within each cluster, which is the centroid of the cluster. Iterate until the centroid stops changing or the maximum number of iterations is reached.

8. The deep learning-based genome variation filtering system according to claim 5, characterized in that: The correlation matrix is ​​constructed using the Spearman correlation coefficient to create a nonlinear correlation matrix between features, where R... x R y The rank of the eigenvalues ​​is given by the formula: Where Cov represents the rank covariance, σ is the standard deviation of the rank, and ρ is the Spearman rank correlation coefficient. The subsequent distance transformation converts the correlation matrix into a distance matrix to eliminate the directional differences between positive and negative correlations. The formula is as follows: Where Distance represents the elimination of positive and negative correlation differences by absolute value, with the result ranging from [0, 1]. The hierarchical clustering uses the average linking method to generate a clustering tree, and extracts the leaf node order to rearrange the feature columns, as shown in the following formula: Where |A| and |B| are the number of samples in clusters A and B, d(a,b) is the distance between sample a and sample b, and D(A,B) represents the distance between clusters A and B.

Images