A genetic testing data cleaning method based on artificial intelligence

Abstract

The present invention relates to the field of bioinformatics data processing technology, and in particular to a gene detection data cleaning method based on artificial intelligence. The method comprises the following steps: obtaining an original gene detection data set; performing detection quality feature recognition on the original gene detection data set to obtain detection quality feature data; performing quality screening on the detection quality feature data to obtain detection quality screening data; performing denoising on the original gene detection data set according to the detection quality screening data to obtain a preliminary gene detection data set; performing high-throughput sequencing analysis on the preliminary gene detection data set to generate high-throughput sequencing data. Therefore, the present invention uses data analysis technology, artificial intelligence technology and adaptive correction technology to achieve sequencing error correction, adaptive batch effect correction and intelligent elimination of redundant data for gene detection data, thereby making the cleaning effect of gene detection data better.

Description

Technical Field

[0001] The present invention relates to the technical field of biological information data processing, and in particular to a gene detection data cleaning method based on artificial intelligence. Background Art

[0002] Early genetic testing technology relied on manual processing and analysis of data, which was inefficient and prone to errors. With the development of high-throughput sequencing technology, the amount of genetic testing data has surged, and traditional data processing methods can no longer meet the demand. The introduction of artificial intelligence technology, especially the application of machine learning and deep learning algorithms, has improved the efficiency and accuracy of genetic testing data cleaning to a certain extent. The application of artificial intelligence technology in genetic testing data cleaning mainly includes data preprocessing, variation detection, genotype inference, etc. Through artificial intelligence technology, useful information can be identified from a large amount of complex genomic data, thereby eliminating invalid and erroneous data. Therefore, in the development of genetic testing technology, data cleaning methods based on artificial intelligence have played a key role. However, in terms of genetic testing data cleaning, the existing technology fails to combine high-throughput sequencing data to correct sequencing errors in genetic testing data, and lacks intelligent elimination of redundant data and adaptive batch effect correction for genetic testing data, resulting in poor genetic testing data cleaning effects, which ultimately affects the accuracy of genetic testing. Summary of the invention

[0003] Based on this, it is necessary to provide an artificial intelligence-based genetic testing data cleaning method to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a genetic testing data cleaning method based on artificial intelligence is provided, the method comprising the following steps:

[0005] Step S1: obtaining an original gene detection data set; performing detection quality feature recognition on the original gene detection data set to obtain detection quality feature data; performing quality screening on the detection quality feature data to obtain detection quality screening data; performing denoising processing on the original gene detection data set according to the detection quality screening data to obtain a preliminary gene detection data set;

[0006] Step S2: performing high-throughput sequencing analysis on the preliminary gene detection data set to generate high-throughput sequencing data; performing sequencing error correction on the high-throughput sequencing data to obtain sequencing error correction data; performing automatic coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data;

[0007] Step S3: performing genotype pattern recognition on the balanced coverage detection data to obtain genotype data; performing genotype clustering on the preliminary gene detection data set according to the genotype data to obtain gene detection cluster data; performing intelligent redundant data elimination on the gene detection cluster data to obtain gene detection redundant elimination data; performing adaptive batch effect correction on the gene detection redundant elimination data to obtain batch effect correction data;

[0008] Step S4: performing detection variant site evaluation on the preliminary genetic testing data set according to the batch effect correction data to obtain variant site evaluation data; performing multimodal data cleaning on the preliminary genetic testing data set according to the variant site evaluation data to obtain genetic testing cleaned data; verifying the cleaning results of the genetic testing cleaned data to obtain a genetic testing data cleaning report.

[0009] The present invention can provide data support for subsequent analysis by acquiring the original gene detection data set; the original gene detection data set is subjected to detection quality feature recognition, and outliers and unreliable data points in the data can be identified to ensure the accuracy of subsequent analysis; the detection quality feature data is subjected to quality screening to clarify the relevant data conditions that affect the analysis results, thereby obtaining detection quality screening data, and the original gene detection data set can be subsequently subjected to denoising processing to finally obtain a preliminary gene detection data set. The preliminary gene detection data set is subjected to high-throughput sequencing analysis, which provides an information basis for subsequent gene sequencing situation analysis; sequencing error correction is performed on high-throughput sequencing data, and errors generated during the sequencing process can be corrected to obtain sequencing error correction data; the sequencing error correction data is subjected to automatic coverage homogenization processing to ensure the consistency and comparability of the data, so that the comparison between different sample data is more reasonable and accurate, thereby making the data more evenly covered. Genotype pattern recognition of balanced coverage test data can distinguish different gene types and understand the functions of different gene test data; Genotype clustering of preliminary gene test data sets based on gene type data can cluster similar gene test data together, which helps to discover the association between genes; Intelligent redundant data removal of gene test cluster data can further optimize gene test data, remove unnecessary repeated information, and improve data quality and analysis efficiency; Adaptive batch effect correction of gene test redundant removal data ensures that data between different experimental batches are comparable, which is particularly important for cross-experimental gene test data analysis. According to batch effect correction data, the preliminary gene test data set is evaluated for detection variant sites, which can accurately evaluate variant sites in the preliminary gene test data set and understand data anomalies caused by gene test data variation; Multimodal data cleaning further purifies data, removes potential data contamination and interference, and improves the purity of gene test data; The cleaning results of gene test cleaning data are verified to ensure the effectiveness of the data cleaning process. The final gene test data cleaning report records the relevant situation of gene test data cleaning and ensures the effectiveness of gene test data cleaning. Therefore, the present invention uses data analysis technology, artificial intelligence technology and adaptive correction technology to achieve sequencing error correction, adaptive batch effect correction and intelligent elimination of redundant data for genetic testing data, thereby achieving better cleaning effect of genetic testing data.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: Obtaining the original gene detection data set;

[0012] Step S12: extracting the detection status of the original gene detection data set to generate gene detection status data; performing detection quality feature recognition on the gene detection status data to obtain detection quality feature data; performing quality status analysis on the detection quality feature data to obtain detection quality status data;

[0013] Step S13: setting a quality compliance threshold for the detection quality status data to obtain quality compliance threshold data; comparing the detection quality status data with the quality compliance threshold data, if the detection quality status data is greater than the quality compliance threshold data, marking the detection quality status data as detection quality compliance data; if the detection quality status data is less than or equal to the quality compliance threshold data, marking the detection quality status data as detection quality abnormality data; integrating the detection quality compliance data and the detection quality abnormality data to obtain detection quality screening data;

[0014] Step S14: performing outlier identification on the original gene detection data set according to the detection quality screening data to obtain gene detection quality outliers; performing nonlinear denoising on the gene detection quality outliers to obtain quality nonlinear denoised data; performing data stability enhancement on the original gene detection data set according to the quality nonlinear denoised data to obtain a preliminary gene detection data set.

[0015] The present invention obtains the original gene detection data set, thereby ensuring that subsequent analysis can have comprehensive gene detection data; by extracting gene detection situation data and performing detection quality feature recognition, outliers and unreliable points in the data can be marked, thereby improving the accuracy of the analysis; quality status analysis is performed on the detection quality feature data to provide a basis for subsequent quality compliance judgment; a quality compliance threshold is determined, and the detection quality status data is compared with the detection quality status data, so as to clearly distinguish between detection quality compliance data and detection quality abnormal data, thereby improving the objectivity and accuracy of data screening; the detection quality compliance data and detection quality abnormal data are integrated to obtain detection quality screening data, thereby providing accurate data for subsequent outlier recognition and denoising processing; by identifying gene detection quality abnormal values ​​and performing nonlinear denoising processing, noise in the data is effectively eliminated, the signal-to-noise ratio and mean square error of the data are improved, and the stability and reliability of the data are enhanced; data stability is enhanced for the original gene detection data set according to the nonlinear denoising data, thereby obtaining a preliminary gene detection data set, thereby providing a more accurate and stable data basis for subsequent analysis.

[0016] Preferably, step S2 comprises the following steps:

[0017] Step S21: determining the gene expression status of the preliminary gene detection data set to obtain gene expression data; performing gene activity analysis on the gene expression data to generate detection gene activity data;

[0018] Step S22: performing gene group identification on the preliminary gene detection data set according to the detection gene activity data to obtain detection gene group data; performing inter-group interaction analysis on the detection gene group data to obtain detection gene network relationship data; performing high-throughput sequencing analysis on the preliminary gene detection data set according to the detection gene network relationship data to generate high-throughput sequencing data;

[0019] Step S23: extracting sequencing status from the high-throughput sequencing data to obtain gene sequencing status data; identifying sequencing error status from the gene sequencing status data to generate sequencing error status data; performing error type analysis on the sequencing error status data to obtain sequencing error type data; performing error type distribution detection on the high-throughput sequencing data according to the sequencing error type data to obtain sequencing error distribution data;

[0020] Step S24: performing error sequencing positioning on the high-throughput sequencing data according to the sequencing error distribution data to obtain error sequencing positioning point data; performing error correction on the high-throughput sequencing data according to the error sequencing positioning point data to obtain sequencing error correction data;

[0021] Step S25: Automatically perform coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data.

[0022] The present invention determines the gene expression status of the preliminary gene detection data set, which can reveal the activity state of the gene under different conditions; performs gene activity analysis on the gene expression data, which is helpful to identify the functional role of the key gene; performs gene group identification based on the detected gene activity data, which is helpful to find the gene detection group with similar expression pattern; performs group interaction analysis on the detected gene group data, further reveals the relationship between the gene detection data; extracts the sequencing status of the high-throughput sequencing data to obtain the gene sequencing status data, which provides a basis for subsequent error identification; sequencing error status identification and error type analysis are helpful to identify the sequencing data error status occurring in the sequencing process; performs error sequencing location according to the sequencing error distribution data, which is helpful to accurately identify the location of the sequencing error; performs error correction on the high-throughput sequencing data according to the error sequencing location point data, thereby improving the accuracy of the sequencing data; performs automatic coverage uniformization processing on the sequencing error correction data, ensures the consistency and comparability of the sequencing data, and provides a high-quality data basis for the subsequent gene detection data cleaning.

[0023] Preferably, step S25 comprises the following steps:

[0024] Step S251: performing adaptive normalization processing on the sequencing error correction data to obtain standardized correction data; performing multi-scale coverage analysis on the standardized correction data to obtain coverage distribution data;

[0025] Step S252: locating abnormal regions of the coverage distribution data to obtain abnormal region data; dynamically adjusting abnormal region balance of the preliminary gene detection data set according to the abnormal region data to obtain region balance adjustment data;

[0026] Step S253: performing iterative optimization processing on the sequencing error correction data according to the regional balance adjustment data to obtain correction iterative optimization data; performing coverage optimization processing on the preliminary gene detection data set according to the correction iterative optimization data to generate correction coverage optimization data;

[0027] Step S254: Perform multi-dimensional balance verification on the correction coverage optimization data to obtain balanced verification data; automatically equalize the coverage of the sequencing error correction data based on the balanced verification data to obtain balanced coverage detection data.

[0028] The present invention performs adaptive normalization processing on sequencing error correction data, specifically by adjusting the scale of the data, so that the data between different samples are comparable, which can provide a unified data basis for subsequent analysis; multi-scale coverage analysis is performed on the standardized correction data, which is helpful to identify abnormal patterns and potential sequencing problems in the data; abnormal region positioning is performed on the coverage distribution data, which is helpful to identify and locate abnormal regions that affect the gene detection results; abnormal region dynamic balance adjustment is performed on the preliminary gene detection data set according to the abnormal region data, which is helpful to optimize the overall quality of the data set and reduce the influence of abnormal values ​​on the gene detection results; the sequencing error correction data is iteratively optimized according to the regional balance adjustment data to further improve the accuracy and reliability of the data; the coverage optimization processing of the preliminary gene detection data set is performed according to the correction iterative optimization data, which is helpful to improve the coverage uniformity of the data set and provide a more stable data basis for subsequent gene detection cleaning; multi-dimensional balance verification is performed on the correction coverage optimization data to ensure that the data has reached a balanced state in multiple dimensions, thereby improving the quality of the data and the accuracy of the analysis; the coverage of the sequencing error correction data is automatically uniformized according to the balanced verification data to ensure the reliability and validity of the gene detection data.

[0029] Preferably, step S3 comprises the following steps:

[0030] Step S31: performing gene expression pattern mining on the balanced coverage detection data to obtain gene expression pattern data; performing pattern similarity analysis on the gene expression pattern data to obtain pattern similarity data;

[0031] Step S32: performing pattern stability analysis on the gene expression pattern data according to the pattern similarity data to obtain expression pattern stability data; performing gene type identification on the preliminary gene detection data set according to the expression pattern stability data to obtain gene type data; performing contribution feature type selection on the gene type data to generate contribution feature type data;

[0032] Step S33: performing gene type association analysis on the gene type data according to the contribution characteristic type data to obtain gene type association data; performing cluster density verification on the gene type association data to obtain cluster density verification data; performing abnormal cluster density removal on the gene type association data according to the cluster density verification data to generate abnormal density removal data; performing genotype clustering on the preliminary gene detection data set based on the abnormal density removal data and the gene type association data to obtain gene detection cluster data;

[0033] Step S34: performing intelligent redundant data elimination on the gene detection clustering data to obtain gene detection redundant elimination data;

[0034] Step S35: Adaptively correct the batch effect of the gene detection redundant elimination data to obtain batch effect corrected data.

[0035] The present invention mines gene expression patterns on balanced coverage detection data, which helps to reveal the activity status of genes under different conditions; performs pattern similarity analysis on gene expression pattern data to further clarify the similarity between gene expression patterns and provide important information for the subsequent cleaning of gene detection data; performs pattern stability analysis based on pattern similarity data to help identify gene patterns that show stability under different conditions; through gene type identification and contribution feature type selection, the feature type that contributes the most to gene function can be screened out; performs gene type association analysis based on contribution feature type data to understand the relationship between different gene types; cluster density verification and abnormal cluster density removal ensure the accuracy and reliability of genotype cluster analysis; performs intelligent redundant data elimination on gene detection cluster data, specifically by eliminating unnecessary repeated information, thereby improving the quality of detection data; performs adaptive batch effect correction on gene detection redundant elimination data, specifically by correcting data differences between different experimental batches, thereby ensuring data consistency and comparability.

[0036] Preferably, step S34 includes the following steps:

[0037] Step S341: performing deep data dimensionality reduction on the gene detection clustering data to obtain gene detection dimensionality reduction data; performing deep feature mapping on the gene detection dimensionality reduction data to obtain gene detection feature mapping data;

[0038] Step S342: performing sparsity analysis on the gene detection feature mapping data to obtain detection sparsity data; performing sparsity enhancement processing on the detection sparsity data to obtain sparsity enhanced data;

[0039] Step S343: performing redundant feature identification on the gene detection cluster data according to the sparsity enhancement data to obtain cluster redundant feature data; performing redundant weight analysis on the cluster redundant feature data to generate cluster redundant weight data to obtain weight decay trend data;

[0040] Step S344: dynamically adjust the weight of the weight attenuation trend data to generate dynamic weight adjustment data; intelligently remove redundant data from the gene detection clustering data based on the dynamic weight adjustment data to obtain gene detection redundant removal data.

[0041] The present invention performs deep data dimensionality reduction on gene detection cluster data, specifically by reducing the number of variables in the data set, thereby improving the efficiency of data processing and reducing the consumption of computing resources; deep feature mapping further extracts key features in the dimensionality reduction data, and enhances the ability to understand and interpret gene expression patterns; sparsity analysis is performed on gene detection feature mapping data, which is helpful to identify and process missing values ​​or abnormal values ​​in the data; sparsity enhancement processing is performed on the detection sparsity data, and the structure of the gene detection data is further optimized, thereby obtaining sparsity enhanced data, and improving the stability and robustness of the data; redundant feature identification is performed on the gene detection cluster data according to the sparsity enhanced data, which is helpful to remove redundant features that do not increase the amount of information; redundant weight analysis is performed on the cluster redundant feature data, and the importance of the features is further refined; weight decay trend detection is performed on the cluster redundant weight data, and the weight decay trend of the redundant weight data is clarified; dynamic weight adjustment is performed on the weight decay trend data, and the representation of the gene detection cluster data is optimized; intelligent redundant data elimination is performed to obtain gene detection redundant elimination data, which can improve the quality accuracy of the gene detection data.

[0042] Preferably, step S35 includes the following steps:

[0043] Step S351: performing detection batch feature recognition on the preliminary gene detection data set to obtain detection batch feature data; performing effect feature extraction on the detection batch feature data to generate batch effect feature data;

[0044] Step S352: constructing a batch effect association map for the gene detection redundant elimination data according to the batch effect characteristic data to generate a batch effect association map; performing effect component decomposition on the batch effect association map to obtain effect component decomposition data;

[0045] Step S353: performing intelligent batch effect suppression on the effect component decomposition data to obtain batch effect suppression data; performing effect suppression verification on the preliminary gene detection data set according to the batch effect suppression data to obtain effect suppression verification data;

[0046] Step S354: performing batch effect allele group analysis on the effect suppression verification data to obtain effect allele group data; performing adaptive batch effect correction on the preliminary gene detection data set based on the effect allele group data to obtain batch effect correction data.

[0047] The present invention helps to distinguish the differences between different batches of test data through batch feature identification; effect feature extraction generates batch effect feature data, which further clarifies the difference features between batches and provides important information for subsequent batch effect analysis; constructs a batch effect association map based on the batch effect feature data, which helps to visualize and analyze the correlation between batches; effect component decomposition obtains effect component decomposition data, which helps to identify and separate batch effect components that affect data quality; intelligent batch effect suppression is performed on the effect component decomposition data, which effectively reduces the impact of batch effects on genetic testing data; effect suppression verification is performed on a preliminary genetic testing data set based on the batch effect suppression data, ensuring the effectiveness of batch correction; batch effect allele group analysis is performed on the effect suppression verification data, providing data support for understanding the impact of batch effects at the genomic level; adaptive batch effect correction is performed on the preliminary genetic testing data set based on the effect allele group data, the correction process is optimized by an adaptive method, and the consistency of genetic testing data is further improved.

[0048] Preferably, step S4 comprises the following steps:

[0049] Step S41: identifying detection variant sites on the preliminary gene detection data set according to the batch effect correction data to generate detection variant site data; performing variant site stability analysis on the detection variant site data to obtain site stability data;

[0050] Step S42: performing variant site confidence calculation on the site stability data to obtain site confidence calculation data; performing variant site association analysis on the preliminary gene detection data set according to the site confidence calculation data to obtain variant site association data; performing variant site effect evaluation on the variant site association data to obtain site effect evaluation data;

[0051] Step S43: integrating the site stability data, the site confidence calculation data and the site effect evaluation data to obtain the variant site evaluation data;

[0052] Step S44: performing multimodal data cleaning on the preliminary gene detection data set according to the variant site evaluation data to obtain gene detection cleaned data;

[0053] Step S45: verify the cleaning results of the gene detection cleansing data to obtain a gene detection data cleaning report.

[0054] The present invention can reduce the variation of data between different experimental batches and improve the consistency and reliability of gene detection data through batch effect correction; the identified detection variant site data provides accurate gene variation information for subsequent analysis; stability analysis ensures the reliability of variant sites and provides stable basic data for subsequent research; site confidence calculation enhances the accuracy of variant site data and provides high-confidence variant site information for association analysis; variant site association analysis reveals the potential connection between variant sites and gene detection data; site effect evaluation further quantifies the importance of variant sites; integrating site stability, confidence and effect evaluation data to form variant site evaluation data can improve the comprehensiveness and depth of variant site analysis and provide richer and more accurate data support for subsequent multimodal data cleaning; performing multimodal data cleaning on the preliminary gene detection data set according to the variant site evaluation data, effectively removing noise and outliers in the gene detection data set, and improving the quality and credibility of the data; verifying the cleaning results of the gene detection cleaning data to ensure the effectiveness and correctness of the gene detection data cleaning; and finally generating a gene detection data cleaning report to record the cleaning process and results in detail.

[0055] Preferably, step S44 includes the following steps:

[0056] Step S441: dividing the preliminary gene detection data set into data structure hierarchies according to the variant site assessment data to obtain gene detection structure hierarchy parameters; reconstructing the preliminary gene detection data set according to the gene detection structure hierarchy parameters to obtain gene detection reconstructed data;

[0057] Step S442: performing reconstruction feature mapping enhancement on the gene detection reconstructed data to obtain reconstructed feature enhanced data; performing multimodal type determination on the reconstructed feature enhanced data to generate multimodal type data; performing multimodal data synchronization on the reconstructed feature enhanced data according to the multimodal type data to generate a multimodal data synchronization result;

[0058] Step S443: performing intelligent modality conversion on the multimodal data synchronization result according to the multimodal type data to obtain intelligent modality conversion data; performing dynamic modality fusion on the gene detection reconstruction data according to the intelligent modality conversion data to obtain dynamic modality fusion data;

[0059] Step S444: performing adaptive modal optimization on the dynamic modal fusion data to obtain adaptive modal optimization data; performing dynamic modal data cleaning on the preliminary gene detection data set according to the adaptive modal optimization data to obtain gene detection cleaning data.

[0060] The present invention divides the data structure of the preliminary gene detection data set into hierarchical levels according to the variant site assessment data, providing a clear organizational structure for subsequent data reconstruction; the detection data reconstruction further optimizes the data structure, making the gene detection reconstructed data more in line with the analysis requirements and improving the efficiency and accuracy of data cleaning; the reconstruction feature mapping enhancement significantly improves the characterization ability of the gene detection reconstructed data, making the useful information in the data more prominent; multimodal type determination and data synchronization ensure the consistency and coordination of different types of data, and intelligent modal conversion and dynamic modal fusion further integrate multimodal data, enhancing the comprehensiveness of the data and the depth of analysis; adaptive modal optimization further improves the quality and availability of dynamic modal fusion data, ensuring the efficiency and accuracy of the data in the cleaning process; dynamic modal data cleaning is a key step to ensure the quality of gene detection data, and the cleaning process guided by adaptive modal optimization data removes noise and outliers from the obtained gene detection cleaning data, thereby improving the purity and reliability of the data.

[0061] Preferably, step S45 includes the following steps:

[0062] Step S451: performing data integrity check on the gene detection cleansing data to obtain integrity check data; performing data consistency analysis on the integrity check data to obtain consistency analysis data;

[0063] Step S452: performing a data distribution feature check on the consistency analysis data to obtain distribution feature check data; performing an outlier review process on the distribution feature check data to obtain outlier review data;

[0064] Step S453: performing data quality assessment on the outlier review data to obtain data quality assessment data; performing comprehensive verification of the cleaning results on the data quality assessment data to obtain data cleaning comprehensive verification results;

[0065] Step S454: Generate a cleaning effect report for the comprehensive verification results of data cleaning to obtain a genetic testing data cleaning report.

[0066] The present invention ensures the accuracy and reliability of genetic testing cleansing data through data integrity verification, providing a high-quality data foundation for subsequent analysis; data consistency analysis further ensures the stability and coordination of data in the entire processing flow, avoiding the problem of inconsistent data at different stages; data distribution feature inspection helps to identify and process deviations and anomalies in the data, ensuring the balance and representativeness of the data; outlier review processing purifies genetic testing data, thereby removing extreme values ​​or erroneous data that affect genetic testing; data quality assessment provides quantitative indicators for data cleaning, identifies weaknesses and deficiencies in the data, and thus performs targeted optimization; comprehensive verification of cleaning results ensures the effectiveness of the data cleaning process and improves the transparency and credibility of data cleaning; the generation of cleaning effect reports provides detailed records and evaluations for the entire genetic testing data cleaning process, so that the results of data cleaning can be clearly understood and verified, providing researchers with important data references. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A schematic diagram of the steps of a genetic testing data cleaning method based on artificial intelligence;

[0068] Figure 2 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0069] Figure 3 for Figure 2 Detailed implementation steps of step S44 are shown in the flowchart;

[0070] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0073] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0074] To achieve this, please refer to Figures 1 to 3 , a genetic testing data cleaning method based on artificial intelligence, the method comprising the following steps:

[0075] Step S1: obtaining an original gene detection data set; performing detection quality feature recognition on the original gene detection data set to obtain detection quality feature data; performing quality screening on the detection quality feature data to obtain detection quality screening data; performing denoising processing on the original gene detection data set according to the detection quality screening data to obtain a preliminary gene detection data set;

[0076] Step S2: performing high-throughput sequencing analysis on the preliminary gene detection data set to generate high-throughput sequencing data; performing sequencing error correction on the high-throughput sequencing data to obtain sequencing error correction data; performing automatic coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data;

[0077] Step S3: performing genotype pattern recognition on the balanced coverage detection data to obtain genotype data; performing genotype clustering on the preliminary gene detection data set according to the genotype data to obtain gene detection cluster data; performing intelligent redundant data elimination on the gene detection cluster data to obtain gene detection redundant elimination data; performing adaptive batch effect correction on the gene detection redundant elimination data to obtain batch effect correction data;

[0078] Step S4: performing detection variant site evaluation on the preliminary genetic testing data set according to the batch effect correction data to obtain variant site evaluation data; performing multimodal data cleaning on the preliminary genetic testing data set according to the variant site evaluation data to obtain genetic testing cleaned data; verifying the cleaning results of the genetic testing cleaned data to obtain a genetic testing data cleaning report.

[0079] The present invention can provide data support for subsequent analysis by acquiring the original gene detection data set; the original gene detection data set is subjected to detection quality feature recognition, and outliers and unreliable data points in the data can be identified to ensure the accuracy of subsequent analysis; the detection quality feature data is subjected to quality screening to clarify the relevant data conditions that affect the analysis results, thereby obtaining detection quality screening data, and the original gene detection data set can be subsequently subjected to denoising processing to finally obtain a preliminary gene detection data set. The preliminary gene detection data set is subjected to high-throughput sequencing analysis, which provides an information basis for subsequent gene sequencing situation analysis; sequencing error correction is performed on high-throughput sequencing data, and errors generated during the sequencing process can be corrected to obtain sequencing error correction data; the sequencing error correction data is subjected to automatic coverage homogenization processing to ensure the consistency and comparability of the data, so that the comparison between different sample data is more reasonable and accurate, thereby making the data more evenly covered. Genotype pattern recognition of balanced coverage test data can distinguish different gene types and understand the functions of different gene test data; Genotype clustering of preliminary gene test data sets based on gene type data can cluster similar gene test data together, which helps to discover the association between genes; Intelligent redundant data removal of gene test cluster data can further optimize gene test data, remove unnecessary repeated information, and improve data quality and analysis efficiency; Adaptive batch effect correction of gene test redundant removal data ensures that data between different experimental batches are comparable, which is particularly important for cross-experimental gene test data analysis. According to batch effect correction data, the preliminary gene test data set is evaluated for detection variant sites, which can accurately evaluate variant sites in the preliminary gene test data set and understand data anomalies caused by gene test data variation; Multimodal data cleaning further purifies data, removes potential data contamination and interference, and improves the purity of gene test data; The cleaning results of gene test cleaning data are verified to ensure the effectiveness of the data cleaning process. The final gene test data cleaning report records the relevant situation of gene test data cleaning and ensures the effectiveness of gene test data cleaning. Therefore, the present invention uses data analysis technology, artificial intelligence technology and adaptive correction technology to achieve sequencing error correction, adaptive batch effect correction and intelligent elimination of redundant data for genetic testing data, thereby achieving better cleaning effect of genetic testing data.

[0080] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of a gene detection data cleaning method based on artificial intelligence of the present invention. In this example, the gene detection data cleaning method based on artificial intelligence includes the following steps:

[0081] Step S1: obtaining an original gene detection data set; performing detection quality feature recognition on the original gene detection data set to obtain detection quality feature data; performing quality screening on the detection quality feature data to obtain detection quality screening data; performing denoising processing on the original gene detection data set according to the detection quality screening data to obtain a preliminary gene detection data set;

[0082] In an embodiment of the present invention, first, it is necessary to obtain an original gene detection data set from a sequencing platform. The gene detection data is saved in a FASTQ format, which includes various data of gene detection. The FastQC tool is used to identify the detection quality characteristics of the original data set. FastQC is a program for generating a sequencing data quality report, which can identify multiple quality characteristics in the data, such as base quality distribution, sequence content, base pairing, over-expressed sequences, and contamination of the sequencing platform. According to the quality report generated by FastQC, a threshold is set to filter the data; for example, bases with a quality score lower than the threshold are removed, or sequences with too high N content (expressed as unclear or unknown nucleotides, usually referred to as "unknown bases") are deleted; further, regions with abnormal GC content (referring to the ratio of guanine and cytosine in the genome) are screened out, and adapter sequences and low-quality reads (sequencing reads) are removed, so as to obtain detection quality feature data; the Trimmomatic tool is used to denoise the original data. Trimmomatic can cut off the low-quality end of the sequence according to the specified quality threshold, or remove reads (sequencing reads) containing adapter sequences, and finally obtain a preliminary gene detection data set.

[0083] Step S2: performing high-throughput sequencing analysis on the preliminary gene detection data set to generate high-throughput sequencing data; performing sequencing error correction on the high-throughput sequencing data to obtain sequencing error correction data; performing automatic coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data;

[0084] In an embodiment of the present invention, BWA software is used, specifically BWA is a software for aligning DNA sequences to reference genomes, high-throughput sequencing analysis is performed on the preliminary gene detection data set to generate high-throughput sequencing data; GATK is used, wherein GATK is a toolkit for analyzing and interpreting high-throughput sequencing data; sequencing errors are corrected for high-throughput sequencing data; the BaseRecalibrator and PrintReads tools of GATK are used to correct the quality score, specifically the quality score of each base is adjusted to correct errors that occur during the sequencing process; the DepthOfCoverage tool of GATK is used to evaluate the coverage, and the coverage is adjusted through a custom script, for example, for regions with higher coverage, the number of reads in the region is reduced by random sampling; for regions with lower coverage, reads are selected from regions with higher coverage for supplementation, or reads are simulated to increase coverage; thereby making the coverage in the data set more uniform; further, the genomecov command of BEDTools is used to generate a coverage file to achieve automatic uniformity of coverage.

[0085] Step S3: performing genotype pattern recognition on the balanced coverage detection data to obtain genotype data; performing genotype clustering on the preliminary gene detection data set according to the genotype data to obtain gene detection cluster data; performing intelligent redundant data elimination on the gene detection cluster data to obtain gene detection redundant elimination data; performing adaptive batch effect correction on the gene detection redundant elimination data to obtain batch effect correction data;

[0086] In an embodiment of the present invention, a pattern recognition algorithm, such as a support vector machine (SVM), is used to perform gene type pattern recognition on a preliminary gene detection data set; specifically, through a training model, the SVM assigns different genes to corresponding gene types according to gene expression data; based on the identified gene type data, a clustering algorithm, such as K-means or hierarchical clustering, is used to perform cluster analysis on the genes; specifically, the genes in the gene detection data set are grouped according to similarity to obtain gene detection cluster data; data deduplication technology, such as a data hashing method, is used to remove redundant data from the gene detection cluster data; by calculating and comparing the hash value of the data, duplicate genotype data is identified and deleted to obtain redundant gene detection data; a batch effect correction algorithm, such as ComBat or deepMNN, is used to correct the redundant gene detection data; and by considering batch information, the batch effect data in the data is adjusted to obtain batch effect corrected data.

[0087] Step S4: performing detection variant site evaluation on the preliminary genetic testing data set according to the batch effect correction data to obtain variant site evaluation data; performing multimodal data cleaning on the preliminary genetic testing data set according to the variant site evaluation data to obtain genetic testing cleaned data; verifying the cleaning results of the genetic testing cleaned data to obtain a genetic testing data cleaning report.

[0088] In an embodiment of the present invention, a variation detection tool, such as HaplotypeCaller of GATK, is used to analyze batch effect correction data to identify variation sites in the genome; wherein HaplotypeCaller is a tool for calling natural variations of a single sample (including SNPs, insertions and deletions, etc.), and specifically uses local alignment and complex comparison strategies to identify variation sites; based on the variation site evaluation data, genotype clustering is performed using software such as CD-HIT, and redundant sequences are removed; specifically, highly similar sequences are quickly identified and merged using CD-HIT software; sequence quality control tools, such as FastQC, are used to verify the cleaned data for genetic testing; the quality of the cleaned data set is specifically evaluated, including sequencing quality, over-expressed sequences, and contamination of the sequencing platform, to ensure the accuracy and reliability of the data set.

[0089] Preferably, step S1 comprises the following steps:

[0090] Step S11: Obtaining the original gene detection data set;

[0091] Step S12: extracting the detection status of the original gene detection data set to generate gene detection status data; performing detection quality feature recognition on the gene detection status data to obtain detection quality feature data; performing quality status analysis on the detection quality feature data to obtain detection quality status data;

[0092] Step S13: setting a quality compliance threshold for the detection quality status data to obtain quality compliance threshold data; comparing the detection quality status data with the quality compliance threshold data, if the detection quality status data is greater than the quality compliance threshold data, marking the detection quality status data as detection quality compliance data; if the detection quality status data is less than or equal to the quality compliance threshold data, marking the detection quality status data as detection quality abnormality data; integrating the detection quality compliance data and the detection quality abnormality data to obtain detection quality screening data;

[0093] Step S14: performing outlier identification on the original gene detection data set according to the detection quality screening data to obtain gene detection quality outliers; performing nonlinear denoising on the gene detection quality outliers to obtain quality nonlinear denoised data; performing data stability enhancement on the original gene detection data set according to the quality nonlinear denoised data to obtain a preliminary gene detection data set.

[0094] In an embodiment of the present invention, an original gene detection data set is obtained from a sequencing platform and saved in a FASTQ format, which includes various data of gene detection; the FastQC tool is used to perform a quality assessment on the original data set, specifically detecting errors and impurities in the sequence data, and generating gene detection status data; the gene detection status data is analyzed by FastQC to identify detection quality feature data, specifically including key parameters such as base quality score, sequence N content, and GC content; the detection quality feature data is analyzed, and a quality status, such as qualified or unqualified, is calculated based on the quality feature data through a customized script, thereby obtaining detection quality status data; a quality compliance threshold is set for the detection quality status data, for example, a base quality threshold: setting an average quality score (Phred-score) greater than Q30, indicating that at least 99.9% of the bases are accurate; a sequence N content threshold: setting the N content to no more than 5% to ensure the reliability of the sequence information; a GC content threshold: setting the N content to no more than 5% to ensure the reliability of the sequence information; and a GC content threshold. Threshold: Set the GC content between 30% and 70% to avoid analytical deviation due to abnormal GC content; compare the detection quality status data with the quality compliance threshold data. If the detection quality status data is greater than the quality compliance threshold data, the detection quality status data will be marked as detection quality compliance data; if the detection quality status data is less than or equal to the quality compliance threshold data, the detection quality status data will be marked as detection quality abnormal data; use data integration technology to integrate the detection quality compliance data and the detection quality abnormal data to form a detection quality screening data set; use statistical methods, such as box plots or standard deviations, to identify outliers in the detection quality screening data and obtain gene detection quality abnormal values; perform nonlinear denoising on the gene detection quality abnormal values, specifically use the DenoiseST denoising method to obtain quality nonlinear denoising data; perform data stability enhancement on the original gene detection data set based on the nonlinear denoising data, and finally obtain a preliminary gene detection data set.

[0095] Preferably, step S2 comprises the following steps:

[0096] Step S21: determining the gene expression status of the preliminary gene detection data set to obtain gene expression data; performing gene activity analysis on the gene expression data to generate detection gene activity data;

[0097] Step S22: performing gene group identification on the preliminary gene detection data set according to the detection gene activity data to obtain detection gene group data; performing inter-group interaction analysis on the detection gene group data to obtain detection gene network relationship data; performing high-throughput sequencing analysis on the preliminary gene detection data set according to the detection gene network relationship data to generate high-throughput sequencing data;

[0098] Step S23: extracting sequencing status from the high-throughput sequencing data to obtain gene sequencing status data; identifying sequencing error status from the gene sequencing status data to generate sequencing error status data; performing error type analysis on the sequencing error status data to obtain sequencing error type data; performing error type distribution detection on the high-throughput sequencing data according to the sequencing error type data to obtain sequencing error distribution data;

[0099] Step S24: performing error sequencing positioning on the high-throughput sequencing data according to the sequencing error distribution data to obtain error sequencing positioning point data; performing error correction on the high-throughput sequencing data according to the error sequencing positioning point data to obtain sequencing error correction data;

[0100] Step S25: Automatically perform coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data.

[0101] In the embodiment of the present invention, RNA sequencing technology is used to determine the gene expression status of the preliminary gene detection data set, specifically measuring the gene expression level at the whole genome level, thereby obtaining gene expression data; gene information analysis tools, such as DESeq2 or edgeR, are used to perform gene activity analysis on the gene expression data, and by identifying differentially expressed genes, the changes in gene activity are specifically revealed, thereby generating detection gene activity data; cluster analysis methods, such as hierarchical clustering, are used. Clustering) or K-means algorithm is used to identify gene groups in the preliminary gene detection data set according to the detection gene activity data to obtain the detection gene group data; network analysis tools, such as Cytoscape or STRING database, are used to analyze the interaction between groups in the detection gene group data to obtain the detection gene network relationship data; high-throughput sequencing technology (HTS) is used again to analyze the detection gene network relationship data to generate high-throughput sequencing data; sequence alignment tools, such as BWA or Bowtie2, are used to extract sequencing status from the high-throughput sequencing data to obtain gene sequencing status data; quality control tools, such as FastQC, are used to identify sequencing error status of gene sequencing status data to generate sequencing error status data; FastQC specifically detects quality problems in sequencing data, such as base calling Error; use variation detection tools, such as GATK or SAMtools, to perform error type analysis on sequencing error status data to obtain sequencing error type data; perform error type distribution detection on high-throughput sequencing data based on sequencing error type data to obtain sequencing error distribution data; use GATK's IndelRealigner or BaseRecalibrator tools to locate error sequencing on high-throughput sequencing data based on sequencing error distribution data to obtain error sequencing location point data; use GATK's PrintReads tool to perform error correction on high-throughput sequencing data based on error sequencing location point data to obtain sequencing error correction data; use tools such as GATK's DepthOfCoverage to automatically even out the coverage of sequencing error correction data to obtain balanced coverage detection data.

[0102] Preferably, step S25 comprises the following steps:

[0103] Step S251: performing adaptive normalization processing on the sequencing error correction data to obtain standardized correction data; performing multi-scale coverage analysis on the standardized correction data to obtain coverage distribution data;

[0104] Step S252: locating abnormal regions of the coverage distribution data to obtain abnormal region data; dynamically adjusting abnormal region balance of the preliminary gene detection data set according to the abnormal region data to obtain region balance adjustment data;

[0105] Step S253: performing iterative optimization processing on the sequencing error correction data according to the regional balance adjustment data to obtain correction iterative optimization data; performing coverage optimization processing on the preliminary gene detection data set according to the correction iterative optimization data to generate correction coverage optimization data;

[0106] Step S254: Perform multi-dimensional balance verification on the correction coverage optimization data to obtain balanced verification data; automatically equalize the coverage of the sequencing error correction data based on the balanced verification data to obtain balanced coverage detection data.

[0107] In an embodiment of the present invention, the sequencing error correction data is adaptively normalized to obtain standardized correction data; further, the sequencing error correction data sample is used as a reference to calculate the corresponding scale factor; specifically, by changing the percentage of differential expression and the number of repetitions, a multi-scale coverage analysis is performed on the standardized correction data to obtain coverage distribution data; the coverage distribution data is abnormally located, and the image anomaly detection and location is performed using the SimpleNet network, specifically by applying noise to normal features in the feature space to identify abnormal features, thereby obtaining abnormal region data; according to the abnormal region data, the preliminary gene detection data set is dynamically balanced to obtain a regional balance. Adjust data; iteratively optimize the sequencing error correction data according to the regional balance adjustment data, which is specifically achieved by updating the parameters of the optimizer, for example, using the Adam optimizer to iteratively update the sequencing error correction data to obtain the corrected iterative optimization data; perform coverage optimization on the preliminary gene detection data set according to the corrected iterative optimization data to generate corrected coverage optimization data; perform multi-dimensional balance verification on the corrected coverage optimization data to obtain balanced verification data; automatically uniformize the coverage of the sequencing error correction data according to the uniformized verification data to obtain balanced coverage detection data; specifically evaluate the uniformity of the multi-dimensional point distribution through the nearest neighbor distance method (NND) or the Voronoi diagram method.

[0108] Preferably, step S3 comprises the following steps:

[0109] Step S31: performing gene expression pattern mining on the balanced coverage detection data to obtain gene expression pattern data; performing pattern similarity analysis on the gene expression pattern data to obtain pattern similarity data;

[0110] Step S32: performing pattern stability analysis on the gene expression pattern data according to the pattern similarity data to obtain expression pattern stability data; performing gene type identification on the preliminary gene detection data set according to the expression pattern stability data to obtain gene type data; performing contribution feature type selection on the gene type data to generate contribution feature type data;

[0111] Step S33: performing gene type association analysis on the gene type data according to the contribution characteristic type data to obtain gene type association data; performing cluster density verification on the gene type association data to obtain cluster density verification data; performing abnormal cluster density removal on the gene type association data according to the cluster density verification data to generate abnormal density removal data; performing genotype clustering on the preliminary gene detection data set based on the abnormal density removal data and the gene type association data to obtain gene detection cluster data;

[0112] Step S34: performing intelligent redundant data elimination on the gene detection clustering data to obtain gene detection redundant elimination data;

[0113] Step S35: Adaptively correct the batch effect of the gene detection redundant elimination data to obtain batch effect corrected data.

[0114] In an embodiment of the present invention, the balanced coverage detection data is first analyzed to extract the expression of each gene under different conditions; the Euclidean distance of the expression between genes is calculated using a hierarchical clustering method, and a clustering tree is constructed; a threshold value, such as 0.5, is set to extract the gene expression pattern from the clustering tree to obtain gene expression pattern data; the gene expression pattern data is subjected to pattern stability analysis based on the pattern similarity data to specifically compare the consistency of the gene expression patterns under different conditions; for example, if a group of genes shows similar expression trends in multiple tissue samples, the pattern is marked as stable; further, the stability of the pattern is evaluated by calculating the frequency of occurrence of the pattern in more than 70% of the samples, and the relevant information of the expression pattern is recorded to obtain a table. Expression pattern stability data; identifying gene types based on expression pattern stability data; for example, if a group of genes is continuously highly expressed in a specific tissue, they are classified as "tissue-specifically expressed genes", and the association between the gene expression pattern and the specific biological function is determined, so as to identify the gene type with a specific function, thereby obtaining gene type data; after identifying the gene type, analyzing the association between different gene types; for example, the association between the highly expressed gene type in the gene detection data and the lowly expressed gene type in the gene detection data; specifically, by constructing an interaction network between gene types and analyzing the key nodes and connection patterns in the network, the potential relationship between gene types is revealed, and gene type association data is obtained. In order to ensure that the gene clustering results are meaningful, the cluster density of the gene type association data is verified. Specifically, the average distance of the genes in each cluster is calculated to ensure that the gene expression patterns within the cluster are similar enough; a density threshold is set, such as the average distance is less than 0.25, to verify whether the density of the gene detection data reaches the threshold, so as to obtain the cluster density verification data; during the cluster density verification process, if the similarity between the genes within the cluster is low, it indicates that it contains abnormal data or genes with unstable expression patterns. At this time, by setting a density threshold, such as the average distance is greater than 0.5, clusters below the threshold are excluded, thereby removing abnormal clusters; after the genotype clustering is completed, intelligent redundant data removal is performed; specifically identify and delete genes that appear repeatedly in the cluster, and by comparing the similarity of the gene expression patterns, for example, delete duplicate genes with a similarity greater than 0.95, so as to eliminate redundant gene detection data; batch effect correction is performed on the gene detection data, and statistical methods, such as the ComBat algorithm, are used to identify and correct batch effects to ensure that the gene expression data between different batches are comparable.

[0115] Preferably, step S34 includes the following steps:

[0116] Step S341: performing deep data dimensionality reduction on the gene detection clustering data to obtain gene detection dimensionality reduction data; performing deep feature mapping on the gene detection dimensionality reduction data to obtain gene detection feature mapping data;

[0117] Step S342: performing sparsity analysis on the gene detection feature mapping data to obtain detection sparsity data; performing sparsity enhancement processing on the detection sparsity data to obtain sparsity enhanced data;

[0118] Step S343: performing redundant feature identification on the gene detection cluster data according to the sparsity enhancement data to obtain cluster redundant feature data; performing redundant weight analysis on the cluster redundant feature data to generate cluster redundant weight data; performing weight decay trend detection on the cluster redundant weight data to obtain weight decay trend data;

[0119] Step S344: dynamically adjust the weight of the weight attenuation trend data to generate dynamic weight adjustment data; intelligently remove redundant data from the gene detection clustering data based on the dynamic weight adjustment data to obtain gene detection redundant removal data.

[0120] In an embodiment of the present invention, the UMAP algorithm is used to implement deep data dimensionality reduction processing on gene detection clustering data; the parameters n_neighbors of the UMAP algorithm are specifically set to 15, min_dist is 0.1, and n_components is 2, so as to obtain gene detection dimensionality reduction data; deep feature mapping is performed on the gene detection dimensionality reduction data, specifically further mapped to a two-dimensional space through the UMAP algorithm to obtain gene detection feature mapping data; sparsity analysis is performed on the gene detection feature mapping data, specifically, the detection sparse data is obtained by calculating the proportion of non-zero elements to total elements; sparsity enhancement processing is performed on the detection sparse data, for example, by adding small noise or using sparse coding technology to obtain sparse enhanced data; according to the sparse enhanced data, Redundant features are identified on the genetic detection clustering data to obtain clustering redundant feature data; specifically, features with repeated information are identified by calculating the correlation between features and marked as redundant; redundant weight analysis is performed on the clustering redundant feature data, specifically the contribution of each redundant feature in the data set is calculated to generate clustering redundant weight data; weight decay trend detection is performed on the clustering redundant weight data, specifically by measuring the trend of feature weight changes over time or conditions to obtain weight decay trend data; dynamic weight adjustment is performed on the weight decay trend data, and based on the adaptive weight decay technology, the hyperparameters of weight decay are automatically adjusted according to the loss gradient in the process to generate dynamic weight adjustment data; finally, the genetic detection clustering data is intelligently redundantly eliminated according to the dynamic weight adjustment data to obtain genetic detection redundant elimination data.

[0121] Preferably, step S35 includes the following steps:

[0122] Step S351: performing detection batch feature recognition on the preliminary gene detection data set to obtain detection batch feature data; performing effect feature extraction on the detection batch feature data to generate batch effect feature data;

[0123] Step S352: constructing a batch effect association map for the gene detection redundant elimination data according to the batch effect characteristic data to generate a batch effect association map; performing effect component decomposition on the batch effect association map to obtain effect component decomposition data;

[0124] Step S353: performing intelligent batch effect suppression on the effect component decomposition data to obtain batch effect suppression data; performing effect suppression verification on the preliminary gene detection data set according to the batch effect suppression data to obtain effect suppression verification data;

[0125] Step S354: performing batch effect allele group analysis on the effect suppression verification data to obtain effect allele group data; performing adaptive batch effect correction on the preliminary gene detection data set based on the effect allele group data to obtain batch effect correction data.

[0126] In an embodiment of the present invention, the detection batch feature recognition is performed on the preliminary gene detection data set, specifically, the gene expression data from different batches are collected and integrated, and the features related to the specific batch are identified, and the batch effect analysis technology, such as the ComBat algorithm, is used to identify and quantify the batch effect to obtain the detection batch feature data; the effect feature extraction is performed on the detection batch feature data, specifically by calculating the difference in gene expression between batches, and using statistical methods, such as ANOVA (analysis of variance), to determine the difference in gene expression caused by the batch effect, thereby generating batch effect feature data; the batch effect association map construction technology, such as batch effect network analysis, is used to visualize and quantify the batch effect in the gene expression data, construct a batch effect association map, and generate a batch effect association map; the batch effect association map is decomposed into effect components, specifically using the principal component analysis (PCA) technology to reduce the dimension of the data, and identify the main batch effect components that affect the gene expression data, to obtain the effect component decomposition data; the effect component decomposition data is subjected to intelligent batch effect suppression, specifically using a machine learning algorithm Methods, such as random forests, are used to predict and suppress batch effects; the characteristics of batch effects are identified by training models, and the effect is removed in the prediction stage to obtain batch effect suppression data; based on the batch effect suppression data, the preliminary genetic testing data set is validated for effect suppression, and a cross-validation method is specifically used to evaluate the effectiveness of batch effect suppression to obtain effect suppression verification data; batch effect allele group analysis is performed on the effect suppression verification data; specifically, genomic analysis techniques, such as three-dimensional genomics techniques, are used to analyze the distribution of batch effects in the corrected data, and the residual influence of batch effects in the corrected genome is understood to obtain effect allele group data; based on the effect allele group data, the preliminary genetic testing data set is adaptively corrected for batch effects, and the intensity of batch effect correction is dynamically adjusted by adaptive threshold adjustment, specifically, the intensity of batch effect correction is dynamically adjusted by using an adaptive algorithm based on the effect allele group data; for example, an adaptive threshold adjustment algorithm can be designed that adjusts correction parameters in real time according to the size of the batch effect and the stability of the data to ensure that the correction process can remove the batch effect.

[0127] As an example of the present invention, refer to Figure 2 As shown, in this example, step S4 includes:

[0128] Step S41: identifying detection variant sites on the preliminary gene detection data set according to the batch effect correction data to generate detection variant site data; performing variant site stability analysis on the detection variant site data to obtain site stability data;

[0129] Step S42: performing variant site confidence calculation on the site stability data to obtain site confidence calculation data; performing variant site association analysis on the preliminary gene detection data set according to the site confidence calculation data to obtain variant site association data; performing variant site effect evaluation on the variant site association data to obtain site effect evaluation data;

[0130] Step S43: integrating the site stability data, the site confidence calculation data and the site effect evaluation data to obtain the variant site evaluation data;

[0131] Step S44: performing multimodal data cleaning on the preliminary gene detection data set according to the variant site evaluation data to obtain gene detection cleaned data;

[0132] Step S45: verify the cleaning results of the gene detection cleansing data to obtain a gene detection data cleaning report.

[0133] In an embodiment of the present invention, the ComBat algorithm is used to correct the batch effect of the preliminary genetic testing data set; first, the preliminary genetic testing data set is divided into multiple batches, each batch representing a different experimental condition or time point; then, the ComBat algorithm is used to adjust the data of each batch to eliminate technical differences between batches and ensure the consistency of the data set. The corrected data is used to identify variant sites. Specifically, the sequencing data is analyzed to identify single nucleotide polymorphisms (SNPs) and insertions / deletions (indels) in the genome, thereby generating detection variant site data, including position, reference bases, and variant bases; the identified variant sites are compared with known databases such as dbSNP to screen out variant sites that appear consistently in different samples, and the variant site stability analysis is performed to obtain site stability data; the stability data of the variant sites is calculated with confidence, and the quality of each variant site is scored to distinguish between genetic detection variants and sequencing errors, and the site confidence calculation data is obtained; PLINK software or R's genetics package is used for association analysis, and the association between the variant site and the specific genetic detection data is determined by comparing the preliminary genetic detection data set. Specifically, the logistic regression model is used to evaluate the effect size and statistical significance of each variant site, thereby obtaining variant site association data; SIFT or PolyPhen-2 algorithms are used to evaluate the variant site effect of the variant site association data, and the influence of the specific variant site on the genetic detection data is obtained to obtain site effect evaluation data. The site stability data, site confidence calculation data and site effect assessment data were integrated, and the data processing function of the R language was used to merge the data from different sources and convert the data format to form a complete variant site assessment data set; the FastQC tool was used to perform multimodal data cleaning on the preliminary genetic testing data set based on the variant site assessment data to obtain genetic testing cleaned data; the cleaning results of the genetic testing cleaned data were verified to obtain a genetic testing data cleaning report.

[0134] As an example of the present invention, refer to Figure 3 As shown, in this example, step S44 includes:

[0135] Step S441: dividing the preliminary gene detection data set into data structure hierarchies according to the variant site assessment data to obtain gene detection structure hierarchy parameters; reconstructing the preliminary gene detection data set according to the gene detection structure hierarchy parameters to obtain gene detection reconstructed data;

[0136] Step S442: performing reconstruction feature mapping enhancement on the gene detection reconstructed data to obtain reconstructed feature enhanced data; performing multimodal type determination on the reconstructed feature enhanced data to generate multimodal type data; performing multimodal data synchronization on the reconstructed feature enhanced data according to the multimodal type data to generate a multimodal data synchronization result;

[0137] Step S443: performing intelligent modality conversion on the multimodal data synchronization result according to the multimodal type data to obtain intelligent modality conversion data; performing dynamic modality fusion on the gene detection reconstruction data according to the intelligent modality conversion data to obtain dynamic modality fusion data;

[0138] Step S444: performing adaptive modal optimization on the dynamic modal fusion data to obtain adaptive modal optimization data; performing dynamic modal data cleaning on the preliminary gene detection data set according to the adaptive modal optimization data to obtain gene detection cleaning data.

[0139] In the embodiments of the present invention, bioinformatics tools such as GATK are used to identify and control variant sites to ensure the accuracy of the data; hierarchical clustering algorithms are used to structure the gene detection data set according to the distribution characteristics of the variant sites to form gene detection structure hierarchy parameters; a reconstruction algorithm such as SVIM technology is used to reconstruct the preliminary gene detection data set to obtain gene detection reconstructed data; deep learning technologies such as convolutional neural networks (CNN) are used to extract features and enhance mapping of the reconstructed data to obtain reconstructed feature enhanced data; based on the characteristics of the reconstructed feature enhanced data, the multimodal type of the data is determined, such as image, text, sequence, etc.; a synchronization mechanism is used, such as spatial alignment of the detection data, and multimodal type data is performed. Synchronize to generate multimodal data synchronization results; according to the multimodal type data, use machine learning algorithms to perform intelligent modal conversion on the multimodal data synchronization results to obtain intelligent modal conversion data; use fusion algorithms, such as weighted average technology, to perform dynamic modal fusion on the intelligent modal conversion data and genetic detection reconstruction data to obtain dynamic modal fusion data; optimize the dynamic modal fusion data to obtain adaptive modal optimization data; perform final data cleaning on the adaptive modal optimization data, including removing outliers, smoothing noise and other operations to obtain genetic detection cleaned data; use statistical methods, such as t-SNE or PCA, to reduce the dimension and visualize the cleaned data to obtain genetic detection cleaned data to verify the quality and consistency of the data.

[0140] Preferably, step S45 includes the following steps:

[0141] Step S451: performing data integrity check on the gene detection cleansing data to obtain integrity check data; performing data consistency analysis on the integrity check data to obtain consistency analysis data;

[0142] Step S452: performing a data distribution feature check on the consistency analysis data to obtain distribution feature check data; performing an outlier review process on the distribution feature check data to obtain outlier review data;

[0143] Step S453: performing data quality assessment on the outlier review data to obtain data quality assessment data; performing comprehensive verification of the cleaning results on the data quality assessment data to obtain data cleaning comprehensive verification results;

[0144] Step S454: Generate a cleaning effect report for the comprehensive verification results of data cleaning to obtain a genetic testing data cleaning report.

[0145] In an embodiment of the present invention, an MD5 check algorithm is used to perform integrity check on the gene detection cleaned data to obtain integrity check data; a Kappa coefficient test and an ICC intragroup correlation coefficient test method are used to analyze the consistency analysis data to ensure the consistency and reliability of the data in different samples or repeated measurements; by comparing data sets from different sources or time points, the Kendall coordination coefficient is used to analyze their consistency, thereby obtaining consistency analysis data; a descriptive statistical analysis method is used, including calculating the mean, median, standard deviation, etc. of the data, to perform a feature check on the distribution feature inspection data to obtain distribution feature inspection data; an outlier is detected using a Z-Score method and an IQR method, and a box plot is used to assist in visualization to review the detected outliers to determine their specific variation and obtain outlier review data; for outliers confirmed to be erroneous, a mean or median filling method is used to correct them, or a data interpolation algorithm such as K-nearest neighbor (KNN) is used to fill them. Through data quality assessment standards and indicators, such as accuracy, recall, precision and F1 score, the data quality assessment data is quantitatively evaluated to obtain data quality assessment data; using statistical methods, such as t-SNE or PCA, the data quality assessment data is reduced in dimension and visualized to obtain the comprehensive verification results of data cleaning; based on the comprehensive verification results of data cleaning, a detailed genetic testing data cleaning report is written; the report includes data integrity verification results, consistency analysis data, distribution characteristic inspection data, outlier review data and detailed description and analysis of data quality assessment data.

[0146] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0147] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A gene detection data cleaning method based on artificial intelligence, characterized in that: The following steps are involved: Step S1: obtaining an original gene detection data set; performing detection quality feature recognition on the original gene detection data set to obtain detection quality feature data; Performing quality screening on the detection quality characteristic data to obtain detection quality screening data; The original gene detection data set is denoised according to the detection quality screening data to obtain a preliminary gene detection data set; Step S2: performing high-throughput sequencing analysis on the preliminary gene detection data set to generate high-throughput sequencing data; performing sequencing error correction on the high-throughput sequencing data to obtain sequencing error correction data; performing automatic coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data; Step S3: performing genotype pattern recognition on the balanced coverage detection data to obtain genotype data; performing genotype clustering on the preliminary gene detection data set according to the genotype data to obtain gene detection cluster data; performing intelligent redundant data elimination on the gene detection cluster data to obtain gene detection redundant elimination data; Adaptively correcting the batch effect of the gene detection redundant elimination data to obtain batch effect corrected data, step S3 includes: Step S31: performing gene expression pattern mining on the balanced coverage detection data to obtain gene expression pattern data; performing pattern similarity analysis on the gene expression pattern data to obtain pattern similarity data; Step S32: performing pattern stability analysis on the gene expression pattern data according to the pattern similarity data to obtain expression pattern stability data; performing gene type identification on the preliminary gene detection data set according to the expression pattern stability data to obtain gene type data; performing contribution feature type selection on the gene type data to generate contribution feature type data; Step S33: performing gene type association analysis on the gene type data according to the contribution characteristic type data to obtain gene type association data; performing cluster density verification on the gene type association data to obtain cluster density verification data; performing abnormal cluster density removal on the gene type association data according to the cluster density verification data to generate abnormal density removal data; performing genotype clustering on the preliminary gene detection data set based on the abnormal density removal data and the gene type association data to obtain gene detection cluster data; Step S34: performing intelligent redundant data elimination on the gene detection clustering data to obtain gene detection redundant elimination data; Step S35: performing adaptive batch effect correction on the gene detection redundant elimination data to obtain batch effect corrected data; Step S4: evaluating the detection variant sites of the preliminary gene detection data set according to the batch effect correction data to obtain variant site evaluation data; performing multimodal data cleaning on the preliminary gene detection data set according to the variant site evaluation data to obtain gene detection cleaned data; verifying the cleaning results of the gene detection cleaned data to obtain a gene detection data cleaning report. Step S4 includes: Step S41: identifying detection variant sites on the preliminary gene detection data set according to the batch effect correction data to generate detection variant site data; performing variant site stability analysis on the detection variant site data to obtain site stability data; Step S42: performing variant site confidence calculation on the site stability data to obtain site confidence calculation data; performing variant site association analysis on the preliminary gene detection data set according to the site confidence calculation data to obtain variant site association data; performing variant site effect evaluation on the variant site association data to obtain site effect evaluation data; Step S43: integrating the site stability data, the site confidence calculation data and the site effect evaluation data to obtain the variant site evaluation data; Step S44: performing multimodal data cleaning on the preliminary gene detection data set according to the variant site evaluation data to obtain gene detection cleaned data; Step S45: verify the cleaning results of the gene detection cleaning data to obtain a gene detection data cleaning report.

2. The method for cleaning genetic testing data based on artificial intelligence according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Obtaining the original gene detection data set; Step S12: extracting the detection status of the original gene detection data set to generate gene detection status data; performing detection quality feature recognition on the gene detection status data to obtain detection quality feature data; performing quality status analysis on the detection quality feature data to obtain detection quality status data; Step S13: setting a quality compliance threshold for the detection quality status data to obtain quality compliance threshold data; comparing the detection quality status data with the quality compliance threshold data, if the detection quality status data is greater than the quality compliance threshold data, marking the detection quality status data as detection quality compliance data; if the detection quality status data is less than or equal to the quality compliance threshold data, marking the detection quality status data as detection quality abnormality data; integrating the detection quality compliance data and the detection quality abnormality data to obtain detection quality screening data; Step S14: performing outlier identification on the original gene detection data set according to the detection quality screening data to obtain gene detection quality outliers; performing nonlinear denoising on the gene detection quality outliers to obtain quality nonlinear denoised data; performing data stability enhancement on the original gene detection data set according to the quality nonlinear denoised data to obtain a preliminary gene detection data set.

3. The method for cleaning genetic testing data based on artificial intelligence according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: determining the gene expression status of the preliminary gene detection data set to obtain gene expression data; performing gene activity analysis on the gene expression data to generate detection gene activity data; Step S22: performing gene group identification on the preliminary gene detection data set according to the detection gene activity data to obtain detection gene group data; performing inter-group interaction analysis on the detection gene group data to obtain detection gene network relationship data; performing high-throughput sequencing analysis on the preliminary gene detection data set according to the detection gene network relationship data to generate high-throughput sequencing data; Step S23: extracting sequencing status from the high-throughput sequencing data to obtain gene sequencing status data; identifying sequencing error status from the gene sequencing status data to generate sequencing error status data; performing error type analysis on the sequencing error status data to obtain sequencing error type data; performing error type distribution detection on the high-throughput sequencing data according to the sequencing error type data to obtain sequencing error distribution data; Step S24: performing error sequencing positioning on the high-throughput sequencing data according to the sequencing error distribution data to obtain error sequencing positioning point data; performing error correction on the high-throughput sequencing data according to the error sequencing positioning point data to obtain sequencing error correction data; Step S25: Automatically perform coverage uniformization processing on the sequencing error correction data to obtain balanced coverage detection data.

4. The method for cleaning genetic testing data based on artificial intelligence according to claim 3, characterized in that: Step S25 includes the following steps: Step S251: performing adaptive normalization processing on the sequencing error correction data to obtain standardized correction data; performing multi-scale coverage analysis on the standardized correction data to obtain coverage distribution data; Step S252: locating abnormal regions of the coverage distribution data to obtain abnormal region data; dynamically adjusting abnormal region balance of the preliminary gene detection data set according to the abnormal region data to obtain region balance adjustment data; Step S253: performing iterative optimization processing on the sequencing error correction data according to the regional balance adjustment data to obtain correction iterative optimization data; performing coverage optimization processing on the preliminary gene detection data set according to the correction iterative optimization data to generate correction coverage optimization data; Step S254: Perform multi-dimensional balance verification on the correction coverage optimization data to obtain balanced verification data; automatically equalize the coverage of the sequencing error correction data based on the balanced verification data to obtain balanced coverage detection data.

5. The method for cleaning genetic testing data based on artificial intelligence according to claim 1, characterized in that: Step S34 includes the following steps: Step S341: performing deep data dimensionality reduction on the gene detection clustering data to obtain gene detection dimensionality reduction data; performing deep feature mapping on the gene detection dimensionality reduction data to obtain gene detection feature mapping data; Step S342: performing sparsity analysis on the gene detection feature mapping data to obtain detection sparsity data; performing sparsity enhancement processing on the detection sparsity data to obtain sparsity enhanced data; Step S343: performing redundant feature identification on the gene detection cluster data according to the sparsity enhancement data to obtain cluster redundant feature data; performing redundant weight analysis on the cluster redundant feature data to generate cluster redundant weight data; performing weight decay trend detection on the cluster redundant weight data to obtain weight decay trend data; Step S344: dynamically adjust the weight of the weight attenuation trend data to generate dynamic weight adjustment data; intelligently remove redundant data from the gene detection clustering data based on the dynamic weight adjustment data to obtain gene detection redundant removal data.

6. The method for cleaning genetic testing data based on artificial intelligence according to claim 5, characterized in that: Step S35 includes the following steps: Step S351: performing detection batch feature recognition on the preliminary gene detection data set to obtain detection batch feature data; performing effect feature extraction on the detection batch feature data to generate batch effect feature data; Step S352: constructing a batch effect association map for the gene detection redundant elimination data according to the batch effect characteristic data to generate a batch effect association map; performing effect component decomposition on the batch effect association map to obtain effect component decomposition data; Step S353: performing intelligent batch effect suppression on the effect component decomposition data to obtain batch effect suppression data; performing effect suppression verification on the preliminary gene detection data set according to the batch effect suppression data to obtain effect suppression verification data; Step S354: performing batch effect allele group analysis on the effect suppression verification data to obtain effect allele group data; performing adaptive batch effect correction on the preliminary gene detection data set based on the effect allele group data to obtain batch effect correction data.

7. The method for cleaning genetic testing data based on artificial intelligence according to claim 5, characterized in that: Step S44 includes the following steps: Step S441: dividing the preliminary gene detection data set into data structure hierarchies according to the variant site assessment data to obtain gene detection structure hierarchy parameters; reconstructing the preliminary gene detection data set according to the gene detection structure hierarchy parameters to obtain gene detection reconstructed data; Step S442: performing reconstruction feature mapping enhancement on the gene detection reconstructed data to obtain reconstructed feature enhanced data; performing multimodal type determination on the reconstructed feature enhanced data to generate multimodal type data; performing multimodal data synchronization on the reconstructed feature enhanced data according to the multimodal type data to generate a multimodal data synchronization result; Step S443: performing intelligent modality conversion on the multimodal data synchronization result according to the multimodal type data to obtain intelligent modality conversion data; performing dynamic modality fusion on the gene detection reconstruction data according to the intelligent modality conversion data to obtain dynamic modality fusion data; Step S444: performing adaptive modal optimization on the dynamic modal fusion data to obtain adaptive modal optimization data; performing dynamic modal data cleaning on the preliminary gene detection data set according to the adaptive modal optimization data to obtain gene detection cleaning data.

8. The method for cleaning genetic testing data based on artificial intelligence according to claim 7, characterized in that: Step S45 includes the following steps: Step S451: performing data integrity check on the gene detection cleansing data to obtain integrity check data; performing data consistency analysis on the integrity check data to obtain consistency analysis data; Step S452: performing a data distribution feature check on the consistency analysis data to obtain distribution feature check data; performing an outlier review process on the distribution feature check data to obtain outlier review data; Step S453: performing data quality assessment on the outlier review data to obtain data quality assessment data; performing comprehensive verification of the cleaning results on the data quality assessment data to obtain data cleaning comprehensive verification results; Step S454: Generate a cleaning effect report for the comprehensive verification results of data cleaning to obtain a genetic testing data cleaning report.

Images