Gene detection data cleaning method and system based on artificial intelligence

Abstract

The invention relates to the technical field of genomics, in particular to a gene detection data cleaning method and system based on artificial intelligence. The method comprises the following steps: acquiring gene detection data, and performing preliminary filtering on the gene detection data based on removal of low-quality reads to obtain preliminary filtering data; performing linker sequence identification and removal on the preliminarily filtered data, and performing sequencing error correction to generate gene sequence data; performing GC content analysis and repetitive sequence identification on the gene sequence data to obtain sequence feature data; performing dynamic quality threshold adjustment according to the sequence feature data, and performing secondary filtering on the gene sequence data to generate high-quality sequence data; and performing sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data. According to the method, deep analysis is carried out by utilizing sequence characteristics such as GC content and nucleotide distribution, and the recognition capability on sequence complexity and variation modes is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of genomics, and in particular to a gene detection data cleaning method and system based on artificial intelligence. Background Art

[0002] Genetic testing data is data information obtained by sequencing and analyzing DNA or RNA samples extracted from organisms through modern molecular biology technology. This type of data is used to reveal the genetic information, gene expression, genetic variation, etc. of organisms. With the development of high-throughput sequencing (HTS) technology, genetic testing data has become an important resource in the fields of medicine, agriculture, forensics, etc. Genetic testing data cleaning refers to the process of removing noise, redundant information and low-quality fragments from the raw data obtained by sequencing to ensure the accuracy and reliability of subsequent analysis. Due to the complexity of genetic sequencing technology and the specific problems caused by different samples and platforms, data cleaning is a key step in the genetic testing process.

[0003] However, traditional AI-based genetic testing data cleaning methods often have the following steps: In traditional genetic testing data cleaning methods, they mainly rely on static threshold settings and standardized filtering rules, which are difficult to adapt to diverse and complex data types; when dealing with different sample sources, different sequencing platforms and different genome complexities, traditional methods are insufficient in accuracy and flexibility, often resulting in high false positives or missing key variant sites. Summary of the invention

[0004] Based on this, it is necessary for the present invention to provide a genetic testing data cleaning method and system based on artificial intelligence to solve at least one of the above technical problems.

[0005] To achieve the above purpose, a genetic testing data cleaning method based on artificial intelligence includes the following steps:

[0006] Step S1: Obtaining gene detection data, performing preliminary filtering on the gene detection data based on removing low-quality reads to obtain preliminary filtered data; performing linker sequence identification and removal on the preliminary filtered data, and performing sequencing error correction to generate gene sequence data;

[0007] Step S2: GC content analysis and repetitive sequence identification are performed on the gene sequence data to obtain sequence feature data;

[0008] Step S3: dynamically adjusting the quality threshold according to the sequence feature data, and performing secondary filtering on the gene sequence data to generate high-quality sequence data; performing sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; performing nucleotide distribution analysis on the sequence feature data, and performing weight adjustment on the sequence pattern feature data to generate weighted feature data;

[0009] Step S4: perform genome alignment on the gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the alignment result, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on the candidate variant data by weighted feature data, so as to obtain credible variant site data;

[0010] Step S5: Confirm the variant sites by integrating multiple algorithms based on the credible variant site data and high-quality sequence data to generate preliminary variant detection result data; perform rare variant and structural variant analysis on the preliminary variant detection result data to obtain the final variant detection data.

[0011] The present invention can effectively eliminate those sequencing reads with poor quality through preliminary filtering, reduce the interference of low-quality data on subsequent analysis, thereby improving the overall quality of the data and the accuracy of the analysis results. Residual adapter sequences may remain during the detection process. Identifying and removing adapter sequences can reduce the interference of irrelevant sequences on the analysis, ensuring that the generated gene sequence is more accurate. Through the error correction algorithm, the random errors generated during the sequencing process are corrected, the data quality is further improved, and the stability of downstream analysis is ensured. The distribution of GC content is an important feature in the genome, and uneven GC content may cause sequencing deviations. By analyzing the GC content, abnormal regions can be effectively identified, thereby improving the accuracy of the data and the effectiveness of biological interpretation. Repeated sequences in the genome are prone to misleading results during the analysis process. Identifying these regions helps to avoid incorrect comparisons and incorrect variant detections, and reduces false positive results. By dynamically adjusting the quality threshold, customized quality control can be performed for different types of sequence fragments, improving the flexibility and accuracy of data filtering, and then generating high-quality sequence data. Recognizing specific patterns in the sequence (such as functional elements, repeated sequences, etc.) helps to detect potential functional regions and conserved regions, providing a more biologically meaningful interpretation. The distribution analysis of nucleotides can reveal the regional specificity in the sequence. Combined with the weight adjustment of pattern feature data, it can more accurately highlight important features and help subsequent variant identification. By comparing known genomic libraries, the variant sites in the gene sequence can be accurately located to ensure the accuracy of variant detection. Alignment can also help identify potential mismatches and reduce false positive rates. Combining dynamic quality control with alignment results can ensure the authenticity of candidate variants and effectively filter out low-quality and unreliable variants. Credibility screening based on previously generated weighted feature data can further improve the accuracy of variant detection and ensure that the selected variant sites have high biological credibility. Through the integrated analysis of multiple variant detection algorithms, the sensitivity and specificity of variant detection can be improved, the possible deviations of a single algorithm can be reduced, and the reliability of the test results can be ensured. Rare variants and structural variants often have important biological functions. Through special analysis steps, these variants can be identified, providing more valuable references for clinical applications. Through methods such as redundancy removal and consistency verification, the final output variant data is ensured to have a high degree of accuracy and credibility, laying a solid foundation for downstream gene function analysis and clinical applications. This method greatly improves the quality of genetic testing data and the accuracy of variant detection through multiple steps of filtering, correction, comparison and analysis. In particular, the reliability of the test results is ensured through technical means such as dynamic quality threshold adjustment, multi-algorithm integration, and the use of weighted feature data, making this method of great value in applications such as precision medicine and clinical diagnosis.

[0012] The present invention also provides an artificial intelligence-based gene detection data cleaning system, which is used to execute the artificial intelligence-based gene detection data cleaning method mentioned above. The artificial intelligence-based gene detection data cleaning system includes:

[0013] A data preprocessing module is used to obtain gene detection data, perform preliminary filtering on the gene detection data based on removing low-quality reads, and obtain preliminary filtered data; identify and remove adapter sequences on the preliminary filtered data, and perform sequencing error correction to generate gene sequence data;

[0014] The sequence feature extraction module is used to perform GC content analysis and repetitive sequence identification on gene sequence data to obtain sequence feature data;

[0015] The dynamic quality optimization module is used to adjust the dynamic quality threshold according to the sequence feature data, and perform secondary filtering on the gene sequence data to generate high-quality sequence data; perform sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; perform nucleotide distribution analysis on the sequence feature data, and perform weight adjustment on the sequence pattern feature data to generate weighted feature data;

[0016] The variant site screening module is used to perform genome comparison on gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the comparison results, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on candidate variant data through weighted feature data, so as to obtain credible variant site data;

[0017] The variation confirmation module is used to perform variation site confirmation based on the trusted variation site data and high-quality sequence data using multiple algorithms to generate preliminary variation detection result data; perform rare variation and structural variation analysis on the preliminary variation detection result data to obtain the final variation detection data.

[0018] The present invention ensures that the basic data of subsequent analysis is reliable by removing low-quality reads, reduces noise interference, and improves the accuracy of subsequent analysis. The identification and removal of the adapter sequence can eliminate the non-target sequences introduced during the sequencing process, so that the gene sequence data finally obtained is purer and more accurate. The correction of sequencing errors can effectively reduce errors in the data, ensure the accuracy of the gene sequence data, and lay a good foundation for subsequent analysis. The analysis of GC content can reveal the composition characteristics of the genome and help understand the function and regulatory mechanism of the gene. The identification of repeated sequences can provide information for subsequent structural variation analysis. The extracted sequence feature data provides key input information for dynamic quality optimization and variant site screening, so that these subsequent steps can be executed more accurately. The adjustment of the dynamic quality threshold can be flexibly responded to according to the specific sequence features, effectively eliminate low-quality data that does not meet the standards, and further improve the data quality. Through the generation of sequence pattern recognition and weighted features, the influence of important features in subsequent analysis is enhanced, making variation detection more accurate. The analysis of nucleotide distribution helps to understand the sequence features more comprehensively, so as to consider more dimensional information in variation detection. Genome alignment can help accurately identify the variant sites in the genome and improve the sensitivity and accuracy of variation recognition. Through the integration of adaptive quality thresholds and the application of weighted feature data, the credibility screening of candidate variants can be effectively improved, misidentification can be reduced, and the reliability of the final variant sites can be improved. By combining multiple variant detection algorithms, the advantages of different algorithms can be combined to improve the accuracy and comprehensiveness of variant detection, ensuring that multiple types of variants are identified. Through the analysis of rare variants and structural variants, important variants associated with specific diseases can be identified, providing valuable information for clinical applications and biological research. The variant detection data finally generated has been optimized and confirmed in multiple steps to ensure high quality and high credibility of the data, laying the foundation for subsequent biological research and clinical applications. Through the organic combination of these modules, from data preprocessing to variant confirmation, each module plays a key role in improving data quality, enhancing feature expression, accurately identifying variants, and improving the reliability of results. The high-quality variant detection data finally generated provides a solid foundation for subsequent scientific research and clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0020] Figure 1 This is a schematic diagram of the steps of the artificial intelligence-based genetic testing data cleaning method of the present invention;

[0021] Figure 2 for Figure 1 Detailed step flow diagram of step S1;

[0022] Figure 3 for Figure 1 Detailed step flow chart of step S2 in FIG. DETAILED DESCRIPTION

[0023] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0025] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0026] To achieve this, please refer to Figures 1 to 3 The present invention provides a gene detection data cleaning method based on artificial intelligence, the method comprising the following steps:

[0027] Step S1: Obtaining gene detection data, performing preliminary filtering on the gene detection data based on removing low-quality reads to obtain preliminary filtered data; performing linker sequence identification and removal on the preliminary filtered data, and performing sequencing error correction to generate gene sequence data;

[0028] Step S2: GC content analysis and repetitive sequence identification are performed on the gene sequence data to obtain sequence feature data;

[0029] Step S3: dynamically adjusting the quality threshold according to the sequence feature data, and performing secondary filtering on the gene sequence data to generate high-quality sequence data; performing sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; performing nucleotide distribution analysis on the sequence feature data, and performing weight adjustment on the sequence pattern feature data to generate weighted feature data;

[0030] Step S4: perform genome alignment on the gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the alignment result, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on the candidate variant data by weighted feature data, so as to obtain credible variant site data;

[0031] Step S5: Confirm the variant sites by integrating multiple algorithms based on the credible variant site data and high-quality sequence data to generate preliminary variant detection result data; perform rare variant and structural variant analysis on the preliminary variant detection result data to obtain the final variant detection data.

[0032] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of a gene detection data cleaning method based on artificial intelligence of the present invention. In this example, the gene detection data cleaning method based on artificial intelligence includes the following steps:

[0033] Step S1: Obtaining gene detection data, performing preliminary filtering on the gene detection data based on removing low-quality reads to obtain preliminary filtered data; performing linker sequence identification and removal on the preliminary filtered data, and performing sequencing error correction to generate gene sequence data;

[0034] The gene detection data of the embodiment of the present invention is generated by a high-throughput sequencing platform (such as Illumina HiSeq or NovaSeq) in the format of FASTQ. First, the gene detection data needs to be initially filtered to remove low-quality reads. The specific operation includes calculating the Phred quality score of each base, screening and removing reads below Q20, and filtering out these unqualified data to obtain preliminary filtered data. Next, the preliminary filtered data is identified and removed by a joint sequence recognition algorithm based on sequence alignment (such as BWA or Cutadapt). Then, sequencing errors are corrected by the k-mer frequency correction method, and commonly used error correction tools include KMC or BCFtools, and the corrected gene sequence data is finally generated. This process is applicable to human genome detection in practical applications, setting the filtering threshold to Q20, and using a k-mer with a length of 36bp for error correction to ensure that the generated gene sequence data reaches high precision.

[0035] Step S2: GC content analysis and repetitive sequence identification are performed on the gene sequence data to obtain sequence feature data;

[0036] The embodiment of the present invention performs GC content analysis and repetitive sequence identification on the gene sequence data generated in step S1. First, the sliding window method is used to calculate the GC content of the gene sequence data, with a window size of 100bp and a step size of 10bp. The GC content distribution data is generated by calculating the ratio of GC bases in each window. For areas with abnormal GC content distribution, the abnormal areas are identified and marked to generate GC abnormal area marking data. At the same time, the RepeatMasker tool is used to scan the gene sequence data, identify the repetitive sequence areas and generate repetitive sequence feature data. Finally, the GC abnormal area data is feature fused with the repetitive sequence feature data to generate sequence feature data. This process is applied to multi-sample genome detection projects to identify gene fragments with more copy number variations. The window size and step size can be adjusted according to specific application requirements.

[0037] Step S3: dynamically adjusting the quality threshold according to the sequence feature data, and performing secondary filtering on the gene sequence data to generate high-quality sequence data; performing sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; performing nucleotide distribution analysis on the sequence feature data, and performing weight adjustment on the sequence pattern feature data to generate weighted feature data;

[0038] The embodiment of the present invention performs dynamic quality threshold adjustment on gene sequence data according to the sequence feature data obtained in step S2. First, a multidimensional feature space is constructed, and each fragment of the gene sequence data is clustered and analyzed. An unsupervised clustering algorithm (such as DBSCAN or K-means) is used to generate sequence clustering results, and its quality distribution characteristics (including mean, variance, skewness and kurtosis) are calculated. Based on these characteristics, a quality threshold decision model is constructed, and the model is optimized using the degree of sequence quality improvement, and the quality thresholds of different sequence fragments are dynamically adjusted. Sequence fragments that do not meet quality requirements are eliminated to generate high-quality sequence data. Subsequently, sequence pattern recognition is performed on high-quality sequence data using k-mer analysis to obtain sequence pattern feature data, and nucleotide distribution analysis is performed on sequence feature data at the same time, potential periodic patterns are identified using Fourier transform, and weight adjustment is performed in combination with sequence pattern features, and weighted feature data is finally generated. This process is suitable for high-precision processing of genomic data, particularly for optimizing sequence quality before variation detection.

[0039] Step S4: perform genome alignment on the gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the alignment result, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on the candidate variant data by weighted feature data, so as to obtain credible variant site data;

[0040] The embodiment of the present invention compares high-quality gene sequence data according to a preset genome library (such as the hg38 human genome reference library). The BWA-MEM tool is used to perform genome alignment to generate alignment result data, and the adaptive quality threshold data is integrated with the alignment result data through the Bayesian fusion method to obtain heterogeneous integration feature data. On this basis, the genome alignment results are used to preliminarily identify the variant sites in the gene sequence data to generate candidate variant data. The candidate variant data is screened for credibility using the weighted feature data obtained in step S3, and low-credibility candidate variants are eliminated, and finally credible variant site data is generated. This embodiment is suitable for cancer genome research, and variant site detection is performed on patient samples. The threshold of genome alignment is set to 90% alignment coverage.

[0041] Step S5: Confirm the variant sites by integrating multiple algorithms based on the credible variant site data and high-quality sequence data to generate preliminary variant detection result data; perform rare variant and structural variant analysis on the preliminary variant detection result data to obtain the final variant detection data.

[0042] The embodiment of the present invention uses multiple variation detection algorithms (such as Mutect2, Freebayes, VarScan, etc.) to perform parallel analysis on gene sequence data based on credible variation site data and high-quality sequence data to generate multi-algorithm variation detection results. Then, an integrated analysis method based on machine learning (such as random forest or XGBoost) is used to integrate and analyze the multi-algorithm detection results to obtain preliminary variation detection result data. The preliminary result data is further subjected to rare variation analysis, rare variation frequencies are calculated, and potential rare variations are identified. Finally, a local assembly algorithm is used to perform structural analysis on potential structural variation regions, confirm structural variations, and generate structural variation candidate data. By classifying and verifying the structural variation candidate data, credible structural variation data is obtained, and the final consistency check and redundancy removal are performed in combination with the rare variation data to generate the final variation detection data. This step is applicable to complex variation detection of the human genome, especially in genomic research on tumor samples, where the machine learning model in the integrated analysis can be optimized based on millions of training samples.

[0043] Preferably, step S1 comprises the following steps:

[0044] Step S11: Obtaining genetic testing data in FASTQ format;

[0045] Step S12: Perform integrity check on the gene detection data based on the data file size, number of lines and sequence length distribution, and remove invalid files to obtain valid gene detection data;

[0046] Step S13: Calculate the Phred quality score of each base for the valid gene detection data, thereby obtaining sequencing quality distribution data;

[0047] Step S14: performing preliminary filtering on the effective gene detection data based on removing low-quality reads according to the sequencing quality distribution data to obtain preliminary filtered data;

[0048] Step S15: performing linker sequence identification based on a sequence alignment algorithm on the preliminary filtered data, and removing the linker sequence from the read segment, thereby obtaining linker-free sequence data;

[0049] Step S16: quickly comparing the removed linker sequence data according to the preset contamination source data, identifying and removing the contamination sequence, thereby obtaining purified sequence data;

[0050] Step S17: Perform sequencing error correction on the purified sequence data using an error correction algorithm based on k-mer frequency to obtain corrected gene sequence data.

[0051] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in the embodiment of the present invention, step S1 includes the following steps:

[0052] Step S11: Obtaining genetic testing data in FASTQ format;

[0053] The genetic detection data of the embodiment of the present invention is generated by the Illumina NovaSeq 6000 platform and stored in FASTQ format. The data set contains two files: one is the forward read segment (R1) and the other is the reverse read segment (R2). The size of each file is about 100GB, the data volume is about 300 million pairs of read segments, and the length of each read segment is 150bp. These FASTQ files contain the base sequence of the genomic fragment and its corresponding quality value information. These files are read as input for genetic detection data for subsequent processing. This process is suitable for large-scale whole genome sequencing projects, such as human genome sequencing, and parameters such as file size and read segment length may vary depending on the specific platform and experimental conditions.

[0054] Step S12: Perform integrity check on the gene detection data based on the data file size, number of lines and sequence length distribution, and remove invalid files to obtain valid gene detection data;

[0055] The embodiment of the present invention performs integrity detection on the genetic detection data obtained in step S11, and first calculates the number of lines, file size and sequence length distribution of the FASTQ file. The FastQC tool is used to analyze the file to detect the read integrity of the file, such as whether each read segment contains the corresponding base sequence and quality value information, and whether it conforms to the expected file format. Files with abnormal number of lines or insufficient file size are marked, and invalid files are removed. During the detection process, the distribution of sequence lengths is also analyzed to ensure that all read segment lengths are consistent or within a reasonable range. Finally, invalid or abnormal data is removed to obtain valid genetic detection data. This process is suitable for large-scale genome research and can effectively eliminate system errors and invalid data in the sequencing process.

[0056] Step S13: Calculate the Phred quality score of each base for the valid gene detection data, thereby obtaining sequencing quality distribution data;

[0057] The embodiment of the present invention uses FastQC to calculate the Phred quality score of the valid gene detection data obtained in step S12. The Phred quality score is used to indicate the accuracy of each base in the sequencing process. The sequencing quality distribution data is obtained by calculating the Phred value of each base. For the base position of each read segment, the mean and variance of its Phred quality score are calculated, and the quality distribution of the entire data set is analyzed. Bases with sequencing quality lower than Q20 are marked as low quality, and Q20 indicates that the base recognition error rate is 1%. This process is suitable for genome sequencing projects with a higher sequencing depth, and the generated sequencing quality distribution data can guide subsequent data filtering and correction.

[0058] Step S14: performing preliminary filtering on the effective gene detection data based on removing low-quality reads according to the sequencing quality distribution data to obtain preliminary filtered data;

[0059] The embodiment of the present invention performs preliminary filtering of valid gene detection data based on the sequencing quality distribution data generated in step S13. First, a low quality threshold is set, such as Q20, indicating that all bases with a quality score lower than 20 will be regarded as low-quality reads. Then, the data is filtered using tools such as Trim Galore to remove low-quality reads and low-quality bases at the ends. Short reads with a length of less than 50bp are also removed during the filtering process. After this step, preliminary filtering data is generated, and the proportion of removed low-quality reads is generally between 1% and 5%, and the specific proportion varies according to the sample and experimental conditions. This process is suitable for the quality control stage of high-throughput sequencing data to ensure the accuracy and reliability of subsequent analysis.

[0060] Step S15: performing linker sequence identification based on a sequence alignment algorithm on the preliminary filtered data, and removing the linker sequence from the read segment, thereby obtaining linker-free sequence data;

[0061] The embodiment of the present invention identifies and removes the joint sequence of the preliminary filtered data, and uses the BWA tool for sequence alignment. First, the common joint sequence (such as Illumina sequencing joint sequence) is used as a reference sequence, and the read segment in the preliminary filtered data is compared to identify the joint sequence position. For the read segment containing the joint sequence, the joint sequence part is cut off using the Cutadapt tool to obtain the joint sequence data. In this embodiment, the joint sequence length is generally 20-30bp, and the joint sequence identified and removed accounts for 5%-10%. This process is suitable for data cleaning generated by different sequencing platforms to ensure that the technical noise in the data is effectively removed.

[0062] Step S16: quickly comparing the removed linker sequence data according to the preset contamination source data, identifying and removing the contamination sequence, thereby obtaining purified sequence data;

[0063] The embodiment of the present invention uses Bowtie2 to perform rapid comparison based on a preset pollution source database (such as genomic data of bacteria, viruses and other common environmental pollutants) to identify and remove contamination sequences in genetic detection data. First, the linker-free sequence data is compared with the pollution source database, and for sequences with a comparison score higher than the threshold, they are marked as contamination sequences and removed. Through this step, non-target sequences introduced by environmental pollution or sample cross-contamination are removed to generate purified sequence data. In practical applications, the proportion of contamination sequences is usually low, but it may account for more than 10% in microbial population sequencing, so this step is particularly important.

[0064] Step S17: Perform sequencing error correction on the purified sequence data using an error correction algorithm based on k-mer frequency to obtain corrected gene sequence data.

[0065] The embodiment of the present invention uses an error correction algorithm based on k-mer frequency to correct the purified sequence data obtained in step S16. In this embodiment, the KMC tool is used to perform k-mer frequency analysis on the purified sequence data, the k value is set to 21bp, the frequency of occurrence of each k-mer is calculated, and the low-frequency erroneous k-mer is identified. By comparing with the high-frequency k-mer, the base errors corresponding to the low-frequency k-mer are corrected, thereby performing sequencing error correction. After correction, gene sequence data with higher accuracy is generated. This process is applicable to large-scale whole genome sequencing or transcriptome sequencing projects, and the corrected data can be used for further variation detection or genome assembly analysis.

[0066] The FASTQ format of the present invention is a standard format for gene sequencing data, which contains sequence information and quality scores, ensuring that the data can be subsequently processed in a unified manner; using the standard FASTQ format ensures that the data can be seamlessly connected to various analysis tools and algorithms, which is convenient for subsequent operations. By detecting the file size, number of lines, and sequence length distribution, missing parts or abnormal files in the data can be quickly discovered to ensure that only complete and standard data enter the subsequent process; by eliminating invalid or abnormal files, low-quality or erroneous files are avoided from entering the subsequent analysis link, thereby improving the efficiency and accuracy of the overall data processing. The Phred quality score is a standard measure of sequencing quality, which can evaluate the sequencing accuracy of each base and help identify which bases may have errors, thereby providing a reference for subsequent filtering steps; by calculating the distribution of Phred scores, the quality status of the overall sequencing data can be understood, and data support can be provided for formulating data filtering and optimization strategies. According to the analysis of Phred quality scores in the previous step, reads with quality below the threshold are removed, which reduces the impact of low-quality data on subsequent analysis, thereby improving the overall credibility of the data; filtering out low-quality reads helps improve the accuracy of subsequent alignment and analysis, and reduces the occurrence of false positives and false negatives. During the sequencing process, adapter sequences will be mixed into the reads. By identifying and removing these irrelevant sequences, the data can be purified, false positive results can be reduced, and the purity of the data can be improved; eliminating adapter sequences helps to avoid them being misidentified as variants or other biological features in subsequent alignments and analyses, thereby reducing the error rate. Exogenous contaminants may be mixed in the sequencing data. By comparing with a database of known contamination sources, these contamination sequences can be effectively identified and removed to ensure data purity; after eliminating the contamination source, the remaining data is more accurate and reliable, which helps to improve the accuracy of subsequent gene alignment and variation detection. The k-mer frequency error correction algorithm can detect and correct random sequencing errors based on the repetitive patterns of short fragments in the sequence, further improving the accuracy of the data; through effective error correction, the noise introduced by sequencing errors can be reduced, thereby improving the credibility of the gene sequence data and making it perform better in subsequent analysis (such as variant detection). Through these steps, the data has undergone multi-level filtering and optimization, including integrity checks, quality assessments, removal of adapter sequences and contamination sequences, and error correction. Each step is designed to improve the quality, accuracy, and reliability of the data, and ultimately generate high-quality gene sequence data. These processing steps not only reduce errors, but also reduce the possibility of false positive detection, providing a more reliable data basis for tasks such as variant detection and sequence alignment in genomic research and clinical applications.

[0067] Preferably, step S2 comprises the following steps:

[0068] Step S21: performing sliding window analysis on the gene sequence data and calculating the GC content of each window to obtain GC content distribution data;

[0069] Step S22: identifying abnormal GC content regions based on the GC content distribution data, and generating GC abnormal region marking data;

[0070] Step S23: Scan the gene sequence data, identify and mark the repeated sequence region, and obtain repeated sequence feature data;

[0071] Step S24: feature fusion of the GC abnormal region marking data and the repeated sequence feature data to obtain sequence feature data.

[0072] As an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 Detailed step flow diagram of step S2 in the embodiment of the present invention, step S2 includes the following steps:

[0073] Step S21: performing sliding window analysis on the gene sequence data and calculating the GC content of each window to obtain GC content distribution data;

[0074] The embodiment of the present invention uses a sliding window analysis method to calculate the GC content of gene sequence data. Specifically, the size of the sliding window is set to 100bp, and the sliding step is 10bp. The GC content in each window is calculated to obtain the GC ratio of each window. The GC ratio refers to the ratio of the sum of the number of G (guanine) and C (cytosine) bases in the window to the total number of bases in the entire window. In this way, the GC content distribution data of the entire sequence is generated, and these data are recorded in a table, marking the position of each window and the corresponding GC ratio. In genome analysis, the calculation of GC content is used to evaluate the complexity and variability of genomic regions, and is suitable for sequence quality evaluation and variation analysis in large-scale genome sequencing projects.

[0075] Step S22: identifying abnormal GC content regions based on the GC content distribution data, and generating GC abnormal region marking data;

[0076] The embodiment of the present invention identifies regions with abnormal GC content based on the GC content distribution data generated in step S21. First, a normal range threshold of the GC content is set. For example, the normal range can be set to 35%-65%. When the GC content of a window is lower or higher than this range, the window is marked as a region with abnormal GC content. Then, marker data is generated for these abnormal regions, and the marker data includes the starting position, the ending position and the GC content deviation value of the abnormal region. GC abnormal regions are usually associated with non-coding regions, repetitive regions or gene deletions in the genome, so this step plays an important role in studying genomic structural variations or abnormal gene regions, especially in genomic studies of rare diseases.

[0077] Step S23: Scan the gene sequence data, identify and mark the repeated sequence region, and obtain repeated sequence feature data;

[0078] The embodiment of the present invention uses the RepeatMasker tool to scan the gene sequence data, identify and mark the repetitive sequence regions. Repetitive sequences include interspersed repetitive sequences and tandem repetitive sequences, which are prone to sequencing errors or analysis errors. The specific operation is to compare the gene sequence with a known repetitive sequence database, identify the regions that match the database sequence, and mark these regions as repetitive sequence regions. The marking data includes the starting position, the ending position, the repetition type (such as short tandem repeats, long interspersed repeats, etc.) and the repetition frequency of the repetitive sequence. In the human genome, repetitive sequences usually account for 30%-50% of the genome, so this step can effectively help analysts eliminate the interference of these regions with subsequent variation detection.

[0079] Step S24: feature fusion of the GC abnormal region marking data and the repeated sequence feature data to obtain sequence feature data.

[0080] The embodiment of the present invention performs feature fusion on the GC abnormal region marking data generated in step S22 and the repetitive sequence characteristic data generated in step S23 to obtain sequence characteristic data. First, the two data sets are merged according to the sequence position, and the GC abnormality information and repetitive sequence information of each region are retained. If a region contains both GC abnormalities and repetitive sequences, it is marked as a high-complexity region, and its sequence characteristics are recorded in detail. Through feature fusion, the complexity characteristics of the gene sequence can be described more comprehensively, and the generated sequence characteristic data contains information such as the GC content, repetitive type, and position of each gene region. This step is applicable to the data preprocessing stage in genomic variation research, and is helpful for filtering or weighted processing of complex sequence regions in subsequent analysis.

[0081] The sliding window analysis method of the present invention can perform local analysis on the gene sequence, and generate GC content distribution data by calculating the GC content in each window. This distribution can better reflect the local characteristics of the sequence and avoid the deviation caused by a single overall GC content. The GC content may vary significantly in different regions of the genome, and some functional regions may show extreme GC content. Sliding window analysis can reveal these local differences and help identify regions that may be functionally important. By analyzing the GC content distribution data, abnormally high or abnormally low GC content regions can be identified, which may correspond to fragments with poor sequencing quality, genome rearrangements, or abnormal functional regions. Marking these abnormal regions helps to pay special attention to these fragments in subsequent analysis. Identifying and marking GC abnormal regions can help researchers avoid misjudgment or over-reliance on these abnormal regions in subsequent comparisons and variation detection processes, thereby improving the accuracy and reliability of the entire analysis. By scanning the gene sequence, repeated sequences present in the genome can be identified. These repeated sequences may cause false positives in subsequent variation detection and genome comparison processes, so it is very important to mark these regions as soon as possible. Repeated sequences play an important role in genome function and structure, and may be related to gene regulation, structural variation, etc. Marking these regions helps to gain a deeper understanding of the structural characteristics and potential functional impacts of the genome. Fusion of GC abnormal region marker data and repetitive sequence feature data can comprehensively reflect the potential abnormal features in the gene sequence. This fused sequence feature data can provide richer information for subsequent gene alignment, variation detection, gene function prediction, etc. By combining different types of feature data, truly abnormal or important sequence regions can be identified more accurately, avoiding false positives caused by a single feature. This provides a more targeted and accurate reference for subsequent analysis. These steps help identify local abnormal regions and repetitive fragments in gene sequences through sliding window analysis, GC content calculation, and repetitive sequence detection. The effect of these steps is to improve the quality of data, identify potential abnormal regions, avoid false positives, and improve the accuracy of the overall analysis. The final sequence feature data provides a more accurate and detailed feature description for subsequent genome alignment, variation detection, and functional analysis, which helps to improve the efficiency and reliability of the entire genome analysis process.

[0082] Preferably, step S3 comprises the following steps:

[0083] Step S31: dynamically adjusting the quality threshold according to the sequence feature data, thereby obtaining adaptive quality threshold data;

[0084] The embodiment of the present invention uses sequence feature data to adjust the dynamic quality threshold. The specific method is to set the initial quality threshold range (for example, the minimum value of the Phred quality score is set to 20) according to the data such as the GC content distribution and the repetitive sequence characteristics obtained in step S24, and dynamically adjust the threshold to adapt to the sequence complexity of different regions. For example, for areas with abnormal GC content or more repetitive sequences, the quality threshold is lowered (such as to 15) to reduce filtering errors; while for areas with normal GC content, a higher quality threshold is maintained. Through this dynamic adjustment, adaptive quality threshold data is generated for subsequent sequence data filtering. This method is suitable for quality control in large-scale genome sequencing, and can effectively improve the accuracy of sequence filtering and avoid information loss caused by excessive filtering.

[0085] Step S32: performing secondary filtering on the gene sequence data according to the adaptive quality threshold data to remove sequence fragments that do not meet the quality requirements, thereby obtaining high-quality sequence data.

[0086] The embodiment of the present invention performs secondary filtering on the gene sequence data according to the adaptive quality threshold data generated in step S31. In this embodiment, a filtering algorithm based on a dynamic threshold is adopted to remove low-quality sequence fragments that do not meet the quality requirements. The specific operation is to calculate the average Phred quality score for each sequence fragment and compare it with the adaptive quality threshold. All sequence fragments below the threshold will be eliminated, and the remaining sequence fragments will be retained as high-quality sequence data. After the filtering is completed, the obtained high-quality sequence data will be retained in a new data file for subsequent analysis. The filtering strategy of this step can retain valuable gene sequence information to the maximum extent while ensuring data quality, and is suitable for the post-processing link of high-throughput sequencing data.

[0087] Step S33: performing sequence pattern recognition based on functional element features on the high-quality sequence data, thereby obtaining sequence pattern feature data;

[0088] The embodiment of the present invention carries out sequence pattern recognition based on functional element characteristics to high-quality sequence data. In the present embodiment, MEME (Multiple Em for Motif Elicitation) software is used to carry out pattern recognition to functional elements that may exist in the sequence, and functional elements include promoter regions, enhancer sequences, etc. The identification of these elements is by scanning the high-quality sequence data, extracting sequence fragments with specific patterns, and determining the significance of these patterns by statistical analysis. For example, for a sequence pattern with a length of 6bp, a probability model is used to judge the probability that the pattern occurs in a random sequence, and a significant pattern with a biological function is screened out. The target of this step is to identify the functional sequence related to gene expression regulation, generate sequence pattern feature data, and be applicable to element prediction in genome function research.

[0089] Step S34: performing nucleotide distribution analysis on the sequence feature data and weight adjustment on the sequence pattern feature data to generate weighted feature data.

[0090] The embodiment of the present invention performs nucleotide distribution analysis on the sequence feature data generated in step S24, and the specific analysis method is to perform position distribution statistics on the four bases (A, T, C, G) in each gene sequence to obtain a density distribution map of nucleotides. Next, in conjunction with the sequence pattern feature data generated in step S33, a weighted algorithm is used to integrate these data. According to the biological importance of different sequence patterns (such as promoter patterns or repetitive sequence patterns), different weights are assigned to different feature data. For example, a higher weight is assigned to the promoter region, and a lower weight is assigned to the repetitive sequence. Weighted feature data is generated by weight adjustment, and the data is used for subsequent gene function analysis and variation detection, which can highlight key areas with biological functions.

[0091] The present invention can flexibly set the threshold value according to the characteristics of different sequences (such as GC content, repeated sequences, etc.) by dynamically adjusting the quality threshold value, rather than adopting a unified static quality standard. This adaptive strategy can better adapt to the variability of different gene regions and improve the overall quality of the data. Dynamically adjusting the threshold value can avoid excessively eliminating sequences with potential important information while maintaining high-quality data, thereby balancing data quality and information retention. This is particularly important in complex genome sequences and can reduce the loss of misjudged regions. By performing secondary filtering based on data of adaptive quality threshold values, sequence fragments with poor quality can be more accurately eliminated to ensure the accuracy of subsequent analysis. Compared with preliminary filtering, the precision of secondary filtering is higher, especially more effective when filtering complex or marginal low-quality sequence fragments. After eliminating sequences that do not meet quality requirements, the remaining data will have higher reliability, providing a more stable foundation for subsequent genome comparison, functional analysis and variation detection. Through sequence pattern recognition, important elements or patterns (such as promoters, enhancers, etc.) with biological functions in gene sequences can be detected. The identification of these functional elements is crucial to gene regulation, expression and functional research. Pattern recognition based on functional elements helps to reveal potential important regions in the genome. By identifying functionally relevant pattern features, researchers can gain a deeper understanding of the functions of specific gene sequences and their possible biological significance. This pattern information is of great value in fields such as genetic analysis, gene editing, and drug development. Nucleotide distribution analysis can further identify key features in gene sequences by examining the local and global distribution of bases. Adjusting the weights of these features helps to enhance the expression of important features, while weakening or excluding noise features and improving the expressiveness of the overall feature set. The weighted feature data can more accurately locate key regions and patterns in the sequence in subsequent analysis. This significantly improves the accuracy of tasks such as genome alignment, variant identification, and gene function prediction, helping to reduce errors and enhance the credibility of the results. These steps further optimize the quality and feature expression of gene sequence data by dynamically adjusting quality thresholds, secondary filtering sequences, identifying functional element patterns, and weighting features. The effects of this process are reflected in many aspects, including enhancing the flexibility and reliability of data, accurately identifying functionally relevant sequence patterns, and optimizing the expression of feature information, thereby providing a higher quality foundation and more reliable feature data for subsequent gene analysis. This refined processing can significantly improve the accuracy and efficiency of the entire genome analysis process.

[0092] Preferably, step S31 includes the following steps:

[0093] Step S311: construct a multidimensional feature space according to the sequence feature data, map the sequence fragments in the gene sequence data into the multidimensional feature space, perform clustering analysis on the data points based on an unsupervised clustering algorithm, and obtain sequence clustering result data;

[0094] The embodiment of the present invention first constructs a multidimensional feature space based on the sequence feature data generated in step S24. The dimensions of the feature space can include GC content, sequence length, repeat sequence characteristics, nucleotide distribution, etc., and a high-dimensional space is constructed by standardizing these features. Then, each sequence fragment in the gene sequence data is mapped to the multidimensional feature space to generate corresponding data points. Then, an unsupervised clustering algorithm (such as K-means or DBSCAN) is used to perform cluster analysis on these data points to find regions with similar potential patterns and characteristics in the gene sequence. Each clustering result represents a group of fragments with similar sequence characteristics, and finally generates sequence clustering result data. This method is widely used in large-scale genome data analysis and is suitable for distinguishing different gene regulatory regions or repeat sequence regions.

[0095] Step S312: Calculating the quality distribution characteristics of the sequence clustering result data to obtain quality distribution characteristic data, wherein the quality distribution characteristics include mean, variance, skewness and kurtosis;

[0096] The embodiment of the present invention calculates the quality distribution characteristics of the sequence fragments in each cluster according to the sequence clustering result data generated in step S311. Specifically calculate the statistics such as the mean, variance, skewness and kurtosis of the quality scores of the sequence fragments of each cluster. The quality score is based on the Phred score, and statistics of these features can help analyze the overall quality distribution of the sequence. For example, the mean reflects the average quality of the sequence in the cluster, the variance represents the discrete degree of the quality, and the skewness and kurtosis reflect the morphological characteristics of the quality distribution. Through these statistical features, the sequence quality stability of each cluster can be evaluated, and then provide a basis for subsequent quality threshold decision-making. This step can quickly evaluate the quality characteristics of the clustering results in large-scale genome sequence quality analysis.

[0097] Step S313: constructing an initial quality threshold decision model based on the quality distribution feature data, and optimizing the initial quality threshold decision model with the sequence quality improvement degree as a reward signal, thereby obtaining a quality threshold decision model;

[0098] The embodiment of the present invention constructs an initial quality threshold decision model based on the quality distribution feature data calculated in step S312. The core of the model is to determine an initial quality threshold based on the quality distribution characteristics of different clusters. The initial model can use a decision tree or a random forest algorithm to optimize the model by using the degree of sequence quality improvement as a reward signal. During the optimization process, if the sequence quality of a cluster is improved after filtering, the quality threshold of the cluster will be lowered; conversely, if the effect after filtering is not good, the quality threshold is appropriately increased. After multiple rounds of training, the model will gradually stabilize and eventually form a quality threshold decision model that can be adaptively adjusted according to the sequence feature data. This method helps to accurately adjust the filtering strategy and optimize data quality in high-throughput genome analysis.

[0099] Step S314: dynamically assigning quality thresholds to sequence segments with different feature combinations in the gene sequence data according to the quality threshold decision model, thereby obtaining adaptive quality threshold data.

[0100] The embodiment of the present invention dynamically allocates quality thresholds to sequence fragments of different feature combinations in gene sequence data according to the quality threshold decision model obtained in step S313. The specific operation is to allocate each sequence fragment to a corresponding cluster according to its feature, and then dynamically adjust the quality threshold in each cluster according to the output of the quality threshold decision model. For example, for clusters with abnormal GC content or more repetitive sequences, the threshold is appropriately reduced (such as reduced to a Phred score of 15), while for high-quality, non-abnormal feature fragments, a higher threshold is maintained (such as a Phred score of 20). By this dynamic allocation method, adaptive quality threshold data is generated to provide high-quality input data for subsequent gene sequence data processing and analysis. This step is applicable to quality control tasks in large-scale genome data processing.

[0101] The present invention can use a variety of sequence features (such as GC content, repeat sequences, functional elements, etc.) to comprehensively describe sequence fragments by constructing a multidimensional feature space. The construction of this high-dimensional feature space can effectively capture the potential similarities and hidden patterns in the sequence, and reveal the intrinsic connections between sequence fragments. Unsupervised clustering algorithms can automatically group gene sequence fragments according to the similarity between data points without pre-defined labels. This clustering method can discover the natural distribution and pattern between sequence fragments, help to further reveal the functional characteristics of different fragments and their roles in the genome, and provide important references for subsequent quality assessment and feature analysis. By calculating statistical features such as mean, variance, skewness and kurtosis, the quality distribution in each sequence clustering group can be quantified. These features can reflect the overall situation of sequence quality, such as whether there are significant high-quality or low-quality fragments, whether the sequence quality is concentrated at a certain level, or whether there are extreme situations (such as long tail effect). By analyzing these quality distribution features, the overall quality performance of gene sequences can be more accurately evaluated. Features such as variance and skewness can reveal the discrete degree of the quality of different sequence fragments, thereby providing a basis for the dynamic adjustment of subsequent quality thresholds. The initial quality threshold decision model is constructed based on the quality distribution feature data to ensure that it can make reasonable decisions based on the actual quality of each sequence fragment. This model based on actual data can adapt to the characteristics of different sequence fragments, thereby formulating more effective quality thresholds. The degree of sequence quality improvement is used as a reward signal to optimize the initial model, which enables the model to continuously adjust the threshold strategy in multiple iterations and automatically learn the quality patterns of different sequence fragments. As the optimization process progresses, the model can better predict and assign the optimal quality threshold, thereby improving the overall gene sequence processing effect. The quality threshold decision model can dynamically assign quality thresholds based on different feature combinations (such as GC content, repeat sequences, etc.), thereby achieving refined control in the quality assessment of each sequence fragment. Unlike the unified threshold setting, this adaptive strategy can adjust the threshold according to the specific situation, making the assessment more accurate and flexible. Adaptive quality threshold data ensures that each gene sequence fragment obtains the most appropriate quality control based on its unique characteristics. In this way, the efficiency of data processing can be improved, and the quality assessment results of each sequence fragment can be guaranteed to be more in line with the actual situation. This is particularly important for data quality control in large-scale genome analysis. By constructing a multidimensional feature space and using an unsupervised clustering algorithm, the intrinsic patterns and features of gene sequence fragments can be effectively discovered. Subsequently, the quality distribution feature calculation and decision model construction and optimization can be used to further improve the adaptability and accuracy of quality assessment. The effect of this process is mainly reflected in the dynamic adjustment of quality thresholds, the refined control of sequence fragment quality, and the improvement of data processing efficiency and accuracy. This method not only enhances the quality control capabilities of gene sequences, but also provides a more reliable high-quality data foundation for subsequent analysis.

[0102] Preferably, step S33 includes the following steps:

[0103] Step S331: Acquire a functional element feature library, which includes feature sequences of regulatory sequences, promoter sequences, and intron splicing sites;

[0104] The embodiment of the present invention first obtains a functional element feature library, which contains feature sequence information of known gene regulatory sequences, promoter sequences, and intron splicing sites. The data source of the functional element feature library can be a public genome database, such as Ensembl or UCSC genome browser, or a custom feature library constructed by literature mining. Each functional element contains detailed sequence patterns and features, such as the promoter region may have a TATA box or other conserved sequence features, which will be used for functional element identification of high-quality sequence data in subsequent steps.

[0105] Step S332: performing k-mer analysis with a k value of 3-8 on the high-quality sequence data, and counting the k-mer occurrence frequency, thereby generating a k-mer frequency matrix; performing local sequence complexity calculation on the high-quality sequence data, and obtaining sequence complexity distribution data;

[0106] According to the high-quality sequence data obtained in step S331, the embodiment of the present invention uses the k-mer analysis method to calculate the k-mer frequencies of different lengths, and the k value range is set to 3 to 8, indicating a subsequence of 3 bases to 8 bases in the gene sequence. Through the sliding window technology, the frequency of occurrence of these k-mers in the entire genome sequence is counted to generate a corresponding k-mer frequency matrix. Each element in the matrix represents the number of occurrences of a specific k-mer in the sequence. At the same time, the complexity of the local sequence is evaluated using Shannon entropy or other complexity calculation methods to generate sequence complexity distribution data. High complexity regions usually mean the non-repetitiveness of the sequence, and vice versa, it means the repetitiveness or conservatism of the sequence. These data will be used for subsequent sequence pattern recognition and analysis.

[0107] Step S333: performing self-alignment on the high-quality sequence data, identifying the repetitive patterns and conserved regions within the sequence, and generating repetitive pattern feature data;

[0108] The embodiment of the present invention utilizes the high-quality sequence data of step S332 to perform self-alignment to identify internal repetitive patterns and conserved regions in the gene sequence. The alignment algorithm can adopt BLAST or Smith-Waterman algorithm to generate repetitive pattern feature data by identifying the repetitive fragments inside the sequence. The repetitive pattern usually has functions in specific regions in the genome, such as satellite DNA or microsatellite regions, and the conservatism of these regions may imply potential functions. The conserved regions obtained by alignment can also be used to infer gene regulatory functions. The generated repetitive pattern feature data provide important basis for subsequent functional annotation and sequence pattern feature recognition.

[0109] Step S334: identifying known functional elements on the high-quality sequence data according to the functional element feature library, thereby obtaining functional element annotation data;

[0110] The embodiment of the present invention is based on the functional element feature library constructed in step S331, and the high-quality sequence data is identified with known functional elements. A sequence alignment algorithm (such as Bowtie or BWA) is used to match the high-quality sequence data with the functional elements in the feature library. The characteristic sequences of functional elements such as promoters or intron splicing sites will be marked and annotated to generate functional element annotation data. These annotation data will help distinguish regions with potential regulatory functions in the sequence, such as gene promoter regions that may be involved in transcriptional regulation, and intron splicing sites that determine the correct splicing and expression of genes. This step can effectively reveal the regulatory element layout of the genome.

[0111] Step S335: construct a multidimensional feature vector based on the k-mer frequency matrix, sequence complexity distribution data, repetitive pattern feature data and functional element annotation data, and perform t-SNE dimensionality reduction processing to obtain sequence pattern feature data.

[0112] The embodiment of the present invention combines the k-mer frequency matrix generated in step S332, the sequence complexity distribution data, the repeated pattern feature data generated in step S333, and the functional element annotation data generated in step S334 to construct a multidimensional feature vector. Each gene sequence fragment will be represented as a high-dimensional feature vector based on these features. In order to facilitate subsequent analysis, t-SNE (t-Distributed Stochastic Neighbor Embedding) will be used for dimensionality reduction processing, and the high-dimensional data will be mapped into a two-dimensional or three-dimensional space to generate sequence pattern feature data. t-SNE can retain the local structural features of high-dimensional data, so that fragments with similar sequence patterns are more easily gathered together in low-dimensional space. In this way, the obtained sequence pattern feature data can reveal the potential function and structural relationship of the sequence fragment.

[0113] The functional element feature library of the present invention contains known functional regions in the genome, such as regulatory sequences, promoters, splicing sites, etc. These sequences play an important role in biological processes such as gene regulation and expression. By citing these functional features, a clear biological basis can be provided for subsequent analysis, and support can be provided for feature annotation in sequence pattern recognition. By introducing an external functional element feature library, the recognition accuracy of specific functional regions in gene sequences can be greatly improved; this lays the foundation for subsequent functional analysis and helps to discover potential gene regulatory mechanisms and gene functions. K-mer analysis is an effective means of sequence pattern recognition. By calculating the frequency of occurrence of k-mers of different lengths (k value 3-8) in the sequence, short segment repetitions and pattern information in the gene sequence can be captured. The generated k-mer frequency matrix provides comprehensive basic data for subsequent sequence feature analysis. By calculating the local sequence complexity, the degree of information redundancy in the gene sequence can be measured. The higher the sequence complexity, the richer the amount of information contained in the sequence, and the region with lower complexity may be a repeated region or a conserved region. Sequence complexity distribution data can help distinguish functionally active regions from redundant regions. By self-alignment, self-repeating fragments and conserved regions in the gene sequence can be identified. These repeated and conserved regions may have important functions in evolution, and are often related to gene expression regulation, structural maintenance, etc. The generated repeat pattern feature data provides a basis for subsequent functional element identification and annotation. Self-alignment reveals the structural features within the gene sequence, helping to identify regions in the sequence with important functions or structural effects, such as highly conserved gene regulatory regions. This helps to further understand the functional organization and regulatory mechanism of gene sequences. By comparing the functional element feature library, known functional elements (such as promoters, regulatory sequences, etc.) in the gene sequence can be quickly identified, thereby generating accurate functional annotation data. This not only provides important biological information for sequence feature analysis, but also provides data support for understanding the potential functions and regulatory mechanisms of gene sequences. The identification of known functional elements helps to provide comparative references for unknown regions in gene sequences, allowing researchers to better identify potential functional regions and further explore new gene regulatory elements. By combining k-mer frequency, sequence complexity, repeat pattern and functional element annotation data, a multidimensional feature vector is constructed, which can fully describe the characteristics of gene sequences. This feature fusion method can more comprehensively reveal complex patterns in sequences and help discover potential sequence patterns and functional relationships. t-SNE dimensionality reduction processing can effectively reduce the dimension of feature data, while maintaining the main feature information, reducing the computational complexity, making subsequent pattern recognition and analysis more efficient. After dimensionality reduction processing, the sequence pattern feature data can more intuitively display the clustering structure and pattern characteristics of the gene sequence, facilitating further analysis and research.The above steps reveal the pattern characteristics of gene sequences in all aspects through the reference of functional element feature library, k-mer analysis, complexity calculation, self-alignment and other means. Through the construction of multi-dimensional feature vectors and dimensionality reduction processing, various feature information can be integrated to discover important patterns and functional regions in the sequence. This method not only improves the accuracy and efficiency of pattern recognition, but also provides strong support for subsequent biological function research.

[0114] Preferably, step S34 includes the following steps:

[0115] Step S341: performing sequence subclass identification on the sequence pattern feature data, thereby obtaining sequence subclass data;

[0116] The embodiment of the present invention analyzes the sequence pattern feature data and uses an unsupervised classification algorithm (such as k-means or DBSCAN) to identify different subclasses in the gene sequence. Each sequence fragment is classified according to its feature vector to generate corresponding sequence subclass data. The purpose of this step is to divide the gene sequence into different subclasses, such as conserved regions, functional element regions, and repeat sequence regions. The algorithm will cluster these fragments together according to the similarity of the sequence pattern features, thereby achieving effective classification of the sequence subclasses. The classification results will be used for subsequent feature association analysis and weight adjustment.

[0117] Step S342: Calculating the occurrence frequency of each type of nucleotide in each window based on the sliding window for the high-quality sequence data, thereby generating a nucleotide frequency distribution diagram;

[0118] The embodiment of the present invention performs sliding window analysis on high-quality sequence data, and the size of the sliding window can be set to 100 to 500 bases, which can be adjusted according to the specific scenario. For each sliding window, the frequency of occurrence of nucleotides such as A, T, C, and G is counted to generate a nucleotide frequency distribution map. The distribution map will show the relative frequency changes of nucleotides in different windows, helping to identify the local distribution pattern of nucleotides in the sequence. This method is often used to detect the distribution of repetitive fragments, low-complexity regions, or functional elements that may exist in the sequence.

[0119] Step S343: Perform Fourier transform-based periodicity analysis on the nucleotide frequency distribution graph, and identify potential periodic patterns to obtain sequence periodicity feature data;

[0120] The embodiment of the present invention performs Fourier transform (FFT) processing on the nucleotide frequency distribution diagram generated in step S342 to identify periodic signals therein. Fourier transform can convert time domain signals into frequency domain signals, thereby revealing potential periodic patterns in the sequence. This method can be used to identify periodically repeated DNA sequences or potential gene regulatory elements. Periodic feature data can provide clues to regulatory structures hidden in the sequence, such as periodic patterns of satellite DNA or simple repeat sequences.

[0121] Step S344: performing multi-dimensional tensor decomposition analysis on the sequence period feature data and the sequence subclass data to obtain nucleotide-sequence subclass association data;

[0122] In the embodiment of the present invention, the sequence periodic feature data obtained in step S343 and the sequence subclass data in step S341 are subjected to multi-dimensional tensor decomposition analysis. Tensor decomposition is a method of extending matrix decomposition for processing multi-dimensional data. In this step, the periodic feature is combined with the sequence subclass feature by tensor decomposition to generate nucleotide-sequence subclass association data. This data is used to explore the correlation between different subclass sequences and specific nucleotide patterns or periodic distributions, providing a basis for subsequent sequence pattern optimization.

[0123] Step S345: weighting the sequence pattern feature data based on the nucleotide-sequence subclass association data to obtain weighted sequence pattern feature data;

[0124] According to the nucleotide-sequence subclass association data obtained in step S344, the embodiment of the present invention uses weighting factors to adjust the sequence pattern feature data to generate weighted sequence pattern feature data. The weighting factors are allocated according to the degree of association between different nucleotide-subclasses. For example, a specific subclass may have a higher correlation with a specific nucleotide frequency pattern, and the weight will be increased accordingly. This step improves the accuracy of sequence pattern recognition through weight adjustment and optimizes the functional annotation effect of different subclasses.

[0125] Step S346: extracting regions whose deviation from the background distribution is greater than a preset deviation threshold according to the nucleotide frequency distribution map, thereby generating nucleotide abnormal region data;

[0126] The embodiment of the present invention identifies regions that deviate significantly from the background distribution according to the nucleotide frequency distribution diagram generated in step S342. A preset deviation threshold, such as a multiple of the standard deviation, is set. When the nucleotide frequency in a certain region deviates significantly from the background distribution, these regions are extracted and nucleotide abnormal region data are generated. This method can help identify potential variant regions or mutation hotspots in the sequence, thereby providing strong support for subsequent sequence analysis.

[0127] Step S347: performing abnormal pattern recognition and elimination processing on the weighted sequence pattern feature data according to the nucleotide abnormal region data, thereby obtaining weighted feature data.

[0128] According to the abnormal nucleotide region data of step S346, the embodiment of the present invention uses an abnormal pattern recognition algorithm (such as an abnormal detection based on a support vector machine) to perform abnormal elimination processing on the weighted sequence pattern feature data. The purpose of this step is to filter out abnormal patterns in the sequence to ensure the accuracy of the analysis results. After the abnormal pattern is eliminated, the final weighted feature data is generated.

[0129] The present invention can divide the gene sequence into multiple subclasses with similar characteristics through sequence subclass identification of sequence pattern feature data; this helps to distinguish sequence fragments with different functions or structural characteristics, thereby providing a more fine-grained perspective for further analysis. After the complex sequence data is divided into different subclasses, the characteristics of different types of sequences can be captured more clearly, which provides a more accurate basis for subsequent functional prediction and sequence pattern analysis. Sliding window analysis can generate a detailed nucleotide frequency distribution map by calculating the frequency of occurrence of various types of nucleotides in the local area. This can reveal regional nucleotide enrichment or sparse features in the sequence, which helps to identify potential functional regions or regulatory elements. By analyzing the frequency of nucleotides in the window, the changes in local features in the sequence can be more intuitively observed, helping to identify important sequence patterns, such as regulatory elements, repeated sequences or structural elements. Fourier transform is a powerful signal processing tool that can reveal periodic patterns that may exist in gene sequences by performing periodic analysis on nucleotide frequency distribution maps. These patterns may be related to gene regulation or structural functions, especially in highly conserved gene fragments. The identification of periodic features can help analyze sequences with repetitive properties or specific functions and improve the accuracy of subsequent sequence pattern recognition, especially for the discovery of gene regulatory regions. Through multidimensional tensor decomposition analysis, sequence periodic features can be associated with sequence subclass data. This process can discover hidden association patterns in high-dimensional data and reveal the connection between nucleotide distribution and specific sequence subclasses. Tensor decomposition can effectively reduce the data dimension while retaining important information, helping to extract valuable patterns and trends from multidimensional features, so as to better understand the association between nucleotide distribution and sequence function. By adjusting the weights of sequence pattern features through nucleotide-sequence subclass association data, the recognition weights of features that are more important for biological function or structure can be increased; this makes the feature extraction process more accurate and helps to highlight the biological significance of key sequence regions. Weight adjustment can improve the performance of the model in sequence pattern recognition, making subsequent functional prediction more efficient and accurate. By setting a predetermined deviation threshold, nucleotide regions that deviate from the normal background distribution can be extracted; these abnormal regions may indicate potential functional regions or regulatory elements, especially in gene sequences with functional mutations or structural changes. This process helps to screen out those abnormal distribution regions with important biological significance and provide more accurate positioning for subsequent biological function research. By performing abnormal pattern recognition and elimination on weighted sequence pattern feature data, false positive patterns that do not have biological functions can be eliminated to ensure that the final weighted feature data is more biologically meaningful and functionally relevant. The elimination process can reduce noise and misleading features, making the final weighted feature data more reliable and providing a solid foundation for subsequent gene function annotation and prediction.Through a series of data processing methods such as sequence pattern feature analysis, nucleotide frequency distribution and periodicity analysis, tensor decomposition, etc., the accuracy and efficiency of sequence feature recognition have been comprehensively improved. The effect of these steps is reflected in the accurate capture of important patterns and features in the sequence through multi-dimensional feature analysis and abnormal region detection, providing strong data support for gene function prediction and structure analysis.

[0130] Preferably, step S4 comprises the following steps:

[0131] Step S41: performing genome alignment on the gene sequence data according to a preset genome library to generate alignment result data;

[0132] The embodiment of the present invention uses a preset genome library (such as a reference human genome or other species genome) to compare gene sequence data. The comparison tool can use efficient gene comparison software such as BWA, Bowtie or HISAT2. During the comparison process, the gene sequence fragments are accurately matched or partially matched with the genome library to generate comparison result data. The comparison result includes the matching position of the sequence, the matching score, and the statistical information of the unmatched fragments, which will be used for subsequent variant identification and analysis.

[0133] Step S42: integrating the adaptive quality threshold data and the comparison result data based on multi-dimensional Bayesian fusion to obtain heterogeneous integrated feature data;

[0134] The embodiment of the present invention integrates the adaptive quality threshold data generated in step S31 with the comparison result data obtained in step S41, and uses a multi-dimensional Bayesian fusion model to handle the heterogeneity of the data. Bayesian fusion can combine data features from different sources and assign weights according to their respective confidence levels to generate heterogeneous integrated feature data. This process enhances the ability to detect potential variant sites in the sequence by fusing the adaptive quality threshold and the position information in the comparison result.

[0135] Step S43: Preliminary identification of mutation sites in gene sequence data based on heterogeneity integration feature data to generate candidate mutation data;

[0136] The embodiment of the present invention preliminarily identifies potential variant sites in the gene sequence based on the heterogeneous integrated feature data generated in step S42. Use a variation detection algorithm such as GATK or VarScan to scan the integrated data and identify possible single nucleotide variations (SNVs) and insertions and deletions (INDELs). During the preliminary identification process, a certain variation confidence threshold is set to exclude noise and generate candidate variation data. The candidate data will be used for credibility screening in subsequent steps to ensure the accuracy of variation identification.

[0137] Step S44: screening the candidate variant data for credibility by weighted feature data, thereby obtaining credible variant site data.

[0138] The embodiment of the present invention uses weighted feature data to perform credibility screening on the candidate variant data generated in step S43. First, the weighted scores of the candidate variant sites are calculated, and the sequence pattern features are combined with the functional element features. By comprehensively considering the functional relevance, variation pattern, and nucleotide distribution characteristics, highly credible variant sites are screened. Candidate variants with scores below the set threshold will be eliminated, and finally credible variant site data will be generated to provide high-quality variation information for subsequent in-depth analysis.

[0139] The genome alignment of the present invention is the basis of variation detection. By comparing the gene sequence data with the reference genome library, the difference region in the gene sequence can be quickly determined; this is a key step in discovering gene variations such as single nucleotide polymorphisms (SNPs) and insertions / deletions (Indels). The alignment results provide the corresponding relationship between the gene sequence and the reference genome, so that subsequent variation analysis can be performed in the context of known gene functions. This analysis based on the reference sequence helps to interpret the potential biological significance of the variation. The adaptive quality threshold data reflects the sequence quality, and the alignment results provide the similarity between the sequence and the reference genome. Through multi-dimensional Bayesian fusion, these heterogeneous data are effectively integrated, thereby reducing the noise and uncertainty in the data and improving the accuracy of variation identification. Bayesian fusion can dynamically adjust the weights of different data sources by updating the prior probability; this flexible integration strategy makes the variation detection process more robust, especially when dealing with complex or low-quality sequence data. Based on heterogeneous integration feature data, potential variation sites in gene sequences can be effectively identified. Since the feature data combines the quality threshold information and the alignment results, the initially identified variant data is more reliable and can significantly reduce false positive results caused by low-quality data or alignment errors. Under the premise of ensuring quality, the initially identified candidate variant data has a wider coverage, covering all possible variant regions in the sequence, laying a good foundation for subsequent credibility screening. Through credibility screening with weighted feature data, the reliability of candidate variants can be comprehensively evaluated based on multiple features (such as sequence patterns, quality distribution, nucleotide features, etc.). This process ensures that the variant sites finally screened have high accuracy and biological significance. The weighted feature data combines feature information from different dimensions, such as k-mer analysis, sequence complexity, etc., which can effectively identify and eliminate variant sites with low credibility, reduce false positives and redundant results, and improve the accuracy of the final variant site data. The screened credible variant data has a high degree of credibility and can provide reliable basic data support for subsequent biological function analysis, disease association studies, etc.

[0140] Preferably, step S5 comprises the following steps:

[0141] Step S51: performing parallel analysis using multiple variation detection algorithms based on the credible variation site data and high-quality sequence data, thereby obtaining multi-algorithm variation detection result data;

[0142] The embodiment of the present invention uses multiple variation detection algorithms for parallel analysis based on the credible variation site data and high-quality sequence data obtained in step S44. Commonly used algorithms include GATK, Samtools, DeepVariant, etc. These algorithms each have different detection sensitivities and error handling capabilities. By running multiple algorithms simultaneously, variation detection from multiple angles can be performed on the same set of data, thereby reducing missed detections and false detections. By integrating the detection results of these algorithms, multi-algorithm variation detection result data is generated, including the variation sites detected by each algorithm and related information.

[0143] Step S52: performing integrated analysis based on machine learning on the multi-algorithm variation detection result data, thereby generating preliminary variation detection result data;

[0144] The embodiment of the present invention inputs the multi-algorithm variation detection result data generated in step S51 into an integrated analysis model based on machine learning. The model integrates the results of multiple algorithms. Commonly used methods include random forests, XGBoost, etc. These models can dynamically adjust weights and generate comprehensive results based on the historical detection performance of each algorithm and the current data characteristics. Finally, the preliminary variation detection result data generated by the machine learning model includes verified and integrated variation sites, and is accompanied by its confidence score, which further improves the accuracy of variation detection.

[0145] Step S53: performing rare variation identification based on rare variation frequency calculation on the preliminary variation detection result data, thereby obtaining rare variation data;

[0146] The embodiment of the present invention performs rare variant identification on the preliminary variant detection result data generated in step S52. Public variant frequency databases such as gnomAD, ExAC, etc. are used to compare the detected variant sites and calculate the frequency of occurrence of each variant in different populations. For variant sites with an occurrence frequency lower than a set threshold (such as 0.01%), they are marked as rare variants. The identification of rare variant data helps to locate potential functional or pathogenic variants, especially in the study of rare genetic diseases.

[0147] Step S54: using a local assembly algorithm to perform a variation structure analysis on the potential structural variation regions in the preliminary variation detection result data, thereby obtaining structural variation candidate data;

[0148] The embodiment of the present invention uses a local assembly algorithm to conduct an in-depth analysis of the structural variation regions that may exist in the preliminary variation detection result data generated in step S52. Local assembly can reassemble complex structural variations such as insertions, deletions, and inversions in the sequence. Commonly used tools include LUMPY, Manta, etc. The algorithm reassembles local regions of the original sequence data to generate structural variation candidate data, helping to identify the location and specific type of complex structural variations.

[0149] Step S55: classifying and verifying the candidate structural variation data to obtain credible structural variation data;

[0150] The embodiment of the present invention performs classification verification processing on the structural variation candidate data generated in step S54. First, by comparing with the structural variation database (such as DGV), the known structural variation is identified. At the same time, the functional annotation tool (such as SnpEff) is used to predict the functional impact of these variations, and a comprehensive evaluation is performed in combination with the sequence depth, variation position and sample characteristics to screen out credible structural variation candidates, and finally generate credible structural variation data for subsequent analysis.

[0151] Step S56: performing redundant information removal and consistency verification on the preliminary variation detection result data according to the rare variation data and the trusted structural variation data, so as to obtain the final variation detection data.

[0152] The embodiment of the present invention removes redundant information and performs consistency verification on the preliminary variation detection result data based on the rare variation data generated in step S53 and the trusted structural variation data generated in step S55. First, the repeatedly detected variation sites and redundant information are removed to simplify the data. Then, all remaining variation are verified for consistency to ensure the consistency of the variation site detection results between different algorithms and data sources, and to remove abnormal results with significant differences. After this series of processing, accurate and non-redundant final variation detection data is finally obtained, which can be used for clinical or scientific research applications.

[0153] The present invention can utilize different algorithm characteristics and advantages on the same data set to cover more types of mutations (such as SNP, Indel, structural variation, etc.), thereby improving the comprehensiveness and accuracy of mutation detection by applying multiple mutation detection algorithms in parallel. Different algorithms may have different assumptions and limitations when processing data. By combining the results of multiple algorithms, the bias caused by a single algorithm can be reduced, thereby obtaining more objective and accurate mutation detection results. By using machine learning models for integrated analysis, the detection results of different algorithms can be fully utilized, and the accuracy and reliability of mutation detection can be further improved by learning patterns and features in the data. Integrated learning methods can usually effectively improve the performance of the model and reduce misjudgment. Through the training and optimization of machine learning models, the weights and decision strategies of the algorithm can be automatically adjusted, the adaptability of the system in a complex data environment can be enhanced, manual intervention can be reduced, and the analysis efficiency can be improved. Rare mutations are usually associated with specific diseases or individual characteristics. Identifying these mutations can provide important information for subsequent biological research and clinical applications, and help personalized medicine and genetic research; by calculating the frequency of rare mutations, the test results can be compared with known databases to identify those mutations with extremely low frequencies, thereby reducing false positive results and improving the reliability of mutation identification. Local assembly algorithms can perform detailed analysis within the identified potential structural variation regions, revealing the specific characteristics of structural variation, such as insertions, deletions, or rearrangements. This process helps to gain a deeper understanding of genomic changes and their biological significance; local assembly methods can usually effectively capture small structural variations in a complex genomic background, thereby improving the overall recognition sensitivity. By classifying and verifying candidate data, true structural variations can be effectively screened out, and false positive or low-quality candidate data can be excluded; this verification process helps to improve the credibility of the final results. The obtained credible structural variation data can provide a solid data foundation for subsequent biological research, functional verification, and clinical applications, ensuring the reliability of research conclusions. By removing redundant information, the final variation detection results can be made more concise and clear, which is helpful for subsequent data analysis and interpretation. Such processing can avoid repeated calculation and analysis of information and improve data processing efficiency. Consistency verification can confirm the results in a variety of ways to ensure that the final variation data is highly accurate and credible. This is of great value for subsequent scientific research and clinical applications.

[0154] The present invention also provides an artificial intelligence-based gene detection data cleaning system, which is used to execute the artificial intelligence-based gene detection data cleaning method mentioned above. The artificial intelligence-based gene detection data cleaning system includes:

[0155] A data preprocessing module is used to obtain gene detection data, perform preliminary filtering on the gene detection data based on removing low-quality reads, and obtain preliminary filtered data; identify and remove adapter sequences on the preliminary filtered data, and perform sequencing error correction to generate gene sequence data;

[0156] The sequence feature extraction module is used to perform GC content analysis and repetitive sequence identification on gene sequence data to obtain sequence feature data;

[0157] The dynamic quality optimization module is used to adjust the dynamic quality threshold according to the sequence feature data, and perform secondary filtering on the gene sequence data to generate high-quality sequence data; perform sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; perform nucleotide distribution analysis on the sequence feature data, and perform weight adjustment on the sequence pattern feature data to generate weighted feature data;

[0158] The variant site screening module is used to perform genome comparison on gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the comparison results, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on candidate variant data through weighted feature data, so as to obtain credible variant site data;

[0159] The variation confirmation module is used to perform variation site confirmation based on the trusted variation site data and high-quality sequence data using multiple algorithms to generate preliminary variation detection result data; perform rare variation and structural variation analysis on the preliminary variation detection result data to obtain the final variation detection data.

[0160] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0161] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A gene detection data cleaning method based on artificial intelligence, characterized in that: The following steps are involved: Step S1: Obtaining gene detection data, performing preliminary filtering on the gene detection data based on removing low-quality reads to obtain preliminary filtered data; performing linker sequence identification and removal on the preliminary filtered data, and performing sequencing error correction to generate gene sequence data; Step S2: GC content analysis and repetitive sequence identification are performed on the gene sequence data to obtain sequence feature data; Step S3: dynamically adjusting the quality threshold according to the sequence feature data, and performing secondary filtering on the gene sequence data to generate high-quality sequence data; performing sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; Performing nucleotide distribution analysis on the sequence feature data and weight adjustment on the sequence pattern feature data to generate weighted feature data; Step S4: perform genome alignment on the gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the alignment result, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on the candidate variant data by weighted feature data, so as to obtain credible variant site data; Step S5: Confirm the variant sites using multiple algorithms based on the reliable variant site data and high-quality sequence data to generate preliminary variant detection result data; The preliminary mutation detection result data were analyzed for rare mutations and structural mutations to obtain the final mutation detection data.

2. The method for cleaning genetic testing data based on artificial intelligence according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Obtaining genetic testing data in FASTQ format; Step S12: Perform integrity check on the gene detection data based on the data file size, number of lines and sequence length distribution, and remove invalid files to obtain valid gene detection data; Step S13: Calculate the Phred quality score of each base for the valid gene detection data, thereby obtaining sequencing quality distribution data; Step S14: performing preliminary filtering on the effective gene detection data based on removing low-quality reads according to the sequencing quality distribution data to obtain preliminary filtered data; Step S15: performing linker sequence identification based on a sequence alignment algorithm on the preliminary filtered data, and removing the linker sequence from the read segment, thereby obtaining linker-free sequence data; Step S16: quickly comparing the removed linker sequence data according to the preset contamination source data, identifying and removing the contamination sequence, thereby obtaining purified sequence data; Step S17: Perform sequencing error correction on the purified sequence data using an error correction algorithm based on k-mer frequency to obtain corrected gene sequence data.

3. The method for cleaning genetic testing data based on artificial intelligence according to claim 2, characterized in that: Step S2 includes the following steps: Step S21: performing sliding window analysis on the gene sequence data and calculating the GC content of each window to obtain GC content distribution data; Step S22: identifying abnormal GC content regions based on the GC content distribution data, and generating GC abnormal region marking data; Step S23: Scan the gene sequence data, identify and mark the repeated sequence region, and obtain repeated sequence feature data; Step S24: feature fusion of the GC abnormal region marking data and the repeated sequence feature data to obtain sequence feature data.

4. The method for cleaning genetic testing data based on artificial intelligence according to claim 3, characterized in that: Step S3 includes the following steps: Step S31: dynamically adjusting the quality threshold according to the sequence feature data, thereby obtaining adaptive quality threshold data; Step S32: performing secondary filtering on the gene sequence data according to the adaptive quality threshold data to remove sequence fragments that do not meet the quality requirements, thereby obtaining high-quality sequence data. Step S33: performing sequence pattern recognition based on functional element features on the high-quality sequence data, thereby obtaining sequence pattern feature data; Step S34: performing nucleotide distribution analysis on the sequence feature data and weight adjustment on the sequence pattern feature data to generate weighted feature data.

5. The method for cleaning genetic testing data based on artificial intelligence according to claim 4, characterized in that: Step S31 includes the following steps: Step S311: construct a multidimensional feature space according to the sequence feature data, map the sequence fragments in the gene sequence data into the multidimensional feature space, perform clustering analysis on the data points based on an unsupervised clustering algorithm, and obtain sequence clustering result data; Step S312: Calculating the quality distribution characteristics of the sequence clustering result data to obtain quality distribution characteristic data, wherein the quality distribution characteristics include mean, variance, skewness and kurtosis; Step S313: constructing an initial quality threshold decision model based on the quality distribution feature data, and optimizing the initial quality threshold decision model with the sequence quality improvement degree as a reward signal, thereby obtaining a quality threshold decision model; Step S314: dynamically assigning quality thresholds to sequence segments with different feature combinations in the gene sequence data according to the quality threshold decision model, thereby obtaining adaptive quality threshold data.

6. The method for cleaning genetic testing data based on artificial intelligence according to claim 5, characterized in that: Step S33 includes the following steps: Step S331: Acquire a functional element feature library, which includes feature sequences of regulatory sequences, promoter sequences, and intron splicing sites; Step S332: performing k-mer analysis with a k value of 3-8 on the high-quality sequence data, and counting the k-mer occurrence frequency, thereby generating a k-mer frequency matrix; performing local sequence complexity calculation on the high-quality sequence data, and obtaining sequence complexity distribution data; Step S333: performing self-alignment on the high-quality sequence data, identifying the repetitive patterns and conserved regions within the sequence, and generating repetitive pattern feature data; Step S334: identifying known functional elements on the high-quality sequence data according to the functional element feature library, thereby obtaining functional element annotation data; Step S335: construct a multidimensional feature vector based on the k-mer frequency matrix, sequence complexity distribution data, repetitive pattern feature data and functional element annotation data, and perform t-SNE dimensionality reduction processing to obtain sequence pattern feature data.

7. The method for cleaning genetic testing data based on artificial intelligence according to claim 6, characterized in that: Step S34 includes the following steps: Step S341: performing sequence subclass identification on the sequence pattern feature data, thereby obtaining sequence subclass data; Step S342: Calculating the occurrence frequency of each type of nucleotide in each window based on the sliding window for the high-quality sequence data, thereby generating a nucleotide frequency distribution diagram; Step S343: Perform Fourier transform-based periodicity analysis on the nucleotide frequency distribution graph, and identify potential periodic patterns to obtain sequence periodicity feature data; Step S344: performing multi-dimensional tensor decomposition analysis on the sequence period feature data and the sequence subclass data to obtain nucleotide-sequence subclass association data; Step S345: weighting the sequence pattern feature data based on the nucleotide-sequence subclass association data to obtain weighted sequence pattern feature data; Step S346: extracting regions whose deviation from the background distribution is greater than a preset deviation threshold according to the nucleotide frequency distribution map, thereby generating nucleotide abnormal region data; Step S347: performing abnormal pattern recognition and elimination processing on the weighted sequence pattern feature data according to the nucleotide abnormal region data, thereby obtaining weighted feature data.

8. The method for cleaning genetic testing data based on artificial intelligence according to claim 7, characterized in that: Step S4 includes the following steps: Step S41: performing genome alignment on the gene sequence data according to a preset genome library to generate alignment result data; Step S42: integrating the adaptive quality threshold data and the comparison result data based on multi-dimensional Bayesian fusion to obtain heterogeneous integrated feature data; Step S43: Preliminary identification of mutation sites in gene sequence data based on heterogeneity integration feature data to generate candidate mutation data; Step S44: screening the candidate variant data for credibility by weighted feature data, thereby obtaining credible variant site data.

9. The method for cleaning genetic testing data based on artificial intelligence according to claim 8, characterized in that: Step S5 includes the following steps: Step S51: performing parallel analysis using multiple variation detection algorithms based on the credible variation site data and high-quality sequence data, thereby obtaining multi-algorithm variation detection result data; Step S52: performing integrated analysis based on machine learning on the multi-algorithm variation detection result data, thereby generating preliminary variation detection result data; Step S53: performing rare variation identification based on rare variation frequency calculation on the preliminary variation detection result data, thereby obtaining rare variation data; Step S54: using a local assembly algorithm to perform a variation structure analysis on the potential structural variation regions in the preliminary variation detection result data, thereby obtaining structural variation candidate data; Step S55: classifying and verifying the candidate structural variation data to obtain credible structural variation data; Step S56: performing redundant information removal and consistency verification on the preliminary variation detection result data according to the rare variation data and the trusted structural variation data, so as to obtain the final variation detection data.

10. A genetic testing data cleaning system based on artificial intelligence, characterized in that: Used to perform the artificial intelligence-based gene detection data cleaning method according to claim 1, the artificial intelligence-based gene detection data cleaning system comprises: A data preprocessing module is used to obtain gene detection data, perform preliminary filtering on the gene detection data based on removing low-quality reads, and obtain preliminary filtered data; identify and remove adapter sequences on the preliminary filtered data, and perform sequencing error correction to generate gene sequence data; The sequence feature extraction module is used to perform GC content analysis and repetitive sequence identification on gene sequence data to obtain sequence feature data; The dynamic quality optimization module is used to adjust the dynamic quality threshold according to the sequence feature data, and perform secondary filtering on the gene sequence data to generate high-quality sequence data; perform sequence pattern recognition on the high-quality sequence data to obtain sequence pattern feature data; perform nucleotide distribution analysis on the sequence feature data, and perform weight adjustment on the sequence pattern feature data to generate weighted feature data; The variant site screening module is used to perform genome comparison on gene sequence data according to the preset genome library, integrate the adaptive quality threshold data with the comparison results, perform preliminary identification of variant sites, and generate candidate variant data; perform credibility screening on candidate variant data through weighted feature data, so as to obtain credible variant site data; The variation confirmation module is used to perform variation site confirmation based on the trusted variation site data and high-quality sequence data using multiple algorithms to generate preliminary variation detection result data; perform rare variation and structural variation analysis on the preliminary variation detection result data to obtain the final variation detection data.

Images