Structural variation real-time detection system and method

Abstract

The invention discloses a structure variation real-time detection system and method, and relates to the field of structure variation real-time detection, in particular to the structure variation real-time detection system and method. The invention aims to solve the problems of high cost and difficulty in realizing real-time detection under the condition of limited sequencing and computing resources due to calculation separation of sequencing in the existing real-time structural variation detection. The system comprises a file monitoring and task queue management module which is used for monitoring a newly generated FASTQ file and adding the newly generated file into a task queue; the preliminary SV feature extraction module is used for extracting preliminary features of structural variation from the files in the task queue; the SV clustering and genetic typing module is used for carrying out SV clustering and genetic typing on the extracted preliminary features of the structural variation and outputting an SV detection result in a VCF format; and the real-time feedback mechanism module is used for comparing the SV detection result with the target SV set and calculating the proportion covered by the current detection result in the target SV set.

Description

Technical Field

[0001] The present invention relates to a real-time detection system and method for structural variation. Background Art

[0002] Nanopore sequencing technology, characterized by long read lengths and real-time sequencing, has become one of the most widely used technologies in cutting-edge genomics, clinical diagnostics, and related fields. Compared with traditional second-generation sequencing, the long read length advantage of nanopore sequencing significantly improves the ability to detect genomic structural variations. Currently, structural variation (SV) detection algorithms developed specifically for nanopore sequencing include cuteSV, Sniffles2, SVIM, etc. After collecting complete sequencing data, these methods sequentially perform sequence alignment, signal extraction, and feature clustering to detect structural variations, playing an important role in understanding the distribution of structural variations in the human genome and its relationship with health and disease.

[0003] However, these methods generally adopt a serial "sequencing first, computing later" process, that is, sequencing is performed first, then analysis is performed. Therefore, it is difficult to meet the requirements of real-time applications such as clinical diagnosis that require rapid response. This delay hinders the timely identification of key variant information. Some studies have attempted to address this problem. For example, some scholars have proposed to speed up sequencing output by stacking sequencing instruments, expanding computing resources, and using cloud storage to reduce data transmission time; others have proposed a skeleton-based structural variation detection toolkit that uses alignment skeletons as input to achieve rapid read mapping. However, these methods still do not fundamentally break the separation of sequencing and computing, and are expensive, making them difficult to promote and use under conditions of limited sequencing and computing resources. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that sequencing and computing in existing real-time detection of structural variations are separated, which is costly and difficult to achieve real-time detection under conditions of limited sequencing and computing resources, and to propose a real-time detection system and method for structural variation.

[0005] A real-time structural variation detection system comprises: A real-time structural variation detection system includes: a file monitoring and task queue management module, a preliminary SV feature extraction module, an SV clustering and genotyping module, and a real-time feedback mechanism module; The file monitoring and task queue management module is used to monitor newly generated FASTQ files and add the newly generated FASTQ files to the task queue; The preliminary SV feature extraction module is used to extract preliminary features of structural variations from the FASTQ files in the task queue; The SV clustering and genotyping module is used to perform SV clustering and genotyping on the extracted preliminary features of structural variations and output SV detection results in VCF format; The real-time feedback mechanism module is used to compare the SV detection result with the target SV set and calculate the proportion of the target SV set covered by the current detection result.

[0006] A method for real-time detection of structural variations has the following specific steps: Step 1: Use the watchdog file monitoring module developed in Python to automatically detect newly generated FASTQ files in the user-specified directory and register the detected new FASTQ files in the pending task queue in chronological order; Step 2: Use minimap2 to align the first FASTQ file in the pending task queue with the human reference genome, and remove the first FASTQ file from the queue; minimap2 determines whether a sequence alignment failure occurs; If no failure occurs, a SAM format alignment result is obtained. The alignment result contains the position of the bases in the FASTQ file on the human reference genome, and step 3 is performed; If a failure occurs, the incomplete reads in the current FASTQ file are discarded, and the current FASTQ file after discarding the incomplete reads is re-aligned to obtain a SAM format alignment result. The alignment result contains the position of the bases in the FASTQ file on the human reference genome, and step 3 is executed; Step 3: Use samtools to sort the SAM format alignment results from front to back according to their positions on the human reference genome to obtain the sorted SAM format alignment file; Use samtools to convert the sorted SAM format alignment results to generate a BAM format alignment file, and execute steps 4 and 5; Step 4: Use the pandepth tool to calculate the sequencing depth of the BAM format alignment file generated in step 2; Step 5: Use cuteSV tool to extract structural variation feature signals from the BAM format alignment file generated in step 3; The cuteSV tool determines whether there is a fault in the structural variation feature signal in the BAM format alignment file generated in step 3 of the extraction; If a failure occurs, the structural variation feature signal extracted from the BAM format comparison file generated in step 3 is discarded, and the structural variation feature signal is extracted again from the BAM format comparison file generated in step 3; If no fault occurs, the extracted structural variation feature signal is used as the preliminary feature of the structural variation; Step 6: Determine whether the number of BAM format comparison files extracted in step 5 reaches the clustering trigger condition; If reached, proceed to step seven; If not, re-execute steps 2 to 5 until the number of BAM format comparison files extracted in step 5 reaches the clustering trigger condition; Step 7: Use the cuteSV feature clustering module to cluster and genotype the preliminary features of structural variations corresponding to all extracted BAM format comparison files, and output the SV detection results in VCF format; cuteSV determines whether the clustering fails; If clustering fails, the feature clustering module of cuteSV is reused to cluster and genotype the preliminary features of structural variations extracted from all BAM format alignment files, and the real-time SV detection results in VCF format are output; If the clustering does not fail, the real-time SV detection results in VCF format are output; Step 8: Compare the real-time SV detection results in the VCF format with the target SV set, and calculate the proportion of the target SV set covered by the real-time SV detection results in the VCF format; When the coverage ratio reaches the ratio threshold of the target SV set, sequencing is stopped; If the coverage ratio does not reach the ratio threshold of the target SV set, steps 2 to 5 are executed for the first FASTQ file in the processing task queue.

[0007] The beneficial effects of the present invention are: This paper proposes a more efficient solution that integrates real-time data analysis with the sequencing process. The computation process is performed simultaneously with sequencing, enabling the analysis of the current batch of data to proceed simultaneously with the generation of the next batch of data. This overlapping workflow allows the analysis process to be nearly synchronized with data production, significantly improving detection speed, and can be run even with only a single sequencer and standard edge computing equipment. The present invention proposes cuteSV-OL, a real-time structural variation detection framework for nanopore sequencing devices. This method performs real-time sequence alignment and variant signal feature extraction for each data batch output by the sequencer, while clustering the accumulated variant features to achieve real-time detection of structural variations. Furthermore, the method includes a real-time SV detection rate assessment module that compares the test results with a user-defined reference set to determine whether sequencing can be terminated early, thereby saving time and cost. cuteSV-OL also has a fault recovery mechanism that can resume the analysis process in the event of an unexpected outage, ensuring data integrity. This solution can be seamlessly integrated into mainstream desktop systems and used in conjunction with nanopore sequencers, providing an efficient solution for time-sensitive scenarios such as real-time clinical diagnosis.

[0008] This paper introduces a task buffer queue mechanism, introducing an asynchronous relay layer between file monitoring and data processing (asynchronous means that new sequencing data is not immediately entered into the workflow upon receipt, but must be queued). This helps mitigate any mismatch between sequencing and analysis rates. Within the data processing flow, this paper employs a producer-consumer model for concurrent processing, with each task item representing a FASTQ file.

[0009] In addition, the present invention proposes a three-stage fault tolerance and failure recovery mechanism that can recover from system crashes or runtime anomalies. The system persistently stores the operating status, including the set of task batches that have arrived, the set of task batches that have been processed, and the batch numbers completed before the last crash. Because the system includes three main processing stages: sequence alignment, signal extraction, and feature clustering, the fault tolerance mechanism of the present invention can process each stage separately; In summary, through modular design, overlapping workflows, and asynchronous mechanisms, this invention improves the scalability and cross-platform deployment capabilities of the structural variation detection process while ensuring system real-time performance, stability, and data integrity. It is suitable for real-time analysis of long-read sequencing data in a variety of clinical and research scenarios.

[0010] This invention addresses the lack of effectiveness and computational resource constraints of existing SV detection methods, improving the effectiveness and usability of SV sequencing analysis. The system features cross-platform compatibility, a buffer queue management mechanism, configurable clustering and evaluation modules, and fault recovery capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0012] Specific embodiment 1: This embodiment is a real-time detection system for structural variation (SV) including: cuteSV-OL is a redeveloped version of the widely used offline SV detection tool cuteSV. A typical cuteSV-OL process includes file monitoring and task queue management, preliminary SV feature extraction, SV clustering and genotyping, and real-time feedback mechanism. Figure 1 As shown; A real-time structural variation detection system includes: a file monitoring and task queue management module, a preliminary SV feature extraction module, an SV clustering and genotyping module, and a real-time feedback mechanism module; The file monitoring and task queue management module is used to monitor newly generated FASTQ files and add the newly generated FASTQ files to the task queue; The preliminary SV feature extraction module is used to extract preliminary features of structural variations from the FASTQ files in the task queue; The SV clustering and genotyping module is used to perform SV clustering and genotyping on the extracted preliminary features of structural variations and output SV detection results in VCF format; The real-time feedback mechanism module is used to compare the SV detection results with the target SV set, calculate the proportion of the target SV set covered by the current detection results, and use the coverage ratio as an evaluation indicator.

[0013] File monitoring and task queue management: cuteSV-OL uses a file monitoring daemon (watchdog) to continuously monitor newly generated FASTQ files. Once a file is generated stably, the system automatically adds it to the task queue, preparing it for subsequent analysis.

[0014] Preliminary SV feature extraction: cuteSV-OL processes each FASTQ file sequentially in queue order, performs sequence alignment using minimap2, processes BAM files using samtools, and calls cuteSV's signal extraction module to extract preliminary structural variant features. In addition, pandepth is called to assess the sequencing depth of each FASTQ file.

[0015] SV Clustering and Genotyping: After feature extraction is complete for all FASTQ files in the current batch, cuteSV-OL launches the cuteSV clustering module to cluster and genotype the extracted structural variant features. This step generates a temporary VCF file containing the test results, taking into account the current sequencing depth.

[0016] Real-time feedback mechanism: Finally, cuteSV-OL launches the variant detection rate evaluation module, compares the detected structural variants with the built-in population SV dataset (or user-defined reference dataset), and calculates the coverage of high-frequency variants as an evaluation indicator, thereby achieving real-time feedback on sequencing progress.

[0017] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the file monitoring and task queue management module is used to monitor newly generated FASTQ files and add the newly generated FASTQ files to the task queue; the specific process is as follows: The watchdog file monitoring module developed based on Python is used to automatically detect newly generated FASTQ files (supporting compressed formats) in the user-specified directory, and register the detected new FASTQ files in the pending task queue in chronological order.

[0018] To enable cross-platform file monitoring, this paper uses the watchdog file monitoring module, developed in Python, to replace the underlying inotify API that relies on the Linux operating system. Although inotify offers slightly better performance, the difference is not significant when monitoring a small number of files. Watchdog offers greater platform versatility and ease of deployment, and can be used on Windows, Linux, and Mac. This module automatically detects newly generated FASTQ files (supporting compressed formats) in a user-specified directory and registers the detected new file paths in a pending task queue.

[0019] Other steps and parameters are the same as those in the first embodiment.

[0020] Specific embodiment three: This embodiment differs from specific embodiments one or two in that the preliminary SV feature extraction module is used to extract preliminary features of structural variations from the FASTQ files in the task queue; the specific process is as follows: (1) Use minimap2 to align the FASTQ file ranked first in the task queue with the human reference genome, and remove the FASTQ file ranked first in the task queue from the task queue; minimap2 determines whether a sequence alignment failure occurs; If no failure occurs, a SAM format alignment result is obtained, which contains the position of the bases in the FASTQ file on the human reference genome, and execute (2); If a failure occurs, the incomplete reads in the current FASTQ file are discarded, and the current FASTQ file after discarding the incomplete reads is re-aligned to obtain a SAM format alignment result. The alignment result includes the position of the bases in the FASTQ file on the human reference genome, and execute (2); (2) Use samtools to sort the SAM format alignment results from front to back according to their positions on the human reference genome to obtain the sorted SAM format alignment file; Use samtools to convert the sorted SAM format alignment results to generate a BAM format alignment file and execute (3) and (4); (3) Use the pandepth tool to calculate the sequencing depth of the BAM format alignment file generated in (2) to provide reference information for subsequent structural variation clustering; (4) Use cuteSV tool to extract the structural variation signature signal (svsignature) in the BAM format alignment file generated in (2); Only the streamlined SVsignature data is retained, and the original FASTQ and BAM files can be cleared in real time when no longer in use to save storage resources; The cuteSV tool determines whether there is a fault in the structural variation feature signal in the BAM format alignment file generated by extraction (2); If a failure occurs, the structural variation feature signal extracted from the BAM format comparison file generated by (2) is discarded, and the structural variation feature signal is extracted from the BAM format comparison file generated by (2) again to avoid repeated extraction of the same signal; If no fault occurs, the extracted structural variation feature signal is used as the preliminary feature of the structural variation.

[0021] Other steps and parameters are the same as those in the first or second embodiment.

[0022] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that: the SV clustering and genotyping module is used to perform SV clustering and genotyping on the extracted preliminary features of structural variations, and output SV detection results in VCF format; The specific process is: Execute the preliminary SV feature extraction module process on the first FASTQ file in the processing task queue to determine whether the number of BAM format alignment files extracted in step 5 meets the clustering trigger condition; If not, continue to execute the preliminary SV feature extraction module process on the FASTQ file ranked first in the processing task queue until the clustering trigger condition is met; If it is reached, cuteSV's feature clustering module is used to cluster the structural variation feature signals extracted from all BAM format comparison files; cuteSV determines whether the clustering fails; If clustering fails, cuteSV’s feature clustering module is reused to cluster the structural variation feature signals extracted from all BAM format alignment files, and the real-time SV detection results in VCF format are output; If the clustering is successful, the real-time SV detection results in VCF format are output.

[0023] The other steps and parameters are the same as those in the first to third embodiments.

[0024] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that the real-time feedback mechanism module is used to compare the SV detection results with the target SV set, calculate the proportion of the target SV set covered by the current detection results, and use the coverage ratio as an evaluation indicator; The specific process is: Compare the real-time SV detection results in VCF format with the target SV set, and calculate the proportion of the target SV set covered by the real-time SV detection results in the current VCF format; When the coverage ratio reaches the ratio threshold of the target SV set, sequencing is stopped; When the coverage ratio does not reach the ratio threshold of the target SV set, the preliminary SV feature extraction module process is executed on the FASTQ file ranked first in the processing task queue.

[0025] The other steps and parameters are the same as those in the first to fourth embodiments.

[0026] Specific embodiment six: This embodiment differs from any one of specific embodiments one to five in that the target set is a set of common human structural variations constructed in the HGSVC database. The HGSVC database supports customizing the minimum allele frequency (minorallele frequency) threshold, and a subset is taken from the HGSVC as the target set.

[0027] The other steps and parameters are the same as those in the first to fifth embodiments.

[0028] Specific embodiment seven: This embodiment differs from any one of specific embodiments one to six in that when the real-time structural variation detection system crashes (shuts down), execution is continued from the last unfinished batch based on the recorded operating status (if the system crashes during the execution of a batch, the FASTQ sequence alignment is started from a FASTQ sequence alignment in the corresponding batch, and the FASTQ sequence alignment, SAM file sorting and format conversion, preliminary SV feature extraction module, SV clustering and genotyping module, and real-time feedback mechanism module are re-executed), thereby avoiding resource waste caused by starting from the beginning; The one batch is to execute a preliminary SV feature extraction module, an SV clustering and genotyping module and a real-time feedback mechanism module on one FASTQ.

[0029] The other steps and parameters are the same as those in the first to sixth embodiments.

[0030] Specific embodiment 8: This embodiment is a method for real-time detection of structural variation (SV). The specific process is as follows: Step 1: Use the watchdog file monitoring module developed in Python to automatically detect newly generated FASTQ files (supporting compressed formats) in the user-specified directory and register the detected new FASTQ files in the pending task queue in chronological order; Step 2: Use minimap2 to align the first FASTQ file in the pending task queue with the human reference genome, and remove the first FASTQ file from the queue; minimap2 determines whether a sequence alignment failure occurs; If no failure occurs, a SAM format alignment result is obtained. The alignment result contains the position of the bases in the FASTQ file on the human reference genome, and step 3 is performed; If a failure occurs, the incomplete reads in the current FASTQ file are discarded, and the current FASTQ file after discarding the incomplete reads is re-aligned to obtain a SAM format alignment result. The alignment result contains the position of the bases in the FASTQ file on the human reference genome, and step 3 is executed; Step 3: Use samtools to sort the SAM format alignment results from front to back according to their positions on the human reference genome to obtain the sorted SAM format alignment file; Use samtools to convert the sorted SAM format alignment results to generate a BAM format alignment file, and execute steps 4 and 5; Step 4: Use the pandepth tool to calculate the sequencing depth of the BAM format alignment file generated in step 2 to provide reference information for subsequent structural variation clustering; Step 5: Use cuteSV tool to extract structural variation feature signals from the BAM format alignment file generated in step 3; Only the streamlined SVsignature data is retained, and the original FASTQ and BAM files can be cleared in real time when no longer in use to save storage resources; The cuteSV tool determines whether there is a fault in the structural variation feature signal in the BAM format alignment file generated in step 3 of the extraction; If a failure occurs, the structural variation feature signal (svsignature) extracted from the BAM format comparison file generated in step 3 is discarded, and the structural variation feature signal is extracted again from the BAM format comparison file generated in step 3 to avoid repeated extraction of the same signal; If no fault occurs, the extracted structural variation feature signal is used as the preliminary feature of the structural variation; Step 6: Determine whether the number of BAM format comparison files extracted in step 5 reaches the clustering trigger condition; If reached, proceed to step seven; If not, re-execute steps 2 to 5 until the number of BAM format comparison files extracted in step 5 reaches the clustering trigger condition; Step 7: Use the cuteSV feature clustering module to cluster and genotype the preliminary features of structural variations corresponding to all extracted BAM format comparison files, and output the SV detection results in VCF format; cuteSV determines whether the clustering fails; If clustering fails, the feature clustering module of cuteSV is reused to cluster and genotype the preliminary features of structural variations extracted from all BAM format alignment files, and the real-time SV detection results in VCF format are output; If the clustering does not fail, the real-time SV detection results in VCF format are output; Step 8: Compare the real-time SV detection results in the VCF format with the target SV set, and calculate the proportion of the target SV set covered by the real-time SV detection results in the VCF format; When the coverage ratio reaches the ratio threshold of the target SV set, sequencing is stopped; If the coverage ratio does not reach the ratio threshold of the target SV set, steps 2 to 5 are executed for the first FASTQ file in the processing task queue.

[0031] Specific embodiment 9: This embodiment differs from specific embodiment 8 in that the structural variation feature signal extracted in step 5 includes the following five attributes, as shown in the following formula: in, Indicates the starting position of the read on the reference genome; Indicates the end position of the read segment on the reference genome; Indicates whether it is the main alignment (obtained by minimap2 during alignment, recorded in the BAM file); is the read segment identifier; Indicates the chromosome position information of the read segment on the reference genome.

[0032] It can be directly used for subsequent genotyping of structural variation types (such as TRA): determine whether the reads crossing the breakpoint are non-primary alignments. If a certain ratio is met, the breakpoint is considered to be credible, thereby improving the detection accuracy of TRA type structural variations.

[0033] Other steps and parameters are the same as those in the eighth embodiment.

[0034] Specific embodiment ten: This embodiment differs from specific embodiment nine in that if a real-time structural variation detection system crashes (shuts down) during the execution of steps two to eight, execution is continued from the last unfinished batch based on the recorded running status (if the system crashes during the execution of a batch, steps two to eight are re-executed starting from step two of the corresponding batch), thus avoiding resource waste caused by restarting from the beginning. The one batch is steps 2 to 8.

[0035] Other steps and parameters are the same as those in the ninth embodiment.

[0036] This invention relates to an optimized analysis framework for structural variation detection. Specifically, it presents a modularized reconstruction and improved efficiency of cuteSV, an existing structural variation detection tool, to adapt it to the operational requirements of real-time batch processing. This approach significantly improves cuteSV's adaptability and efficiency in real-time processing systems, making it suitable for online detection of structural variations in long-read sequencing.

[0037] First, this paper proposes a structural improvement to the cuteSV tool by separating its modules and decoupling its functions. The highly coupled structural variation feature extraction and feature clustering steps in the original workflow are deconstructed into two completely independent modules: the "SV Signal Extraction Module" and the "SV Feature Clustering Module." These modules independently process datasets of different sizes, thus meeting the data batch processing requirements of real-time analysis systems.

[0038] In the original cuteSV workflow, two modules share a single BAM file: SV signal extraction identifies candidate variant events, while feature clustering integrates and genotypes these events. However, in the cuteSV-OL real-time processing architecture, data is batched. The extraction module executes on a single batch, while the clustering module processes SV candidate information accumulated from multiple batches. The BAM file ranges required by the two modules are inconsistent. Merging all BAM files for each clustering step would incur significant time and storage overhead.

[0039] To solve the above problems, the present invention proposes a lightweight read information caching method, which extracts and stores the minimum read level information required by the clustering module in advance during the signal extraction stage, thereby avoiding repeated parsing and merging operations of BAM files in the subsequent clustering process.

[0040] This information can be directly used for subsequent genotyping of structural variation types (such as TRA): determine whether the reads crossing the breakpoint are non-primary alignments. If a certain ratio is met, the breakpoint is considered to be credible, thereby improving the detection accuracy of TRA type structural variations.

[0041] Compared to the original cuteSV approach, which relied on the pysam library to dynamically parse BAM files, the intermediate read metadata file proposed in this paper allows for region localization and read support calculation without relying on BAM, significantly improving processing speed and resource efficiency. Furthermore, this paper rewrites the highly reproducible code of the signal extraction and feature clustering modules into Cython, achieving an approximately 20% improvement in overall computational speed while maintaining functional equivalence.

[0042] Through the above optimization, the present invention realizes the high-performance integration of cuteSV in real-time and batch structural variation detection scenarios, with the advantages of strong module independence, low computing resource usage, and good scalability, and is suitable for a variety of real-time genome detection platforms.

[0043] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A real-time structural variation detection system, characterized by: The system includes: a file monitoring and task queue management module, a preliminary SV feature extraction module, an SV clustering and genotyping module, and a real-time feedback mechanism module; The file monitoring and task queue management module is used to monitor newly generated FASTQ files and add the newly generated FASTQ files to the task queue; The preliminary SV feature extraction module is used to extract preliminary features of structural variations from the FASTQ files in the task queue; The SV clustering and genotyping module is used to perform SV clustering and genotyping on the extracted preliminary features of structural variations and output SV detection results in VCF format; The real-time feedback mechanism module is used to compare the SV detection result with the target SV set and calculate the proportion of the target SV set covered by the current detection result.

2. The real-time structural variation detection system according to claim 1, characterized in that: The file monitoring and task queue management module is used to monitor newly generated FASTQ files and add them to the task queue. The specific process is as follows: The watchdog file monitoring module developed based on Python language is used to automatically detect newly generated FASTQ files in the user-specified directory, and register the detected new FASTQ files into the pending task queue in chronological order.

3. The real-time structural variation detection system according to claim 2, characterized in that: The preliminary SV feature extraction module is used to extract preliminary features of structural variations from the FASTQ files in the task queue; The specific process is: (1) Use minimap2 to align the FASTQ file ranked first in the task queue with the human reference genome, and remove the FASTQ file ranked first in the task queue from the task queue; minimap2 determines whether a sequence alignment failure occurs; If no failure occurs, a SAM format alignment result is obtained, which contains the position of the bases in the FASTQ file on the human reference genome, and execute (2); If a failure occurs, the incomplete reads in the current FASTQ file are discarded, and the current FASTQ file after discarding the incomplete reads is re-aligned to obtain a SAM format alignment result. The alignment result includes the position of the bases in the FASTQ file on the human reference genome, and execute (2); (2) Use samtools to sort the SAM format alignment results from front to back according to their positions on the human reference genome to obtain the sorted SAM format alignment file; Use samtools to convert the sorted SAM format alignment results to generate a BAM format alignment file and execute (3) and (4); (3) Use the pandepth tool to calculate the sequencing depth of the BAM format alignment file generated in (2); (4) Use cuteSV tool to extract the structural variation feature signals in the BAM format alignment file generated in (2); The cuteSV tool determines whether there is a fault in the structural variation feature signal in the BAM format alignment file generated by extraction (2); If a failure occurs, the structural variation feature signal extracted from the BAM format comparison file generated by (2) is discarded, and the structural variation feature signal is extracted from the BAM format comparison file generated by (2) again; If no fault occurs, the extracted structural variation feature signal is used as the preliminary feature of the structural variation.

4. A real-time structural variation detection system according to claim 3, characterized in that: The SV clustering and genotyping module is used to perform SV clustering and genotyping on the extracted preliminary features of structural variations and output SV detection results in VCF format. The specific process is as follows: Execute the preliminary SV feature extraction module process on the first FASTQ file in the processing task queue to determine whether the number of BAM format alignment files extracted in step 5 meets the clustering trigger condition; If not, continue to execute the preliminary SV feature extraction module process on the FASTQ file ranked first in the processing task queue until the clustering trigger condition is met; If it is reached, cuteSV's feature clustering module is used to cluster the structural variation feature signals extracted from all BAM format comparison files; cuteSV determines whether the clustering fails; If clustering fails, cuteSV’s feature clustering module is reused to cluster the structural variation feature signals extracted from all BAM format alignment files, and the real-time SV detection results in VCF format are output; If the clustering is successful, the real-time SV detection results in VCF format are output.

5. The real-time structural variation detection system according to claim 4, characterized in that: The real-time feedback mechanism module is used to compare the SV detection results with the target SV set, calculate the proportion of the target SV set covered by the current detection results, and use the coverage ratio as an evaluation indicator; the specific process is as follows: Compare the real-time SV detection results in VCF format with the target SV set, and calculate the proportion of the target SV set covered by the real-time SV detection results in the current VCF format; When the coverage ratio reaches the ratio threshold of the target SV set, sequencing is stopped; When the coverage ratio does not reach the ratio threshold of the target SV set, the preliminary SV feature extraction module process is executed on the FASTQ file ranked first in the processing task queue.

6. A real-time structural variation detection system according to claim 5, characterized in that: The target set is the structural variation set in the HGSVC database.

7. The real-time structural variation detection system according to claim 6, characterized in that: When the real-time structural variation detection system crashes, execution continues from the last unfinished batch; The one batch is to execute a preliminary SV feature extraction module, an SV clustering and genotyping module and a real-time feedback mechanism module on one FASTQ.

8. A method for real-time detection of structural variation, characterized by: The specific process of the method is: Step 1: Use the watchdog file monitoring module developed in Python to automatically detect newly generated FASTQ files in the user-specified directory and register the detected new FASTQ files in the pending task queue in chronological order; Step 2: Use minimap2 to align the first FASTQ file in the pending task queue with the human reference genome, and remove the first FASTQ file from the queue; minimap2 determines whether a sequence alignment failure occurs; If no failure occurs, a SAM format alignment result is obtained. The alignment result contains the position of the bases in the FASTQ file on the human reference genome, and step 3 is performed; If a failure occurs, the incomplete reads in the current FASTQ file are discarded, and the current FASTQ file after discarding the incomplete reads is re-aligned to obtain a SAM format alignment result. The alignment result contains the position of the bases in the FASTQ file on the human reference genome, and step 3 is executed; Step 3: Use samtools to sort the SAM format alignment results from front to back according to their positions on the human reference genome to obtain the sorted SAM format alignment file; Use samtools to convert the sorted SAM format alignment results to generate a BAM format alignment file, and execute steps 4 and 5; Step 4: Use the pandepth tool to calculate the sequencing depth of the BAM format alignment file generated in step 2; Step 5: Use cuteSV tool to extract structural variation feature signals from the BAM format alignment file generated in step 3; The cuteSV tool determines whether there is a fault in the structural variation feature signal in the BAM format alignment file generated in step 3 of the extraction; If a failure occurs, the structural variation feature signal extracted from the BAM format comparison file generated in step 3 is discarded, and the structural variation feature signal is extracted again from the BAM format comparison file generated in step 3; If no fault occurs, the extracted structural variation feature signal is used as the preliminary feature of the structural variation; Step 6: Determine whether the number of BAM format comparison files extracted in step 5 reaches the clustering trigger condition; If reached, proceed to step seven; If not, re-execute steps 2 to 5 until the number of BAM format comparison files extracted in step 5 reaches the clustering trigger condition; Step 7: Use the cuteSV feature clustering module to cluster and genotype the preliminary features of structural variations corresponding to all extracted BAM format comparison files, and output the SV detection results in VCF format; cuteSV determines whether the clustering fails; If clustering fails, the feature clustering module of cuteSV is reused to cluster and genotype the preliminary features of structural variations extracted from all BAM format alignment files, and the real-time SV detection results in VCF format are output; If the clustering does not fail, the real-time SV detection results in VCF format are output; Step 8: Compare the real-time SV detection results in the VCF format with the target SV set, and calculate the proportion of the target SV set covered by the real-time SV detection results in the VCF format; When the coverage ratio reaches the ratio threshold of the target SV set, sequencing is stopped; If the coverage ratio does not reach the ratio threshold of the target SV set, steps 2 to 5 are executed for the first FASTQ file in the processing task queue.

9. The method for real-time detection of structural variation according to claim 8, characterized in that: The structural variation feature signal extracted in step 5 includes the following five attributes, as shown in the following formula: in, Indicates the starting position of the read on the reference genome; Indicates the end position of the read segment on the reference genome; Indicates whether it is the main comparison; is the read segment identifier; Indicates the chromosome position information of the read segment on the reference genome.

10. A method for real-time detection of structural variations according to claim 9, characterized in that: If a real-time structural variation detection system crashes during the execution of steps 2 to 8, execution will continue from the last unfinished batch; The one batch is steps 2 to 8.

Images