A Cluster-Based High-Throughput Data Analysis Method

Abstract

The invention discloses a high-throughput data analyzing method. For processing of high-throughput sequencing offline data, the high-throughput data analyzing method comprises the following steps: partitioning offline data; generating a plurality of front data slice files after the offline data are partitioned, and combining a plurality of generated comparison result slice files into a comparison result file after all comparisons with a reference genome are finished; pre-specifying one zone file, and partitioning the zone file into a plurality of specified zone sub-files; extracting data from the comparison result file according to the plurality of specified zone sub-files, partitioning the extracted data once again to generate a plurality of back data slice files, and processing the plurality of back data slice files in a subsequent step. For operation processing of a plurality of data slices obtained after partitioning, computing resources including computing nodes, a corresponding CPU (Central Processing Unit) and a corresponding memory are allocated by a cluster management tool.

Description

technical field [0001] The invention belongs to the technical field of gene sequencing, in particular to a cluster-based high-throughput data analysis method. Background technique [0002] High-throughput gene sequencing technology, also known as "next-generation sequencing (NGS) technology, can determine hundreds of thousands or even millions of sequences at one time, and is the most widely used sequencing technology today. Compared with the traditional Sanger sequencing technology, NGS has the advantages of high speed, high throughput, and low price. [0003] Variation detection based on high-throughput sequencing has developed rapidly in recent years, and now coincides with the vigorous promotion of precision medicine, the demand for mutation detection is facing explosive growth. In the process of mutation detection, it is very common to process GB-level or even hundreds of gigabytes of data. The routine analysis time ranges from a few hours to a few days. It takes a lon...

AI Technical Summary

Problems solved by technology

The application believes that the above-mentioned solution solves the problem of too long analysis time of human resequencing biological information. However, this document does not involve high-throughput gene sequencing technology, and does not provide a solution to the existing problems in the existing high-throughput gene sequencing solution. suggestion

[0007] It can be seen that the existing solutions can only perform parallel processing of whole genome data, and cannot cope well with the situation of whole exon or target sequencing. In addition, if the number of parallel processing tasks is changed, the reference genome needs to be re-segmented and indexed , which increases analysis time and inconvenience

Images