Re-sequencing sequence alignment method based on Spark framework

Abstract

The invention relates to the technical field of computer science and bioinformatics, and particularly to a re-sequencing sequence alignment method based on Spark framework. The method comprises threesteps of a RDDs creating step, a Map step and a Reduce step. The corresponding RDDs are created based on the FASTQ file and are stored in an HDFS. Then a sequence alignment algorithm of a BWA is applied on each RDDs. Furthermore the RDDs perform multi-node mapping. Finally whether to execute a final combining step is determined according to a processing requirement. According to the method, a sequence alignment BWA which is used in a re-sequencing step is integrated in a Spark big data processing frame, and re-sequencing procedure optimization is finished in a distributed calculation manner, thereby effectively improving re-sequencing data analysis efficiency.

Description

technical field [0001] The invention relates to the technical fields of computer science and bioinformatics, in particular to a resequencing sequence alignment method based on the Spark framework. Background technique [0002] Whole genome resequencing is to perform genome sequencing on different individuals of species with existing reference sequences (Reference Sequence), and based on this, analyze differences at the individual or group level. Through genome-wide resequencing, researchers can find a large number of single nucleotide polymorphism sites (SNP), copy number variation (Copy Number Variation, CNV), insertion deletion (InDel, Insertion / Deletion), structural variation (Structure Variation, SV) and other variant sites, which have great guiding significance in the research of human diseases and animal and plant breeding. As the cost of sequencing decreases, it is foreseeable that we will accumulate a large number of genome resequencing data of biological individual...

AI Technical Summary

Problems solved by technology

[0004] In genome resequencing, there are many gene comparison tools such as SOAP, BWA, bowtie2, etc. SOAP is the first gene comparison tool that can use a small computer memory to compare gene sequences, but its paired The support for -end data is poor, while BWA and bowtie2 appeared later than SOAP, but they can handle gene sequencing data in single-end and paired-end formats well. In terms of data processing speed, bowtie2 is better than BWA Obvious speed advantage, but the accuracy rate is slightly insufficient. However, the methods of these comparison tools are mostly based on a single node. At present, the preprocessing, management and analysis of sequencing data sets with diverse situations and sources have exceeded the capabilities of many bioinformatics scientists. At present, it often takes several days to complete the analysis of a patient's whole genome sequencing data, which greatly delays the progress of follow-up life and medical scientific research.

Images