The invention discloses a distributed
sequence alignment method and
system based on Spark, and relates to the technical field of
bioinformatics, and the method comprises the steps: receiving a Minimap2 reference sequence, and generating a multi-part index of the reference sequence based on a multi-part index generation function of the Minimap2; partitioning the multi-part index of the reference sequence based on a character string in the multi-part index of the reference sequence as an identifier to obtain a partitioned copy, and distributing the partitioned copy to each node of a Spark cluster; receiving a query sequence, partitioning the query sequence through a
file format of FASTA or FASTQ, and generating a multi-node shared task
queue based on the number of partitions of the reference sequence, the number of partitions of the query sequence and partition copies on each node; and based on the multi-node shared task
queue, each node selects the task in the task
queue which needs to process all the remaining tasks and takes the longest time in the task queues corresponding to all the index file partition copies of the node to process, and a task
processing result is obtained. According to the method, strategies of query sequence partitioning and reference
sequence database partitioning are combined,
dynamic load balancing is achieved by establishing a multi-node shared task queue, and the efficiency of comparison between Minimap2 and a large-scale
data set is optimized. The method is suitable for long sequence data generated by a third-generation sequencing technology, and is especially suitable for application scenes of on-site rapid sequencing and analysis.