Method for realizing parallel compression and parallel decompression on FASTQ file containing DNA (deoxyribonucleic acid) sequence read data

A decompression and data technology, applied in concurrent instruction execution, electrical digital data processing, special data processing applications, etc., can solve problems such as parallel algorithm research articles that have not yet seen multi-core CPUs

CN103559020AInactive Publication Date: 2014-02-05INST OF SOFTWARE - CHINESE ACAD OF SCI +1
3 Cites 43 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2014-02-05
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a method for realizing parallel compression and parallel decompression on an FASTQ file containing DNA (deoxyribonucleic acid) sequence read data. By aiming at the compression and the decompression of the FASTQ file containing the DNA sequence read data, by utilizing circular double buffering queues, circular double memory mapping and memory mapping and by combining the data segmentation processing technology, the multi-thread streamline parallel compression and parallel decompression technology, the read-write sequence two-dimensional array technology and the like, the parallel compression and the parallel decompression between multiple processes of the FASTQ file and between in-process multiple threads is realized. The parallel compression and parallel decompression can be realized based on MPI and OpenMP, and also can be realized based on the MPI and Pthread (POSIX thread). According to the method disclosed by the invention, by fully utilizing all computational nodes and the powerful computational capability of an intra-node multi-core CPU (central processing unit), constraints of resources, such as a processor, a memory and the like, on a serial compression and decompression program, can be released.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the fields of biological information, data compression and high-performance computing, in particular to a method for parallel compression and parallel decompression of DNA reading sequence data FASTQ files. Background technique

[0002] One of the main tasks of bioinformatics is to collect and analyze large amounts of genetic data. These data are critical for genetic research, helping to identify genetic components that prevent or cause disease and develop targeted therapies. High-throughput sequencing methods and equipment generate massive amounts of short-read sequence data. The common way to store, manage and transmit DNA read sequence data is to use the FASTQ file format, which mainly contains DNA read sequence data and annotation information corresponding to each DNA base, such as Quality, which represents the uncertainty of the sequencing labeling process. Scores information. Read sequence markers and other description...

Examples

Embodiment Construction

[0062] The present invention provides a method for parallel compression and decompression of FASTQ files of DNA reading sequence data. In order to make the purpose, technical solution and effect of the present invention clearer and clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0063] The raw data reading thread in the parallel compression method of the FASTQ file is explained in detail below, and its specific implementation steps are as follows:

[0064] (1) Open the FASTQ compressed file of the raw DNA read sequence data to be compressed.

[0065] (2) Obtain the memory paging size of the file system of the currently running machine.

[0066] (3) Set the memory mapping space size according to the memory paging size.

[0067] (4) According to the rang...