Multi-thread fast storage lossless compression method and system for FASTQ data

A fast storage and lossless compression technology, applied in the direction of code conversion, electrical components, etc., can solve the problems of slow compression speed and poor compression effect of DNA sequence data, and achieve the effect of improving compression speed, compression rate and applicability

Inactive Publication Date: 2016-11-09
SHENZHEN UNIV
View PDF2 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In view of this, the object of the present invention is to provide a multi-threaded fast storage lossless compression method for FASTQ data and its system, aiming at solving the problems of poor compression effect and slow compression speed for DNA sequence data in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-thread fast storage lossless compression method and system for FASTQ data
  • Multi-thread fast storage lossless compression method and system for FASTQ data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0034] The specific embodiment of the present invention provides a multi-threaded fast storage lossless compression method for FASTQ data, which is applied to the compression of DNA sequences, wherein the method mainly includes the following steps:

[0035] S11, data classification step: input the original FASTQ data, and divide the short reads of the original FASTQ data into three data streams of metadata, quality score and base sequence;

[0036] S12. Data compression step: For metadata, use incremental encoding to detect and eliminate redundant information in metadata; for quality scores, use bi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-thread fast storage lossless compression method for FASTQ data, and is applied to compression of a DNA sequence. The method comprises a data classification step of inputting original FASTQ data, and dividing short reading of the original FASTQ data into three data flows, namely metadata, a mass fraction and a base sequence; a data compression step of aiming at the metadata, using an incremental encoding way to detect and eliminate redundant information of the metadata; aiming at the mass fraction, using a bit level PPM (Prediction by partial matching) prediction model and arithmetic coding to compress; aiming at the base sequence, using improved arithmetic coding of a fixed order to compress; and a data output step of archiving and merging compression results of different data flows, and outputting final data after compression. The invention further provides a multi-thread fast storage lossless compression system for the FASTQ data. The technical scheme provided by the invention can improve the compression efficiency and compression speed.

Description

technical field [0001] The invention relates to the field of data compression, in particular to a multi-thread fast storage lossless compression method for FASTQ data and a system thereof. Background technique [0002] With the development of DNA sequencing technology, the cost of genome sequencing is getting lower and lower. In 2014, the milestone of controlling the cost of determining a human genome to $1,000 was reached. Due to the improvement of sequencing efficiency, the amount of DNA sequence data has shown explosive growth. Since the growth rate of DNA sequencing data has far exceeded the growth rate of computer microprocessors and storage devices, the DNA data "tsunami" generated by storage and analysis of DNA sequencing technology and large-scale genome projects has become a constraint for the further development of the DNA sequencing industry. important bottleneck. Moreover, since DNA sequencing technology is developing from high-throughput sequencing (High-Thro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H03M7/30
CPCH03M7/30
Inventor 朱泽轩黄志安孙怡雯文振焜
Owner SHENZHEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap