A Fast Differential Compression Method
A compression method and fast technology, applied in the direction of code conversion, electrical components, etc., can solve the problems of low data compression efficiency, poor scalability, large indexing overhead, etc., achieve fast differential compression encoding speed, maintain locality, height difference The effect of volume compression efficiency
Active Publication Date: 2019-04-12
HUAZHONG UNIV OF SCI & TECH
View PDF3 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0008] Aiming at the above defects or improvement needs of the prior art, the present invention provides a fast differential compression method, the purpose of which is to quickly segment words, calculate word hashes, and index lookups by similar data blocks (or files) based on content. Repeat operations such as words, identify different data between similar data blocks, and realize the final differential encoding storage, thereby saving storage space and solving the problems of slow compression encoding, large index overhead, and data compression efficiency in existing differential compression technologies Technical issues with low and poor scalability
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0040] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.
[0041] like figure 1As shown, the differential compression of the present invention mainly includes three parts, namely rapid segmentation of words based on content, a method for enlarging repeated words, and differential encoding for merging non-repetitive words.
[0042] The content-based rapid word segmentation method proposed by the present invention avoids a large number of words with over...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a rapid delta compression method, which comprises the following steps of carrying out content-based rapid segmentation on a reference block B in delta compression to obtain a plurality of words in order to form a word library; carrying out content-based rapid segmentation on a reference block A similar to the reference block B and amplifying repeated words detected during rapid segmentation to obtain repeated words and non-repeated words; encoding and storing the obtained repeated words and non-repeated words in sequence according to a segmentation order and respectively utilizing two different data formats to record the repeated words and the non-repeated words in order to obtain a delta data block deltaB,A; and when a decoding operation on the delta data block deltaB,A is needed, obtaining records of two data formats from the deltaB,A in sequence in order to obtain all words of the data block A in sequence, and writing the words into an output stream in sequence to restore the complete data block A. The method has the advantages of high search efficiency for the repeated words, small computing cost, high data compression efficiency and the like.
Description
technical field [0001] The invention belongs to the field of data compression for computer storage, and more specifically relates to a fast differential compression method. Background technique [0002] In recent years, with the development and popularization of computer technology and networks, the amount of data information storage in the world has shown an explosive growth trend. Although the price of storage devices has been declining, it is far behind the speed of data expansion. Data deduplication (Data Deduplication), as a technology to effectively eliminate redundant data on a large scale, has become a hotspot in storage system research in recent years. Data deduplication can not only greatly save storage space and improve the utilization of storage resources, but also improve the transmission efficiency of network bandwidth by avoiding redundant data transmission. [0003] However, with the development of deduplication technology, deduplication technology also fac...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & AuthorityPatents(China)
IPC IPC(8): H03M7/30
CPCH03M7/30H03M7/3091
Inventor夏文冯丹李春光江泓
OwnerHUAZHONG UNIV OF SCI & TECH