Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Historical sensed data duplicate removal fragment eliminating method and system

A fragmentation and data technology, applied in the field of computer storage, can solve the problem of increasing economic losses in the recovery time window, and achieve the effect of large recovery performance and less memory resources

Active Publication Date: 2014-09-17
HUAZHONG UNIV OF SCI & TECH
View PDF3 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Excessively long recovery time windows greatly increase economic losses

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Historical sensed data duplicate removal fragment eliminating method and system
  • Historical sensed data duplicate removal fragment eliminating method and system
  • Historical sensed data duplicate removal fragment eliminating method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

[0038] like figure 1 Shown is the architecture of the invention. The present invention has a fingerprint index module on the disk, which stores the fingerprints of data blocks in the system and is used to judge duplicate data blocks; the container pool is a container storage module that provides operations for reading and writing containers; historical information files record the above Histor...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a fragment eliminating method applied to a data duplicate removal system. The method can accurately identify fragments by utilizing historical information, and is small in memory expenditure and high in recovering throughput rate. The method comprises the steps of first, partitioning files in a data stream, working out fingerprints of the files, and searching index to find out repeating data blocks; then, searching a container I for the repeating data blocks from a sparse container set in the last duplicate record, and judging which repeating data blocks belong to the sparse container. The repeating data blocks belonging to the sparse container are rewroted into a new container. In the backup procedure, the method only needs to record the utilization rate of related containers, so that the memory expenditure is extremely small. The invention also provides a corresponding historical sensed data duplicate removal fragment eliminating system. The fragments are accurately identified by adopting the method, so the amount of rewroted data is very small, and accordingly higher duplicate removal rate and back-up performance are maintained, and the recovering performance is also obviously improved.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more particularly relates to a method and system for eliminating duplicate fragments of data with history perception. Background technique [0002] In recent years, with the popularization of the Internet, the amount of data information storage in the world is exploding. The storage and management of massive data has become a major problem faced by academia and industry. Researchers have found that there is a large amount of redundant data in various storage systems (such as backup and archive storage systems, primary storage systems, and high-performance data centers). By eliminating this redundant data, storage costs can be greatly reduced. Therefore, data deduplication (Data Deduplication), as a technology to effectively eliminate redundant data on a large scale, has become a hotspot in storage system research in recent years. Data deduplication can not only save storage space ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F11/14G06F3/06
Inventor 冯丹付忞华宇夏文黄方亭柳青
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com