Continuous data reading oriented data placement method of deduplication and erasure correcting combined system

A technology oriented to data and hybrid systems, which is applied to data error detection, electrical digital data processing, instruments, etc. in the direction of redundancy in computing, to achieve the effects of ensuring reliability, eliminating read load bottlenecks, and improving read performance

Inactive Publication Date: 2016-08-03
NAT UNIV OF DEFENSE TECH
View PDF6 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, the present invention optimizes the problem of continuous reading of data in a deduplication-correction hybrid storage system, and proposes a method of independently and continuously placing all data blocks and all redundant blocks of multiple groups in order to improve System performance when data is continuously read

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Continuous data reading oriented data placement method of deduplication and erasure correcting combined system
  • Continuous data reading oriented data placement method of deduplication and erasure correcting combined system
  • Continuous data reading oriented data placement method of deduplication and erasure correcting combined system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] figure 1It is a schematic diagram of a storage system that only uses data deduplication. The file is first divided into blocks, and then the corresponding fingerprint is generated by calculating the hash value for each block, and the fingerprint is compared with the fingerprint in the index table to remove duplicate data. Finally, the unique data blocks are rotated in sequence according to the node number Place and store to the corresponding node.

[0031] figure 2 It is the basic flow chart of the hybrid system of data deduplication and erasure code, including data block, calculation of characteristic value, query index table, deletion of duplicate blocks, redundant encoding and placement and storage of blocks. The placement strategy is to place the blocks sequentially and sequentially according to the strip grouping without distinguishing between redundant blocks and data blocks.

[0032] image 3 It is a schematic diagram of the data placement method of the dedu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a continuous data reading oriented data placement method of a deduplication and erasure correcting combined system. Based on various types of distributed data deduplication and erasure code combined storage system, through the change of the placement strategy for a data block and a verification block, on the premise of ensuring that the system reliability is free of any influence, the reading performance in the continuous data reading is further improved. The data placement method is characterized in that the constitution of tapes is not changed, and all data elements and all verification elements in the tapes are respectively continuously placed, so that the continuity of placement of the all data elements is ensured, load bottlenecks caused by data element interrupt placement of the original verification elements are eliminated, the degree of parallelism in continuous data reading is improved to the maximum extent, the individual node parallelism is utilized to the maximum extent, and the system read performance of the continuous reading is improved.

Description

technical field [0001] The present invention is applicable to the technical fields of data deduplication and erasure code, and provides a data placement method for a hybrid system of data deduplication (DataDeduplication) and erasure code (Erasurecode) for continuous data reading, without changing the reliability of the system. Under the premise of eliminating the load bottleneck of continuous reading of data, the reading performance of the system is improved. Background technique [0002] In the era of big data, the explosive growth of data and the rapid growth of computing performance of processing devices represented by CPU and GPU have put forward higher requirements for storage system capacity, performance and reliability, and storage systems are facing huge challenges. [0003] On the one hand, as far as the huge and growing data scale is concerned, it is not an effective way to solve the capacity problem by blindly adding storage devices to expand the storage scale, a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/14
CPCG06F11/1453
Inventor 肖侬邓明翥陈志广刘芳
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap