Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing method, device and deduplication processor

A data processing device and data processing technology, applied in the field of storage, can solve the problems that the index occupies a large memory and cannot meet the storage requirements of data volume, etc.

Active Publication Date: 2015-09-23
HUAWEI TECH CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, the inventors found that in the prior art data deduplication method, the index occupies a large amount of memory, which cannot meet the storage requirements of the gradually increasing data volume

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device and deduplication processor
  • Data processing method, device and deduplication processor
  • Data processing method, device and deduplication processor

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0061] The embodiment of the present invention provides a data processing system. The data processing system includes at least one deduplication processor and at least one storage node. The deduplication processor and the storage node are deployed in two ways. The embodiment of the present invention provides two deployment methods, for...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Embodiments of the present invention, by indexing data primarily according to index values corresponding to data of a first query unit, where the first query unit comprises multiple minimum data blocks, and by extracting some bits from each minimum data block to compose an index value corresponding to the first query unit, greatly reduce index matching time, increase index matching efficiency, and at the same time, make possible a significant reduction in the amount of memory occupied by indexes.

Description

Technical field [0001] The embodiments of the present invention relate to storage technology, and in particular, to a data processing method, device, and deduplication processor. Background technique [0002] Data deduplication is also called smart compression or single-instance storage. It can automatically search for duplicate data, keep only one copy of the same data, and use a pointer to the single copy to replace other duplicate copies to eliminate redundancy Data storage technology that reduces storage capacity requirements. [0003] In the prior art, the deduplication method can adopt a fixed-length block algorithm. [0004] The fingerprint algorithm is used to calculate the fingerprint of the data object in the sliding window. If the predetermined conditions are met, the start and end positions of the sliding window are used as the boundary of the data block, and the data object is divided into blocks by continuously sliding the window and calculating fingerprints. For each...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
CPCG06F17/30G06F16/328G06F16/1756
Inventor 于传帅张程伟张宗全林春恭游俊刘强
Owner HUAWEI TECH CO LTD