Data de-weighting method and device

A data and block data technology, applied in the field of data deduplication methods and devices, to achieve the effect of reducing data storage space and reducing storage space occupation

CN107644081AInactive Publication Date: 2018-01-30RUIJIE NETWORKS CO LTD
4 Cites 2 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
Publication Date
2018-01-30
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a data de-weighting method and device and relates to the field of computer storage. The data de-weighting method and device are used for removing repeated data and reducing a data storage space. The method comprises the steps that a data stream is segmented at equal intervals to obtain to-be-stored segment data, and the to-be-stored segment data is partitioned at equal intervals to obtain to-be-stored block data; according to the to-be-stored block data, block Hash of the to-be-stored block data and segment Hash of the to-be-stored segment data are obtained; the block data identical to the data stream is searched for in existing data according to the segment Hash and the block Hash of the data stream and a segment list; a block pointer of the same block data in thedata stream is set to be a block pointer of the same block data in the existing data; the to-be-stored block data in the block data which is not the same in the data stream is stored, and a block pointer of the to-be-stored block data where the same block data is not found through search is set to be a storage position corresponding to the to-be-stored block data; and the segment Hash, the block Hash and the block pointers of the data stream are stored into the segment list. The embodiment is applied to data storage.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of computer storage, in particular to a data deduplication method and device. Background technique

[0002] Hyper Converged Infrastructure (HCI for short) is an infrastructure that integrates storage, network, and computing. In such an architecture environment, virtualization software is integrated in one chassis to support smooth horizontal expansion. Scalable to both large-scale clusters and multiple data centers. Its storage is to pool the local storage of each server node in the cluster through Software Defined Storage (SDS) to form a unified distributed storage. Therefore, the hyper-converged infrastructure solves the problems of long deployment time and difficult expansion of traditional information technology (Information Technology, IT) architecture, and realizes rapid deployment and expansion; simplifies storage and data management in virtual environments, and achieves performance acceleration and capacity o...

Examples

Embodiment 2

[0084] The embodiment of the present application provides a data deduplication device, which is applied to the above-mentioned network architecture. Refer to Figure 9 As shown in, the device 90 includes:

[0085] Segmentation unit 901, for segmenting the data stream at equal intervals to obtain segment data to be stored, and performing equal-interval block division of each segment data to be stored to obtain block data to be stored;

[0086] The obtaining unit 902 is configured to obtain the block hash value of the block data to be stored and the segment hash value of the segment data to be stored according to the block data to be stored obtained by the segmentation unit 901, wherein the block hash value of the block data to be stored It is used to uniquely index the block data to be stored, and the segment hash value of the segment data to be stored is used to uniquely index the segment data to be stored;

[0087] The search unit 903 is configured to search the block data i...