Data processing method and device
A data processing and data block technology, applied in the storage field, can solve the problems of increasing server-side data storage, reducing the efficiency of deduplication, and increasing network bandwidth consumption, so as to achieve the effect of reducing data storage and improving the deduplication rate.
Inactive Publication Date: 2011-09-28
HUAWEI DIGITAL TECH (CHENGDU) CO LTD
View PDF3 Cites 11 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] In the existing data deduplication technology, most of the fixed-length data block cutting methods are used to cut the files to be backed up. Update, etc. At this time, if the traditional fixed-length data block cutting method is used, even if the amount of data modified in the original backup file is small, the existing data blocks in the original file will be moved sequentially, so the changed file The number of duplicate data blocks found in the previous backup will be reduced, which will reduce the efficiency of data deduplication and cause more data blocks to be transmitted to the server. This will increase the consumption of network bandwidth on the one hand, and on the other hand will Increase server-side data storage
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
[0038] see Figure 1 , is a data processing method disclosed in an embodiment of the present invention;
[0039] In the embodiment of the present invention, the client has completed the first backup of the file. After a period of time, the original file changes, that is, the changed file needs to be backed up to the server again to complete the u...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention discloses a data processing method, which comprises the following steps of: calculating fingerprint data of each data block of a file to be backed up and fingerprint data of the data block after the offset of the address of each data block of the file to be backed up is progressively increased; sending the fingerprint data of each data block of the file to be backed up and the fingerprint data of the data block after the offset of the address of each data block of the file to be backed up is progressively increased to a server to compare with the fingerprint data of each data block of a file stored by the server and the fingerprint data of the data block after the offset of each data bock of the stored file is progressively increased; and sending the changed data to the server according to the comparison result sent by the server. The deletion rate of the repeated data can be improved.
Description
technical field [0001] The invention relates to the storage field, in particular to a data processing method and device. Background technique [0002] As the amount of enterprise data continues to increase, a large amount of duplicate data poses severe challenges to storage. Data de-duplication (De-Dupe for short), as an important technology for effectively reducing data and reducing data storage costs, has become the focus of everyone's attention. [0003] In the implementation of deduplication technology, the system calculates and checks the fingerprint data of a data block (or file), which is used to uniquely identify a file or a data block of a file, and judges whether the data block is consistent with Stored metadata is duplicated. If it is a duplicate, you only need to keep a pointer to this metadata, and if the fingerprint data shows that the data block is brand new, then keep the data block and use it as metadata for later use. [0004] In the existing data dedupl...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08H04L29/06
CPCH04L29/08G06F11/1453G06F11/10H04L29/06G06F17/30156
Inventor 任欣何非
Owner HUAWEI DIGITAL TECH (CHENGDU) CO LTD