Unlock instant, AI-driven research and patent intelligence for your innovation.

A data deduplication method, system, device and computer-readable storage medium

A data and target data technology, applied in the field of storage, can solve problems affecting device performance, etc., to achieve the effect of improving efficiency, reducing resource consumption, and improving computing efficiency

Active Publication Date: 2021-10-22
SUZHOU METABRAIN INTELLIGENT TECH CO LTD
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, the core idea of ​​judging whether the data is duplicate data is to calculate the fingerprint value of the data, and the calculation of the fingerprint value needs to occupy a large amount of CPU (central processing unit, central processing unit) resources, thereby affecting the performance of the device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data deduplication method, system, device and computer-readable storage medium
  • A data deduplication method, system, device and computer-readable storage medium
  • A data deduplication method, system, device and computer-readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0053] see figure 1 , figure 1 It is a flowchart of a data deduplication method provided by the embodiment of this application.

[0054] A data deduplication method provided in the embodiment of the present application can be applied to devices such as servers and user terminals, and includes the following steps:

[0055] Step S101: Read target data of a preset size from the target storage device.

[0056] In practical applications, target data of a preset size may be read from t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application discloses a data deduplication method, system, device and computer-readable storage medium, in which target data of a preset size is read in the target storage device; the target data is calculated by the SSE instruction set to obtain the preset size Corresponding calculation data; perform hash operation on the calculation data to obtain the corresponding hash value; obtain the fingerprint value of the target data in the target storage device; judge whether the hash value is consistent with the fingerprint value, and if so, no longer save the target data Data is written to the target storage device. The data deduplication method provided by this application realizes the use of the SSE instruction set to improve the efficiency of computing the target data, thereby improving the computing efficiency of the hash value, and only needs to judge whether the hash value is consistent with the fingerprint value. Judging whether the target data is duplicate data can reduce CPU resource consumption. The data deduplication system, equipment and computer-readable storage medium provided by the present application also solve corresponding technical problems.

Description

technical field [0001] The present application relates to the field of storage technology, and more specifically, to a data deduplication method, system, device, and computer-readable storage medium. Background technique [0002] At present, in the field of storage, the query and storage of massive data require huge resources, seriously affecting the performance of data storage. In order to reduce the resources required to store data and improve data storage performance, an existing method is to deduplicate data, which means to delete duplicate data, so that only one copy of the same data remains in the storage device , on the premise of not affecting data consistency, reduce the amount of data stored on the disk. [0003] However, the core idea of ​​judging whether the data is repeated data is to calculate the fingerprint value of the data, and the calculation of the fingerprint value needs to occupy a large amount of CPU (central processing unit, central processing unit) ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06
CPCG06F3/0608G06F3/0641G06F3/0659G06F3/0676G06F3/0679
Inventor 岳斌
Owner SUZHOU METABRAIN INTELLIGENT TECH CO LTD