Data processing method, device, equipment and storage medium

A data processing and data technology, which is applied in the storage field and can solve the problems of amplification of upper-layer business write delay and so on.

Pending Publication Date: 2020-07-07
SANGFOR TECH INC
View PDF3 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In related technologies, data deduplication is often performed when data is written. For example, by fingerprint calculation and comparison of written data, it is decided whether to send a data write request to the back-end hardware storage device. Since each write request is Fingerprint calculation and comparison are required, which will increase the writing delay of upper-layer services

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device, equipment and storage medium
  • Data processing method, device, equipment and storage medium
  • Data processing method, device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of the invention. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

[0028] An embodiment of the present invention provides a data processing method, which is applied to a data processing device. The data processing device may be a client or server for data reading and writing, or a data reading and writing processing node in a distributed storage system. Each data reading and writing processing node can write data into the shared storage pool or read data from the storage pool. Such as figure 1 As shown, the method includes:

[0029] Step 10...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method, a device, equipment and a storage medium. The method comprises the steps of obtaining to-be-written data in a writing request; determining metadata corresponding to the to-be-written data; writing the to-be-written data and metadata corresponding to the to-be-written data into a target file; determining that the to-be-written data and metadata corresponding to the to-be-written data are written into the target file, and generating response information that the to-be-written data are successfully written; and carrying out duplicate removal on the data written into the target file in the write request. According to the method of the invention, the write delay is reduced, the write request can be quickly responded, and the data written into the target file in the write request is subjected to duplicate removal, so that the write delay of an upper-layer service during data writing can be reduced, the data duplicate removal can be effectively realized, and the utilization rate of a storage space is improved.

Description

technical field [0001] The present invention relates to the field of storage, in particular to a data processing method, device, equipment and storage medium. Background technique [0002] Data deduplication, also known as data deduplication (Data Deduplication), is a technology applied in a storage system to globally identify and eliminate redundant data. In order to allow limited storage resources to store more data, most distributed storage systems use data deduplication. [0003] In related technologies, data deduplication is often performed when data is written. For example, by fingerprint calculation and comparison of written data, it is decided whether to send a data write request to the back-end hardware storage device. Since each write request is Fingerprint calculation and comparison are required, which will increase the writing delay of upper-layer services. Contents of the invention [0004] In view of this, the embodiments of the present invention provide a ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0608G06F3/0611G06F3/0641G06F3/067
Inventor 张欢
Owner SANGFOR TECH INC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap