Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A data processing method, system and client

A data processing system and data processing technology, applied in the storage field, can solve problems such as the deduplication performance of the system, and achieve the effects of improving the deduplication performance, reducing the delay, and reducing the occupation of network bandwidth.

Active Publication Date: 2016-11-09
HUAWEI TECH CO LTD
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The inventor found through research that in the prior art cluster deduplication technology, the sampled fingerprint values ​​need to be sent to all physical nodes for query, resulting in too many interactions between physical nodes during the deduplication process. When the data processing system When there are many physical nodes, when each physical node performs deduplication, the calculation amount will increase with the increase of physical nodes in the data processing system, resulting in a decrease in deduplication performance of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing method, system and client
  • A data processing method, system and client
  • A data processing method, system and client

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0046] An embodiment of the present invention provides a data processing system, and the data processing system includes at least one client and multiple storage nodes, and the client and storage nodes may be deployed in various manners. The embodiment of the present invention provides two deployment methods, for example, as Figure 1-A As shown, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the present invention provides a data processing method, system, and client. The target storage is determined by comparing the second vector of received data with the first vector corresponding to all storage nodes pre-stored on the client receiving data. node, instead of sampling a part of the fingerprint value from the received data and sending it to all storage nodes in the data processing system for query and waiting for the feedback from the storage node to determine the target storage node, thereby avoiding the need The multiple interactions between them improve deduplication performance, reduce network bandwidth occupation, and reduce latency.

Description

technical field [0001] Embodiments of the present invention relate to storage technologies, and in particular, to a data processing method, system, and client. Background technique [0002] Data deduplication, also known as intelligent compression or single instance storage, is a method that can automatically search for duplicate data, keep only one copy of the same data, and replace other duplicate copies with pointers to the single copy to eliminate redundancy Data storage technology that reduces storage capacity requirements. [0003] In the prior art, the data deduplication technology is widely used in backup, virtual desktop and other application environments. The data processing system is composed of multiple storage nodes. Each storage node has its own deduplication processing engine and storage medium, such as a hard disk. When data needs to be written to a file, the data is divided into blocks in the cache. Obtain multiple data blocks, calculate the fingerprint va...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
CPCG06F12/0813G06F16/1752G06F16/215G06F16/2282
Inventor 黄岩
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com