A method and device for deleting duplicate data

A technology for removing duplicate data and duplicate files, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., to achieve the effect of easy expansion, improved system performance, and easy nodes

Active Publication Date: 2017-12-15
ZHEJIANG UNIVIEW TECH CO LTD
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] This application provides a method and device for deleting duplicate data based on the Openstack Object Storage system to solve the problem of deleting duplicate data in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and device for deleting duplicate data
  • A method and device for deleting duplicate data
  • A method and device for deleting duplicate data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

[0035] The terminology used in this application is for the purpose of describing particular embodiments only, and is not intended to limit the application. As used in this application and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise.

[0036] This application provides a set o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses method and device for deleting duplicate data on the basis of the Openstack Object Storage system. The method and device are characterized in that a proxy node comprises a duplicate removing middleware module; a storage node comprises a duplicate removing service process module; a duplicate removing hash ring is built in the duplicate middleware module, and each node of the duplicate removing hash ring is a root node of a red-black tree; a fingerprint file is sent to the duplicate removing duplicate module through the duplicate removing service process module; a duplicate file can be determined by the duplicate removing middleware module after finding the root node of the red-black tree; then one duplicate file is remained in the storage node, and other duplicate files are deleted; a redirection file directed to the remained duplicate file is stored at the position in which other duplicate files are stored before; if no duplicate file is found, the value of a virtual node partition in each fingerprint file and the MD5 value of the file content are inserted into a sub-node of the red-black tree. According to the method and device, the transverse expansion advantages of linear increase of the performance of the Openstack Object Storage system are fully utilized, and thus the node can be easily expanded.

Description

technical field [0001] The present application relates to Openstack Object Storage cloud storage technology, in particular to a method and device for deleting duplicate data based on the Openstack Object Storage system. Background technique [0002] Openstack Object Storage (swift) is an object storage sub-solution of the Openstack open source cloud computing project, which provides powerful scalability, redundancy and persistence. Its structure is as follows: figure 1 Framework diagram of Openstack Object Storage. [0003] Such as figure 1 , Openstack Object Storage mainly consists of two types of nodes: proxy (proxy) nodes and storage (storage) nodes. The proxy node is responsible for receiving the client's request and communicating with the storage node. It locates and forwards the request to the storage node according to the object requested by the client. The storage node is mainly responsible for data storage, providing data security guarantees such as backup, fau...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
CPCG06F16/1748
Inventor 张朝潞
Owner ZHEJIANG UNIVIEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap