Method and device for deleting duplicate data

A technology for deleting duplicate data and duplicate files, which is applied in electrical digital data processing, special data processing applications, instruments, etc., to achieve the effects of improving system performance, easy expansion, and reducing search time

Active Publication Date: 2015-03-11
ZHEJIANG UNIVIEW TECH
View PDF5 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] This application provides a method and device for deleting duplicate data based on the O

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for deleting duplicate data
  • Method and device for deleting duplicate data
  • Method and device for deleting duplicate data

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0034] The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

[0035] The terms used in this application are only for the purpose of describing specific embodiments, and are not intended to limit the application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0036] This application provides a set of solut...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses method and device for deleting duplicate data on the basis of the Openstack Object Storage system. The method and device are characterized in that a proxy node comprises a duplicate removing middleware module; a storage node comprises a duplicate removing service process module; a duplicate removing hash ring is built in the duplicate middleware module, and each node of the duplicate removing hash ring is a root node of a red-black tree; a fingerprint file is sent to the duplicate removing duplicate module through the duplicate removing service process module; a duplicate file can be determined by the duplicate removing middleware module after finding the root node of the red-black tree; then one duplicate file is remained in the storage node, and other duplicate files are deleted; a redirection file directed to the remained duplicate file is stored at the position in which other duplicate files are stored before; if no duplicate file is found, the value of a virtual node partition in each fingerprint file and the MD5 value of the file content are inserted into a sub-node of the red-black tree. According to the method and device, the transverse expansion advantages of linear increase of the performance of the Openstack Object Storage system are fully utilized, and thus the node can be easily expanded.

Description

technical field [0001] This application relates to the Openstack Object Storage cloud storage technology, in particular to a method and device for deleting duplicate data based on the Openstack Object Storage system. Background technique [0002] Openstack Object Storage (swift) is an object storage sub-solution of the Openstack open source cloud computing project, which provides powerful scalability, redundancy and persistence. Its structure is as follows: figure 1 Framework diagram of Openstack Object Storage. [0003] Such as figure 1 , Openstack Object Storage mainly consists of two types of nodes: proxy (proxy) nodes and storage (storage) nodes. The proxy node is responsible for receiving the client's request and communicating with the storage node. It locates and forwards the request to the storage node according to the object requested by the client. The storage node is mainly responsible for data storage, providing data security guarantees such as backup, fault ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/1748
Inventor 张朝潞
Owner ZHEJIANG UNIVIEW TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap