Fog storage data redundancy removal method based on repeated data deletion

A technology for deduplication and data storage, applied in the input/output process of data processing, electrical digital data processing, digital data information retrieval, etc., to prevent sudden crashes, improve query speed, and improve fingerprint query speed.

Active Publication Date: 2019-12-27
SHANGHAI UNIVERSITY OF ELECTRIC POWER
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The present invention aims at the problem of data duplication existing in fog nodes at present, and proposes a fog storage data de-redundancy method based on data deduplication. Putting it in memory ensures query efficiency, reduces unnecessary disk IO, and avoids network delays

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fog storage data redundancy removal method based on repeated data deletion
  • Fog storage data redundancy removal method based on repeated data deletion
  • Fog storage data redundancy removal method based on repeated data deletion

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Such as figure 1 The schematic diagram of the deduplication system for fog storage data based on data deduplication is shown. The system is divided into: mobile terminal area, fog computing area and remote cloud computing area. The fog computing area and remote cloud computing area are also called fog nodes and cloud nodes. .

[0027] 1. Mobile terminal area: mobile devices that can be connected to the local area network, such as mobile phones, computers, tablets, smart watches, etc.

[0028] 2. Fog computing area: Considering the limited storage space and computing power of lightweight mobile devices, the fog computing area receives requests / services from mobile devices in the mobile terminal area, processes and stores a small amount of frequently accessed data that does not need to be placed in the cloud . Each fog computing area consists of four parts: fog manager (Fog Manager), process log management server (Services Logging Server), fog computing server (Fog Comp...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a fog storage data redundancy removal method based on repeated data deletion, which comprises the following steps: aiming at data with higher access frequency in a fog node, constructing an index table in a memory, and taking a red-black tree corresponding to each index value as a structure for storing data fingerprints, so that I / O (Input / Output) between a disk and the memory is reduced, and the query speed is increased; judging whether the data blocks with the same data fingerprint are repeated or not by using a cyclic redundancy code (CRC) technology, and storing conflict data blocks in fingerprint nodes by using a linked list structure, so that the hash conflict problem generated during calculation of the data fingerprint is solved; persistently storing the fingerprint table in the memory, and dividing the fingerprint table into a mapping file of the memory at a certain moment and a log file for recording update to prevent sudden crash of the system; the data redundancy elimination scheme conforming to the actual situation of the fog node is provided, disk I / O is reduced in a network with high requirements for communication efficiency, and the fingerprint query speed is increased.

Description

technical field [0001] The invention relates to a data storage technology, in particular to a data de-redundancy method for fog storage based on data de-duplication. Background technique [0002] The explosive growth of digital information in today's society, the rapid increase and flood of data volume has posed a huge challenge to data storage management technology, how to efficiently manage and store data has become a research hotspot. In the primary storage system and secondary storage system produced by Microsoft and EMC, there are 50% and 85% redundant data respectively. As time goes by, the proportion of redundant data increases exponentially. Enterprises need to store these data costs will also increase. So technicians turned to cloud storage systems with larger capacity and lower cost. Cloud storage adopts a data outsourcing model. In order to reduce costs, many cloud service providers set up data centers in low-cost remote areas. When the cloud server is far away...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F16/174
CPCG06F3/0608G06F3/0641G06F3/0662G06F16/1748Y02D10/00
Inventor 温蜜陈思佳李晋国陈珊吴佳乐徐健郭帅王轩楷李孟特
Owner SHANGHAI UNIVERSITY OF ELECTRIC POWER
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap