Method for recovering junk data in cloud storage log file system

A technology of garbage data and log files, which is applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problem of poor utilization of storage space and achieve the effects of improving utilization, flexible recycling, and saving storage space

Active Publication Date: 2015-11-11
NORTHWESTERN POLYTECHNICAL UNIV
View PDF2 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In order to overcome the shortcomings of poor storage space utilization in existing garbage data recovery methods, the present invention provides a garbage data recovery method in a cloud storage log file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for recovering junk data in cloud storage log file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0010] The specific steps of the garbage data recovery method in the cloud storage log file system of the present invention are as follows:

[0011] The present invention is based on a kind of distributed file system, and this distributed file system realizes the distributed file system of log structure in conjunction with log file system on Hadoop DistributedFileSystem (HDFS), and the present invention is designed and realized in order to solve the HDFS etc. similar The problem of garbage data recovery that cannot be solved by the distributed file system has also been improved. Although the garbage recovery system has been implemented in storage systems such as Sheepdog, there are still some problems. For example, garbage recovery needs to consume additional storage space, and the garbage recovery is not strong enough. flexible.

[0012] In order to solve the problem of garbage data recovery, the present invention firstly judges garbage data, and each time the distributed fil...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for recovering junk data in a cloud storage log file system, used for solving the technical problem that the storage space of the existing junk data recovery method is poor in utilization rate. The technical scheme provided by the invention is as follows: the junk data is judged at first; a new log is established when the file system is updated; an index node in the log comprises a three-grade index structure; each file corresponds to one index node; an index address of a data block is stored in the three-grade index structure; the junk data is searched by taking the latest log as the base; the index addresses in the logs are sequentially compared from the earliest log; the data is stored in the log by the cloud storage log file system; the log is also stored in a segment storage file; and the junk data is recovered when the junk data amount of the segment storage file is more than a threshold value. According to the invention, the junk data is recovered manually or automatically; manual configuration of a segment file recovery threshold value is supported; the storage space occupied by the junk data can be recovered at any time; and thus, the utilization rate of the storage space is improved.

Description

technical field [0001] The invention relates to a garbage data recovery method, in particular to a garbage data recovery method in a cloud storage log file system. Background technique [0002] With the increasing amount of Internet big data, major Internet giants have launched their own storage systems, and these storage systems have also become industry standards. Google designed and implemented Google File System (GFS) and key-value storage system LevelDB, Amazon designed and implemented SimpleStorageSystem (S3) and key-value storage system Dynamo, Yahoo! Design and implement PNUTS, Facebook design and implement Cassandra, etc. Most of these storage systems are not open source, so open source organizations have also designed and implemented open source storage systems based on their published papers. For example, the Apache Foundation designed and implemented the GFS open source version of Hadoop DistributedFileSystem. These storage systems are designed for the characte...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/134G06F16/148G06F16/162G06F16/174G06F16/1815
Inventor 贾威威张延园林奕
Owner NORTHWESTERN POLYTECHNICAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap