Garbage data recovery method in cloud storage log file system

A technology of garbage data and log files, which is applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problem of poor utilization of storage space and achieve the effects of improving utilization, flexible recycling, and saving storage space

Active Publication Date: 2018-08-14
NORTHWESTERN POLYTECHNICAL UNIV
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In order to overcome the shortcomings of poor storage space utilization in existing garbage data recovery methods, the present invention provides a garbage data recovery method in a cloud storage log file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Garbage data recovery method in cloud storage log file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0010] The specific steps of the garbage data recovery method in the cloud storage log file system of the present invention are as follows:

[0011] The present invention is based on a kind of distributed file system, and this distributed file system realizes the distributed file system of log structure in conjunction with journal file system on Hadoop DistributedFile System (HDFS), the present invention is designed and realized in order to solve the HDFS of Apache Foundation Similar to the garbage data recovery problem that the distributed file system cannot solve, it also improves the storage system such as Sheepdog. Although the garbage recovery system is implemented, there are still some problems, such as garbage recovery needs to consume additional storage space, and the garbage recovery is strong. , not flexible enough.

[0012] In order to solve the problem of garbage data recovery, the present invention firstly judges garbage data, and each time the distributed file sy...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for recovering junk data in a cloud storage log file system, used for solving the technical problem that the storage space of the existing junk data recovery method is poor in utilization rate. The technical scheme provided by the invention is as follows: the junk data is judged at first; a new log is established when the file system is updated; an index node in the log comprises a three-grade index structure; each file corresponds to one index node; an index address of a data block is stored in the three-grade index structure; the junk data is searched by taking the latest log as the base; the index addresses in the logs are sequentially compared from the earliest log; the data is stored in the log by the cloud storage log file system; the log is also stored in a segment storage file; and the junk data is recovered when the junk data amount of the segment storage file is more than a threshold value. According to the invention, the junk data is recovered manually or automatically; manual configuration of a segment file recovery threshold value is supported; the storage space occupied by the junk data can be recovered at any time; and thus, the utilization rate of the storage space is improved.

Description

technical field [0001] The invention relates to a garbage data recovery method, in particular to a garbage data recovery method in a cloud storage log file system. Background technique [0002] With the increasing amount of Internet big data, major Internet giants have launched their own storage systems, and these storage systems have also become industry standards. Google designed and implemented Google File System (GFS) and key-value storage system LevelDB, Amazon designed and implemented Simple Storage System (S3) and key-value storage system Dynamo, Yahoo! Design and implement PNUTS, Facebook design and implement Cassandra, etc. Most of these storage systems are not open source, so open source organizations have also designed and implemented open source storage systems based on their published papers. For example, the Apache Foundation designed and implemented the GFS open source Hadoop Distributed File System. These storage systems are designed for the characteristics...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
CPCG06F16/134G06F16/148G06F16/162G06F16/174G06F16/1815
Inventor 贾威威张延园林奕
Owner NORTHWESTERN POLYTECHNICAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap