Method and system for data cleaning suitable for mass storage

A data cleaning and mass storage technology, applied in the fields of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as data cleaning work cannot be carried out normally, and achieve the effect of reliable Internet service, efficiency and reliability improvement

Inactive Publication Date: 2013-10-16
BEIJING NETEAST TECH
View PDF4 Cites 27 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the huge disk storage space, the number of saved files is much larger than when no cloud storage is used. When the cloud storage space is a

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for data cleaning suitable for mass storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] The content of the present invention will be described in detail below with reference to the accompanying drawings.

[0041] In order to achieve this objective, the data cleaning method and system adapted to mass storage provided by the present invention need to include the following subsystems:

[0042] 1. Data storage distribution subsystem.

[0043] In order to ensure the normal progress of subsequent data cleaning, the system needs to participate in and make corresponding decisions and processing when data is saved to cloud storage, which mainly includes the following:

[0044] Data is stored in a multi-level directory.

[0045] In order to facilitate the use of a large number of disks in the prior art, cloud storage technology is usually used. Cloud storage technology provides a mount point for upper-level applications. This mount point is a directory for upper-level applications, and the capacity is several hundred. TB, or even several petabytes, the upper application does...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and system for data cleaning suitable for mass storage. The method includes the steps that step (101), a plurality of levels of catalogues are built below a mount point of cloud storage, and files are stored in the mounted catalogues, wherein the names of all the levels of catalogues are formed according to a plurality of bytes of file names; step (102), a distribution strategy is adopted for carrying out searching on one appointed level of catalogue, last access time of all the files below the catalogue is obtained, and the distribution strategy is that a plurality of processes are started simultaneously; step (103), according to the difference values among the last access time of all the files and current scanning time, which files needing to be deleted is judged, the concrete steps are that an initial threshold value is set, the files of which the different values are larger than the initial threshold value are searched and serve as the files to be deleted; if the files to be deleted are not searched, the initial threshold value is reduced, the files of which the different values are larger than the reduced initial threshold value are searched again and serve as the files to be deleted, and the operation is carried out until released storage space meets needs.

Description

Technical field [0001] The invention relates to the problem of mass storage cleaning, and specifically a method and system for data cleaning of mass storage media. Background technique [0002] With the rapid development of the Internet, network operators continue to build basic network facilities, and bandwidth is constantly improving, but at the same time, Internet applications based on high bandwidth are constantly introducing new ones. Netizens are more pursuing online, real-time, and high-definition Internet application experience. As a result, Internet applications generate a large number of data files, and the capacity of storage media has also grown from GB to TB, and then to the current PB level. [0003] Because the capacity of a single disk is very limited, if you want to build a storage of hundreds of terabytes or even several petabytes, the disk data that needs to be managed will be very large. Therefore, mass storage technology has emerged. Cloud storage is one of the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 鲁冬林王超峰
Owner BEIJING NETEAST TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap