System and method for eliminating redundancy file of file storage system

A redundant file and file storage technology, which is applied in the field of data processing, can solve the problems of redundant file space occupied by redundant processing and low efficiency of redundant processing, and achieve the effects of reducing storage costs, ensuring security, and load sharing

Active Publication Date: 2007-11-28
SHENZHEN TENCENT COMP SYST CO LTD
View PDF0 Cites 65 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The technical problem to be solved by the present invention is to provide a new system and method for eliminating redundant files in the file storage

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • System and method for eliminating redundancy file of file storage system
  • System and method for eliminating redundancy file of file storage system
  • System and method for eliminating redundancy file of file storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] As shown in Figure 1, the system for eliminating redundant files in the file storage system of the present invention includes an index server module 11, a storage server module 12, a preprocessing module 13, a quick access module 16 and a deduplication service module 15, wherein the index server The module 11, the storage server module 12, and the preprocessing module 13 are connected sequentially, and the quick access module 16, the index server module 11, the storage server module 12, and the preprocessing module 13 are respectively connected to the deduplication service module 15.

[0031] The storage server module 12 and the index server module 11 are a part of the mass storage system, wherein the storage server module 12 mainly completes the preservation of specific file data, and is generally composed of dozens or even hundreds of servers to realize distributed storage of files; The server module 11 mainly saves the logical directory of each user and the logical na...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a system to remove redundant file in file storing system, which is characterized by the following: comprising sea quantity memory system; constructing with the memory server module and index server module; also comprising pre-treating module, quick storing module and removing repeat server module; getting file MD5 value and file size with the pre-treating module; proceeding file quoting number statistic with the quick storing module; deleting repeated file when the file quoting number reach intended value with the removing repeat server module; connecting the index server module, the storing server module and the pre-treating module consecutively; connecting the quick access module, index server module, storing server module and the pre-treating module to the removing repeat server module. This invention also discloses a corresponding method to remove redundant file in accessing system. This invention can control carrying cost effectively.

Description

technical field [0001] The invention relates to the field of data processing, more specifically, to a system and method for quickly eliminating redundant files in a file storage system. Background technique [0002] In any large-scale storage system, there are a large number of redundant files. For example, there are often multiple copies of a file in the local hard disk of a personal computer. Many of these copies are due to operational errors or user forgetting. caused by copying. However, in a system shared by many people, due to poor sharing between users, the redundancy of files is very high. With the rise of network hard drives, many Internet companies provide users with many redundant files in the network hard drives. In addition, there are also a large number of repeated web pages and files in web search engines. For a file storage system, the main cost is the equipment cost, and eliminating redundant files in the system can greatly save equipment costs, and elimi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 熊展志张立明
Owner SHENZHEN TENCENT COMP SYST CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap