Fragmented-file storage method based on distributed storage system

A distributed storage and file storage technology, which is applied in the direction of memory system, memory address/allocation/relocation, special data processing applications, etc., can solve the problem of low efficiency of fragmented file processing, and achieve the effect of improving efficiency and improving utilization efficiency

Active Publication Date: 2014-01-01
上海网达软件股份有限公司
View PDF5 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to overcome the low efficiency of fragmented file processing in the prior art, the present invention provides a fragmented file storage method based on a distributed storage system. The main idea of ​​the present invention is to continuously map fragmented files to In memory, merging into a large file and then storing the large file can greatly reduce the space occupied by fragmented files and improve the efficiency of file processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fragmented-file storage method based on distributed storage system
  • Fragmented-file storage method based on distributed storage system
  • Fragmented-file storage method based on distributed storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0039] Such as figure 1 As shown, the present invention provides a method for storing fragmented files based on a distributed storage system, comprising the following steps:

[0040] Step S1, allocate a fragmented file mapping space in the memory of the storage node, and the fragmented file mapping space is continuous.

[0041] Step S2, in the storage node, judge several source files according to the preset threshold value, if the size of a certain source file is smaller than the threshold value, then the source file is a fragment file, and obtain several fragment files after judgment; wherein, each Each source file is preset with a corresponding source file number.

[0042] Wherein, the source file refers to the file uploaded by the client application program to the storage node through the router. The source file number is assigned by the router when the source file is uploaded by the router. A request table is set in the router, and the request table is used to record th...

Embodiment 2

[0056] The difference between this embodiment and Embodiment 1 is that the fragmented file mapping space adopts non-swap memory. Due to the use of non-swappable memory, the fragmented file mapping space will not automatically release the content in the space, and some swap methods need to be set to realize the exchange of files between the fragmented file mapping space and the disk.

[0057] The priority is preset for each large file, and the exchange of large files between the fragmented file mapping space and the disk is determined according to the priority. The priority may be set according to the following criteria: the frequency of file access, or the time of the last access, or the weighting of historical access records; but not limited thereto, other criteria may also be used. Take the standard of the frequency of file access as an example, if a large file has more access times, the priority of the large file is higher, otherwise, the priority is lower; large files with...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a fragmented-file storage method based on a distributed storage system. A memory map file technology is adopted to continuously map fragmented files into a memory, the fragmented files are combined into a large file, and the large file is stored. Corresponding to the method, the invention further provides storage nodes comprising a fragmented-file judgment unit, a fragmented-file mapping space, a fragmented-file metadata unit, a memory mapping space metadata unit, a pointer list unit and a memory exchange control unit. The method and the storage nodes have the following beneficial effects that as the fragmented files are combined into the large file to be stored, the utilization efficiency of a disk is greatly improved, and the efficiency of processing continuous fragmented files is greatly improved; the memory mapping space adopts the non-exchange memory, and corresponding file priority and exchange strategy are matched, so that the efficiency of processing random fragmented files is greatly improved.

Description

technical field [0001] The invention belongs to the technical field of computer file storage, and in particular relates to a method for storing fragmented files based on a distributed storage system. Background technique [0002] The file system is the interface between the computer operating system and the drive. When the operating system requests to read a file from the hard disk, it will request the corresponding file system (FAT 16 / 32, NTFS, Ext2 / 3 / 4) to open the file. A sector is the smallest physical storage unit of a disk, but since the operating system cannot address a large number of sectors, the operating system combines adjacent sectors together to form a cluster, and then manages the cluster. Each cluster can include 2, 4, 8, 16, 32 or 64 sectors. Obviously, a cluster is a logical concept used by the operating system, not a physical property of the disk. In order to better manage disk space and read data from the hard disk more efficiently, the operating system...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F12/08G06F12/0802
CPCG06F16/1724G06F16/1827
Inventor 杨源杰陈化峰李庆瑜戴立言王振邦戴彩霞
Owner 上海网达软件股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap