File storage method and system for distributed file system

A distributed file and system storage technology, applied in the field of distributed file systems, can solve the problems of wasting disk space, reducing the performance of distributed file systems, and data objects occupying large stripes, so as to avoid waste, improve performance and Adaptability and the effect of increasing availability

Inactive Publication Date: 2014-05-07
INSPUR BEIJING ELECTRONICS INFORMATION IND
View PDF5 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since the current stripe size is fixed, there will be a problem that a data object with a smaller capacity occupies a larger stripe, resulting in a waste of disk space
And it will reduce the performance of distributed file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • File storage method and system for distributed file system
  • File storage method and system for distributed file system
  • File storage method and system for distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] Example 1 as image 3 As shown, its specific implementation is as follows:

[0047] The function of adaptively adjusting stripes provided by the present invention includes three module groups: a file attribute parameter collector, an adaptive strategy engine and a strategy executor. Through the mutual cooperation of these three modules, the step of adaptively and dynamically adjusting the strips is jointly realized. The specific implementation steps are as follows:

[0048] The file attribute parameter collector is responsible for collecting various attribute parameters of the file during the process of writing the file into the distributed file system, such as block size, proportion of reading and writing, frequency of reading and writing, file name and other attribute parameters. Then the file attribute parameter collector organically saves these attribute parameters as the input of the subsequent adaptive policy engine. When collecting file attribute parameters, t...

Embodiment 2

[0054] Periodically collect file attribute parameters, or dynamically collect file attribute parameters on demand. Statistical file characteristics based on file attribute parameters. The specific characteristics include at least, within a fixed time period T, the characteristics of all files written into the folder. The characteristics here include file size, file read and write frequency, and file type. Wait.

[0055] Input the feature data collected within T time into the strategy engine generator. The strategy engine generator generates predicted adjustment strategies based on these data. The policy engine generator generates adjustment policies according to specified conditions. The specified condition at least includes: the frequency and size of the file written within T time conform to the specified characteristics. For example, the writing frequency of 1M files reaches more than 80%.

[0056] After the policy engine generator generates the adjustment policy, it com...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a file storage method for a distributed file system. The method comprises the following steps of acquiring attribute parameters of a file; generating an adaptive strategy for regulating a strip according to the attribute parameters of the file; regulating the size of the strip according to the adaptive strategy, and storing the file in the strip. According to the method, a mechanism for generating the adaptive strategy according to the size of a written file is established to dynamically regulate the strip, so that the performance and the adaptability of the distributed file system are improved, and the application range of the distributed file system is widened.

Description

technical field [0001] The invention relates to distributed file system technology, in particular to a method and system for storing files in a distributed file system. Background technique [0002] The computer distributed system is connected by a group of loosely integrated computer software and hardware, and the computer distributed system completes related computing work through highly close cooperation. At present, due to the explosive growth of personal and enterprise data, more and more distributed file systems are deployed on the cluster, and their capacity is also increasing. Moreover, due to the wide variety of system applications, the read-write characteristics and sizes of files also vary widely, which puts a lot of pressure on the distributed file system. [0003] At present, general distributed file systems are based on fixed and adjustable stripes, and the stripes are fixed before files are written into the distributed file system. Among them, striping (stri...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F12/06
CPCG06F16/122G06F16/182
Inventor 张敬海
Owner INSPUR BEIJING ELECTRONICS INFORMATION IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap