Method for optimizing distributed file system small-file writing cache

A technology of distributed files and small files, applied in special data processing applications, instruments, electrical digital data processing, etc., can solve the problem that the content of small files cannot be effectively merged, and achieve the effect of improving writing efficiency

Inactive Publication Date: 2017-06-06
ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
View PDF7 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] When writing large files, the data can be merged on the client side to improve the refresh efficiency, but the content of small files cannot be effectively merged, so it is urgent to invent a method for optimizing the write cache of small files

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for optimizing distributed file system small-file writing cache

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0032] In the present invention, when the file system performs small file writing operations, the written data is saved to the client cache and managed according to the inode distinction, and then the timer regularly refreshes the dirty data to the back-end storage pool, writes in batches, and refreshes in batches To improve the writing efficiency of small files.

[0033] A method for optimizing write cache of small files in a distributed file system, the specific method steps are as follows:

[0034] S1. Mount the file system, and specify a directory to enable the cache function; the opening and closing function of the cache function uses the directory as the basic unit of operation, and the subdirectory inherits the opening and closing function of the cache function.

[0035] S2. Save the written data to the client cache;

[0036] S3. The cache management module performs dirty data management according to the inode; add an index in each inode of the client to manage the dir...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for optimizing distributed file system small-file writing cache, and relates to the field of computer storage. The method particularly includes the steps: S1 mounting a file system, and opening a cache function by a designating catalogue; S2 saving written data to a client to cache; S3 performing dirty data differentiated management by a cache management module according to inode; S4 regularly refreshing the dirty data to rear end storage pool by a cache refreshing module. According to the method, catalogue layout is performed, opening and closing control of cache functions is achieved and is inherited by the aid of a sub catalogue, operations for refreshing inode are triggered by a timer, synchronous refreshing is changed into asynchronous refreshing, batch refreshing of inode cab achieved, refreshed data contents are connected with inode intensity, small-file writing efficiency can be effectively improved, and writing operations of a large file and the small file can be achieved.

Description

technical field [0001] The invention relates to the field of computer storage, in particular to a method for optimizing small file write cache in a distributed file system. Background technique [0002] With the rapid development of computer technology, the amount of information and data in various forms is increasing, and the storage of massive data at the PB level is becoming more and more important. Traditional file systems can no longer meet the requirements of large capacity, high reliability, and high performance of existing applications. In order to meet these new requirements, distributed file systems have received extensive attention. [0003] The existing research on distributed file systems is mainly divided into separate management of metadata and actual data storage. Through the isolation of metadata and actual data, the dynamic increase and decrease of capacity and the current expansion of performance of distributed file systems are realized. [0004] However,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/172G06F16/1727G06F16/182
Inventor 于鲁宪
Owner ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap