File storage method and device

A file storage and file technology, applied in the field of file storage methods and devices, can solve problems such as inability to store files, and achieve the effects of reducing size and space, optimizing utilization, and reducing write amplification

Active Publication Date: 2019-07-23
BEIJING QIHOO TECH CO LTD
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

For example, under the common xfs file system, the size of the index node inode is 536bytes, and the size of ordinary files is 40kb. We use a storage model with 32G memory size and 88T disk space online, so in this storage Under the model, 32G / 536byte is much smaller than 88T / 40kb, which means we cannot save so many files

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • File storage method and device
  • File storage method and device
  • File storage method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0066] figure 1 It is a schematic flowchart of a file storage method according to an embodiment of the present invention. The file storage method is applied to a storage engine for storing multiple small files, wherein the multiple small files may be a large number of small files, and a small file means that the storage space occupied by a file is smaller than a file threshold (for example, 1M). Such as figure 1 As shown, the met...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a file storage method and device. The file storage method and device are applied to a storage engine for storing multiple small files, wherein the small files are files with the occupied storage space smaller than a file threshold value. The method comprises the steps that the storage engine is used for partitioning a storage space, and unit storage spaces are distributed in advance; corresponding index nodes are set in the storage engine for the small files to be stored in the unit storage spaces; when the storage engine receives a data writing-in request of a new small file, the index modes are maintained unchanged, the new small file is written into the corresponding unit storage space and saved. According to the technical scheme, fragments of the space of a disk are reduced; besides, the content in the index nodes does not need to be modified when data is written into the storage space, so that system calling is reduced, and the utilization rate of the IO of the disk is optimized.

Description

technical field [0001] The invention relates to the technical field of file processing, in particular to a file storage method and device. Background technique [0002] Since the emergence of massive small file storage, it has always been a major problem in the field of data storage. The leveldb database based on the LSMtree structure is more suitable for scenarios where certain keywords are frequently written online, and the goal defined by our small file storage engine is to store user file data, and the characteristic of file data is that the data is stored after writing , There are very few cases of modification, such as after a picture is stored, it is either deleted, but there is no case of modifying the content of the picture. Since the LSM tree has a hierarchical concept, a piece of data will be frequently written and merged, so the LSMtree structure based on the leveldb database has the problem of write amplification. [0003] In related technologies, the storage ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/13
CPCG06F16/13
Inventor 陈宗志
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap