Distributed file system and method for storing lots of small files

A technology for distributed files and massive small files, applied in file systems, file access structures, special data processing applications, etc., can solve the problem of low efficiency of massive small file storage, simplify IO access process, and reduce the number of reads and writes , Improve the effect of processing performance

Active Publication Date: 2015-12-09
SICHUAN CHANGHONG ELECTRIC CO LTD
View PDF4 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a distributed file system and its method for storing small files, so as to solve the problem of low storage efficiency of massive small files

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system and method for storing lots of small files

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] The first embodiment provides a distributed file system, figure 1 It is a distributed file system provided by Embodiment 1 of the present invention. Such as figure 1 As shown, the system includes a client, a resource location module, and a data storage module,

[0049] The client is the user's operation portal, which is respectively connected to the resource locating module and the data storage module. The client is configured to send a small file upload request to the resource locating module and upload the small file to the data storage module.

[0050] The resource locating module includes at least one resource locator, the resource locator is configured to: process the small file upload request sent by the client, allocate the source node for the client, and designate a data storage node in the data storage server as the main server of the combined storage , The main server is configured to: allocate free storage units for small files and complete the combined storage ...

Embodiment 2

[0055] The second embodiment provides a method for storing a large number of small files based on the distributed file system in the first embodiment. The method includes the following steps: building a distributed file system, uploading small files, synchronization of source nodes, and synchronization of the main server. The steps are explained in detail.

[0056] (1) Establish a distributed file system

[0057] Specifically, a distributed file system of a component embodiment.

[0058] (2) Small file upload

[0059] The upload of small files includes steps 2.1 to 2.10, and steps 2.1 to 2.10 are described in detail below.

[0060] Step 2.1: When the user needs to upload a small file, the client initiates an upload request to the resource locator.

[0061] Step 2.2: The resource locator queries available data storage nodes as source nodes and designates the main server.

[0062] Step 2.3: The resource locator returns the IP and port information of the source node to the client.

[0063] S...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of data storage and provides a distributed file system and a method for storing lots of small files to solve the problem of relatively low efficiency for storing lots of small files at present. The system comprises a client, a resource location module and a data storage module. According to the technical scheme provided by the invention, the small files are combined into a large file to be stored and the small files are accessed by positioning a specific position of the large file, so that the file read-write frequency is reduced, the IO access process is simplified, and the processing performance of a file system is improved.

Description

Technical field [0001] The invention relates to the field of data storage, in particular to a distributed file system and a method for storing massive small files. Background technique [0002] Nowadays, human beings have entered the information age, and the total amount of data generated by all walks of life is exploding. For example, the data generated by the New York Stock Exchange in one day can reach a level of T. The well-known social networking site Facebook is global There are hundreds of millions of users, so the user data that needs to be stored is very large. Among these huge user data, the proportion of massive small files is gradually increasing. In this context, how to store these file data generated by various applications such as social networks, mobile communications, network video and audio, scientific experiments, etc., effectively and safely, and convenient for users to find them at any time, has become information technology. Problems that need to be solved ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/13G06F16/1824
Inventor 杨雪莲李强姜韬
Owner SICHUAN CHANGHONG ELECTRIC CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap