Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Distributed file system and method based on multi-client collaboration and client

A distributed file, multi-client technology, applied in the field of distributed file systems, can solve the problems of slow metadata access, slow real-time file writing, and inability to fully access real-time data from multiple clients, saving bandwidth and speeding up query access. Effect

Active Publication Date: 2021-07-30
JINQIANMAO TECH CO LTD
View PDF4 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] To this end, it is necessary to provide a distributed file system based on multi-client collaboration to solve the problems of slow metadata access, slow real-time file writing, and incomplete access to multi-client real-time data in existing distributed file systems.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system and method based on multi-client collaboration and client
  • Distributed file system and method based on multi-client collaboration and client
  • Distributed file system and method based on multi-client collaboration and client

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0067] Such as figure 1 and figure 2 As shown, a distributed file system 100 based on multi-client collaboration includes: a server 102, a client 101, and an in-memory database 103; the server 102 includes: a metadata server cluster 1021 and a data server cluster 1022, the The metadata server cluster 1021 and the data server cluster 1022 separately provide independent volumes for the client 101 to mount and use; the client 101 includes: a user interface module 1011, a caching module 1012 and a communication module 1013. The files are written in blocks; the memory database 103 is used to: record the data block block information and distribution information of the files uploaded to the data server.

[0068] Among them, the metadata server cluster 1021 is mainly used to maintain metadata and is responsible for controlling functions such as garbage collection and load balancing. The metadata server uses SSD disks to speed up data access; the data server cluster 1022 is responsib...

Embodiment 2

[0073] In practical applications, the implementation of data upload based on the multi-client 101-based distributed file system is as follows:

[0074] When there is data to be written, at first through the user interface module 1011 of the client 101, the write request is sent to the metadata server cluster 1021, such as the owner of the file, time, and file size, creating the file client 101, and uploading the file Data server completion status and other information. It should be noted that the file size is only the information recorded in the extended attribute of the file, without specific file data, and does not occupy disk space. After receiving the request, the metadata server cluster 1021 records the metadata information of the file, and returns a confirmation message to the client 101. After the client 101 receives the confirmation message, it starts to write data. First, the data is written into the local cache module 1012 in blocks (such as 4M). After the data is w...

Embodiment 3

[0079] In practical applications, the implementation of data reading based on the multi-client 101-based distributed file system is as follows:

[0080] When the client 101 reads data, it first accesses the metadata server to obtain the identification information of the file. Afterwards, through the identification information of the file, the client 101 queries the memory database 103, obtains the current block information of the file, and obtains the information of the data server where the data block is located, and then the client 101 establishes a link with the data server where the data block of the file is located, and finally obtains All data blocks. Afterwards, the data is synthesized on the client 101 to complete the pulling of the entire file.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of file processing, in particular to a distributed file system and method based on multi-client collaboration and clients. The distributed file system based on multi-client collaboration comprises a server side, clients and a memory database, the server side comprises a metadata server cluster and a data server cluster, and the metadata server cluster and the data server cluster respectively and independently provide independent volumes for the client side to mount and use, so that complete separation of files and directory information of the files is realized, and metadata query access is accelerated. The file of the client is written in blocks, so that the de-duplication uploading function based on the data blocks is realized, and the bandwidth is saved. The memory database is used for recording data block partitioning information and distribution information of the file. Through cooperation of multiple clients, the problem that real-time data of the multiple clients cannot be completely accessed is solved.

Description

technical field [0001] The invention relates to the technical field of file processing, in particular to a distributed file system, method and client based on multi-client collaboration. Background technique [0002] In the era of big data, the amount of data is growing exponentially, which poses a great challenge to the storage system. At present, there are three common metadata organization and management methods: (1) centralized metadata management; (2) distributed metadata management; (3) no metadata management. In order to achieve high throughput of data access, many systems use dedicated metadata servers. When users access the system, they first obtain access permission and file metadata information from the metadata server, such as data location and file attributes, and then directly access the files. content. In addition, some systems adopt a metadata-free architecture in order to avoid the hidden dangers brought by a single metadata. Due to the rapid growth of st...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/182G06F16/172G06F3/06G06F16/28
CPCG06F16/182G06F16/172G06F3/06G06F3/0611G06F3/064G06F3/067G06F16/28
Inventor 程永红许锡顺庄世勇林宝栋
Owner JINQIANMAO TECH CO LTD
Features
  • Generate Ideas
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com