Distributed file system and method based on multi-client collaboration and client

A distributed file, multi-client technology, applied in the field of distributed file systems, can solve the problems of slow metadata access, slow real-time file writing, and inability to fully access real-time data from multiple clients, saving bandwidth and speeding up query access. Effect

Active Publication Date: 2021-07-30
JINQIANMAO TECH CO LTD
View PDF4 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] To this end, it is necessary to provide a distributed file system based on multi-client collaboration to solve the problems of slo

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system and method based on multi-client collaboration and client
  • Distributed file system and method based on multi-client collaboration and client
  • Distributed file system and method based on multi-client collaboration and client

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0066] Example 1:

[0067] like figure 1 and figure 2 As shown, a distributed file system 100 based on multi-client synergies, includes: server end 102, client 101, and memory database 103; server end 102 includes: metadata server cluster 1021 and data server cluster 1022, The metadata server cluster 1021 and the data server cluster 1022 separately provide a separate volume to the client 101 mounted; the client 101 includes a user interface module 1011, a cache module 1012, and a communication module 1013, the client 101. The file is written for blocking; the memory database 103 is used to record data block subunit information and distribution information of the file of the upload data server.

[0068] The metadata server cluster 1021 is mainly used to maintain metadata and control garbage collection, load balancing and other functions. The metadata server uses SSD disks, speeds up data access speed; data server cluster 1022 is responsible for saving data and receiving processing ...

Example Embodiment

[0072] Example 2

[0073] In practical applications, the implementation of data uploading based on the distributed file system based on the multi-client 101 coordinated distributed file system is as follows:

[0074] When there is data writing, the write request is first sent to the write request to the metadata server cluster 1021, such as the primary, time, and file size of the file, and the file client 101, and file upload Data server completion states and other information. It should be noted that the file size is only recorded in the file extension attribute, no specific file data, does not take up disk space. After receiving the request, the metadata server cluster 1021 is recorded, and the metadata information of the file is recorded and returned to the client 101 confirms the message. After the client 101 receives a confirmation message, start data writing, first written to the local cache module 1012 in a block (e.g., 4m), to be written to a certain size, performing data ...

Example Embodiment

[0078] Example 3

[0079] In practical applications, the implementation of data reading based on the distributed file system based on the multi-client 101 synergistic is as follows:

[0080] When the client 101 reads the data, first access the metadata server to obtain the identification information of the file. Thereafter, the memory database 103 is then queried by the file, the client 101 is obtained, and the current block information is obtained, and the data server information is located, then the client 101 establishes a link with the data server where the file is located, and finally acquire All data blocks. The data is then synthesized on the client 101 to complete the tensile of the entire file.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of file processing, in particular to a distributed file system and method based on multi-client collaboration and clients. The distributed file system based on multi-client collaboration comprises a server side, clients and a memory database, the server side comprises a metadata server cluster and a data server cluster, and the metadata server cluster and the data server cluster respectively and independently provide independent volumes for the client side to mount and use, so that complete separation of files and directory information of the files is realized, and metadata query access is accelerated. The file of the client is written in blocks, so that the de-duplication uploading function based on the data blocks is realized, and the bandwidth is saved. The memory database is used for recording data block partitioning information and distribution information of the file. Through cooperation of multiple clients, the problem that real-time data of the multiple clients cannot be completely accessed is solved.

Description

technical field [0001] The invention relates to the technical field of file processing, in particular to a distributed file system, method and client based on multi-client collaboration. Background technique [0002] In the era of big data, the amount of data is growing exponentially, which poses a great challenge to the storage system. At present, there are three common metadata organization and management methods: (1) centralized metadata management; (2) distributed metadata management; (3) no metadata management. In order to achieve high throughput of data access, many systems use dedicated metadata servers. When users access the system, they first obtain access permission and file metadata information from the metadata server, such as data location and file attributes, and then directly access the files. content. In addition, some systems adopt a metadata-free architecture in order to avoid the hidden dangers brought by a single metadata. Due to the rapid growth of st...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/182G06F16/172G06F3/06G06F16/28
CPCG06F16/182G06F16/172G06F3/06G06F3/0611G06F3/064G06F3/067G06F16/28
Inventor 程永红许锡顺庄世勇林宝栋
Owner JINQIANMAO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap