Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Ceph-based rbd file transmission method and system thereof

A file and fast transmission technology, applied in the field of file transmission of distributed storage, can solve the problems of complicated operation and increase in cost, and achieve the effect of reducing intermediate steps, reducing transmission cost and improving transmission efficiency.

Pending Publication Date: 2020-06-02
GUANGDONG EFLYCLOUD COMPUTING CO LTD
View PDF2 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] 1) The method of exporting the rbd file to a local file, then transferring the file, and finally importing the file to a different place is cumbersome, and requires sufficient space to store the rbd file locally and in a different place;
[0005] 2) The rbd file transfer is a file stream, that is, in the actual scene, the file capacity and the transfer flow are 1:1, which greatly increases the transfer time and cost

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Ceph-based rbd file transmission method and system thereof
  • Ceph-based rbd file transmission method and system thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] The purpose of the present invention is to realize the conversion of rbd files into compressed data streams, realize fast data transmission; and realize a rbd file backup mode across ceph clusters.

[0034] Therefore, this embodiment proposes a ceph-based rbd file fast transfer method, such as figure 1 As shown, the steps include:

[0035] S10, the local rbd client and remote rbd client are all connected to the ceph distributed storage platform.

[0036] S11, the local rbd client accesses and opens the rbd file of the ceph distributed storage platform;

[0037] S12, the remote rbd client creates an rbd file of the ceph distributed storage platform for storing the received data stream;

[0038] S13, the local rbd client and the remote rbd client establish a tcp long connection;

[0039] S14, the local rbd client reads the rbd file of the ceph distributed storage platform, compresses it and transmits it to the remote rbd client;

[0040] In this embodiment, lz4 compre...

Embodiment 2

[0047] This embodiment discloses a ceph-based rbd file rapid transfer system based on the method of embodiment 1, such as figure 2 Shown, comprise local rbd client, off-site rbd client and ceph distributed storage platform, described local rbd client, off-site rbd client are all connected with described ceph distributed storage platform;

[0048] The local rbd client includes an access module, a first connection module, a compressed transmission module and an end sign generation module;

[0049] Described off-site rbd client comprises new rbd file generating module, the second connection module, receiving decompression module and end sign receiving module;

[0050] The first connection module is used to connect the ceph distributed storage platform, and is used to connect the second connection module and establish a tcp long connection; the second connection module is also used to connect the ceph distributed storage platform ;

[0051] The access module is used to access and...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a ceph-based rbd file transmission method and a system thereof. The rbd file transmission method comprises the following steps of: accessing an rbd file of a ceph distributed storage platform by a local rbd client; newly establishing an rbd file of the ceph distributed storage platform by the remote rbd client; a tcp long connection is established between the local rbd client and the remote rbd client; compressing the rbd file of the local rbd client and transmitting the rbd file to a remote rbd client; the remote rbd client decompresses the received rbd file data flowand writes the rbd file data flow into the rbd file newly built in the step S2; and reading the rbd files of all the ceph distributed storage platforms and sending an end mark to the remote rbd clientby the local rbd client, and stopping the rbd files by the remote rbd client after receiving the end mark. According to the method, the rbd file transmission operation is simplified, the storage space is saved, the transmission time is shortened, and the rbd file transmission cost is reduced.

Description

technical field [0001] The invention relates to the technical field of file transmission in distributed storage, in particular to a ceph-based rbd file fast transmission method and system thereof. Background technique [0002] Ceph is currently the most popular virtualized storage architecture. In the industry, the best scenarios for ceph distributed storage clusters are block storage and file storage. The block storage file (rbd file) of ceph only supports RAW format (non-sparse file format). Therefore, more network resources and storage resources need to be consumed when storing or transmitting RAW format files. [0003] In the existing technology, there are problems in cross-cluster file transfer: [0004] 1) The method of exporting the rbd file to a local file, then transferring the file, and finally importing the file to a different place is cumbersome, and requires sufficient space to store the rbd file locally and in a different place; [0005] 2) The rbd file tra...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/182G06F16/16G06F16/17G06F16/172G06F11/14H04L29/06H04L29/08
CPCG06F16/182G06F16/16G06F16/1737G06F16/172G06F11/1464H04L67/1097H04L67/141H04L69/04
Inventor 曾星史伟闵宇
Owner GUANGDONG EFLYCLOUD COMPUTING CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com