Method, device and system of data migration of distributed type storage system

A distributed storage and data technology, applied in the field of communication, can solve the problems of rarely used large-scale Internet systems, complex distributed storage systems, complex logic, etc., achieve real-time smooth linear expansion, reduce deployment complexity, and improve system performance. Effect

Active Publication Date: 2013-04-24
ZHEJIANG TMALL TECH CO LTD
View PDF3 Cites 33 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] In the above method (1), since it will affect the availability of the distributed storage system, that is, it will cause the distributed storage system to be unavailable, so it is rarely used in practical applications, especially in large-scale Internet systems
[0011] The above method (2) is the current mainstream distributed storage architecture, but it depends on the Namenode. Every read and write access needs to visit the Namenode, which increases the overhead and reduces the performance; when the Namenode fa

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device and system of data migration of distributed type storage system
  • Method, device and system of data migration of distributed type storage system
  • Method, device and system of data migration of distributed type storage system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0035] Example one

[0036] The first embodiment of the present application provides a data migration method of a distributed storage system, which is applied to a distributed storage system including a client, a data management server, a source storage server, and a target storage server; the client is used for users The storage access provides services to complete the data storage process. For example, if a user visiting a certain website needs to upload data to the website, the client of the distributed storage system of the website receives the data uploaded by the user and stores it in the storage On the server, at this time, the client needs to determine which storage server should store the data according to its own strategy, and store the data on the determined storage server; of course, it is understandable that the client is not just for receiving The data uploaded by users who visit the website can also be used to receive data uploaded by any user who uses the distribu...

Example Embodiment

[0066] Example two

[0067] Based on the same application concept as the above method, an embodiment of the application also provides a storage server, which is used as a source storage server in a distributed storage system including a client, a data management server, the source storage server, and a target storage server , After the data management server notifies the source storage server to migrate data to the target storage server according to a preset routing algorithm, such as Image 6 As shown, the storage server includes:

[0068] The receiving module 11 is used to receive data to be written from the client;

[0069] The routing module 12 is configured to determine that the data needs to be written to the target storage server according to the preset routing algorithm;

[0070] The proxy module 13 is configured to write the data to the target storage server when it is determined that the data needs to be written to the target storage server.

[0071] In this application, the...

Example Embodiment

[0078] Example three

[0079] Based on the same application concept as the above method, an embodiment of this application also provides a distributed storage system, including a client, a data management server, a source storage server, and a target storage server, where:

[0080] The data management server is configured to notify the source storage server to migrate data to the target storage server according to a preset routing algorithm;

[0081] The client is configured to send data to be written to the source storage server when data needs to be written before the source storage server data migration is completed; after the source storage server data migration is completed , When there is data to be written, determining according to the preset routing algorithm to send the data to be written to the source storage server, or to send the data to be written to the target storage server;

[0082] The source storage server is configured to start its own write proxy function after the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method, device and system of data migration of a distributed type storage system. The method includes the steps that a source storage server receives data which are from a client-side and need to be written into, and the data is written into a target storage server by the source storage server when the data are determined to need to be written into the target storage server by the source storage server according to preset routing algorithm. In the implemented cases, in the process of migrating the data to the target storage server by the source storage server, the data which need to be written into are still sent to the source storage server by the client-side, the source storage server acts on behalf of the client-side in a responsibility position and stores the data onto the target storage server provided that the memory address of the data is the target storage server, and thereby the distributed type storage system can achieve smooth linear scale in real time, improve performance of the system, improve usability and reduce deployment complexity.

Description

technical field [0001] This application relates to the field of communication technology, and in particular to a data migration method, device and system for a distributed storage system Background technique [0002] In a distributed storage system (that is, data is stored in multiple independent storage servers), when the data storage requirement reaches the actual physical storage capacity, the distributed storage system needs to be physically expanded to meet the demand. The usual method of physical expansion is Add a new storage server to the storage cluster. [0003] In the prior art, the ways to realize the expansion of the distributed storage system include: [0004] (1) The expansion of the distributed storage system is realized by shutting down the storage server for maintenance, which will affect the availability of the distributed storage system. [0005] (2) Realize the expansion of the distributed storage system through HDFS (Hadoop Distributed File System, di...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08
Inventor 李智慧何坤余俊周异
Owner ZHEJIANG TMALL TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap