Partition expansion method and device and data storage method and device

An expansion method and distributed storage technology, applied in the field of partition expansion method, data storage method and device, can solve problems such as difficulty in ensuring migration, and achieve the effect of improving efficiency

Active Publication Date: 2019-07-09
HUAWEI CLOUD COMPUTING TECH CO LTD
View PDF7 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In the process of realizing the above-mentioned system expansion, it is difficult to ensure that the partition data of the original storage node is accurately migrated to the newly added partition on the new storage node

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Partition expansion method and device and data storage method and device
  • Partition expansion method and device and data storage method and device
  • Partition expansion method and device and data storage method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] The application will be described in further detail below in conjunction with the accompanying drawings.

[0046] In the field of distributed storage, various failures often occur, so certain redundancy strategies must be adopted, such as multiple copies and EC (Erasure Codes, erasure codes). Multi-copy is to store multiple copies of the original data. As long as there is more than one copy of data remaining in the event of a failure, the data will not be lost. EC divides the original data into k small original data blocks, and then calculates m small verification data blocks based on the k data, and finally stores k+m small data blocks. When a fault occurs, the k+m small data blocks As long as there are more than k remaining, the original data can be recovered. Regardless of multi-copy or EC, data is usually stored in units of data blocks (chunk / plog). Data blocks are stored in storage nodes, but the location is not fixed. With expansion or kick-off, data blocks will ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a partition expansion method and device and a data storage method and device. The method comprises the following steps: the target storage node receives a first partition viewsent by the controller server, the first partition view comprises two mapping relationships; wherein the first mapping relationship is a mapping relationship between each storage node identifier and each partition identifier in the distributed storage system; the second mapping relationship is a mapping relationship between each newly added partition identifier and the partition identifier to be migrated; therefore, the target storage node determines a first to-be-migrated partition corresponding to the first newly added partition on the target storage node according to the second mapping relation, determines a source storage node corresponding to the first partition to be migrated according to a first mapping relation; then obtains the data of the first partition to be migrated from the source storage node, and writies the obtained data into the first newly-added partition on the target storage node, so that the data of the partition of the original storage node can be accurately migrated to the newly-added partition on the new storage node.

Description

technical field [0001] The present application relates to the field of information technology, and in particular to a partition extension method, data storage method and device. Background technique [0002] In a distributed storage system, a distributed hash table (Distributed Hash Table, DHT) is generally used to determine the data storage location. That is, the Hash value is calculated by a hash Hash (hash) function, and the data is mapped to a Hash space according to the Hash value. The Hash space is a ring space connected end to end. The Hash ring can be divided into several equal parts, and each part is called a partition, and then evenly distribute the partitions to each storage node, and at the same time, establish a mapping relationship from data to partitions and from partitions to storage nodes and store them in the DHT. Among them, the reading and writing of data, the expansion of distributed clusters, and fault handling all depend on the changes of DHT. Theref...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08G06F3/06
CPCH04L67/1097G06F3/0644H04L67/563H04L67/55
Inventor 于传帅刘强夏勇
Owner HUAWEI CLOUD COMPUTING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap