Data storage method, device and system

A data storage system and data technology, applied in the IT field, can solve problems such as large data volume and storage system performance impact, and achieve the effects of avoiding write failures, reducing IO data volume, and reducing the number of generated data.

Active Publication Date: 2018-11-09
HUAWEI CLOUD COMPUTING TECH CO LTD
View PDF10 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the amount of data stored by storage nodes is generally relatively large, the migration and ret

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage method, device and system
  • Data storage method, device and system
  • Data storage method, device and system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0064] Such as figure 1 Shown is a schematic diagram of the architecture of a data storage system provided by an embodiment of the present invention. The data storage system includes a storage client 100 and a distributed storage system 200. The distributed storage system 200 includes a management server 201 and multiple Storage node 202. The user writes data to the storage nodes included in the partition of the distributed storage system 200 through the storage client 100 in the persistence layer. The storage client 100 processes read and write requests from user equipment, and reads and writes data to the distributed storage system 200. The storage client uses EC technology to generate multiple EC blocks for the data to be stored, and then sends them to a partition of the distributed storage system for storage. Data distributed storage can reduce the risk of data loss and improve the reliability of distributed storage systems.

[0065] The data storage system can use EC techn...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data storage method, device and system. A storage client needs to store data to be written into a distributed storage system. Storage nodes corresponding to a first data unitallocated by a management server for the data to be written are only a part of a storage node group. When the state of the first data unit returned by the management server is received by the storageclient, the numbers of data blocks and check blocks to be generated when EC encoding is performed on the data to be written can be determined, the storage client stores the generated data block and check block in the partial storage nodes designated by the management server in a partition in which the first data unit is located. Thus, dynamic adjustment of the EC redundancy ratio is realized, andthe management server can exclude nodes in the partition from the range of data storage to be written according to the needs, thereby reducing the IO amount of data storage.

Description

technical field [0001] The present invention relates to the field of IT technology, in particular to a data storage method, device and system. Background technique [0002] More and more distributed storage systems at the present stage use erasure code (Erasure Code, EC) technology to store data. The principle of erasure code technology is to divide the data into N data blocks, and use the redundancy algorithm to encode the N data blocks to generate M check blocks. The N data blocks and M check blocks are called an EC Bands. Among them, data blocks or check blocks are also called EC blocks. When the lost EC blocks are not more than M, the original data can be recovered. The redundancy ratio of EC stripes is r=M / N or r=M / (N +M). Exemplarily, when N is equal to 4 and M is equal to 2, the EC strip can be expressed as EC 4+2. [0003] In order to ensure the reliability of storage, the distributed storage system will allocate N+M EC blocks contained in each EC stripe to N+M s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06G06F11/10
CPCG06F3/0619G06F3/064G06F3/0644G06F3/0647G06F3/067G06F11/1004G06F11/1044G06F11/1076G06F3/0652G06F9/5061
Inventor 刘小威伍华涛殷丽慧
Owner HUAWEI CLOUD COMPUTING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap