Unlock instant, AI-driven research and patent intelligence for your innovation.

A method, device and system for data storage

A data storage system and data technology, applied in the IT field, can solve problems such as large data volume and storage system performance impact, and achieve the effects of avoiding write failures, reducing IO data volume, and reducing the number of generated data.

Active Publication Date: 2020-09-04
HUAWEI CLOUD COMPUTING TECH CO LTD
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the amount of data stored by storage nodes is generally relatively large, the migration and retrieval of data in the faulty node will have a greater impact on the performance of the storage system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method, device and system for data storage
  • A method, device and system for data storage
  • A method, device and system for data storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0064] like figure 1 As shown, it is a schematic diagram of the architecture of a data storage system provided by an embodiment of the present invention. The data storage system includes a storage client 100 and a distributed storage system 200. The distributed storage system 200 includes a management server 201 and multiple Storage node 202. The user writes data to the storage nodes contained in the partitions of the distributed storage system 200 through the storage client 100 in the persistence layer. The storage client 100 processes read and write requests from user equipment, and reads and writes data to the distributed storage system 200 . The storage client uses EC technology to generate multiple EC blocks (blocks) for the data to be stored, and then sends them to a partition of the distributed storage system for storage. Data distributed storage can reduce the risk of data loss and improve the reliability of distributed storage system.

[0065] The data storage syst...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A method, device and system for data storage, the storage client needs to store the data to be written in a distributed storage system, and the storage node corresponding to the first data unit allocated by the management server for the data to be written is only the storage node For some nodes in the group, when the storage client receives the status of the first data unit returned by the management server, it can determine the number of data blocks and check blocks that need to be generated when EC coding is performed on the data to be written, and the storage client The generated data blocks and check blocks are stored in some storage nodes designated by the management server in the partition where the first data unit is located, thereby realizing the dynamic adjustment of the EC redundancy ratio. The management server can store some nodes in the partition as needed. The scope of the data storage to be written is excluded, and the IO amount of the data storage is reduced.

Description

technical field [0001] The present invention relates to the field of IT technology, in particular to a data storage method, device and system. Background technique [0002] More and more distributed storage systems at the present stage use erasure code (Erasure Code, EC) technology to store data. The principle of erasure code technology is to divide the data into N data blocks, and use the redundancy algorithm to encode the N data blocks to generate M check blocks. The N data blocks and M check blocks are called an EC Bands. Among them, data blocks or check blocks are also called EC blocks. When the lost EC blocks are not more than M, the original data can be recovered. The redundancy ratio of EC stripes is r=M / N or r=M / (N +M). Exemplarily, when N is equal to 4 and M is equal to 2, the EC strip can be expressed as EC 4+2. [0003] In order to ensure the reliability of storage, the distributed storage system will allocate N+M EC blocks contained in each EC stripe to N+M s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06G06F11/10
CPCG06F3/0619G06F3/064G06F3/0644G06F3/0647G06F3/067G06F11/1004G06F11/1044G06F11/1076G06F3/0652G06F9/5061
Inventor 刘小威伍华涛殷丽慧
Owner HUAWEI CLOUD COMPUTING TECH CO LTD