Adaptive data recovery flow control method, device, electronic equipment and storage medium

An adaptive, data-based technology, applied in the computer field, can solve problems such as high probability of failure, achieve good flow control effect, ensure IO load, and reduce the effect of failure recovery intensity

Active Publication Date: 2021-01-29
PING AN TECH (SHENZHEN) CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the larger the size of the storage system, the higher the probability of failure

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Adaptive data recovery flow control method, device, electronic equipment and storage medium
  • Adaptive data recovery flow control method, device, electronic equipment and storage medium
  • Adaptive data recovery flow control method, device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0054] figure 1 It is a flow chart of the adaptive data recovery flow control method provided by Embodiment 1 of the present invention. According to different requirements, the execution sequence in the flow chart can be changed, and some steps can be omitted.

[0055] S11. Periodically synchronize the information of each storage node in the distributed storage system.

[0056] In a preferred embodiment of the present invention, the distributed storage system (hereinafter referred to as the storage system for short) implements distributed data storage in a cluster storage manner.

[0057] The distributed storage is a data storage technology, which uses the remaining disk space on each storage system in the cluster through the network, and integrates the scattered storage resources of the remaining disk space to form a virtual Storage devices store data in various corners of the cluster.

[0058] Therefore, each storage node described in the present invention is each sub-sto...

Embodiment 2

[0120] figure 2 It is a functional block diagram of a preferred embodiment of the self-adaptive data restoration flow control device of the present invention.

[0121] In some embodiments, the adaptive data recovery flow control device 20 (hereinafter simply referred to as "data recovery flow control device 20") runs in an electronic device. The data restoration flow control device 20 may include a plurality of functional modules composed of program code segments. The program codes of each program segment in the data recovery flow control device 20 can be stored in a memory, and executed by at least one processor to execute (see for details figure 1 and related descriptions) an adaptive data recovery flow control method.

[0122] In this embodiment, the data recovery flow control device 20 of the electronic device can be divided into multiple functional modules according to the functions it performs. The functional modules may include: a synchronization module 201 , a dete...

Embodiment 3

[0187] image 3 It is a schematic diagram of the electronic device provided by Embodiment 5 of the present invention.

[0188] The electronic device 3 includes: a memory 31 , at least one processor 32 , a computer program 33 stored in the memory 31 and operable on the at least one processor 32 , and at least one communication bus 34 .

[0189] When the at least one processor 32 executes the computer program 33, the steps in the above embodiments of the adaptive data recovery flow control method are implemented.

[0190] Exemplarily, the computer program 33 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 31 and executed by the at least one processor 32, To complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 33 in t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

An adaptive data recovery flow control method, including: regularly synchronizing information of each storage node in a distributed storage system; when detecting that a storage node has failed, obtaining a storage list of the failed storage node; identifying The IO load category of the user application in the last statistical period; calculate the flow control threshold corresponding to the current statistical period according to the IO load category in the previous statistical period; according to the storage list and the flow control threshold corresponding to the current statistical period , perform a recovery operation on the data in the current statistical period of the failed storage node; repeat the above steps until all the data is recovered. The invention also provides an adaptive data restoration flow control device, electronic equipment and storage medium. The present invention can improve the data repair efficiency of a large-scale distributed storage system and reduce the risk of data loss, while avoiding obvious impact on normal input and output service performance, and has a good flow control effect.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to an adaptive data recovery flow control method, device, electronic equipment and storage medium. Background technique [0002] With the advent of the era of big data and cloud computing, the amount of data in various fields is showing a trend of rapid growth. These ever-growing massive data need to rely on large-scale distributed storage systems to achieve reliable storage and efficient access. However, the larger the storage system, the higher the probability of failure. In order to cope with possible failures at any time and ensure the reliability of data storage, distributed storage systems need data redundancy. A common data redundancy strategy is to store multiple copies of data on different physical nodes. When some copies are damaged, the damaged copies can be repaired based on the intact copies. [0003] In addition, when expanding the capacity of a distributed storag...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06
CPCG06F3/0613G06F3/064G06F3/067G06F3/0673
Inventor 陈学伟
Owner PING AN TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap