Data consistency check flow control method, device, electronic equipment and storage medium

A consistency and data technology, applied in the computer field, can solve problems such as reducing user application experience, and achieve the effects of ensuring data consistency, good flow control effect, and improving efficiency

Active Publication Date: 2021-01-01
PING AN TECH (SHENZHEN) CO LTD
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If the data consistency check is triggered just when the input / output (IO) (write down the full name) of the user application is under high pressure, the IO that may be performing the consistency check will affect the user application. IO has an impact, thereby reducing the experience of user applications and even causing system failures

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data consistency check flow control method, device, electronic equipment and storage medium
  • Data consistency check flow control method, device, electronic equipment and storage medium
  • Data consistency check flow control method, device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0056] figure 1 It is a flow chart of the flow control method for data consistency verification provided by Embodiment 1 of the present invention. According to different requirements, the execution sequence in the flow chart can be changed, and some steps can be omitted.

[0057] S11. When a write request of user data is received, store the user data as multiple copies.

[0058] In order to provide data reliability, distributed storage systems are generally implemented by storing data in multiple copies. For example, if a user stores a txt document, in the underlying distributed storage system, this document will be stored in three copies and placed on different hard disks in different fault domains. In this way, even if a hard disk is damaged, the txt file will not be lost. Or when two hard disks are damaged at the same time, the data will still not be lost. However, after the hard disk is damaged, the distributed storage system will generally sense and complete the lost ...

Embodiment 2

[0082] figure 2 It is a flow chart of the method for determining the flow control threshold corresponding to the current statistical period according to the IO load of the user application in the previous statistical period provided by the second embodiment of the present invention.

[0083] S21. Obtain the data block size of each IO applied by the user in the last statistical period, and calculate the average data block size of the IOs in the last statistical period.

[0084] The average data block size of the IOs in the last statistical period may be calculated using an arithmetic mean algorithm, a geometric mean algorithm, or a root mean square mean algorithm.

[0085] For example, suppose it is detected that the user application had ten IOs in the last statistical period, and the data block sizes of the ten IOs are: 2M, 1M, 3M, 0.5M, 10M, 4M, 0.1M, 1.2M, 5M and 8M. Using the arithmetic mean algorithm to calculate the average data block size of the IO in the last statist...

Embodiment 3

[0119] image 3 It is a functional block diagram of a preferred embodiment of the flow control device for data consistency verification in the present invention.

[0120] In some embodiments, the data consistency verification flow control device 30 runs in an electronic device. The data consistency check flow control device 30 may include a plurality of functional modules composed of program code segments. The program codes of each program segment in the data consistency check flow control device 30 can be stored in a memory, and executed by at least one processor for execution (see Figure 1-2 and related descriptions) data consistency check flow control method.

[0121] In this embodiment, the data consistency verification flow control device 30 can be divided into multiple functional modules according to the functions it performs. The functional modules may include: a copy storage module 301 , a detection module 302 , a flow control acquisition module 303 , a copy verifi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A method and apparatus for data consistency checking and flow control, an electronic device and a storage medium. The method comprises: when a write request for user data is received, storing said user data as a plurality of copies (S11); detecting whether a trigger condition for data consistency checking is met (S12); when it is detected that the trigger condition for data consistency checking is met, obtaining a flow control threshold corresponding to a current statistical period in a checking period (S13); performing data consistency checking on the plurality of copies on the basis of the flow control threshold corresponding to the current statistical period (S14). The described method may prevent an obvious impact on normal input and output service performance, while simultaneously improving the data consistency checking efficiency of a large-scale distributed storage system, and has a very good flow control effect.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a data consistency check flow control method, device, electronic equipment and storage medium. Background technique [0002] The distributed storage system stores data dispersedly on multiple independent devices, adopts a scalable system structure and multiple redundancy strategies, uses multiple storage servers to share the storage load, and searches for storage information according to the corresponding positioning algorithm. The distributed storage system can not only improve the reliability, availability and access efficiency of the system, but also be easy to expand, and can eliminate single point of failure. When a fault occurs, the impact on the front-end user application is small. [0003] Nevertheless, while ensuring the corresponding performance indicators, it is also very important for the distributed storage system to ensure the consistency of replica data in each ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06G06F11/14
CPCG06F3/0653G06F3/0655G06F3/067G06F11/1464
Inventor 陈学伟
Owner PING AN TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap