Fast data verification method, device and system for cloud storage system

A cloud storage system and data verification technology, applied in the field of cloud storage, can solve the problems of high error risk and slow verification speed

Active Publication Date: 2022-08-02
深圳市木浪云科技有限公司
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Embodiments of the present invention provide a fast data verification method, device and system for a cloud storage system, to at least solve the problem that data needs to go through a longer path than a traditional file system when reading a file from a distributed file system client. The technical problems caused by slow verification speed and high risk of error

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fast data verification method, device and system for cloud storage system
  • Fast data verification method, device and system for cloud storage system
  • Fast data verification method, device and system for cloud storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0039] According to an embodiment of the present invention, a fast data verification method for a cloud storage system is provided, such as image 3 As shown, the method includes:

[0040] Step S302, receiving a sampling verification instruction;

[0041] Step S304, extract the data files required for verifying the target file from the sampling verification instruction, obtain the data sets that need to be sampled based on the extracted data files, form metadata of the data sets that need to be verified, and analyze all the data sets. Describe the metadata of the dataset to be verified;

[0042] The extracted data file includes: the target file, a check code file corresponding to the target file, and a sampling check ratio, wherein the check code file includes a check algorithm, a check code length, a check code file, and a check code file. Check code array length, check code array and check code.

[0043] In an exemplary embodiment, based on the extracted data file, a samp...

Embodiment 2

[0054] According to the embodiment of the present invention, another fast data verification method for a cloud storage system is provided, such as Figure 4 As shown, the method includes:

[0055] Step S402, verifying the initiation of the task;

[0056] The step of initiating the verification task is that the user issues a sampling verification instruction through the interface of the backup module, so as to sample the disk file of a certain virtual machine to check whether there is any data error.

[0057] In an exemplary embodiment, combining Figure 5 , the initiation of the verification task may include the following steps:

[0058] 1) The user application initiates a sampling verification command to the backup module;

[0059] 2) After receiving the sampling verification instruction, the backup module generates a fixed-format sampling verification task plan file according to the verification requirements of the backup data, that is, the data file required for verifyin...

Embodiment 3

[0087] According to an embodiment of the present invention, another method for fast data verification of a cloud storage system is provided, such as Figure 5 As shown, the method includes:

[0088] Step S502, initiating a verification task;

[0089] The step of initiating the verification task is that the user issues a sampling verification instruction through the interface of the backup module, so as to sample the disk file of a certain virtual machine to check whether there is any data error.

[0090] Step S504, preprocessing the verification task;

[0091] The file system service of the storage system reads the verification task plan file and obtains the necessary information for data verification, including the target file, the verification code file corresponding to the target file, and the sampling verification ratio. The file system of the storage system calls the data sampling module of the metadata service cluster through the sampling verification interface module,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a fast data verification method, device and system of a cloud storage system. Wherein, the method includes: the method is executed in a cloud storage system, and the method includes: receiving a sampling verification instruction; extracting a data file required for verifying the target file from the sampling verification instruction, and based on the extracted obtain the data set that needs to be sampled, form the metadata of the data set that needs to be verified, and analyze the metadata of the data set that needs to be verified; according to the analysis result of the metadata and the target file The number of service nodes of the data block is divided into multiple check sub-tasks; the multiple check sub-tasks are executed in parallel to check the data. The invention solves the technical problems of slower verification speed and higher error risk caused by the fact that the data needs to go through a longer path than the traditional file system when reading files from the distributed file system client.

Description

technical field [0001] The present invention relates to the field of cloud storage, and in particular, to a method, device and system for fast data verification of a cloud storage system. Background technique [0002] When a computer system transmits and stores data, errors will inevitably occur. In order to avoid this kind of error, on the one hand, we should start from the hardware side to improve the anti-interference ability and reliability of the hardware; This kind of data encoding with error detection or automatic error correction capability is usually called data check code. The principle is to add some check bits to the data to form a data check code, and to judge whether there is an error or correct it by checking the validity of the data check code. Commonly used data check codes include parity check code, Hamming check code, cyclic redundancy check code (CRC) and so on. [0003] The verification of the file is mainly based on the calculation method of the veri...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/10G06F16/182G06F3/06
CPCG06F11/1004G06F16/182G06F3/0608G06F3/067G06F3/0641
Inventor 陈元强蔡涛吴健辉
Owner 深圳市木浪云科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap