Data verification method and device and storage medium

A data verification and data block technology, which is applied in the computer field, can solve problems affecting the read and write performance of the user's front end, and achieve the effect of improving the overall verification speed, solving the read and write performance, and reducing resource occupation

Active Publication Date: 2020-01-07
ALIBABA GRP HLDG LTD
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The embodiment of the present invention provides a data verification method and device, and a storage medium to at least solve the technical problem in the related art that affects the read and write performance of the user front end during the data verification process

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data verification method and device and storage medium
  • Data verification method and device and storage medium
  • Data verification method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0031] According to an embodiment of the present invention, an embodiment of a data verification method is also provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and , although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0032] During the data verification process, the storage node reads the data block and calculates the verification code according to the verification request, and there will be multiple storage nodes in the distributed storage system, and the data verification will occupy a certain amount of storage nodes being verified. disk and network bandwidth, and this application load balances the storage nodes to prevent multiple verification tasks running concurrently from being concentrated on a few storage nodes, res...

Embodiment 2

[0088] According to an embodiment of the present invention, a device for implementing the above data verification method is also provided, such as Figure 4 As shown, among them, Figure 4 It is a structural block diagram of a data verification device provided according to an embodiment of the present invention, and the device includes:

[0089] The determining module 42 is configured to determine a data block to be verified among the plurality of data blocks corresponding to a predetermined file in the distributed storage system, wherein the storage node where the data block to be verified is located includes: the distributed storage system satisfies Storage nodes for load balancing strategies;

[0090] It should be noted that the above load balancing strategy can be used to balance the verification tasks of the storage nodes storing data blocks in the distributed storage system, for example: there are many verification tasks on a certain storage node (the number of verifica...

Embodiment 3

[0111] Embodiments of the present invention may provide a computer terminal, and the computer terminal may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the foregoing computer terminal may also be replaced with a terminal device such as a mobile terminal.

[0112] Optionally, in this embodiment, the foregoing computer terminal may be located in at least one network device among multiple network devices of the computer network.

[0113] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data verification method of the application program: determine the data block to be verified among the multiple data blocks included in the predetermined file in the distributed storage system, wherein the data block to be verified is The storage node where the verification data block is located is a storage node that satisfies the load balancing strategy in the distributed stora...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data verification method and device and a storage medium. The method comprises the steps that to-be-verified data blocks in a plurality of data blocks corresponding to predetermined files in the distributed storage system are determined, and storage nodes where the to-be-verified data blocks are located comprise storage nodes meeting a load balancing strategy in the distributed storage system; and the to-be-verified data block is verified. According to the invention, the technical problem that the read-write performance of the user front end is influenced in the dataverification process in the prior art is solved.

Description

technical field [0001] The invention relates to the field of computers, in particular to a data verification method and device, and a storage medium. Background technique [0002] In a large-scale distributed storage system, due to long input-output (input-output, IO) links and a large number of nodes, the probability of data corruption on a single node increases greatly. Therefore, it is very necessary to check the consistency of the data blocks stored in the system and check whether multiple copies of a data block are consistent. However, data verification itself will occupy additional network and storage node bandwidth resources in the distributed storage system, and users of distributed storage systems often have high demands for low latency and high throughput. Therefore, a data verification method is required. The verification scheme, while quickly completing the data consistency verification, can also minimize the impact on the user's front-end read and write. [00...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/10
CPCG06F11/1004H04L1/0061G06F3/061G06F3/0659G06F3/0671G06F3/067G06F3/0619G06F3/065G06F3/0604G06F3/064G06F21/64
Inventor 宋平凡谷跃胜
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap