Snapshot processing method and system, equipment and storage medium

A processing method and snapshot technology, applied in the storage field, can solve problems such as business bandwidth congestion, difficulty, and implementation complexity, and achieve the effect of reducing bandwidth occupation, time-consuming, and crowding.

Active Publication Date: 2021-08-13
PING AN TECH (SHENZHEN) CO LTD
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Traditional incremental snapshot algorithms, such as copy-on-write or redirect-on-write, are complex to implement, and it is difficult to achieve high-performance and low-occupancy indicators at the same time. Taking a snapshot requires additional disk operations, which occupy a high bandwidth of the underlying file system, and thus occupy a certain amount of business bandwidth, which is very unfavorable in some performance-sensitive business scenarios

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Snapshot processing method and system, equipment and storage medium
  • Snapshot processing method and system, equipment and storage medium
  • Snapshot processing method and system, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0036] As we all know, the function of the snapshot is mainly to perform online data backup and recovery. When the storage device has an application failure or the file is damaged, it can quickly recover the data and restore the data to the state at a certain available point in time. Another function of the snapshot is Provide another data access channel for storage users. When the original data is processed by online applications, users can access snapshot data and use snapshots for testing and other work. All storage systems, regardless of high, medium and low end, as long as they are applied to online systems, then Snapshots become an indispensable feature.

[0037] The principle of the present invention is to use the data engine to keep the content unchanged after replaying the log. The storage system using the consistency algorithm does not need to strictly solidify the data content at each time point because of the log replay. For example, in a replication group A snapsh...

Embodiment 1

[0039] refer to figure 1 , the snapshot processing method of the present invention includes:

[0040] Use the names of all files in the data engine to construct a file name list, store the file name list in the metadata file under the snapshot directory, and record the log number at the current moment;

[0041] Based on the principle that the content of the data engine remains unchanged before and after the log is replayed by the data engine, the file deletion operation of the distributed storage system using the consensus algorithm is performed. Specifically, when the deletion operation instruction sent by the requester is detected, the file will be deleted in the snapshot directory. Create a blank file with the same name as the file to be deleted in the delete operation command, and then delete the file to be deleted in the data engine.

[0042] For example: when the name of the file to be deleted is file, a file file is created in the snapshot directory, and the file file is...

Embodiment 2

[0045] refer to figure 1 , the snapshot processing method of the present invention includes:

[0046] Use the names of all files in the data engine to construct a file name list, store the file name list in the metadata file under the snapshot directory, and record the log number at the current moment;

[0047] Based on the principle that the content of the data engine remains unchanged before and after the log is replayed by the data engine, the file copy operation of the distributed storage system using the consistency algorithm is performed. The specific process is as follows:

[0048] When the data copy operation instruction sent by the requester is detected, in the snapshot directory, according to the data copy operation instruction, the metadata file to be copied is sent to the requester;

[0049] When the pull file request sent by the requester is detected, the file is searched from the data engine according to the file name in the pull file request, and when the file ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a snapshot processing method and system, equipment and a storage medium. The method comprises the steps: building a file name list through the names of all files in a data engine, storing the file name list in a metadata file under a snapshot directory, and recording the log number at the current moment; and when an operation instruction sent by a requester is detected, performing a file deletion operation, a file copying operation or a file recovery operation of the distributed storage system using a consistency algorithm based on a principle that the content of the data engine is kept unchanged before and after the data engine replays the log. According to the method, the system, the equipment and the storage medium, additional operation on the disk is comprehensively removed, the bandwidth occupation of a bottom-layer file system is relatively low, and the occupation of service bandwidth is avoided.

Description

technical field [0001] The invention belongs to the technical field of storage, and relates to a snapshot processing method, system, equipment and storage medium. Background technique [0002] A distributed storage system that uses a consensus algorithm generally divides replication groups within the cluster, and then each replication group uses a consensus algorithm to keep the data carried by members of the same group exactly the same. The specific method is to combine all client IO Encapsulate it into a log, and then distribute the log inside the replication group. Members of the replication group will append the received log to their own log file, and at the same time restore the log into a specific operation that acts on their own data engine (that is, persistence stored data). In this way, if one or more (less than the minimum majority of the replication group) members go offline, after they are restarted, only the missing logs can be delivered to these members, and t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/11G06F16/13G06F16/16
CPCG06F16/128G06F16/162G06F16/166G06F16/13
Inventor 赵鑫
Owner PING AN TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap