Snapshot processing method, device, node, system, medium and product

Abstract

The embodiment of the invention discloses a snapshot processing method and device, nodes, a system, a medium and a product. The method comprises the following steps: sending a snapshot preparation message to each following node; under the condition that each following node is ready, a snapshot creation message is sent to each following node, so that each following node creates a corresponding snapshot at the same moment, and each following node compresses metadata of the corresponding snapshot and then sends the metadata to the leader node; and decompressing the metadata sent by each following node, and generating global snapshot information according to the decompressed metadata. According to the technical scheme of the embodiment, the leader node is used for sending the snapshot preparation message and the snapshot creation message to the following node, so that each node is ensured to create the snapshot at the same time, and the reliability of the generated global snapshot information is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a snapshot processing method, device, node, system, medium, and product. Background Art

[0002] With the widespread use of container technology, the deployment and management of containerized applications has become increasingly complex. Especially in large-scale distributed systems, how to quickly fix vulnerabilities and roll back systems while ensuring high system availability has become an important challenge. Traditional snapshot and rollback mechanisms often cause downtime in large-scale environments, affecting business continuity. In addition, in a multi-node distributed environment, ensuring the consistency of all nodes is also a difficult problem. Summary of the invention

[0003] The embodiments of the present disclosure provide a snapshot processing method, device, node, system, medium and product, which ensure that each node creates a snapshot at the same time and improve the reliability of the generated global snapshot information.

[0004] In a first aspect, a snapshot processing method is provided, which is applied to a leader node, where the leader node is elected based on a distributed consistency algorithm; the method comprises:

[0005] Send a snapshot preparation message to each follower node;

[0006] When each of the follower nodes is ready, sending a snapshot creation message to each of the follower nodes, so that each of the follower nodes creates a corresponding snapshot at the same time, and each of the follower nodes compresses metadata of the corresponding snapshot and sends it to the leader node;

[0007] The metadata sent by each of the follower nodes is decompressed, and global snapshot information is generated according to the decompressed metadata.

[0008] In a second aspect, a snapshot processing method is provided, which is applied to a follower node, and the method includes:

[0009] When receiving the snapshot preparation message, perform the preparation operation before creating the snapshot, and send a ready message to the leader node after being ready;

[0010] When a snapshot creation message is received, a snapshot is created at a specified time, and metadata of the snapshot is compressed and sent to the leader node.

[0011] In a third aspect, a snapshot processing device is provided, comprising:

[0012] A snapshot preparation message sending module is used to send a snapshot preparation message to each follower node;

[0013] A snapshot creation message sending module is used to send a snapshot creation message to each follower node when each follower node is ready, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node;

[0014] The global snapshot information generation module is used to decompress the metadata sent by each follower node and generate global snapshot information according to the decompressed metadata.

[0015] In a fourth aspect, a snapshot processing device is provided, including:

[0016] A preparation operation execution module, used for executing the preparation operation before creating the snapshot when receiving the snapshot preparation message, and sending a preparation message to the leader node after being ready;

[0017] The snapshot creation module is used to create a snapshot at a specified time when a snapshot creation message is received, and compress the metadata of the snapshot and send it to the leader node.

[0018] In a fifth aspect, a leadership node is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the snapshot processing method as described in the first aspect above when executing the computer program.

[0019] In a sixth aspect, a follower node is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the snapshot processing method as described in the second aspect above when executing the computer program.

[0020] In a seventh aspect, a snapshot creation system is provided, comprising the leader node as described in the fifth aspect, and at least two follower nodes as described in the sixth aspect;

[0021] The leader node is elected based on a distributed consistency algorithm.

[0022] In an eighth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the snapshot processing method described in the first aspect and the snapshot processing method described in the second aspect are implemented.

[0023] In a ninth aspect, a computer program product is provided, the computer program product comprising a computer program, and the computer program, when executed by a processor, implements the snapshot processing method described in the first aspect and the snapshot processing method described in the second aspect.

[0024] The disclosed embodiment discloses a snapshot processing method, device, node, system, medium and product, including: sending a snapshot preparation message to each follower node; when each follower node is ready, sending a snapshot creation message to each follower node, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node; decompresses the metadata sent by each follower node, and generates global snapshot information according to the decompressed metadata. The technical solution of this embodiment uses the leader node to send snapshot preparation messages and snapshot creation messages to the follower nodes, ensuring that each node creates a snapshot at the same time, and improving the reliability of the generated global snapshot information.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the embodiments of the present disclosure. Other features of the embodiments of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 is a flowchart of a snapshot processing method provided by an embodiment of the present disclosure;

[0028] Figure 2 is a schematic diagram of a rollback operation process provided by an embodiment of the present disclosure;

[0029] Figure 3 is a timing diagram of a rollback operation provided by an embodiment of the present disclosure;

[0030] Figure 4 is a process schematic diagram of a snapshot processing method provided by an embodiment of the present disclosure;

[0031] Figure 5 is a timing diagram of a snapshot processing method provided by an embodiment of the present disclosure;

[0032] Figure 6 is a flowchart of another snapshot processing method provided by an embodiment of the present disclosure;

[0033] Figure 7 is a schematic diagram of a rollback execution process provided by an embodiment of the present disclosure;

[0034] Figure 8is a timing diagram of a rollback execution process provided by an embodiment of the present disclosure;

[0035] Fig. 9 is a schematic diagram of a process of creating a snapshot provided by an embodiment of the present disclosure;

[0036] Fig.10 is a timing diagram of a process of creating a snapshot provided by an embodiment of the present disclosure;

[0037] Fig.11 is a structural schematic diagram of a snapshot processing device provided by an embodiment of the present disclosure;

[0038] Fig.12 is a structural schematic diagram of another snapshot processing device provided by an embodiment of the present disclosure;

[0039] Fig.13 is a schematic diagram of the structure of a leader node provided by an embodiment of the present disclosure;

[0040] Fig.14 is a structural diagram of a follower node provided by an embodiment of the present disclosure;

[0041] Fig.15 It is a structural diagram of a snapshot creation system provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0043] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0044] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0045] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the relevant content of the solution.

[0046] Figure 1The present invention provides a flowchart of a snapshot processing method, which can be applied to the situation of processing snapshots. The method can be applied to the leader node, which is elected based on the distributed consistency algorithm. The method can be executed by a snapshot processing device, which can be implemented in the form of hardware and / or software. The snapshot processing device can be configured in the leader node, which includes but is not limited to computers, computers, terminals, servers and other devices with data processing capabilities. Figure 1 As shown, the method includes:

[0047] S110. Send a snapshot preparation message to each follower node.

[0048] In this embodiment, a leader node (Leader Node) can be elected based on a distributed consensus algorithm, wherein the distributed consensus algorithm can be an algorithm that ensures that all nodes in a distributed system reach a consensus on the replication and state of data in the face of network partitions, node failures, etc., such as: Raft Consensus Algorithm, Raft algorithm is a consistency algorithm for distributed systems, which can achieve consistency in distributed systems in a simple and easy-to-understand manner. The leader node can be a key component in a distributed system and can be used to coordinate and manage other nodes in the cluster. Exemplarily, the functions that the leader node can implement include: processing client requests, log replication, fault recovery, data consistency detection, decision making, state synchronization and / or monitoring, etc. The snapshot processing method provided in this embodiment can be applied to the leader node.

[0049] Continuing from the above description, the follower node is usually not responsible for initiating operations or decisions, but responding to the instructions of the leader node. Exemplary functions that can be implemented by the follower node include: waiting for the instruction information of the leader node, election voting, state synchronization, fault detection and / or timeout triggering, etc.

[0050] Specifically, the leader node may send a snapshot preparation message to each follower node, wherein the snapshot may be a copy of the state of the system at a certain moment, the snapshot preparation message may be a preparation message before taking the snapshot, and the snapshot preparation message may include a snapshot identifier and a predetermined snapshot timestamp, and the snapshot identifier may be used to distinguish different snapshots; the predetermined snapshot timestamp may be a preset time for creating the snapshot. S120. When each follower node is ready, send a snapshot creation message to each follower node, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node.

[0051] Specifically, when each follower node is ready, the leader node can send a snapshot creation message to each ready follower node. When the follower node receives the snapshot creation message, each follower node can create a corresponding snapshot at the same time and generate corresponding metadata; wherein the snapshot creation message can be a message for creating a snapshot, and the metadata can be various descriptive information about the snapshot. Exemplarily, the metadata can include a snapshot identifier, a creation timestamp, a follower node identifier, a container list, a data location, a data size and / or a checksum, etc., wherein the snapshot identifier can be a unique identifier; the creation timestamp can be the snapshot creation time accurate to milliseconds; the node identifier can be the node identifier that creates this snapshot; the container list can be a list of identifiers of all containers contained in the snapshot; the data location can be the location of the snapshot data in the local storage system; the data size can be the total size of the snapshot data; the checksum can be a checksum used to verify the integrity of the data.

[0052] Continuing with the above description, each follower node can also compress the corresponding metadata and send it to the leader node to ensure the consistency, integrity and efficiency of creating the snapshot. Exemplarily, after the metadata is generated, it can be compressed using the LZ77 (Lempel-Ziv 1977) compression algorithm. LZ77 is a commonly used lossless data compression algorithm that reduces the amount of data by finding repeated patterns in the data.

[0053] It should be noted that a reliable network transmission protocol can be used to send the compressed metadata to the leader node to ensure that the data can reach the leader node completely and accurately. If a network problem is encountered during the sending process, a retry mechanism can be implemented. Usually a maximum number of retries can be set (for example, 3 times). If the data still cannot be sent successfully within the maximum number of retries, an exception can be thrown to mark the snapshot operation as failed. Throughout the process, logs can be continuously recorded to track the execution of each step of the operation. If an exception occurs at any stage, immediately record detailed error information, including the error type, occurrence time, related data, etc.

[0054] It should be explained that this embodiment can implement a distributed snapshot creation mechanism to ensure that all nodes can create consistent snapshots in a large-scale distributed environment. This mechanism mainly includes two key parts: time synchronization and a two-stage snapshot protocol. First, an accurate global time synchronization mechanism can be executed. This mechanism ensures that all nodes participating in snapshot creation remain highly consistent in time. Time synchronization is crucial for creating consistent snapshots because it allows all nodes to start the snapshot process at the same time. Next, a two-stage snapshot protocol can be implemented. This protocol is divided into two phases, "prepare" and "create", which are used to coordinate the snapshot creation process of all nodes. In the preparation phase, the leader node can send a snapshot preparation message to each follower node, and in the creation phase, the leader node can send a snapshot creation message to each follower node.

[0055] S130 . Decompress the metadata sent by each follower node, and generate global snapshot information according to the decompressed metadata.

[0056] It can be known that after receiving the metadata sent by each follower node, the leader node can decompress the received metadata, such as using the LZ77 algorithm. After the decompression is completed, the integrity and consistency of the decompressed metadata can be checked. The integrity can be to check whether the decompressed data contains necessary fields and / or whether it contains necessary content. The consistency can be to check whether the decompressed metadata is consistent with the expected information, such as: the snapshot identifier is consistent with the snapshot identifier of this operation. Continuing with the above description, if the decompressed data passes the verification, global snapshot information can be generated according to the decompressed metadata, wherein the global snapshot information can be a comprehensive and versioned snapshot information, and the global snapshot information can include: snapshot creation time; snapshot version generated based on the snapshot creation time; follower node information for creating the snapshot; data size of the snapshot of the follower node; total data volume of each follower node; container list; data location; verification information for verifying data integrity. The comprehensiveness of the snapshot can be guaranteed based on the global snapshot information. Exemplarily, global snapshot information may include: snapshot version: a unique version number generated based on the current timestamp; creation time: the snapshot creation time accurate to milliseconds; number of participating nodes: the total number of nodes participating in this snapshot; total data size: the sum of all node snapshot data; node metadata: detailed metadata information of each node, including data location, size, checksum, etc.

[0057] It should be noted that if the decompressed data has not been verified, such as incomplete metadata, checksum mismatch, etc., the leader node will send a request to the corresponding follower node to resend the metadata. A maximum number of retries can be set for this process to prevent an infinite loop.

[0058] This embodiment provides a snapshot processing method, including: sending a snapshot preparation message to each follower node; when each follower node is ready, sending a snapshot creation message to each follower node, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node; decompresses the metadata sent by each follower node, and generates global snapshot information based on the decompressed metadata, ensuring that all nodes in the distributed system can create consistent snapshots at the same time, and can maintain consistency even in the face of complex situations such as network delays and node failures. At the same time, it also provides a good error handling mechanism, which can promptly discover and handle problems in the snapshot creation process, and improve the reliability of the generated global snapshot information.

[0059] As an optional implementation of this embodiment, the snapshot processing method provided by this embodiment may further include:

[0060] 1) upon receiving ready messages from a specified number of follower nodes, confirming that each of the follower nodes is ready;

[0061] It should be noted that, when a follower node receives a snapshot preparation message, it can perform preparation operations before creating a snapshot, and after the preparation operations are completed, it can send a readiness message to the leader node. Specifically, when the leader node receives readiness messages from a specified number of follower nodes, it can determine that each follower node is ready. The specified number is a pre-set number.

[0062] 2) When the number of the received ready messages does not reach the specified number, an indication message of terminating the snapshot is sent to each of the follower nodes.

[0063] It should be noted that when the leader node does not receive the specified number of ready messages, the leader node can send an instruction message to terminate the snapshot to each follower node. Since the preparation time of each follower node may be different, entering the creation phase after receiving the specified number of ready messages ensures that each follower node completes the preparation work separately, and then can create snapshots at the same time, and can maintain consistency even in the face of complex situations such as network delays and node failures. At the same time, it also provides a good error handling mechanism, which can promptly discover and handle problems in the snapshot creation process.

[0064] As an optional implementation of this embodiment, before generating the global snapshot information according to the decompressed metadata, the following is further included:

[0065] Checking whether the decompressed metadata contains necessary fields, and whether the necessary fields are consistent with information in the snapshot creation message;

[0066] The necessary fields include a snapshot identifier, a follow-up node identifier, and a snapshot timestamp.

[0067] Specifically, after receiving the metadata sent by the follower node, the leader node can decompress the received metadata. For example, the metadata can be decompressed using the LZ77 algorithm. After the metadata is decompressed, it can be checked whether the metadata after decompression contains necessary fields. The necessary fields can be fields that must be included in the metadata. The necessary fields can include a snapshot identifier, a follower node identifier, and a snapshot timestamp.

[0068] Continuing with the above description, when it is determined that the decompressed metadata includes necessary fields, it can be determined whether the necessary fields are consistent with the information in the snapshot creation message. The integrity and consistency of the decompressed metadata are checked through the necessary fields, thereby improving the reliability of the snapshot.

[0069] It should be noted that when checking the decompressed metadata, if there is a problem with the decompressed metadata, such as incomplete metadata, mismatched checksum, etc., the leader node will send a request to the corresponding follower node to resend the metadata. The resending process can set a maximum number of retries (such as 3 times) to prevent an infinite loop.

[0070] As an optional implementation of this embodiment, the snapshot processing method provided by this embodiment may further include:

[0071] Sending a rollback instruction to each of the follower nodes, wherein the rollback instruction is used to instruct each of the follower nodes to perform a rollback operation;

[0072] In this embodiment, the rollback operation can be used to safely restore to the state before the transaction started when failures and errors occur, thereby ensuring the data consistency and reliability of the entire system. The rollback instruction can be an instruction for starting the rollback operation. When performing the rollback operation, the Raft algorithm can also be used to determine the leader node. Exemplarily, when the system needs to perform a rollback operation, a leader node election can be triggered first. Each node has the opportunity to become a candidate. The candidate can increase its term number and send a message requesting a vote to all other nodes; after receiving the voting request, other nodes can decide whether to vote in support according to certain rules (such as a higher term number). Each node can only vote for one candidate during a term; the candidate can count the number of votes received. If the support of the majority of nodes is received, the candidate will become the new leader node; the new leader node can immediately send a heartbeat message to all other nodes to establish its leadership position; throughout the process, the system can dynamically adjust the heartbeat interval and election timeout according to the current network status and system load. For example, increase the heartbeat interval when the network is congested, and extend the election timeout when the system load is high.

[0073] Before starting the actual rollback operation, the system can also create a new snapshot to capture the current system state and provide a recovery point for possible rollback failures. Exemplarily, the specific process may include: initializing snapshot creation: the newly elected leader node will initiate a snapshot creation request, using the same process as regular snapshot creation; all nodes synchronize their clocks to ensure that the snapshot is created at a consistent time point; perform the preparation phase and the creation phase to ensure that all nodes create the snapshot at the same time; after each node creates the snapshot, it will generate and compress metadata and then send it to the leader node; the leader node collects and processes the metadata of all nodes, generates a global snapshot information (global versioned snapshot information), and stores the global snapshot information in the distributed storage system as a potential rollback point.

[0074] The rollback operation includes:

[0075] 1) Read and verify the global snapshot information, and copy the global snapshot information to the local cache;

[0076] Specifically, the leader node can decompress the global snapshot information, read the global snapshot information, and verify the global snapshot information. Exemplarily, the leader node can verify the validity of the snapshot version number, check whether the snapshot creation time is within the allowed range, verify whether the number of participating nodes meets expectations, and / or check whether the total data size is within a reasonable range.

[0077] Continuing with the above description, after the global snapshot information verification is completed, the local high-speed storage path can be used as a cache location to copy the global snapshot information to the local cache through an asynchronous input / output mechanism. A reasonable timeout period (e.g., 1 hour) can also be set to handle large snapshots.

[0078] It should be noted that in the rollback operation, the leader node can also start a two-stage snapshot rollback protocol (i.e., the preparation phase and the creation phase) to ensure that all nodes are ready for rollback. Specifically, the leader node can send a rollback preparation message to all follower nodes, including the identifier of the target snapshot. After receiving the message, each follower node will verify whether it is capable of performing the rollback (such as checking local resources, verifying local snapshot data, etc.). Among them, the local snapshot data can be that each node stores the snapshot data in the local file system when creating the snapshot. For example, a compressed file is generated to save the current state data. The follower node returns a ready or unready response to the leader node. The leader node counts the response results. Only when the ready response is received from the majority of nodes will it proceed to the next step. At the same time, after confirming that it is ready, each node will reserve the necessary system resources (such as storage space, computing resources, etc.) to perform the rollback operation. If the preparation phase is successfully completed, the leader node will update the global state and mark the system into the rollback preparation state. If a problem is found at any stage (such as snapshot verification failure, node not ready, etc.), the leader node will abort the rollback operation, clean up the reserved resources, and notify all nodes that the rollback preparation has failed.

[0079] 2) Create a container instance for each container and transfer the metadata corresponding to each container from the original container to the created corresponding container;

[0080] Specifically, the container configuration information in the global snapshot information can be obtained, and a container instance is created for each container according to the container configuration information, and the network settings are configured to ensure consistency with the original container. When the network settings are completed, the rollback status can be updated to "prepared to complete".

[0081] Continuing with the above description, you can query the rollback status again and confirm that the rollback status is "Ready to Complete". After confirmation, you can start the container instance, and when the container instance reaches a healthy state (the application in the container instance can run normally), you can set a timeout mechanism (such as 60 seconds), update the load balancer, and transfer the metadata corresponding to each container from the original container to the corresponding container created. After the transfer is complete, the meta-container can be closed. Exemplarily, the meta-container can be closed by: sending a software interrupt (SIGTERM) signal; or waiting for a certain period of time (such as 30 seconds) for the container to process the current request; if the container is still running, a software interrupt (SIGTERM) signal can be sent to force termination.

[0082] In this embodiment, the container instance is a cloud service product based on container technology, which allows users to run container applications without managing underlying servers. The container can be a way to store and manage data.

[0083] 3) Verify the metadata of each created container, and complete the rollback if the verification passes.

[0084] Following the above description, after the metadata corresponding to each container is transferred from the original container to the corresponding container created, the metadata of each created container can be verified, which improves the reliability of the rollback. The verification can specifically include: defining a series of key data points, such as: important configuration files, database files, etc.; verifying and calculating these key data points of each container; comparing the checksums of each container to ensure that they are completely consistent. If the verification of each created container passes, it can be determined that the rollback is completed, and the rollback status can be updated to "rolled back".

[0085] Figure 2 A schematic diagram of a rollback operation process provided in this embodiment is as follows: Figure 2 As shown, the container snapshot (global snapshot information) can be verified. If the verification is successful, a rollback plan can be created and the rollback data can be preheated: such as obtaining the snapshot data location from the storage system; selecting a local high-speed storage path as the cache location; using an asynchronous I / O mechanism (asynchronous input / output mechanism) to copy data to the local cache; setting a reasonable timeout (such as 1 hour) to handle large snapshots.

[0086] Following the above description, after data preheating is completed, a new container instance can be created. After successful creation, the rollback status (rollback mark) can be updated to "Ready to Complete". You can confirm the rollback status (rollback mark) as "Ready to Complete" again, and after confirmation, perform container switching (i.e., transfer the metadata corresponding to each container from the original container to the corresponding container created). After the switch is completed, these key data points of all new containers are verified and calculated to ensure data consistency. Update the rollback status, clean up the old container resources, update the container list, and the rollback operation is completed.

[0087] Figure 3 A timing diagram of a rollback operation provided in this embodiment, such as Figure 3As shown, the rollback operation is divided into two main stages: the preparation of container snapshot rollback stage and the submission of container snapshot rollback stage. In the preparation of container snapshot rollback stage, the container snapshot can be verified first. After the verification result is obtained, the rollback data can be preheated and the required data can be loaded into the local high-speed storage. After the preheating is completed, a new container instance is created for each container. After the creation is completed, the submission of container snapshot rollback stage can be entered: start the new container. After the container is started, the health status of the container can be checked. After determining that the container has reached a healthy state, the load balancer can be updated, and the metadata corresponding to each container can be transferred from the original container (old container) to the corresponding container created. After the update is completed, the old container can be closed. After passing the data consistency check, the original container (old container) can be deleted, and the data and network configuration related to the original container can be cleaned up. After the rollback is completed, the created new container instance and the related preset data can also be deleted.

[0088] It should be noted that during the rollback operation, if an error occurs, detailed error information can be recorded, and the rollback operation is performed for the switched container: restart the old container (if it has been stopped); update the load balancer to redirect traffic to the old container; stop and delete the newly created container instance; and clean up all temporary resources and states. During the rollback operation, in order to ensure that no other operations interfere, a global lock mechanism can be used to ensure that during the rollback operation.

[0089] It should be noted that during the rollback operation, snapshot information can also be recorded, such as: generating a unique version number and timestamp for each snapshot; recording detailed metadata, including information such as the number of nodes and data size; and recording the location, size, and checksum of each node's data. These snapshot information are used in the rollback operation to verify and restore the system state.

[0090] Continuing from the above description, during the entire rollback process, you can also update the progress of the rollback operation in real time, record detailed operation logs, and send notifications to administrators or monitoring systems at key nodes (such as stage completion, error occurrence). After the rollback process is successful, you can clean up all temporary data and intermediate states, perform a final system health check, and generate a detailed rollback report, including the operation time, affected containers, any problems encountered, etc.

[0091] Through the two-stage rollback protocol (prepare and create two stages), the system can perform safe and reliable rollback operations in a distributed environment, ensure data consistency and operation atomicity, provide powerful error handling and rollback mechanisms, and greatly improve the robustness and reliability of the system in the face of complex situations. The rollback operation provided in this embodiment adopts a two-stage rollback protocol, that is, the rollback operation is divided into two main stages: the prepare rollback stage and the commit rollback stage. This two-stage protocol ensures that either all containers are successfully rolled back or all containers remain in their original state. Prepare rollback stage: first verify the validity of the target snapshot; create a detailed rollback plan, including snapshot identification, rollback timestamp and other information; preheat the rollback data and load the required data into local high-speed storage; create new instances for each container, but do not start them; set the rollback mark to indicate that the preparation stage is completed.

[0092] Commit rollback phase: Check the rollback marker to ensure that the preparation phase has been completed; perform container switching, including starting new containers, updating load balancers, shutting down old containers, etc.; perform data consistency checks to ensure that the key data points of all new containers are consistent; if the consistency check passes, update the rollback status to completed; otherwise, roll back all switched containers; clean up old container resources. A global lock mechanism is used in the rollback operation to ensure that no other operations interfere during the rollback operation. This lock runs through the entire preparation and submission phases to ensure the atomicity of the operation. The atomicity of the rollback operation can also be ensured in the rollback operation through the following mechanisms: create new container instances in the preparation phase, but do not start them to avoid affecting the currently running system; only when all preparations are completed will the submission phase be entered. In the submission phase, transactional operations are used to switch containers to ensure that either all succeed or all fail; if any errors occur in the submission phase, the compensation mechanism will be triggered to roll back the switched containers to their original state.

[0093] Continuing with the above description, after the container switch is completed, a data consistency check is performed: define a series of key data points that are critical to the correct operation of the system; perform checksum calculations on these key data points for each new container; compare the checksums of all containers to ensure that they are completely consistent; if any inconsistency is found, trigger a rollback operation immediately. If an error occurs during the rollback process, the system will try to restore the switched container to its original state, such as: restarting the original container (if it has been stopped); waiting for the original container to return to the ready state; updating the load balancer to redirect traffic to the original container; stopping and deleting the newly created container instance.

[0094] Continuing with the above description, you can also use versioned snapshot information to manage rollback status, such as: generating a unique version number and timestamp for each snapshot; recording detailed metadata, including information such as the number of nodes and data size; and recording the location, size, and checksum of each node's data. This versioning mechanism allows the system to accurately track the status of each snapshot, facilitating accurate rollback operations.

[0095] Optionally, decompressing metadata sent by each follower node includes:

[0096] 1) If the metadata of all the following nodes are received within the set time period, then the metadata of all the following nodes are decompressed;

[0097] It is known that a set period can be set, and the set period can be a pre-set time range. The determination of the set period needs to weigh factors such as system scale and network conditions, and can usually be set to a few minutes to tens of minutes. If the leader node receives metadata from all follower nodes within the set period, it can decompress the metadata of all follower nodes.

[0098] 2) If metadata of some follower nodes are received within a set period of time, indication information of terminating the snapshot or the received metadata of some follower nodes is sent to each of the follower nodes.

[0099] Specifically, if only some of the metadata of the follower nodes are received within the set period, it can be decided based on the system design and current status whether to send an indication message of terminating the snapshot to each follower node, or to receive some of the metadata of the follower nodes and decompress some of the metadata. Receiving metadata within the set period prevents the phenomenon of termination due to a node not sending metadata in time, prevents the snapshot failure rate from being too high, and affects data storage and rollback. Only the received metadata is decompressed, which improves the flexibility of storage and rollback and ensures storage and rollback efficiency.

[0100] For example, in some scenarios with high requirements for data consistency (such as the financial field), data loss or inconsistency may not be tolerated. If the leader node believes that the current metadata is not sufficient to ensure consistency, it can decide to send an instruction to terminate the snapshot to each follower node. If a new node joins the cluster or an existing node recovers from a failure, if the leader node believes that the received metadata is sufficient to support the rapid synchronization or recovery of the new node, it can decide to receive the metadata of some follower nodes. Otherwise, if critical data is missing, it may choose to abort the operation and wait for more metadata.

[0101] Continuing with the above description, in the Raft algorithm, when the log is too large, the number of logs will be reduced through the snapshot mechanism, and the leader node can decide to receive metadata from some follower nodes. If the metadata management mechanism can effectively handle large-scale data, the leader node can send an indication of terminating the snapshot to each follower node for subsequent acquisition of all metadata.

[0102] In the snapshot processing method provided in this embodiment, when an unrecoverable error occurs, the leader node can record detailed error information, including the error type, occurrence time, impact scope, etc.; it can also notify all participating nodes to terminate the current snapshot operation, clean up the temporary data and status that have been created, and report the operation result (failure) to the upper system or administrator who initiated the snapshot request. If all steps of the snapshot processing method are successfully completed, the leader node can update the system status, mark the snapshot operation as successful, send a confirmation message to all participating nodes, allow them to clean up temporary data, and report the operation result (success) to the upper system or administrator who initiated the snapshot request.

[0103] Figure 4 A schematic diagram of a snapshot processing method provided in this embodiment is shown in FIG. Figure 4 As shown in the figure, when the snapshot creation process can be started, a Leader node is first elected through the distributed consensus algorithm (Raft), and the heartbeat interval is dynamically adjusted according to the system load to achieve a balance between resource consumption and fault detection speed; the election timeout is dynamically adjusted according to the network status to achieve adaptive threshold adjustment. The precise global time synchronization mechanism is used to synchronize the time of all nodes. The two-stage ("preparation phase" and "creation phase") snapshot protocol is implemented. The Leader node coordinates all Follower nodes and sends a snapshot preparation message. After all nodes are ready (preparation phase is successful), a snapshot creation message is sent to ensure that all nodes create snapshots at the same time. Each node stores the snapshot data locally, compresses the metadata using the LZ77 algorithm, and synchronizes the snapshot metadata to the Leader node. The Leader node collects the snapshot metadata of all nodes, decompresses the synchronized metadata information, adds version information, generates versioned snapshot information (global snapshot information), and stores it in the distributed storage system after compression to complete the snapshot creation. If the storage is not successful, the storage can be retried. If the storage is successful, the snapshot is confirmed to be completed. If the storage is unsuccessful, a storage exception is thrown.

[0104] Figure 5 A timing diagram of a snapshot processing method provided in this embodiment, such as Figure 5As shown, the leader node is selected by dynamically adjusting the heartbeat interval and the election timeout. After the leader node is determined, the precise global time synchronization mechanism is used to synchronize the time of all nodes based on the two-stage snapshot protocol: the leader node sends a snapshot preparation message to other nodes (follower nodes, such as node 2 and node 3), and each follower node sends a ready message to the leader node after being ready; after receiving each ready message, the leader node sends a snapshot creation message to each follower node, and each follower node sends a creation confirmation message to the leader node to confirm the creation of the snapshot. Each follower node creates a local snapshot and compresses the metadata of the snapshot, and sends the metadata to the leader node. The leader node can decompress the metadata, generate versioned snapshot information (global snapshot information), and store it.

[0105] It should be noted that the snapshot and rollback mechanism provided in this embodiment can perform snapshot and rollback operations on a single node or centralized storage system, which is relatively simple to implement, suitable for small-scale systems, and relatively easy to manage and configure. It is also possible to perform snapshot and rollback operations based on virtual machines to achieve system recovery, which is suitable for virtualized environments. Through virtual machine management tools, a certain degree of automated management can be achieved. It is also possible to perform snapshot and rollback operations at the application layer to ensure the consistency of the application state, and can be optimized for specific applications to improve the efficiency of snapshot and rollback operations. It is suitable for specific application scenarios and has high flexibility.

[0106] Figure 6 This is a flowchart of another snapshot processing method provided by an embodiment of the present disclosure, which can be applied to a follower node. The method is applicable to the case of processing a snapshot, and the method can be executed by a snapshot processing device, which can be implemented in the form of hardware and / or software, and the snapshot processing device can be configured in a follower node, which includes but is not limited to a computer, a terminal, a server, and other devices with data processing capabilities. Figure 6 As shown, the method includes:

[0107] S210, when receiving the snapshot preparation message, performing preparation operations before creating the snapshot, and sending a preparation message to the leader node after being ready;

[0108] In this embodiment, after receiving the snapshot preparation message sent by the leader node, the follower node can perform preparations before creating the snapshot, such as clearing the cache, suspending non-critical operations, etc. After each follower node completes the preparations (is ready), it can send a ready message to the leader node.

[0109] S220. When a snapshot creation message is received, a snapshot is created at a specified time, and metadata of the snapshot is compressed and sent to the leader node.

[0110] In this embodiment, when the follower node receives the snapshot creation message sent by the leader node, it can create a snapshot at a specified time. Specifically, the process of creating a snapshot may include: obtaining all relevant data status of each follower node at present, and the data status includes but is not limited to: a list of running containers, configuration information of containers, network settings, storage volume information, etc. It should be noted that a specific application programming interface (Application Programming Interface, API) or interface can be used to obtain this information to ensure that a consistent view is obtained.

[0111] Following the above description, the acquired data state will be serialized and stored in the local storage system. In this process, the Copy-on-Write (CoW) mechanism may be used to minimize the impact on the running system. The CoW mechanism allows the system to create a new copy only when the data is modified, thereby greatly reducing storage overhead and performance impact. According to the data state, a detailed metadata structure can be generated to describe the various information of this snapshot. Among them, CoW can be a programming technique that is commonly used in computer programs, especially in the fields of operating systems and memory management. The CoW mechanism allows multiple processes or threads to share a copy of the same data until one of the processes or threads needs to modify the data, and then the actual copy operation will be performed, effectively reducing unnecessary data copying and improving resource utilization efficiency. After obtaining the metadata of the snapshot, the metadata of the snapshot can be compressed and the compressed metadata can be sent to the leader node.

[0112] This embodiment provides a snapshot processing method, including: when a snapshot preparation message is received, performing preparation operations before creating a snapshot, and sending a ready message to the leader node after preparation is complete; when a snapshot creation message is received, creating a snapshot at a specified time, and compressing the metadata of the snapshot and sending it to the leader node, ensuring that each node creates a snapshot at the same specified time, thereby improving the reliability of the generated global snapshot.

[0113] As an optional implementation of this embodiment, the snapshot processing method provided by this embodiment further includes: when a rollback instruction is received, performing the following rollback operation:

[0114] 1) Read and verify the global snapshot information, and copy the global snapshot information to the local cache;

[0115] In this embodiment, after receiving the rollback indication, the follower node can read the global snapshot information and verify the global snapshot information. The global snapshot information may include: snapshot creation time; snapshot version generated based on the snapshot creation time; follower node information that creates the snapshot; data size of the follower node's snapshot; total data volume of each follower node; container list; data location; and verification information for verifying data integrity.

[0116] Exemplarily, the validity of the snapshot version number in the global snapshot information can be verified, whether the snapshot creation time is within the allowed range, whether the number of participating nodes meets expectations, and / or whether the total data size is within a reasonable range.

[0117] Continuing with the above description, after the global snapshot information verification is completed, the local high-speed storage path can be used as a cache location to copy the global snapshot information to the local cache through an asynchronous input / output mechanism. A reasonable timeout period (e.g., 1 hour) can also be set to handle large snapshots.

[0118] It should be noted that a detailed rollback plan may also be generated, the rollback plan including information such as a snapshot identifier and a rollback timestamp, and the rollback state is initialized to "preparing".

[0119] 2) Create a container instance for each container and transfer the metadata corresponding to each container from the original container to the created corresponding container;

[0120] Specifically, the container configuration information in the global snapshot information can be obtained, and a container instance is created for each container according to the container configuration information, and the network settings are configured to ensure consistency with the original container. When the network settings are completed, the rollback status can be updated to "prepared to complete".

[0121] Continuing with the above description, you can query the rollback status again and confirm that the rollback status is "Ready to Complete". After confirmation, you can start the container instance, and when the container instance reaches a healthy state (the application in the container instance can run normally), you can set a timeout mechanism (such as 60 seconds), update the load balancer, and transfer the metadata corresponding to each container from the original container to the corresponding container created. After the transfer is complete, the meta-container can be closed. Exemplarily, the meta-container can be closed by: sending a software interrupt (SIGTERM) signal; or waiting for a certain period of time (such as 30 seconds) for the container to process the current request; if the container is still running, a software interrupt (SIGTERM) signal can be sent to force termination.

[0122] 3) Verify the metadata of each created container, and complete the rollback if the verification passes.

[0123] Specifically, after the metadata corresponding to each container is transferred from the original container to the corresponding container created, the metadata of each created container can be verified. Specifically, it includes: defining a series of key data points, such as: important configuration files, database files, etc.; verifying and calculating these key data points of each container; comparing the checksums of each container to ensure that they are completely consistent. If the verification of each created container passes, it can be determined that the rollback is completed, and the rollback status can be updated to "rolled back"

[0124] Figure 7 A schematic diagram of a rollback execution process provided in this embodiment is as follows: Figure 7 As shown, the leader node is elected. After the leader node is determined, the versioned snapshot information (global snapshot information) is decompressed to verify the integrity and consistency of the snapshot. After the preparation is completed, the rollback data is distributed. After the distribution is completed, all nodes perform the rollback operation synchronously. After completion, each node reports the rollback status to the leader node. The leader node collects the rollback status of all nodes. If all nodes roll back successfully, the rollback is confirmed to be complete; otherwise, the rollback failure processing process is started, and an attempt is made to restore to the state before the rollback, and a rollback report is generated.

[0125] Specifically, the rollback execution process can be described in detail as follows: First, elect a leader node: This step is similar to the leader election step in the snapshot creation process, and the same Raft algorithm is used for leader election. The system will maintain a dynamic heartbeat interval and election timeout adjustment mechanism to adapt to different network environments and system loads. The election process includes: Initialization election: When the system needs to perform a rollback operation, a leader election will be triggered first. Each node has the opportunity to become a candidate. Increase term: The candidate will increase its term number and send a message to all other nodes requesting a vote.

[0126] Voting process: After receiving a voting request, other nodes will decide whether to vote in favor based on certain rules (such as a higher term number). Each node can only vote for one candidate during a term. Counting votes: The candidate will count the number of votes received. If it receives support from a majority of nodes, the candidate will become the new Leader. Establishing leadership: The new Leader will immediately send a heartbeat message to all other nodes to establish its leadership. Dynamic parameter adjustment: Throughout the process, the system will dynamically adjust the heartbeat interval and election timeout based on the current network conditions and system load. For example, increase the heartbeat interval when the network is congested, and extend the election timeout when the system load is high.

[0127] After the leader node is selected, the snapshot creation process is called to capture the system state at the start of the rollback. Before starting the actual rollback operation, a new snapshot can be created to capture the current system state. The purpose of this step is to provide a recovery point for possible rollback failures. The specific process is as follows:

[0128] Initialize snapshot creation: The newly elected Leader node will initiate a snapshot creation request, using the same process as regular snapshot creation. Synchronize time: All nodes synchronize their clocks to ensure that the snapshot is created at a consistent point in time. Two-stage snapshot creation: Execute the preparation phase and the creation phase to ensure that all nodes create the snapshot at the same time. Metadata collection: After each node creates a snapshot, it generates and compresses metadata and then sends it to the Leader node. Metadata processing: The Leader node collects and processes the metadata of all nodes to generate a global versioned snapshot information. Storing snapshot information: Store the versioned snapshot information in a distributed storage system as a potential rollback point.

[0129] After calling the snapshot creation process and capturing the system state at the start of the rollback, you can verify the snapshot and prepare for the rollback. Specifically, the system will verify the validity of the target snapshot and prepare for the rollback operation. The specific process is as follows:

[0130] Retrieve snapshot information: The leader node retrieves the version information of the target snapshot from the distributed storage system. Decompression and verification: Decompress the snapshot information and verify its integrity and consistency. Checks include but are not limited to: verifying the validity of the snapshot version number; checking whether the snapshot creation time is within the allowed range; verifying whether the number of participating nodes is in line with expectations; and checking whether the total data size is within a reasonable range. Preparation phase protocol: The leader node initiates a two-phase protocol to ensure that all nodes are ready for rollback: a) The leader sends a rollback preparation message to all follower nodes, including the identifier of the target snapshot. b) After receiving the message, each follower node verifies whether it is capable of performing the rollback (such as checking local resources, verifying local snapshot data, etc.). c) The follower node returns a ready or not ready response to the leader. d) The leader counts the response results. Only when a ready response is received from the majority of nodes will it proceed to the next step. Resource reservation: After confirming that it is ready, each node will reserve the necessary system resources (such as storage space, computing resources, etc.) to perform the rollback operation. Status update: If the preparation phase is successfully completed, the leader node will update the global status and mark the system as ready for rollback. If a problem is found at any stage (such as snapshot verification failure, node not ready, etc.), the leader will abort the rollback operation, clean up the reserved resources, and notify all nodes that the rollback preparation has failed.

[0131] After verifying the snapshot and preparing for rollback, the rollback data can be distributed. In this step, the Leader node is responsible for distributing the data required for the rollback to all participating Follower nodes. This process needs to be carefully designed to ensure data integrity and consistency. The specific process is as follows:

[0132] Data preparation: The leader node first retrieves the complete snapshot data from the distributed storage system. This includes the snapshot data location, size, and checksum information for each node. Data distribution: For each node participating in the rollback, the leader performs the following operations: a) Determine the snapshot data required by the node; b) If the data is located in remote storage, the leader will first download the data to the local cache; c) Use a reliable data transfer protocol (such as RDMA or optimized TCP protocol) to send the data to the target node; d) Use a checksum mechanism to ensure data integrity during transmission. Parallel transmission: To improve efficiency, the leader will perform data transmission in parallel as much as possible, while taking into account network bandwidth limitations. Error handling: If problems are encountered during data transmission (such as network interruption, checksum mismatch, etc.), the system will take the following measures: a) Retry the transmission operation, usually setting a maximum number of retries (such as 3 times); b) If the retry fails, try to obtain data from an alternative data source; c) If it still fails, mark the node as a fault and consider whether to abort the entire rollback operation; Confirm reception: After successfully receiving and verifying the data, each Follower node will send a confirmation message to the Leader; Completion monitoring: The Leader node will monitor the progress of data distribution in real time and may report the progress to the administrator or monitoring system.

[0133] After the rollback data distribution is completed, the rollback can be performed. This is the core step of the rollback operation. All nodes synchronously perform the actual rollback operation. The process is as follows:

[0134] Synchronous signal: The leader node sends a synchronous rollback execution signal to all ready follower nodes. Stop related services: Each node first gracefully stops the affected services and containers. This process includes: a) sending a SIGTERM signal to related processes to allow them to perform cleanup operations; b) waiting for a predefined time (such as 30 seconds) to allow the process enough time to clean up; if the process does not exit within the predetermined time, send a SIGKILL signal to force termination. Apply snapshot data: Each node begins to apply the received snapshot data to the local system. This process may include: a) replacing files and directories in the file system; b) updating database content; c) resetting system configuration. Consistency check: After applying the snapshot data, each node will perform a local consistency check to ensure that all critical data has been correctly restored. Start services: Start each service and container in a predefined order. This process needs to consider the dependencies between services. Health check: Perform a health check on each started service to ensure that they are running normally. Report status: After completing the above steps, each node reports its rollback status (success or failure) to the leader.

[0135] After the rollback is completed, you can confirm that the rollback is complete. This is the last step of the rollback operation and is used to confirm the success of the entire rollback process. The details are as follows:

[0136] Collect status: The leader node collects the rollback status reports from all participating nodes. Evaluate results: The leader analyzes the reports from all nodes to determine whether the rollback is successful. The criterion for success is usually that all nodes (or a predefined set of key nodes) report that the rollback is successful. Success processing: If the rollback is judged to be successful, the leader will: a) update the system global status and mark it as "rolled back"; b) send a confirmation message to all nodes that the rollback is complete; c) trigger any necessary post-rollback operations, such as updating the configuration management database, notifying the monitoring system, etc. Failure processing: If the rollback is judged to be a failure, the leader will start the rollback failure processing process: a) record the detailed failure reason and the affected nodes; b) try to restore the system to the state before the rollback started (using the snapshot created in step 2); c) notify all nodes of the rollback failure and ask them to perform any necessary cleanup operations; d) generate a detailed failure report, including the failure reason, the scope of impact, and the recommended repair steps. Resource cleanup: Whether the rollback succeeds or fails, the system will perform necessary resource cleanup, such as deleting temporary files and releasing reserved system resources. Notification: Send notification of the final result of the rollback operation to the system administrator or upper-level management system. Log archiving: Archive detailed logs of the entire rollback process for subsequent analysis and auditing. Figure 8 A timing diagram of a rollback execution process provided in this embodiment, such as Figure 8As shown, the client requests to roll back to the specified snapshot version. First, the Leader node is elected. The Leader node retrieves and decompresses the versioned snapshot information from the distributed storage system. Verify the integrity and consistency of the snapshot. The Leader node sends a rollback preparation message to the Follower node, and the Follower node confirms that it is ready to the Leader node. The Leader node obtains the snapshot data from the distributed storage system and distributes the rollback data to all Follower nodes. Each Follower node verifies the integrity of the received data. The Leader node sends an execution rollback command, and each Follower node executes the rollback operation synchronously. After completion, each node reports the rollback status to the Leader node, and the Leader node determines that all nodes have rolled back successfully. If all nodes roll back successfully, the Leader node sends a rollback completion notification to each Follower node and reports the rollback success to the client. If the rollback fails, the Leader node notifies each Follower node of the rollback failure, collects the cause and status of the rollback failure, attempts to restore each Follower node to the state before the rollback, and reports the rollback failure to the client.

[0137] Optionally, perform preparations before creating a snapshot, including:

[0138] 1) Creating initial snapshot metadata for each container, the snapshot metadata including a snapshot identifier, a container identifier, and a snapshot timestamp;

[0139] In this embodiment, when the follower node receives the snapshot preparation message, it can create initial snapshot metadata for each container, and the initial metadata includes: a snapshot identifier, a container identifier, and a snapshot timestamp.

[0140] 2) Enable the copy-on-write mechanism for each container to maintain data status;

[0141] Specifically, each container can enable the Copy-on-Write (CoW) mechanism to maintain data status, allowing the system to capture a consistent data status without interrupting normal operations.

[0142] 3) Setting a write operation hook for each container, where the write operation hook is used to capture all write operations after the snapshot is created;

[0143] In this embodiment, a write operation hook can also be set for each container, and the write operation hook is used to capture all write operations after the snapshot is created; the write operation hook can also achieve: intercepting all write operation requests; checking whether the target file needs to perform write-time copy. If necessary, create a file copy before performing the write operation; record the write operation details (file path, offset, data, etc.) in the incremental log; and perform the actual write operation.

[0144] 4) Create an initialized incremental log for each container, which is used to record all data change information after the snapshot is created;

[0145] In this embodiment, an initialized incremental log can also be created for each container to record all change information after the snapshot is created. Wherein, the change information can be the change of all data after the snapshot is created. Exemplarily, each incremental log can include: the incremental log is an append-type log file or a similar data structure for recording all data changes after the snapshot is created. Each log contains: operation type (such as write, delete, create, etc.), file path, timestamp, change data (for write operations), metadata changes (such as permission changes). The incremental log can also include an asynchronous incremental log. The asynchronous application incremental log can be a logging technology that allows the application to asynchronously write log information to the log storage while executing the main task. The asynchronous application incremental log can include: creating an independent background thread or process, continuously reading the next entry of the incremental log, applying the change to the snapshot file system, updating the applied log position and / or if all logs have been applied, marking the snapshot as a "fully synchronized" state, etc. Through mechanisms such as copy-on-write, write operation hooks, and incremental logs, it can be used to record various changes in the system after the snapshot is created.

[0146] 5) Create a reference point for the file system as the starting point for the snapshot;

[0147] In this embodiment, a reference point may also be created at the file system level, and the reference point may be used as a starting point for the snapshot.

[0148] 6) Update the snapshot status to Ready.

[0149] In this embodiment, the snapshot status may also be updated, for example, the snapshot status may be updated to “ready”.

[0150] Optionally, create a snapshot at a specified time, including:

[0151] 1) serializing the list of containers, configuration information, network settings and / or storage volume information of the follower node running at a specified time to create a snapshot;

[0152] Specifically, when a follower node creates a snapshot at a specified time, it can capture all relevant data states of the follower node at present, where the data states include but are not limited to: a list of running containers, configuration information of containers, network settings, storage volume information, etc. The follower node can use a specific application programming interface (Application Programming Interface, API) or interface to obtain these data states. The obtained list of running containers, configuration information of containers, network settings and / or storage volume information can be serialized to create a snapshot.

[0153] 2) Apply all data change information and all write operations after the snapshot is created to the created snapshot;

[0154] Specifically, after creating a snapshot, all data change information and all write operations created by the snapshot can be applied to the created snapshot, thereby improving the reliability of the snapshot, wherein the data change information can be recorded in the incremental log. Exemplarily, applying all data change information and all write operations after the snapshot is created to the created snapshot can include: replacing files and directories in the file system, updating database content, resetting system configuration, etc.

[0155] 3) Update the snapshot status to creation completed.

[0156] Specifically, after all data change information and all write operations after the snapshot is created are applied to the created snapshot, the snapshot status may be updated to “creation completed”.

[0157] Fig. 9 A schematic diagram of a process of creating a snapshot provided in this embodiment, such as Fig. 9 As shown, the snapshot creation is initialized to prepare all nodes for the snapshot. When all nodes are ready, all nodes start to create snapshots at the same time. After the snapshot creation of all nodes is completed, the incremental log is applied asynchronously. When the snapshot is not completed, incremental changes can be applied. When the snapshot is completed, the snapshot creation is completed.

[0158] Specifically, the two-stage snapshot creation protocol is used to ensure that all nodes in a distributed environment can create consistent snapshots. Each node is responsible for managing the snapshot creation process of the containers running on it, as described below:

[0159] Node management: Each node is responsible for managing the snapshot creation process for the containers running on it. The node maintains a list of containers and a snapshot metadata store. This information is essential for coordinating the snapshot creation process.

[0160] Preparation phase: a) Initialize snapshot metadata: Create initial snapshot metadata for each container, including snapshot ID, container ID and timestamp; b) Enable copy-on-write mechanism: Enable copy-on-write (CoW) mechanism for each container. This allows the system to capture consistent data state without interrupting normal operation; Set write operation hook: Set write operation hook for each container to capture all write operations after snapshot creation; Initialize incremental log: Create an incremental log for each container to record all changes after snapshot creation; Create file system reference point: Create a reference point at the file system level as the starting point of the snapshot; Update snapshot status: Update the snapshot status to "prepared".

[0161] Creation phase: a) Create a file system snapshot: Use the snapshot function of the underlying file system (such as B-tree File System (Btrfs) or Zettabyte File System (ZFS)) to create a file system-level snapshot for each container; b) Start recording incremental changes: Start the incremental change recording mechanism to capture all data modifications after the snapshot is created; c) Update the snapshot status: Update the snapshot status to "creation completed"; d) Asynchronously start applying incremental logs: Start a process in the background to apply the recorded incremental changes to the snapshot.

[0162] Write operation hook implementation: Write operation hook is an interception mechanism used to capture and record all write operations of the file system without affecting the normal execution of these operations. Modify the file system driver or use a file system filter. a) Intercept all write operation requests; b) Check whether the target file needs to perform copy-on-write. If necessary, create a copy of the file before performing the write operation; c) Record the write operation details (file path, offset, data, etc.) to the incremental log; d) Perform the actual write operation.

[0163] Incremental log implementation: The incremental log is an append-type log file or similar data structure that records all data changes after the snapshot is created. It can be used to record data change mechanisms and store all modification operations that have occurred since the snapshot was created. Use an append-type log file or similar data structure. Use asynchronous writing to minimize performance impact. Each log can contain: a) operation type (such as write, delete, create, etc.); b) file path; c) timestamp; d) change data (for write operations); e) metadata changes (such as permission changes)

[0164] Asynchronous Apply Delta Log: Asynchronous Apply Delta Log is a background process that continuously applies recorded changes to the snapshot to keep the snapshot up to date. Create a separate background thread or process. Asynchronous Apply Delta Log can achieve: a) Create a separate background thread or process; b) Continue to read the next entry of the delta log; c) Apply the changes to the snapshot file system; d) Update the applied log position; e) If all logs have been applied, mark the snapshot as "fully synchronized".

[0165] Error handling: Throughout the process, the system will continuously record logs to track the execution of each step. If an exception occurs at any stage: a) immediately record detailed error information, including error type, occurrence time, related data, etc.; b) try to execute predefined error recovery processes, such as retrying operations or rolling back to a safe state; c) if the error cannot be recovered, mark the snapshot as a failed state and notify the coordination node (such as Leader).

[0166] Snapshot abort mechanism: If the operation needs to be aborted during the snapshot creation process (for example, due to a system-level error), the system will: a) stop all ongoing snapshot-related operations; b) clean up the created temporary data and file system reference points; c) restore the container to a normal non-snapshot state; d) delete the related snapshot metadata.

[0167] The two-stage snapshot protocol system is able to create consistent snapshots in complex distributed environments while minimizing the impact on normal business operations. This approach not only ensures data consistency, but also improves the efficiency of the entire process through technologies such as asynchronous processing and incremental logging.

[0168] Fig.10 A timing diagram of a process of creating a snapshot provided in this embodiment, such as Fig.10 As shown in the figure, the snapshot coordinator sends a snapshot preparation message to each node. After receiving the snapshot preparation message, each node can enable the copy-on-write mechanism of the container, set the write operation hook of the file system, initialize the incremental log of the file system, and create a file system reference point; after each node is ready, the snapshot coordinator updates the snapshot status to "ready completed". The snapshot coordinator sends a snapshot creation instruction to each node, and starts to create a file system snapshot and start recording incremental changes. After the creation is completed, the application incremental log is started asynchronously for each node: the container performs a write operation on the file system, and the file system can trigger the write operation hook of each node (container), perform copy-on-write for each node (container) through the file system, record the incremental log, and apply the incremental changes to the snapshot. Until all nodes (containers) complete the snapshot.

[0169] It should be noted that at the beginning of snapshot creation, all files and directories are marked as CoW. When a write operation occurs: you can check whether the target file is marked as CoW; if so, copy the original file to the new location before performing the write operation; and update the file system metadata so that the original snapshot reference points to the copied file, and perform the write operation at the original location.

[0170] Optionally, compressing metadata of the snapshot includes:

[0171] 1) Starting from the current character, search for the longest string that matches the forward buffer in the sliding window;

[0172] In this embodiment, the metadata of the snapshot can be compressed using the LZ77 compression algorithm. LZ77 is a commonly used lossless data compression algorithm that reduces the amount of data by finding repeated patterns in the data. At the beginning of compression, a sliding window and a forward buffer can be set, wherein the sizes of the sliding window and the forward buffer can be preset.

[0173] Continuing with the above description, after the sliding window and the forward buffer are set, the longest string matching the forward buffer can be searched in the sliding window starting from the current character.

[0174] 2) If there is a matching longest string, record the matching position and length, as well as the next unmatched character after the matching longest string;

[0175] Specifically, if there is a longest character string matching the forward buffer in the sliding window, the position of the longest character string and the next unmatched character after the matching longest character string may be recorded.

[0176] 3) If there is no matching longest string, record the current character;

[0177] Specifically, if there is no longest character string matching the forward buffer in the sliding window, the current character may be recorded.

[0178] 4) Move the sliding window and the forward buffer and perform the above-mentioned search and record operations until the metadata compression of the snapshot is completed.

[0179] Specifically, after the current character recording is completed, the sliding window and the forward buffer can be moved, and the above-mentioned search and recording operations cannot be repeated until the metadata compression of the snapshot is completed.

[0180] It should be explained that the compressed metadata can be stored in a reliable distributed storage system. This storage system should have high availability and data redundancy to ensure the security of snapshot metadata. Storage operations usually set a retry mechanism. If the storage fails, the system will try to re-store multiple times (e.g., 3 times).

[0181] Exemplarily, the compression process is as follows: set a sliding window, the window size is usually several thousand bytes (such as 4096 bytes); set a forward buffer, the size is usually tens of bytes (such as 15 bytes); starting from the current position, search the sliding window for the longest string that matches the forward buffer; if a match is found, record the position and length of the match, as well as the next unmatched character; if no match is found, directly record the current character; move the sliding window and the forward buffer, and repeat the above process until all data is processed. By compressing the data, the data transmission efficiency is improved.

[0182] Beneficial effects of the technical solution provided by this embodiment:

[0183] 1) This technical solution introduces a distributed consensus algorithm (Raft) to ensure that the system can continue to run during snapshot and rollback operations, achieving zero downtime. This means that during snapshot and rollback operations, the system service will not be interrupted and the business can continue, greatly improving the high availability of the system.

[0184] 2) This technical solution uses a distributed consistency algorithm to ensure that the data of all nodes remain consistent when performing snapshot and rollback operations in a multi-node environment. By electing a leader node and coordinating all follower nodes, it ensures that all nodes create or roll back snapshots at the same time, avoiding data inconsistency problems and improving the stability and reliability of the system.

[0185] 3) This technical solution simplifies the configuration and management of snapshots and rollbacks through intelligent automated management tools, reduces human intervention, and reduces operation and maintenance costs. Automated tools can automatically coordinate and execute snapshot and rollback operations, greatly improving the maintainability and operation and maintenance efficiency of the system.

[0186] 4) The two-stage snapshot creation protocol and the two-stage snapshot rollback protocol adopted by this technical solution provide fine-grained state control. For example, in the snapshot creation process, through the two stages of "prepare" and "create", it is ensured that all nodes create snapshots at the same time and can handle possible errors in a timely manner. Similarly, in the rollback process, through the two stages of "prepare rollback" and "commit rollback", a more reliable rollback operation is achieved. This fine-grained control greatly improves the reliability and success rate of the operation.

[0187] 5) This technical solution can adapt to different network environments and system loads by dynamically adjusting the heartbeat interval and election timeout. For example, when the system load is high, the heartbeat interval will be appropriately increased to reduce unnecessary network communication; when the network delay is large, the election timeout will be increased accordingly to avoid unnecessary re-election. This adaptive mechanism significantly improves the stability and efficiency of the system in different environments.

[0188] 6) The Copy-on-Write (CoW) mechanism adopted by the present invention creates a new copy only when the data changes, greatly reducing storage overhead. The incremental log technology allows the system to record only the changed data, rather than a complete copy of the data, further reducing storage requirements and network transmission volume. The asynchronous application of incremental logs minimizes the impact on the normal operation of the system. The combined application of these technologies enables the present invention to significantly reduce performance overhead while ensuring data consistency.

[0189] The technical solution provided in this embodiment solves the downtime, consistency and complexity problems existing in the prior art. By introducing the Raft distributed consistency algorithm, the high availability and consistency of the system during snapshot and rollback operations are ensured, and management and operation are simplified through intelligent automation tools.

[0190] Fig.11 is a schematic diagram of the structure of a snapshot processing device provided by an embodiment of the present disclosure; Fig.11 As shown, the device includes: a snapshot preparation message sending module 310, a snapshot creation message sending module 320, and a global snapshot information generating module 330.

[0191] The snapshot preparation message sending module 310 is used to send a snapshot preparation message to each follower node;

[0192] The snapshot creation message sending module 320 is used to send a snapshot creation message to each follower node when each follower node is ready, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node;

[0193] The global snapshot information generating module 330 is used to decompress the metadata sent by each of the follower nodes and generate global snapshot information according to the decompressed metadata.

[0194] The embodiment of the present disclosure provides a snapshot processing device to ensure that each node creates a snapshot at the same time, thereby improving the reliability of the generated global snapshot.

[0195] Furthermore, the device also includes:

[0196] A confirmation module, configured to confirm that each follower node is ready when receiving ready messages from a specified number of follower nodes;

[0197] The termination indication sending module is used to send indication information of terminating the snapshot to each of the follower nodes when the number of the received ready messages does not reach the specified number.

[0198] Furthermore, the global snapshot information generating module 330 is further used to:

[0199] If metadata of all follower nodes are received within a set period, then the metadata of all follower nodes are decompressed;

[0200] If metadata of some follower nodes are received within the set time period, indication information of terminating the snapshot or the received metadata of some follower nodes is sent to each of the follower nodes.

[0201] Furthermore, the device also includes:

[0202] A metadata checking module, used to check whether the decompressed metadata contains necessary fields, and whether the necessary fields are consistent with information in the snapshot creation message;

[0203] The necessary fields include a snapshot identifier, a follow-up node identifier, and a snapshot timestamp.

[0204] Furthermore, the global snapshot information includes:

[0205] Snapshot creation time; snapshot version generated based on snapshot creation time; follower node information for creating snapshot; data size of snapshot of follower node; total data volume of each follower node; container list; data location; verification information for verifying data integrity.

[0206] Furthermore, the device also includes:

[0207] A rollback instruction sending module, used to send a rollback instruction to each follower node, wherein the rollback instruction is used to instruct each follower node to perform a rollback operation;

[0208] The rollback operation includes:

[0209] Read and verify global snapshot information, and copy the global snapshot information to a local cache;

[0210] Create a container instance for each container, and transfer the metadata corresponding to each container from the original container to the created corresponding container;

[0211] Verify the metadata of each created container, and complete the rollback if the verification passes.

[0212] The snapshot processing device provided in the embodiments of the present disclosure can execute the snapshot processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0213] Fig.12 is a schematic diagram of the structure of another snapshot processing device provided by an embodiment of the present disclosure; Fig.12 As shown, the device includes: a preparation operation execution module 410 and a snapshot creation module 420.

[0214] The preparation operation execution module 410 is used to perform the preparation operation before creating the snapshot when receiving the snapshot preparation message, and send a preparation message to the leader node after being ready;

[0215] The snapshot creation module 420 is used to create a snapshot at a specified time when a snapshot creation message is received, and compress metadata of the snapshot and send it to the leader node.

[0216] The embodiment of the present disclosure provides a snapshot processing device to ensure that each node creates a snapshot at the same specified time, thereby improving the reliability of the generated global snapshot.

[0217] Furthermore, the preparation operation execution module 410 is also used for:

[0218] Creating initial snapshot metadata for each container, the snapshot metadata including a snapshot identifier, a container identifier, and a snapshot timestamp;

[0219] Enable the copy-on-write mechanism for each container to maintain data status;

[0220] A write operation hook is set for each container, where the write operation hook is used to capture all write operations after the snapshot is created;

[0221] Create an initial incremental log for each container, which is used to record all data change information after the snapshot is created;

[0222] Create a reference point for the file system as the starting point for the snapshot;

[0223] Update the snapshot status to Ready.

[0224] Furthermore, the snapshot creation module 420 is further configured to:

[0225] Serializing a list of containers, configuration information, network settings, and / or storage volume information of the follower node running at a specified time to create a snapshot;

[0226] Apply all data change information and all write operations after the snapshot is created to the created snapshot;

[0227] Update the snapshot status to creation completed.

[0228] Furthermore, the snapshot creation module 420 is further configured to:

[0229] Starting from the current character, search for the longest string that matches the forward buffer in the sliding window;

[0230] If there is a longest matching string, the matching position and length, as well as the next unmatched character after the longest matching string, are recorded;

[0231] If there is no matching longest string, record the current character;

[0232] The sliding window and the forward buffer are moved and the above-mentioned searching and recording operations are performed until the metadata compression of the snapshot is completed.

[0233] Furthermore, the device also includes:

[0234] The rollback execution module is used to perform the following rollback operations upon receiving a rollback instruction:

[0235] Read and verify global snapshot information, and copy the global snapshot information to a local cache;

[0236] Create a container instance for each container, and transfer the metadata corresponding to each container from the original container to the created corresponding container;

[0237] Verify the metadata of each created container, and complete the rollback if the verification passes.

[0238] Furthermore, the global snapshot information includes:

[0239] Snapshot creation time; snapshot version generated based on snapshot creation time; follower node information for creating snapshot; data size of snapshot of follower node; total data volume of each follower node; container list; data location; verification information for verifying data integrity.

[0240] The snapshot processing device provided in the embodiments of the present disclosure can execute the snapshot processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0241] Fig.13A schematic diagram of a leader node 50 that can be used to implement embodiments of the present disclosure is shown. Leader nodes are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present disclosure described and / or claimed herein.

[0242] like Fig.13 As shown, the leader node 50 includes at least one processor 55, and a memory connected to the at least one processor 51 in communication, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 to the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the leader node 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other through a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0243] A number of components in the leader node 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the leader node 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0244] The processor 51 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microprocessor, etc. The processor 51 performs the various methods and processes described above, such as the snapshot processing method.

[0245] In some embodiments, the snapshot processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on the leader node 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the snapshot processing method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the snapshot processing method in any other appropriate manner (e.g., by means of firmware).

[0246] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0247] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0248] To provide interaction with a user, the systems and techniques described herein may be implemented on a leader node having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the leader node. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0249] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0250] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0251] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the embodiments of the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions of the embodiments of the present disclosure can be achieved, and this document does not limit this.

[0252] The above specific implementations do not constitute a limitation on the protection scope of the embodiments of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the embodiments of the present disclosure shall be included in the protection scope of the embodiments of the present disclosure.

[0253] Fig.14 A schematic diagram of a follower node 60 that can be used to implement an embodiment of the present disclosure is shown. The follower node is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present disclosure described and / or claimed herein.

[0254] like Fig.14 As shown, the follower node 60 includes at least one processor 61, and a memory connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 to the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the follower node 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other through a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0255] A number of components in the follower node 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the follower node 60 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0256] The processor 61 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microprocessor, etc. The processor 61 performs the various methods and processes described above, such as the snapshot processing method.

[0257] In some embodiments, the snapshot processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on the follower node 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the snapshot processing method described above may be performed. Alternatively, in other embodiments, the processor 61 may be configured to perform the snapshot processing method in any other appropriate manner (e.g., by means of firmware).

[0258] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0259] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0260] In the context of the disclosed embodiments, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0261] To provide interaction with a user, the systems and techniques described herein may be implemented on a follower node having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the follower node. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0262] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0263] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0264] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the embodiments of the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions of the embodiments of the present disclosure can be achieved, and this document does not limit this.

[0265] The above specific implementations do not constitute a limitation on the protection scope of the embodiments of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the embodiments of the present disclosure shall be included in the protection scope of the embodiments of the present disclosure.

[0266] The embodiments of the present disclosure also provide a computer program product, including a computer program and / or instructions, which, when executed by a processor, implements the snapshot processing method provided in any embodiment of the present application.

[0267] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to write computer program codes for performing the operation of the disclosed embodiments, the programming languages ​​including object-oriented programming languages, such as Java, Smalltalk, C++, and also including conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0268] Note that the above are only preferred embodiments of the embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the embodiments of the present disclosure are not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the embodiments of the present disclosure. Therefore, although the embodiments of the present disclosure are described in more detail through the above embodiments, the embodiments of the present disclosure are not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the embodiments of the present disclosure, and the scope of the embodiments of the present disclosure is determined by the scope of the attached claims.

[0269] The disclosed embodiment also provides a snapshot creation system. The system includes: a leader node provided by the embodiment and a follower node provided by the embodiment; the leader node is elected based on a distributed consistency algorithm. The leader node can be used to send a snapshot preparation message to each follower node; and when each follower node is ready, it sends a snapshot creation message to each follower node, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node; decompresses the metadata sent by each follower node, and generates global snapshot information according to the decompressed metadata. The follower node can be used to perform preparation operations before creating a snapshot when receiving a snapshot preparation message, and send a ready message to the leader node when ready; when receiving a snapshot creation message, it creates a snapshot at a specified time, and compresses the metadata of the snapshot and sends it to the leader node. By electing a leader node and coordinating all follower nodes, it is ensured that all nodes create or roll back snapshots at the same time, avoiding data inconsistency problems and improving the stability and reliability of the system. At the same time, the leader node can send rollback instructions to each follower node. After receiving the rollback instructions, the follower node executes the rollback operation, which simplifies the configuration and management of snapshots and rollbacks, reduces human intervention, reduces operation and maintenance costs, automatically coordinates and executes snapshot and rollback operations, and greatly improves the system's maintainability and operation and maintenance efficiency.

[0270] It should be explained that the process of electing a leader node through the distributed consensus algorithm (Raft) can be as follows: the system will dynamically adjust the heartbeat interval according to the current system load. When the system load is high, the heartbeat interval will be appropriately increased to reduce unnecessary network communication; when the system load is low, the heartbeat interval will be reduced to improve the system response speed. The heartbeat interval will be adjusted between the preset minimum value (such as 50 milliseconds) and the maximum value (such as 500 milliseconds). At the same time, the election timeout can also be dynamically adjusted according to the network conditions. When the network delay is large, the election timeout will be increased accordingly to avoid unnecessary re-election; when the network condition is good, the election timeout will be reduced to speed up the leader election process. The election timeout will also be adjusted between the preset minimum value (such as 150 milliseconds) and the maximum value (such as 1000 milliseconds). The system load and network delay are monitored in real time, and data is collected within a certain time window (such as a load sampling window of 60 seconds and a network delay sampling window of 30 seconds) to dynamically adjust these parameters. Through this adaptive mechanism, the system can maintain stability and efficiency in different operating environments.

[0271] Continuing with the above description, during the leader node election process, each node maintains its own status information, including the current term, who to vote for, the current leader node identifier, etc. When a node initiates an election, it increases its term number and sends a message to other nodes requesting a vote. After receiving the voting request, other nodes will decide whether to vote in support according to certain rules (such as a larger term number, log update, etc.). If a node receives the support of the majority of nodes, it will become the new leader node. After the election is successful, the new leader node will immediately send a heartbeat message to all other nodes to establish its leadership position. After receiving the heartbeat message from the new leader node, other nodes will update their own status and recognize the status of the new leader node.

[0272] Specifically, Fig.15 FIG. 7 is a schematic diagram of a snapshot creation system 70 provided in an embodiment of the present disclosure. Fig.15 As shown, the system includes: a leader node 701 and a follower node 702.

[0273] A snapshot creation system provided by an embodiment of the present disclosure can be used to execute the snapshot creation method provided by any of the above embodiments, and has corresponding functions and beneficial effects.

Claims

1. A snapshot processing method, characterized in that: Applied to a leader node, the leader node is elected based on a distributed consistency algorithm; the method comprises: Send a snapshot preparation message to each follower node; When each of the follower nodes is ready, sending a snapshot creation message to each of the follower nodes, so that each of the follower nodes creates a corresponding snapshot at the same time, and each of the follower nodes compresses metadata of the corresponding snapshot and sends it to the leader node; The metadata sent by each of the follower nodes is decompressed, and global snapshot information is generated according to the decompressed metadata.

2. The method according to claim 1, characterized in that Also includes: Upon receiving ready messages from a specified number of follower nodes, confirming that each of the follower nodes is ready; When the number of the received ready messages does not reach the specified number, indication information of terminating the snapshot is sent to each of the follower nodes.

3. The method according to claim 1, characterized in that Decompress the metadata sent by each follower node, including: If metadata of all follower nodes are received within a set period, then the metadata of all follower nodes are decompressed; If metadata of some follower nodes are received within the set time period, indication information of terminating the snapshot or the received metadata of some follower nodes is sent to each of the follower nodes.

4. The method according to claim 1, characterized in that Before generating global snapshot information based on the decompressed metadata, it also includes: Checking whether the decompressed metadata contains necessary fields, and whether the necessary fields are consistent with information in the snapshot creation message; The necessary fields include a snapshot identifier, a follow-up node identifier, and a snapshot timestamp.

5. The method according to claim 1, characterized in that The global snapshot information includes: Snapshot creation time; snapshot version generated based on snapshot creation time; follower node information for creating snapshot; data size of snapshot of follower node; total data volume of each follower node; container list; data location; verification information for verifying data integrity.

6. The method according to claim 1, characterized in that Also includes: Sending a rollback instruction to each of the follower nodes, wherein the rollback instruction is used to instruct each of the follower nodes to perform a rollback operation; The rollback operation includes: Read and verify global snapshot information, and copy the global snapshot information to a local cache; Create a container instance for each container, and transfer the metadata corresponding to each container from the original container to the created corresponding container; Verify the metadata of each created container, and complete the rollback if the verification passes.

7. A snapshot processing method, characterized in that: Applied to a follower node, the method comprises: When receiving the snapshot preparation message, perform the preparation operation before creating the snapshot, and send a ready message to the leader node after being ready; When a snapshot creation message is received, a snapshot is created at a specified time, and metadata of the snapshot is compressed and sent to the leader node.

8. The method according to claim 7, characterized in that Perform the following operations before creating a snapshot: Creating initial snapshot metadata for each container, the snapshot metadata including a snapshot identifier, a container identifier, and a snapshot timestamp; Enable the copy-on-write mechanism for each container to maintain data status; A write operation hook is set for each container, where the write operation hook is used to capture all write operations after the snapshot is created; Create an initial incremental log for each container, which is used to record all data change information after the snapshot is created; Create a reference point for the file system as the starting point for the snapshot; Update the snapshot status to Ready.

9. The method according to claim 7, characterized in that: Create a snapshot at a specified time, including: Serializing a list of containers, configuration information, network settings, and / or storage volume information of the follower node running at a specified time to create a snapshot; Apply all data change information and all write operations after the snapshot is created to the created snapshot; Update the snapshot status to creation completed.

10. The method according to claim 7, characterized in that Compressing the metadata of the snapshot includes: Starting from the current character, search for the longest string that matches the forward buffer in the sliding window; If there is a longest matching string, the matching position and length, as well as the next unmatched character after the longest matching string, are recorded; If there is no matching longest string, record the current character; The sliding window and the forward buffer are moved and the above-mentioned searching and recording operations are performed until the metadata compression of the snapshot is completed.

11. The method according to claim 7, characterized in that Also includes: When a rollback instruction is received, the following rollback operations are performed: Read and verify global snapshot information, and copy the global snapshot information to a local cache; Create a container instance for each container, and transfer the metadata corresponding to each container from the original container to the created corresponding container; Verify the metadata of each created container, and complete the rollback if the verification passes.

12. The method according to claim 11, characterized in that The global snapshot information includes: Snapshot creation time; snapshot version generated based on snapshot creation time; follower node information for creating snapshot; data size of snapshot of follower node; total data volume of each follower node; container list; data location; verification information for verifying data integrity.

13. A snapshot processing device, characterized in that: include: A snapshot preparation message sending module is used to send a snapshot preparation message to each follower node; A snapshot creation message sending module, used for sending a snapshot creation message to each follower node when each follower node is ready, so that each follower node creates a corresponding snapshot at the same time, and each follower node compresses the metadata of the corresponding snapshot and sends it to the leader node; The global snapshot information generation module is used to decompress the metadata sent by each follower node and generate global snapshot information according to the decompressed metadata.

14. A snapshot processing device, characterized in that: include: A preparation operation execution module, used for executing the preparation operation before creating the snapshot when receiving the snapshot preparation message, and sending a preparation message to the leader node after being ready; The snapshot creation module is used to create a snapshot at a specified time when a snapshot creation message is received, and compress the metadata of the snapshot and send it to the leader node.

15. A leader node, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the snapshot processing method according to any one of claims 1 to 6 is implemented.

16. A follower node, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the snapshot processing method according to any one of claims 7 to 12 is implemented.

17. A snapshot creation system, characterized in that: comprising a leader node as claimed in claim 15, and at least two follower nodes as claimed in claim 16; The leader node is elected based on a distributed consistency algorithm.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the snapshot processing method as described in any one of claims 1 to 12 is implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the snapshot processing method according to any one of claims 1 to 12.

Images