Data disaster recovery method and device in cluster system, electronic equipment and storage medium

Abstract

The invention discloses a data disaster recovery method and device in a cluster system, electronic equipment and a storage medium, the cluster system comprises a management platform and a plurality of cluster nodes, disks in all the nodes are divided into a plurality of storage volume groups according to disk types, and each storage volume group correspondingly creates one or more logic storage volumes. Two logic storage volumes distributed at different nodes and corresponding to the same storage volume group can form a main and standby logic storage volume mapping pair, and the method is applied to a management platform and comprises the following steps: determining an abnormal cluster node and an abnormal main logic storage volume thereof in a cluster system; determining a standby logic storage volume corresponding to the abnormal main logic storage volume and a target cluster node to which the standby logic storage volume belongs according to the main and standby logic storage volume mapping relationship; and releasing the mounting relationship between the application container corresponding to the abnormal main logic storage volume and the abnormal cluster node, and establishing the mounting relationship between the application container and the target cluster node. According to the scheme, high availability of the host disk in the cluster system can be realized so as to effectively deal with the host exception problem.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a data disaster recovery method in a cluster system, and also to a data disaster recovery device, an electronic device, and a computer-readable storage medium in the cluster system. Background Art

[0002] With the continuous development of cloud-native technology, Kubernetes (K8s) has become an indisputable fact as the de facto standard in the field of container orchestration. Cloud-native applications deployed on K8s need to mount container storage volumes to persist business data. Compared with mounting external shared storage, using host disks can provide higher read and write performance and is the first choice for persistent data of some IO-sensitive businesses. However, when the host crashes or the disk is damaged, the data cannot be automatically migrated, resulting in the business being unable to continue to provide external services. In order to ensure high availability, the business itself needs to have high availability capabilities, which cannot take advantage of K8s' natural support for high availability, bringing additional workload to business vendors.

[0003] Therefore, how to achieve high availability of host disks in a K8s cluster to effectively deal with host abnormalities in the K8s cluster is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0004] The purpose of the present application is to provide a data disaster recovery method in a cluster system, which can achieve high availability of host disks in a K8s cluster to effectively deal with host abnormalities in the K8s cluster; another purpose of the present application is to provide a data disaster recovery device, electronic device, computer-readable storage medium and computer program product in a cluster system, all of which have the above-mentioned beneficial effects.

[0005] In a first aspect, the present application provides a data disaster recovery method in a cluster system, the cluster system comprising a management platform and a plurality of cluster nodes, the disks in all the cluster nodes being divided into a plurality of storage volume groups according to the disk type, one or more logical storage volumes being created corresponding to each storage volume group, two logical storage volumes distributed in different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume mapping pair, the method being applied to the management platform, comprising:

[0006] Performing a health status check on each of the cluster nodes in the cluster system according to the data disaster recovery instruction, and determining an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node;

[0007] Determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary and backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs;

[0008] The mounting relationship between the application container corresponding to the abnormal primary logical storage volume and the abnormal cluster node is released, and a mounting relationship between the application container and the target cluster node is established.

[0009] Optionally, the data disaster recovery method in the cluster system further includes:

[0010] For each normal cluster node in the cluster system, determining each primary logical storage volume in the normal cluster node;

[0011] Determine the backup logical storage volume corresponding to the primary logical storage volume according to the primary and backup logical storage volume mapping relationship;

[0012] The running data of the application container corresponding to the primary logical storage volume is stored in the primary logical storage volume, and is backed up to the corresponding backup logical storage volume.

[0013] Optionally, storing the running data of the application container corresponding to the primary logical storage volume in the primary logical storage volume, and backing up the running data in the corresponding backup logical storage volume includes:

[0014] Determine the file directory of the primary logical storage volume and the file directory of the corresponding backup logical storage volume;

[0015] Storing the running data of the application container corresponding to the primary logical storage volume in a file directory of the primary logical storage volume;

[0016] The running data is incrementally synchronized to the file directory of the standby logical storage volume.

[0017] Optionally, after performing a health status check on each cluster node in the cluster system according to the data disaster recovery instruction and determining an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node, the method further includes:

[0018] Record the node information of the abnormal cluster node to the abnormal node list; wherein the node information includes a combination of one or more of the node name, node ID, and node IP of the abnormal cluster node;

[0019] Accordingly, the data disaster recovery method in the cluster system further includes:

[0020] Performing a health status check on each of the cluster nodes in the cluster system according to the data disaster recovery instruction to determine a normal cluster node;

[0021] When the normal cluster node hits the abnormal node list, determining each primary logical storage volume in the normal cluster node as a restored primary logical storage volume;

[0022] Determine the original backup logical storage volume corresponding to the restored primary logical storage volume according to the primary and backup logical storage volume mapping relationship;

[0023] The running data of the application container corresponding to the original standby logical storage volume is reversely synchronized to the restored primary logical storage volume.

[0024] Optionally, the reverse synchronizing the running data of the application container corresponding to the original standby logical storage volume to the restored logical storage volume includes:

[0025] Determine the abnormality detection time of the normal cluster nodes that hit the abnormal node list;

[0026] Determine the most recent normal detection time based on the abnormal detection time and the detection time interval;

[0027] The running data of the application container corresponding to the original standby logical storage volume is reversely synchronized to the restored primary logical storage volume according to the most recently detected normal time.

[0028] Optionally, the data disaster recovery method in the cluster system further includes:

[0029] Creating a logical storage volume on each of the cluster nodes by using a preset LVM command, and establishing each of the primary and standby logical storage volume mapping pairs according to the cluster nodes and corresponding storage volume groups to which the logical storage volumes are distributed;

[0030] Generating the primary and backup logical storage volume mapping relationship based on each of the primary and backup logical storage volume mapping pairs;

[0031] The primary logical storage volume in the primary-backup logical storage volume mapping pair is mapped to the local storage volume of the corresponding application container, so as to mount the application container to the cluster node to which the primary logical storage volume belongs.

[0032] Optionally, performing a health status check on each cluster node in the cluster system according to the data disaster recovery instruction to determine an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node includes:

[0033] Triggering the data disaster recovery instruction according to a preset timing detection mechanism;

[0034] A health status check is performed on each cluster node in the cluster system according to the data disaster recovery instruction to determine the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node.

[0035] In a second aspect, the present application further discloses a data disaster recovery device in a cluster system, the cluster system comprising a management platform and a plurality of cluster nodes, the disks in all the cluster nodes being divided into a plurality of storage volume groups according to the disk type, one or more logical storage volumes being created corresponding to each storage volume group, two logical storage volumes distributed in different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume pair, the device being applied to the management platform, comprising:

[0036] A first determination module is used to perform a health status check on each cluster node in the cluster system according to a data disaster recovery instruction, and determine an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node;

[0037] A second determination module is used to determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary and backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs;

[0038] The mounting switching module is used to release the mounting relationship between the application container corresponding to the abnormal primary logical storage volume and the abnormal cluster node, and to establish a mounting relationship between the application container and the target cluster node.

[0039] In a third aspect, the present application further discloses an electronic device, comprising:

[0040] Memory for storing computer programs;

[0041] The processor is used to implement the steps of any one of the data disaster recovery methods in the cluster system described above when executing the computer program.

[0042] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data disaster recovery method in any cluster system as described above are implemented.

[0043] In a fifth aspect, the present invention further discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any of the data disaster recovery methods in the cluster system as described above.

[0044] The present application provides a data disaster recovery method in a cluster system, the cluster system includes a management platform and multiple cluster nodes, the disks in all the cluster nodes are divided into multiple storage volume groups according to disk type, one or more logical storage volumes are created corresponding to each of the storage volume groups, two logical storage volumes distributed in different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume mapping pair, the method is applied to the management platform, including: performing a health status check on each of the cluster nodes in the cluster system according to a data disaster recovery instruction, determining an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node; determining a backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary-backup logical storage volume mapping relationship, and determining a target cluster node to which the backup logical storage volume belongs; unmounting an application container corresponding to the abnormal primary logical storage volume from the abnormal cluster node, and establishing a mounting relationship between the application container and the target cluster node.

[0045] Applying the technical solution provided by the present application, for the disks on all cluster nodes in the cluster system, multiple storage volume groups are obtained by dividing them according to the disk type, that is, one disk type corresponds to one storage volume group, and all disks in the same storage volume group have the same disk type and may be distributed on different cluster nodes, wherein one or more logical storage volumes are created corresponding to each storage volume group, then, two logical storage volumes distributed on different cluster nodes and corresponding to the same storage volume group can form a primary and backup logical storage volume mapping pair, and the primary and backup logical storage volume mapping pair can be used to realize data disaster recovery in the cluster system. On this basis, the management platform can determine the abnormal cluster node and the abnormal primary logical storage volume in the cluster system through health status check, and can determine the backup logical storage volume corresponding to the abnormal primary logical storage volume and its target cluster node according to the primary and backup logical storage volume mapping relationship, thereby, the mounting relationship between the application container corresponding to the abnormal primary logical storage volume and the abnormal cluster node can be released, and the mounting relationship between the application container and the target cluster node can be established, thereby realizing a data disaster recovery solution based on the disks of each host in the cluster, and it can be seen that the present technical solution realizes the high availability of the host disks in the cluster system to effectively deal with sudden host abnormalities in the cluster system.

[0046] In one embodiment of the present application, when it is detected that an abnormal cluster node has been restored to a normal cluster node, the primary logical storage volume on the restored normal cluster node can be reversely synchronized to restore the application container running data that was missed during the abnormal period. In this process, the time when the abnormal cluster node was detected to be abnormal (detection abnormality time) and the time interval of the detection cycle can be referred to to determine the time when it was last detected to be in a normal state (last detected normal time). Thus, the application container running data after the last detected normal time can be reversely synchronized to the primary logical storage volume in the restored normal cluster node, so as to effectively reduce the amount of data to be synchronized and achieve fast and efficient reverse data synchronization.

[0047] The data disaster recovery device, electronic device, computer-readable storage medium and computer program product in the cluster system provided by the present application also have the above-mentioned technical effects, which will not be described in detail in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0049] Figure 1 A schematic diagram of the structure of a cluster system provided in this application;

[0050] Figure 2 A schematic diagram of a data disaster recovery method in a cluster system provided by the present application;

[0051] Figure 3 A schematic diagram of a data disaster recovery method in a cluster system provided by the present application;

[0052] Figure 4 A schematic diagram of the structure of a data disaster recovery device in a cluster system provided by the present application;

[0053] Figure 5 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0054] The core of the present application is to provide a data disaster recovery method in a cluster system, which can achieve high availability of host disks in the K8s cluster to effectively deal with host abnormalities in the K8s cluster; another core of the present application is to provide a data disaster recovery device, electronic device, computer-readable storage medium and computer program product in a cluster system, all of which have the above-mentioned beneficial effects.

[0055] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0056] An embodiment of the present application provides a data disaster recovery method in a cluster system.

[0057] First, please refer to Figure 1 , Figure 1 A schematic diagram of the structure of a cluster system provided by the present application, the cluster system includes a management platform and multiple cluster nodes, in the cluster system, the disks in all cluster nodes can be divided into multiple storage volume groups according to the disk type, that is, one disk type corresponds to one storage volume group, the disks in the same storage volume group have the same disk type, and may be distributed on different cluster nodes, wherein each storage volume group corresponds to one or more logical storage volumes, then, two logical storage volumes distributed on different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume mapping pair, and the primary-backup logical storage volume mapping pair can be used to implement data disaster recovery in the cluster system. The data disaster recovery method in the cluster system provided in the embodiment of the present application is implemented based on the cluster system.

[0058] For further information, please refer to Figure 2 , Figure 2 A schematic diagram of a data disaster recovery method in a cluster system provided by the present application. The data disaster recovery method in the cluster system is applied to Figure 1 The management platform shown may include the following S101 to S103.

[0059] S101: Perform a health status check on each cluster node in the cluster system according to a data disaster recovery instruction to determine an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node.

[0060] This step aims to determine the abnormal cluster nodes in the cluster system and the abnormal main logical storage volumes therein, which can be achieved by performing a health status check on each cluster node in the cluster system. Specifically, when a data disaster recovery instruction is received, a health status check can be performed on each cluster node in the cluster system, so that the abnormal cluster nodes in the cluster system and the abnormal main logical storage volumes in the abnormal cluster nodes can be determined. It can be understood that the abnormal main logical storage volumes in the abnormal cluster nodes may be all the main logical storage volumes in the abnormal cluster nodes, or may be part of the main logical storage volumes in the abnormal cluster nodes. Of course, the abnormal cluster nodes may also include backup logical storage volumes.

[0061] In one embodiment of the present application, performing a health status check on each cluster node in the cluster system according to the data disaster recovery instruction to determine an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node may include:

[0062] Trigger data disaster recovery instructions according to the preset timing detection mechanism;

[0063] A health status check is performed on each cluster node in the cluster system according to the data disaster recovery instruction to determine the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node.

[0064] The embodiment of the present application provides a method for triggering a data disaster recovery instruction, that is, an automatic response method based on a preset timing detection mechanism, that is, data disaster recovery detection can be performed on a cluster system at a regular time. The specific value of the timing time interval does not affect the implementation of the present technical solution, and can be set by the technician according to the actual situation, and the present application does not limit this.

[0065] The above-mentioned checking the health status of each cluster node in the cluster system according to the data disaster recovery instruction to determine the abnormal cluster node may include:

[0066] Perform health status check on each cluster node in the cluster system according to the data disaster recovery instructions;

[0067] When it is detected that a disk of a cluster node is not found and / or the disk name is changed and / or the disk reading and writing is abnormal, the cluster node is determined to be an abnormal cluster node.

[0068] The embodiment of the present application provides an implementation method for determining whether a cluster node is an abnormal cluster node. Specifically, for each cluster node in the cluster system, when performing a health status check on it, it can be checked whether a combination of one or more of a disk not found event, a disk name change event, and a disk read / write abnormal event occurs therein. If so, the cluster node can be regarded as an abnormal cluster node, otherwise it can be regarded as a normal cluster node.

[0069] S102: Determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary-backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs.

[0070] This step aims to determine the backup logical storage volume corresponding to the abnormal primary logical storage volume and the cluster node to which it belongs. As described above, two logical storage volumes distributed on different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume mapping pair. Then, all primary-backup logical storage volume mapping pairs in the cluster system can generate a primary-backup logical storage volume mapping relationship. Thus, the backup logical storage volume corresponding to the abnormal primary logical storage volume can be determined based on the primary-backup logical storage volume mapping relationship, and the cluster node to which it belongs, that is, the target cluster node, can be further determined. It should be noted that the primary logical storage volume maintains a data synchronization relationship with its corresponding backup logical storage volume to facilitate the implementation of data disaster recovery services in the cluster system.

[0071] In one embodiment of the present application, the data disaster recovery method in the cluster system may further include:

[0072] Create logical storage volumes on each cluster node through the preset LVM (Logical Volume Management) command, and establish primary and backup logical storage volume mapping pairs according to the cluster nodes and corresponding storage volume groups where the logical storage volumes are distributed;

[0073] Generate a primary and backup logical storage volume mapping relationship based on each primary and backup logical storage volume mapping pair;

[0074] The primary logical storage volume in the primary-backup logical storage volume mapping pair is mapped to the local storage volume of the corresponding application container, so as to mount the application container to the cluster node to which the primary logical storage volume belongs.

[0075] The embodiment of the present application provides a pre-configuration scheme in a cluster system, which is used to realize the generation of the mapping relationship between the primary and backup logical storage volumes and the initial mounting of each application container. First, the preset LVM command can be used to realize the creation of logical storage volumes on each cluster node, so that each primary and backup logical storage volume mapping pair can be established on each cluster node according to the cluster node where each logical storage volume is distributed and its corresponding storage volume group to generate the primary and backup logical storage volume mapping relationship; further, each application container in the cluster system corresponds to a primary and backup logical storage volume mapping pair, and during the initial configuration, each application container is mounted on the cluster node to which the primary logical storage volume in the corresponding primary and backup logical storage volume mapping pair belongs, and the backup logical storage volume only serves as a backup service.

[0076] S103: Unmounting the application container corresponding to the abnormal primary logical storage volume from the abnormal cluster node, and establishing a mounting relationship between the application container and the target cluster node.

[0077] This step aims to achieve the mounting switch of the application container. Specifically, since the abnormal cluster node and the abnormal main logical storage volume therein can no longer continue to provide services for the application container mounted thereon, the mounting relationship between the application container corresponding to the abnormal main logical storage volume and the abnormal cluster node can be directly released; on this basis, in order to ensure the normal operation of the original business of the application container, the mounting relationship between the application container and the target cluster node can continue to be established, and the target cluster node and the backup logical storage volume corresponding to the abnormal main logical storage volume therein continue to provide services for the application container.

[0078] It can be seen that the data disaster recovery method in the cluster system provided by the embodiment of the present application, for the disks on all cluster nodes in the cluster system, is divided according to the disk type to obtain multiple storage volume groups, that is, one disk type corresponds to one storage volume group, and all disks in the same storage volume group have the same disk type and may be distributed on different cluster nodes, wherein each storage volume group corresponds to one or more logical storage volumes created, then, two logical storage volumes distributed on different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume mapping pair, and the primary-backup logical storage volume mapping pair can be used to realize data disaster recovery in the cluster system. On this basis, the management platform can determine the abnormal cluster nodes and the abnormal main logical storage volumes in the cluster system through health status checks, and can determine the backup logical storage volume corresponding to the abnormal main logical storage volume and the target cluster node to which it belongs based on the mapping relationship between the primary and backup logical storage volumes. In this way, the mounting relationship between the application container corresponding to the abnormal main logical storage volume and the abnormal cluster node can be released, and the mounting relationship between the application container and the target cluster node can be established, thereby realizing a data disaster recovery plan based on the host disks in the cluster. It can be seen that this technical solution realizes the high availability of the host disks in the cluster system, so as to effectively deal with sudden host abnormalities in the cluster system.

[0079] Based on the above embodiments:

[0080] In one embodiment of the present application, the data disaster recovery method in the cluster system may further include:

[0081] For each normal cluster node in the cluster system, determining each primary logical storage volume in the normal cluster node;

[0082] Determine the backup logical storage volume corresponding to the primary logical storage volume according to the primary and backup logical storage volume mapping relationship;

[0083] The running data of the application container corresponding to the primary logical storage volume is stored in the primary logical storage volume, and backed up to the corresponding backup logical storage volume.

[0084] The data disaster recovery method in the cluster system provided by the embodiment of the present application can realize data synchronization between the primary and backup logical storage volumes, so as to realize data disaster recovery service. Specifically, for each normal cluster node in the cluster system, the backup logical storage volume corresponding to each primary logical storage volume can be determined according to the primary and backup logical storage volume mapping relationship, thereby, the running data of the application container corresponding to the primary logical storage volume can be stored in the primary logical storage volume and backed up to the corresponding backup logical storage volume at the same time, so as to realize disaster recovery backup of the running data of the application container.

[0085] The step of storing the running data of the application container corresponding to the primary logical storage volume in the primary logical storage volume and backing it up in the corresponding backup logical storage volume may include:

[0086] Determine the file directory of the primary logical storage volume and the file directory of the corresponding backup logical storage volume;

[0087] Storing the running data of the application container corresponding to the primary logical storage volume in the file directory of the primary logical storage volume;

[0088] Incrementally synchronize running data to the file directory of the backup logical storage volume.

[0089] The embodiment of the present application provides a method for implementing data synchronization between primary and backup logical storage volumes, namely, a data synchronization method based on incremental backup. It can be understood that incremental backup and only backing up changed data information without repeatedly backing up original data information can effectively reduce the amount of backup data and greatly improve the data synchronization efficiency between primary and backup logical storage volumes.

[0090] In one embodiment of the present application, after performing a health status check on each cluster node in the cluster system according to the data disaster recovery instruction and determining the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node, the method may further include: recording the node information of the abnormal cluster node into an abnormal node list; wherein the node information includes a combination of one or more of the node name, node ID, and node IP of the abnormal cluster node;

[0091] Accordingly, the data disaster recovery method in the cluster system may further include:

[0092] Perform health status checks on each cluster node in the cluster system according to the data disaster recovery instructions to determine normal cluster nodes;

[0093] When a normal cluster node hits the abnormal node list, each primary logical storage volume in the normal cluster node is determined to be a restored primary logical storage volume;

[0094] Determine the original backup logical storage volume corresponding to the restored primary logical storage volume according to the primary and backup logical storage volume mapping relationship;

[0095] Reversely synchronize the running data of the application container corresponding to the original backup logical storage volume to the restored primary logical storage volume.

[0096] The data disaster recovery method in the cluster system provided by the embodiment of the present application can further realize the reverse data synchronization operation of the cluster node after abnormal recovery. Specifically, after the abnormal cluster node in the cluster system is monitored, the relevant information of the abnormal cluster node can be recorded in the abnormal node list; on this basis, if in a subsequent round of cluster node health check, it is found that the abnormal cluster node originally recorded in the abnormal node list is restored to a normal cluster node, then each of the primary logical storage volumes therein can be regarded as the restored primary logical storage volume, thereby, the original backup logical storage volume corresponding to the restored primary logical storage volume can be determined according to the mapping relationship between the primary and backup logical storage volumes, and the original backup logical storage volume can be used to perform reverse data synchronization on the corresponding restored primary logical storage volume pair, so as to continue to provide disaster recovery services for the current application container.

[0097] The reverse synchronization of the running data of the application container corresponding to the original standby logical storage volume to the restored logical storage volume may include:

[0098] Determine the abnormality detection time of the normal cluster nodes that hit the abnormal node list;

[0099] Determine the most recent normal detection time based on the abnormal detection time and the detection time interval;

[0100] The running data of the application container corresponding to the original standby logical storage volume is reversely synchronized to the restored primary logical storage volume according to the most recent normal detection time.

[0101] The embodiment of the present application provides a method for implementing reverse synchronization based on time information. Specifically, the abnormal cluster list can also record the time when the cluster node is detected as an abnormal cluster node, that is, the above-mentioned abnormal detection time. Then, the time when the cluster node was last detected as a normal cluster node can be obtained by subtracting the detection time interval from the abnormal detection time, that is, the above-mentioned most recent normal detection time. It can be imagined that the application container operation data generated before the most recent normal detection time has been stored and backed up normally, and the application container operation data generated after the most recent normal detection time may not be stored normally in part or in whole. Therefore, the application container operation data generated after the most recent normal detection time can be reversely synchronized to the main logical storage volume after recovery.

[0102] It can be seen that in an embodiment of the present application, when it is detected that an abnormal cluster node has been restored to a normal cluster node, the main logical storage volume on the restored normal cluster node can be reversely synchronized to restore the application container running data that was missed during the abnormal period. In this process, the time when the abnormal cluster node was detected to be abnormal (detection abnormality time) and the time interval of the detection cycle can be referred to to determine the time when it was last detected to be in a normal state (the last detected normal time). As a result, the application container running data after the last detected normal time can be reversely synchronized to the main logical storage volume in the restored normal cluster node, so as to effectively reduce the amount of data to be synchronized and achieve fast and efficient reverse data synchronization.

[0103] Based on the above embodiments, the embodiment of the present application takes the K8s cluster (an infrastructure platform that provides container orchestration capabilities and deploys cloud-native microservice applications) as an example to provide another data disaster recovery method in a cluster system.

[0104] First, the basic idea of ​​the data disaster recovery method in the cluster system provided by the embodiment of the present application mainly includes:

[0105] 1. When deploying a K8s cluster, group the data disks on the cluster nodes into VGs (volume groups) according to the disk type. The VG names on different cluster nodes of the same disk type should be consistent to form a mutual backup relationship.

[0106] 2. When creating a local storage volume for an application container, specify a VG to create primary and backup LVs (storage logical volumes) on different cluster nodes through CSI (Container Storage Interface) and associate them with the local storage volume of the application container for business data persistence.

[0107] 3. When the business application is mounted to the local storage volume, the application container is scheduled to the cluster node where the primary LV is located, and the corresponding business data (application container running data) is persisted to the file system directory corresponding to the primary LV, and is backed up to the file system directory corresponding to the standby LV in an incremental manner to complete the backup of business data on different cluster nodes.

[0108] 4. When a cluster node crashes or the disk is damaged, swap the roles of the primary and backup LVs, use the K8s scheduling mechanism to reschedule the application container to the cluster node where the original backup LV is located, read and write business data in the file system directory corresponding to the LV, and after the original cluster node or disk is repaired, reverse synchronize the business data.

[0109] For further information, please refer to Figure 3 , Figure 3 The present application provides a schematic diagram of a data disaster recovery method in a cluster system, wherein the management platform of the cluster system mainly includes:

[0110] 1. Data synchronization device: realizes real-time synchronization of data under specific directories of different cluster nodes and completes mutual backup of business data on different cluster nodes.

[0111] 2. CSI: Implement the CSI interface of the K8s standard, use LVM (Logical Volume Management) to create primary and backup LVs on the VGs of different cluster nodes, bind them to the local storage volumes of the application container, and mount them to the application container for data persistence.

[0112] 3. Host status check device: Checks the health status of cluster nodes and disks, and generates node abnormality events when abnormalities occur.

[0113] 4. Automatic application migration device: automatically senses node abnormal events, switches the roles of the primary and backup LVs, and uses K8s's high availability mechanism to reschedule application containers to normal cluster nodes.

[0114] based on Figure 3 As shown, the implementation process of the data disaster recovery method in the cluster system may include:

[0115] 1. Users deploy Kubernetes clusters. The hosts that serve as K8s cluster nodes have disks. Disks of the same type are grouped into VGs. The names of VGs of the same disk type on all cluster nodes remain consistent.

[0116] 2. Deploy CSI based on LVM to implement local storage volume management. CSI is used to create primary and backup LVs according to VG. The specific creation method is: create LVs on different cluster nodes through LVM commands, respectively as primary and backup LVs, and save the association between the two; when creating the local storage volume of the application container, save the primary and backup LV information to the local storage volume of the application container to achieve the association between the primary and backup LVs and the local storage volume. After the local storage volume is mounted to the application container, CSI initializes the LV and creates a file system, mapping it to the inside of the application container. The specific implementation method is: using the K8s mechanism, write the information of the local storage volume to the orchestration file of the container application, CSI monitors the changes in the orchestration file information, finds the device name of the primary LV mapped to the cluster node operating system, and initializes the file system in the general way of creating a file system for the operating system device and mounts it to a specific file directory. Finally, through the application container mapping, the file directory on the cluster node is mapped to the inside of the application container. For the backup LV, the file system is created in the same way and mapped to the LV with the same path.

[0117] 3. Deploy a data synchronization device to synchronize business data from the file directory corresponding to the primary LV to the file directory corresponding to the backup LV in an incremental manner according to the association relationship between the primary and backup LVs to achieve data backup.

[0118] 4. Deploy the host health check device to regularly check the health status of cluster nodes and their disks. Cluster node abnormality means that the node cannot be accessed normally, and disk abnormality mainly means that the operating system cannot find the disk, the disk device name changes, and the disk cannot read and write data normally. When an abnormality is detected, the abnormality is uniformly processed as an event that the file directory mounted by the LV cannot be read and written normally, and the health check device can report the abnormal information through events.

[0119] 5. Deploy an automatic application migration device. When it senses abnormal information about a cluster node, it switches the roles of the primary and backup LVs, and changes the cluster node where the local storage volume of the application container can be scheduled to the cluster node where the current backup LV is located. K8s reschedules the application container to the node to complete application reconstruction, thereby achieving high availability of applications with mounted local storage volumes. At the same time, the data synchronization device automatically senses the changes in the roles of the primary and backup LVs and stops data synchronization. When the host health check device reports that the file directory of the original LV has resumed normal reading and writing time, the data synchronization device starts reverse data synchronization and synchronizes data to the file directory of the original LV at a regular interval.

[0120] It can be seen that the data disaster recovery method in the cluster system provided by the embodiment of the present application has the following advantages:

[0121] 1. The K8s cluster uses the data disk of the cluster node to provide storage volumes mounted to the application container in the form of LVM for data persistence, avoiding the performance loss of remote data transmission and meeting the performance requirements of high IO and low latency of the business;

[0122] 2. Through the incremental synchronization of business data across cluster nodes, local data can be backed up across nodes without occupying too much bandwidth, thus avoiding the problem of important business loss caused by single point failure;

[0123] 3. After a cluster node or disk fails, the application container mounted with the local storage volume can be automatically migrated to a normal cluster node to continue to provide external services, achieving the same effect as mounting external shared storage. When the application container uses local storage, it can also recover from cross-node failures. The business itself no longer needs to achieve high availability. Developers only need to focus on the development of their own business logic, which effectively improves R&D efficiency.

[0124] 4. No manual data backup and manual business migration are required, which reduces the operation and maintenance costs of applications mounted on local storage volumes.

[0125] An embodiment of the present application provides a data disaster recovery device in a cluster system.

[0126] Please refer to Figure 4 , Figure 4A structural schematic diagram of a data disaster recovery device in a cluster system provided by the present application, wherein the cluster system includes a management platform and multiple cluster nodes, the disks in all cluster nodes are divided into multiple storage volume groups according to the disk type, each storage volume group is corresponding to one or more logical storage volumes, and two logical storage volumes distributed in different cluster nodes and corresponding to the same storage volume group can form a primary and backup logical storage volume mapping pair, and the data disaster recovery device in the cluster system is applied to Figure 1 The management platform shown may include:

[0127] The first determination module 1 is used to perform a health status check on each cluster node in the cluster system according to the data disaster recovery instruction, and determine the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node;

[0128] The second determination module 2 is used to determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary and backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs;

[0129] The mounting switching module 3 is used to release the mounting relationship between the application container corresponding to the abnormal primary logical storage volume and the abnormal cluster node, and to establish a mounting relationship between the application container and the target cluster node.

[0130] It can be seen that the data disaster recovery device in the cluster system provided by the embodiment of the present application divides the disks on all cluster nodes in the cluster system into multiple storage volume groups according to the disk type, that is, one disk type corresponds to one storage volume group, and all disks in the same storage volume group have the same disk type and may be distributed on different cluster nodes, wherein each storage volume group corresponds to one or more logical storage volumes created, then, two logical storage volumes distributed on different cluster nodes and corresponding to the same storage volume group can form a primary-backup logical storage volume mapping pair, and the primary-backup logical storage volume mapping pair can be used to realize data disaster recovery in the cluster system. On this basis, the management platform can determine the abnormal cluster nodes and the abnormal main logical storage volumes in the cluster system through health status checks, and can determine the backup logical storage volume corresponding to the abnormal main logical storage volume and the target cluster node to which it belongs based on the mapping relationship between the primary and backup logical storage volumes. In this way, the mounting relationship between the application container corresponding to the abnormal main logical storage volume and the abnormal cluster node can be released, and the mounting relationship between the application container and the target cluster node can be established, thereby realizing a data disaster recovery plan based on the host disks in the cluster. It can be seen that this technical solution realizes the high availability of the host disks in the cluster system, so as to effectively deal with sudden host abnormalities in the cluster system.

[0131] In one embodiment of the present application, the data disaster recovery device in the cluster system may also include a backup module, which is used to determine each primary logical storage volume in the normal cluster node for each normal cluster node in the cluster system; determine the backup logical storage volume corresponding to the primary logical storage volume according to the primary-backup logical storage volume mapping relationship; store the running data of the application container corresponding to the primary logical storage volume to the primary logical storage volume, and back up it to the corresponding backup logical storage volume.

[0132] In one embodiment of the present application, the above-mentioned backup module can be specifically used to determine the file directory of the primary logical storage volume and the file directory of the corresponding backup logical storage volume; store the running data of the application container corresponding to the primary logical storage volume in the file directory of the primary logical storage volume; and synchronize the running data increments to the file directory of the backup logical storage volume.

[0133] In one embodiment of the present application, the data disaster recovery device in the cluster system may further include:

[0134] A recording module is used to perform a health status check on each cluster node in the cluster system according to the data disaster recovery instruction, and after determining the abnormal cluster node and the abnormal main logical storage volume in the abnormal cluster node, record the node information of the abnormal cluster node into the abnormal node list; wherein the node information includes a combination of one or more of the node name, node ID, and node IP of the abnormal cluster node;

[0135] The recovery module is used to check the health status of each cluster node in the cluster system according to the data disaster recovery instruction and determine the normal cluster node; when the normal cluster node hits the abnormal node list, determine each logical storage volume in the normal cluster node as the restored logical storage volume; determine the original backup logical storage volume corresponding to the restored logical storage volume according to the mapping relationship between the primary and backup logical storage volumes; and reversely synchronize the running data of the application container corresponding to the original backup logical storage volume to the restored logical storage volume.

[0136] In one embodiment of the present application, the above-mentioned recovery module can be specifically used to determine the detection anomaly time of the normal cluster node that hits the abnormal node list; determine the most recent detection normal time based on the detection anomaly time and the detection time interval; and reversely synchronize the running data of the application container corresponding to the original backup logical storage volume to the restored logical storage volume according to the most recent detection normal time.

[0137] In one embodiment of the present application, the data disaster recovery device in the cluster system may further include:

[0138] The configuration module is used to create a logical storage volume on each cluster node through a preset LVM command, and establish each primary and backup logical storage volume mapping pair according to the cluster nodes where the logical storage volume is distributed and the corresponding storage volume group; generate a primary and backup logical storage volume mapping relationship based on each primary and backup logical storage volume mapping pair; map the primary logical storage volume in the primary and backup logical storage volume mapping pair to the local storage volume of the corresponding application container, so as to mount the application container to the cluster node to which the primary logical storage volume belongs.

[0139] In one embodiment of the present application, the above-mentioned first determination module 1 can be specifically used to trigger a data disaster recovery instruction according to a preset timing detection mechanism; perform a health status check on each cluster node in the cluster system according to the data disaster recovery instruction, and determine the abnormal cluster node and the abnormal main logical storage volume in the abnormal cluster node.

[0140] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0141] An embodiment of the present application provides an electronic device.

[0142] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application, and the electronic device may include:

[0143] A memory 11, used for storing computer programs;

[0144] The processor 10 can implement the steps of any one of the above-mentioned data disaster recovery methods in the cluster system when executing a computer program.

[0145] like Figure 5 , which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0146] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0147] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the data disaster recovery method in a cluster system.

[0148] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0149] Perform a health status check on each cluster node in the cluster system according to the data disaster recovery instruction, and determine the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node;

[0150] Determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary and backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs;

[0151] The application container corresponding to the abnormal primary logical storage volume is unmounted from the abnormal cluster node, and a mounting relationship between the application container and the target cluster node is established.

[0152] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0153] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0154] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0155] Of course, it should be noted that Figure 5 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 5 More or fewer components than shown, or combinations of certain components.

[0156] An embodiment of the present application provides a computer-readable storage medium.

[0157] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data disaster recovery methods in a cluster system can be implemented.

[0158] Among them, the computer-readable storage medium can be any available medium that can be stored in a computer or a data storage device such as a server or data center that includes one or more available media integrated. For example, it can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape, etc.), an optical medium (such as a DVD) or a semiconductor medium (such as a solid-state hard disk) and other media that can store computer program codes.

[0159] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0160] An embodiment of the present application provides a computer program product.

[0161] The computer program product provided in the embodiment of the present application includes a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of any of the above-mentioned data disaster recovery methods in a cluster system can be implemented.

[0162] Specifically, in the above embodiments, all or part of them can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of them can be implemented in the form of a computer program product.

[0163] Among them, the computer program product may include one or more computer programs / instructions, and when the computer program / instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application may be generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0164] For an introduction to the computer program product provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0165] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0166] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0167] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0168] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A data disaster recovery method in a cluster system, characterized in that: The cluster system includes a management platform and multiple cluster nodes. The disks in all the cluster nodes are divided into multiple storage volume groups according to disk types. One or more logical storage volumes are created corresponding to each storage volume group. Two logical storage volumes distributed in different cluster nodes and corresponding to the same storage volume group can form a primary and backup logical storage volume mapping pair. The method is applied to the management platform and includes: Performing a health status check on each of the cluster nodes in the cluster system according to the data disaster recovery instruction, and determining an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node; Determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary and backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs; The mounting relationship between the application container corresponding to the abnormal primary logical storage volume and the abnormal cluster node is released, and a mounting relationship between the application container and the target cluster node is established.

2. The data disaster recovery method in a cluster system according to claim 1, characterized in that: Also includes: For each normal cluster node in the cluster system, determining each primary logical storage volume in the normal cluster node; Determine the backup logical storage volume corresponding to the primary logical storage volume according to the primary and backup logical storage volume mapping relationship; The running data of the application container corresponding to the primary logical storage volume is stored in the primary logical storage volume, and is backed up to the corresponding backup logical storage volume.

3. The data disaster recovery method in a cluster system according to claim 2, characterized in that: The storing the running data of the application container corresponding to the primary logical storage volume to the primary logical storage volume and backing up to the corresponding backup logical storage volume includes: Determine the file directory of the primary logical storage volume and the file directory of the corresponding backup logical storage volume; Storing the running data of the application container corresponding to the primary logical storage volume in a file directory of the primary logical storage volume; The running data is incrementally synchronized to the file directory of the standby logical storage volume.

4. The data disaster recovery method in a cluster system according to claim 1, characterized in that: After checking the health status of each cluster node in the cluster system according to the data disaster recovery instruction and determining the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node, the method further includes: Record the node information of the abnormal cluster node to the abnormal node list; wherein the node information includes a combination of one or more of the node name, node ID, and node IP of the abnormal cluster node; Accordingly, the method further comprises: Performing a health status check on each of the cluster nodes in the cluster system according to the data disaster recovery instruction to determine a normal cluster node; When the normal cluster node hits the abnormal node list, determining each primary logical storage volume in the normal cluster node as a restored primary logical storage volume; Determine the original backup logical storage volume corresponding to the restored primary logical storage volume according to the primary and backup logical storage volume mapping relationship; The running data of the application container corresponding to the original standby logical storage volume is reversely synchronized to the restored primary logical storage volume.

5. The data disaster recovery method in a cluster system according to claim 4, characterized in that: The reverse synchronization of the running data of the application container corresponding to the original standby logical storage volume to the restored primary logical storage volume includes: Determine the abnormality detection time of the normal cluster nodes that hit the abnormal node list; Determine the most recent normal detection time based on the abnormal detection time and the detection time interval; The running data of the application container corresponding to the original standby logical storage volume is reversely synchronized to the restored primary logical storage volume according to the most recently detected normal time.

6. The data disaster recovery method in a cluster system according to claim 1, characterized in that: Also includes: Creating a logical storage volume on each of the cluster nodes by using a preset LVM command, and establishing each of the primary and standby logical storage volume mapping pairs according to the cluster nodes and corresponding storage volume groups to which the logical storage volumes are distributed; Generating the primary and backup logical storage volume mapping relationship based on each of the primary and backup logical storage volume mapping pairs; The primary logical storage volume in the primary-backup logical storage volume mapping pair is mapped to the local storage volume of the corresponding application container, so as to mount the application container to the cluster node to which the primary logical storage volume belongs.

7. The data disaster recovery method in a cluster system according to claim 1, characterized in that: The step of performing a health status check on each cluster node in the cluster system according to the data disaster recovery instruction to determine an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node includes: Triggering the data disaster recovery instruction according to a preset timing detection mechanism; A health status check is performed on each cluster node in the cluster system according to the data disaster recovery instruction to determine the abnormal cluster node and the abnormal primary logical storage volume in the abnormal cluster node.

8. A data disaster recovery device in a cluster system, characterized in that: The cluster system includes a management platform and multiple cluster nodes. The disks in all the cluster nodes are divided into multiple storage volume groups according to disk types. One or more logical storage volumes are created corresponding to each storage volume group. Two logical storage volumes distributed in different cluster nodes and corresponding to the same storage volume group can form a primary and backup logical storage volume pair. The device is applied to the management platform and includes: A first determination module is used to perform a health status check on each cluster node in the cluster system according to a data disaster recovery instruction, and determine an abnormal cluster node and an abnormal primary logical storage volume in the abnormal cluster node; A second determination module is used to determine the backup logical storage volume corresponding to the abnormal primary logical storage volume according to the primary and backup logical storage volume mapping relationship, and determine the target cluster node to which the backup logical storage volume belongs; The mounting switching module is used to release the mounting relationship between the application container corresponding to the abnormal primary logical storage volume and the abnormal cluster node, and to establish a mounting relationship between the application container and the target cluster node.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the data disaster recovery method in a cluster system as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data disaster recovery method in a cluster system according to any one of claims 1 to 7 are implemented.

Images