Electronic equipment and data backup method based on data storage service

Abstract

The invention provides an electronic device and a data backup method based on a data storage service, and mainly relates to the technical field of computers, the electronic device comprises a processor configured to obtain a data synchronization rule, the data synchronization rule comprising a first cluster address, a second cluster address and a synchronization data identifier; acquiring to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address; and synchronizing the to-be-synchronized data to a second cluster corresponding to the second cluster address. Only the to-be-synchronized data in the specified range is synchronized from the first cluster to the second cluster, and the to-be-synchronized data is less than total data, so that the CTED data is synchronized to the standby cluster in real time under the conditions of insufficient disk IO and network delay, the method has the characteristic of quick switching, and meanwhile, the data synchronization in the specified range is realized, and the data synchronization efficiency is improved. And synchronization delay caused by full-amount synchronization is avoided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and more specifically, to an electronic device and a data backup method based on a data storage service. Background Art

[0002] With the continuous growth of business volume, in the era of big data explosion, the requirements for high availability of systems are constantly increasing. As the industry's leading distributed storage system, ETCD (a distributed key-value storage system) often appears as a coordinator in various large-scale distributed system frameworks. ETCD is a distributed key-value storage developed by CoreOS based on the Raft protocol, which can be used for service discovery, shared configuration, and consistency assurance (such as database master election, distributed locks, etc.).

[0003] In order to ensure that everything is safe, you need to establish a backup ETCD cluster to ensure that if the main cluster fails as a whole, you can quickly switch to the backup cluster and use the backup cluster to provide services. ETCD is an extremely important service in the K8S cluster, storing all the data information of the cluster. Similarly, the backup cluster stores ETCD data that is completely consistent with the main cluster. If the ETCD data is lost, it will affect the recovery of the cluster data.

[0004] In the case of large amounts of data, due to disk input / output (IO) and network latency, it is impossible to ensure that CTED data is synchronized to the backup cluster in real time. The backup cluster is prone to incomplete ETCD data, affecting cluster recovery. Summary of the invention

[0005] The exemplary implementation of the present application provides an electronic device and a data backup method based on a data storage service, which realizes real-time synchronization of ETCD data and has a fast switching feature, while realizing data synchronization within a specified range and avoiding synchronization delays caused by full synchronization.

[0006] The technical solutions provided by the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of the present application provides an electronic device, comprising: a processor, configured to: obtain a data synchronization rule, wherein the data synchronization rule comprises a first cluster address, a second cluster address, and a synchronization data identifier; obtain the data to be synchronized corresponding to the synchronization data identifier from a first cluster via the first cluster address; and synchronize the data to be synchronized to a second cluster corresponding to the second cluster address.

[0008] In a second aspect, an embodiment of the present application provides a data backup method based on a data storage service, which is applied to an electronic device, and the method includes: obtaining a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier; obtaining the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the first cluster address; and synchronizing the data to be synchronized to the second cluster corresponding to the second cluster address.

[0009] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computing device, the computing device implements the data backup method based on the data storage service provided in any embodiment of the second aspect.

[0010] In a fourth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, the computer implements the data backup method based on the data storage service provided by any embodiment of the second aspect.

[0011] It can be seen from the above technical solutions that the embodiment of the present application provides an electronic device and a data backup method based on a data storage service, the electronic device comprising: a processor, configured to: obtain data synchronization rules, wherein the data synchronization rules include a first cluster address, a second cluster address, and a synchronization data identifier; obtain the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the first cluster address; synchronize the data to be synchronized to the second cluster corresponding to the second cluster address. Since only the data to be synchronized within the specified range is synchronized from the first cluster to the second cluster, the data to be synchronized is less than the full amount of data, in the case of insufficient disk IO and network delay, the CTED data can be synchronized to the backup cluster in real time, with a fast switching feature, and data synchronization within the specified range can be achieved at the same time, avoiding synchronization delays caused by full synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0013] Figure 1 A schematic diagram of the structure of a Raft protocol provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram of a process for node election in an ETCD cluster provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of an ETCD cluster architecture provided in an embodiment of the present application;

[0016] Figure 4 A flowchart of a data backup method based on a data storage service provided in an embodiment of the present application;

[0017] Figure 5 A flowchart of another data backup method based on a data storage service provided in an embodiment of the present application;

[0018] Figure 6 A flowchart of another data backup method based on a data storage service provided in an embodiment of the present application;

[0019] Figure 7 A flowchart of another data backup method based on a data storage service provided in an embodiment of the present application;

[0020] Figure 8 A schematic diagram of the structure of a data synchronization system based on an ETCD cluster provided in an embodiment of the present application;

[0021] Fig. 9 An interactive schematic diagram of a data synchronization method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0023] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0024] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0025] With the continuous growth of business volume, in the era of big data explosion, the requirements for high availability of systems are constantly increasing. As the industry's leading distributed storage system, ETCD (a distributed key-value storage system) often appears as a coordinator in various large-scale distributed system frameworks.

[0026] In a cloud platform cluster environment, such as a container cloud cluster based on Kubernetes (K8S for short), ETCD, as the backend storage software, stores all data during the operation of the K8S cluster. Its importance is self-evident. ETCD uses the Raft protocol to maintain the consistency of the status of each node in the cluster. Simply put, the ETCD cluster is a distributed system consisting of multiple nodes communicating with each other to form an overall external service. Each node stores complete data, and the Raft protocol is used to ensure that the data maintained by each node is consistent. Each ETCD node maintains a state machine, and there is at most one valid master node at any time. The master node handles all write operations from the client, and the Raft protocol is used to ensure that the changes to the state machine caused by the write operation will be reliably synchronized to other nodes.

[0027] The core of ETCD's working principle lies in the Raft protocol and the watch mechanism. Figure 1 As shown in Figure 1. The Raft protocol is a protocol used to maintain data consistency among a group of service nodes. This group of service nodes forms a cluster, and there is a master node to provide external services. When the cluster is initialized or the master node fails, it faces an election problem. Each node in the cluster is in one of the three roles of Leader, Follower, or Candidate at any time. Figure 2 , as shown in Figure 3, the election features are as follows: when the cluster is initialized, each node is a Follower; there is at most one valid master node in the cluster, which synchronizes data with other nodes through heartbeats; when a Follower node does not receive a heartbeat from the master node within a certain period of time, it will change its role to a Candidate node and initiate an election vote; when more than half of the nodes, including itself, agree, the election is successful, and the Candidate node becomes the Leader node. If the number of votes received is less than half, the election fails, or the election times out. If the master node is not elected in this round, the next round of elections will be held (this happens because multiple nodes are elected at the same time, and all nodes have received more than half of the votes). After the Candidate node receives information from the master node, it will immediately terminate the election process and become a Follower node. In order to avoid falling into a cycle of election failures, the time when each node does not receive a heartbeat to initiate an election is a random value within a certain range, which can avoid two nodes initiating elections at the same time.

[0028] Due to the importance of the ETCD cluster, if a disaster occurs or etcd data is lost, the recovery of the cluster data will be affected. In addition to the regular ETCD backup, an ETCD real-time synchronization mechanism is also required to ensure that in the event of machine failure or power outage that causes etcd data loss, it can quickly switch to the backup cluster. The previous solutions were all based on backup and recovery, and the backup and recovery mechanism took at least minutes to restore the business, which was unacceptable for the production environment.

[0029] In order to ensure that everything is foolproof, it is necessary to establish a standby ETCD cluster to ensure that when the main cluster is down as a whole, it can quickly switch to the standby cluster and provide services. ETCD is an extremely important service in the K8S cluster, storing all the data information of the cluster. Similarly, if a disaster occurs or the data of ETCD is lost, it will affect the recovery of cluster data. At present, the most widely used method is to build a multi-node ETCD cluster and use the Raft protocol to maintain the consistency of the status of each node in the cluster. Simply put, the ETCD cluster is a distributed system, which consists of multiple nodes communicating with each other to form an overall external service. Each node stores complete data, and the Raft protocol is used to ensure that the data maintained by each node is consistent. The current mainstream solution is to back up the ETCD cluster, and then restore it in the standby cluster when needed or use the ETCD official tool make-mirror for synchronization. The problem with using the backup recovery solution is that recovery takes time and it is impossible to quickly switch to the standby cluster. The problem with make-mirror is that the synchronization range cannot be specified. For large data volumes, it is limited by disk IO and network latency, and real-time synchronization cannot be guaranteed. The standby cluster is prone to incomplete ETCD data, which affects cluster recovery.

[0030] The synchronization method using the official tool make-mirror is as follows: insert a data synchronization command into the running code of the cluster. After the code runs to the data synchronization command, the data synchronization command calls the make-mirror tool. The make-mirror tool synchronizes all the data in the source ETCD cluster to the destination ETCD cluster.

[0031] When using the make-mirror tool for data synchronization, you cannot specify the data synchronization range and can only perform full data synchronization. In addition, the make-mirror tool can only synchronize the full data in a source ETCD cluster to a destination ETCD cluster. In the same period, the make-mirror tool can only achieve one-to-one cluster data synchronization. In other words, the make-mirror tool synchronizes the full data in ETCD cluster A to ETCD cluster B. If you want to synchronize the full data in ETCD cluster A to ETCD cluster C, or if you want to synchronize the full data in ETCD cluster D to ETCD cluster E, you must wait until the data synchronization from ETCD cluster A to ETCD cluster B is completed before you can perform the next cluster data synchronization.

[0032] To solve the above technical problems, an embodiment of the present application provides an electronic device and a data backup method based on a data storage service. The technical solution of the present application is described in detail below in conjunction with the accompanying drawings and embodiments.

[0033] In one possible implementation, an embodiment of the present application provides an electronic device, including: a processor, configured to: obtain a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier; obtain the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the first cluster address; and synchronize the data to be synchronized to the second cluster corresponding to the second cluster address.

[0034] In a possible implementation, the processor is configured to: monitor the data range corresponding to the synchronization data identifier in the first cluster; when the data in the data range corresponding to the synchronization data identifier is updated, synchronize the updated incremental data to the second cluster corresponding to the second cluster address.

[0035] In a possible implementation, the processor is configured to: monitor the data range corresponding to the synchronization data identifier in the first cluster through a watch interface.

[0036] In a possible implementation, the processor is configured to: read the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the first cluster address; cache the data to be synchronized in the memory first; and synchronize the data to be synchronized from the memory to the second cluster corresponding to the second cluster address.

[0037] In one possible implementation, the processor is configured to: establish a data synchronization channel between the first cluster and the second cluster based on a data synchronization rule; obtain the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the data synchronization channel; and synchronize the data to be synchronized to the second cluster corresponding to the second cluster address through the data synchronization channel.

[0038] In one possible implementation, the processor is configured to: save data synchronization rules; after the electronic device crashes and restarts, obtain the pre-included data synchronization rules; establish a data synchronization channel between the first cluster and the second cluster based on the data synchronization rules; obtain the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the data synchronization channel; and synchronize the data to be synchronized to the second cluster corresponding to the second cluster address through the data synchronization channel.

[0039] In one possible implementation, the processor is configured to: determine whether certificate authentication is enabled for the second cluster; if certificate authentication is enabled for the second cluster, parse the certificate of the second cluster from the data synchronization rule, and the certificate of the second cluster is used to verify the user identity; if the user identity authentication is passed, synchronize the data to be synchronized to the second cluster corresponding to the second cluster address.

[0040] In a possible implementation, the first cluster is a main ETCD cluster, and the second cluster is a standby ETCD cluster; or, the first cluster is a standby ETCD cluster, and the second cluster is a main ETCD cluster.

[0041] In a possible implementation, the processor is configured to: during the process of connecting the first cluster with the service, if it is determined that the first cluster fails, switch the service to connect with the second cluster; after the failure of the first cluster is recovered, switch the service to connect with the first cluster.

[0042] In a possible implementation, the processor is configured to: obtain updated data in the second cluster within a first time period, wherein the first time period refers to the period during which the second cluster is connected to the service; synchronize the updated data in the second cluster to the first cluster, wherein the updated data is data after the data to be synchronized is updated.

[0043] In a possible implementation manner, the processor is configured to: after the service is connected to the second cluster, control the service to transmit the unsynchronized data to the second cluster.

[0044] In a possible implementation, the present application embodiment provides a data backup method based on a data storage service, such as Figure 4 As shown, the data backup method based on the data storage service provided in the embodiment of the present application mainly includes steps S101 to S103.

[0045] S101. Obtain a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier.

[0046] First, the application scenarios of the data backup method based on the data storage service are briefly introduced. When the ETCD cluster is used as the storage software of the K8S cluster, in the case of the overall downtime of the main ETCD cluster, in order to ensure the normal operation of the K8S cluster, it is necessary to establish a backup ETCD cluster for the main ETCD cluster, so that in the case of the overall downtime of the main ETCD cluster, the backup ETCD cluster can be switched to provide services. At this time, it is necessary to ensure that the data in the main ETCD cluster and the backup ETCD cluster are completely consistent. The commonly used method is to synchronize the data in the main ETCD cluster to the backup ETCD cluster.

[0047] The first cluster is the main ETCD cluster, and the second cluster is the backup ETCD cluster; alternatively, the first cluster is the backup ETCD cluster, and the second cluster is the main ETCD cluster.

[0048] When the first cluster is the main ETCD cluster and the second cluster is the standby ETCD cluster, the data in the main ETCD cluster needs to be synchronized to the standby ETCD cluster. When the first cluster is the standby ETCD cluster, the second cluster is the main ETCD cluster and the data in the standby ETCD cluster needs to be synchronized to the main ETCD cluster. In the embodiment of the present application, the first cluster is the main ETCD cluster and the second cluster is the standby ETCD cluster as an example for introduction.

[0049] The synchronization data identifier refers to an identifier used to indicate the synchronization data range. The synchronization data identifier may be one or more of a folder name, a data prefix, a data address, etc., which is not specifically limited in the present embodiment. The first cluster address is used to access or log in to the first cluster, and the second cluster address is used to access or log in to the second cluster.

[0050] The developer (hereinafter referred to as the user) can set the data synchronization rules through the client, and the user can select the first cluster address, the second cluster address and the synchronization data identifier through the interactive interface in the user terminal. According to the confirmation control in the interactive interface, the data synchronization rules are generated based on the selected first cluster address, the second cluster address and the synchronization data identifier. The client sends the generated data synchronization rules to the processor of the electronic device, so that the processor of the electronic device can receive the data synchronization rules and execute the data backup process based on the data synchronization rules.

[0051] In a specific implementation scenario, the processor of the above-mentioned electronic device can be a service in the K8S platform, which can be called a data synchronization service, or a data backup service. The service is used to realize data synchronization between each ETCD cluster. The K8S platform includes a first ETCD cluster, a second ETCD cluster and the above-mentioned service, and the service can receive the data synchronization rules sent by the client. The service parses the data synchronization rules to obtain the first ETCD cluster address, the second ETCD cluster address and the synchronization data identifier in the data synchronization rules. The service can perform the data backup method based on the data storage service provided in the embodiment of the present application based on the first ETCD cluster address, the second ETCD cluster address and the synchronization data identifier.

[0052] Among them, the above services are written in golang language, which is a statically strongly typed, compiled, concurrent programming language with garbage collection function.

[0053] S102: Acquire to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address.

[0054] The above service can read the data to be synchronized corresponding to the synchronization data identifier from the first ETCD cluster, and synchronize the data to be synchronized to the second ETCD cluster.

[0055] The data to be synchronized can be understood as data that needs to be synchronized from the first cluster to the second cluster, and the data range of the data to be synchronized is determined by the synchronization data identifier. For example, if the synchronization data identifier is / data, it means that the processor only needs to synchronize the data under / data in the first cluster to the second cluster as the data to be synchronized.

[0056] The above service can access the first cluster through the first cluster address. By accessing the first cluster, all the data under / data in the first cluster is used as data to be synchronized.

[0057] If the service cannot access the first cluster normally through the first cluster address after receiving the data synchronization rule, the service performs polling access to the first cluster. In other words, the service starts to periodically access the first cluster until the service can access the first cluster normally, and then obtains the data to be synchronized corresponding to the synchronization data identifier from the first cluster. If the service still cannot access the first cluster after the preset time, a prompt message is generated. The prompt message is used to provide cluster maintenance personnel with the information that the service is currently unable to obtain the data to be synchronized.

[0058] S103: Synchronize the data to be synchronized to the second cluster corresponding to the second cluster address.

[0059] In a possible implementation, the data to be synchronized corresponding to the synchronization data identifier is read from the first cluster through the first cluster address; the data to be synchronized is first cached in the memory; and the data to be synchronized is synchronized from the memory to the second cluster corresponding to the second cluster address.

[0060] Specifically, the data to be synchronized corresponding to the synchronization data identifier is read from the first cluster through the first cluster address, the data to be synchronized is first cached in the memory, and then the data to be synchronized in the memory is gradually written to the second cluster.

[0061] When it is determined that the data to be synchronized exists in the first cluster, the service underlying framework automatically allocates a piece of memory and caches the data to be synchronized in the memory. Then the above service accesses the second cluster according to the second cluster address. If the second cluster can be correctly accessed and the second cluster can receive data normally, the data to be synchronized cached in the memory will be synchronized to the second cluster. After the data to be synchronized is sent to the second cluster, the service underlying framework automatically reclaims the memory allocated to the data to be synchronized.

[0062] If the service cannot access the second cluster normally, that is, the connection between the service and the second cluster is disconnected, the service caches the data to be synchronized in the memory and starts polling the second cluster. In other words, the service starts to access the second cluster periodically until the service can access the second cluster normally, and then synchronizes the data to be synchronized in the service memory to the second cluster.

[0063] If the service can access the second cluster normally, but the second cluster cannot receive data due to its own reasons, the service caches the data to be synchronized in the memory and starts polling the second cluster. In other words, the service starts to periodically access the second cluster until the second cluster can receive data normally, and then synchronizes the data to be synchronized in the service memory to the second cluster.

[0064] In another implementation scenario, if the service still cannot access the second cluster or the second cluster still cannot receive data after a preset period of time, a prompt message is generated to inform cluster maintenance personnel that the service is currently unable to synchronize data to the second cluster.

[0065] The embodiment of the present application provides an electronic device and a data backup method based on a data storage service, including: obtaining a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier; obtaining the data to be synchronized corresponding to the synchronization data identifier from the first cluster through the first cluster address; synchronizing the data to be synchronized to the second cluster corresponding to the second cluster address. Since only the data to be synchronized within the specified range is synchronized from the first cluster to the second cluster, the data to be synchronized is less than the full amount of data. In the case of insufficient disk IO and network delay, the CTED data can be synchronized to the backup cluster in real time, with a fast switching feature, and data synchronization within the specified range can be achieved at the same time, avoiding synchronization delays caused by full synchronization.

[0066] The K8S platform includes multiple ETCD clusters, and obtaining data synchronization rules includes: obtaining multiple data synchronization rules, each data synchronization rule includes its corresponding first ETCD cluster address, second ETCD cluster address and synchronization data identifier, and each data synchronization rule corresponds to a data synchronization channel, which is used to obtain the data to be synchronized corresponding to the synchronization data identifier from the corresponding first cluster and synchronize it to the second cluster. In other words, each data synchronization rule constructs a data synchronization channel. The above service can establish a data synchronization channel between multiple clusters and synchronize data between multiple clusters at the same time.

[0067] In a specific implementation scenario, data synchronization rule 1 is obtained, and data synchronization rule 1 includes the address of the first ETCD cluster A, the address of the second ETCD cluster A', and the synchronization data identifier. Data synchronization channel 1 is established between the first ETCD cluster A and the second ETCD cluster A', and the data synchronization channel 1 is used for data synchronization between the first ETCD cluster A and the second ETCD cluster A'. Data synchronization rule 2 is obtained, and data synchronization rule 2 includes the address of the first ETCD cluster B, the address of the second ETCD cluster B', and the synchronization data identifier. Data synchronization channel 2 is established between the first ETCD cluster B and the second ETCD cluster B', and the data synchronization channel 2 is used for data synchronization between the first ETCD cluster B and the second ETCD cluster B'. ..., and so on, data synchronization rule N is obtained, and data synchronization rule N includes the address of the first ETCD cluster C, the address of the second ETCD cluster C', and the synchronization data identifier. Data synchronization channel N is established between the first ETCD cluster C and the second ETCD cluster C', and the data synchronization channel N is used for data synchronization between the first ETCD cluster C and the second ETCD cluster C'. The number of the above data synchronization channels is the same as the number of data synchronization rules.

[0068] Furthermore, multiple data synchronization channels are deployed in one data synchronization service. Figure 8 In the data synchronization controller (DataSync Controller) shown in the figure, each data synchronization channel does not affect each other. In other words, whether the data synchronization channel 1 can realize data synchronization normally or not, it does not affect the data synchronization between the first ETCD cluster B and the second ETCD cluster B' through the data synchronization channel 2.

[0069] In the embodiment of the present application, a corresponding data synchronization channel is established based on the data synchronization rule to achieve data synchronization between each source cluster and the destination cluster without interfering with each other, and data synchronization between multiple clusters can be achieved simultaneously.

[0070] Furthermore, multiple data synchronization rules can be obtained at the same time, and multiple data synchronization channels can be established at the same time. If a new data synchronization rule is received after multiple data synchronization channels are established, a new data synchronization channel can be established according to the new data synchronization rule. The various data synchronization channels do not interfere with each other.

[0071] In one possible implementation, data synchronization rules are saved; after the electronic device crashes and restarts, pre-included data synchronization rules are obtained; a data synchronization channel is established between the first cluster and the second cluster based on the data synchronization rules; data to be synchronized corresponding to the synchronization data identifier is obtained from the first cluster through the data synchronization channel; and the data to be synchronized is synchronized to the second cluster corresponding to the second cluster address through the data synchronization channel.

[0072] The above multiple data synchronization rules are stored in the data synchronization service as a configuration file of the data synchronization service. After the above data synchronization service crashes and restarts, the data synchronization service can read the stored data synchronization rules from the configuration file, establish a data synchronization channel according to the read data synchronization rules, and use the established data synchronization channel to achieve cluster-to-cluster data synchronization.

[0073] In a specific implementation scenario, the data synchronization service can read the stored data synchronization rule 1 from the configuration file, and the data synchronization rule 1 includes the address of the first ETCD cluster A, the address of the second ETCD cluster A', and the synchronization data identifier. A data synchronization channel 1' is established between the first ETCD cluster A and the second ETCD cluster A', and the data synchronization channel 1' is used for data synchronization between the first ETCD cluster A and the second ETCD cluster A'.

[0074] Furthermore, after the data synchronization channel 1' is established, the full amount of data corresponding to the synchronization data identifier is read from the first ETCD cluster A as the data to be synchronized based on the address of the first ETCD cluster A, and all the data to be synchronized are transmitted to the second ETCD cluster A' through the data synchronization channel 1'. After receiving the data to be synchronized, the second ETCD cluster A' deletes the previously received data to be synchronized and saves the data to be synchronized as new synchronization data. Alternatively, after receiving the new data to be synchronized, the second ETCD cluster A' compares the new data to be synchronized with the stored data to be synchronized, and saves the data to be synchronized with a later update time as the final data to be synchronized.

[0075] In another implementation, the data synchronization service records the synchronized data identifiers of each data synchronization channel. After the data synchronization service crashes and restarts, the synchronized data identifiers are compared with the full amount of data to be synchronized corresponding to the synchronization data identifiers obtained from the first ETCD cluster A. If the data to be synchronized obtained from the first ETCD cluster A has updated data, the updated data is synchronized to the second ETCD cluster A' as incremental data. If the data to be synchronized obtained from the first ETCD cluster A does not have updated data, data synchronization to the second ETCD cluster A' is no longer performed, and the data range corresponding to the synchronization data identifier in the first cluster is monitored.

[0076] In another implementation, after the data synchronization service crashes and restarts, the data synchronization service obtains the data to be synchronized (hereinafter referred to as synchronized data) received by the second ETCD cluster from the second ETCD cluster through various data synchronization channels, and compares the synchronized data with the data corresponding to the synchronization data identifier in the first ETCD cluster. If the data corresponding to the synchronization data identifier in the first ETCD cluster has updated incremental data, the data synchronization service obtains the incremental data and transmits it to the second ETCD cluster through the data synchronization channel.

[0077] Specifically, after the data synchronization service crashes and restarts, the data synchronization service obtains the synchronized data received by the second ETCD cluster from the second ETCD cluster A' through the data synchronization channel 1, and compares the synchronized data with the data corresponding to the synchronization data identifier in the first ETCD cluster A. If there is updated incremental data in the first ETCD cluster A, the data synchronization service obtains the incremental data and transmits it to the second ETCD cluster through the data synchronization channel.

[0078] In a possible implementation, the present application embodiment provides another data backup method based on a data storage service, such as Figure 5As shown, the data backup method based on the data storage service provided in the embodiment of the present application mainly includes steps S201 to S205.

[0079] S201. Obtain a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier.

[0080] S202: Acquire the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address.

[0081] S203: Synchronize the data to be synchronized to the second cluster corresponding to the second cluster address.

[0082] The specific process of steps S201 to S203 in the embodiment of the present application is the same as that of steps S101 to S103 in the above embodiment. For details, please refer to the description in the above embodiment, and the embodiment of the present application will not be repeated.

[0083] S204: Monitor the data range corresponding to the synchronization data identifier in the first cluster.

[0084] The data range corresponding to the synchronization data identifier can be understood as the data range indicated by the synchronization data identifier. For example, when the synchronization data identifier is a file name, all data under the file is used as the monitored data range. For another example, when the synchronization data identifier is a data prefix, all data indicated by the data prefix is ​​used as the monitored data range.

[0085] In a possible implementation, the data range corresponding to the synchronization data identifier in the first cluster is monitored through the watch interface. The version number corresponding to the data range to be monitored is informed to the watch interface, and all change events after the version can be monitored through the watch interface, that is, when any data in the data range corresponding to the synchronization data identifier in the first cluster changes, it will be monitored by the watch interface.

[0086] S205: When data within the data range corresponding to the synchronization data identifier is updated, the updated incremental data is synchronized to the second cluster corresponding to the second cluster address.

[0087] When the watch interface detects that the data within the above data range is updated, the updated incremental data is obtained from the first cluster, and the updated incremental data is synchronized to the second cluster. For example: when the watch interface detects that file 1 within the above data range is updated, the updated file 1 is obtained from the first cluster, and the updated file 1 is synchronized to the second cluster.

[0088] Furthermore, the original file 1 in the second cluster is deleted. In other words, the original file 1 in the second cluster is replaced with the updated file 1.

[0089] When the watch interface detects that the data within the above data range is updated, the updated incremental data is obtained from the first cluster, the updated incremental data is first cached in the memory, and then the updated incremental data is gradually replaced in the memory with the data before the update in the second cluster.

[0090] When the data to be synchronized is cached in the memory, and the service polls the second cluster, if it detects that the data in the data range corresponding to the synchronization data identifier in the first cluster is updated, the updated incremental data will be cached in the memory. In the above case, there are two processing methods:

[0091] First: cache the updated incremental data as a new data in the memory. After the service can access the second cluster normally and the second cluster can receive data normally, synchronize all the data in the memory (the full data obtained for the first time and the updated incremental data) to the second cluster. The second cluster processes all the received data. Specifically, for the same data, keep the file with the latest update time or the latest version number, and delete other files.

[0092] Second: Replace the corresponding pre-update data in the memory with the updated incremental data, so that all the data stored in the memory is the latest data. After the service can access the second cluster normally and the second cluster can receive data normally, synchronize the cached data in the memory to the second cluster.

[0093] In the present application example, incremental synchronization of data in a specified range is achieved through monitoring, avoiding the problem of high network and disk pressure caused by full synchronization.

[0094] In a possible implementation, the present application embodiment provides another data backup method based on a data storage service, such as Figure 6 As shown, the data backup method based on the data storage service provided in the embodiment of the present application mainly includes steps S301 to S307.

[0095] S301. Obtain a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier.

[0096] The specific process of step S301 in the embodiment of the present application is the same as that of step S101 in the above embodiment. For details, please refer to the description in the above embodiment, and the embodiment of the present application will not be repeated.

[0097] S302: Determine whether certificate authentication is enabled for the first cluster.

[0098] Read the authentication identifier in the first cluster. If the authentication identifier is on, it indicates that the first cluster has enabled certificate authentication. If the authentication identifier is off, it indicates that the first cluster has not enabled certificate authentication. The data to be synchronized corresponding to the synchronization data identifier is directly obtained from the first cluster.

[0099] S303: When certificate authentication is enabled for the first cluster, parse the certificate of the first cluster from the data synchronization rule, and the certificate of the first cluster is used to verify the user identity.

[0100] If the authentication flag in the first cluster is on, it indicates that the first cluster turns on certificate authentication. At this time, the certificate of the first cluster is parsed from the data synchronization rule. If the certificate of the first cluster can be correctly parsed, the certificate is authenticated. If the authentication passes, step S304 is executed.

[0101] If the certificate of the first cluster cannot be correctly parsed, or if the certificate authentication fails, a certificate acquisition request is sent to the client, and the certificate of the first cluster is requested from the client again.

[0102] S304: When the user identity authentication is passed, obtain the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster.

[0103] The specific process of step S304 in the embodiment of the present application is the same as that of step S102 in the above embodiment. For details, please refer to the description in the above embodiment, and the embodiment of the present application will not be repeated.

[0104] S305: Determine whether certificate authentication is enabled for the second cluster.

[0105] Read the authentication identifier in the second cluster. If the authentication identifier is on, it indicates that the second cluster has enabled certificate authentication. If the authentication identifier is off, it indicates that the second cluster has not enabled certificate authentication. The data to be synchronized corresponding to the synchronization data identifier is directly obtained from the second cluster.

[0106] S306: When certificate authentication is enabled in the second cluster, the certificate of the second cluster is parsed from the data synchronization rule, and the certificate of the second cluster is used to verify the user identity.

[0107] If the authentication flag in the second cluster is on, it indicates that the second cluster turns on certificate authentication. At this time, the certificate of the second cluster is parsed from the data synchronization rule. If the certificate of the second cluster can be correctly parsed, the certificate is authenticated. If the authentication passes, step S307 is executed.

[0108] If the certificate of the second cluster cannot be correctly parsed, or if the certificate authentication fails, a certificate acquisition request is sent to the client, and the certificate of the second cluster is requested from the client again.

[0109] S307: When the user identity authentication is passed, synchronize the data to be synchronized to the second cluster corresponding to the second cluster address.

[0110] The specific process of step S307 in the embodiment of the present application is the same as that of step S103 in the above embodiment. For details, please refer to the description in the above embodiment, and the embodiment of the present application will not be repeated.

[0111] In the embodiment of the present application, identity authentication is performed before data synchronization to ensure the security of the data.

[0112] In a possible implementation, the present application embodiment provides another data backup method based on a data storage service, such as Figure 7 As shown, the data backup method based on the data storage service provided in the embodiment of the present application mainly includes steps S401 to S40.

[0113] S401. Obtain a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier.

[0114] S402: Acquire the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address.

[0115] S403: Synchronize the data to be synchronized to the second cluster corresponding to the second cluster address.

[0116] The specific process of steps S401 to S403 in the embodiment of the present application is the same as that of steps S101 to S103 in the above embodiment. For details, please refer to the description in the above embodiment, and the embodiment of the present application will not be repeated.

[0117] S404: When it is determined that a failure occurs in the first cluster during the process of connecting the first cluster to the service, the service is switched to connecting to the second cluster.

[0118] In the embodiment of the present application, the first cluster is the main ETCD cluster as an example for explanation. When the main ETCD cluster is normal, the main ETCD cluster provides services. If the main ETCD cluster fails during the service provided by the main ETCD cluster, the entire system will be down. At this time, the service needs to be switched to the standby ETCD cluster. The service switching method is not repeated in the embodiment of the present application.

[0119] S405: After the service is connected to the second cluster, the control service transmits the unsynchronized data to the second cluster.

[0120] It can be seen from the above embodiment that only the data within the specified range in the main ETCD cluster (data corresponding to the synchronization data identifier) ​​is synchronized to the second cluster, and part of the data is not synchronized from the main ETCD cluster to the standby ETCD cluster.

[0121] The above-mentioned unsynchronized data can be understood as data that changes with the service type or service content, such as: the node status of each node in the ETCD cluster, the service status of each service connected to the ETCD cluster, etc. The above-mentioned data to be synchronized can be understood as data that does not change with the service type or service content, such as: service list, service configuration, etc.

[0122] The unsynchronized data may be stored in the service, and therefore, the service may be controlled to transmit the unsynchronized data to the second cluster to ensure that the second cluster can operate normally.

[0123] S406: After the failure of the first cluster is recovered, switch the service to connect with the first cluster.

[0124] After receiving the fault recovery signal of the first cluster, the service is switched from the second cluster to the first cluster, and the service is provided by the first cluster.

[0125] In another implementation, after the second cluster is connected to the service, the second cluster can be used as the main cluster to continuously connect to the service, and the first cluster after failure recovery can be used as the backup cluster until the second cluster fails, and then the service is switched to the first cluster as the backup cluster.

[0126] S407: Acquire update data in the second cluster within a first time period, where the first time period refers to a period during which the second cluster is connected to the service.

[0127] Since the service is in normal working state during the connection with the second cluster, some updated data will be generated in the second cluster. If the data in the second cluster is updated during this period, the updated data is obtained and cached in the memory.

[0128] S408: Synchronize the updated data in the second cluster to the first cluster, where the updated data is data after the data to be synchronized is updated.

[0129] The updated data acquired in the above step S407 is synchronized to the first cluster to ensure the normal operation of the first cluster.

[0130] Furthermore, the updated data refers to updated data within a specified range. For example, if the data under / data is synchronized from the first cluster to the second cluster, only the updated data under / data is synchronized back to the first cluster as updated data.

[0131] In an embodiment of the present application, a structural diagram of a data synchronization system based on an ETCD cluster is provided; Figure 8As shown, the system includes a source ETCD cluster, a destination ETCD cluster, a data synchronization processor (ETCD DataSyncController) and a client, wherein the ETCD DataSync Controller reads the data synchronization rules set by the user on the client, and the data range to be synchronized needs to be configured in the data synchronization rules, such as setting to synchronize only the data under / data. After the ETCD DataSync Controller reads the data synchronization rules and confirms the synchronization data range, it obtains the data under the specified / data from the original ETCD cluster and writes this part to the destination ETCD cluster. When the data under / data changes subsequently, the ETCD DataSync Controller will monitor the changes in the corresponding data and write it to the destination ETCD cluster in real time.

[0132] In an embodiment of the present application, an interactive schematic diagram of a data synchronization method based on an ETCD cluster is provided; Fig. 9 As shown, the data synchronization method provided in the embodiment of the present application mainly includes the following steps:

[0133] S501. The user configures data synchronization rules on the client.

[0134] The data synchronization rules need to configure the original ETCD cluster address, the destination ETCD cluster address, and the synchronization data prefix. If ETCD has certificate authentication enabled, you also need to configure the certificates of the original and destination ETCD clusters.

[0135] S502: The processor reads the data synchronization rule, reads the relevant information, and obtains the full amount of data of the synchronization prefix through the original ETCD address.

[0136] S503: After reading this part of the full data, cache it in the memory first, and then gradually write the data to the destination ETCD cluster.

[0137] S504, monitor the data changes within the original ETCD synchronization data range in real time.

[0138] When any changes occur to the data in the original ETCD, they will be monitored. Use the watch interface to specify the version of the data returned and monitor all change events after this version.

[0139] S505: After relevant changes are monitored, the corresponding change data is written to the destination ETCD cluster.

[0140] The embodiment of the present application realizes the synchronization of the specified range of ETCD in real time, has the characteristic of fast switching, and can also specify the synchronization range to avoid the synchronization delay caused by full synchronization.

[0141] In some embodiments, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computing device, the computing device implements the data backup method based on the data storage service of any of the above embodiments.

[0142] In some embodiments, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to implement a data backup method based on a data storage service in any embodiment of the first aspect or the second aspect.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0144] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. An electronic device, characterized in that: include: The processor is configured as: Acquire a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier; Acquire the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address; The data to be synchronized is synchronized to the second cluster corresponding to the second cluster address.

2. The electronic device according to claim 1, characterized in that: The processor is configured to: Monitor the data range corresponding to the synchronization data identifier in the first cluster; When data within the data range corresponding to the synchronization data identifier is updated, the updated incremental data is synchronized to the second cluster corresponding to the second cluster address.

3. The electronic device according to claim 1, characterized in that: The processor is configured to: The data range corresponding to the synchronization data identifier in the first cluster is monitored through the watch interface.

4. The electronic device according to claim 1, characterized in that: The processor is configured to: Reading the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address; Cache the data to be synchronized into memory first; The data to be synchronized is synchronized from the memory to the second cluster corresponding to the second cluster address.

5. The electronic device according to claim 1, characterized in that: The processor is configured to: Establishing a data synchronization channel between the first cluster and the second cluster based on the data synchronization rule; Acquire the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the data synchronization channel; The data to be synchronized is synchronized to the second cluster corresponding to the second cluster address through the data synchronization channel.

6. The electronic device according to claim 5, characterized in that: The processor is configured to: Saving the data synchronization rules; After the electronic device crashes and restarts, obtaining a pre-saved data synchronization rule; Establishing a data synchronization channel between the first cluster and the second cluster based on the data synchronization rule; Acquire the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the data synchronization channel; The data to be synchronized is synchronized to the second cluster corresponding to the second cluster address through the data synchronization channel.

7. The method according to claim 1, characterized in that The processor is configured to: During the process of connecting the first cluster to the service, when it is determined that the first cluster fails, switching the service to connecting to the second cluster; After the failure of the first cluster is recovered, the service is switched to be connected to the first cluster.

8. The method according to claim 7, characterized in that The processor is configured to: Acquire updated data in the second cluster within a first time period, wherein the first time period refers to a period during which the second cluster is connected to the service; The updated data in the second cluster is synchronized to the first cluster, wherein the updated data is data after the data to be synchronized is updated.

9. The method according to claim 7, characterized in that: The processor is configured to: After the service is connected to the second cluster, the service is controlled to transmit the unsynchronized data to the second cluster.

10. A data backup method based on a data storage service, characterized in that: The method is applied to an electronic device, and the method comprises: Acquire a data synchronization rule, wherein the data synchronization rule includes a first cluster address, a second cluster address, and a synchronization data identifier; Acquire the to-be-synchronized data corresponding to the synchronization data identifier from the first cluster through the first cluster address; The data to be synchronized is synchronized to the second cluster corresponding to the second cluster address.

Images