Deployment method and device of cross-site cluster based on ETCD
By building a backup cluster in the ETCD cluster and synchronizing data, the cluster unavailability problem caused by the failure of most node sites is solved, rapid recovery and high availability are achieved, and business impact is reduced.
Patent Information
- Application Number
- CN202210840717.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-18
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-07-18
AI Technical Summary
In cross-site deployment of ETCD clusters, when most node sites fail, the entire cluster will be unavailable and cannot provide services to the outside world, resulting in business impact.
Build a backup cluster corresponding to the primary cluster, and synchronize the data of the primary cluster to the backup cluster. Switch communication to connect to the backup cluster in case of site-level failures. The backup cluster becomes the new primary cluster, and the primary cluster is converted into the backup cluster to be restored. .
It realizes rapid recovery and service continuity when site-level failures occur, reduces the impact of failures on business, ensures data consistency, and improves the high availability of ETCD across sites.
Smart Images

Figure CN115208743B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of big data, and in particular to a method and device for deploying a cross-site cluster based on ETCD. Background Art
[0002] With the continuous growth of business volume and the explosion of big data, the requirements for high availability of systems are constantly increasing. As the industry's leading distributed storage system, ETCD often appears as a coordinator in various large-scale distributed system frameworks.
[0003] In the current cross-site deployment scheme of the ETCD cluster, based on the Raft protocol rules, since the number of nodes at a site is generally odd, there will usually be a scenario where most nodes and a few nodes are deployed at different sites, that is, among the sites included in the ETCD cluster, there is always one site that deploys more ETCD nodes.
[0004] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because they are described in the background technology section of the present application. Summary of the invention
[0005] The inventors have discovered that in the above-mentioned existing ETCD cluster cross-site deployment scheme, when a site with a few nodes fails, the ETCD cluster to which the site belongs can still provide communication services. However, when a site with a majority of nodes fails, the majority election rule of the Raft protocol will be triggered, and the entire ETCD cluster to which the site belongs will become unavailable; that is, when a site-level failure occurs, there is a 50% probability that communication will be interrupted and services will no longer be provided to the outside.
[0006] In order to solve the above technical problems or similar problems, the embodiments of the present application provide a cross-site cluster deployment method and device based on ETCD, so as to avoid the situation in which a site-level failure occurs in the ETCD cross-site deployment solution and services can no longer be provided to the outside world, causing business impact.
[0007] An embodiment of the present application provides a cross-site cluster deployment method based on ETCD, wherein the method includes: building a backup cluster corresponding to the current main cluster based on the current main cluster; synchronously backing up the data of the current main cluster to the backup cluster; and when a site-level failure occurs in the current main cluster, switching the communication connection with the current main cluster to the communication connection with the backup cluster, wherein the backup cluster is switched to a new main cluster, and the current main cluster is switched to a backup cluster to be restored.
[0008] In some embodiments, the main cluster and the backup cluster include a first site and a second site, respectively, wherein building a backup cluster corresponding to the current main cluster based on the current main cluster includes: deploying the number of nodes of the first site and the second site in the backup cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster, wherein the number of nodes included in the first site and the second site in the current main cluster is inversely proportional to the number of nodes included in the first site and the second site in the backup cluster.
[0009] In some embodiments, deploying the number of nodes of the first site and the second site in the standby cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster includes: deploying the number of nodes included in the second site in the standby cluster as the number of nodes included in the first site in the current main cluster; deploying the number of nodes included in the first site in the standby cluster as the number of nodes included in the second site in the current main cluster.
[0010] In some embodiments, synchronously backing up the data of the current primary cluster to the backup cluster includes: starting a Mirror data synchronization tool in the current primary cluster to trigger data synchronization from the current primary cluster to the backup cluster, and creating a first listener in the full path of the current primary cluster based on the triggering of the data synchronization; and using the Mirror data synchronization tool to synchronously back up the data of the current primary cluster to the backup cluster based on the listening result of the first listener.
[0011] In some embodiments, before switching the communication connection with the current primary cluster to the communication connection with the backup cluster, the method further includes: determining whether a site-level failure occurs in the current primary cluster according to a monitoring result of a second monitor for performing heartbeat detection on the current primary cluster.
[0012] In some embodiments, judging whether a site-level failure occurs in the current main cluster according to a monitoring result of a second listener for performing heartbeat detection on the current main cluster includes: when the monitoring result of the second listener is isHealthy, determining that a site-level failure occurs in the current main cluster.
[0013] In some embodiments, switching the communication connection with the current main cluster to the communication connection with the backup cluster includes: when it is determined that a site-level failure has occurred in the current main cluster, disconnecting the communication connection with the current main cluster; destroying the second listener; obtaining the cluster address of the backup cluster in a preset backup cluster parameter table; and establishing a communication connection with the backup cluster according to the cluster address of the backup cluster.
[0014] In some embodiments, switching the communication connection with the current primary cluster to the communication connection with the standby cluster further includes: creating a new second listener in the new primary cluster for performing heartbeat detection on the new primary cluster.
[0015] In some embodiments, the method further includes: determining whether the failure of the standby cluster to be restored has been eliminated, and when the determination result is that the failure of the standby cluster to be restored has been eliminated, switching the standby cluster to be restored to a new standby cluster.
[0016] In some embodiments, the method further includes: starting a Mirror data synchronization tool in the new primary cluster to trigger data synchronization from the new primary cluster to the new standby cluster.
[0017] An embodiment of the present application also provides a cross-site cluster deployment device based on ETCD, wherein the device includes: a building unit, which builds a backup cluster corresponding to the current main cluster based on the current main cluster; a backup unit, which synchronously backs up the data of the current main cluster to the backup cluster; and a switching unit, which switches the communication connection with the current main cluster to the communication connection with the backup cluster when a site-level failure occurs in the current main cluster, wherein the backup cluster is switched to a new main cluster, and the current main cluster is switched to a backup cluster to be restored.
[0018] An embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned ETCD-based cross-site cluster deployment method when executing the computer program.
[0019] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned ETCD-based cross-site cluster deployment method is implemented.
[0020] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned ETCD-based cross-site cluster deployment method.
[0021] One of the beneficial effects of the embodiments of the present application is that by building a backup cluster corresponding to the main cluster and synchronously backing up the data of the main cluster to the backup cluster, external services can be restored in time when a site-level failure occurs, thereby improving the stability of the entire distributed service, greatly reducing the impact of the failure on the business, and effectively ensuring data consistency, thereby ensuring the high availability of ETCD across sites.
[0022] With reference to the following description and accompanying drawings, the embodiments of the present application are disclosed in detail. It should be understood that the embodiments of the present application are not limited in scope. Within the spirit and scope of the appended claims, the embodiments of the present application include many changes, modifications and equivalents.
[0023] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0025] Figure 1 It is a schematic diagram of a deployment method of a cross-site cluster based on ETCD according to an embodiment of the present application;
[0026] Figure 2 yes Figure 1 A schematic diagram of step 101;
[0027] Figure 3 yes Figure 1 A schematic diagram of step 102;
[0028] Figure 4 yes Figure 1 A schematic diagram of step 103;
[0029] Figure 5 is another schematic diagram of a cross-site cluster deployment method based on ETCD according to an embodiment of the present application;
[0030] Figure 6 It is a flowchart of a cross-site cluster deployment method based on ETCD according to an embodiment of the present application;
[0031] Figure 7 It is a schematic diagram of a deployment device of a cross-site cluster based on ETCD according to an embodiment of the present application;
[0032] Figure 8 yes Figure 7 A schematic diagram of a building unit;
[0033] Fig. 9 yes Figure 7 A schematic diagram of a backup unit;
[0034] Fig.10 yes Figure 7 Schematic diagram of the switching unit. DETAILED DESCRIPTION
[0035] With reference to the accompanying drawings, the above and other features of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the present application includes all modifications, variations and equivalents falling within the scope of the attached claims.
[0036] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.
[0037] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.
[0038] The implementation of the embodiments of the present application will be described below with reference to the accompanying drawings.
[0039] An embodiment of the present application provides a cross-site cluster deployment method based on ETCD. Figure 1 It is a schematic diagram of a cross-site cluster deployment method based on ETCD according to an embodiment of the present application.
[0040] like Figure 1 As shown, the deployment method of the cross-site cluster based on ETCD may include:
[0041] 101, building a backup cluster corresponding to the current main cluster based on the current main cluster;
[0042] 102, synchronously backing up the data of the current main cluster to the standby cluster;
[0043] 103. When a site-level failure occurs in the current primary cluster, the communication connection with the current primary cluster is switched to the communication connection with the backup cluster.
[0044] After the above switching is completed, the standby cluster becomes the new main cluster, and the current main cluster becomes the standby cluster to be restored.
[0045] According to the above embodiment, by building a backup cluster corresponding to the main cluster and synchronously backing up the data of the main cluster to the backup cluster, external services can be restored in time when a site-level failure occurs, thereby improving the stability of the entire distributed service (ETCD), greatly reducing the impact of failures on the business, and effectively ensuring data consistency, thereby ensuring the high availability of ETCD across sites.
[0046] In some embodiments of the present application, each site in an ETCD-based cluster (hereinafter referred to as "ETCD cluster") may include one or more nodes. If the number of failed nodes in the site is too large and causes the entire site to fail, it is called a site-level failure. That is, a site-level failure refers to the failure of a site in the ETCD cluster (i.e., the current main cluster), rather than the failure of one or some nodes in a site.
[0047] In some embodiments of the present application, the ETCD cluster usually includes two sites (a first site and a second site), so the current main cluster and the backup cluster built based on the main cluster also include the first site and the second site respectively.
[0048] Figure 2 yes Figure 1 Schematic diagram of step 101.
[0049] like Figure 2 As shown, building a standby cluster corresponding to the current main cluster according to the current main cluster (step 101) may include:
[0050] 201. Deploy the number of nodes of the first site and the second site in the backup cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster, so that the number of nodes included in the first site and the second site in the backup cluster is inversely proportional to the number of nodes included in the first site and the second site in the current main cluster.
[0051] Specifically, if the number of nodes included in the first site A in the current main cluster is greater than the number of nodes included in the second site B in the current main cluster, then the number of nodes included in the first site A of the established standby cluster is less than the number of nodes included in the second site B of the standby cluster. For example, the number of nodes included in the first site A in the current main cluster is N1, and the number of nodes included in the second site B in the current main cluster is N2, where N1>N2, then the number of nodes in the first site A in the established standby cluster is N3, and the number of nodes in the second site B in the standby cluster is N4, where N3<N4. In other words, whether the first site A or the second site B in the ETCD cluster has a site-level failure, it can be guaranteed that the other site has a majority node.
[0052] Therefore, the standby cluster is reversely built by making the number of nodes in the site of the standby cluster inversely proportional to the number of nodes in the site of the current main cluster, thereby ensuring the high availability of the entire cluster.
[0053] In the above embodiment, further, step 201 may include: deploying the number of nodes included in the second site in the backup cluster as the number of nodes included in the first site in the current main cluster; and deploying the number of nodes included in the first site in the backup cluster as the number of nodes included in the second site in the current main cluster.
[0054] Specifically, the number of nodes included in the first site A in the current main cluster is M1, and the number of nodes included in the second site B in the current main cluster is M2. Then the number of nodes in the first site A in the constructed backup cluster is M2, and the number of nodes in the second site B in the backup cluster is M1.
[0055] Therefore, by reversely building a backup cluster with the same number of nodes, the high availability of the site can be improved.
[0056] In the present application, only an example of building a standby cluster by reversely setting the number of nodes at two sites is cited, but the present application is not limited to this, and all methods of building a corresponding standby cluster based on a main cluster are included therein.
[0057] Figure 3 yes Figure 1 Schematic diagram of step 102.
[0058] like Figure 3 As shown, synchronously backing up the data of the current primary cluster to the standby cluster may include:
[0059] 301, start the Mirror data synchronization tool in the current primary cluster to trigger data synchronization from the current primary cluster to the standby cluster;
[0060] 302, according to the trigger of data synchronization, create a listener (first listener) in the full path of the current main cluster;
[0061] 303 . According to the monitoring result of the first monitor, use the Mirror data synchronization tool to synchronously back up the data of the current active cluster to the standby cluster.
[0062] Therefore, by utilizing the high-performance real-time response of the listener's watch mechanism, when data changes are detected in the main cluster, the Mirror data synchronization tool can complete data synchronization within milliseconds without affecting external services.
[0063] In some embodiments of the present application, before step 103, the ETCD-based cross-site cluster deployment method may further include:
[0064] It is determined whether a site-level failure occurs in the current main cluster according to the monitoring result of the monitor (second monitor) used for performing heartbeat detection on the current main cluster.
[0065] Among them, the watch mechanism of the second listener for heartbeat detection is similar to the current technology and will not be repeated here.
[0066] Specifically, when the monitoring result of the second listener is isHealthy, it is determined that a site-level failure occurs in the current main cluster.
[0067] Figure 4 yes Figure 1 Schematic diagram of step 103.
[0068] In the above embodiment, whether switching is required is determined according to the judgment result. When switching is required, the following steps are performed: Figure 4 Steps shown.
[0069] like Figure 4 As shown, when a site-level failure occurs in the current primary cluster, switching the communication connection with the current primary cluster to the communication connection with the standby cluster may include:
[0070] 401, when it is determined that a site-level failure occurs in the current main cluster, disconnect the communication connection with the current main cluster;
[0071] 402, destroy the second listener;
[0072] 403, obtaining the cluster address of the standby cluster in the preset standby cluster parameter table;
[0073] The standby cluster parameter table is stored in advance.
[0074] 404. Establish a communication connection with the standby cluster according to the cluster address of the standby cluster.
[0075] Therefore, by using this method to perform active-standby switching, the entire switching process is fast and efficient, and the overall time consumption can be in seconds; after the active-standby switching, the standby cluster provides services to the outside as the new active cluster, and the current active cluster switches to the standby cluster to be restored.
[0076] like Figure 4 As shown, when a site-level failure occurs in the current primary cluster, switching the communication connection with the current primary cluster to the communication connection with the standby cluster may also include:
[0077] 405 , create a new listener (new second listener) in the new primary cluster for performing heartbeat detection on the new primary cluster.
[0078] Therefore, the watch mechanism of the new second listener can be further used to monitor in real time whether the new main cluster fails, thereby improving the stability of the entire distributed service.
[0079] Figure 5 This is another schematic diagram of the ETCD-based cross-site cluster deployment method of an embodiment of the present application.
[0080] like Figure 5 As shown, the deployment method of the cross-site cluster based on ETCD may include:
[0081] 501, building a backup cluster corresponding to the current main cluster based on the current main cluster;
[0082] 502, synchronously backing up the data of the current main cluster to the backup cluster;
[0083] 503, when a site-level failure occurs in the current primary cluster, the communication connection with the current primary cluster is switched to the communication connection with the backup cluster; wherein, after the switching is completed, the backup cluster becomes the new primary cluster, and the current primary cluster becomes the backup cluster to be restored.
[0084] The above 501 to 503 are related to Figure 1 101 to 103 correspond to the method steps respectively, and 501 to 503 will not be repeated here.
[0085] In some embodiments of the present application, Figure 5 As shown, in addition to the above 501 to 503, the deployment method of the cross-site cluster based on ETCD may also include:
[0086] 504, determining whether the failure of the standby cluster to be restored has been eliminated;
[0087] 505. When the judgment result is that the failure of the standby cluster to be restored has been eliminated, the standby cluster to be restored is switched to a new standby cluster.
[0088] Therefore, while ensuring the continuity of external services, it can provide enough time for operation and maintenance personnel to troubleshoot problems and restore the failed ETCD cluster (the backup cluster to be restored). In addition, a new backup cluster can be further provided to the new primary cluster, thereby avoiding the problem of the entire ETCD cluster being unavailable when a site-level failure occurs in the new primary cluster, thereby further ensuring the stability of the entire distributed service.
[0089] In the above embodiment, if Figure 5 As shown, the deployment method of the cross-site cluster based on ETCD may also include:
[0090] 506 , starting the Mirror data synchronization tool in the new primary cluster to trigger data synchronization from the new primary cluster to the new standby cluster.
[0091] In this way, reverse synchronization of data from the new primary cluster to the new backup cluster can be achieved, thereby utilizing the efficient synchronization mechanism and bidirectional availability of cluster data to further effectively ensure data consistency.
[0092] The specific implementation method of the cross-site cluster deployment method based on ETCD is described in detail below.
[0093] Figure 6 It is a flowchart of a cross-site cluster deployment method based on ETCD according to an embodiment of the present application.
[0094] like Figure 6 As shown, the deployment method of the cross-site cluster based on ETCD may include:
[0095] 601. Building a backup cluster corresponding to the current main cluster based on the current main cluster.
[0096] For example, the number of nodes included in the second site in the standby cluster is deployed as the number of nodes included in the first site in the current main cluster; and the number of nodes included in the first site in the standby cluster is deployed as the number of nodes included in the second site in the current main cluster, thereby building a standby cluster based on the main cluster.
[0097] 602 , start the Mirror data synchronization tool in the current primary cluster to synchronously back up data from the current primary cluster to the standby cluster according to the monitoring result of the watch mechanism.
[0098] 603, judging whether a site-level failure occurs in the current main cluster according to the monitoring result of the monitor that performs heartbeat detection on the current main cluster, if yes, proceeding to 604, if no, performing a loop judgment.
[0099] 604, disconnecting the communication connection with the current main cluster, and the current main cluster becomes the standby cluster to be restored;
[0100] 605, destroy the listener that performs heartbeat detection on the current main cluster;
[0101] 606, obtaining the cluster address of the standby cluster;
[0102] 607, establishing a communication connection with the standby cluster according to the cluster address of the standby cluster, and the standby cluster becomes the new main cluster;
[0103] 608, creating a new listener in the new primary cluster for performing heartbeat detection on the new primary cluster;
[0104] 609, when the failure of the standby cluster to be restored has been eliminated, the standby cluster to be restored is switched to a new standby cluster;
[0105] 610 , start the Mirror data synchronization tool in the new primary cluster to trigger data synchronization from the new primary cluster to the new standby cluster.
[0106] According to the above embodiment, the cross-site high availability method effectively ensures that when a site-level failure occurs, external services can be restored within a few seconds; and while ensuring service continuity, it provides sufficient time for operation and maintenance personnel to troubleshoot problems and restore the failed ETCD cluster; and, by utilizing the efficient synchronization mechanism and two-way availability of cluster data, data consistency is effectively guaranteed, which globally improves the stability of the entire distributed service and greatly reduces the impact of failures on the business.
[0107] The implementation of the above specific application is only an example, and the remaining implementation methods will not be described in detail one by one.
[0108] The embodiment of the present application also provides a cross-site cluster deployment device based on ETCD, as described in the following embodiment. Since the principle of the problem solved by the cross-site cluster deployment device based on ETCD is similar to the cross-site cluster deployment method based on ETCD, the implementation of the cross-site cluster deployment device based on ETCD can refer to the implementation of the cross-site cluster deployment method based on ETCD, and the repeated parts will not be repeated.
[0109] Figure 7 Schematic diagram of a cross-site cluster deployment device based on ETCD according to an embodiment of the present application. Figure 7As shown, the cross-site cluster deployment device 700 based on ETCD may include: a construction unit 701, a backup unit 702 and a switching unit 703; wherein the construction unit 701 builds a backup cluster corresponding to the current main cluster based on the current main cluster; the backup unit 702 synchronously backs up the data of the current main cluster to the backup cluster; the switching unit 703 switches the communication connection with the current main cluster to the communication connection with the backup cluster when a site-level failure occurs in the current main cluster. wherein the backup cluster is switched to the new main cluster, and the current main cluster is switched to the backup cluster to be restored.
[0110] According to the above embodiment, by building a backup cluster corresponding to the main cluster and synchronously backing up the data of the main cluster to the backup cluster, external services can be restored in time when a site-level failure occurs, thereby improving the stability of the entire distributed service (ETCD), greatly reducing the impact of failures on the business, and effectively ensuring data consistency, thereby ensuring the high availability of ETCD across sites.
[0111] Figure 8 yes Figure 7 Schematic diagram of the building unit. Figure 8 As shown, the building unit 701 may include a deployment module 7011 .
[0112] In some embodiments of the present application, the deployment module 7011 can deploy the number of nodes of the first site and the second site in the standby cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster, so that the number of nodes included in the first site and the second site in the current main cluster is inversely proportional to the number of nodes included in the first site and the second site in the standby cluster. Specifically, the deployment module 7011 can deploy the number of nodes included in the second site in the standby cluster as the number of nodes included in the first site in the current main cluster; and deploy the number of nodes included in the first site in the standby cluster as the number of nodes included in the second site in the current main cluster.
[0113] Fig. 9 yes Figure 7 Schematic diagram of the backup unit. Fig. 9 As shown, the backup unit 702 may include a startup module 7021 , a creation module 7022 , and a data synchronization module 7023 .
[0114] Among them, the startup module 7021 is used to start the Mirror data synchronization tool in the current main cluster to trigger data synchronization from the current main cluster to the backup cluster; the creation module 7022 is used to create a first listener in the full path of the current main cluster according to the trigger of data synchronization; the data synchronization module 7023 is used to use the Mirror data synchronization tool to synchronously back up the data of the current main cluster to the backup cluster according to the listening results of the first listener.
[0115] In some embodiments of the present application, the ETCD-based cross-site cluster deployment device 700 may also include a first judgment unit (not shown), which is used to judge whether a site-level failure has occurred in the current main cluster based on the listening result of the second listener that performs heartbeat detection on the current main cluster; and when the listening result of the second listener is isHealthy, the judgment unit determines that a site-level failure has occurred in the current main cluster.
[0116] Fig.10 yes Figure 7 Schematic diagram of the switching unit. Fig.10 As shown, the switching unit 703 may include a disconnection module 7031, a destruction module 7032, an acquisition module 7033, and a connection module 7034. The disconnection module 7031 is used to disconnect the communication connection with the current main cluster when it is determined that a site-level failure has occurred in the current main cluster; the destruction module 7032 is used to destroy the second listener; the acquisition module 7033 is used to obtain the cluster address of the standby cluster in the preset standby cluster parameter table; and the connection module 7034 is used to establish a communication connection with the standby cluster according to the cluster address of the standby cluster.
[0117] In some embodiments of the present application, Fig.10 As shown, the switching unit 703 may further include a creating module 7035, and the creating module 7035 is used to create a new second listener in the new primary cluster for performing heartbeat detection on the new primary cluster.
[0118] In some embodiments of the present application, the ETCD-based cross-site cluster deployment device 700 may also include a second judgment unit (not shown), which is used to determine whether the failure of the standby cluster to be restored has been eliminated. When the judgment result is that the failure of the standby cluster to be restored has been eliminated, the standby cluster to be restored is switched to a new standby cluster.
[0119] In some embodiments of the present application, the ETCD-based cross-site cluster deployment device 700 may also include a starting unit (not shown), which is used to start the Mirror data synchronization tool in the new primary cluster to trigger data synchronization from the new primary cluster to the new backup cluster.
[0120] According to the above embodiment, the cross-site high availability method effectively ensures that when a site-level failure occurs, external services can be restored within a few seconds; and while ensuring service continuity, it provides sufficient time for operation and maintenance personnel to troubleshoot problems and restore the failed ETCD cluster; and, by utilizing the efficient synchronization mechanism and two-way availability of cluster data, data consistency is effectively guaranteed, which globally improves the stability of the entire distributed service and greatly reduces the impact of failures on the business.
[0121] An embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned ETCD-based cross-site cluster deployment method when executing the computer program.
[0122] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned ETCD-based cross-site cluster deployment method.
[0123] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned ETCD-based cross-site cluster deployment method.
[0124] According to the above embodiment, by building a backup cluster corresponding to the main cluster and synchronously backing up the data of the main cluster to the backup cluster, external services can be restored in time when a site-level failure occurs, thereby improving the stability of the entire distributed service (ETCD), greatly reducing the impact of failures on the business, and effectively ensuring data consistency, thereby ensuring the high availability of ETCD across sites.
[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0129] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.
Claims
1. A cross-site cluster deployment method based on ETCD, characterized in that: The method comprises: Building a backup cluster corresponding to the current main cluster based on the current main cluster; Synchronously backing up the data of the current primary cluster to the standby cluster; and When a site-level failure occurs in the current primary cluster, the communication connection with the current primary cluster is switched to the communication connection with the backup cluster, wherein the backup cluster is switched to the new primary cluster and the current primary cluster is switched to the backup cluster to be restored, The main cluster and the standby cluster both include a first site and a second site, wherein building a standby cluster corresponding to the current main cluster based on the current main cluster includes: Deploy the number of nodes of the first site and the second site in the standby cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster, The ratio of the number of nodes included in the first site and the second site in the current main cluster is inversely proportional to the ratio of the number of nodes included in the first site and the second site in the standby cluster.
2. The cross-site cluster deployment method according to claim 1, characterized in that: Deploying the number of nodes of the first site and the second site in the standby cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster includes: Deploy the number of nodes included in the second site in the standby cluster to the number of nodes included in the first site in the current main cluster; The number of nodes included in the first site in the standby cluster is deployed as the number of nodes included in the second site in the current main cluster.
3. The cross-site cluster deployment method according to claim 1, characterized in that: Synchronously backing up the data of the current primary cluster to the standby cluster includes: Starting a Mirror data synchronization tool in the current primary cluster to trigger data synchronization from the current primary cluster to the standby cluster, Creating a first listener in the full path of the current primary cluster according to the triggering of the data synchronization; According to the monitoring result of the first monitor, the Mirror data synchronization tool is used to synchronously back up the data of the current primary cluster to the standby cluster.
4. The cross-site cluster deployment method according to claim 1, characterized in that: Before switching the communication connection with the current main cluster to the communication connection with the standby cluster, the method further includes: It is determined whether a site-level failure occurs in the current main cluster according to a monitoring result of a second monitor used for performing heartbeat detection on the current main cluster.
5. The cross-site cluster deployment method according to claim 4, characterized in that: Determining whether a site-level failure occurs in the current main cluster according to a monitoring result of a second monitor for performing heartbeat detection on the current main cluster includes: When the monitoring result of the second listener is isHealthy, it is determined that a site-level failure occurs in the current main cluster.
6. The cross-site cluster deployment method according to claim 5, characterized in that: Switching the communication connection with the current main cluster to the communication connection with the standby cluster includes: When it is determined that a site-level failure occurs in the current main cluster, disconnecting the communication connection with the current main cluster; Destroy the second listener; Obtaining the cluster address of the standby cluster in a preset standby cluster parameter table; A communication connection with the standby cluster is established according to the cluster address of the standby cluster.
7. The cross-site cluster deployment method according to claim 6, characterized in that: Switching the communication connection with the current main cluster to the communication connection with the standby cluster further includes: A new second listener is created in the new primary cluster for performing heartbeat detection on the new primary cluster.
8. The cross-site cluster deployment method according to claim 1, characterized in that: The method further comprises: Determine whether the failure of the standby cluster to be restored has been eliminated, When the judgment result is that the failure of the standby cluster to be restored has been eliminated, the standby cluster to be restored is switched to a new standby cluster.
9. The cross-site cluster deployment method according to claim 8, characterized in that: The method further comprises: A Mirror data synchronization tool is started in the new primary cluster to trigger data synchronization from the new primary cluster to the new standby cluster.
10. A cross-site cluster deployment device based on ETCD, characterized in that: The device comprises: A building unit, which builds a backup cluster corresponding to the current main cluster based on the current main cluster; A backup unit, which synchronously backs up the data of the current primary cluster to the backup cluster; and a switching unit, which switches the communication connection with the current main cluster to the communication connection with the backup cluster when a site-level failure occurs in the current main cluster, wherein the backup cluster is switched to a new main cluster and the current main cluster is switched to a backup cluster to be restored, The primary cluster and the backup cluster both include a first site and a second site, Wherein, the building unit comprises: A deployment module deploys the number of nodes of the first site and the second site in the standby cluster according to the number of nodes respectively included in the first site and the second site in the current main cluster, so that the ratio of the number of nodes included in the first site and the second site in the current main cluster is inversely proportional to the ratio of the number of nodes included in the first site and the second site in the standby cluster.
11. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.
13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.
