A method, system and storage medium for avoiding brain split in a dual-node high-availability cluster

Abstract

The present invention provides a method and system for avoiding brain split in a dual-node high-availability cluster, and a storage medium. A standby heartbeat link is added in the dual-node high-availability cluster, and the IP in the standby heartbeat link selects the IP address of the business system network; the dual-node high-availability cluster adds a pingd resource and is configured as a clone resource; the pingd resource and the business service to be protected by the high-availability cluster are constrained and bound; the pingd resource is started, and the connectivity with the business system network is monitored through a ping request based on the standby heartbeat link; if the pingd resource cannot ping the business system network, it is determined whether to automatically stop the business service to be protected according to the constraint conditions of the constraint binding. The present invention monitors whether the "brain split" situation occurs and starts and stops the business service, thereby realizing the function of avoiding the "brain split" phenomenon in the dual-node cluster.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a method, system and storage medium for avoiding brain split in high-availability services under network heartbeat failure in a dual-node cluster. Background Art

[0002] High availability cluster is a computer cluster technology, whose main purpose is to ensure the continuous availability and reliability of key business systems. In a high availability cluster, multiple server devices are used to form a cluster, and various fault-tolerant technologies are adopted. When one server fails, other servers can immediately take over the work of the failed server, thereby achieving seamless switching and continuous operation of the business system. Among them, the dual-node high availability cluster is widely used in small and medium-sized enterprises with limited resources or relatively simple business needs due to its advantages of low configuration cost and easy management and maintenance.

[0003] When a high-availability cluster fails to exchange heartbeat signals and other control information between nodes due to network failure or software or hardware equipment problems, each node acts independently and cannot uniformly judge the system status and perform failover, causing confusion or data inconsistency when the application protected in the cluster provides services to the outside world. This phenomenon is called "brain split" in the field of high-availability clusters.

[0004] To address the above issues, the commonly used solution is to use Fence devices to add to the cluster as resources. When the cluster scheduling manager detects a "brain split" in the cluster, it will perform Fence actions (usually restarting) on ​​some nodes according to the priority or strategy set by the cluster, thereby achieving fault isolation. This mechanism ensures data consistency and maintains the stable operation of the cluster.

[0005] However, in some special scenarios, some servers may run on customized hardware without compatible fence devices and interfaces, or these fence devices are not allowed from the perspective of data and network security. In a two-node cluster, since there is no third independent arbitration point, it is difficult for the cluster to determine which node should be considered as an active node and which node should be considered as a faulty node when the cluster heartbeat is disconnected. Therefore, in the absence of fences or arbitration devices, how to avoid "brain split" in a two-node cluster is an urgent problem to be solved. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for avoiding brain split in a dual-node high-availability cluster, and a storage medium to monitor whether a "brain split" situation occurs and start and stop business services, thereby realizing the function of avoiding the "brain split" phenomenon in a dual-node cluster.

[0007] In order to achieve the above object, the technical solution of the present invention is implemented as follows:

[0008] A method for avoiding brain split in a dual-node high-availability cluster, comprising:

[0009] S1. Add a standby heartbeat link in a two-node high-availability cluster. The IP in the standby heartbeat link uses the IP address of the business system network.

[0010] S2. Add pingd resources to the two-node high-availability cluster and configure them as clone resources;

[0011] S3. Constrain and bind the pingd resources and the business services to be protected by the high-availability cluster;

[0012] S4, start the pingd resource, and monitor the connectivity with the business system network through a ping request based on the standby heartbeat link;

[0013] S5. If the pingd resource cannot ping the business system network, it is determined whether to automatically stop the business service to be protected according to the constraint conditions of the constraint binding.

[0014] Furthermore, in step S1, a backup heartbeat link is added by modifying the configuration file corosync.conf of all nodes in the cluster.

[0015] Furthermore, when the pingd resource is configured as a clone resource in step S2, the configuration parameters include the IP address of the pingd resource for pinging requests and the score obtained by pinging an IP address.

[0016] Furthermore, the constraint binding in step S3 includes:

[0017] S301: If the pingd resource is not started on a node in the cluster, the business service bound to the pingd resource is not allowed to be started on the node;

[0018] S302: If the total score obtained by the current ping request is less than multiplier*N, the business service bound to the pingd resource is not allowed to start on the node; N is the number of IP addresses that the pingd resource performs ping requests on in the pingd resource configuration parameters, and multiplier is the score obtained by pinging an IP address in the pingd resource configuration parameters.

[0019] Another aspect of the present invention provides a system for avoiding brain split in a dual-node high-availability cluster, comprising:

[0020] Backup heartbeat module: Add a backup heartbeat link in a two-node high-availability cluster. The IP in the backup heartbeat link uses the IP address of the business system network.

[0021] Resource configuration module: Add pingd resources to a two-node high-availability cluster and configure them as clone resources;

[0022] Constraint binding module: constrains and binds pingd resources with the business services to be protected by the high-availability cluster;

[0023] Start the monitoring module: start the pingd resource, and monitor the connectivity with the business system network through a ping request based on the backup heartbeat link;

[0024] Automatic stop module: If the pingd resource cannot ping the business system network, it is determined whether to automatically stop the business service to be protected according to the constraint conditions of the constraint binding.

[0025] Furthermore, in the standby heartbeat module, a standby heartbeat link is added by modifying the configuration file corosync.conf of all nodes in the cluster.

[0026] Furthermore, when the pingd resource is configured as a clone resource in the resource configuration module, the configuration parameters include the IP address of the pingd resource for pinging requests and the score obtained by pinging an IP address.

[0027] Furthermore, the constraint binding module includes:

[0028] First constraint unit: If the pingd resource is not started on a node in the cluster, the business service bound to the pingd resource is not allowed to start on the node;

[0029] Second constraint unit: If the total score of the current ping request is less than multiplier*N, the business service bound to the pingd resource is not allowed to start on the node; N is the number of IP addresses that the pingd resource pings in the pingd resource configuration parameters, and multiplier is the score obtained by pinging an IP address in the pingd resource configuration parameters.

[0030] The present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned method for avoiding brain split in a dual-node high-availability cluster.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention pings the service system network based on pingd resources to monitor whether a "brain split" situation occurs, and starts and stops service services based on constraint conditions. Based on this design principle, the function of avoiding the "brain split" phenomenon in a dual-node cluster is finally achieved.

[0033] 2. The present invention does not require the installation of additional Fence equipment, nor does it require any additional auxiliary software and hardware. The load impact on the system nodes can also be ignored, and it has low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the process of the first embodiment of the present invention;

[0035] Figure 2 It is a schematic diagram of network connection of the first embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0037] The design idea of ​​the present invention is to provide a solution for avoiding brain split in a two-node cluster by using pacemaker & corosync as core components and pingd resources as core resources to cooperate with each other in a scenario where arbitration devices such as Fence or Stonith devices cannot be used.

[0038] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0039] Embodiment 1:

[0040] In the first embodiment, the method for avoiding brain split in a dual-node high-availability cluster proposed by the present invention is as follows: Figure 1 As shown, including:

[0041] S1. Configure the cluster and add a business network segment heartbeat as a backup heartbeat; specifically, build and configure a dual-node high-availability cluster, with each node server containing at least two network cards. In addition, the server system version of each node in the cluster is consistent with the high-availability cluster software version. Based on the first group of heartbeat links in the cluster, combined with the IP address planning of the business network, a group of IP addresses of the business network segment is required to set the backup network heartbeat of the cluster. This group of redundant heartbeats can also further improve the reliability, availability and stability of the system to a certain extent.

[0042] When configuring a standby heartbeat link, you can complete the configuration by modifying the configuration information in the nodelist section of the cluster configuration file / etc / corosync / corosync.conf. Note that this configuration step requires modifying the configuration parameters of all nodes in the cluster, and after the modification, you need to restart the cluster for the configuration to take effect.

[0043] S2. Deploy ping clone resources in the cluster; add pingd resources in the cluster and set them as clone resources so that multiple nodes in the cluster can run redundant ping instances at the same time. The normal or not of the ping status on the node represents the communication status of the cluster network and the business network.

[0044] The parameter options that must be set for the pingd resource are:

[0045] host_list: The host IP address of the pingd resource that makes ping requests. Multiple addresses (separated by spaces) can be set based on the specific situation. Usually, it is the gateway address of the business network.

[0046] multiplier: The score that can be obtained by pinging a host IP in host_list; this score is used to calculate the location score when the pingd resource and business resource set the location constraint in the subsequent steps. The specific value of the score needs to be set according to the business scenario.

[0047] name: The name used when adding constraints to pingd. It can be modified. Please make sure it is consistent with the name of the configured pingd resource.

[0048] Currently, the settings of these three parameters basically meet the requirements of most scenarios. Other parameters of pingd resources are optional and can also be set based on specific business scenarios.

[0049] The cluster configuration clones the pingd resource, which starts a ping service on each node in the cluster. The ping service simulates the behavior of the client trying to connect to the server by sending an ICMP echo request (i.e., ping command) to the host IP in the host_list (the gateway of the business network). If all hosts in the host_list can successfully respond to the ping request, it means that the gateway of the business network is in normal status.

[0050] S3. Constrain and bind the pingd resource to the high-availability business service to be protected. Since the pingd resource can monitor the network status, the pingd resource is logically bound to the key business service in this step. This constraint ensures the consistency of business data in the event of a split-brain in the cluster.

[0051] Constraint binding method: Use the command interface provided by the pacemaker upper layer package pcs to set it or use the corresponding graphical interface tool to set it. The specific constraints set by the pcs command are as follows:

[0052] Constraint 1:

[0053] pcs constraint location res_name rule score=-INFINITY not_definedpingd;

[0054] Among them, res_name is the name of the resource to be managed, not the name of the pingd resource itself. pingd needs to be consistent with the name attribute in the parameter.

[0055] Constraint 1 means that if the pingd resource is not started on a node in the cluster, the business service bound to the pingd resource is not allowed to start on the node.

[0056] This constraint can prevent business services from being deployed on nodes with network problems, thereby ensuring that business services can be started and run in a good network environment.

[0057] Constraint 2:

[0058] pcs constraint location res_name rule score=-INFINITY pingd lt score;

[0059] The "score" needs to be set in combination with the multiplier of the pingd resource and the number of host IPs set in host_list. If one host IP is set, the location constraint score of the restriction condition is 1*multiplier. If N host IPs are added, it is N*multiplier. For example, if the multiplier of the above resource is 3000, if there is only one host IP in host_list, the location constraint score is 3000.

[0060] Constraint 2 means that when the current location constraint score is less than multiplier*N (where N is the number of hosts set in host_list and multiplier is the score set in the pingd resource configuration), the business service bound to the pingd resource is not allowed to start on the node.

[0061] Through this constraint, when the ping request cannot successfully access the host IP in host_list, the location constraint score is less than multiplier*N, and the business service bound to the ping service constraint will be restricted to run on the current node. This is because if the host in host_list cannot respond to the ping request of the current node, it usually indicates that the communication of the current node to the external network has failed. When the node cannot access the external network normally, stopping the service can avoid the service from generating or processing data on the unavailable network, thereby avoiding the problem of inconsistent business data.

[0062] S4. Start the pingd resource and related business resources; in a high-availability cluster, the cluster manages and monitors business services through resources. After the pingd resource is started, it will continuously ping the business gateway address to monitor the connectivity between the two. In addition, after the pingd resource is started, you can first query the score of the pingd resource to see the score after the initial startup to ensure the normal startup of subsequent business services. The query method is to query through the attrd_updater -n pingd -Q command.

[0063] S5. Determine whether the pingd resource can ping the service gateway; if the pingd resource cannot ping the service gateway, it means that the node is faulty and is not suitable for running the service, then execute S6. Otherwise, the pingd resource will continue to monitor the connectivity of the service gateway.

[0064] S6. Stop the high-availability resources to be protected. Based on the mechanism of pingd resources, if the service gateway set in host_list cannot be pinged, the location constraint score of pingd will be less than N*multiplier. Based on constraint 2 in S3, the service resources bound to pingd will be automatically stopped on the current node. At the same time, based on the high-availability cluster scheduling mechanism, other healthy nodes in the cluster will start the service on this node, providing high-availability services continuously and uninterruptedly, and avoiding the occurrence of brain split.

[0065] Figure 2 This is a network connection diagram of this embodiment. As can be seen from the figure, the overall solution actually includes three parts, namely the cluster basic service component, the business service resources to be protected in the upper layer, and the pingd resource component; based on the pingd resource, ping the business gateway to monitor whether the "brain split" situation occurs, and start and stop the business service based on the constraint conditions.

[0066] The method for avoiding brain split in a high-availability cluster provided in the embodiment of the present invention does not require the installation of additional Fence devices, nor does it require any additional auxiliary software and hardware, and the load impact on the system nodes is negligible, so it is also low-cost.

[0067] Embodiment 2:

[0068] This second embodiment proposes a system for avoiding brain split in a dual-node high-availability cluster, including:

[0069] Backup heartbeat module: Add a backup heartbeat link in a two-node high-availability cluster. The IP in the backup heartbeat link uses the IP address of the business system network. In the backup heartbeat module, add a backup heartbeat link by modifying the configuration file corosync.conf of all nodes in the cluster.

[0070] Resource configuration module: A two-node high-availability cluster adds a pingd resource and configures it as a clone resource. When a pingd resource is configured as a clone resource in the resource configuration module, the configuration parameters include the IP address that the pingd resource uses to make ping requests and the score obtained when a ping is successful.

[0071] Constraint binding module: constrains and binds pingd resources and business services to be protected by the high-availability cluster; the constraint binding module includes: a first constraint unit: if the pingd resource is not started on a node in the cluster, the business service bound to the pingd resource is not allowed to be started on the node; a second constraint unit: if the total score obtained by the current ping request is less than multiplier*N, the business service bound to the pingd resource is not allowed to be started on the node; wherein N is the number of IP addresses to which the pingd resource performs ping requests in the pingd resource configuration parameters, and the multiplier is the score obtained by pinging an IP address in the pingd resource configuration parameters.

[0072] Start the monitoring module: start the pingd resource, and monitor the connectivity with the business system network through ping requests based on the backup heartbeat link.

[0073] Automatic stop module: If the pingd resource cannot ping the business system network, it is determined whether to automatically stop the business service to be protected according to the constraint conditions of the constraint binding.

[0074] The system for avoiding brain split in a dual-node high-availability cluster proposed in the second embodiment can implement the method for avoiding brain split in a dual-node high-availability cluster described in the first embodiment, and has the same beneficial effects as the first embodiment.

[0075] Embodiment three:

[0076] This embodiment provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute the method for avoiding brain split in a dual-node high-availability cluster as described in the first embodiment.

[0077] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0078] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0079] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0080] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0081] The above-mentioned embodiments are only preferred implementations of the present invention, and are only used to help understand the method and core ideas of the present application. The protection scope of the present invention is not limited to the above-mentioned embodiments. All technical solutions under the idea of ​​the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for avoiding brain split in a dual-node high-availability cluster, characterized in that: include: S1. Add a standby heartbeat link in a two-node high-availability cluster. The IP in the standby heartbeat link uses the IP address of the business system network. Specifically, based on the first group of heartbeat links in the cluster, combined with the IP address planning of the business network, a group of IP addresses of the business network segment is required to set up the backup network heartbeat of the cluster; S2. Add pingd resources to the two-node high-availability cluster and configure them as clone resources; S3. Constrain and bind the pingd resources and the business services to be protected by the high-availability cluster; the constraint binding includes: S301: If the pingd resource is not started on a node in the cluster, the business service bound to the pingd resource is not allowed to be started on the node; S302: If the total score obtained by the current ping request is less than multiplier*N, the business service bound to the pingd resource is not allowed to be started on the node; N is the number of IP addresses that the pingd resource performs ping requests on in the pingd resource configuration parameters, and multiplier is the score obtained by pinging an IP address in the pingd resource configuration parameters; S4, start the pingd resource, and monitor the connectivity with the business system network through the ping request based on the standby heartbeat link; after the pingd resource is started, continue to ping the business gateway address to monitor the connectivity between the two; in addition, after the pingd resource is started, first query the score of the pingd resource to see the score after the initial startup to ensure the normal startup of subsequent business services; S5. If the pingd resource cannot ping the business system network, it is determined whether to automatically stop the business service to be protected based on the constraint conditions of the constraint binding; based on the mechanism of the pingd resource, if the business gateway set in the host_list cannot be pinged, the location constraint score of pingd is less than N*multiplier; based on the constraint S302 in S3, the business resource bound to pingd is automatically stopped on the current node; at the same time, based on the high-availability cluster scheduling mechanism, other healthy nodes in the cluster start the business service on this node.

2. The method for avoiding brain split in a dual-node high-availability cluster according to claim 1, characterized in that: In step S1, a backup heartbeat link is added by modifying the configuration file corosync.conf of all nodes in the cluster.

3. The method for avoiding brain split in a dual-node high-availability cluster according to claim 1, characterized in that: When the pingd resource is configured as a clone resource in step S2, the configuration parameters include the IP address of the pingd resource for pinging requests and the score obtained by pinging an IP address.

4. A system for avoiding brain split in a dual-node high-availability cluster, characterized in that: include: Backup heartbeat module: Add a backup heartbeat link in a two-node high-availability cluster. The IP in the backup heartbeat link uses the IP address of the business system network. Specifically, based on the first group of heartbeat links in the cluster, combined with the IP address planning of the business network, a group of IP addresses of the business network segment is required to set up the backup network heartbeat of the cluster; Resource configuration module: Add pingd resources to a two-node high-availability cluster and configure them as clone resources; Constraint binding module: constrains and binds pingd resources and business services to be protected by the high-availability cluster; the constraint binding module includes: a first constraint unit: if the pingd resource is not started on a node in the cluster, the business service bound to the pingd resource is not allowed to be started on the node; a second constraint unit: if the total score obtained by the current ping request is less than multiplier*N, the business service bound to the pingd resource is not allowed to be started on the node; wherein N is the number of IP addresses that the pingd resource performs ping requests on in the pingd resource configuration parameters, and the multiplier is the score obtained by pinging an IP address in the pingd resource configuration parameters; Start the monitoring module: start the pingd resource, and monitor the connectivity with the business system network through ping requests based on the backup heartbeat link; after the pingd resource is started, continue to ping the business gateway address to monitor the connectivity between the two; in addition, after the pingd resource is started, first query the score of the pingd resource to see the score after the initial startup to ensure the normal startup of subsequent business services; Automatic stop module: If the pingd resource cannot ping the business system network, it is determined whether to automatically stop the business service to be protected based on the constraint conditions of the constraint binding; based on the mechanism of the pingd resource, if the business gateway set in the host_list cannot be pinged, the location constraint score of pingd is less than N*multiplier; based on the constraint S302 in S3, the business resource bound to pingd is automatically stopped on the current node; at the same time, based on the high-availability cluster scheduling mechanism, other healthy nodes in the cluster start the business service on this node.

5. The system for avoiding brain split in a dual-node high-availability cluster according to claim 4, characterized in that: In the standby heartbeat module, add a standby heartbeat link by modifying the configuration file corosync.conf of all nodes in the cluster.

6. The system for avoiding brain split in a dual-node high-availability cluster according to claim 4, characterized in that: When a pingd resource is configured as a clone resource in the resource configuration module, the configuration parameters include the IP address that the pingd resource uses to make a ping request and the score obtained by pinging an IP address.

7. A computer-readable storage medium storing a computer program, characterized in that: The computer program is used to execute the method for avoiding brain split in a dual-node high-availability cluster as described in any one of claims 1-3.

Images