Grouping multi-machine hot backup method and system and storage medium

Abstract

The invention discloses a grouped multi-machine hot backup method and system and a storage medium. The method comprises the steps that the number of hot backup nodes with key functions is determined, a group number and an intra-group number are marked for each hot backup node, each hot backup node operates independently, an online node list is established through heartbeat messages, and working nodes are known through working messages; if one hot backup node cannot receive the working message for a period of time, the working node is judged to be offline, a new working node is selected according to a selection rule, and if the selected working node is the node, the node begins to participate in system work and then sends a heartbeat message and a working message according to a certain period; if the selected working node is not the current node, the selected working node only sends a heartbeat message; the method further comprises the step of assigning a working node in a manual intervention mode. According to the invention, the number of the hot backup nodes is expanded, and when the working node is offline, a new working node is quickly selected from the online node list, so that the problem that key functions of the whole system cannot work after the working node is offline due to failure is avoided.

Description

Background Art

[0002] In some large systems, there are some key working nodes that affect system availability once problems occur. These working nodes require relatively high reliability, so these key working nodes need to use hot backup technology to increase reliability. Generally, dual-machine hot backup technology is used. This technology usually uses an arbitration device and two hot backup nodes. The hot backup node notifies the arbitration device through a heartbeat message that it is running normally. The arbitration device determines whether the two hot backup nodes are online according to the arbitration cycle. If the arbitration device determines that one of the hot backup nodes as a working node is not online, it notifies the other hot backup node to start working as a replacement working node.

[0003] The above method has the following problems:

[0004] There are only two hot backup nodes, which is not expandable enough to meet complex application scenarios. Once the arbitration device fails, the ability to determine whether the hot backup is online is lost. Due to communication delays or information loss, it is possible that both machines may work or neither may work. Summary of the invention

[0005] The present invention provides a group multi-machine hot backup method, system and storage medium to solve the reliability problem caused by key function failure during system operation.

[0006] The technical solution of a group multi-machine hot backup method of the present invention is as follows:

[0007] The number of hot backup nodes in a system is determined based on the key functions in the system. The number of hot backup nodes in each group is determined as G, and the number of hot backup nodes in each group is determined as N, where G and N are both positive integers. A hot backup node can be considered as an independent processor, and each independent processor has a different IP address. All hot backup nodes are connected via Ethernet. Among the hot backup nodes, there is only one working node that participates in the system work and realizes the system function.

[0008] The hot standby nodes are marked, and the mark of each hot standby node includes the group number of the hot standby node and the number within the group, wherein the hot standby node with the group number 1 and the number within the group 1 is the node that immediately participates in the work of the system after startup, that is, the working node;

[0009] Each hot standby node sends a heartbeat message to the outside; the working node in the hot standby node sends a work message to the outside in addition to the heartbeat message; when the working node is manually intervened, the hot standby node designated as the working node sends a switching work message to the outside; each hot standby node sends a heartbeat message once every 100ms; the working node sends a work message once every 100ms; when the working node is manually intervened, the designated working node sends a switching work message for 3 consecutive times;

[0010] Each hot backup node has two functions. The first function is the collection function, which is used to receive heartbeat messages, work messages and switching work messages from other hot backup nodes in the network; the second function is the judgment function, which is used to determine whether the working node is offline and whether the node needs to be started to participate in the system work; after the hot backup node is started, the collection function is turned on, and it waits for 3000ms before automatically turning on the judgment function.

[0011] The working steps of the method are:

[0012] Step 1, start the collection function of all hot backup nodes. For any hot backup node (G, N), send a heartbeat message immediately after startup and then send it through Ethernet with a period of 100ms. The content of the heartbeat message includes: heartbeat message identifier, group number, and group number, for example (0x10, G, N), indicating that the hot backup node is online; if the mark of the hot backup node is (1, 1), it will directly participate in the system work after startup. In addition to sending the above-mentioned heartbeat message, it will send a work message immediately after startup and then send it through Ethernet with a period of 100ms. The content of the work message includes: work message identifier, group number, and group number, such as (0x20, 1, 1), indicating that the hot backup node is participating in the system work.

[0013] When any hot backup node (G, N) receives a heartbeat message from another hot backup node, the content of the heartbeat message received by the hot backup node is written into the online node list of the node, and the information of the online node list includes: the group number, the internal number, the IP address, and the port number of the other hot backup nodes contained in the received heartbeat message, and a counter is assigned to each other hot backup node in the list, and when a heartbeat message from another hot backup node is received each time, the value of the corresponding counter is set to a positive integer, such as 10; in addition, any hot backup node (G, N) also writes its own group number, internal number, IP address, and port number into the online node list of the node;

[0014] When any hot standby node (G, N) of a non-working node receives a working message from another hot standby node, the hot standby node (G, N) of the non-working node records the content of the working message separately, including: group number, group number, address, port, and allocates a counter to the working node of the record, and sets the initial value of the counter to a positive integer, such as 10;

[0015] Step 2: Start the determination function of all hot backup nodes. Any hot backup node (G, N) reduces the counter value corresponding to other hot backup nodes in the online node list of this node by one every 100ms. When the counter value is reduced to 0, it is determined that the corresponding other hot backup node is offline. If the offline hot backup node determined is the current working node, any hot backup node (G, N) needs to select the next working node from the online node list of this node.

[0016] The process of selecting a working node is: compare all hot backup nodes in the online node list of this node, compare them one by one according to the group number, group number, IP address, and port, and select the working node. The comparison rules are:

[0017] First compare the group numbers and select the hot standby node with the smaller group number as the working node;

[0018] When the group numbers are the same, the numbers within the group are compared and the hot standby node with the smaller number is selected as the working node;

[0019] When the group number and the number within the group are the same, compare the IP addresses and select the hot standby node with the smaller IP address as the working node;

[0020] When the group number, group number, and IP address are the same, compare the port numbers and select the hot backup node with the smaller port number as the working node.

[0021] For example, there are two hot standby nodes in the online node list: Node A (A, B, IP1, Port 1) and Node B (C, D, IP2, Port 2). The comparison rule is:

[0022] If group number A < group number C, select node A as the working node;

[0023] If group number A = group number C, and group number B < group number D, then select node A as the working node;

[0024] If group number A = group number C, group number B = group number D, address IP1 < address IP2, then select node A as the working node;

[0025] If group number A = group number C, group number B = group number D, address IP1 = address IP2, port 1 < port 2, then select node A as the working node.

[0026] Otherwise, select node B as the working node.

[0027] If the selected working node is this node, this node participates in the system work, and this node sends work messages through Ethernet with a period of 100ms; if the selected working node is not this node, this node does not participate in the system work and only sends heartbeat messages.

[0028] In addition, the alternative process of step two is: start the judgment function of all hot backup nodes, manually intervene to designate a hot backup node of a non-working node as a working node, then the designated hot backup node participates in the system work, and sends three work switching messages continuously through Ethernet, and sends work messages with a period of 100ms; after other hot backup nodes receive the work switching message, they do not participate in the system work, only send heartbeat messages, and update the working node information being recorded based on the received work message.

[0029] The method may also include the following process:

[0030] For any hot standby node that is not a working node, if it receives working messages from two or more other hot standby nodes in succession, the working node is determined based on the working message received last;

[0031] For a working node, if a working message is received from another hot standby node, the working node is selected by comparing the current node with the hot standby node corresponding to the received working message according to the comparison rule;

[0032] The comparison rules are:

[0033] First compare the group numbers and select the hot standby node with the smaller group number as the working node;

[0034] When the group numbers are the same, the numbers within the group are compared and the hot standby node with the smaller number is selected as the working node;

[0035] When the group number and the number within the group are the same, compare the IP addresses and select the hot standby node with the smaller IP address as the working node;

[0036] When the group number, group number, and IP address are the same, compare the port numbers and select the hot backup node with the smaller port number as the working node.

[0037] A storage medium of the present invention stores instructions, and when a computer reads the instructions, the computer executes any one of the above-mentioned grouped multi-machine hot backup methods.

[0038] The beneficial effects of the present invention are as follows:

[0039] The grouped multi-machine hot backup method allows the working nodes of the system's key functions to quickly switch to other hot backup nodes as new working nodes when a failure occurs, thereby ensuring business continuity and stability; through this decentralized design without the need for an arbitration device, a series of problems caused by the failure of the arbitration device are solved; in addition, the present invention also supports dynamic increase or decrease of hot backup nodes, thereby enhancing the scalability of the system; multiple hot backup nodes run in parallel, thereby greatly improving the data protection capability; in summary, the present invention has the beneficial effects of ensuring business continuity, improving system flexibility, reducing the risk of data loss, etc., and provides strong support and guarantee for system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A working method of a group multi-machine hot backup method;

[0041] Figure 2 It is an alternative method for step 2 of a group multi-machine hot backup method;

[0042] Figure 3 The invention is a method for handling special situations of a group multi-machine hot backup method. DETAILED DESCRIPTION

[0043] like Figure 1-3 As shown, the method of the present invention is described below by means of specific embodiments:

[0044] The external communication of a certain system is completed by two links. Each link corresponds to a group of hot backup computers. Each group uses two computers as hot backup nodes. A total of four computers A, B, C, and D ensure communication. At the same time, only one computer participates in the system work to complete the system's external communication. The communication requirements are that the first group of hot backup computers communicate first. If the first group of hot backup nodes fail, the second group of hot backup nodes will take over its work.

[0045] First, assign hot standby node tags to the above four computers:

[0046] Computer A: Group number 1, group number 1, IP address 192.168.1.10, port 8000;

[0047] Computer B: group number 1, group number 2, IP address 192.168.1.11, port 8000;

[0048] Computer C: group number 2, group number 1, IP address 192.168.1.12, port 8000;

[0049] Computer D: group number 2, group number 2, IP address 192.168.1.13, port 8000;

[0050] After the four computers A, B, C, and D are started, computer A is group number 1 and numbered 1 within the group. After starting, it starts to participate in the system work, that is, as a working node. Computers B, C, and D enter the collection state after starting. After receiving the working message sent by computer A, they only send heartbeat messages, do not participate in the system work, and do not communicate with the outside of the system. Within 3 seconds of starting the collection function, each machine establishes an online node list and records the working node. After 3 seconds, each machine starts the judgment function;

[0051] The four computers A, B, C, and D send heartbeat messages at a period of 100ms, and computer A sends work messages in addition;

[0052] Computer A receives the heartbeat messages from computers B, C, and D, adds the information of computers A, B, C, and D to the local online node list of computer A, and allocates three corresponding counters to computers B, C, and D, setting the initial values ​​to 10.

[0053] Computer B receives the heartbeat messages from computers A, C, and D, adds the information of computers A, B, C, and D to the local online node list of computer B, and allocates three corresponding counters to computers A, C, and D, setting the initial value to 10. At the same time, computer B also receives the work message from computer A and records the work node as computer A.

[0054] Computer C and Computer D perform similar processing as Computer B.

[0055] Computers A, B, C, and D detect the offline status of the working node every 100ms, and reduce the counter value of other hot standby nodes in the online node list by 1; usually within 1 to 2 cycles, any computer can receive the heartbeat message sent by other computers, so that the counters of other computers in the local online node list are reset to 10;

[0056] When computer A fails, and the counter value of computer A in the local online node list of computers B, C, and D is reduced to 0, computers B, C, and D can determine that computer A is offline;

[0057] For computer B, it is determined that computer A is offline, and because computer A is a working node, computer B executes the working node selection process. In the local online node list of computer B, only computers B, C, and D are online. After comparison, the group number of computer B is 1, which is smaller than computer C and computer D. Therefore, the working node selects computer B, and then computer B participates in the system work, replacing computer A to perform external communication functions, and sends heartbeat messages and working messages at a period of 100ms;

[0058] For computer C, it is determined that computer A is offline, and computer A is a working node, so computer C executes the working node selection process. In the local online node list of computer C, only computers B, C, and D are online. After comparison, the group number of computer B is 1, which is smaller than computer C and computer D. Therefore, the working node selects computer B, so computer C does not participate in the system work and only sends heartbeat messages with a period of 100ms; at the same time, computer C receives the working message of computer B, and computer C updates the working node recorded locally;

[0059] For computer D, its processing process is the same as that of computer C. The final result is that computer B is selected as the working node. Computer D does not participate in the system work and only sends heartbeat messages with a period of 100ms.

[0060] If subsequent computers C or D fail, it will not affect computer B as a working node;

[0061] If computer B fails, computer C and computer D follow the process of selecting computer B as the working node. Computer C selects computer C as the working node to replace B to participate in system work and perform external communication functions.

[0062] In addition, when computers A, B, C, and D are all online, and computer A is a working node without any failure, and computer D is manually designated to participate in the system work, computer D sends three consecutive work switching messages, and then sends work messages at a period of 100ms;

[0063] After receiving the work switching message from computer D, computer A no longer participates in the system work and only sends heartbeat messages. At the same time, it updates the working node in the local record to computer D.

[0064] The processing of computers B and C is similar to that of computer A.

[0065] In particular, when computer A receives the work switching message from computer D, the normal situation is that after receiving the work message from computer A, computers B and C receive the work message from computer D, and computer A no longer sends work messages. In this way, the working node recorded locally by computers B and C is changed from computer A to computer D. However, due to a certain time error, computers B and C may first receive the work message from computer D and then receive the work message from computer A. At this time, the working node recorded by computers B and C is computer A, and an error occurs that the locally recorded working node is inconsistent with the actual working node. In this special case, it is only a short-term state. Since the actual state of computers A and D has changed, computer A will no longer continue to send work messages, while computer D will continue to send work messages. The working node recorded by computers B and C will eventually be updated to computer D, and the error will be automatically corrected.

[0066] The above-mentioned parameters and steps for each unit module to implement corresponding functions in a group multi-machine hot backup system of the present invention can refer to the parameters and steps in the embodiment of a group multi-machine hot backup method above, and will not be repeated here.

[0067] A storage medium according to an embodiment of the present invention is characterized in that instructions are stored in the storage medium, and when a computer reads the instructions, the computer executes any one of the above-mentioned group multi-machine hot backup methods.

[0068] Those skilled in the art will appreciate that the present invention may be implemented as a system, method or computer program product.

[0069] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present invention may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0070] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0071] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A group multi-machine hot backup method, characterized in that: include: The number of groups G of hot backup nodes and the number N of hot backup nodes in each group are determined for key functions in a system, where G and N are both positive integers; each hot backup node has a different IP address; all hot backup nodes are connected via Ethernet; and there is only one working node among the hot backup nodes that participates in system work and realizes system functions; Marking the hot standby nodes, wherein the mark of each hot standby node includes the group number of the hot standby node and the number within the group; Each hot standby node sends a heartbeat message to the outside; the working node in the hot standby node also sends a working message to the outside; when the working node is manually intervened, the hot standby node designated as the working node sends a switching working message to the outside; each hot standby node sends a heartbeat message to the outside at a certain period; the working node also sends a working message to the outside at a certain period; when the working node is manually intervened, the designated working node sends a switching working message to the outside continuously at least 2 times; Each hot backup node has two functions. The first function is the collection function, which is used to receive heartbeat messages, work messages and switching work messages from other hot backup nodes in the network; the second function is the judgment function, which is used to determine whether the working node is offline and whether the node needs to be started to participate in the system work; after the hot backup node is started, the collection function is turned on first, and then the judgment function is automatically turned on after a period of time.

2. The method according to claim 1, characterized in that The hot standby node with the group number set to 1 and the number within the group set to 1 is a node that immediately participates in the work of the system after startup, that is, a working node.

3. The method according to claim 1, characterized in that The specific working steps of the method described are: Step 1, start the collection function of all hot standby nodes, for any hot standby node (G, N), immediately after starting and then send a heartbeat message via Ethernet at a certain period, the heartbeat message content includes: heartbeat message identifier, group number, group number, indicating that the hot standby node is online; When any hot standby node (G, N) receives a heartbeat message from another hot standby node, the content of the heartbeat message received by the hot standby node is written into the online node list of the node, wherein the information of the online node list includes: the group number, the internal number, the IP address, and the port number of the other hot standby nodes contained in the received heartbeat message, and a counter is assigned to each other hot standby node in the list, and when a heartbeat message from another hot standby node is received each time, the value of the corresponding counter is set to a positive integer; in addition, any hot standby node (G, N) also writes its own group number, internal number, IP address, and port number into the online node list of the node; When any hot standby node (G, N) of a non-working node receives a working message from another hot standby node, the hot standby node (G, N) of the non-working node records the content of the working message separately, including: group number, group number, address, port, and allocates a counter to the working node of the record, and sets the initial value of the counter to a positive integer; Step 2: Start the determination function of all hot backup nodes. Any hot backup node (G, N) reduces the counter value corresponding to other hot backup nodes in the online node list of the node by one at a certain period. When the counter value is reduced to 0, the corresponding other hot backup node is determined to be offline. If the determined offline hot backup node is the current working node, any hot backup node (G, N) needs to select the next working node from the online node list of the node. If the selected working node is this node, this node updates the recorded working node and participates in the system work, and this node sends work messages through Ethernet at a certain period; if the selected working node is not this node, this node does not participate in the system work and only sends heartbeat messages.

4. The method according to claim 3, characterized in that The process of selecting the next working node from the online node list of the current node in step 2 is as follows: Compare all hot backup nodes in the online node list of this node, and select the working node based on the group number, group number, IP address, and port. The comparison rules are as follows: First compare the group numbers and select the hot standby node with the smaller group number as the working node; When the group numbers are the same, the numbers within the group are compared and the hot standby node with the smaller number is selected as the working node; When the group number and the number within the group are the same, compare the IP addresses and select the hot standby node with the smaller IP address as the working node; When the group number, group number, and IP address are the same, compare the port numbers and select the hot backup node with the smaller port number as the working node.

5. The method according to claim 3, characterized in that: The process of the replacement of step 2 is as follows: starting the determination function of all hot standby nodes, manually intervening to designate a hot standby node of a non-working node as a working node, and then the designated hot standby node participates in the system work, and at the same time continuously sends at least 2 switching work messages to the outside through Ethernet, and sends work messages at a certain period; After receiving the work switching message, other hot standby nodes do not participate in the system work, only send heartbeat messages, and update the recorded working node information based on the received work message.

6. The method according to any one of claims 3 to 5, characterized in that: The method further comprises the following process: For any hot standby node that is a non-working node, when it receives working messages from two or more other hot standby nodes in succession, the working node is determined based on the working message received last; For a working node, if it receives a working message from another hot standby node, it compares the current node with the hot standby node corresponding to the received working message according to the comparison rule and selects the working node. The comparison rules are: First compare the group numbers and select the hot standby node with the smaller group number as the working node; When the group numbers are the same, the numbers within the group are compared and the hot standby node with the smaller number is selected as the working node; When the group number and the number within the group are the same, compare the IP addresses and select the hot standby node with the smaller IP address as the working node; When the group number, group number, and IP address are the same, compare the port numbers and select the hot backup node with the smaller port number as the working node.

7. The method according to any one of claims 1-6, wherein the certain period is 100 ms and the period of time is 3000 ms.

8. A grouped multi-machine hot standby system using the method described in claim 7.

9. A storage medium, characterized in that: The storage medium stores instructions, and when a computer reads the instructions, the computer executes the method described in any one of claims 1 to 7.

Images