High availability method based on parallel management of mock mechanism

Abstract

The invention discloses a high-availability method based on parallel management of a mock mechanism, which comprises the following steps of: completing the construction of a dependency relationship between transmission control and main / standby synchronization confirmation and disk dropping completion confirmation, regularly checking main / standby synchronization and disk dropping completion conditions by a transmitting end, setting a platform to be in a trigger mode when the pressure is low, and performing service non-perception transmission control. Primary and standby synchronization and disk dropping of service processing data and data are carried out in parallel, a primary and standby synchronization channel is established between primary and standby nodes in a cluster, a communication protocol determines that a standby point replies a confirmation message when receiving primary point synchronization data, a mock mechanism is established, and a receiving node without a synchronization channel actually sending data also replies the confirmation message. According to the method, main and standby point logics are not distinguished outside a communication layer, during main and standby switching, breakpoint interaction does not need to be carried out on cluster management, logic switching can be completed, the problem that high-availability confirmation and services are mutually dependent is solved, cluster processing time delay, service processing state independence and cluster state management seamless switching are reduced, and the overall performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster management, and in particular to a high-availability method based on parallel management of a mock mechanism. Background Art

[0002] In order to meet the high availability requirements of the financial system, important nodes are often deployed in clusters. Among them, the cluster that queues multiple input source data and synchronizes the master and the standby is the most common cluster model. The data consistency of this cluster model requires the master point to synchronize the data to be processed to multiple standby nodes in the cluster. After receiving the confirmation of the standby point receiving the data and the data is persisted on the disk, business processing can be carried out. In addition, the breakpoint interaction of the new master and standby nodes during switching ensures that the data is not lost or disordered after the cluster switch. The mock mechanism comes from the common method used in software development and testing. It constructs or obtains objects and services in a simulated way. Here, a channel is established for the standby point to send data to the master point. Since the actual standby point does not send data, it is simulated to "send data". The subsequent logic is the same as the processing logic of the master point.

[0003] From the above, we can see that the high availability guarantee of the cluster's overall external services and the business processing of the nodes often adopt the simplest serial mechanism, that is, the data input by the cluster must be received by the backup point and persisted on the disk before business processing can be carried out, and then the processing results can be sent to the outside; or the business processing needs to provide additional high-availability confirmation logic to improve parallelism, which to a certain extent increases the complexity and coupling of business processing. At the same time, when switching between primary and backup nodes, cluster nodes also need to perform complex breakpoint interaction logic to ensure the consistency of data processed by the new primary node. The biggest problem with the above is that the internal processing delay of the cluster is large, and the sending control of the business and the breakpoint interaction of the cluster are highly coupled, which is time-consuming, labor-intensive and error-prone. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art that the internal processing delay of the cluster is large, the sending control of the service and the breakpoint interaction coupling of the cluster are highly interactive, time-consuming, labor-intensive and error-prone.

[0005] In order to solve the above problems, the present invention provides a high availability method based on mock mechanism parallel management, comprising the following steps:

[0006] Step 1: Build a dependency relationship between the data to be sent and the completion of master-slave synchronization and disk storage. That is, the data sending module registers a return function to obtain the sequence number of the data in the input data queue that has completed master-slave synchronization and disk storage.

[0007] Step 2: Implement sending control;

[0008] The return function obtains the data sequence number that has completed the master-slave synchronization and disk storage. Here, the algebraic ack_seq is used to represent the data sequence number. When a business generates data to be sent, the sequence number of the input data corresponding to the data to be sent must be less than ack_seq before it can be sent.

[0009] Step 3: The data sending module starts the timer, regularly checks the sending control of step 2, and completes the sending of the data to be sent;

[0010] Step 4: When the master-slave synchronization and disk storage are completed under low pressure, the sending data module can be notified through asynchronous events to trigger the completion of the sending control calculation described in step 2, and the external sending of business processing data can be completed;

[0011] Step 5: Based on step 2, the service-unaware sending control is established;

[0012] Step 2: The master node in the cluster establishes a master-slave synchronization channel for all standby nodes through TCP or UDP protocol, and sends the data queued by the master node. When receiving the synchronization data from the master node, the standby node will reply with a confirmation message, that is, the queue number of the input data; establish a mock mechanism, that is, the standby node establishes a synchronization channel without actually sending data, and the receiving node of the channel, the master node and other standby nodes also reply with the existing data volume to simulate the reception of the channel;

[0013] In a preferred embodiment, the mock mechanism is that a backup point without synchronization data also establishes a master-slave synchronization channel, and the corresponding receiving end continuously replies to the received data point.

[0014] In a preferred embodiment, the event triggering mode is to actively send an asynchronous event notification when the master-slave synchronization is confirmed to be completed and the disk placement is confirmed to be completed.

[0015] In a preferred manner, low pressure means that the amount of input data per unit time is small, that is, the amount of input data written to the main disk per unit time is small.

[0016] In the preferred mode, the imperceptible sending control is to perform the master-slave synchronization and disk placement of the data in the input queue, the business processing data and the data in parallel, and complete the output control of the data to be sent generated by the business.

[0017] In the preferred mode, the logic of the master and backup points is no longer distinguished except for the synchronization channel established in step 5, and the sending control and business processing are not aware of it; when the master and backup switch is performed, the cluster management does not need to interact with the breakpoints to complete the switching logic. Asynchronous event notification is a common solution in the reactor model of the software industry.

[0018] The beneficial effects of the present invention are as follows: solving the interdependence problem between high availability confirmation and business processing, reducing the internal processing delay of the cluster, business processing status independence, seamless switching of cluster status management, and improving overall performance.

[0019] 1. Based on the sending control mechanism, business processing is synchronized with the cluster, and the logic of persistence and disk placement is parallel. The dependency between high availability and business is postponed to the time of sending. The batch processing of each pipeline output greatly improves the cluster processing performance.

[0020] 2. You can choose between timer or event triggering, which are used for control timeliness and balance performance requirements under high and low pressure conditions respectively.

[0021] 3. Cluster confirmation based on the mock mechanism does not require any breakpoint interaction, truly achieving seamless state switching, simple and efficient.

[0022] 4. The solution is completely encapsulated within the platform framework, with no perception of business processing and easy use. More sending control dependencies can be added by simply setting up, and it has good scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the main process of data processing in the current cluster;

[0024] Figure 2 This is a schematic diagram of the current cluster state switching breakpoint interaction;

[0025] Figure 3 This is a schematic diagram of a high-availability method based on parallel management of the mock mechanism. DETAILED DESCRIPTION

[0026] Example 1: Figure 1 , Figure 2 , Figure 3 As shown, the present invention provides a high-availability rack based on parallel management of the mock mechanism, and the specific steps are as follows:

[0027] A confirmation dependency relationship between cluster external transmission and completion of master-slave synchronization and disk storage is established, that is, cluster input data must be received and confirmed by the backup point and persisted on the disk before the generated business output can be sent externally. The business no longer participates in high-availability management and performs business processing independently, without the existence of perceived transmission control, so as to isolate the coupling relationship between input and output data and business processing. At the same time, business processing is parallel to master-slave synchronization confirmation and disk storage completion, and control is only performed when sending externally to achieve high availability of the cluster. It greatly improves latency performance and business independence, and makes control simpler and more efficient.

[0028] In high-availability clusters, it is common for the master node to establish a master-slave synchronization channel, queue the received multi-channel data and synchronize it to the backup node to ensure the high availability consistency of the cluster. This method cleverly designs a mock mechanism for master-slave synchronization, that is, the backup node without synchronized data also establishes a master-slave synchronization channel, and its corresponding receiving end also continuously replies to the received data point. This simulated fake communication pipeline is called mock. Based on this mock mechanism, this method isolates breakpoints at the communication layer, greatly simplifying the breakpoint maintenance logic during cluster switching. At the same time, the business logic of the master and backup nodes is made independent of the state, making cluster management simple and efficient, and truly achieving seamless switching.

[0029] Step 1: The platform framework completes the construction of the dependency relationship between sending control and the confirmation of master-slave synchronization and disk placement. That is, the data sent externally depends on the completion of master-slave synchronization confirmation and disk placement.

[0030] Step 2: The sending end of the platform framework uses a timer to periodically check the master-slave synchronization and disk placement completion status, and controls that only data that has been confirmed to be completed by the master-slave synchronization and disk placement can be sent externally.

[0031] Step 3: The platform framework can also use event-triggered mode under low pressure, that is, when the master-slave synchronization confirmation is completed and the disk is completed and confirmed, it will actively send an asynchronous event notification. After receiving the notification, the sending thread triggers a sending control calculation to complete the external sending of business processing data.

[0032] Step 4: Based on the above steps and methods, the business is not aware of the sending control, and the business processing data and the primary and backup synchronization and storage of the data are carried out in parallel, which greatly improves the processing efficiency of the cluster's high availability.

[0033] Step 5: A master-slave synchronization channel is established between the master and slave nodes in the cluster. The communication protocol itself determines that the slave node will reply with a confirmation message when it receives the synchronization data from the master node. With the established mock mechanism, other receiving nodes of the synchronization channel that do not actually send data also "reply" with the existing data volume to simulate the reception of communication.

[0034] Step 6: There is no distinction between the primary and standby logic outside the communication layer, such as sending control and business processing. At the same time, when the primary and standby switch is performed, the cluster management can complete the switching logic without breakpoint interaction. For example, the new primary point can automatically start sending new data through the corresponding "reply" breakpoint, without generating dirty data, and without additional breakpoint interaction logic, making cluster switching management convenient and efficient.

[0035] The following are Figure 3 Logical process of data processing:

[0036] 1. The master node of the cluster receives multiple data sources (n data sources) and queues them together and puts them into the receiving queue.

[0037] 2. The data in the queue is processed in parallel by three consumers (threads):

[0038] 2-1, business processing thread, reads queue data for business processing, and puts the sent data generated after processing into the send data queue;

[0039] 2-2. Persistent disk write thread, write queue data to disk;

[0040] 2-3. The master-slave synchronization thread sends the data in the queue to the backup point (possibly multiple backup points);

[0041] 3. The backup node receives the data synchronized by the primary node and replies with confirmation (data point received);

[0042] 4. The master point completes the sending control calculation through (timer and event notification mechanism) and sends out the output data generated by the backup point confirmation and data after disk.

[0043] 5. For the backup point, the data synchronized by the primary point is placed in its own receiving data queue, and the data is also processed in parallel internally:

[0044] 5-1, business processing thread, reads queue data for business processing, and puts the sent data generated after processing into the send data queue;

[0045] 5-2. Persistent disk write thread, write queue data to disk;

[0046] 5-3. The master-slave synchronization thread also establishes a communication transmission channel with the master point and another backup point, which is called mock mode transmission;

[0047] 6. The primary point will reply to the backup point for confirmation (the data points it owns) at a fixed time or after accumulating a certain number of times. This reply confirmation method is called mock mode confirmation.

[0048] 7. The backup node also establishes a synchronous sending channel (mock) and the corresponding receiving end (primary node and another backup node) replies with confirmation (mock) based on its own data volume. This mechanism ensures the independence of the primary and backup states of node processing and greatly simplifies cluster state switching and breakpoint management.

[0049] 7-1. The synchronization module of the backup point can also receive confirmation from other receiving points. The business processing and sending control are consistent with the operation of the main point, achieving state independence.

[0050] 7-2. When the primary point fails, the backup point with a larger amount of data is elected as the new primary point. The new primary point continues to send synchronization data from the confirmation points replied by other receiving points.

[0051] Embodiment 2:

[0052] Step 1: Build a dependency relationship between the data to be sent and the completion of master-slave synchronization and disk storage. That is, the data sending module registers a return function to obtain the sequence number of the data in the input data queue that has completed master-slave synchronization and disk storage.

[0053] Step 2: Implement sending control;

[0054] The return function obtains the data sequence number that has completed the master-slave synchronization and disk storage. Here, the algebraic ack_seq is used to represent the data sequence number. When a business generates data to be sent, the sequence number of the input data corresponding to the data to be sent must be less than ack_seq before it can be sent.

[0055] Step 3: The data sending module starts the timer, regularly checks the sending control of step 2, and completes the sending of the data to be sent;

[0056] Step 4: When the master-slave synchronization and disk storage are completed under low pressure, the sending data module can be notified through asynchronous events to trigger the completion of the sending control calculation described in step 2, and the external sending of business processing data can be completed;

[0057] Step 5: Based on step 2, the service-unaware sending control is established;

[0058] Step 2: The master node in the cluster establishes a master-slave synchronization channel for all standby nodes through the TCP or UDP protocol, and sends the data queued by the master node. When receiving the synchronization data from the master node, the standby node will reply with a confirmation message, that is, the queue number of the input data; a mock mechanism is established, that is, the standby node establishes a synchronization channel without actually sending data, and the receiving node of the channel, the master node and other standby nodes also reply with the existing data volume to simulate the reception of the channel.

[0059] The mock mechanism is that a backup point without synchronization data also establishes a master-slave synchronization channel, and the corresponding receiving end continuously replies to the received data point.

[0060] The event triggering mode is to actively send an asynchronous event notification when the master-slave synchronization confirmation is completed and the disk is completed.

[0061] Low pressure means less input data per unit time.

[0062] The imperceptible sending control is to synchronize the data in the input queue, the business processing data and the data between the master and the backup, and to store them on the disk in parallel. The framework completes the output control of the data to be sent generated by the business.

[0063] Except for the synchronization channel established in step 5, the logic of the primary and backup points is no longer distinguished, and the sending control and business processing are unaware; when performing the primary and backup switching, the cluster management no longer needs to interact with the breakpoints to complete the switching logic.

[0064] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A high availability method based on mock mechanism parallel management, characterized in that: The steps include: Step 1: Build a dependency relationship between the data to be sent and the completion of master-slave synchronization and disk storage. That is, the data sending module registers a return function to obtain the sequence number of the data in the input data queue that has completed master-slave synchronization and disk storage. Step 2: Implement sending control; The return function obtains the data sequence number that has completed the master-slave synchronization and disk storage. Here, the algebraic ack_seq is used to represent the data sequence number. When a business generates data to be sent, the sequence number of the input data corresponding to the data to be sent must be less than ack_seq before it can be sent. Step 3: The data sending module starts the timer, regularly checks the sending control of step 2, and completes the sending of the data to be sent; Step 4: When the master-slave synchronization and disk storage are completed under low pressure, the sending data module can be notified through asynchronous events to trigger the completion of the sending control calculation described in step 2, and the external sending of business processing data can be completed; Step 5: Based on step 2, the service-unaware sending control is established; Step 2: The master node in the cluster establishes a master-slave synchronization channel for all standby nodes through the TCP or UDP protocol, and sends the data queued by the master node. When receiving the synchronization data from the master node, the standby node will reply with a confirmation message, that is, the queue number of the input data; a mock mechanism is established, that is, the standby node establishes a synchronization channel without actually sending data, and the receiving node of the channel, the master node and other standby nodes also reply with the existing data volume to simulate the reception of the channel.

2. According to claim 1, the high availability method based on parallel management of mock mechanism is characterized in that: The mock mechanism is that a backup point without synchronization data also establishes a master-slave synchronization channel, and the corresponding receiving end continuously replies to the received data point.

3. According to claim 1, the high availability method based on parallel management of mock mechanism is characterized in that: The event triggering mode is to actively send an asynchronous event notification when the master-slave synchronization confirmation is completed and the disk placement is completed.

4. According to claim 1, the high availability method based on parallel management of mock mechanism is characterized in that: Low pressure means less input data per unit time.

5. The high availability method based on parallel management of mock mechanism according to claim 1 is characterized in that: The imperceptible sending control is to synchronize the data in the input queue, the business processing data and the data between the master and the backup, and to store them on the disk in parallel. The framework completes the output control of the data to be sent generated by the business.

6. The high availability method based on parallel management of mock mechanism according to claim 1 is characterized in that: Except for the synchronization channel established in step 5, the logic of the primary and backup points is no longer distinguished, and the sending control and business processing are unaware; when performing the primary and backup switching, the cluster management no longer needs to interact with the breakpoints to complete the switching logic.

Images