Micro-service distributed consistency method suitable for weak network environment

Abstract

The invention discloses a micro-service distributed consistency method suitable for a weak network environment, and the method comprises the steps: defining network nodes, including inactive nodes, active nodes and appointment nodes, constructing an appointment mode and a non-appointment mode, maintaining the data consistency of a whole micro-service registration center cluster, and carrying out the initial configuration of the cluster and the appointment nodes in the appointment mode; the active node sends a pre-detection packet and selects an optimal appointment node; the active node compresses the data and the version number and sends the data and the version number to the appointment node, and only sends the version number in the second period and after; the dating node collects data of all active nodes in a synchronization period and sends the data to other nodes in the cluster; other nodes establish lease or continuation storage after receiving the data; in the non-appointment mode, the active node compresses the data and the version number and sends the data and the version number to other nodes of the cluster, and only the version number is sent in the second period and later; and other nodes establish lease or continuation storage after receiving the data. According to the invention, the micro-service registration center cluster can still be efficiently synchronized in an environment with limited bandwidth and unstable network connection.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more specifically, to a microservice distributed consistency method applicable to weak network environments. Background Art

[0002] In a microservice registry cluster, servers are connected through a network to form a logical cluster. In a weak network environment, there will be situations such as network packet loss, large latency, and network disconnection in the cluster, resulting in frequent offline of cluster nodes or even grouping. At this time, it is necessary to ensure that the cluster data is synchronized as normally as possible, and when the node recovers from being out of the group, it can still join the cluster to work and maintain the same state as the cluster.

[0003] As a distributed consistency protocol, the Raft algorithm is widely used in the core components of various distributed microservices. However, in a weak network environment, the Raft algorithm has some significant disadvantages:

[0004] 1. Requirement for more than half of the nodes to be online. The Raft algorithm requires more than half of the nodes to be working properly to perform data updates and synchronization. This means that in an environment with unstable network conditions or frequent offline of nodes, the system may not be able to reach a sufficient proportion of online nodes, resulting in service unavailability or delayed processing of requests.

[0005] 2. Heartbeat election and synchronization consume bandwidth. To maintain the leader state of the cluster and achieve data consistency, the Raft algorithm conducts leader election and data synchronization by periodically sending heartbeat packets and log entries. In an environment with poor network conditions, this frequent network communication will consume a large amount of bandwidth resources, further reducing the effective load capacity of the network.

[0006] 3. High requirement for network performance. The Raft algorithm relies on the low-latency characteristics of the network to quickly complete key operations such as leader election and log replication. In a weak network environment, the high network latency will directly affect the efficiency of the Raft algorithm, increase the time for election and the latency of data synchronization, and even cause the cluster to be in an election state all the time, thus affecting the response speed and stability of the entire system.

[0007] 4. High requirement for disk performance. When the Raft algorithm performs log replication and state machine submission, it needs to perform frequent disk read and write operations. In the case of poor disk performance, these operations will become the bottleneck of system performance. Especially in scenarios where a large number of write operations need to be processed, the disk I / O performance directly determines the speed of data synchronization and the overall performance of the system.

[0008] These limitations make the application effect of the Raft algorithm in microservice clusters with poor network conditions or limited resources greatly reduced. Summary of the Invention

[0009] The object of the present invention is to propose a microservice distributed consistency method applicable to weak network environments, so as to solve the problems in the prior art that in the case of poor network environments, the microservice registry cluster synchronization is extremely unstable and the availability is low.

[0010] The technical solution for achieving the object of the present invention is: a microservice distributed consistency method applicable to weak network environments, the steps are as follows:

[0011] (1) Define network nodes including inactive nodes, active nodes and rendezvous nodes, where:

[0012] Active node: A node in the microservice registry cluster that has service data. The active node will send the service data on this node to the rendezvous node in each synchronization cycle;

[0013] Inactive node: A node in the microservice registry cluster that does not have service data and will only receive data during the synchronization process;

[0014] Rendezvous node: In the initialization stage, configure the nodes in the cluster with low latency and packet loss rate as the rendezvous node. The rendezvous node receives the local data from the active nodes within a synchronization cycle and integrates and periodically sends these data to other nodes in the cluster;

[0015] (2) Construct a rendezvous mode and a non-rendezvous mode to keep the data of the entire microservice registry cluster consistent, where:

[0016] (1) Rendezvous mode, the specific process is:

[0017] (11) Initialization configuration of the cluster and the rendezvous node, including the IP of the nodes in the cluster and the rendezvous node marker;

[0018] (12) The active node sends a pre-detection packet to select the optimal rendezvous node;

[0019] (13) The active node compresses the data and the version number and sends them to the rendezvous node. Only the version number is sent in the second cycle and later;

[0020] (14) The rendezvous node collects the data of all active nodes within a synchronization cycle and sends them to other nodes in the cluster;

[0021] (15) Other nodes establish a lease or renew the lease and store it after receiving the data;

[0022] (2) Non-rendezvous mode, the specific process is:

[0023] (21) The active node compresses the data and the version number and sends them to other nodes in the cluster. Only the version number is sent in the second cycle and later;

[0024] (22) After other nodes receive data, they establish or renew lease storage.

[0025] Further, the active node sends a pre-detection packet and selects the optimal rendezvous node. Specifically:

[0026] Before the active node sends service data to the rendezvous node, it sends a 1-byte pre-detection packet to all rendezvous nodes. The content of the pre-detection packet is any one in the ASCII code. If the rendezvous node receives this pre-detection packet, it will immediately reply with the same 1-byte data to the active node. The active node first compares whether the received reply is the same as the pre-detection data it sent before, to prevent it from being data sent in other cycles. If they are the same, it will select the rendezvous node with the fastest reply to send service data.

[0027] Further, after other nodes receive data, they establish or renew lease storage. Specifically:

[0028] The data received remotely has a valid time, which is called the lease time. The lease time is greater than twice the synchronization period and less than three times the synchronization period. After the lease is refreshed, the lease time is reset. If the service data of the remote node does not refresh the lease within the lease time, it is considered that the service has gone offline and the information of this service should not be queried.

[0029] Further, after other nodes receive data, if they find that the received data version numbers are inconsistent, they will actively request the corresponding active node to obtain the latest version number and data.

[0030] Further, the period for the active node to send data is the same as the period for the rendezvous node to collect data.

[0031] Further, if the data sent by the rendezvous node is mixed data of multiple nodes, when sending it to the active node, the data containing this active node in the mixed data will be deleted.

[0032] Further, the rendezvous mode and non-rendezvous mode are automatically switched according to the situation. The specific switching principles are as follows:

[0033] (1) If no rendezvous node is configured, enter the non-rendezvous mode;

[0034] (2) If a rendezvous node is configured and the rendezvous node is not out of the group for the active node, enter the rendezvous mode;

[0035] (3) If a rendezvous node is configured and the rendezvous node is out of the group for the active node, enter the non-rendezvous mode.

[0036] Further, if the cluster is grouped, the group with a rendezvous node will enter the rendezvous mode, and the group without a rendezvous node will enter the non-rendezvous mode.

[0037] Further, the Zlib algorithm is used to compress data.

[0038] Compared with the prior art, the present invention has the following remarkable advantages: aiming at the network latency and frequent outlier problems in the mobile weak network environment, a large amount of compression is performed on the synchronized data in the cluster. At the same time, the setting of leases and cycles is introduced, which enhances the fault tolerance for network latency and packet loss and synchronizes data quickly, and can keep the data of the entire microservice registry cluster as up-to-date and consistent as possible; considering the diversity and uncertainty of the network environment, the microservice registry cluster can still synchronize efficiently in an environment with limited bandwidth and unstable network connections. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the synchronization of the dating mode of the present invention.

[0040] Figure 2 It is a schematic diagram of the synchronization of the non-dating mode of the present invention.

[0041] Figure 3 It is a schematic diagram of the method flow of the present invention.

[0042] Figure 4 It is a schematic diagram of the beneficial effects of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0043] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The present invention is a distributed consistency method applicable to the microservice network in a weak network environment. The network nodes are divided into inactive nodes, active nodes and dating nodes.

[0045] Active nodes and inactive nodes: In the microservice registry cluster, service data does not exist on all network nodes. The nodes with service data are called active nodes, and active nodes will send the service data on their own nodes to the dating nodes in each synchronization cycle; the nodes without service data are called inactive nodes, and there will only be actions to receive data during the synchronization process.

[0046] Dating Node: In the initialization phase, nodes in the cluster with relatively low latency and packet loss rate are configured as dating nodes. If the latency and packet loss rate are basically the same, any 2 nodes in the cluster are randomly selected as dating nodes. The role of dating nodes is to receive local data from active nodes within a synchronization period, integrate this data, and periodically send it to other nodes in the cluster. Dating nodes are not necessary. If dating nodes do not exist, the cluster will enter the non-dating mode.

[0047] The described distributed consensus method includes two modes: dating and non-dating.

[0048] The dating mode is as Figure 1 shown and includes the following steps:

[0049] (1) Initial configuration of the cluster and dating nodes, including the IP addresses of nodes in the cluster and dating node markers. In the present invention, XML is used as the configuration file format;

[0050] (2) Active nodes send pre-detection packets to select the optimal dating node. The specific implementation method is as follows: Before an active node sends service data to a dating node, it will first send a 1-byte pre-detection packet to all dating nodes. The content of this packet is any one in the ASCII code. At the same time, if a dating node receives this pre-detection packet, it will immediately reply with the same 1-byte data to the active node. The active node will first compare whether the received reply is the same as the pre-detection data it sent before, to prevent it from being data sent in other cycles. If they are the same, it will select the dating node with the fastest reply to send service data;

[0051] (3) Active nodes use the Zlib algorithm to compress data and the version number and send them to the dating node. Only the version number is sent in the second cycle and subsequent cycles;

[0052] (4) Dating nodes collect data from all active nodes within a synchronization period and send it to other nodes in the cluster;

[0053] (5) Other nodes except this dating node establish a lease or renew the lease storage after receiving the data. The specific design of the lease is as follows: The data received remotely has a valid time, which is called the lease time. The lease time is slightly longer than twice the synchronization time, which can allow one data packet to be lost during synchronization and increase the tolerance for network latency. After refreshing the lease, the lease time is reset. The reason for designing the lease is because of the real-time nature of the microservice framework. If the service data of a remote node does not refresh the lease within the lease time, it is considered that the service has gone offline and the information of this service should not be queried.

[0054] The non-dating mode is as Figure 2 shown and includes the following steps:

[0055] (1) The active nodes compress the data and send the version number to other nodes in the cluster. In the second cycle and subsequent cycles, only the version number is sent.

[0056] (2) After other nodes receive the data, they establish a lease or renew the storage.

[0057] In steps 3 and 4 of the above dating mode and step 1 of the non-dating mode, the data sent is compressed using the Zlib algorithm compression algorithm. At the same time, in steps 5 of the above dating mode and step 2 of the non-dating mode, the lease time is greater than two synchronization cycles and less than three synchronization cycles. If a node finds that the version numbers of the received data are inconsistent, it will actively request the corresponding active node to obtain the latest version number and data.

[0058] The distributed consensus method will automatically switch between the dating mode and the non-dating mode according to the situation, and the specific situation is as follows:

[0059] (1) If no dating nodes are configured, enter the non-dating mode;

[0060] (2) If dating nodes are configured and the dating nodes are not out of the group for the active nodes, enter the dating mode;

[0061] (3) If dating nodes are configured and the dating nodes are out of the group for the active nodes, enter the non-dating mode.

[0062] Among them, if dating nodes are configured, the nodes with relatively low latency and packet loss rate should be configured.

[0063] The schematic diagram of the process of the distributed consensus method is as Figure 3 shown.

[0064] As Figure 4 shown, the distributed consensus method greatly compresses the synchronized data and the number of connections in the cluster, and introduces the setting of leases and cycles, enhancing the fault tolerance for network latency and packet loss, and enabling the microservice registry cluster to still be able to synchronize efficiently in an environment with limited bandwidth and unstable network connections.

[0065] The following takes a specific number of nodes as an example to further explain the distributed consensus method of the present invention.

[0066] Suppose there are 12 nodes from A to L in a microservice registry cluster, and nodes A and B are selected as dating nodes because of their relatively low latency and packet loss rate.

[0067] At the beginning, all nodes have no service information. At this time, there is no data exchange in the cluster, that is, there is no bandwidth consumption in the cluster when it is idle.

[0068] Service S is registered to node C, and node C becomes the active node. At this time, synchronization begins. Node C will first send a 1-byte random ASCII pre-detection packet, such as ".", to all appointment nodes (A and B). After receiving the pre-detection packet, nodes A and B will immediately reply "." to node C. Node C will first compare whether the reply is "." to prevent the reply of pre-detection packets sent in other cycles. Then, according to the reply time, it will select the node with the fastest reply for data transmission. If there is no reply to the pre-detection packet from A and B within the specified time, it is considered that there is no appointment node online, and it will enter the no-appointment mode. The active node will send the version number and data to other nodes in the cluster in a similar broadcast manner.

[0069] Node C sends a data packet to the preferred appointment node (suppose it is node A). The data sent includes the version number and data content of this data. The data is the data after being compressed using a compression method.

[0070] Node A will continuously receive data within a synchronization cycle, and then synchronize the data received within this cycle to all other nodes, including appointment nodes. The cycle for the active node to send data is the same as the cycle for the appointment node to collect data, which can ensure that no duplicate data is received within the same cycle. At the same time, if the data sent by the appointment node is a mixed data of multiple nodes, the data containing this active node in the mixed data will be deleted when sent to the active node.

[0071] At the end of the first synchronization, each node can query the data of service S, and the remote data has a lease, and the lease time is greater than two synchronization cycles. If the lease is not refreshed within two cycles, the data will become invalid and the service is considered to be offline. This is because of the real-time nature of microservices, and there cannot be an expired service data in the service list. After that, if the data of node C does not change, only the version number will be synchronized in each subsequent synchronization cycle. If a node has just recovered to the cluster or network packet loss causes it not to receive the latest data of node C, and it is found that the version number is missing or inconsistent when comparing the version numbers, it will actively request the latest data from node C once.

[0072] When the data of the active node C changes, it will immediately perform a synchronization and update the version number. At this time, the data synchronized by the active node C is incremental data, that is, it will include the addition and subtraction operations of the data, the updated data version number, and the version number before the update. When other nodes receive the synchronized data, they will first compare whether their version is the same as the version number before the updated data. If they are the same, they will update according to the addition and subtraction operations. If they are different, they will actively request the latest data from the active node.

[0073] In a weak network environment, the cluster may be grouped. If the cluster is grouped, the group with the appointment node still synchronizes according to the appointment node mechanism, and the group without the appointment node will enter the no-appointment mode, that is, the active node directly sends the data or version number of this node to other nodes in the cluster in a form similar to broadcasting, such as Figure 2 shown.

[0074] If the groups are merged, since the data has a version number, and since the modification of the data in the microservice registry only occurs at the data generation end, that is, each node does not modify the data of other nodes, there will be no data conflict after the merge.

[0075] The lease time of the remote data is greater than twice the synchronization time. If the synchronization time is 2 seconds, the lease time can be set to 5 seconds. At this time, the remote data is allowed to have a 3-second delay or 1 packet loss. If only 1 packet is lost, the lease refresh time changes from one cycle to 2 cycles, 4 seconds. At this time, the lease is still valid, and the service data will not be repeatedly invalidated and rebuilt due to packet loss. However, if two or more packets are lost, it is considered that the service has gone offline, and the real-time nature of the microservice is followed.

[0076] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0077] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A distributed consistency method for microservices in a weak network environment, characterized in that: Here are the steps: (I) Definition of network nodes: Inactive nodes, active nodes and dating nodes, among which: Active node: A node with service data in the microservice registry cluster. The active node will send the service data on this node to the dating node in each synchronization cycle; Inactive nodes: nodes in the microservice registry cluster that do not contain service data. During the synchronization process, only data is received. Dating nodes: During the initialization phase, nodes with low latency and packet loss rates in the cluster are configured as dating nodes. Dating nodes receive local data from active nodes within a synchronization cycle and periodically integrate these data to other nodes in the cluster. (II) Construct the appointment mode and non-appointment mode to keep the data of the entire microservice registry cluster consistent, including: (1) Dating mode: The specific process is as follows: (11) Initial configuration of cluster and dating nodes, including IP addresses of nodes in the cluster and dating node tags; (12) The active node sends a pre-detection packet and selects the optimal dating node; (13) The active node compresses the data and the version number and sends them to the meeting node. In the second cycle and thereafter, only the version number is sent; (14) The dating node collects data from all active nodes within a synchronization cycle and sends it to other nodes in the cluster; (15) After receiving the data, other nodes establish a lease or renew the storage; (2) Non-dating mode: The specific process is as follows: (21) The active node compresses the data and sends the version number to other nodes in the cluster. In the second cycle and thereafter, only the version number is sent; (22) After receiving the data, other nodes establish a lease or renew the storage.

2. According to the method for distributed consistency of microservices in a weak network environment according to claim 1, it is characterized in that: The active node sends a pre-detection packet and selects the best dating node, specifically: Before the active node sends service data to the dating node, it sends a 1-byte pre-detection packet to all dating nodes. The content of the pre-detection packet is any ASCII code. If the dating node receives this pre-detection packet, it will immediately reply with the same 1-byte data to the active node. The active node first compares the received reply with the pre-detection data it sent before to see if it is the same as the data sent in other cycles. If they are the same, the dating node with the fastest reply will be selected to send the service data.

3. The distributed consistency method for microservices in a weak network environment according to claim 1 is characterized in that: After receiving the data, other nodes establish a lease or renew the storage. Specifically: The data received remotely has a validity period, which is called the lease time. The lease time is greater than twice the synchronization period and less than three times the synchronization period. After the lease is refreshed, the lease time is reset. If the service data of the remote node is not refreshed within the lease time, it is considered that the service has been offline and the information of the service should not be found.

4. The distributed consistency method for microservices in a weak network environment according to claim 1 is characterized in that: After other nodes receive the data, if they find that the version number of the received data is inconsistent, they will actively request the corresponding active node to obtain the latest version number and data.

5. The distributed consistency method for microservices in a weak network environment according to claim 1 is characterized in that: The period in which the active nodes send data is the same as the period in which the dating nodes collect data.

6. The distributed consistency method for microservices in a weak network environment according to claim 1 is characterized in that: If the data sent by the dating node is mixed data from multiple nodes, the data of the active node included in the mixed data will be deleted when it is sent to the active node.

7. The distributed consistency method for microservices in a weak network environment according to claim 1 is characterized in that: Automatically switch between dating mode and non-dating mode according to the situation. The specific switching principles are as follows: (1) No dating node is configured, and the non-dating mode is entered; (2) Configure the dating node. For active nodes, the dating node has not left the group and enters the dating mode; (3) Configure the dating node. For active nodes, the dating node leaves the group and enters the non-dating mode.

8. The distributed consistency method for microservices in a weak network environment according to claim 7 is characterized in that: If the cluster is grouped, the group with dating nodes enters dating mode, and the group without dating nodes enters no dating mode.

9. The distributed consistency method for microservices in a weak network environment according to claim 1 is characterized in that: Compress data using the Zlib algorithm.

Images