A distributed logical time synchronization method and terminal based on ZooKeeper listening mechanism

Abstract

This invention discloses a distributed logical time synchronization method and terminal based on the Zookeeper listening mechanism. The method includes: creating a persistent node in Zookeeper to store a global logical time identifier; periodically writing updated logical time identifiers to this node by a time source service; each service node in the distributed system registering to listen to this node upon startup, and reading the new logical time identifier and updating its local cache when it detects a change in the time data stored therein; and prioritizing reading from the local cache when a service node needs to obtain the time. This invention achieves highly available, low-overhead, and easily scalable logical time synchronization by uniformly storing logical time identifiers in Zookeeper nodes and relying on its distributed consistency and listening mechanism to synchronize the logical time identifiers to the local caches of each distributed service node in real time, thus meeting the time consistency requirements of new distributed systems.

Description

Technical Field

[0001] This invention relates to the field of time synchronization technology for distributed systems, and in particular to a distributed logical time synchronization method and terminal based on the ZooKeeper listening mechanism. Background Technology

[0002] Currently, the main technical solutions for time synchronization in distributed systems fall into the following categories: First, synchronization based on the Network Time Protocol (NTP). This method involves configuring a time client on each node, periodically sending requests to a standard time server, and adjusting the local clock based on the returned results, thereby achieving uniformity among distributed service nodes. Second, using dedicated clock synchronization hardware or protocols, such as GPS time synchronization or PTP (Precision Time Protocol). This type of approach is typically used in scenarios with high time accuracy requirements, achieving nanosecond or microsecond-level synchronization through hardware time synchronization or high-precision protocols. This solution carries logical timestamps in messages or transactions, enabling different nodes to maintain a logical time sequence.

[0003] Although existing technologies can achieve time synchronization in distributed systems to a certain extent, they still have many shortcomings. First, NTP-based time synchronization relies on an external time server, and its synchronization accuracy is easily affected by network latency, jitter, and packet loss, making it difficult to guarantee stability in distributed systems with high concurrency and low latency requirements.

[0004] Secondly, while GPS time synchronization or PTP protocols can provide higher accuracy, they require additional hardware or dedicated network support, resulting in higher costs and application limitations in some deployment environments. Logical clocks or vector clocks, although capable of maintaining the relative order of events, typically rely on appending time information to communication messages, increasing system complexity and communication overhead, and lacking a centralized and unified time reference. Summary of the Invention

[0005] The purpose of this invention is to provide a distributed logical time synchronization method and terminal based on the ZooKeeper listening mechanism, aiming to solve the problems of limited accuracy, high cost, excessive overhead, and single point of failure in existing technologies. This invention uniformly stores logical time identifiers in ZooKeeper nodes and, relying on its distributed consistency and listening mechanism, synchronizes the logical time identifiers to the local caches of each distributed service node in real time. This achieves highly available, low-overhead, and easily scalable logical time synchronization, meeting the time consistency requirements of modern distributed systems.

[0006] The technical solution adopted in this invention is:

[0007] A distributed logical time synchronization method based on the ZooKeeper listening mechanism includes the following steps:

[0008] Create a node in ZooKeeper to store a global logical time identifier;

[0009] The time source service deployed in the distributed system periodically writes updated logical time identifiers to nodes in Zookeeper;

[0010] When a distributed service node in a distributed system starts up, it registers a listener for the node in ZooKeeper.

[0011] When a distributed service node detects a change in the logical time stamp of a node in ZooKeeper, the distributed service node reads the changed logical time stamp from ZooKeeper and updates it to its local cache.

[0012] When a distributed service node obtains a logical time identifier, it first reads it from the local cache.

[0013] Furthermore, the nodes in Zookeeper are persistent nodes.

[0014] Furthermore, before creating a node in Zookeeper, the process includes: establishing a unified node path in Zookeeper for time synchronization, under which the node is created.

[0015] Furthermore, the logical time identifier is a 13-bit millisecond timestamp.

[0016] Furthermore, periodically write updated logical time markers to the nodes in ZooKeeper, specifically including:

[0017] The time source service obtains the current time from the local system clock at fixed intervals and generates corresponding logical time identifiers, which are then written to nodes in Zookeeper.

[0018] Furthermore, the fixed period is 100 milliseconds.

[0019] Furthermore, the distributed service nodes update the modified logical timestamp to their local cache, specifically including:

[0020] The distributed service node compares the modified logical time identifier it reads with the logical time identifier currently stored in its local cache;

[0021] If a local clock rollback is detected, or if the changed logical time identifier is not greater than the logical time identifier currently stored in the local cache, then the logical time identifier currently stored in the local cache is retained.

[0022] Otherwise, update the local cache using the modified logical time identifier.

[0023] Furthermore, the method of the present invention also includes:

[0024] When the connection between the distributed service node and Zookeeper is interrupted, the distributed service node continues to use the logical time stamp of the most recently updated one in the local cache;

[0025] When the connection between the distributed service node and Zookeeper is restored, the distributed service node automatically re-registers to listen to the nodes in Zookeeper and synchronizes the latest logical time stamp.

[0026] Furthermore, when the distributed service node obtains the logical time identifier, it prioritizes reading from the local cache, specifically including:

[0027] The distributed service nodes first attempt to read the logical time identifier from the local cache;

[0028] If the local cache does not contain a logical time identifier, or the logical time identifier stored therein is invalid, then the logical time identifier is read from the node in Zookeeper and updated to the local cache.

[0029] A terminal device is characterized by comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned distributed logical time synchronization method.

[0030] This invention has the following essential features:

[0031] Zookeeper-based global time synchronization mechanism:

[0032] This invention uses Zookeeper coordination service as the core infrastructure, in which a globally unified logical time stamp storage node is created and maintained. By utilizing Zookeeper's inherent distributed consistency protocol, the logical time stamp is ensured to be highly synchronized and strongly consistent among all access nodes.

[0033] Definition of Unified Logical Time Stamp:

[0034] This invention uses a 13-bit millisecond timestamp as a unified logical time identifier in a distributed system. This identifier is generated and maintained by a time source service and features high precision, ease of generation, and universal readability, thus simplifying system implementation.

[0035] Real-time synchronization driven by a listening mechanism:

[0036] Each distributed service node subscribes to change events in real time by registering a listener (such as a Watcher) with the storage nodes in ZooKeeper. When the time source updates the identifier, each node can receive the notification and synchronize in near real time, completely avoiding the network and computational overhead caused by the traditional polling method.

[0037] Local cache-first read mechanism:

[0038] To optimize read performance and reduce reliance on ZooKeeper, each distributed service node prioritizes reading from its local cache when logical timestamps are needed. Only when the local cache is not yet established or the stored timestamp is deemed invalid does it fall back to directly reading from the ZooKeeper node, thus achieving high performance while ensuring consistency.

[0039] Strategies for handling exceptions and clock rollback:

[0040] To address potential anomalies such as local physical clock rollback in a distributed environment, this invention designs a robust update strategy: when a rollback or newly acquired logical time identifier is detected to be no greater than the cached value, the original value in the cache is retained. This max(cache value, new value) strategy ensures that the logical time identifier always remains monotonically increasing throughout the entire distributed system, meeting the basic requirements of business for event order.

[0041] Self-healing ability during connection interruption:

[0042] This invention exhibits excellent fault tolerance. When the connection between the distributed service node and Zookeeper is temporarily interrupted, the node can continue to operate by relying on the most recently updated valid logical time identifier in its local cache, ensuring service continuity. Once the connection is restored, the node can automatically re-register for listening and synchronize to the latest identifier, achieving service self-healing.

[0043] Achieving a globally consistent time source:

[0044] By combining the above mechanisms, this invention provides a highly available, high-performance, and highly consistent global logical time source for large-scale distributed systems, without relying on external clock services (such as NTP). All service nodes share the same logical time reference, fundamentally solving the problems of accuracy, overhead, and consistency in time synchronization in distributed environments.

[0045] Compared with the closest prior art, the present invention has significant advantages:

[0046] 1. Higher consistency: Based on Zookeeper's distributed consistency protocol (ZAB), it ensures that the timestamp is highly consistent across nodes after it is written, avoiding time differences caused by network jitter.

[0047] 2. Lighter architecture: Only 13-bit millisecond timestamps are needed as a unified logical time, avoiding complex logical clock or vector clock models, reducing implementation difficulty and system overhead.

[0048] 3. Better scalability: This invention is naturally applicable to distributed, microservice, and large-scale cluster environments. It can maintain time synchronization capability as the cluster expands, and there is no single point of bottleneck in centralized time services. Attached Figure Description

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0050] Figure 1 This is a schematic diagram illustrating the interaction between the time source service, Zookeeper, and multiple distributed service nodes in this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0052] This invention provides a distributed logical time synchronization method based on the Zookeeper listening mechanism, aiming to provide a highly consistent, low-overhead, and highly available logical time source for distributed systems. The following is combined with... Figure 1 The implementation of this method is explained in detail.

[0053] Step S101, Namespace initialization:

[0054] In a ZooKeeper coordination service cluster, a unified node path for time synchronization is first established, such as / global-time / epoch. Then, a node is created under this path. This node is a persistent node and will serve as the unified storage location for the global logical time identifier of the entire distributed system.

[0055] Step 102, Deploy and run the Time Source service:

[0056] One or more time source services are deployed in a distributed system. A primary time source service can be designated to periodically generate the latest timestamps and write them to ZooKeeper nodes. Once activated, the primary time source service begins its periodic operation. Specifically, it retrieves the current time from the local system clock at a fixed interval (e.g., 100 milliseconds) and generates a corresponding logical time identifier. In this embodiment, the logical time identifier uses a 13-bit millisecond timestamp. Subsequently, the time source service writes (updates) the newly generated logical time identifier to a pre-created persistent node ( / global-time / epoch) in ZooKeeper. The ZooKeeper cluster utilizes its distributed consistency protocol (such as ZAB) to ensure the consistency and visibility of this write operation across all ZooKeeper server nodes.

[0057] Step S103, Distributed service node registers listener:

[0058] When each distributed service node in a distributed system starts up, it registers a listener with the persistent node ( / global-time / epoch) in ZooKeeper to ensure that it can receive a notification in real time when the logical time stamp changes.

[0059] Step S104: Listen for triggers and local cache updates.

[0060] When the logical time stamp of the " / global-time / epoch" node in ZooKeeper changes, ZooKeeper notifies all registered listening distributed service nodes of the change event. Upon receiving the notification, the distributed service nodes immediately read the changed logical time stamp from ZooKeeper and update it in their local cache.

[0061] Before updating the modified logical timestamp to the local cache, exception handling logic will be executed:

[0062] Compare the read modified logical timestamp with the currently stored logical timestamp in the local cache:

[0063] If a local clock rollback is detected, or the changed logical time identifier is not greater than the currently stored logical time identifier in the local cache, the currently stored logical time identifier in the local cache is retained; otherwise, the changed logical time identifier is used to update the local cache. The max(cache value, new value) strategy ensures that the logical time identifier in each node's local cache always remains monotonically increasing.

[0064] Step S105: Read the timestamp from the local cache first.

[0065] The distributed service nodes first attempt to read the logical time identifier from the local cache;

[0066] The distributed service node will only fall back to the node that directly reads the logical time identifier in Zookeeper, read the logical time identifier, and synchronously update the local cache if the logical time identifier does not exist in the local cache or the logical time identifier stored therein is invalid.

[0067] S106, Disconnection and Self-Healing Process:

[0068] When the connection between the distributed service node and Zookeeper is interrupted, the distributed service node continues to use the most recently updated logical timestamp in the local cache to ensure the continuous operation of basic services.

[0069] When the connection between the distributed service node and Zookeeper is restored, the distributed service node automatically re-registers to listen to the nodes in Zookeeper and synchronizes the latest logical time stamp, thereby achieving self-healing of the service and restoring it to a normal synchronization state.

[0070] Through the above mechanism, all distributed service nodes can use a consistent 13-bit millisecond timestamp locally without relying on external NTP services, thus achieving the unification of logical time in the distributed system.

[0071] The present invention also provides a terminal device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the distributed logical time synchronization method described in the above-described method embodiments. This terminal device can be a distributed server, an edge computing terminal, or an embedded device to achieve time unification across different scenarios.

[0072] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Unless otherwise specified, the embodiments and features described and illustrated in this application can be combined with each other. The components of the embodiments of this application generally described and illustrated in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A distributed logical time synchronization method based on the ZooKeeper listening mechanism, characterized in that: Includes the following steps: Create a node in ZooKeeper to store a global logical time identifier; The time source service deployed in the distributed system periodically writes updated logical time identifiers to nodes in Zookeeper; When a distributed service node in a distributed system starts up, it registers a listener for the node in ZooKeeper. When a distributed service node detects a change in the logical time stamp of a node in ZooKeeper, the distributed service node reads the changed logical time stamp from ZooKeeper and updates it to its local cache. When a distributed service node obtains a logical time identifier, it first reads it from the local cache.

2. The distributed logical time synchronization method based on the ZooKeeper listening mechanism according to claim 1, characterized in that: The nodes in Zookeeper are persistent nodes.

3. A distributed logical time synchronization method based on ZooKeeper listening mechanism according to claim 1 or 2, characterized in that: Before creating a node in ZooKeeper, the process also includes: establishing a unified node path in ZooKeeper for time synchronization, under which the node is created.

4. The distributed logical time synchronization method based on the ZooKeeper listening mechanism according to claim 1, characterized in that: The logical time identifier is a 13-bit millisecond timestamp.

5. A distributed logical time synchronization method based on ZooKeeper listening mechanism according to claim 1, characterized in that: Periodically write updated logical time markers to nodes in ZooKeeper, specifically including: The time source service obtains the current time from the local system clock at fixed intervals and generates corresponding logical time identifiers, which are then written to nodes in Zookeeper.

6. A distributed logical time synchronization method based on ZooKeeper listening mechanism according to claim 5, characterized in that: The fixed period is 100 milliseconds.

7. A distributed logical time synchronization method based on ZooKeeper listening mechanism according to claim 1, characterized in that: The distributed service nodes update the modified logical timestamp to their local cache, specifically including: The distributed service node compares the modified logical time identifier it reads with the logical time identifier currently stored in its local cache; If a local clock rollback is detected, or if the changed logical time identifier is not greater than the logical time identifier currently stored in the local cache, then the logical time identifier currently stored in the local cache is retained. Otherwise, update the local cache using the modified logical time identifier.

8. A distributed logical time synchronization method based on ZooKeeper listening mechanism according to claim 1, characterized in that: The method further includes: When the connection between the distributed service node and Zookeeper is interrupted, the distributed service node continues to use the logical time stamp of the most recently updated one in the local cache; When the connection between the distributed service node and Zookeeper is restored, the distributed service node automatically re-registers to listen to the nodes in Zookeeper and synchronizes the latest logical time stamp.

9. A distributed logical time synchronization method based on ZooKeeper listening mechanism according to claim 1, characterized in that: When a distributed service node obtains a logical time identifier, it first reads it from the local cache, specifically including: The distributed service nodes first attempt to read the logical time identifier from the local cache; If the local cache does not contain a logical time identifier, or the logical time identifier stored therein is invalid, then the logical time identifier is read from the node in Zookeeper and updated to the local cache.

10. A terminal device, characterized in that: It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the distributed logical time synchronization method as described in any one of claims 1 to 9.

Images