A method and tool for data synchronization between time series databases

Abstract

The application discloses a data synchronization method and tool between time sequence databases, relates to the technical field of time sequence databases, and has the implementation process as follows: a node needing to collect data is defined as an edge main node, and an edge backup node exists for each edge main node; a cloud end manages data of the edge main node and the edge backup node; a user writes cloud end information for managing the edge main node and edge backup node information of the edge main node into a configuration file; whether the cloud end information for managing the edge main node and the edge backup node information of the edge main node exist in the configuration file is checked; if not, the cloud end information for managing the edge main node and the edge backup node information of the edge main node are written into the configuration file, and then data synchronization of the edge main node to the cloud end and the edge backup node is started; if yes, data synchronization of the edge main node to the cloud end and the edge backup node is restarted. The application can simply and quickly realize data synchronization and improve user experience.

Description

Technical Field

[0001] This invention relates to the field of time-series database technology, specifically a method and tool for data synchronization between time-series databases. Background Technology

[0002] A time-series database is often divided into two different types of data: metadata and time-series data.

[0003] Generally, data synchronization between time-series databases requires not only the synchronization of time-series data but also the synchronization of related metadata. Furthermore, data from one time-series database cluster may be synchronized to other different time-series database clusters. For example, after a time-series database collects data at the edge, it needs to synchronize the data to cloud storage and also to the edge backup node to prevent the primary node from failing suddenly. During this process, metadata changes may also occur. Therefore, at the edge, it is necessary to establish synchronization relationships for metadata and time-series data with both the cloud and the backup node. While all these processes involve establishing synchronization relationships, this involves four synchronization steps, making the operation relatively cumbersome and reducing the user experience. Summary of the Invention

[0004] This invention addresses the needs and shortcomings of current technological development by providing a method and tool for data synchronization between time-series databases, enabling users to more easily and conveniently synchronize data between the edge master node in the cloud and its edge backup nodes, thereby improving the user experience.

[0005] First, the present invention provides a data synchronization method between time-series databases, and the technical solution adopted to solve the above-mentioned technical problem is as follows:

[0006] A method for data synchronization between time-series databases, the implementation steps of which are as follows:

[0007] Define the node that needs to collect data as the edge master node, and each edge master node has one edge backup node;

[0008] Cloud-based management of data for edge master and edge backup nodes;

[0009] Users write the cloud information of the management edge master node and the edge backup node information of the edge master node into the configuration file;

[0010] Users can enable or disable data synchronization of the edge master node, which means enabling data synchronization from the edge master node to the cloud and its edge backup nodes.

[0011] Specifically, the user executes the statement "Start replication" on the edge master node to start data synchronization from the edge master node to the cloud and its edge backup nodes;

[0012] Users can execute the statement "Pause replication" on the edge master node to disable data synchronization between the edge master node and the cloud and its edge backup nodes.

[0013] Optionally, before enabling data synchronization on the edge master node, the user needs to check the configuration file to see if there is cloud information managing the edge master node and edge backup node information for the edge master node.

[0014] If the configuration file does not contain cloud information for managing the master edge node or information for the backup edge node, it means that data synchronization for the master edge node has not been enabled before. In this case, the aforementioned cloud information and backup edge node information need to be added to the configuration file before enabling data synchronization for the master edge node.

[0015] If the configuration file contains cloud information for managing the edge master node and edge backup node information for the edge master node, it means that data synchronization was previously enabled for the edge master node, and data synchronization for the edge master node needs to be re-enabled.

[0016] Alternatively, if data synchronization has not been enabled on the edge master node before, write the cloud information for managing the edge master node and the edge backup node information of the edge master node into the configuration file, then enable data synchronization of the edge master node, create the synchronization status of each data in the edge master node time series database in the configuration file, and record the data status as running.

[0017] After disabling data synchronization on the edge master node, the data status of the edge master node's time-series database is changed from running to stopped in the configuration file.

[0018] Optionally, if the edge master node has previously enabled data synchronization and then re-enables data synchronization, after obtaining the cloud information managing the edge master node, the edge backup node information of the edge master node, and the data synchronization status from the configuration file, the data status of the edge master node's time-series database is changed from stopped to running.

[0019] Secondly, the technical solution adopted by the data synchronization tool between time-series databases of the present invention to solve the above-mentioned technical problems is as follows:

[0020] A data synchronization tool between time-series databases, comprising:

[0021] The get-write module is used to get the cloud information of the management edge master node and the edge backup node information of the edge master node, and write them into the configuration file;

[0022] The inspection and processing module checks whether the cloud information managing the edge master node and the edge backup node information exist in the configuration file when the user enables data synchronization of the edge master node. If they exist, it restarts the data synchronization from the edge master node to the cloud and the edge backup node. If they do not exist, it starts the acquisition and writing module and then starts the data synchronization from the edge master node to the cloud and the edge backup node.

[0023] Optionally, the user can execute the statement "Start replication" on the edge master node to enable data synchronization from the edge master node to the cloud and its edge backup nodes.

[0024] Users can execute the statement "Pause replication" on the edge master node to disable data synchronization between the edge master node and the cloud and its edge backup nodes.

[0025] Optionally, if the configuration file does not contain cloud information for managing the edge master node or edge backup node information for the edge master node, it indicates that data synchronization has not been enabled for the edge master node before. In this case, it is necessary to start the acquisition and writing module, write the cloud information for managing the edge master node and edge backup node information for the edge master node into the configuration file, then enable data synchronization from the edge master node to the cloud and edge backup nodes, create the synchronization status of each data in the edge master node time-series database in the configuration file, and record the data status as running.

[0026] Optionally, if the configuration file contains cloud information managing the edge master node and edge backup node information for the edge master node, it indicates that data synchronization was previously enabled on the edge master node. When data synchronization from the edge master node to the cloud and edge backup node is re-enabled, the cloud information managing the edge master node, edge backup node information for the edge master node, and data synchronization status are retrieved from the configuration file, and then the data status of the edge master node's time-series database is changed from stopped to running.

[0027] The data synchronization method and tool between time-series databases of the present invention have the following advantages compared with the prior art:

[0028] (1) This invention achieves data synchronization between the edge master node and the cloud and its edge backup nodes by writing the cloud information of the management edge master node and the edge backup node information of the edge master node into the configuration file. That is, it achieves data synchronization between the metadata and time series data in the time series database of the edge master node and the time series database of the cloud and its edge backup nodes.

[0029] (2) The present invention writes the cloud information of the management edge master node and the edge backup node information of the edge master node into the configuration file. When the user enables or disables the data synchronization of the edge master node, there is no need to be aware of the cloud or backup node, metadata or time series data. The user only needs to obtain the cloud information and edge backup node information from the configuration file to enable or disable all data synchronization, which greatly improves the user experience.

[0030] (3) This invention records the data synchronization status of the edge master node. When a data synchronization line has a problem, it is easier to find the cause of the error and facilitate the maintenance of data synchronization. Attached Figure Description

[0031] Appendix Figure 1 This is a flowchart of data synchronization performed according to Embodiment 1 of the present invention;

[0032] Appendix Figure 2 This is a module connection diagram of Embodiment 2 of the present invention. Detailed Implementation

[0033] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.

[0034] Example 1:

[0035] This embodiment proposes a data synchronization method between time-series databases, and the implementation steps are as follows:

[0036] Define the node that needs to collect data as the edge master node, and each edge master node has one edge backup node;

[0037] Cloud-based management of data for edge master and edge backup nodes;

[0038] Users write the cloud information of the management edge master node and the edge backup node information of the edge master node into the configuration file;

[0039] Users execute the statement "Start replication" on the edge master node to initiate data synchronization from the edge master node to the cloud and its edge backup nodes.

[0040] Users can execute the statement "Pause replication" on the edge master node to disable data synchronization between the edge master node and the cloud and its edge backup nodes.

[0041] In this embodiment, combined with the appendix Figure 1 Before a user enables data synchronization on the edge master node, they need to check the configuration file to see if there is cloud information managing the edge master node and edge backup node information for the edge master node.

[0042] If the cloud information for managing the edge master node and the edge backup node information for the edge master node are not present in the configuration file, it means that the edge master node has not enabled data synchronization before. In this case, you need to write the cloud information for managing the edge master node and the edge backup node information for the edge master node into the configuration file, then enable data synchronization of the edge master node, create the synchronization status of each data in the edge master node time series database in the configuration file, and record the data status as running. After disabling data synchronization of the edge master node, change the data status of the edge master node time series database from running to stopped in the configuration file.

[0043] If the configuration file contains cloud information for managing the edge master node and edge backup node information for the edge master node, it means that data synchronization was previously enabled for the edge master node. In this case, data synchronization for the edge master node needs to be re-enabled. After obtaining the cloud information for managing the edge master node, the edge backup node information for the edge master node, and the data synchronization status from the configuration file, the data status of the edge master node's time-series database should be changed from stopped to running.

[0044] It should be added that: the data synchronization from the edge master node to the cloud and its edge backup nodes is actually the data synchronization from the edge master node's time series database to the cloud's time series database and its edge backup node's time series database.

[0045] Example 2:

[0046] Combined with appendix Figure 2 This embodiment proposes a data synchronization tool between time-series databases, which includes:

[0047] The get-write module is used to get the cloud information of the management edge master node and the edge backup node information of the edge master node, and write them into the configuration file;

[0048] The processing module checks whether the cloud information managing the edge master node and the edge backup node information exist in the configuration file when the user enables data synchronization of the edge master node. ① If the cloud information managing the edge master node and the edge backup node information do not exist in the configuration file, it means that the edge master node has not enabled data synchronization before. In this case, the acquisition and writing module needs to be started to write the cloud information managing the edge master node and the edge backup node information into the configuration file, and then data synchronization from the edge master node to the cloud and the edge backup node is enabled. ① Create a synchronization status for each data entry in the edge master node time series database in the configuration file and record the data status as running. ② If the configuration file contains cloud information managing the edge master node and edge backup node information of the edge master node, it means that the edge master node has previously enabled data synchronization. When re-enabling data synchronization between the edge master node and the cloud and edge backup nodes, retrieve the cloud information managing the edge master node, the edge backup node information of the edge master node, and the data synchronization status from the configuration file, and then change the data status of the edge master node time series database from stopped to running.

[0049] In this embodiment, the user executes the statement Start replication on the edge master node to start data synchronization from the edge master node to the cloud and its edge backup nodes.

[0050] Users can execute the statement "Pause replication" on the edge master node to disable data synchronization between the edge master node and the cloud and its edge backup nodes.

[0051] It should be added that: the data synchronization from the edge master node to the cloud and its edge backup nodes is actually the data synchronization from the edge master node's time series database to the cloud's time series database and its edge backup node's time series database.

[0052] In summary, the data synchronization method and tool for time-series databases of this invention writes the cloud information of the management edge master node and the edge backup node information of the edge master node into a configuration file. When users enable or disable data synchronization of the edge master node, they do not need to be aware of the cloud or backup node, metadata or time-series data. They only need to obtain the cloud information and edge backup node information from the configuration file to enable or disable all data synchronization. This realizes the data synchronization of metadata and time-series data in the edge master node time-series database to the cloud time-series database and its edge backup node time-series database, greatly improving the user experience.

[0053] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A method for data synchronization between time-series databases, characterized in that, The implementation steps are as follows: Define the node that needs to collect data as the edge master node, and each edge master node has one edge backup node; Cloud-based management of data for edge master and edge backup nodes; Users write the cloud information of the management edge master node and the edge backup node information of the edge master node into the configuration file; Check the configuration file to see if there is cloud information managing the master node and backup node information for the master node. If there is no such information, it means that the master node has not previously enabled data synchronization. In this case, you need to add the aforementioned cloud information and backup node information to the configuration file and then enable data synchronization for the master node. If there is such information, it means that the master node has previously enabled data synchronization. In this case, you need to re-enable data synchronization for the master node. Users can enable or disable data synchronization of the edge master node, which means enabling data synchronization from the edge master node to the cloud and its edge backup nodes.

2. The data synchronization method between time-series databases according to claim 1, characterized in that, Users execute the statement Start replication on the edge master node to start data synchronization from the edge master node to the cloud and its edge backup nodes. Users can execute the statement "Pause replication" on the edge master node to disable data synchronization between the edge master node and the cloud and its edge backup nodes.

3. The data synchronization method between time-series databases according to claim 1, characterized in that, If data synchronization has not been enabled on the edge master node before, write the cloud information for managing the edge master node and the edge backup node information of the edge master node into the configuration file. Then enable data synchronization of the edge master node, create the synchronization status of each data in the time series database of the edge master node in the configuration file, and record the data status as running. After disabling data synchronization on the edge master node, the data status of the edge master node's time-series database is changed from running to stopped in the configuration file.

4. The data synchronization method between time-series databases according to claim 3, characterized in that, When the edge master node has previously enabled data synchronization and then re-enables data synchronization, it retrieves the cloud information managing the edge master node, the edge backup node information of the edge master node, and the data synchronization status from the configuration file, and then changes the data status of the edge master node's time-series database from stopped to running.

5. A data synchronization tool between time-series databases, characterized in that, It includes: The get-write module is used to get the cloud information of the management edge master node and the edge backup node information of the edge master node, and write them into the configuration file; The inspection and processing module checks whether the cloud information managing the edge master node and the edge backup node information of the edge master node exist in the configuration file when the user enables data synchronization of the edge master node. If they exist, the module restarts the data synchronization from the edge master node to the cloud and the edge backup node. If they do not exist, the module starts the acquisition and writing module and then starts the data synchronization from the edge master node to the cloud and the edge backup node. If the configuration file does not contain cloud information for managing the edge master node or edge backup node information for the edge master node, it means that the edge master node has not previously enabled data synchronization. In this case, it is necessary to start the acquisition and writing module, write the cloud information for managing the edge master node and the edge backup node information for the edge master node into the configuration file, and then enable data synchronization from the edge master node to the cloud and the edge backup node. Create the synchronization status of each data in the edge master node time series database in the configuration file and record the data status as running. The configuration file contains cloud information for managing the edge master node and edge backup node information for the edge master node, indicating that data synchronization was previously enabled on the edge master node. When data synchronization from the edge master node to the cloud and edge backup node is re-enabled, the cloud information for managing the edge master node, edge backup node information for the edge master node, and data synchronization status are retrieved from the configuration file. Then, the data status of the edge master node's time-series database is changed from stopped to running.

6. A data synchronization tool between time-series databases according to claim 5, characterized in that, Users execute the statement Start replication on the edge master node to start data synchronization from the edge master node to the cloud and its edge backup nodes. Users can execute the statement "Pause replication" on the edge master node to disable data synchronization between the edge master node and the cloud and its edge backup nodes.

Images