Methods, systems, equipment, and media for synchronizing configuration data in real-time database systems

Abstract

This invention relates to a method, system, device, and medium for synchronizing configuration data in a real-time database system. The method includes: when a configuration client publishes configuration data for configuration updates, the business configuration synchronization modules of each node in the business subsystem receive the configuration data distributed by the configuration server and synchronize the configuration data to their respective nodes according to a configuration update strategy. When a user publishes configuration updates to various business subsystems of the real-time database, the configuration server synchronizes data with one or more first-level nodes in the system. Then, the first-level nodes synchronize data with their subscribed slave nodes. The subscribed slave nodes select the node with the highest version and latest priority from among the subscribed first-level nodes as the update data source. This invention, based on the interconnection of various configuration business nodes and using a multi-master, multi-slave strategy and a multi-stage publishing mechanism, ensures high availability and high performance of real-time database configuration distribution.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a method, system, device, and medium for synchronizing configuration data in a real-time database system. Background Technology

[0002] As enterprise informatization deepens, real-time databases are becoming increasingly large-scale, and their performance requirements are rising. The application scenarios for real-time databases have also evolved from single-machine deployments to various deployment modes such as single-machine, dual-machine redundancy, clusters, and cloud deployments. As the metadata of real-time databases, the deployment and synchronization solutions must accommodate single-machine, dual-machine, and cluster modes to ensure high availability in various scenarios while maximizing data synchronization performance.

[0003] Current mainstream distributed real-time databases typically employ a master-slave cluster architecture, where the master node performs configuration updates, and the slave nodes synchronize with the master. These solutions involve master election, often using strongly consistent algorithms like Paxos or its variants such as Raft. These algorithms require at least three available nodes in the cluster. On one hand, this can lead to resource waste for small clients; on the other hand, if a cluster failure reduces the number of healthy nodes to less than three, the cluster will be unable to elect a master, rendering the entire service system unavailable. Furthermore, there is a possibility of a "split-brain" scenario due to network failures, resulting in more severe system configuration asynchrony and data unavailability. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for synchronizing configuration data in a real-time database system, which solves the technical problem that the existing cluster architecture is limited by the number of nodes, cannot take into account both single-machine mode and dual-machine redundancy mode, and will fall into the split-brain problem.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, embodiments of the present invention provide a method for synchronizing configuration data in a real-time database system, comprising:

[0009] When the configuration client publishes configuration data for configuration updates, the business configuration synchronization module of each node in the business subsystem receives the configuration data distributed by the configuration server and synchronizes and updates the configuration data to its respective node according to the configuration update strategy.

[0010] The configuration client, configuration server, and business subsystem are all components of a distributed real-time database system; each node in the business subsystem is equipped with a configuration synchronization module; the business subsystem includes n-level nodes, and the n-th level node subscribes to data from at least one node of the previous level, where n is a natural number greater than or equal to 1.

[0011] The configuration update strategy involves each level node selecting the configuration data of the highest priority node among all subscribed parent nodes for configuration synchronization.

[0012] Optionally, the deployment modes of the business subsystem include: single-machine mode, dual-machine redundancy mode, and cluster mode.

[0013] Optionally,

[0014] In the business subsystems of dual-machine redundancy mode and cluster mode, the first-level node is the management master node selected according to preset selection conditions, and the second-level node that subscribes to the management master node information is the subscription slave node.

[0015] The management master node and the subscription slave node change according to the node update rules, and the service configuration synchronization module in any of the management master node and the subscription slave node obtains the location information of the node to which the service configuration synchronization module belongs in real time. The service configuration synchronization module in the subscription slave node is also used to obtain the management master node information subscribed to by the node to which the service configuration synchronization module belongs in real time.

[0016] The service configuration synchronization module within the subscription slave node confirms the information of the next-level node receiving the configuration data based on the location information, the subscription management master node information, and the configuration update strategy.

[0017] Optionally, in standalone mode,

[0018] The configuration server is configured as the management master node. When the configuration server has the distribution function, it is also used to distribute the configuration data published by the configuration client to each business subsystem, and each business subsystem only needs to perform synchronization once.

[0019] Optionally, in dual-machine redundancy mode,

[0020] Each business subsystem is configured with two Level 1 nodes, which directly synchronize data with the configuration server.

[0021] Optionally, in cluster mode,

[0022] In each business subsystem, the first-level node will send the acquired configuration data to the second-level node with which a subscription relationship has been established;

[0023] After receiving multiple sets of configuration data, the Level 2 node compares the configuration versions of each Level 1 node and selects the Level 1 node with the latest version and the highest priority as the configuration update data source for data synchronization.

[0024] Optionally, the configuration versions of each Level 1 node can be compared, including by version number, version ID, and version release time.

[0025] The version number increments unidirectionally. Only when the number and ID are equal are the configuration versions determined to be equal and no configuration synchronization is performed. Otherwise, if the version number of the first-level node is newer and the version ID of the same number is consistent, only incremental update is performed. Otherwise, a full configuration update is performed and the configuration of the business subsystem is reloaded.

[0026] Optionally,

[0027] When the second-level node is online and all first-level nodes are down, the second-level node is automatically promoted to a first-level node in order to synchronize data with the configuration server;

[0028] If the original Level 1 node recovers, the automatically upgraded Level 2 node will revert to its un-upgraded working state.

[0029] Secondly, embodiments of the present invention provide a synchronization system for real-time database system configuration data, comprising:

[0030] The configuration client is configured to publish configuration data for configuration updates;

[0031] The configuration server is configured to acquire configuration data published by the configuration client and distribute it to various business subsystems for sequential distribution to multiple nodes, so as to keep the data of various nodes in the same business subsystem synchronized.

[0032] The business subsystem includes: n-level nodes, where each n-level node subscribes to data from at least one node at the previous level, and n is a natural number greater than or equal to 1.

[0033] In each business subsystem

[0034] The first-level node will send the acquired configuration information to the second-level node with which it has established a subscription relationship;

[0035] The second-level node subscribes to at least one of the first-level nodes. After receiving multiple sets of configuration data, it compares the configuration versions of each first-level node and selects the first-level node with the latest version and the highest priority as the configuration update data source for data synchronization.

[0036] Thirdly, embodiments of the present invention provide a synchronization device for real-time database system configuration data, characterized in that it includes: at least one database; and a memory communicatively connected to the at least one database; wherein the memory stores instructions executable by the at least one database, the instructions being executed by the at least one database to enable the at least one database to perform a real-time database system configuration data synchronization method as described above.

[0037] Fourthly, embodiments of the present invention provide a computer-readable medium having computer-executable instructions stored thereon, wherein the executable instructions, when executed by a processor, implement a method for synchronizing configuration data of a real-time database system as described above.

[0038] (III) Beneficial Effects

[0039] This invention employs a multi-master, multi-slave strategy for cluster configuration synchronization, ensuring high data availability. The configuration system adopts a business-separated architecture and achieves load balancing of configuration data through a two-phase (or multi-phase) release strategy, improving data synchronization performance and guaranteeing eventual consistency and availability. Simultaneously, the configuration system can automatically match single-machine, dual-machine redundancy, and cluster deployment modes based on the number of nodes in the business subsystems, eliminating the need for additional compatibility code development and improving system reusability and scalability. Attached Figure Description

[0040] Figure 1 A flowchart illustrating a method for synchronizing configuration data in a real-time database system provided by the present invention;

[0041] Figure 2 An interactive timing diagram of a method for synchronizing configuration data in a real-time database system provided by the present invention;

[0042] Figure 3 This invention provides a schematic diagram of the composition of a real-time database system configuration data synchronization system.

[0043] [Explanation of Labels in the Attached Image]

[0044] 100: High-availability synchronization system; 101: Configuration client; 102: Configuration server; 103: Business subsystem. Detailed Implementation

[0045] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 This is a flowchart illustrating a method for synchronizing configuration data in a real-time database system provided by the present invention, as shown below. Figure 1As shown in the embodiment of the present invention, a method for synchronizing configuration data in a real-time database system includes: when a configuration client publishes configuration data for configuration updates, the business configuration synchronization module of each node in the business subsystem receives the configuration data distributed by the configuration server and synchronizes and updates the configuration data to its respective node according to the configuration update strategy.

[0047] This invention employs a multi-master, multi-slave strategy for cluster configuration synchronization, ensuring high data availability. The configuration system adopts a business-separated architecture and achieves load balancing of configuration data through a two-phase (or multi-phase) release strategy, improving data synchronization performance and guaranteeing eventual consistency and availability. Simultaneously, the configuration system can automatically match single-machine, dual-machine redundancy, and cluster deployment modes based on the number of nodes in the business subsystems, eliminating the need for additional compatibility code development and improving system reusability and scalability.

[0048] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0049] Specifically, the present invention provides a method for synchronizing configuration data in a real-time database system, comprising:

[0050] When the configuration client publishes configuration data for configuration updates, the business configuration synchronization module of each node in the business subsystem receives the configuration data distributed by the configuration server and synchronizes and updates the configuration data to its respective node according to the configuration update strategy.

[0051] The configuration client, configuration server, and business subsystem are all components of the distributed real-time database system. Each node in the business subsystem is equipped with a configuration synchronization module. The business subsystem includes n-level nodes, and each n-level node subscribes to data from at least one node in the previous level, where n is a natural number greater than or equal to 1. The configuration update strategy is that each level node selects the configuration data of the highest priority node among all the subscribed previous level nodes for configuration synchronization.

[0052] Furthermore, the deployment modes of the business subsystems include: single-machine mode, dual-machine redundancy mode, and cluster mode.

[0053] In the business subsystems under dual-machine redundancy and cluster modes, the first-level node is the management master node selected according to preset selection criteria, and the second-level nodes that subscribe to the management master node's information are subscription slave nodes. In the business subsystem cluster, the first-level node is responsible for synchronizing configuration service updates and distributing them to the second-level nodes (subscription slave nodes). Configuration updates and publication are implemented by the upper-level configuration server. The main purpose of setting up multi-level nodes is to reduce the load on configuration service publication in the cluster, and to perform secondary distribution of configuration through the master node.

[0054] The management master node and the subscription slave node change according to the node update rules. The service configuration synchronization module within any management master node or subscription slave node obtains the location information of the node to which it belongs in real time. The service configuration synchronization module within the subscription slave node is also used to obtain the management master node information subscribed to by the node to which it belongs in real time. The service configuration synchronization module within the subscription slave node confirms the information of the next-level node receiving the configuration data based on the location information, the subscribed management master node information, and the configuration update strategy.

[0055] The selection of Level 1 nodes can be manually configured by the user or selected autonomously by the cluster. Cluster-based selection can be based on node ID sorting, online time, node status, or other methods. When Level 2 nodes are online and all Level 1 nodes are offline, Level 2 nodes automatically upgrade to Level 1 nodes to synchronize data with the configuration server. If the original Level 1 nodes recover, the automatically upgraded Level 2 nodes revert to their unupgraded working state.

[0056] In this embodiment of the invention, based on the number of nodes in the business subsystem, three deployment modes are automatically matched: single-machine, dual-machine redundancy, and cluster.

[0057] When the number of nodes is 1, it automatically matches the stand-alone mode;

[0058] When the number of nodes is 2, it automatically matches the dual-machine redundancy mode;

[0059] When the number of nodes is greater than 2, the cluster mode will be automatically matched.

[0060] (1) In standalone mode, perform the following steps:

[0061] Configure the configuration server as a level 1 node. When the configuration server publishes configuration data, it will directly publish the configuration update notification to each business subsystem.

[0062] In standalone mode, the local node acts as both a configuration server and the first-level configuration node for each business subsystem. When the configuration server publishes, it only needs to issue configuration update notifications to each business subsystem on the local machine, and each business subsystem only needs to perform a synchronization operation once.

[0063] (2) In dual-machine redundancy mode, perform the following steps:

[0064] Both nodes of each business subsystem are configured as Level 1 nodes, directly obtaining configuration change information from the configuration server.

[0065] If one of the nodes loses connection with the configuration server, the configuration is synchronized from the other level 1 node.

[0066] (3) In cluster mode, perform the following steps:

[0067] When the configuration server is updating the configuration, it publishes a configuration update notification to the first-level nodes of each business subsystem. The first-level nodes obtain the latest update data from the configuration server or other first-level nodes.

[0068] In each business subsystem, the first-level node distributes the acquired configuration data to the second-level nodes that have established a subscription relationship;

[0069] After receiving multiple sets of configuration update information, the Level 2 node compares the configuration versions of each Level 1 node and selects the Level 1 node with the latest version and the highest priority as the configuration update data source for data synchronization.

[0070] The configuration versions of each Level 1 node are compared, using version number, version ID, and release time. Version numbers increment unidirectionally; only when both the version number and ID are equal are the configuration versions considered equal, and no configuration synchronization is performed. Otherwise, if the master node's version number is newer and the version IDs of the same version number are identical, only an incremental update is performed; otherwise, a full configuration update is executed, and the configuration of the business subsystem is reloaded.

[0071] As shown in Figure 2, the specific process of the above method is as follows: The configuration server obtains the configuration data published by the configuration client and distributes it to various business subsystems for multi-level node sequential distribution, so as to keep the data of each node in the same business subsystem synchronized. When a user publishes a configuration update to each business subsystem of the real-time database, the configuration server will first synchronize the data with one or more first-level nodes in the system, and then the first-level nodes will synchronize the data with the second-level nodes they have subscribed to. After receiving multiple data update messages, the second-level nodes will select the node with the highest version and the latest priority from the multiple first-level nodes they have subscribed to as the update data source, thereby achieving high availability and high performance. To improve availability, the configuration data of the business subsystems in the cluster mode can be persisted to the local machine; at the same time, each sub-node can also act as some observation nodes outside the cluster to provide configuration subscription services for further synchronization expansion.

[0072] It is worth mentioning that this invention can also be implemented through dynamic master selection and a one-master-many-slave configuration. However, this would greatly increase the complexity and maintenance cost of the system, and also reduce its scalability.

[0073] like Figure 3 As shown, the present invention also discloses a synchronization system 100 for real-time database system configuration data, comprising:

[0074] Configuration client 101 is configured to publish configuration data for configuration updates.

[0075] Configuration server 102 is configured to acquire configuration data published by configuration clients and distribute it to various business subsystems for sequential distribution to multiple nodes, so as to keep the data of various nodes in the same business subsystem synchronized.

[0076] Business subsystem 103 includes: n-level nodes, where each n-level node subscribes to data from at least one node in the previous level, and n is a natural number greater than or equal to 1.

[0077] In each business subsystem

[0078] The first-level node will send the acquired configuration information to the second-level node with which it has established a subscription relationship.

[0079] The second-level node subscribes to at least one first-level node. After receiving multiple sets of configuration data, it compares the configuration versions of each first-level node and selects the first-level node with the latest version and the highest priority as the data source for configuration update to synchronize the data.

[0080] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0081] Furthermore, the present invention also discloses a synchronization device for real-time database system configuration data, comprising: at least one database; and a memory communicatively connected to the at least one database; wherein the memory stores instructions executable by the at least one database, the instructions being executed by the at least one database to enable the at least one database to perform a real-time database system configuration data synchronization method as described above.

[0082] The present invention also discloses a computer-readable medium storing computer-executable instructions, which, when executed by a processor, implement a method for synchronizing configuration data of a real-time database system as described above.

[0083] In a specific embodiment, the present invention is applied to a distributed real-time database cluster, which has two clusters: a real-time service cluster and a storage service cluster. Based on the solution of the present invention, the configuration related to the real-time service and the configuration related to the historical storage service are synchronized through two-level nodes. Furthermore, the historical storage service provides a secondary subscription service for the historical query service, further synchronizing the configuration to the historical data query service.

[0084] In summary, this invention discloses a method, system, device, and medium for synchronizing configuration data in a real-time database system. This invention sets the client's configuration data update to a high-availability mode, without clustering, to ensure the consistency of configuration data writes. At the same time, the configuration is distributed and deployed to the cluster services of each business subsystem. Through the interconnectivity of the cluster, multiple masters and multiple slaves are achieved, ensuring the timeliness and eventual consistency of data synchronization.

[0085] Compared with traditional cluster data systems, this invention offers the following advantages through strategies such as multi-master multi-slave, business separation, and two-phase (or multi-phase) deployment:

[0086] (1) It ensures the high availability of the real-time database configuration system.

[0087] (2) Improve the consistency of the real-time database configuration system and reduce maintenance costs.

[0088] (3) Improved the balance of configuration data synchronization in the cluster and reduced the network pressure on the system.

[0089] (4) It improves the scalability of the configuration system, meets the compatibility of different modes such as single machine, dual machine redundancy, and cluster, and reduces the development cost of the system.

[0090] (5) Improve the overall stability of the system and avoid excessive load on a single server.

[0091] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0093] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0094] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0095] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0096] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A method for synchronizing configuration data in a real-time database system, characterized in that, include: When the configuration client publishes configuration data for configuration updates, the business configuration synchronization module of each node in the business subsystem receives the configuration data distributed by the configuration server and synchronizes and updates the configuration data to its respective node according to the configuration update strategy. The configuration client, configuration server, and business subsystem are all components of a distributed real-time database system; each node in the business subsystem is equipped with a configuration synchronization module; the business subsystem includes n-level nodes, and the n-th level node subscribes to data from at least one node of the previous level, where n is a natural number greater than or equal to 1. The configuration update strategy involves each level node selecting the configuration data of the highest priority node among all subscribed parent nodes for configuration synchronization.

2. The method for synchronizing configuration data in a real-time database system as described in claim 1, characterized in that, The deployment modes of the business subsystems include: single-machine mode, dual-machine redundancy mode, and cluster mode.

3. The method for synchronizing configuration data in a real-time database system as described in claim 2, characterized in that, In the business subsystems of dual-machine redundancy mode and cluster mode, the first-level node is the management master node selected according to preset selection conditions, and the second-level node that subscribes to the management master node information is the subscription slave node. The management master node and the subscription slave node change according to the node update rules, and the service configuration synchronization module in any of the management master node and the subscription slave node obtains the location information of the node to which the service configuration synchronization module belongs in real time. The service configuration synchronization module in the subscription slave node is also used to obtain the management master node information subscribed to by the node to which the service configuration synchronization module belongs in real time. The service configuration synchronization module within the subscription slave node confirms the information of the next-level node receiving the configuration data based on the location information, the subscription management master node information, and the configuration update strategy.

4. The method for synchronizing configuration data in a real-time database system as described in claim 2, characterized in that, In single-player mode The configuration server is configured as the management master node. When the configuration server has the distribution function, it is also used to distribute the configuration data published by the configuration client to each business subsystem, and each business subsystem only needs to perform synchronization once.

5. The method for synchronizing configuration data in a real-time database system as described in claim 2, characterized in that, In cluster mode, In each business subsystem, the first-level node will send the acquired configuration data to the second-level node with which a subscription relationship has been established; After receiving multiple sets of configuration data, the Level 2 node compares the configuration versions of each Level 1 node and selects the Level 1 node with the latest version and the highest priority as the configuration update data source for data synchronization.

6. The method for synchronizing configuration data in a real-time database system as described in claim 5, characterized in that, The configuration versions of each Level 1 node are compared, including by version number, version ID, and version release time. The version number increments unidirectionally. Only when the number and ID are equal are the configuration versions determined to be equal and no configuration synchronization is performed. Otherwise, if the version number of the first-level node is newer and the version ID of the same number is consistent, only incremental update is performed. Otherwise, a full configuration update is performed and the configuration of the business subsystem is reloaded.

7. The method for synchronizing configuration data in a real-time database system as described in claim 2, characterized in that, When the second-level node is online and all first-level nodes are down, the second-level node is automatically promoted to a first-level node in order to synchronize data with the configuration server; If the original Level 1 node recovers, the automatically upgraded Level 2 node will revert to its un-upgraded working state.

8. A synchronization system for configuration data in a real-time database system, characterized in that, include: The configuration client is configured to publish configuration data for configuration updates; The configuration server is configured to acquire configuration data published by the configuration client and distribute it to various business subsystems for multi-level node sequential distribution, so as to keep the data of various nodes in the same business subsystem synchronized. The business subsystem comprises n-level nodes, wherein each n-level node subscribes to data from at least one node in its parent level, and n is a natural number greater than or equal to 1. In each business subsystem The first-level node will send the acquired configuration information to the second-level node with which it has established a subscription relationship; The second-level node subscribes to at least one of the first-level nodes. After receiving multiple sets of configuration data, it compares the configuration versions of each first-level node and selects the first-level node with the latest version and the highest priority as the configuration update data source for data synchronization.

9. A device for synchronizing configuration data in a real-time database system, characterized in that, include: At least one database; The system includes a memory communicatively connected to the at least one database; wherein the memory stores instructions executable by the at least one database to enable the at least one database to perform a method for synchronizing configuration data in a real-time database system as described in any one of claims 1-7.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that, When the executable instructions are executed by the processor, they implement a method for synchronizing configuration data in a real-time database system as described in any one of claims 1-7.

Images