Data synchronization method, node and system
By sending messages to edge nodes in a distributed database and generating parallel synchronization data packets, the problem of maintaining consistency between data nodes is solved, data synchronization efficiency is improved, and synchronization process is simplified.
Patent Information
- Application Number
- CN202410225155.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-28
- Publication Date
- 2025-01-07
- Estimated Expiration
- 2044-02-28
AI Technical Summary
In distributed databases, data consistency between data nodes is difficult to achieve, especially when there are many nodes and the data synchronization process is cumbersome, the efficiency is inefficient.
By sending messages to edge nodes at the central node, generating and synchronizing data packets in parallel, multi-terminal data synchronization is realized, and the timer mechanism is used to wait for feedback information to ensure data consistency.
It improves the efficiency of data synchronization in distributed databases, simplifies the process of maintaining consistency of multi-terminal data, and reduces synchronization delay.
Smart Images

Figure CN119271740A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data processing, and in particular to a data synchronization method, node and system. Background Art
[0002] With the rapid growth of data volume, the application of distributed databases is becoming more and more widespread. A distributed database refers to a logically unified database composed of multiple physically dispersed data storage nodes connected by a high-speed computer network. Therefore, a distributed database can store data in different data nodes connected by a network, thereby obtaining a larger storage capacity and higher concurrent access volume.
[0003] Since the data in a distributed database is stored in different data nodes, how to maintain consistency between these different data nodes becomes an urgent problem to be solved. Summary of the invention
[0004] To solve the above problems, the embodiments of the present application provide a data synchronization method, node and system, which can realize multi-terminal synchronization between data nodes in a distributed database with high efficiency, and ensure that the data in each data node remains consistent.
[0005] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:
[0006] In a first aspect, a data synchronization method is provided, which is applied to a first node in a distributed data system, wherein the distributed data system further includes a plurality of second nodes, and the method includes: sending a first message to each second node. The first message is used to instruct the second node to feedback newly added data. Receive synchronization information from the second node. The synchronization information indicates the newly added data. Based on the synchronization information and the data in the first node, a synchronization data packet is sent to each second node. The synchronization data packet is obtained based on the newly added data fed back by each second node and the data in the first node.
[0007] Based on this solution, when data changes occur in a data node in a distributed database, the central node (i.e., the first node) can simultaneously initiate multi-end synchronization in each edge node (i.e., the second node), generate a synchronization data packet based on the data to be synchronized (i.e., the new data) of each data node, and synchronize the synchronization data packet to each data node in parallel, thereby achieving data synchronization of each data node with higher efficiency.
[0008] In some possible implementations, sending the first message to each second node includes: sending the first message to each second node when a first synchronization trigger condition is detected. The first synchronization trigger condition includes at least one of the following: there is new data in the first node, the first node receives a first synchronization request for requesting to initiate data synchronization, and the first node receives a first synchronization instruction for instructing to initiate data synchronization.
[0009] In some possible implementations, based on the feedback information and the data in the first node, a synchronization data packet is sent to each second node, including: performing a first processing on the newly added data fed back by each second node and the data in the first node to obtain a synchronization data packet. A synchronization data packet is sent to each second node. Among them, the first processing is used to add data in the newly added data fed back by each second node that has a different index value from the data of the first node, data in the newly added data of the first node that has a different index value from the newly added data fed back by each second node, data in the newly added data fed back by each second node that has the same index value as the data of the first node and has the latest timestamp to the synchronization data packet, and data in the newly added data fed back by each second node that has the same index value as the data of the first node and has a timestamp that is not the latest, and then renaming the index value and adding it to the synchronization data packet.
[0010] In some possible implementations, the method further includes: when there is new data in the first node, using the synchronization data packet to cover the new data in the first node. When there is no new data in the first node, storing the synchronization data packet.
[0011] In some possible implementations, when there is data in the first node whose operation record value is greater than the synchronization water level value, there is new data in the first node, the operation record value is used to mark the position of the data in the database, and the synchronization water level value is used to indicate the maximum operation record value during the last data synchronization.
[0012] In some possible implementations, the method further includes: starting a first timer. The timing upper limit of the first timer is a first preset duration. Before the first timer expires, if feedback information returned by the second node is received, the first timer is triggered to restart. The feedback information is used to indicate whether the corresponding second node participates in this data synchronization.
[0013] In some possible implementations, before the first timer expires, if feedback information returned by the second node is received, the first timer is triggered to restart timing, including: before the first timer expires, if first feedback information returned by the second node is received, the first timer is triggered to restart timing, and the first feedback information is used to indicate that the corresponding second node participates in this data synchronization. The method also includes: starting a second timer corresponding to the second node. The timing upper limit of the second timer is a second preset duration. Before the second timer expires, if synchronization information sent by the second node corresponding to the second timer is received, the newly added data indicated by the synchronization information is stored.
[0014] In some possible implementations, the method further includes: obtaining an online device list, wherein the online device list includes online nodes in the distributed data system, and taking the online nodes in the online device list other than the first node as a plurality of second nodes.
[0015] In a second aspect, a data synchronization method is provided, which is applied to a second node in a distributed data system, where the second node is any one of a plurality of second nodes in the distributed data system. The method comprises: determining synchronization information in response to receiving a first message sent by a first node. The first message is used to instruct the second node to feedback newly added data. The synchronization information is used to indicate the newly added data. The synchronization information is sent to the first node. The synchronization data packet from the first node is stored. The synchronization data packet is obtained according to the newly added data fed back by each second node and the data in the first node.
[0016] Based on this scheme, when the edge node (i.e., the second node) in the distributed database receives the first message sent by the central node (i.e., the first node), it will send its own newly added data to the central node, so that the central node can generate a synchronization data packet based on the data to be synchronized (i.e., the newly added data) of each data node, and synchronize the synchronization data packet to each data node in parallel, thereby achieving data synchronization of each data node with higher efficiency.
[0017] In some possible implementations, after receiving the synchronization data packet from the first node, the method further includes: when there is new data in the second node, using the synchronization data packet to cover the new data in the second node. When there is no new data in the second node, storing the synchronization data packet.
[0018] In some possible implementations, in response to receiving a first message sent by a first node, determining synchronization information includes: in response to receiving the first message sent by the first node, obtaining the state and configuration information of the second node. The state of the second node is used to indicate the device state of the second node, and the configuration information is used to indicate the conditions for the second node to participate in data synchronization. When the state of the second node meets the requirements of the configuration information, first feedback information is fed back to the first node. The first feedback information is used to indicate that the second node participates in this data synchronization. Determine the synchronization information. When the state of the second node does not meet the requirements of the configuration information, second feedback information is fed back to the first node. The second feedback information is used to indicate that the second node does not participate in this data synchronization.
[0019] In some possible implementations, the state of the second node includes whether it is in a charging mode, whether it is in a screen-on state, whether it is in a low-power state, and whether it is in a local area network. The configuration information includes any one or any combination of the following: the second node is in a charging mode. The second node is in a screen-on state. The second node is in a local area network. The second node is in a non-low-power state.
[0020] In some possible implementations, the method further includes: when a second synchronization trigger condition is detected, sending a second synchronization request to the first node. The second synchronization trigger condition includes at least one of the following: the second node is online for the first time in the distributed data system, and the synchronization water level value in the second node is greater than the maximum operation record value. The operation record value is used to mark the location of the data in the database, and the synchronization water level value is used to indicate the maximum operation record value during the last data synchronization. When receiving a full data packet sent by the first node, the full data packet is stored in the second node. The full data packet is all the data in the first node.
[0021] In a third aspect, a data synchronization system is provided, which is applied to a distributed data system. The system includes: a first node and multiple second nodes. The first node is used to send a first message to each second node. The first message is used to instruct the second node to feedback the newly added data. The second node is used to send synchronization information to the first node in response to receiving the first message sent by the first node. The synchronization information is used to indicate the newly added data. The first node is also used to send a synchronization data packet to each second node based on the synchronization information from each second node and the data in the first node. The synchronization data packet is obtained based on the newly added data fed back by each second node and the data in the first node. The second node is also used to receive the synchronization data packet and store the synchronization data packet.
[0022] In a fourth aspect, a node is provided, comprising: one or more processors, one or more memories, and a communication module for implementing a communication connection. The one or more processors are connected to the communication module. The one or more memories are coupled to the one or more processors, and the one or more memories store a computer program. When the one or more processors execute the computer program, the node executes the data synchronization method of any one of claims 1 to 8. Or, executes the data synchronization method of any one of claims 9 to 13.
[0023] It should be understood that the node may be the first node or the second node.
[0024] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the data synchronization method of any one of the first aspect is implemented, or the data synchronization method of any one of the second aspect is implemented.
[0025] In a sixth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the data synchronization method of any one of the first aspect, or implements the data synchronization method of any one of the second aspect.
[0026] It should be understood that the technical solutions provided in the second, third and fourth aspects above and their technical features can all correspond to the data synchronization methods provided in the first, second aspects and their possible designs, and therefore the beneficial effects that can be achieved are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 A schematic diagram of a distributed database provided in an embodiment of the present application;
[0028] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;
[0029] Figure 3 A schematic diagram of a software framework of an electronic device provided in an embodiment of the present application;
[0030] Figure 4 A schematic diagram of a data synchronization system provided in an embodiment of the present application;
[0031] Figure 5 A flowchart of a data synchronization method provided in an embodiment of the present application;
[0032] Figure 6 A schematic diagram of an operation record value in a database provided in an embodiment of the present application;
[0033] Figure 7 A flowchart of a synchronous waiting mechanism provided in an embodiment of the present application;
[0034] Figure 8 A schematic diagram of a process for generating feedback information by an edge node provided in an embodiment of the present application;
[0035] Fig. 9 A schematic diagram of various moments in a synchronous waiting mechanism provided in an embodiment of the present application;
[0036] Fig.10 A schematic diagram of data to be synchronized in each data node provided in an embodiment of the present application;
[0037] Fig.11 A flowchart of another data synchronization method provided in an embodiment of the present application;
[0038] Fig.12 A flowchart of another data synchronization method provided in an embodiment of the present application;
[0039] Fig.13 A schematic diagram of an operation record value in a database provided in an embodiment of the present application;
[0040] Fig.14 An interactive schematic diagram of a data synchronization method provided in an embodiment of the present application;
[0041] Fig.15 A flowchart of an execution of a data synchronization method provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] The words "first", "second" and "third" in the embodiments of the present application are used to distinguish different objects rather than to limit a specific order. In addition, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.
[0043] In the digital age, as the scale of data continues to grow, the pressure on data storage and management continues to increase. Distributed databases are increasingly widely used due to their high availability, easy scalability, and high access efficiency. Among them, high availability means that in a distributed database, data is distributed on different data nodes, so even if some data nodes fail, the data is still available. Easy scalability means that a distributed database can cope with the increase in data scale by adding data nodes without having to reconstruct the entire system. High access efficiency means that users can access data concurrently through different data nodes in a distributed database, and the access speed is fast.
[0044] In a distributed database, data access and processing usually need to be performed on multiple data nodes simultaneously, so the data in different data nodes need to be consistent. The data in different data nodes can be kept consistent through data synchronization. For example, when the data in one or some data nodes changes, the changed data will be synchronized to other data nodes to ensure that the data in all data nodes remains consistent.
[0045] Data nodes in a distributed database can be divided into central nodes and edge nodes according to different permissions. Central nodes refer to data nodes with permissions such as data writing, data reading, and data processing. Edge nodes refer to data nodes that only have permissions to write and read data, but not data processing.
[0046] In the embodiment of the present application, data processing may refer to modifying and calculating data, etc. The central node may also be referred to as a central device or a master node, and the edge node may also be referred to as an edge device or a slave node, which will not be described in detail later.
[0047] The current synchronization method of distributed databases is usually two-end synchronization. Two-end synchronization can also be called 1v1 synchronization, which refers to data synchronization between two data nodes. Two-end synchronization is a serial synchronization method. When a distributed database includes multiple data nodes, the process of maintaining consistency of data in all data nodes through two-end synchronization is relatively cumbersome.
[0048] Below Figure 1 The distributed database shown is used as an example for exemplary description.
[0049] Please refer to Figure 1 , is a schematic diagram of a distributed database provided in an embodiment of the present application. Figure 1 As shown, the data nodes in the distributed database include mobile phones, tablets, personal computers, network attached storage (NAS) devices, watches, disaster recovery modules, etc. Among them, the NAS device is the central node, and the mobile phone, tablet, personal computer, and watch are edge nodes. The NAS device is connected to the mobile phone, tablet, personal computer, watch, and disaster recovery module ( Figure 1 The disaster recovery module is used to provide disaster recovery function for the distributed database, such as providing backup data when data in the NAS device is lost. For ease of explanation, the following takes the edge nodes in the distributed database including mobile phones, tablets, and personal computers as an example for introduction.
[0050] At the initial moment, the data in each data node may be as shown in Table 1, that is, the data in each data node is data a.
[0051] Data Node Data in the data node NAS Devices Dataa cell phone Dataa flat Dataa PC Dataa
[0052] Table 1
[0053] At the first moment after the initial moment, data b is added to the mobile phone, and the data in other data nodes remains unchanged.
[0054] The data in each data node is shown in Table 2.
[0055]
[0056]
[0057] Table 2
[0058] Since the data in the mobile phone has changed, the mobile phone will initiate data synchronization to the NAS device and synchronize data b to the NAS device. In this way, the data in the NAS device becomes data a and data b. Then, the NAS device will synchronize data b to the tablet and personal computer in turn. However, during the above data synchronization process, if the data in the tablet or personal computer also changes, the data synchronization process will be greatly extended. For example, the NAS device first synchronizes data b to the tablet; before the NAS device synchronizes data b to the personal computer or when synchronizing data b to the personal computer, data n is added to the personal computer. Then, the data synchronization between the NAS device and the personal computer includes both the NAS device synchronizing data b to the personal computer and the personal computer synchronizing data n to the NAS device. At this time, the data in each data node is shown in Table 3.
[0059] Data Node Data in the data node NAS Devices Data a, data b, data n cell phone Data a, Data b flat Data a, Data b PC Data a, data b, data n
[0060] Table 3
[0061] As can be seen from Table 3, after the above data synchronization is completed between the NAS device and the personal computer, the data in each data node is not consistent. The NAS device also needs to synchronize data n to the mobile phone and tablet in turn to make the data in each data node consistent.
[0062] Based on the above description, it can be seen that when there are many data nodes, the use of a serial dual-end synchronization solution will make the data synchronization process more cumbersome and time-consuming, and inefficient.
[0063] In order to solve the above problems, the embodiments of the present application provide a data synchronization method, node and system, which can support multi-terminal data synchronization of distributed databases, thereby making the data synchronization process more convenient and improving data synchronization efficiency.
[0064] In the embodiment of the present application, the data node may be an electronic device such as a smart wearable device, a tablet, a laptop computer, a desktop computer, a mobile phone, a NAS device, etc., which is not specifically limited here.
[0065] As an example, see Figure 2 , is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The data synchronization method provided in an embodiment of the present application can be applied to Figure 2 The electronic device 200 shown in the figure. The structure of the electronic device provided in the embodiment of the present application may also include Figure 2 The structure of the electronic device shown in FIG. Figure 2 As shown, the electronic device 200 may include a processor 201 and a communication module 202 .
[0066] The processor 201 may include one or more processing units. For example, the processing unit may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video stream codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. ( Figure 2 Different processing units may be independent devices or integrated into one or more processors 201.
[0067] The processing unit may further include an internal memory 211 for storing instructions and data. In the embodiment of the present application, when the electronic device 200 is a data node in a distributed database, the internal memory 211 may be used to store data in the distributed database.
[0068] The internal memory 211 may also be used to store computer executable program codes, which include instructions. The processor 201 executes various functional applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 211 .
[0069] Exemplarily, the internal memory 211 may store one or more computer programs corresponding to the data synchronization method provided in the embodiment of the present application. In this way, the processor 201 can execute the data synchronization method provided in the embodiment of the present application by running the computer program.
[0070] The communication module 202 is used to provide a wireless communication solution for the electronic device 200. The wireless communication may include 2G (second generation, second generation mobile communication technology), 3G (3rd generation, third generation mobile communication technology), 4G (fourth generation, fourth generation mobile communication technology), 5G (5th generation, fourth generation mobile communication technology), wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.
[0071] In the embodiment of the present application, when the electronic device 200 is a data node in a distributed database, the communication module 202 can be used to realize data transmission and communication with other data nodes. Exemplarily, data synchronization can be realized between the data nodes through the communication module 202.
[0072] It should be understood that Figure 2 The schematic structure does not constitute a specific limitation on the electronic device 200. For example, the electronic device involved in the embodiment of the present application may include: Figure 2 The electronic device shown may have more or fewer components, or may combine or separate certain components, or may have different component arrangements, which are not specifically limited herein.
[0073] The hardware structure of the electronic device in the embodiment of the present application is described above. For ease of understanding, before introducing the software framework of the electronic device in the embodiment of the present application, the data storage method of the distributed database in the embodiment of the present application is first introduced.
[0074] In a distributed database, the stored data is recorded through a database operation record table. Exemplarily, the distributed database includes a central node C, an edge node S1 and an edge node S2. The database operation record table of the database in the edge node S2 may be as shown in Table 4.
[0075] index key conflict src value timestamp 0 Record 1 false S1 v1 2 1 Record 2 false S1 v2 3 2 Record 3 false S1 v3 5
[0076] Table 4
[0077] Referring to Table 4, it can be seen that in the database operation record table of the distributed database, data can be recorded through six parameters, namely, index, key, conflict, src, value, and timestamp. The six parameters are introduced below respectively.
[0078] Index can be called an operation record value, which is used to mark the location of data in the database. In other words, index is used to mark the spatial location occupied by data in the database. Exemplarily, index can be a positive integer, such as 0, 1, 2, 3, etc. Every time a new piece of data is added to the database, the index of the newly added data can be increased by 1 by the index of the last newly added data. For example, if the user changes the value of the data with index 0 and key record 1 in edge node S2 from v1 to v4, the database operation record table in edge node S2 can be changed to Table 5.
[0079] index key conflict src value timestamp 0 Record 1 false S1 v1 2 1 Record 2 false S1 v2 3 2 Record 3 false S1 v3 5 3 Record 1 false S2 v4 7
[0080] Table 5
[0081] It can be seen that compared with Table 4, Table 5 has a new data with index 3, key record 1, conflict false, src S2, value v7, and timestamp 7. This data is the data added due to data change. Before the data change, the index of the previous new data was 2, so the index of the new data is 3.
[0082] The key can be called an index value, which is the index of the data and is used to identify the data. The data node can query or retrieve the data in the database through the index value. In the database, there may be two or more data with the same key, which means that the index values of the two or more data are the same. In this case, the data node can use the data with a larger timestamp as the data corresponding to the index value. For example, the database operation record table of the edge node S2 is shown in Table 5. There are two data with the key of record 1 in the table, one of which has a timestamp of 2 and the other has a timestamp of 7. When the edge node S2 queries or retrieves data, it will use the data with a timestamp of 7 as the data corresponding to record 1.
[0083] Conflict is used to identify whether the data is generated due to data conflict. Data conflict refers to data with the same key. In the embodiment of the present application, only the central node in the distributed database has the authority to modify the conflict of data. When the central node receives data with the same key as the data in the central database, it will compare the timestamp of the data with the same key, rename the key of the data with the same key except the data with the largest timestamp, and save it after setting conflict to true. The central database refers to the database in the central node. The following is an example.
[0084] Exemplarily, the database operation record table in the central node C is the same as the above Table 4. When the central node C receives a piece of data to be synchronized with the key of record 1, conflict is false, src is s2, value is v4, and timestamp is 7 sent by the edge node S2, the central node C can judge through its own database operation record table that there is data in the central database with the same key as the data to be synchronized, that is, the data with index 0, key record 1, conflict is false, src is s1, value is v1, and timestamp is 2 in the above Table 4, and then the central node compares the timestamps of the two pieces of data. It can be seen that the timestamp of the data to be synchronized is greater than the timestamp of the data with index 0 in the central database, then the central node can rename the key of the data with index 0 in the central database, such as renaming it to record 1_bak, record 1_backup, etc., modify the confict from false to true, and then save it again as the data with index 3, which is the data generated by the above data conflict. In addition, the central node will also save the data to be synchronized sent by the edge node S2. As an example, after the above data changes, the database operation record table in the central node C may be as shown in Table 6 below.
[0085] index key conflict src value timestamp 0 Record 1 false S1 v1 2 1 Record 2 false S1 v2 3 2 Record 3 false S1 v3 5 3 Record 1_bak true S1 v1 2 4 Record 1 false S2 v4 7
[0086] Table 6
[0087] src is used to indicate the source of the corresponding data. Taking the data in Table 6 as an example, src is s1, which means the corresponding data comes from edge node s1, and src is s2, which means the corresponding data comes from edge node s2.
[0088] Value is used to indicate the specific value of data. For example, if the stored data is a string, the value of the data is the string.
[0089] Timestamp can also be called a timestamp, which is used to indicate the time when the data was generated.
[0090] Based on the above-introduced data storage method of the distributed database, the software architecture of the electronic device provided by the embodiment of the present application is described below. The electronic device in this embodiment refers to a data node in the distributed database.
[0091] Please refer to Figure 3 , is a schematic diagram of a software framework of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the software framework of the electronic device 300 may include a synchronization calculation unit 301, a synchronization cache module 302, a synchronization control module 303, a synchronization message monitoring module 304, a status monitoring module 305, an account management module 306, a database operation engine 307, a communication message adaptation module 308, a data storage module 309, etc.
[0092] It should be noted that when the electronic device is used as an edge node in a distributed database, not all modules in the above software framework are necessary. For example, the synchronization cache module 302 may not be included in the software framework of the edge node. In other words, the synchronization cache module 302 in the edge node may not participate in the data synchronization process. Figure 3 The functional modules in the module are introduced in detail.
[0093] The synchronization calculation unit 301 is used to perform read and write operations on the database or obtain logs in the database. In addition, the synchronization calculation unit 301 in the central node can also be used to perform a first processing on the data to be synchronized sent by each edge node and the data in the central database to obtain a synchronization data packet. Among them, the central database refers to the database of the central node. Performing a first processing on the data to be synchronized sent by each edge node and the data in the central database refers to adding data in each first data that has a different index value from the data in the central database, data in the data to be synchronized in the central database that has a different index value from each first data, and data with the same index value as the data in the central database and the latest timestamp to the synchronization data packet, and renaming the index value of each first data and the data in the central database that has the same index value and the latest timestamp to add to the synchronization data packet. Among them, the index value refers to key, and the timestamp refers to timestamp.
[0094] The synchronization cache module 302 is set in the central node and is used to cache the to-be-synchronized data sent by each edge node. The synchronization calculation unit 301 performs comparison calculation based on the data cached in the synchronization cache module 302 and the data in the central database.
[0095] The synchronization control module 303 can be set in either a central node or an edge node, and is used to provide the data node with the ability to read and write the database, obtain incremental operation logs and the full database. Among them, the incremental operation log in the database is used to record data changes in the database. The full database refers to all the data in the database. Exemplarily, when a business initiates a database read and write request, the synchronization control module 303 can assign a database operation class for performing database operations to the business, and initiate a request to the database engine to obtain incremental operation logs or a full database. Among them, the database engine is a core service in the database for storing, processing and protecting data. In addition, the synchronization control module is also used to encapsulate the message body to be sent.
[0096] The synchronization message monitoring module 304 can be set in either a central node or an edge node. The synchronization message monitoring module 304 is used to receive messages sent by other data nodes. When receiving a synchronization message for indicating data synchronization, the synchronization message monitoring module 304 transmits the synchronization message to other modules, such as the status monitoring module 305, for processing.
[0097] The status monitoring module 305 can be set in either the central node or the edge node, and is responsible for monitoring the status of the data node, such as whether it is in charging mode, whether it is in the screen-on state, whether it is in the low-power state, whether it is in the local area network, whether it is online, etc. When the status monitoring module 305 receives a synchronization message, it can determine whether to participate in data synchronization based on the status of the data node and the user configuration. Among them, the user configuration may include any one or any combination of the following: support data synchronization when the data node is in charging mode, support data synchronization when the data node is in the screen-on state, support data synchronization when the data node is in the local area network, and support data synchronization when the data node is not in the low-power state.
[0098] The account management module 306 can be set in either a central node or an edge node for logging in or out of an account. Among them, the above account can be a UUID (universally unique identifier) that provides a unique identity authentication identifier in the database. In a distributed database, the premise for each data node to perform data synchronization is that the account logged in by each data node is the same. In some possible implementations, each data node in the distributed database can be an electronic device in a trust ring to ensure that the account logged in by each data node is the same. Among them, the trust ring can also be called a MagicRing trust ring, which is a technical solution based on an identity authentication system to achieve cross-system multi-device trusted interconnection. The accounts logged in by devices in the same trust ring are the same, and a trusted connection can be achieved between each device. In some other possible implementations. The address of the data node that logs in to this account can be stored in the account management module 306, and a synchronization notification or synchronization request is sent according to the address, thereby ensuring that the accounts logged in by each data node for data synchronization are the same.
[0099] The database operation engine 307 can be set in either a central node or an edge node, and is used to perform read and write encapsulation on the database, shield database operation differences, and provide database read and write interfaces and log acquisition interfaces, etc. The database operation engine 307 can support multiple types of data operations.
[0100] The communication message adaptation module 308 can be set in either the central node or the edge node to encapsulate the underlying link capability and provide a send interface. When the synchronization control module 303 calls the send interface to initiate data synchronization, the communication message adaptation module 308 can send a synchronization request to other data nodes through the underlying link.
[0101] The above is the software architecture of the data node in the distributed database provided by the embodiment of the present application. When performing data synchronization, each data node in the distributed database can be combined with the service cloud and the underlying link layer to form a data synchronization system.
[0102] Please refer to Figure 4 , is a schematic diagram of a data synchronization system provided in an embodiment of the present application. Figure 4 As shown, the data synchronization system 400 includes a business cloud 401, a data node 402 and an underlying link layer 403. The business cloud 401 includes a data cloud 411 and an account cloud 421. The data node 402 includes a central node 412 and multiple edge nodes 422.
[0103] Data Cloud 411 is a cloud storage database backup and disaster recovery module of a distributed database, which is used to regularly back up data of the central database, or to restore the central data based on the backup data when problems such as data loss occur in the central database.
[0104] The account cloud 421 is used to store and generate user account information. When each data node in the distributed database performs multi-terminal data synchronization, the data node can perform data isolation based on the user account in the account cloud 421 to avoid data sharing between different user accounts.
[0105] The bottom link layer 403 is used to realize data transmission between different data nodes. Exemplarily, the bottom link layer 403 can be a data transmission layer based on Magic Link, a data transmission layer based on Socket communication, a data transmission layer based on Http communication, etc.
[0106] Based on the above description, the data synchronization method provided in the embodiment of the present application is introduced below. The method is applied to a distributed database, such as a central node and an edge node in a distributed database. Among them, the distributed database can be a multi-terminal distributed database of a home storage center, a backend management system of various township enterprises and institutions, a hotel database management system, a cinema ticketing system, etc., which will not be described in detail here. In the embodiment of the present application, the distributed database can also be referred to as a distributed data system, which will not be described in detail later.
[0107] Since data synchronization refers to data synchronization between the central node and each edge node, for ease of explanation, the central node and the edge node are respectively introduced as the execution subjects. In the embodiment of the present application, the central node can also be called the first node, and the edge node can also be called the second node, which will not be repeated later.
[0108] First, we introduce the data synchronization method applied to the central node, which is an incremental synchronization method. Figure 5 , is a flow chart of a data synchronization method provided in an embodiment of the present application. The method is applied to a central node in a distributed database, such as Figure 5 As shown, the process includes the following steps.
[0109] S501. Send a first message to each second node.
[0110] The first message is used to instruct the second node to feedback the newly added data, which can also be called an incremental synchronization notification. The newly added data can also be called data to be synchronized, first data, etc. S510 can also be stated as: sending an incremental synchronization notification to each edge node. In an embodiment of the present application, the trigger condition for the central node to execute S510 can be the detection of an incremental synchronization trigger condition. Among them, the incremental synchronization trigger condition can also be called the first synchronization trigger condition. That is to say, the central node can send an incremental synchronization notification to each edge node when the incremental synchronization trigger condition is detected.
[0111] The incremental synchronization trigger condition is any one or more of the following combinations: new data is added to the central database; the central node receives an incremental synchronization request for requesting incremental synchronization; the central node receives an incremental synchronization instruction for instructing to initiate incremental synchronization. In an embodiment of the present application, data synchronization can be divided into incremental synchronization and full synchronization. Incremental synchronization refers to synchronizing changed data, and full synchronization refers to synchronizing all data including changed data. The incremental synchronization request in the above synchronization trigger condition is used to request the central node to initiate incremental synchronization, and the incremental synchronization instruction is used to instruct the central node to initiate incremental synchronization.
[0112] In the embodiment of the present application, the incremental synchronization request may also be referred to as a first synchronization request, and the incremental synchronization instruction may also be referred to as a first synchronization instruction.
[0113] The central database can generate new data for users to modify the data in the central database, and can also store new data in the central database for users.
[0114] The incremental synchronization request is a message sent by the edge node to request the central node to initiate incremental data synchronization. When the edge node adds new data in its own database, it sends an incremental synchronization request to the central node.
[0115] In an embodiment of the present application, data nodes such as central nodes and edge nodes can determine whether there is new data in their own database through the two parameters of index and lastindex. Among them, index is the operation record value introduced in the above embodiment, which is used to mark the position of the data in the database, and will not be repeated here. Lastindex can be called the synchronization water level value, which refers to the maximum index at the time of the last data synchronization. The data node can determine that there is new data in its own database when there is data with an index greater than lastindex in the database operation record table, and determine that there is no new data in its own database when there is no data with an index greater than lastindex.
[0116] For example, the distributed database includes a central node C, an edge node S1 and an edge node S2, and the index information in the database operation record table of each data node is as follows: Figure 6 As shown. Among them, the lastindex of each data node is 4. Edge nodes S1 and S2 can determine that there is new data in their own databases by judging that there is data with an index greater than lastindex in their own databases. Central node C can determine that there is no new data in the central database by judging that there is no data with an index greater than lastindex in the central database.
[0117] The incremental synchronization instruction is an instruction issued by the user to the central node to instruct the central node to initiate incremental synchronization. The central node can issue the incremental synchronization instruction to the central node based on the preset interaction logic. For example, an incremental synchronization button or incremental synchronization control can be set in the central node, and the user issues the incremental synchronization instruction to the central node by pressing the incremental synchronization button or clicking the incremental synchronization control.
[0118] The incremental synchronization notification is used to instruct each edge node to return the data to be synchronized. In the incremental synchronization mode, the data to be synchronized is the newly added data. The method for determining whether the newly added data is added can refer to the above, which will not be repeated here.
[0119] The data node can determine the data to be synchronized by using the two parameters index and lastindex. Specifically, the data node can determine the data in the database operation record table with an index greater than lastindex as the data to be synchronized. For example, the distributed database includes a central node C, edge nodes S1 and S2, and the index information in the database operation record table of each data node is as follows: Figure 6 As shown. The edge node S1 can use the data with index greater than lastindex, that is, data with index 5, 6, and 7 as data to be synchronized, and the edge node can use the data with index 5 and 6 as data to be synchronized. In the embodiment of the present application, for the convenience of explanation, the data to be synchronized in the edge node can also be referred to as the first data.
[0120] After the central node sends an incremental synchronization notification to each edge node, it can enable a synchronization waiting mechanism to wait for each edge node to return the data to be synchronized.
[0121] In the synchronization waiting mechanism, the waiting time of the central node can be controlled by a timer. The waiting time includes the time the central node waits for each edge node to return feedback information and the time the central node waits for each edge node to return the data to be synchronized. Among them, the feedback information is the information generated by the edge node in response to the incremental synchronization notification, which is used to indicate whether the corresponding edge node participates in this data synchronization.
[0122] Exemplarily, a first timer and multiple second timers may be provided in the central node. The first timer is a global timer, which is used to control the time the central node waits for each edge node to return feedback information. The second timer is a timer for a single edge node, which is used to control the time the central node waits for the corresponding edge node to return the data to be synchronized. The upper limit of the first timer may be greater than or equal to 3s and less than or equal to 10s, such as 5s, 7s, etc. The upper limit of the second timer may be greater than or equal to 20s and less than or equal to 40s, such as 30s, 32s, etc. The first timer and the second timer will stop timing when they reach the corresponding upper limit.
[0123] Please refer to Figure 7 , is a flow chart of a synchronous waiting mechanism provided in an embodiment of the present application. The synchronous waiting mechanism is executed by a central node. Figure 7 As shown, the process includes the following steps.
[0124] S701: When the central node sends an incremental synchronization notification to each edge node, start a first timer.
[0125] Here, starting the first timer means starting to count until the upper limit of the first timer. For example, if the upper limit of the first timer is 5 seconds, starting the first timer means starting to count for 5 seconds, and the counting ends after 5 seconds.
[0126] Before the first timer expires, if the central node receives feedback information sent by the edge node, the first timer will be refreshed. Refreshing the first timer means refreshing the existing timer to 0. For example, if the first timer receives feedback information sent by an edge node at 2s, the timer of the first timer will be refreshed to 0, that is, the timer of 5s will be restarted.
[0127] In the embodiment of the present application, the feedback information can be divided into first feedback information and second feedback information. The first feedback information is used to indicate that the corresponding edge node participates in the current data synchronization. The second feedback information is used to indicate that the corresponding edge node does not participate in the current data synchronization.
[0128] The edge node can determine whether to generate the first feedback information or the second feedback information based on its own status and configuration information, that is, determine whether to participate in this data synchronization. Among them, the status of the edge node is used to indicate the device status of the edge node or the status of each device in the device. For example, the status of the edge node may include whether it is in charging mode, whether it is in a screen-on state, whether it is in a low-power state, whether it is in a local area network, etc. The configuration information of the edge node is used to indicate the conditions for the edge node to participate in data synchronization. For example, the configuration information may include any one or any combination of the following: the data node is in charging mode; the data node is in a screen-on state; the data node is in a local area network; the data node is in a non-low-power state. In an embodiment of the present application, the state of the edge node only needs to meet any one of the configurations in the configuration information to meet the requirements of the configuration information.
[0129] For example, please refer to Figure 8 , is a schematic diagram of a process of generating feedback information by an edge node provided in an embodiment of the present application. The process can be executed by any edge node in a distributed database. Figure 8 As shown, the process includes the following steps.
[0130] S801. In response to receiving an incremental synchronization notification, obtain its own status and configuration information.
[0131] The status and configuration information of the edge nodes can be found in the above description and will not be repeated here.
[0132] S802: Determine whether the state of the device satisfies the requirements of the configuration information. If yes, execute S803a; if no, execute S803b.
[0133] S803a: Generate first feedback information and send the first feedback information to a central device.
[0134] In addition, when the edge node determines to participate in the data synchronization, that is, generates the first feedback information, it can determine its own data to be synchronized through the two parameters of index and lastindex, and send the data to be synchronized to the central node. The specific determination method can be referred to the description in the above embodiment, which will not be repeated here.
[0135] S803b: Generate second feedback information and send the second feedback information to the central device.
[0136] Exemplarily, the edge node configuration information includes user configuration 1, user configuration 2, user configuration 3, and user configuration 4. Among them, user configuration 1 means that the data node is in charging mode, user configuration 2 means that the data node is in a screen-on state, user configuration 3 means that the data node is in a local area network, and user configuration 4 means that the data node is in a non-low-battery state.
[0137] When the edge node is not in charging mode, not in the screen-on state, not in the local area network, and is in a low-battery state, the edge node can determine that its own state does not meet the requirements of the configuration information. The edge node can generate second feedback information and send the second feedback information to the central device to notify the central device that the edge node does not participate in this data synchronization.
[0138] When the edge node is in charging mode, or in a screen-on state, or in a local area network, or in a non-low-battery state, the edge node can determine that its own state meets the requirements of the configuration information, and then the edge node can generate first feedback information and send the first feedback information to the central device to notify the central device that the edge node participates in this data synchronization. Afterwards, the edge node can send the determined data to be synchronized to the central node.
[0139] In the embodiment of the present application, when a data node sends data to be synchronized, it can first encapsulate the data to be synchronized, and then send the encapsulated data to be synchronized, which is conducive to improving the security of data transmission.
[0140] When the first timer ends, if the central node does not receive feedback information, it can prompt that the data synchronization has failed; if it receives second feedback information sent by the edge node, it can prompt that the edge node does not participate in the data synchronization. The prompt of the failure of the data synchronization or the prompt of not participating in the data synchronization can be in the form of a pop-up window, bubble, etc., which is not limited in this application. The following specifically introduces the solution in which the central node receives the first feedback information before the first timer ends.
[0141] S702: Before the first timer expires, if the central node receives first feedback information sent by the edge node, the central node starts a second timer corresponding to the edge node and refreshes the first timer.
[0142] As described in the above embodiment, each edge node corresponds to a second timer. Starting the second timer means starting the timing until the upper limit of the second timer. For example, if the upper limit of the second timer is 30s, starting the second timer means starting the timing of 30s, and the timing ends after 30s.
[0143] Refreshing the first timer means refreshing the existing timing of the first timer to 0, which will not be described in detail here.
[0144] S703: Before the second timer expires, if the to-be-synchronized data returned by the edge node corresponding to the second timer is received, the to-be-synchronized data is stored in the synchronization cache module.
[0145] As described in the above embodiments, the synchronization cache module is used to cache the data to be synchronized sent by each edge node, so that the subsequent synchronization calculation unit can perform comparison calculation according to each data to be synchronized.
[0146] In addition, before the second timer expires, if the central node receives the to-be-synchronized data returned by the edge node corresponding to the second timer, the central node can stop the timing of the second timer while storing the synchronization data in the synchronization cache module.
[0147] It should be noted that, if the central node does not receive the to-be-synchronized data returned by the corresponding edge node before the second timer expires, the central node may be prompted that the data synchronization of the edge node has failed.
[0148] For ease of understanding, the above-mentioned synchronous waiting mechanism is introduced again through a specific example below. In this example, the distributed database includes a central node C, an edge node S1, an edge node S2, and an edge node S3. Each edge node is online. Among them, the timing upper limit of the first timer is 5s, and the timing upper limit of each second timer is 30s. The second timer corresponding to the edge node S1 is called the second timer T1, the second timer corresponding to the edge node S2 is called the second timer T2, and the second timer corresponding to the edge node S3 is called the second timer T3.
[0149] Please refer to Fig. 9 , is a schematic diagram of various moments in a synchronous waiting mechanism provided in an embodiment of the present application. Among them, the standard moment refers to a standard timing independent of the first timer and each second timer, which is used to more clearly illustrate the relative relationship between each moment in the first timer and each second timer. The moments related to t in the following description all refer to the standard moment.
[0150] like Fig. 9 As shown, at time t, the central node C sends an incremental synchronization notification to each edge node and synchronously starts the first timer. The first timer starts timing for a duration of 5 seconds.
[0151] At time t+2s, the first timer counts 2s. At this time, the central node C receives the first feedback information sent by the edge node S1. The first timer counts 2s and is refreshed to 0s, and the second timer T1 starts counting for 30s.
[0152] At t+3s, the first timer counts 1s and the second timer T1 counts 1s. At this time, the central node C receives the first feedback information sent by the edge node S2, the first timer counts from 1s to 0s, and the second timer T2 starts counting for 30 seconds.
[0153] At t+5s, the first timer is 2s, the second timer T1 is 3s, and the second timer T2 is 2s. At this time, the central node C receives the second feedback information sent by the edge node S3, and the first timer is refreshed from 2s to 0s. Since the second feedback information sent by the edge node S3 indicates that it does not participate in this data synchronization, the central node C will not start the second timer T3 corresponding to the edge node S3, and can prompt the edge node S3 not to participate in this data synchronization. In addition, since the central node C has received the feedback information of all edge nodes, it can also stop the timing of the first timer ( Fig. 9 (not shown in the figure), and this application does not limit this.
[0154] At t+10s, the first timer counts 5s, the second timer T1 counts 8s, the second timer T2 counts 7s, and the third timer T3 counts 5s. At this time, the first timer reaches the upper limit and stops counting.
[0155] At t+17s, the second timer T1 is set to 15s, and the second timer T2 is set to 14s. At this time, the central node C receives the data to be synchronized sent by the edge node S1. The central node C can stop the timing of the second timer T1 (e.g. Fig. 9 As shown), the second timer T1 can also automatically stop after the timing reaches the upper limit ( Fig. 9 In addition, the central node C will also store the received data to be synchronized into the synchronization cache module so that the subsequent synchronization calculation unit can perform comparison calculation.
[0156] At t+30s, the second timer T2 counts for 30s. The central node C still has not received the data to be synchronized sent by the edge node S2. The central node C can prompt the edge node S2 that the data synchronization has failed. The second timer T2 automatically stops.
[0157] The above is a specific example of executing the synchronization waiting mechanism after the central node sends an incremental synchronization notification, waiting for each edge node to return the data to be synchronized. Based on the above examples and descriptions, it should be understood that after the central node sends an incremental synchronization notification, the data to be synchronized returned by each edge node are stored in the synchronization cache module of the central node. The following introduces the steps after the central node receives the data to be synchronized returned by each edge node. In an embodiment of the present application, the data to be synchronized fed back by the edge node, that is, the newly added data fed back by the edge node can also be referred to as synchronization information.
[0158] S502: Receive synchronization information from the second node.
[0159] S503: Send a synchronization data packet to each second node based on the synchronization information and the data in the first node.
[0160] The synchronization data packet is obtained based on the newly added data fed back by each second node and the data in the first node. For example, the synchronization data packet can be obtained by the central node performing a first processing on each first data and the data in the central database. Among them, the first data is the newly added data fed back by the second node, which can also be referred to as the data to be synchronized in the embodiment of the present application.
[0161] The first processing is used to add data in each first data that has a different index value from the data in the central database, data in the central database to be synchronized that has a different index value from each first data, and data in each first data that has the same index value as the data in the central database and has the latest timestamp to the synchronization data packet; the first processing is also used to rename the index value of each first data and the data in the central database that has the same index value and a timestamp that is not the latest, and then add it to the synchronization data packet. The timestamp is the timestamp in the aforementioned embodiment. The central database refers to the database of the central node.
[0162] The first processing is described below by using a specific example. Exemplarily, the distributed database includes a central node C, an edge node S1, and an edge node S2.
[0163] Among them, the database operation record table of the central node C is shown in Table 7 below.
[0164] index key conflict src value timestamp 0 Record 1 false S2 v1 2 1 Record 2 false S2 v2 3 2 Record 3 false S2 v3 5 3 Record 4 false S2 v4 7 4 Record 5 false C v5 9
[0165] Table 7
[0166] The database operation record table of edge node S1 is shown in Table 8 below.
[0167]
[0168]
[0169] Table 8
[0170] The database operation record table of edge node S2 is shown in Table 9 below.
[0171] index key conflict src value timestamp 0 Record 1 false S2 v1 2 1 Record 2 false S2 v2 3 2 Record 3 false S2 v3 5 3 Record 4 false S2 v4 7 4 Record 7 false S2 v8 10 5 Record 3 false S2 v9 11
[0172] Table 9
[0173] In addition, the lastindex in the central database C, edge node S1, and edge node S2 is 3. Fig.10 , is a schematic diagram of data to be synchronized in each data node provided in an embodiment of the present application. Fig.10As shown, according to the method for determining the data to be synchronized introduced in the aforementioned embodiment, the data to be synchronized of the central node C is the data with index 4 in Table 7, the data to be synchronized of the edge node S1 (i.e., the first data) is the data with index 4 and index 5 in Table 8, and the data to be synchronized of the edge node S2 (i.e., the first data) is the data with index 4 and index 5 in Table 9.
[0174] Depend on Fig.10 It can be seen that each first data is the data to be synchronized in the edge node S1 and the data to be synchronized in the edge node S2, and the data of the central database is the data corresponding to Table 7. The data index values of the central database include record 1, record 2, record 3, record 4 and record 5. The index values of each first data include record 6, record 1, record 7, and record 3.
[0175] Therefore, the data in each first data having different index values from the data in the central database include: data with index 4 and index value record 6 in the data to be synchronized of edge node S1, and data with index 4 and index value record 7 in the data to be synchronized of edge node S2.
[0176] The data to be synchronized in the central database that have different index values from the first data include: the data with index 4 and index value record 5 in the data to be synchronized in the central node C.
[0177] The first data have the same index value as the data in the central database, and the data with the latest timestamp include: the data with index 5 and index value record 1 in the data to be synchronized of edge node S1, and the data with index 5 and index value record 3 in the data to be synchronized of edge node S2.
[0178] The data whose index value is the same as that of the data in the central database and whose timestamp is not the latest include: data whose index value is 0 and whose index value is record 1 in the central node C, and data whose index value is 2 and whose index value is record 3 in the central node C. The central node can rename the index value of the data whose index value is 0 and whose index value is record 1, such as renaming the index value from "record 1" to "record 1_bak", "record 1 backup", etc. The central node can also rename the index value of the data whose index value is 2 and whose index value is record 3, such as renaming the index value from "record 3" to "record 3_bak", "record 3 backup", etc.
[0179] It should be noted that the central node can compare the timestamps of each first data with the data in the central database with the same index value, add the data with the latest timestamp to the synchronization data packet, and rename the index value of the data with a less recent timestamp and add it to the synchronization data packet. This process can also be called conflict resolution.
[0180] Based on the above description, it should be understood that in the examples of Tables 7 to 9 above, the central node performs the first processing on each first data and the data of the central database to obtain the information of the synchronization data packet as shown in Table 10 below.
[0181]
[0182]
[0183] Table 10
[0184] The arrangement order of each data is only exemplary and is not limited in this application. In addition, the synchronization data packet may not include index information.
[0185] After the central node determines the synchronization data packet, it can determine whether there is data to be synchronized in the central database. If so, the synchronization data packet is used to overwrite the data to be synchronized in the central database; if not, the synchronization data packet is directly stored, such as storing the synchronization data packet in the synchronization cache module.
[0186] In addition, after receiving the synchronization data packet, each edge node participating in data synchronization can use the synchronization data packet to overwrite the data to be synchronized in its own database. In some possible implementations, if the data node does not have data to be synchronized, the synchronization data packet can be directly stored in its own database.
[0187] For example, the database operation record table of the central node C is shown in Table 7, and the data to be synchronized is as follows: Fig.10 As shown in , after the central node C uses the synchronization data packet shown in Table 10 to cover the data to be synchronized, the operation library record table can be shown in Table 11. Among them, when the synchronization data packet covers the data to be synchronized of the central node, the central node will configure the index value for the newly added data (that is, the data in the synchronization data packet).
[0188] index key conflict src value timestamp 0 Record 1 false S2 v1 2 1 Record 2 false S2 v2 3 2 Record 3 false S2 v3 5 3 Record 4 false S2 v4 7 4 Record 1_bak true S2 v1 2 5 Record 1 false S1 v7 10 6 Record 3_bak true S2 v3 5 7 Record 3 false S2 v9 11 8 Record 5 false C v5 9 9 Record 6 false S1 v6 9 10 Record 7 false S2 v8 10
[0189] Table 11
[0190] In addition, after the edge node S1 corresponding to Table 8 and the edge node S2 corresponding to Table 9 use the synchronization data packet shown in Table 10 to cover their own data to be synchronized, the database operation record table may be as shown in Table 11, which is not described in detail here.
[0191] It should be noted that after data synchronization is completed, each data node can refresh lastindex to the largest index in its own database. For example, in the above example, each data node can refresh lastindex from 3 to 10.
[0192] The above is the data synchronization method applied to the central node provided in the embodiment of the present application. The data synchronization method applied to the edge node is introduced below. Among them, the data synchronization method is an incremental synchronization method.
[0193] Please refer to Fig.11 , is a flow chart of another data synchronization method provided in an embodiment of the present application. The method is applied to edge nodes in a distributed database. Fig.11 As shown, the method includes the following steps.
[0194] S1101. In response to receiving a first message sent by a first node, determine synchronization information.
[0195] In the summary of the embodiments of the present application, S1101 may also be stated as: determining first data in response to receiving an incremental synchronization notification sent by a central node.
[0196] As described in the above embodiments, the central node will send an incremental synchronization notification to each edge node when detecting an incremental synchronization triggering condition. The incremental synchronization triggering condition includes the central node receiving an incremental synchronization request sent by an edge node.
[0197] The edge node will send an incremental synchronization request to the central node when new data is added to its own database. Among them, the edge node can determine whether there is new data in its own database through the two parameters of index and lastindex. The edge node can also determine the first data (i.e., the data to be synchronized) through the two parameters of index and lastindex. For details, please refer to the description in the previous embodiment, which will not be repeated here.
[0198] In addition, when the edge node receives the incremental synchronization notification, it can generate feedback information based on its own status and configuration information, and send the feedback information to the central node. The feedback information is used to indicate whether the edge node participates in this data synchronization, and can be divided into first feedback information indicating participation in this data synchronization, and second feedback information indicating non-participation in this data synchronization. If the edge node participates in this data synchronization, the first data can be sent synchronously when the first feedback information is sent to the central node. For details, please refer to the description in the aforementioned embodiment, which will not be repeated here.
[0199] S1102. Send synchronization information to the first node.
[0200] S1103. Store the synchronization data packet from the first node.
[0201] Specifically, after receiving the first data sent by each edge node, the central node can generate a synchronization data packet according to each first data and the data of the central database, and synchronize the synchronization data packet to the central database and the database of each edge node. After receiving the synchronization data packet, the edge node can use the synchronization data packet to overwrite the first data when the first data exists, and directly store the synchronization data packet when the first data does not exist, thereby realizing multi-terminal synchronization of each data node in the distributed database. For details, please refer to the description in the aforementioned embodiment, which will not be repeated here.
[0202] In some possible implementations, the data synchronization method, nodes and system provided in the embodiments of the present application can also achieve full synchronization, which is described in detail below.
[0203] Please refer to Fig.12 , is a flow chart of another data synchronization method provided in an embodiment of the present application. The method is applied to edge nodes in a distributed database. Fig.12 As shown, the method includes the following steps.
[0204] S1201. When a full synchronization trigger condition is detected, a full synchronization request is sent to a central node.
[0205] The full synchronization trigger conditions may include any of the following: going online for the first time in a distributed database; the lastindex in its own database is greater than the largest index. Among them, the first time an edge node goes online in a distributed database means that the edge node goes online as a data node of the distributed database for the first time. For example, each data node in a distributed database is an electronic device in the same trust ring, and the edge node goes online in the trust ring for the first time.
[0206] The edge node can determine whether it is the first time to go online in the distributed database by whether lastindex is 0. Specifically, when the edge node determines that lastindex is 0, it determines that it is the first time to go online in the distributed database. For example, the distributed database includes a central node C and an edge node S1. The index information in the database operation record table of each data node is as follows: Fig.13 As shown, the edge node S1 can determine that it is online for the first time in the distributed database according to the lastindex being 0, and send a full synchronization request to the central node C.
[0207] The lastindex in the own database is greater than the maximum index, which means that in the database operation record table of the own database, the lastindex is greater than the maximum index. Generally speaking, the lastindex is greater than the maximum index only when there is data loss in the database operation record table.
[0208] S1202: When receiving a full data packet sent by the central node, store the full data packet into its own database.
[0209] The full data packet is all the data in the central database. In response to receiving a full synchronization request sent by an edge node, the central node can send all the data in the central database to the edge node.
[0210] That is to say, in the embodiment of the present application, the full synchronization between data nodes can be 1v1 synchronization, that is, it is carried out between the edge node that initiates the full synchronization request and the central node.
[0211] In some possible implementations, the full synchronization process between data nodes can also be similar to the incremental synchronization method described in the aforementioned embodiment. Specifically, when the central node receives a full synchronization request sent by an edge node (such as a first edge node), it can complete incremental data synchronization with each edge node except the first edge node through the incremental synchronization method described in the aforementioned embodiment. Then, after the incremental data synchronization is completed, all data in the central database is sent to the first edge node as a full data packet. When the first edge node receives the full data packet sent by the central node, it can store the full data packet in its own database. In this way, the full synchronization between the data nodes can be completed.
[0212] Exemplarily, the distributed database includes a central node C, an edge node S1, and an edge node S2. Among them, the database operation record tables of the central node C and the edge node S1 are both empty, and the database operation record table of the edge node S2 is as shown in Table 9 in the above embodiment. When the edge node S1 detects that lastindex is equal to 0, it can send a full synchronization request to the central node C. In response to receiving the full synchronization request, the central node C can perform incremental data synchronization with the edge node S2 through the incremental synchronization method introduced in the above embodiment. Then, the central node C sends all the data in the central database after the incremental data synchronization is completed as a full data packet to the edge node S1. When the edge node S1 receives the full data packet sent by the central node C, it can store the full data packet in its own database. It should be understood that after the above data synchronization is completed, the database operation record tables of the central node C and the edge node S1 also become as shown in Table 9 in the above embodiment.
[0213] In some possible implementations, Fig.12 The steps shown can be found in Figure 8The above S803a is executed later. For example, the edge node sends the first feedback information to the central device through S801-S803a, and after determining to participate in this data synchronization, it can be determined whether the full synchronization trigger condition is detected. If so, the steps corresponding to the above S1201-S1202 are executed. If not, it is determined to use the above embodiment to determine the data to be synchronized, and send the data to be synchronized to the central node.
[0214] The above embodiments introduce the data synchronization method provided by the embodiments of the present application through the central node side and the edge node side. For ease of understanding, the data synchronization method is described below based on the perspective of interaction between the central node and the edge node.
[0215] Please refer to Fig.14 , is an interactive schematic diagram of a data synchronization method provided in an embodiment of the present application. Fig.14 As shown, the method includes the following steps.
[0216] S1401. A first node sends a first message to each second node.
[0217] S1402: The second node sends synchronization information to the first node in response to the first message.
[0218] S1403: The first node sends a synchronization data packet to each second node based on the synchronization information from each second node and the data in the first node.
[0219] S1404. The first node receives and stores a synchronization data packet.
[0220] Among them, the first node, the second node, the first message, the synchronization information, the synchronization data packet and the description related to each step can be found in the description of the aforementioned embodiment and will not be repeated here.
[0221] The data synchronization method provided in the embodiment of the present application is further described below from the perspective of the interaction between the modules of the central node and the modules of the edge node.
[0222] Please refer to Fig.15 , is a flowchart of an execution of a data synchronization method provided in an embodiment of the present application. Fig.15As shown, the central node includes modules such as database engine, device state management, synchronization cache module, data synchronization module, synchronization control set, synchronization message monitoring, communication message adaptation, etc., and the edge node includes modules such as database engine, device state management, data synchronization module, synchronization control set, synchronization message monitoring, synchronization collaboration, etc. Among them, the database engine is the database operation engine in the aforementioned embodiment. The device state management is the state monitoring module in the aforementioned embodiment, the data synchronization module is the synchronization calculation unit in the aforementioned embodiment, the synchronization control set is the synchronization control module in the aforementioned embodiment, the synchronization message monitoring is the synchronization message monitoring module in the aforementioned embodiment, and the communication message adaptation is the communication message adaptation module in the aforementioned embodiment.
[0223] In the embodiment of the present application, when the central node detects the incremental synchronization trigger condition, it can send an incremental synchronization notification to the device state management. The execution process of the following data synchronization method starts from when the device state management receives the incremental synchronization notification. Fig.15 As shown, the data synchronization method may include the following steps.
[0224] S1501. The device status management of the central node obtains a list of online devices.
[0225] The online device list is used to indicate the online devices in the distributed database.
[0226] S1502. The device status management of the central node sends the online device list and the incremental synchronization notification to the data synchronization module of the central node.
[0227] S1503. The data synchronization module of the central node encapsulates the incremental synchronization notification according to the online device list to obtain a packaged notification message.
[0228] In an embodiment of the present application, the data synchronization module can encapsulate the incremental synchronization notification according to the interface of each online device in the online device list to obtain multiple encapsulated notification messages. The interface of each encapsulated notification message corresponds to the interface of each online device. By encapsulating the messages transmitted between data nodes, the security of data transmission can be improved.
[0229] S1504. The data synchronization module of the central node sends the encapsulated notification message to the communication message adapter of the central node.
[0230] In an embodiment of the present application, when the data synchronization module of the central node sends the encapsulation notification message to the communication message adapter, it can simultaneously send a first timing notification to the synchronization cache module of the central node. The first timing notification is used to instruct the synchronization cache module of the central node to start a first timer. For the description of the first timer, please refer to the description in the above embodiment, which will not be repeated here.
[0231] S1505. The communication message adapter of the central node sends the encapsulated notification message to the notification message adapter of the edge node.
[0232] In the embodiment of the present application, the edge node may refer to any edge node in the online device list.
[0233] S1506. The notification message adapter of the edge node sends the encapsulated notification message to the synchronous message listener of the edge node.
[0234] S1507. The synchronization message monitor of the edge node decapsulates the encapsulated notification message to obtain an incremental synchronization notification.
[0235] S1508. The synchronization message monitor of the edge node sends the incremental synchronization notification to the device status management of the edge node.
[0236] S1509: When the device status management of the edge node determines that its own status meets the requirements of the configuration information, first feedback information is generated.
[0237] For the description of the configuration information and the status of the edge node, reference may be made to the description in the above embodiment, which will not be repeated here.
[0238] In addition, when the device state management of the edge node determines that its own state does not meet the requirements of the configuration information, second feedback information can be generated and sent to the central node, which will not be elaborated here.
[0239] S1510: The device state management of the edge node sends first feedback information to the synchronization control set of the edge node.
[0240] For the description of the first feedback information, reference may be made to the description in the above embodiment, which will not be repeated here.
[0241] S1511. In response to receiving the first feedback information, the synchronization control set of the edge node sends a database read-write class and the first feedback information to the data synchronization module of the edge node.
[0242] The database read / write class is used to perform a read operation or a write operation on the data in the database. Exemplarily, the database read / write class can be connected to a database read / write interface provided by a database engine, thereby performing a read operation or a write operation on the data in the database.
[0243] S1512. The data synchronization module of the edge node sends a first data acquisition notification to the database engine of the edge node through the database read-write class.
[0244] The first data acquisition notification is used to instruct the database engine of the edge node to return the data to be synchronized. The method for the central node to determine the data to be synchronized can refer to the description in the above embodiment, which will not be repeated here.
[0245] S1513: In response to receiving the data acquisition notification, the database engine of the edge node returns the data to be synchronized to the data synchronization module of the edge node.
[0246] S1514. The data synchronization module of the edge node encapsulates the first feedback information and the data to be synchronized to obtain encapsulated feedback data.
[0247] It should be understood that the packaged feedback data includes the packaged first feedback information and the packaged data to be synchronized.
[0248] S1515. The data synchronization module of the edge node sends the encapsulated feedback data to the communication message adapter of the edge node.
[0249] S1516. The communication message adapter of the edge node sends the encapsulated feedback data to the communication message adapter of the central node.
[0250] In actual applications, the communication message of the central node may be adapted to receive the encapsulated first feedback information and the encapsulated data to be synchronized at different times. Generally speaking, the encapsulated first feedback information is a signaling message with a small volume, so it can be transmitted from the edge node to the central node faster. The encapsulated data to be synchronized is a data packet with a large volume, so it may take more time to be transmitted from the edge node to the central node.
[0251] That is, the communication message adapter of the central node may first receive the encapsulated first feedback information and then receive the encapsulated data to be synchronized.
[0252] S1517. The communication message adapter of the central node sends the encapsulated feedback data to the synchronization message monitor of the central node.
[0253] S1518. The central node monitors the synchronization message to decapsulate the encapsulated feedback data to obtain the first feedback information and the data to be synchronized.
[0254] S1519: The synchronization message monitor of the central node sends the first feedback information and the data to be synchronized to the synchronization control set of the central node.
[0255] S1520. In response to receiving the first feedback information, the synchronization control set of the central node sends a database read-write class to the data synchronization module of the central node.
[0256] As described in the above description of S1516, the communication message adapter of the central node may first receive the encapsulated first feedback information, and then receive the encapsulated data to be synchronized. Similarly, the synchronization control set of the central node may also first receive the first feedback information, and then receive the data to be synchronized. When the synchronization control set of the central node receives the first feedback information, it will also send a second timing notification to the synchronization cache module of the central node. The second timing notification is used to instruct the synchronization cache module of the central node to refresh the first timer. In addition, when the synchronization cache module refreshes the first timer, it will also start the second timer corresponding to the edge node that sent the above-mentioned first feedback information to wait for the completion of receiving the data to be synchronized. For details, please refer to the description in the above-mentioned embodiment, which will not be repeated here.
[0257] S1521. The data synchronization module of the central node sends a second data acquisition notification to the database engine of the central node through the database read and write class.
[0258] The second data acquisition notification is used to instruct the database engine of the central node to return the data of the central database, such as all the data of the central database and / or the data to be synchronized in the central database.
[0259] S1522: In response to receiving the second data acquisition notification, the database engine of the central node returns the data of the central database to the data synchronization module of the central node.
[0260] S1523. The data synchronization module of the central node performs a first process on the data to be synchronized sent by each edge node and the data in the central database to obtain a synchronization data packet.
[0261] The description of the first process can be found in the aforementioned embodiment and will not be repeated here.
[0262] S1524. The data synchronization module of the central node sends the synchronization data packet to the database engine of the central node.
[0263] S1525. The database engine of the central node uses a synchronization data packet to overwrite the data to be synchronized in the central database.
[0264] In some possible implementations, if the data to be synchronized does not exist in the central database, the database engine of the central node may store the synchronization data packet directly in the central database.
[0265] S1526. The data synchronization module of the central node encapsulates the synchronization data packet to obtain an encapsulated synchronization data packet.
[0266] It should be noted that S1524-S1525 can be executed before S1526, can be executed synchronously with S1526, or can be executed after S1526, and this application does not limit this.
[0267] S1527. The data synchronization module of the central node sends the encapsulated synchronization data packet to the communication message adapter of the central node.
[0268] S1528. The communication message adapter of the central node sends the encapsulated synchronization data packet to the synchronization message adapter of the edge node.
[0269] S1529: The synchronization message adapter of the edge node sends the encapsulated synchronization data packet to the synchronization message listener of the edge node.
[0270] S1530: The synchronization message monitor of the edge node decapsulates the encapsulated synchronization data packet to obtain the synchronization data packet.
[0271] S1531. The synchronization message monitor of the edge node sends a synchronization data packet to the synchronization control set of the edge node.
[0272] S1532. In response to receiving the synchronization data packet, the synchronization control set of the edge node sends the synchronization data packet and the database read and write class to the data synchronization module of the edge node.
[0273] S1533. The data synchronization module of the edge node sends a synchronization data packet and a data write notification to the database engine of the edge node through the database read and write class.
[0274] S1534. The database engine of the edge node uses the synchronization data packet to overwrite the data to be synchronized in its own database.
[0275] In some possible implementations, if the to-be-synchronized data does not exist in the database of the edge node itself, the database engine of the edge node may directly store the synchronization data packet in the database of the edge node itself.
[0276] Based on the above description, it can be seen that the data synchronization method, node and system provided in the embodiments of the present application can realize multi-terminal data synchronization in a distributed database, thereby improving the efficiency of data synchronization in the distributed database.
[0277] It should be noted that the above-mentioned embodiments in the present application are not isolated, but are all implementation methods of the data synchronization method provided by the present application. Therefore, the scheme steps, term explanations, etc. in each embodiment can be referenced to each other, and the present application does not impose any restrictions on this.
[0278] The data synchronization system provided in the embodiment of the present application may include a first node and a plurality of second nodes. The data synchronization system may be a distributed data system. The functions of the first node and the second node may refer to the description in the aforementioned embodiment, and will not be described in detail here.
[0279] An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.
[0280] The embodiments of the present application also provide a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the method in the above-mentioned embodiments.
[0281] In addition, the embodiments of the present application also provide a device, which may be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the methods in the above-mentioned method embodiments. It should be noted that all relevant contents of each step involved in the above-mentioned method embodiments can be referred to the functional description of the corresponding functional module, and will not be repeated here.
[0282] Among them, the electronic device, computer storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
[0283] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of electronic equipment. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0284] The embodiment of the present application can divide the functional modules of the devices involved therein according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0285] The functions or actions or operations or steps in the above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may include one or more servers, data centers and other data storage devices that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).
[0286] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A data synchronization method, characterized in that: Applied to a first node in a distributed data system, the distributed data system further comprising a plurality of second nodes, the method comprising: Sending a first message to each second node; the first message is used to instruct the second node to feed back the newly added data; receiving synchronization information from the second node; the synchronization information indicating the newly added data; Based on the synchronization information and the data in the first node, a synchronization data packet is sent to each of the second nodes; the synchronization data packet is obtained according to the newly added data fed back by each of the second nodes and the data in the first node.
2. The method according to claim 1, characterized in that The sending the first message to each second node includes: When a first synchronization triggering condition is detected, sending the first message to each of the second nodes; The first synchronization triggering condition includes at least one of the following: there is new data in the first node, the first node receives a first synchronization request for requesting to initiate data synchronization, and the first node receives a first synchronization instruction for instructing to initiate data synchronization.
3. The method according to claim 1, characterized in that The sending a synchronization data packet to each of the second nodes based on the feedback information and the data in the first node includes: Performing a first process on the newly added data fed back by each of the second nodes and the data in the first node to obtain the synchronization data packet; and sending the synchronization data packet to each of the second nodes; Among them, the first processing is used to add the data in the newly added data fed back by the each second node that has a different index value from the data of the first node, the data in the newly added data of the first node that has a different index value from the newly added data fed back by the each second node, the newly added data fed back by the each second node that has the same index value as the data of the first node and the latest timestamp to the synchronization data packet, and the newly added data fed back by the each second node that has the same index value as the data of the first node and a different timestamp and then renames the index value and adds it to the synchronization data packet.
4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When there is new data in the first node, using the synchronization data packet to cover the new data in the first node; When there is no newly added data in the first node, the synchronization data packet is stored.
5. The method according to claim 2, characterized in that: When there is data in the first node whose operation record value is greater than the synchronization water level value, there is new data in the first node, the operation record value is used to mark the position of the data in the database, and the synchronization water level value is used to indicate the maximum operation record value during the last data synchronization.
6. The method according to claim 1, characterized in that The method further comprises: Starting a first timer; the timing upper limit of the first timer is a first preset duration; Before the first timer expires, if feedback information returned by the second node is received, the first timer is triggered to restart timing; the feedback information is used to indicate whether the corresponding second node participates in this data synchronization.
7. The method according to claim 6, characterized in that The step of receiving feedback information returned by the second node before the first timer expires, triggering the first timer to restart timing includes: Before the first timer expires, if first feedback information returned by the second node is received, the first timer is triggered to restart timing, and the first feedback information is used to instruct the corresponding second node to participate in this data synchronization; The method also includes: starting a second timer corresponding to the second node; the timing upper limit of the second timer is a second preset duration; before the second timer expires, if synchronization information sent by the second node corresponding to the second timer is received, storing the new data indicated by the synchronization information.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtaining an online device list; the online device list includes online nodes in the distributed data system; The online nodes in the online device list except the first node are used as the multiple second nodes.
9. A data synchronization method, characterized in that: Applied to a second node in a distributed data system, the second node being any one of a plurality of second nodes in the distributed data system; the method comprising: In response to receiving a first message sent by the first node, determining synchronization information; the first message is used to instruct the second node to feed back new data; the synchronization information is used to indicate the new data; Sending the synchronization information to the first node; The synchronization data packet from the first node is stored; the synchronization data packet is obtained according to the newly added data fed back by each of the second nodes and the data in the first node.
10. The method according to claim 9, characterized in that After receiving the synchronization data packet from the first node, the method further includes: When there is new data in the second node, using the synchronization data packet to cover the new data in the second node; When there is no newly added data in the second node, the synchronization data packet is stored.
11. The method according to claim 9, characterized in that The determining synchronization information in response to receiving the first message sent by the first node includes: In response to receiving a first message sent by the first node, acquiring a state and configuration information of the second node; the state of the second node is used to indicate a device state of the second node, and the configuration information is used to indicate a condition for the second node to participate in data synchronization; When the state of the second node meets the requirements of the configuration information, feeding back first feedback information to the first node; the first feedback information is used to instruct the second node to participate in this data synchronization; determining synchronization information; When the state of the second node does not meet the requirements of the configuration information, second feedback information is fed back to the first node; the second feedback information is used to indicate that the second node does not participate in this data synchronization.
12. The method according to claim 11, characterized in that The state of the second node includes whether it is in charging mode, whether it is in a screen-on state, whether it is in a low-battery state, and whether it is in a local area network; The configuration information includes any one or any combination of the following: the second node is in charging mode; the second node is in a screen-on state; the second node is in a local area network; the second node is in a non-low-battery state.
13. The method according to claim 9, characterized in that The method further comprises: When a second synchronization trigger condition is detected, a second synchronization request is sent to the first node; the second synchronization trigger condition includes at least one of the following: the second node is online in the distributed data system for the first time, and the synchronization water level value in the second node is greater than the maximum operation record value; the operation record value is used to mark the position of the data in the database, and the synchronization water level value is used to indicate the maximum operation record value at the last data synchronization; When a full data packet sent by the first node is received, the full data packet is stored in the second node; the full data packet is all data in the first node.
14. A data synchronization system, characterized in that: The data synchronization system is applied to a distributed data system; the system comprises: a first node and a plurality of second nodes; The first node is used to send a first message to each second node; the first message is used to instruct the second node to feed back the newly added data; The second node is used to send synchronization information to the first node in response to receiving the first message sent by the first node; the synchronization information is used to indicate the newly added data; The first node is further configured to send a synchronization data packet to each of the second nodes based on the synchronization information from each of the second nodes and the data in the first node; the synchronization data packet is obtained according to the newly added data fed back by each of the second nodes and the data in the first node; The second node is further configured to receive the synchronization data packet and store the synchronization data packet.
15. A node, characterized in that: include: One or more processors, one or more memories, and a communication module for realizing communication connection; the one or more processors are connected to the communication module; the one or more memories are coupled to the one or more processors, and the one or more memories store a computer program; When the one or more processors execute the computer program, the node executes the data synchronization method described in any one of claims 1-8; or, executes the data synchronization method described in any one of claims 9-13.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data synchronization method according to any one of claims 1 to 8 is implemented; or, the data synchronization method according to any one of claims 9 to 13 is implemented.
17. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements the data synchronization method described in any one of claims 1 to 8; or, implements the data synchronization method described in any one of claims 9 to 13.
Citation Information
Patent Citations
Data synchronization method, apparatus, device and system, and readable storage medium
CN108183965A
Distributed cluster data synchronization method and system, master node, slave node and medium
CN108696595A
Method and device for synchronizing information
CN113133107A
Data synchronization method and data synchronization system
CN113259470A
Data synchronization method and related product
CN114157677A