A data synchronization method, device, readable storage medium and electronic equipment

Abstract

The specification discloses a data synchronization method and device, a readable storage medium and an electronic device. A target slave node of a distributed system initiatively queries data identifiers of business data recently stored by other slave nodes of the distributed system from the other slave nodes, and determines, according to the data identifiers and data identifiers of business data stored by the target slave node itself, business data that the target slave node needs to supplement to participate in a data synchronization process initiated by a master node as business data to be synchronized, and then acquires the business data to be synchronized from the other slave nodes based on a synchronization protocol, so as to realize data synchronization in the distributed system. In a scenario where the target slave node fails to synchronize data with the master node in time, the target slave node can acquire business data from the other slave nodes, thereby reducing a time required for data synchronization and improving synchronization efficiency.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer, and particularly relates to a data synchronization method and device, a readable storage medium and an electronic equipment. BACKGROUND

[0002] At present, with the development of Internet technology and people's increasing concern for their own privacy data, distributed systems are more and more widely used in people's life. In order to ensure that the distributed system can stably provide services to the outside, it is necessary to ensure the data synchronization between the business data stored in each node in the distributed system.

[0003] A common data synchronization method is implemented based on master-slave nodes in a distributed system. The distributed system usually includes a master node and multiple slave nodes. The master node actively sends the data to be synchronized to each slave node, and receives the verification information returned by each slave node according to the data to be synchronized. When the master node determines that the verification is passed according to the received verification information, it sends a synchronization instruction to each slave node to make each slave node store the data to be synchronized, thereby realizing the data synchronization between each node in the distributed system.

[0004] However, in the existing data synchronization method, once a slave node fails to synchronize data in time, the slave node cannot participate in the data synchronization process subsequently, and therefore a new data synchronization method is needed. SUMMARY

[0005] The present specification provides a data synchronization method, device, readable storage medium and electronic equipment to partially solve the above problems existing in the prior art.

[0006] The present specification adopts the following technical solutions:

[0007] The present specification provides a data synchronization method applied to a slave node in a distributed system, wherein the distributed system is pre-deployed with a synchronization protocol, and the slave nodes in the distributed system perform data transmission through the synchronization protocol, and the method comprises the following steps:

[0008] The target slave node in the distributed system sends a query request to other slave nodes in the distributed system;

[0009] Receiving the data identifier of the latest stored business data returned by the other slave nodes according to the query request as a specified identifier;

[0010] When the specified identifier and the data identifiers corresponding to each business data stored by the target slave node itself do not match, determining the data identifier of the business data to be synchronized according to the specified identifier and the data identifier of the latest stored business data of the target slave node itself;

[0011] According to the synchronization protocol, the obtained data identifier is used to obtain the to-be-synchronized business data from other slave nodes of the distributed system, and data synchronization is performed.

[0012] The present specification provides a data synchronization device applied to a slave node in a distributed system, the distributed system being pre-deployed with a synchronization protocol, data transmission being performed between slave nodes in the distributed system through the synchronization protocol, and the data synchronization device comprising:

[0013] A query module is configured to send a query request to other slave nodes in the distributed system by a target slave node in the distributed system.

[0014] An obtaining module is configured to receive data identifiers of recently stored business data returned by the other slave nodes according to the query request as specified identifiers.

[0015] A determining module is configured to determine a data identifier of to-be-synchronized business data according to the specified identifiers and data identifiers of business data recently stored by the target slave node when the specified identifiers and the data identifiers of the business data recently stored by the target slave node do not match.

[0016] A synchronization module is configured to obtain the to-be-synchronized business data from the other slave nodes of the distributed system according to the determined data identifier through the synchronization protocol, and perform data synchronization.

[0017] The present specification provides a computer-readable storage medium, the storage medium storing a computer program, the computer program being executed by a processor to implement the above-mentioned data synchronization method.

[0018] The present specification provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the above-mentioned data synchronization method when executing the program.

[0019] The above-mentioned at least one technical solution adopted by the present specification can achieve the following beneficial effects:

[0020] The target slave node of the distributed system actively queries data identifiers of business data recently stored by other slave nodes of the distributed system from the other slave nodes, and then determines business data that the target slave node needs to supplement in order to participate in a data synchronization process initiated by a master node according to the data identifiers and data identifiers of business data stored by the target slave node itself, as to-be-synchronized business data, and obtains the to-be-synchronized business data from the other slave nodes based on the synchronization protocol, thereby realizing data synchronization in the distributed system.

[0021] In the method, even in the case that the target slave node fails to synchronize data with the master node in time, the target slave node can still acquire service data from other slave nodes, thereby reducing the time required for data synchronization and improving the synchronization efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are included to provide a further understanding of the present description and are incorporated in and constitute a part of this description, illustrate embodiments of the present description and serve to explain the principles of the present description. The drawings of the present description are provided for the purpose of illustration only, and therefore are not intended to limit the present description in any way.

[0023] In the drawings:

[0024] Figure 1 A flowchart of a data synchronization method provided by the present description;

[0025] Figure 2 A flowchart of a data synchronization method provided by the present description;

[0026] Figure 3 A data synchronization device provided by the present description;

[0027] Figure 4 An electronic device corresponding to the data synchronization device provided by the present description. Figure 1 A schematic diagram of the electronic device. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present description clearer, the technical solutions of the present description will be described clearly and completely below in conjunction with the specific embodiments of the present description and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present description, but not all the embodiments. Based on the embodiments in the present description, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present description.

[0029] At present, in the field of data synchronization, a common method for synchronizing data between nodes in a distributed system is to actively initiate synchronization.

[0030] Specifically, the master node first packages the business data to be synchronized and determines the corresponding data identifier for each data item. Next, the master node sends the packaged business data to each slave node in the distributed system. Then, each slave node verifies the received business data and sends the verification information back to the master node. Finally, the master node determines whether the business data has passed verification based on the verification information. If verification is successful, the master node sends a synchronization command to each slave node. Upon receiving the synchronization command, the slave nodes can store the business data to be synchronized, thus achieving data synchronization between nodes in the distributed system.

[0031] As can be seen, currently, when implementing data synchronization between nodes in a distributed system, the master node needs to package the business data to be synchronized, send the packaged business data to each slave node, and then the master node needs to make decisions based on the verification information sent by each slave node, determining whether to store the business data to be synchronized. Clearly, this data synchronization process places high demands on the master node. These demands could be related to network resources or computing resources.

[0032] For slave nodes in a distributed system, the only way for a slave node to obtain business data to be synchronized is when the master node initiates synchronization. In other words, a slave node can only obtain business data from the master node.

[0033] However, in the actual operation of a distributed system, the network conditions are often complex. Network latency and isolation may occur between slave nodes and master nodes. Furthermore, slave nodes may crash but recover and rejoin the distributed system. In such cases, the slave node cannot obtain the necessary business data from the master node, nor can it store it. In other words, the slave node cannot participate in the data synchronization process. Consequently, when the master node initiates a new data synchronization, the slave node will also be unable to participate due to the missing business data. The slave nodes that cannot participate in the data synchronization process due to network latency, network isolation, or node crashes are called lagging nodes.

[0034] In order to avoid the above situation, at present, when there are many slave nodes that cannot participate in the data synchronization process, or the master node is down, etc., a new master node can be re-elected in the distributed system, and the new master node can query the data identifiers of the business data stored in each slave node in the distributed system, and determine each slave node that needs to supplement data according to the data identifiers of the business data stored in the master node itself and the data identifiers of each slave node respectively corresponding to the data identifiers queried. For each slave node that needs to supplement data, the data that the slave node needs to supplement is sent to the slave node, until the data identifier of the business data stored in each slave node is consistent with the data identifier of the business data stored in the master node itself. Only then will the master node initiate a new data synchronization process.

[0035] In addition, at present, a data synchronization method based on a distributed protocol is also provided. Specifically, the distributed protocol deployed in the distributed system in advance is changed. Based on the changed distributed protocol, the slave node in the distributed system can obtain the data identifier of the business data recently stored by other slave nodes from the other slave nodes, and determine the business data to be synchronized by the slave node itself according to the obtained data identifier and the business data stored in the slave node itself, and obtain the business data to be synchronized from the master node, to solve the problem that the business data of the lagging node can only be supplemented by the newly determined master node when the master node is newly determined in the distributed system, resulting in low data synchronization efficiency at present.

[0036] However, the above synchronization method based on the distributed protocol needs to modify the original distributed protocol. The distributed protocol itself is relatively rigorous, and if the protocol is modified recklessly, the accuracy and security of the original distributed protocol cannot be guaranteed, and the node may not be able to exit the consensus, which has a large security risk. The distributed protocol is a distributed consistency protocol.

[0037] Based on this, the present specification provides a new data synchronization method, which does not change the original distributed protocol, but adds a new synchronization protocol to the distributed system. The slave nodes in the distributed system can transmit data through the synchronization protocol. When the slave node determines that it is a lagging node, it can obtain the business data to be synchronized from other slave nodes included in the distributed system based on the synchronization protocol, so that it can be converted from a lagging node to a slave node that can participate in the data synchronization process initiated by the master node, to realize data synchronization between nodes in the distributed system.

[0038] The technical solutions provided by the embodiments of the present specification will be described in detail below with reference to the accompanying drawings.

[0039] Figure 1A flowchart of a data synchronization method provided by the present specification is shown in the figure, and specifically includes the following steps:

[0040] S100: The target slave node in the distributed system sends a query request to other slave nodes in the distributed system.

[0041] S102: Receive the data identifier of the recently stored service data returned by the other slave nodes according to the query request as the specified identifier.

[0042] Unlike the current slave node that can only obtain service data from the master node, and when the slave node is a lagging node, the missing service data of the lagging node can only be supplemented after the master node is re-determined in the distributed system, resulting in low efficiency of current data synchronization.

[0043] The present specification provides a new data synchronization method, which pre-deploys a synchronization protocol in the distributed system for data transmission between slave nodes in the distributed system. Therefore, any slave node in the distributed system can obtain the data identifier of the recently stored service data of other slave nodes in the distributed system based on the synchronization protocol, and determine the service data that the slave node itself needs to supplement as the to-be-synchronized service data based on the obtained data identifier and the service data stored by itself. Then, the slave node obtains the to-be-synchronized service data from other slave nodes in the distributed system based on the synchronization protocol, so that the slave node itself can supplement the missing data based on the synchronization protocol to ensure that the slave node can be converted from a lagging node to a slave node that can participate in the data synchronization process initiated by the master node, thereby realizing data synchronization between nodes in the distributed system.

[0044] Based on the brief description of the data synchronization method provided by the present specification, the data synchronization method provided by the present specification can be applied to a "strong master" type distributed system. The "strong master" type distributed system is the above-mentioned distributed system in which the master node initiates the data synchronization process. Therefore, the data synchronization method can be executed by any slave node in the "strong master" type distributed system.

[0045] Based on this, the target slave node in the distributed system can send a query request to other slave nodes in the distributed system. The target slave node can be any slave node in the distributed system.

[0046] Specifically, the target slave node can randomly select any other slave node from each other slave node included in the distributed system according to the synchronization protocol pre-deployed in the distributed system, and send a query request to the other slave node.

[0047] Thus, the other slave node can determine the data identifier of the service data stored most recently by the other slave node after receiving the query request, and return the data identifier to the target slave node according to the query request.

[0048] The target slave node can receive the data identifier returned by the other slave node, and take the data identifier of the service data stored most recently by the other slave node as the specified identifier.

[0049] The data identifier of the service data is determined by the master node in the distributed system according to the order in which the master node stores each service data when storing the service data. Alternatively, the data identifier of the service data is determined by the master node in the distributed system according to the order corresponding to the service data when the master node initiates data synchronization based on each service data.

[0050] To ensure that the data identifier corresponding to each service data is globally unique and the data identifiers of each service data stored in the master node are continuous, the data identifier of the service data can be determined as needed, and the present specification does not limit how the data identifier of the service data is determined.

[0051] S104: When the specified identifier and the data identifier corresponding to each service data stored by the target slave node do not match, determining the data identifier of the service data to be synchronized according to the specified identifier and the data identifier of the service data stored most recently by the target slave node.

[0052] In one or more embodiments provided in the present specification, as described above, the core idea of the data synchronization method provided in the present specification is that the target slave node of the distributed system actively queries the data identifier of the service data stored most recently by the other slave node of the distributed system, and then determines the service data that the target slave node needs to supplement in order to participate in the data synchronization process initiated by the master node as the service data to be synchronized according to the data identifier and the data identifier of the service data stored by the target slave node itself, and then acquires the service data to be synchronized from the other slave node based on a synchronization protocol, so as to realize data synchronization in the distributed system.

[0053] The master node does not need to supplement the data in the target slave node, which reduces the running pressure of the master node, reduces the time required for data synchronization, and improves the synchronization efficiency.

[0054] Moreover, after the master node of the distributed system is re-determined, the newly determined master node can immediately initiate new data synchronization, without the need for the newly determined master node to supplement data to each lagging node in the distributed system and to initiate data synchronization after supplementing the data, thereby further improving the efficiency of data synchronization.

[0055] Based on this, after the specified identifier is determined, the target slave node can determine the service data to be synchronized by the target slave node according to the specified identifier and the data identifiers corresponding to the service data stored in the target slave node.

[0056] Specifically, the target slave node can match the determined specified identifier with the data identifiers of the service data stored in the target slave node, where the service data stored in the target slave node can be empty, one or multiple.

[0057] If there is no data identifier consistent with the specified identifier in the data identifiers of the service data stored in the target slave node, the target slave node can consider itself as a lag node.

[0058] Therefore, the target slave node can determine the data to be supplemented by the target slave node as the service data to be synchronized according to the specified identifier and the data identifiers of the service data stored in the target slave node.

[0059] If there is a data identifier consistent with the specified identifier in the data identifiers of the service data stored in the target slave node, the target slave node can consider itself as a non-lag node.

[0060] Taking the specified identifier as 99 and the data identifiers of the service data stored in the target slave node as 1-88 as an example, the target slave node can determine the service data to be synchronized according to the specified identifier and the data identifiers of the service data stored in the target slave node, that is, determine the service data with data identifiers 89-99 as the service data to be synchronized.

[0061] S106: According to the determined data identifier, the target slave node acquires the service data to be synchronized from other slave nodes of the distributed system through the synchronization protocol, and performs data synchronization.

[0062] In one or more embodiments provided in the specification, in the scenario of the “strong master” distributed system applied in the specification, the pressure on the master node is large, therefore, if the target slave node needs to acquire the service data to be synchronized from the master node first, the pressure on the master node will be further increased, and since the master node needs to process more data, the rate at which the master node sends the service data to be synchronized is slow. If the service data to be synchronized is acquired from other slave nodes in the distributed system, both the pressure on the distributed database and the execution efficiency of the distributed database will be greatly improved.

[0063] Based on this, after the service data to be synchronized is determined, the target slave node can acquire the service data to be synchronized from other slave nodes of the distributed system through the synchronization protocol based on the service data to be synchronized.

[0064] Specifically, the target slave node can send a transmission request to other slave nodes of the distributed system according to the data identifier of the to-be-synchronized service data according to the pre-deployed synchronization protocol.

[0065] The other slave nodes can determine the to-be-synchronized service data according to the data identifier in the transmission request and send the to-be-synchronized service data to the target slave node.

[0066] The target slave node can receive the service data sent by the other slave nodes, and the data identifier of the latest stored service data in the target slave node is the specified identifier after receiving the service data.

[0067] Based on Figure 1 As shown in the data synchronization method, the target slave node of the distributed system actively queries the data identifier of the service data recently stored by other slave nodes of the distributed system, determines the service data that the target slave node needs to supplement to participate in the data synchronization process initiated by the master node as to-be-synchronized service data according to the data identifier and the data identifier of the service data stored by the target slave node itself, and obtains the to-be-synchronized service data from the other slave nodes based on the synchronization protocol, so as to realize data synchronization in the distributed system. Even in the case that the target slave node fails to synchronize data with the master node in time, the target slave node can still obtain service data from other slave nodes, which reduces the time required for data synchronization and improves synchronization efficiency. Moreover, the method does not require the master node to supplement data in the target slave node, which reduces the time required for data synchronization and reduces the operating pressure of the master node.

[0068] Further, in one or more embodiments provided in the specification, in order to ensure synchronization efficiency and avoid the case that the time required for obtaining to-be-synchronized service data from a single other slave node is too long, the master node of the distributed system initiates data synchronization in the process of receiving all to-be-synchronized service data in the target slave node, and the target slave node cannot obtain the service data recently stored by the master node of the distributed system and cannot participate in the data synchronization process proposed by the master node, the target slave node can also obtain to-be-synchronized service data from multiple other slave nodes to realize data synchronization.

[0069] Specifically, the target slave node can first determine the number of data identifiers of the to-be-synchronized service data.

[0070] If the number of data identifiers of the to-be-synchronized service data is one, the target slave node can select one other slave node as a specified slave node from the other slave nodes of the distributed system, and obtain to-be-synchronized service data from the specified slave node by the target slave node.

[0071] The target slave node can determine the designated slave node according to the attribute information of each other slave node, or randomly select any slave node from each other slave node as the designated slave node. How to determine the designated slave node can be set according to needs, and the present specification does not limit this.

[0072] If the number of data identifiers of the business data to be synchronized is multiple, the target slave node can select a specified number of other slave nodes from each other slave node of the distributed system as designated slave nodes according to the specified number.

[0073] After determining the designated slave nodes, the target slave node can also determine the data identifiers of the business data to be obtained from each designated slave node according to the data identifiers of the business data to be synchronized, the number of data identifiers of the business data to be synchronized, and the specified number.

[0074] Therefore, the target slave node can send a transmission request to each designated slave node according to the data identifiers of the business data to be obtained from the designated slave node.

[0075] For each designated slave node, the designated slave node can return the business data to be obtained from itself to the target slave node according to the received transmission request.

[0076] The target slave node can receive the business data to be synchronized returned by each designated slave node to complete the data synchronization step.

[0077] Furthermore, the load conditions, network conditions, etc. of each slave node are not the same, and if the same amount of data is obtained from each slave node, the other slave nodes with higher load may also be down due to the need to transmit business data to achieve data synchronization.

[0078] Therefore, when determining the data identifiers of the business data to be obtained from each designated slave node, the target node can determine based on the attribute information corresponding to each designated slave node.

[0079] Specifically, for each designated slave node, the attribute information corresponding to the designated slave node can be determined. The attribute information at least includes one of the load condition, the network condition, and the distance between the designated slave node and the target slave node.

[0080] According to the attribute information of each specified slave node, the target slave node can determine a score corresponding to each specified slave node, wherein the score is negatively correlated with the load condition in the attribute information, positively correlated with the network condition in the attribute information, and negatively correlated with the distance between the specified slave node and the target slave node in the attribute information. That is, for each specified slave node, the higher the score of the specified slave node, the better the network condition, the lower the load condition, and the closer the distance between the specified slave node and the target slave node.

[0081] Accordingly, after determining the scores, the target slave node can determine, for each specified slave node, a data identifier of the service data to be obtained from the specified slave node according to the scores, the data identifiers of the service data to be synchronized, the number of the data identifiers of the service data to be synchronized, and the specified number, wherein the score of the specified slave node is positively correlated with the data identifier of the service data to be obtained from the specified slave node.

[0082] In addition, the attribute information includes multiple types, and the target slave node can also preset a weight of each type of attribute information, and then determine the score of each specified slave node based on the preset weight and the determined value of the attribute information. The weight can be 1 or 0, or 0.3 or 0.5, and how to determine the weight can be set as needed, which is not limited in the specification.

[0083] Further, in order to further ensure the efficiency of data synchronization, the target slave node can determine the specified slave node based on the scores corresponding to each other slave node.

[0084] Specifically, for each other slave node in the distributed system, the target slave node can determine attribute information corresponding to the other slave node, wherein the attribute information at least includes one of a load condition, a network condition, and a distance between the other slave node and the target slave node.

[0085] According to the attribute information of each other slave node, the target slave node can determine a score corresponding to each other slave node, wherein the score is negatively correlated with the load condition in the attribute information, positively correlated with the network condition in the attribute information, and negatively correlated with the distance between the other slave node and the target slave node in the attribute information. That is, for each other slave node, the higher the score of the other slave node, the better the network condition, the lower the load condition, and the closer the distance between the other slave node and the target slave node.

[0086] Thus, after the score is determined, the target slave node can determine the specified number of specified slave nodes from each of the other slave nodes included in the distributed system according to the specified number and each score.

[0087] Finally, the target slave node can obtain the service data to be synchronized from the determined specified slave nodes according to the data identifier of the service data to be synchronized.

[0088] In one or more embodiments provided in the present specification, the inconsistency between the service data stored in the target slave node and the specified identifier includes not only the case that the data identifier corresponding to the service data is lower than the specified identifier, but also the case that the data identifier corresponding to the service data is higher than the specified identifier. Only when the data identifier corresponding to the service data is lower than the specified identifier, the target slave node is a lag node and needs to obtain the service data to be synchronized from other slave nodes.

[0089] Therefore, in the step S104, when there is no data identifier matching the specified identifier in the data identifiers corresponding to each service data stored in the target slave node, the target slave node can further determine whether the specified identifier is consistent with the data identifier of the service data most recently stored in the target slave node.

[0090] If the specified identifier is consistent with the data identifier of the service data most recently stored in the target slave node, the target slave node can determine that the other slave node corresponding to the specified identifier is a non-lag node.

[0091] If the specified identifier is inconsistent with the data identifier of the service data most recently stored in the target slave node, the target slave node can determine that the other slave node corresponding to the specified identifier is a lag node, and send prompt information to the other slave node corresponding to the specified identifier. The prompt information is used to prompt the other slave node corresponding to the specified identifier that it is a lag node.

[0092] Further, in order to avoid the case that the target slave node and the other slave node receiving the query request are both lag nodes, but the specified identifier is consistent with the service data stored in the target slave node, so that the target slave node and the other slave node cannot determine that they are lag nodes, when the specified identifier is determined, the target slave node can further send a query request to a plurality of other slave nodes.

[0093] Specifically, the target slave node can send a query request to at least part of the other slave nodes in the distributed system respectively.

[0094] For each other slave node receiving the query request, the data identifier of the service data most recently stored in the other slave node can be determined according to the query request, and the data identifier is returned to the target slave node.

[0095] The target slave node can receive at least part of the other slave nodes that sent the query request, and return the data identifiers of the latest stored service data according to the query request, as the data identifiers corresponding to the at least part of the other slave nodes respectively.

[0096] Therefore, after determining the plurality of data identifiers, the target slave node can select the data identifier with the largest quantity from the data identifiers corresponding to the at least part of the other slave nodes as the specified identifier.

[0097] The target slave node can send the query request to at least part of the other slave nodes in the distributed system, or send the query request to all the other slave nodes in the distributed system. The number and position of the other slave nodes to which the target slave node sends the query request can be set as needed, and the present specification does not limit the same.

[0098] Further, based on the above-mentioned case that the target slave node can send prompt information to the other nodes, the slave nodes in the distributed system can store the service data after receiving the service data, and broadcast the stored service data, so that the slave nodes that receive the data identifier of the service data can determine whether they are the lagging nodes according to the stored service data, thereby improving the efficiency of data synchronization.

[0099] Similarly, the target slave node can receive the reference data identifier sent by the other slave node in the distributed system. The reference data identifier is determined and sent by the other slave node according to the service data stored by the other slave node after the data synchronization between the other slave node and the master node in the distributed system.

[0100] Therefore, when the target slave node determines that the reference data identifier and the data identifier corresponding to the service data stored by the target slave node do not match, the target slave node can determine that it is a lagging node, and determine the data identifier of the service data to be synchronized according to the reference data identifier and the data identifier of the service data stored by the target slave node.

[0101] Based on the same idea, the present specification also provides a data synchronization method, as shown in Figure 2 .

[0102] Figure 2 A flowchart of a data synchronization method provided by the present specification is shown in the figure. On the left side is the target slave node, and on the right side is any other slave node in the distributed system except the target slave node. Wherein:

[0103] S200: The target slave node sends a query request to the other slave nodes in the distributed system.

[0104] S202: The target slave node receives the specified identifier returned by the other slave nodes according to the query request.

[0105] S204: When the specified identifier and the data identifiers corresponding to each business data stored by the target slave node do not match, the target slave node determines the data identifier of the business data to be synchronized based on the specified identifier and the data identifier of the business data most recently stored by the slave node.

[0106] S206: The target slave node sends a transmission request to the other slave nodes based on the data identifier of the service data to be synchronized.

[0107] S208: The target slave node receives the service data to be synchronized returned by the other slave nodes according to the transmission request, and performs data synchronization.

[0108] In the diagram, the target slave node can be any slave node in the distributed system. Other slave nodes can be any slave node in the distributed system other than the target node. Therefore, the target slave node can send a query request to other slave nodes to determine a specified identifier. Then, if the lag node's stored business data is inconsistent with the specified identifier, it can determine the business data it needs to synchronize. After determining the target data, the lag node can send a transmission request to other slave nodes based on the business data to be synchronized and receive the target data to be synchronized returned by the other slave nodes.

[0109] The nodes to which the query request and transmission request are sent can be different nodes.

[0110] Following the same approach, this specification also provides a data synchronization device, such as... Figure 3 As shown.

[0111] Figure 3 The data synchronization device provided in this specification is applied to slave nodes in a distributed system. The distributed system has a pre-deployed synchronization protocol, and the slave nodes in the distributed system transmit data through the synchronization protocol, wherein:

[0112] The query module 300 is used by the target slave node in the distributed system to send query requests to other slave nodes in the distributed system.

[0113] The acquisition module 302 is used to receive the data identifier of the most recently stored business data returned by the other slave nodes according to the query request, as the designated identifier.

[0114] The determining module 304 is configured to determine the data identifier of the business data to be synchronized according to the specified identifier and the data identifier of the business data stored most recently by the target slave node, when the specified identifier and the data identifier of the business data stored by the target slave node itself do not match.

[0115] The synchronizing module 306 is configured to acquire the business data to be synchronized from other slave nodes of the distributed system according to the determined data identifier through the synchronization protocol, and perform data synchronization.

[0116] Optionally, the synchronizing module 306 is configured to determine specified slave nodes from the other slave nodes included in the distributed system according to the specified number, determine the data identifier of the business data to be acquired from each specified slave node according to the data identifier of the business data to be synchronized, the number of the data identifiers of the business data to be synchronized and the specified number, send a transmission request to each specified slave node according to the data identifier of the business data to be acquired from the specified slave node, and receive the business data to be synchronized returned by the specified slave nodes.

[0117] Optionally, the data identifiers of different business data correspond to different data identifiers, the synchronizing module 306 is configured to determine the scores corresponding to the specified slave nodes respectively according to attribute information of the specified slave nodes, the scores are negatively correlated with the load in the attribute information, the scores are positively correlated with the network in the attribute information, the scores are negatively correlated with the distance between the specified slave nodes and the target slave node in the attribute information, determine the data identifier of the business data to be acquired from each specified slave node according to the scores, the data identifier of the business data to be synchronized, the number of the data identifiers of the business data to be synchronized and the specified number, and the number of the data identifiers of the business data to be acquired from the specified slave nodes is negatively correlated with the score of the specified slave node.

[0118] Optionally, the synchronizing module 306 is configured to determine the scores corresponding to the other slave nodes respectively according to attribute information of the other slave nodes, the scores are negatively correlated with the load in the attribute information, the scores are positively correlated with the network in the attribute information, and the scores are negatively correlated with the distance between the other slave nodes and the target slave node in the attribute information, determine the specified slave nodes from the other slave nodes included in the distributed system according to the specified number and the scores, and acquire the business data to be synchronized from the specified slave nodes according to the data identifier of the business data to be synchronized.

[0119] Optionally, the determining module 304 is configured to determine that the target slave node is a non-falling-behind node when there is a data identifier matching the specified identifier in the data identifiers corresponding to the service data stored by the target slave node itself, and determine whether the specified identifier is consistent with the data identifier of the service data most recently stored by the target slave node. If not, send a prompt information to other slave nodes corresponding to the specified identifier, and the prompt information is used to prompt the other slave nodes corresponding to the specified identifier to be falling-behind nodes.

[0120] Optionally, the querying module 300 is configured to send a query request to at least part of other slave nodes in the distributed system respectively, receive the data identifiers of the service data most recently stored by the at least part of other slave nodes respectively according to the query request as the data identifiers corresponding to the at least part of other slave nodes respectively, and select the data identifier with the largest quantity from the data identifiers corresponding to the at least part of other slave nodes respectively as the specified identifier.

[0121] Optionally, the obtaining module 302 is configured to receive a reference data identifier sent by other slave nodes in the distributed system, wherein the reference data identifier is determined by the other slave nodes according to the service data most recently stored by the other slave nodes after the other slave nodes and the master node in the distributed system perform data synchronization, and the data identifier of the service data to be synchronized is determined according to the reference data identifier and the data identifier of the service data most recently stored by the target slave node when the reference data identifier and the data identifier corresponding to each service data stored by the target slave node itself do not match.

[0122] The specification also provides a computer readable storage medium storing a computer program, and the computer program can be used to execute the above Figure 1 data synchronization method.

[0123] The specification also provides Figure 4 The schematic structural diagram of the electronic device is shown in FIG. 4. As shown in FIG. 4, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and of course, other hardware required by the business can also be included. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 data synchronization method. Of course, in addition to the software implementation, the specification does not exclude other implementation manners, such as logic devices or software and hardware combined manner, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic device.

[0124] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0125] The controller can be implemented in any suitable way, e.g. the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91 SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to being implemented in pure computer readable program code form, the controller can perfectly well be implemented by means of logic programmed into logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to perform the same functions. The controller can thus be considered as a hardware component, and the means comprised therein for performing various functions can be considered as structures within the hardware component. Alternatively, or even, the means for performing various functions can be considered as both a software module implementing a method and a structure within a hardware component.

[0126] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0127] For the sake of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of each unit can be implemented in one or more software and / or hardware in implementing the present specification.

[0128] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0130] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 ​

[0132] In one typical configuration, the computing device includes one or more processors (CPU's), input / output interfaces, network interfaces, and memory.

[0133] The memory can include non-persistent memory and / or persistent memory, both of which can be volatile and / or non-volatile. Non-persistent memory can include, for example, a random access memory (RAM), which can be a volatile memory device that can require power to maintain its state. Non-persistent memory can also include, for example, a cache memory, a register found in the CPU, and / or embedded memory in the CPU. Persistent memory can include, for example, read-only memory (ROM), a flash memory, and / or a hard disk. The memory is an example of computer-readable media.

[0134] ​​​​​​Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0135] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0138] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different but related aspects of the description. Each of the various embodiments can stand on its own, and each can be combined with the subject matter of other embodiments to produce further embodiments. Where the same numbers appear in different embodiments, such numbers are used for the sake of ease of understanding only and do not imply that the embodiments in which such numbers appear are the same or similar.

[0139] The above description is embodied in the form of only a description of embodiments of the present specification, and is not intended to limit the present specification. Various changes and modifications can be made by those skilled in the art based on the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.

Claims

1. A data synchronization method applied to slave nodes in a distributed system, wherein the distributed system is pre-deployed with a distributed consistency protocol and a synchronization protocol; during the data synchronization process based on the distributed consistency protocol, the master node in the distributed system synchronizes the business data that needs to be synchronized, and each slave node in the distributed system obtains the business data that needs to be synchronized from the master node; Furthermore, the slave nodes in the distributed system transmit data through the synchronization protocol, the method comprising: The target slave node in the distributed system sends a query request to other slave nodes in the distributed system; Receive the data identifier of the most recently stored business data returned by the other slave nodes according to the query request, and use it as the designated identifier; When the specified identifier and the data identifiers corresponding to each service data stored by the target slave node do not match, the data identifier of the service data to be synchronized is determined based on the specified identifier and the data identifier of the most recently stored service data of the target slave node. Using the synchronization protocol, the business data to be synchronized is obtained from other slave nodes of the distributed system based on the determined data identifier, and data synchronization is performed.

2. The method as described in claim 1, wherein the business data to be synchronized is obtained from other slave nodes of the distributed system according to the determined data identifier through the synchronization protocol, specifically including: When the data identifier of the business data to be synchronized is multiple, the specified slave node is determined from the other slave nodes included in the distributed system according to the specified number; Based on the data identifier of the business data to be synchronized, the number of data identifiers of the business data to be synchronized, and the specified number, determine the data identifier of the business data to be obtained from each specified slave node; For each specified slave node, a transmission request is sent to that specified slave node based on the data identifier of the service data to be obtained from that specified slave node; Receive the business data to be synchronized returned by each designated slave node.

3. The method as described in claim 2, wherein the business data of different services correspond to different data identifiers; Based on the data identifiers of the business data to be synchronized, the number of data identifiers of the business data to be synchronized, and the specified number, the data identifiers of the business data to be obtained from each specified slave node are determined, specifically including: Based on the attribute information of each specified slave node, a score is determined for each specified slave node. The score is negatively correlated with the load information in the attribute information, positively correlated with the network information in the attribute information, and negatively correlated with the distance between the specified slave node and the target slave node in the attribute information. Based on each score, the data identifier of the business data to be synchronized, the number of data identifiers of the business data to be synchronized, and the specified number, the data identifiers of the business data to be obtained from each specified slave node are determined. The score of the specified slave node and the number of data identifiers of the business data to be obtained from the specified slave node are negatively correlated.

4. The method as described in claim 1, wherein the business data to be synchronized is obtained from other slave nodes of the distributed system based on the determined data identifier, specifically including: Based on the attribute information of each other slave node in the distributed system, a score is determined for each other slave node. The score is negatively correlated with the load information in the attribute information, positively correlated with the network information in the attribute information, and negatively correlated with the distance between the other slave node and the target slave node in the attribute information. Based on the specified number and scores, a specified slave node is determined from among the other slave nodes included in the distributed system; Based on the data identifier of the business data to be synchronized, the business data to be synchronized is obtained from the designated slave node.

5. The method of claim 1, further comprising: When there is a data identifier that matches the specified identifier among the data identifiers corresponding to each service data stored by the target slave node itself, the target slave node is determined to be a non-lagging node, and it is determined whether the specified identifier is consistent with the data identifier of the service data most recently stored by the target slave node; If not, send a prompt message to the other slave nodes corresponding to the specified identifier. The prompt message is used to indicate that the other slave nodes corresponding to the specified identifier are lagging nodes.

6. The method as described in claim 1, wherein a query request is sent to other slave nodes in the distributed system, and the data identifier of the most recently stored business data returned by the other slave nodes according to the query request is received as a designated identifier, specifically including: Send query requests to at least some of the other slave nodes in the distributed system; The data identifiers of the most recently stored business data returned by at least some of the other slave nodes according to the query request are used as the data identifiers corresponding to the at least some of the other slave nodes respectively. Select the data identifier with the largest quantity from the data identifiers corresponding to at least some of the other slave nodes, and use it as the designated identifier.

7. The method of claim 1, further comprising: The system receives reference data identifiers sent by other slave nodes in the distributed system, wherein the other slave nodes determine the reference data identifiers based on their most recently stored business data after synchronizing data with the master node in the distributed system. When the reference data identifier and the data identifiers corresponding to each service data stored by the target slave node do not match, the data identifier of the service data to be synchronized is determined based on the reference data identifier and the data identifier of the most recently stored service data of the target slave node.

8. A data synchronization device, applied to a slave node in a distributed system, wherein the distributed system is pre-deployed with a distributed consistency protocol and a synchronization protocol; during the data synchronization process based on the distributed consistency protocol, the master node in the distributed system synchronizes the business data that needs to be synchronized, and each slave node in the distributed system obtains the business data that needs to be synchronized from the master node; Furthermore, the slave nodes in the distributed system transmit data through the synchronization protocol, and the apparatus includes: The query module is used by the target slave node in the distributed system to send query requests to other slave nodes in the distributed system. The acquisition module is used to receive the data identifier of the most recently stored business data returned by the other slave nodes according to the query request, as the designated identifier; The determination module is used to determine the data identifier of the business data to be synchronized based on the specified identifier and the data identifier of the most recently stored business data of the target slave node when the specified identifier and the data identifier of each business data stored by the target slave node do not match. The synchronization module is used to obtain the business data to be synchronized from other slave nodes of the distributed system according to the determined data identifier through the synchronization protocol, and to perform data synchronization.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images