Data synchronization method, apparatus, device, and storage medium

Abstract

The application provides a data synchronization method and device, equipment and a storage medium. It relates to the technical field of big data resource scheduling. The method comprises the following steps: receiving data bitmap encoding sent by a second server; determining a target data set to be synchronized in a first server according to encoding information at each position in the data bitmap encoding; determining a target synchronization mode for synchronizing data to the second server according to the data volume of the target data set, wherein the target synchronization mode comprises at least one of a message mode and a file package mode; and sending the target data set to the second server according to the target synchronization mode. According to the data volume of the data to be synchronized, the application determines to synchronize the data in the form of messages and / or file packages, so that the real-time performance and efficiency of data synchronization can be taken into account, thereby meeting the timeliness and efficiency requirements of data synchronization.

Description

Technical Field

[0001] This application relates to the field of big data resource scheduling technology, and in particular to a data synchronization method, apparatus, device and storage medium. Background Technology

[0002] Various enterprises and institutions, such as governments, banks, hospitals, schools, and businesses, typically use management and information technology (IT) systems that consist of many subsystems. The data stored in these subsystems are dependent and related, and various data synchronization processes are required between them when processing business operations.

[0003] In existing technologies, data synchronization via messages is inefficient when the amount of data to be synchronized is large; data synchronization via file packets cannot guarantee timeliness. Therefore, existing technologies cannot simultaneously achieve both real-time performance and high efficiency in data synchronization, making it difficult to meet the requirements of both simultaneously. Summary of the Invention

[0004] This application provides a data synchronization method, apparatus, device, and storage medium to address the problem that existing technologies cannot simultaneously achieve both real-time and high-efficiency data synchronization, making it difficult to meet the requirements of both at the same time.

[0005] In a first aspect, this application provides a data synchronization method executed by a first server, the method comprising: receiving a data bitmap encoding sent by a second server; determining a target dataset to be synchronized in the first server based on the encoding information at each position in the data bitmap encoding; determining a target synchronization mode for data synchronization with the second server based on the data volume of the target dataset, wherein the target synchronization mode includes at least one of a message mode and a file packet mode; and sending the target dataset to the second server according to the target synchronization mode.

[0006] In one embodiment of this application, determining the target dataset to be synchronized in the first server includes: receiving data synchronization information sent by the second server, wherein the data synchronization information includes at least the data bitmap encoding; and determining the target dataset based on the data synchronization information.

[0007] In one embodiment of this application, the data synchronization information includes a user identifier, the data bitmap encoding, and a first timestamp. Determining the target dataset based on the data synchronization information includes: determining a first dataset from candidate datasets of the first server based on the user identifier; determining a second dataset from the first dataset based on the data bitmap encoding; and determining a dataset that meets the data synchronization conditions from the second dataset based on the first timestamp and a second timestamp of the second dataset, as the target dataset.

[0008] In one embodiment of this application, determining the dataset that satisfies the data synchronization condition from the second dataset based on the first timestamp and the second timestamp of the second dataset includes: comparing the first timestamp with the second timestamp to obtain a candidate second timestamp that is greater than the first timestamp; and using the dataset corresponding to the candidate second timestamp as the dataset that satisfies the data synchronization condition.

[0009] In one embodiment of this application, determining the target dataset to be synchronized in the first server further includes: obtaining a third timestamp of the dataset in the first server; determining whether the dataset has been updated based on the third timestamp; determining whether the dataset meets the data synchronization conditions in response to determining that the dataset has been updated; and determining that the dataset is the target dataset in response to meeting the data synchronization conditions.

[0010] In one embodiment of this application, determining the target synchronization mode for data synchronization with the second server based on the data volume of the target dataset includes: determining the message mode as the target synchronization mode in response to the data volume of the target dataset being less than or equal to a set data volume threshold; determining the message mode and the file packet mode as the target synchronization mode in response to the data volume of the target dataset being greater than the set data volume threshold, or determining the file packet mode as the target synchronization mode.

[0011] In one embodiment of this application, the method further includes: obtaining a target timestamp of the target dataset and sending the target timestamp to the second server, wherein the target timestamp is used to update the first timestamp.

[0012] Secondly, this application provides another data synchronization method, executed by a second server, the method comprising: generating a data bitmap encoding and sending the data bitmap encoding to a first server, the data bitmap encoding being used to determine a target dataset to be synchronized in the first server; receiving the target dataset sent by the first server according to a target synchronization mode, wherein the target synchronization mode includes at least one of a message mode and a file packet mode.

[0013] In one embodiment of this application, the method further includes: obtaining a user identifier, and determining a dataset for current business needs based on the user identifier and the current business scenario; obtaining a data bitmap encoding and a timestamp of the dataset; generating data synchronization information based on the user identifier, the data bitmap encoding, and the timestamp; and sending the data synchronization information to the first server, wherein the data synchronization information is used to determine the target dataset.

[0014] In one embodiment of this application, the method further includes: receiving a target timestamp of the target dataset sent by the first server, and updating the timestamp to the target timestamp.

[0015] Thirdly, this application provides a data synchronization device, comprising: a receiving module for receiving data bitmap encoding sent by a second server; a first determining module for determining a target dataset to be synchronized in a first server based on the encoding information at each position in the data bitmap encoding; a second determining module for determining a target synchronization mode for data synchronization with the second server based on the data volume of the target dataset, the target synchronization mode including at least one of message mode and file packet mode; and a first sending module for sending the target dataset to the second server according to the target synchronization mode.

[0016] In one embodiment of this application, the first determining module is further configured to: receive data synchronization information sent by the second server, wherein the data synchronization information includes at least the data bitmap encoding; and determine the target dataset based on the data synchronization information.

[0017] In one embodiment of this application, the data synchronization information includes a user identifier, the data bitmap encoding, and a first timestamp. The first determining module is further configured to: determine a first dataset from the candidate datasets of the first server based on the user identifier; determine a second dataset from the first dataset based on the data bitmap encoding; and determine a dataset that meets the data synchronization conditions from the second dataset based on the first timestamp and a second timestamp of the second dataset, as the target dataset.

[0018] In one embodiment of this application, the first determining module is further configured to: compare the first timestamp with the second timestamp to obtain a candidate second timestamp that is greater than the first timestamp; and use the dataset corresponding to the candidate second timestamp as the dataset that satisfies the data synchronization condition.

[0019] In one embodiment of this application, the first determining module is further configured to: obtain a third timestamp of the dataset in the first server; determine whether the dataset has been updated based on the third timestamp; in response to determining that the dataset has been updated, determine whether the dataset meets the data synchronization conditions; and in response to meeting the data synchronization conditions, determine that the dataset is the target dataset.

[0020] In one embodiment of this application, the second determining module is further configured to: determine the message mode as the target synchronization mode in response to the data volume of the target dataset being less than or equal to a set data volume threshold; determine the message mode and the file packet mode as the target synchronization mode in response to the data volume of the target dataset being greater than the set data volume threshold, or determine the file packet mode as the target synchronization mode.

[0021] In one embodiment of this application, the apparatus further includes: a second sending module, configured to obtain a target timestamp of the target dataset and send the target timestamp to a second server, wherein the target timestamp is used to update the first timestamp.

[0022] Fourthly, this application provides another data synchronization device, comprising: a first generation module, configured to generate a data bitmap encoding and send the data bitmap encoding to a first server, the data bitmap encoding being used to determine a target dataset to be synchronized in the first server; and a first receiving module, configured to receive the target dataset sent by the first server according to a target synchronization mode, the target synchronization mode including at least one of a message mode and a file packet mode.

[0023] In one embodiment of this application, the apparatus further includes a second generation module, the second generation module being configured to: obtain a user identifier, and determine a dataset for current business needs based on the user identifier and the current business scenario; obtain a data bitmap encoding and a timestamp of the dataset; generate data synchronization information based on the user identifier, the data bitmap encoding, and the timestamp; and send the data synchronization information to the first server, wherein the data synchronization information is used to determine the target dataset.

[0024] In one embodiment of this application, the apparatus further includes: a second receiving module, configured to receive the target timestamp of the target dataset sent by the first server, and update the timestamp to the target timestamp.

[0025] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the data synchronization method as described in the first or second aspect embodiments above.

[0026] In a sixth aspect, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data synchronization method as described in the first or second aspect embodiments above.

[0027] In a seventh aspect, this application provides a computer program product, characterized in that it includes a computer program that, when executed by a processor, implements the data synchronization method as described in the first or second aspect embodiments above.

[0028] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects: based on the amount of data to be synchronized, the data is synchronized in the form of messages and / or file packets, which can take into account both the real-time and efficiency of data synchronization, thereby simultaneously meeting the timeliness and efficiency requirements of data synchronization. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0030] Figure 1 The flow of the data synchronization method provided in the embodiments of this application Figure 1 ;

[0031] Figure 2 The flow of the data synchronization method provided in the embodiments of this application Figure 2 ;

[0032] Figure 3 The flow of the data synchronization method provided in the embodiments of this application Figure 3 ;

[0033] Figure 4 The flow of the data synchronization method provided in the embodiments of this application Figure 4 ;

[0034] Figure 5 The flow of the data synchronization method provided in the embodiments of this application Figure 5 ;

[0035] Figure 6 This is a schematic diagram of the structure of a data synchronization device provided in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of another data synchronization device provided in an embodiment of this application;

[0037] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0038] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0040] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a” and “the” as used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0041] It should be understood that although the terms first, second, third, etc., may be used to describe various information in the embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" and "suppose" as used herein can be interpreted as "when," "when," or "in response to a determination."

[0042] Embodiments of this application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0043] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0044] The data synchronization method, apparatus, electronic device, and storage medium of this application are described below with reference to the accompanying drawings.

[0045] Figure 1 The flow of the data synchronization method provided in the embodiments of this application Figure 1 It should be noted that the data synchronization method provided in this application embodiment is executed by a first server.

[0046] like Figure 1 As shown, the data synchronization method includes:

[0047] S101, Receive the bitmap encoding of the data sent by the second server.

[0048] The data bitmap encoding includes multi-bit encoding, with each bit corresponding to a dataset. Optionally, the encoding can be 0 or 1, where 0 indicates no synchronization is needed and 1 indicates synchronization is required.

[0049] The second server can generate the corresponding data bitmap encoding based on the dataset required by the current business and send the data bitmap encoding to the first server.

[0050] S102. Based on the encoding information at each position in the data bitmap encoding, determine the target dataset to be synchronized in the first server.

[0051] In this application, the data in the first server can be pre-divided into multiple candidate datasets. Optionally, the data of each user in the first server can be used as a candidate dataset. Since users often only use specific data when handling business, the candidate datasets can be further divided into multiple datasets, where each dataset corresponds to the encoding of a position in the data bitmap encoding.

[0052] Optionally, the encoding information includes 0 or 1.

[0053] After receiving the data bitmap encoding sent by the second server, the first server uses the dataset corresponding to the position encoded as 1 in the data bitmap encoding as the target dataset to be synchronized.

[0054] It should be noted that the data in the first server and the data in the second server are partitioned in the same way. The partitioning method can be set according to actual needs, and no restrictions are imposed here.

[0055] S103. Based on the amount of data in the target dataset, determine the target synchronization mode for data synchronization with the second server, wherein the target synchronization mode includes at least one of message mode and file packet mode.

[0056] Message mode is a data synchronization mode that sends data via messages.

[0057] File packet mode is a data synchronization mode that sends data via file packets.

[0058] Optionally, in response to the data volume of the target dataset being less than or equal to a set data volume threshold, the message mode is determined to be the target synchronization mode; in response to the data volume of the target dataset being greater than the set data volume threshold, both the message mode and the file packet mode are determined to be the target synchronization mode; or, the file packet mode is determined to be the target synchronization mode.

[0059] When the amount of data to be synchronized is small, real-time data synchronization is achieved by synchronizing data through messages. When the amount of data to be synchronized is large, data synchronization can be achieved by synchronizing data through messages and file packets, or by synchronizing data through file packets, thus ensuring the efficiency of data synchronization. This balances the real-time nature and efficiency of data synchronization.

[0060] S104. Send the target dataset to the second server according to the target synchronization mode.

[0061] If the target synchronization mode is message mode, the first server sends the target dataset to the second server via messages; if the target synchronization mode is both message mode and file packet mode, the first server sends the target dataset to the second server via messages and file packets; if the target synchronization mode is file packet mode, the first server sends the target dataset to the second server via file packets.

[0062] In this embodiment, a data bitmap encoding sent by a second server is received. Based on the encoding information at each position in the data bitmap encoding, a target dataset to be synchronized in the first server is determined. Based on the data volume of the target dataset, a target synchronization mode for data synchronization with the second server is determined. The target synchronization mode includes at least one of message mode and file packet mode. The target dataset is then sent to the second server according to the target synchronization mode. In this embodiment, determining whether to synchronize data via messages and / or file packets based on the data volume of the data to be synchronized balances the real-time nature and efficiency of data synchronization, thus simultaneously meeting the timeliness and efficiency requirements of data synchronization.

[0063] Figure 2 The flow of the data synchronization method provided in the embodiments of this application Figure 2 It should be noted that the data synchronization method provided in this application embodiment is executed by a first server.

[0064] like Figure 2 As shown, this data synchronization method includes the following steps:

[0065] S201. Receive data synchronization information sent by the second server. The data synchronization information includes at least data bitmap encoding.

[0066] The second server can generate data synchronization information according to data synchronization requirements and send the data synchronization information to the first server.

[0067] Optionally, the second server sends data synchronization information to the first server via a message.

[0068] S202. Determine the target dataset based on the data synchronization information.

[0069] Optionally, the data synchronization information includes user identifier, data bitmap encoding, and first timestamp.

[0070] The user identifier includes the user identity document (ID).

[0071] A timestamp is a complete and verifiable record of data that indicates its existence at a specific point in time. The first timestamp is the timestamp of the dataset indicated by the bitmap encoding of the data on the second server.

[0072] Based on the above embodiments, the target dataset is determined according to the data synchronization information, including:

[0073] Step a1: Determine the first dataset from the candidate datasets of the first server based on the user identifier.

[0074] The first dataset of a user can be found from the candidate datasets in the first server by using the user identifier as a search term.

[0075] Step a2: Determine the second dataset from the first dataset based on the data bitmap encoding.

[0076] Optionally, the first dataset can be divided into multiple datasets according to the business scenario, and the multiple datasets can be arranged in a set order to obtain a dataset sequence. Then, the dataset sequence is represented by data bitmap encoding, where each bit in the data bitmap encoding corresponds to one dataset in the dataset sequence.

[0077] For example, if the first dataset is divided into 10 datasets, the bitmap encoding of the data will be 10 bits, such as 0000000010, 0100000000, etc.

[0078] Since encoding 1 indicates demand synchronization, the dataset corresponding to the position encoded as 1 in the data bitmap can be used as the second dataset. For example, if the data bitmap encoding is 0000000010, then the 9th dataset in the dataset sequence is the second dataset; if the data bitmap encoding is 0100000000, then the 2nd dataset in the dataset sequence is the second dataset.

[0079] Step a3: Based on the first timestamp and the second timestamp of the second dataset, determine the dataset that meets the data synchronization conditions from the second dataset as the target dataset.

[0080] Optionally, the first timestamp is compared with the second timestamp to obtain a candidate second timestamp that is greater than the first timestamp, and the dataset corresponding to the candidate second timestamp is used as the dataset that meets the data synchronization conditions.

[0081] For example, suppose a user (ID: u00001) enters the system of the second server for the first time to conduct business. The user data is stored in the first server. At this time, the second server does not have the user's data. In order to process the business for the user normally, the second server needs to synchronize the user's data with the first server to conduct normal business processing.

[0082] In this case, the second server can send the data synchronization information to the first server in the form of a message, wherein the message format is as follows:

[0083]

[0084] Assuming the first dataset corresponding to user u00001 is divided into 10 second datasets, then the data bitmap encoding Scope: 1111111111 indicates that all data in the first dataset for user u00001 needs to be synchronized, i.e., synchronizing all of user u00001's data. Additionally, since user u00001 is conducting business for the first time in the second service system, there is no data for user u00001 in the second server at this time, therefore the first timestamp is empty (null).

[0085] After receiving the message from the second server, the first server can parse the message to obtain UserID:u00001 and Scope:1111111111. Using u00001 as a key, it searches for the first dataset in its own candidate dataset. After finding the first dataset, since Scope:1111111111 corresponds to each second dataset within the first dataset, it obtains the second timestamp of each second dataset on the first server. Since the first timestamp of the second dataset on the second server is empty, it can be assumed that the first timestamp is less than the second timestamp. Therefore, each second dataset within the first dataset can be identified as the target dataset.

[0086] Assuming user (ID: u00001) has accessed the second server's system before to conduct business, then the second server contains the user's data. The message format would be as follows:

[0087]

[0088]

[0089] In this context, the data bitmap encoding Scope:1000000001 indicates that the first and tenth datasets in the dataset sequence of the first dataset need to be synchronized.

[0090] S203. Based on the amount of data in the target dataset, determine the target synchronization mode for data synchronization with the second server, wherein the target synchronization mode includes at least one of message mode and file packet mode.

[0091] S204. Send the target dataset to the second server according to the target synchronization mode.

[0092] For example, assuming the target synchronization mode is message mode, the message format is as follows:

[0093]

[0094] In this context, Same:0 indicates that the dataset on the first server indicated by Scope:1000000001 is inconsistent with the dataset indicated by Scope:1000000001, requiring data synchronization. Conversely, Same:1 indicates that the dataset on the first server indicated by Scope:1000000001 is consistent with the dataset indicated by Scope:1000000001, and no data synchronization is required. TimeStamp_2 represents the target timestamp of the target dataset. FileName: null indicates that the file package is not included. Data1 and Data10 represent the target dataset to be synchronized to the second server.

[0095] Assuming the target synchronization mode is either message mode or file packet mode, the message format is as follows:

[0096]

[0097] In this case, FileName:fileNameXXX indicates that a file package with the file name fileNameXXX is returned with synchronization data.

[0098] For a detailed description of steps S203-S204, please refer to the relevant content in the above embodiments, which will not be repeated here.

[0099] Furthermore, the data synchronization method also includes: obtaining the target timestamp of the target dataset and sending the target timestamp to the second server, the target timestamp being used to update the first timestamp.

[0100] It can obtain the target timestamp of the target dataset, send the target dataset and the target timestamp together to the second server, so as to update the first timestamp to the target timestamp.

[0101] In this embodiment, data synchronization information sent by a second server is received. This data synchronization information includes at least a data bitmap encoding. A target dataset is determined based on the data synchronization information. Based on the data volume of the target dataset, a target synchronization mode for data synchronization with the second server is determined. The target synchronization mode includes at least one of message mode and file packet mode. The target dataset is then sent to the second server according to the target synchronization mode. In this embodiment, determining whether to synchronize data in the form of messages and / or file packets based on the data volume of the data to be synchronized balances the real-time nature and efficiency of data synchronization, thus simultaneously meeting the timeliness and efficiency requirements of data synchronization.

[0102] When data is updated on the first server, the first server can proactively synchronize the updated data to the second server. The following section combines... Figure 3 Provide an explanation. Figure 3 The flow of the data synchronization method provided in the embodiments of this application Figure 3 It should be noted that the data synchronization method provided in this application embodiment is executed by a first server.

[0103] like Figure 3 As shown, this data synchronization method includes the following steps:

[0104] S301. Obtain the third timestamp of the dataset in the first service.

[0105] S302. Determine whether the dataset has been updated based on the third timestamp.

[0106] S303. In response to determining that the dataset has been updated, determine whether the dataset meets the data synchronization conditions.

[0107] S304. In response to the satisfaction of the data synchronization conditions, the dataset is determined to be the target dataset.

[0108] The system retrieves the third timestamp of the dataset from the first server and determines whether the dataset has been updated based on this timestamp. If so, it sends the corresponding bitmap encoding of the dataset to the second server. Upon receiving the bitmap encoding, the second server identifies the dataset corresponding to the bitmap encoding from its own dataset and sends the fourth timestamp of that dataset to the first server. Upon receiving the fourth timestamp, the first server determines whether it is less than the third timestamp. If so, it determines that the dataset meets the data synchronization conditions and designates that dataset as the target dataset.

[0109] S305. Based on the amount of data in the target dataset, determine the target synchronization mode for data synchronization with the second server. The target synchronization mode includes at least one of message mode and file packet mode.

[0110] S306. Send the target dataset to the second server according to the target synchronization mode.

[0111] For a detailed description of steps S305-S306, please refer to the relevant content in the above embodiments, which will not be repeated here.

[0112] In this embodiment, a third timestamp of the dataset in the first server is obtained. Based on the third timestamp, it is determined whether the dataset has been updated. In response to determining that the dataset has been updated, it is determined whether the dataset meets the data synchronization conditions. In response to meeting the data synchronization conditions, the dataset is determined as the target dataset. Based on the data volume of the target dataset, a target synchronization mode for data synchronization with the second server is determined. The target synchronization mode includes at least one of message mode and file packet mode. The target dataset is then sent to the second server according to the target synchronization mode. In this embodiment, when data in the first server is updated, the first server can proactively synchronize the updated data with the second server. This ensures that the data in the first and second servers is consistent, thereby guaranteeing normal business processing for users.

[0113] Figure 4 The flow of the data synchronization method provided in the embodiments of this application Figure 4 It should be noted that the data synchronization method provided in this application embodiment is executed by a second server.

[0114] like Figure 4 As shown, this data synchronization method includes the following steps:

[0115] S401. Generate a data bitmap encoding and send the data bitmap encoding to the first server. The data bitmap encoding is used to determine the target dataset to be synchronized in the first server.

[0116] The second server can generate a data bitmap encoding according to the data synchronization requirements and send the data bitmap encoding to the first server.

[0117] Optionally, the second server sends the data bitmap encoding to the first server via a message.

[0118] S402. Receive the target dataset sent by the first server according to the target synchronization mode, wherein the target synchronization mode includes at least one of message mode and file packet mode.

[0119] Optionally, after receiving the data bitmap encoding sent by the second server, the first server determines the target dataset based on the data bitmap encoding and obtains the data volume of the target dataset. If the data volume of the target dataset is less than or equal to a set data volume threshold, the message mode is determined to be the target synchronization mode. If the data volume of the target dataset is greater than the set data volume threshold, the message mode and the file packet mode are determined to be the target synchronization mode, or the file packet mode is determined to be the target synchronization mode.

[0120] If the target synchronization mode is message mode, the first server sends the target dataset to the second server via messages; if the target synchronization mode is both message mode and file packet mode, the first server sends the target dataset to the second server via messages and file packets; if the target synchronization mode is file packet mode, the first server sends the target dataset to the second server via file packets.

[0121] In this embodiment, a data bitmap encoding is generated and sent to a first server. The data bitmap encoding is used to determine the target dataset to be synchronized in the first server. The first server receives the target dataset sent by the first server according to a target synchronization mode, wherein the target synchronization mode includes at least one of message mode and file packet mode. In this embodiment, the synchronization method of synchronizing data through messages and / or file packets is determined based on the amount of data to be synchronized, which can balance the real-time nature and efficiency of data synchronization, thereby simultaneously meeting the timeliness and efficiency requirements of data synchronization.

[0122] Figure 5 The flow of the data synchronization method provided in the embodiments of this application Figure 5 It should be noted that the data synchronization method provided in this application embodiment is executed by a second server.

[0123] like Figure 5 As shown, this data synchronization method includes the following steps:

[0124] S501. Obtain the user identifier and, based on the user identifier and the current business scenario, determine the dataset for the current business requirements.

[0125] Optionally, the user identifier includes the user ID.

[0126] When conducting business on the user's secondary server system, the user's identifier can be entered into the secondary server through the system.

[0127] In this embodiment, all data in the second server can be pre-divided into multiple candidate datasets. Optionally, the data of each user in the first server can be used as one candidate dataset.

[0128] Furthermore, each candidate dataset can be divided into multiple datasets according to the business scenario.

[0129] In some embodiments, the target candidate dataset corresponding to the user is determined from the candidate dataset of the second server based on the user identifier, and then the dataset required for the current business scenario is determined from the target candidate dataset, which is the dataset required to be synchronized in the second server.

[0130] S502. Obtain the data bitmap encoding and timestamp of the dataset.

[0131] Since each dataset in the target candidate dataset corresponds to one bit in the data bitmap encoding, the data bitmap encoding can be determined based on the dataset synchronization requirements in the target candidate dataset.

[0132] For example, suppose the target candidate dataset is divided into 10 datasets, then the data bitmap encoding has 10 bits. If the 10th dataset in the target candidate dataset needs to synchronize with the other dataset, then the data bitmap encoding is 1000000001, where encoding 1 indicates synchronization is required and encoding 0 indicates synchronization is not required.

[0133] S503 generates data synchronization information based on user identifier, data bitmap encoding, and timestamp.

[0134] S504. Send the data synchronization information to the first server. The data synchronization information is used to determine the target dataset to be synchronized in the first server.

[0135] After obtaining the user identifier, data bitmap encoding, and timestamp, data synchronization information can be generated based on the data bitmap encoding and sent to the first server. After receiving the data synchronization information, the first server determines the target dataset to be synchronized based on the data synchronization information.

[0136] Optionally, data synchronization information can be sent to the first server via a message.

[0137] For example, the message format is as follows:

[0138]

[0139] Where UserID:u00001 represents the user identifier, Type:1 indicates that the data transmission method is message, Data Bitmap Encoding Scope:1000000001 indicates that the 1st and 10th datasets in the candidate dataset need to be synchronized, and TimeStamp represents the timestamp.

[0140] S505. Receive the target dataset sent by the first server according to the target synchronization mode, where the target synchronization mode includes at least one of message mode and file packet mode.

[0141] For a detailed description of steps S504-S505, please refer to the relevant content in the above embodiments, which will not be repeated here.

[0142] After receiving the target dataset sent by the first server, the second server updates the dataset for the current business requirements to the target dataset. That is, it updates the dataset for the current business requirements in the second server to the target dataset and deletes the dataset for the current business requirements in the second server.

[0143] Furthermore, it receives the target timestamp of the target dataset sent by the first server and updates the timestamp to the target timestamp.

[0144] When the first server sends the target dataset to the second server, it can also send the target timestamp of the target dataset to the second server. After receiving the target timestamp, the second server updates the timestamp of the dataset required by the current business to the target timestamp.

[0145] In this embodiment, a user identifier is obtained, and based on the user identifier and the current business scenario, the dataset required for the current business needs is determined. The data bitmap encoding and timestamp of the dataset are obtained. Based on the user identifier, data bitmap encoding, and timestamp, data synchronization information is generated and sent to a first server. This data synchronization information is used to determine the target dataset to be synchronized in the first server. The target dataset sent by the first server according to the target synchronization mode is received. The target synchronization mode includes at least one of message mode and file packet mode. In this embodiment, determining whether to synchronize data via messages and / or file packets based on the amount of data to be synchronized balances the real-time nature and efficiency of data synchronization, thus simultaneously meeting the timeliness and efficiency requirements of data synchronization.

[0146] Figure 6 This application provides a schematic diagram of the structure of a data synchronization device according to an embodiment. For example... Figure 6 As shown, the data synchronization device 600 includes: a receiving module 610, a first determining module 620, a second determining module 630, and a first sending module 640.

[0147] The receiving module 610 is used to receive the bitmap encoding of the data sent by the second server.

[0148] The first determining module 620 is used to determine the target dataset to be synchronized in the first server based on the encoding information at each position in the data bitmap encoding.

[0149] The second determining module 630 is used to determine the target synchronization mode for data synchronization with the second server based on the amount of data in the target dataset, wherein the target synchronization mode includes at least one of message mode and file packet mode.

[0150] The first sending module 640 is used to send the target dataset to the second server in a target synchronization mode.

[0151] In some embodiments, the first determining module 610 is further configured to: receive data synchronization information sent by the second server, wherein the data synchronization information includes at least data bitmap encoding; and determine the target dataset based on the data synchronization information.

[0152] In some embodiments, the data synchronization information includes a user identifier, a data bitmap encoding, and a first timestamp. The first determining module 620 is further configured to: determine a first dataset from the candidate datasets of the first server based on the user identifier; determine a second dataset from the first dataset based on the data bitmap encoding; and determine a dataset that meets the data synchronization conditions from the second dataset based on the first timestamp and the second timestamp of the second dataset, as the target dataset.

[0153] In some embodiments, the first determining module 620 is further configured to: compare the first timestamp with the second timestamp to obtain a candidate second timestamp that is greater than the first timestamp; and use the dataset corresponding to the candidate second timestamp as the dataset that satisfies the data synchronization condition.

[0154] In some embodiments, the first determining module 620 is further configured to: obtain a third timestamp of the candidate dataset; determine whether the candidate dataset has been updated based on the third timestamp; in response to determining that the candidate dataset has been updated, determine whether the candidate dataset meets the data synchronization conditions; and in response to meeting the data synchronization conditions, determine the candidate dataset as the target dataset.

[0155] In some embodiments, the second determining module 630 is further configured to: determine the message mode as the target synchronization mode in response to the data volume of the target dataset being less than or equal to a set data volume threshold; determine the message mode and the file packet mode as the target synchronization mode in response to the data volume of the target dataset being greater than the set data volume threshold; or determine the file packet mode as the target synchronization mode.

[0156] In some embodiments, the data synchronization device 600 further includes a second sending module 650, which is used to obtain the target timestamp of the target dataset and send the target timestamp to a second server, wherein the target timestamp is used to update the first timestamp.

[0157] The data synchronization device provided in this application embodiment can be used to execute the data synchronization method of the first aspect embodiment above. Its implementation principle and technical effect are similar, and will not be described again here.

[0158] Figure 7 A schematic diagram of the data synchronization device provided in this application embodiment is shown. (See attached diagram.) Figure 7 As shown, the data synchronization device 700 includes: a first generation module 710 and a first receiving module 720.

[0159] The first generation module 710 is used to generate a data bitmap encoding and send the data bitmap encoding to the first server. The data bitmap encoding is used to determine the target dataset to be synchronized in the first server.

[0160] The first receiving module 720 is used to receive the target dataset sent by the first server according to the target synchronization mode, wherein the target synchronization mode includes at least one of message mode and file packet mode.

[0161] In some embodiments, the data synchronization device 700 further includes a second generation module 730, configured to acquire a user identifier, and determine the dataset for the current business needs based on the user identifier and the current business scenario; acquire the data bitmap encoding and timestamp of the dataset; generate data synchronization information based on the user identifier, data bitmap encoding and timestamp; and send the data synchronization information to a first server, wherein the data synchronization information is used to determine the target dataset.

[0162] In some embodiments, the data synchronization device 700 further includes a second receiving module 740, configured to receive the target timestamp of the target dataset sent by the first server and update the timestamp to the target timestamp.

[0163] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device may include: transceiver 121, processor 122, and memory 123.

[0164] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the scheme in the above embodiments. Processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0165] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0166] Transceiver 121 can be used to obtain the task to be run and its configuration information.

[0167] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0168] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0169] This application also provides a chip for executing instructions, which is used to execute the data synchronization method described in the first or second aspect of the above-described embodiments.

[0170] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the data synchronization method described in the first or second aspect of the above-described embodiments.

[0171] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the data synchronization method in the first or second aspect of the above embodiment.

[0172] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0173] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data synchronization method, characterized in that, The method, executed by the first server, includes: The system receives data synchronization information sent by a second server. The data synchronization information includes at least a user identifier, a data bitmap encoding, and a first timestamp. The data bitmap encoding includes a multi-bit encoding, with each bit encoding corresponding to a dataset. The data bitmap encoding is generated by the second server based on the dataset used by the current business needs. The first timestamp is the timestamp of the dataset indicated by the data bitmap encoding. Based on the user identifier, a first dataset is determined from the candidate dataset of the first server; Based on the data bitmap encoding, a second dataset is determined from the first dataset; Based on the first timestamp and the second timestamp of the second dataset, a dataset that meets the data synchronization conditions is determined from the second dataset and used as the target dataset to be synchronized in the first server; Based on the amount of data in the target dataset, a target synchronization mode for data synchronization with the second server is determined, wherein the target synchronization mode includes at least one of message mode and file packet mode; The target dataset is sent to the second server according to the target synchronization mode.

2. The method according to claim 1, characterized in that, The step of determining the dataset that meets the data synchronization conditions from the second dataset based on the first timestamp and the second timestamp of the second dataset includes: Compare the first timestamp with the second timestamp to obtain a candidate second timestamp that is greater than the first timestamp; The dataset corresponding to the candidate second timestamp is taken as the dataset that satisfies the data synchronization condition.

3. The method according to claim 1, characterized in that, The step of determining the target dataset to be synchronized in the first server further includes: Obtain the third timestamp of the dataset in the first server; Based on the third timestamp, determine whether the dataset has been updated; In response to determining that the dataset has been updated, it is then determined whether the dataset meets the data synchronization conditions; In response to the fulfillment of the data synchronization conditions, the dataset is determined to be the target dataset.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the target synchronization mode for data synchronization with the second server based on the data volume of the target dataset includes: If the data volume of the target dataset is less than or equal to a set data volume threshold, then the message mode is determined to be the target synchronization mode. If the data volume of the target dataset exceeds a set data volume threshold, then the message mode and the file packet mode are determined to be the target synchronization mode, or the file packet mode is determined to be the target synchronization mode.

5. The method according to claim 1 or 3, characterized in that, The method further includes: Obtain the target timestamp of the target dataset and send the target timestamp to the second server. The target timestamp is used to update the first timestamp.

6. A data synchronization method, characterized in that, The method, executed by a second server, includes: Obtain the user identifier, and based on the user identifier and the current business scenario, determine the dataset for the current business requirements; Obtain the data bitmap encoding and timestamp of the dataset; the data bitmap encoding includes multi-bit encoding, each position of the encoding corresponds to a dataset, the data bitmap encoding is generated by the second server according to the dataset used by the current business needs, and the timestamp is the timestamp of the dataset indicated by the data bitmap encoding; Based on the user identifier, the data bitmap encoding, and the timestamp, data synchronization information is generated and sent to the first server. The data synchronization information is used to determine the target dataset to be synchronized in the first server. The system receives the target dataset sent by the first server in a target synchronization mode, wherein the target synchronization mode includes at least one of message mode and file packet mode.

7. The method according to claim 6, characterized in that, The method further includes: Receive the target timestamp of the target dataset sent by the first server, and update the timestamp to the target timestamp.

8. A data synchronization device, characterized in that, include: The receiving module is used to receive data synchronization information sent by the second server. The data synchronization information includes at least a user identifier, a data bitmap encoding, and a first timestamp. The data bitmap encoding includes a multi-bit encoding, and the encoding at each position corresponds to a dataset. The data bitmap encoding is generated by the second server according to the dataset used by the current business needs. The first timestamp is the timestamp of the dataset indicated by the data bitmap encoding. The first determining module is configured to determine a first dataset from the candidate datasets of the first server based on the user identifier; determine a second dataset from the first dataset based on the data bitmap encoding; and determine a dataset that meets the data synchronization conditions from the second dataset based on the first timestamp and the second timestamp of the second dataset, as the target dataset to be synchronized in the first server. The second determining module is used to determine the target synchronization mode for data synchronization with the second server based on the amount of data in the target dataset. The target synchronization mode includes at least one of message mode and file packet mode. The first sending module is used to send the target dataset to the second server according to the target synchronization mode.

9. The apparatus according to claim 8, characterized in that, The first determining module is further configured to: Compare the first timestamp with the second timestamp to obtain a candidate second timestamp that is greater than the first timestamp; The dataset corresponding to the candidate second timestamp is taken as the dataset that satisfies the data synchronization condition.

10. The apparatus according to claim 8, characterized in that, The first determining module is further configured to: Obtain the third timestamp of the dataset in the first server; Based on the third timestamp, determine whether the dataset has been updated; In response to determining that the dataset has been updated, it is then determined whether the dataset meets the data synchronization conditions; In response to the fulfillment of the data synchronization conditions, the dataset is determined to be the target dataset.

11. The apparatus according to any one of claims 8-10, characterized in that, The second determining module is further configured to: If the data volume of the target dataset is less than or equal to a set data volume threshold, then the message mode is determined to be the target synchronization mode. If the data volume of the target dataset exceeds a set data volume threshold, then the message mode and the file packet mode are determined to be the target synchronization mode, or the file packet mode is determined to be the target synchronization mode.

12. The apparatus according to claim 8 or 10, characterized in that, The device further includes: The second sending module is used to obtain the target timestamp of the target dataset and send the target timestamp to the second server, wherein the target timestamp is used to update the first timestamp.

13. A data synchronization device, characterized in that, include: The first generation module is used to obtain user identifiers and, based on the user identifiers and the current business scenario, determine the dataset for the current business requirements. Obtain the data bitmap encoding and timestamp of the dataset; the data bitmap encoding includes multi-bit encoding, and the encoding at each position corresponds to a dataset. The data bitmap encoding is generated by the second server according to the dataset used by the current business needs. The timestamp is the timestamp of the dataset indicated by the data bitmap encoding. Based on the user identifier, the data bitmap encoding, and the timestamp, data synchronization information is generated and sent to the first server. The data bitmap encoding is used to determine the target dataset to be synchronized in the first server. The first receiving module is configured to receive the target dataset sent by the first server according to the target synchronization mode, wherein the target synchronization mode includes at least one of message mode and file packet mode.

14. The apparatus according to claim 13, characterized in that, The device further includes: The second receiving module is used to receive the target timestamp of the target dataset sent by the first server and update the timestamp to the target timestamp.

15. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as claimed in any one of claims 1-5 or 6-7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5 or 6-7.

17. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-5 or 6-7.

Images