A data synchronization method and system

Abstract

The present invention discloses a data synchronization method and system, and relates to the field of computer technology. A specific implementation of the method includes: receiving a synchronization write command from a client, loading a predefined synchronization module, and saving the current key in the synchronization write command and the first version information to a source database according to the implementation logic of the synchronization write command in the synchronization module; calling the synchronization read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the synchronization key of the data to be synchronized from the source database; calling the synchronization playback command of the synchronization module to obtain the second version information, and if the update time indicated by the first version information of the key to be synchronized is later than the update time indicated by the second version information, the data to be synchronized is written into the target database. The method realizes data synchronization of the database based on the predefined synchronization module, without rewriting the database source code, and can be upgraded following the open source database version.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a data synchronization method and system. Background Art

[0002] In order to ensure high availability of services and minimize or even eliminate the impact of system failures on normal services, Internet companies can build a multi-active architecture in different locations to provide users with good services even in abnormal situations. Multi-active architecture in different locations refers to a business scenario where multiple data centers distributed in different locations provide services to the outside world at the same time.

[0003] One of the key links in implementing the design of a multi-active architecture is how to synchronize data between multiple data centers. Currently, there are two main technologies for data synchronization. The first is to transform the database's aof (append only log) log into a binlog log, and then scan the binlog log for synchronization; the second is to use a pseudo-slave solution, that is, to implement the redis protocol, disguise as a redis slave, let the redis master push data to the pseudo-slave, and then synchronize. Among them, aof is a log file that only performs append operations, and binlog is a binary format file.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] (1) Both of the above solutions require selecting a database version and rewriting its source code, which is complex to implement and has high development costs, and cannot be upgraded along with the open source database version.

[0006] (2) When implementing bidirectional synchronization, the above two solutions may have a circular synchronization problem, that is, after synchronizing the updated data of the current database to another database, the updated data of the other database is synchronized to the current database. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a data synchronization method and system, which implements database data synchronization based on a predefined synchronization module, does not require rewriting the database source code, and can be upgraded following the open source database version.

[0008] To achieve the above objective, according to one aspect of an embodiment of the present invention, a data synchronization method is provided.

[0009] A data synchronization method according to an embodiment of the present invention includes: receiving a synchronization write command from a client, loading a predefined synchronization module, and saving the current key in the synchronization write command and the first version information used to indicate the update time of the current key in the source database to the source database according to the implementation logic of the synchronization write command in the synchronization module; calling the synchronization read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the key to be synchronized of the data to be synchronized from the source database; wherein the key to be synchronized includes at least one current key; calling the synchronization playback command of the synchronization module to obtain the second version information used to indicate the update time of the key to be synchronized in the target database, and if the update time indicated by the first version information of the key to be synchronized is later than the update time indicated by the second version information, the data to be synchronized is written to the target database.

[0010] Optionally, the first version information and the second version information are globally incremented version numbers; after the step of obtaining the second version information used to represent the update time of the key to be synchronized in the target database, it also includes: comparing the size of the first version information and the second version information to determine the update time of the key to be synchronized in the source database and the target database.

[0011] Optionally, the method further includes: customizing the synchronization module; wherein the synchronization module includes the implementation logic of the synchronous write command, the implementation logic of the synchronous read command, and the implementation logic of the synchronous playback command.

[0012] Optionally, the implementation logic of the synchronous write command is: calling a native write command corresponding to the data type of the current key to write the key value corresponding to the current key into the source database; according to the set data type, writing the current key and the first version information of the current key into the source database.

[0013] Optionally, the implementation logic of the synchronous read command is: respectively call the native read commands corresponding to the data types of the keys to be synchronized, the key values ​​corresponding to the keys to be synchronized, and the first version information, so as to obtain a specified number of the keys to be synchronized, the key values ​​corresponding to the keys to be synchronized, and the first version information from the source database; return the keys to be synchronized, the key values ​​corresponding to the keys to be synchronized, and the first version information.

[0014] Optionally, the synchronous write command of the client is obtained by adding a set prefix to the native write command.

[0015] Optionally, both the source database and the target database are redis databases, and the synchronous write command, the synchronous read command and the synchronous playback command are implemented in a module module of the redis database.

[0016] To achieve the above objective, according to another aspect of an embodiment of the present invention, a data synchronization system is provided.

[0017] A data synchronization system according to an embodiment of the present invention includes: a loading and saving module, which is used to receive a synchronous write command from a client, load a predefined synchronization module, and save the current key in the synchronous write command and the first version information used to represent the update time of the current key in the source database to the source database according to the implementation logic of the synchronous write command in the synchronization module; a calling acquisition module, which is used to call the synchronous read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the key to be synchronized of the data to be synchronized from the source database; wherein the key to be synchronized includes at least one current key; a calling write module, which is used to call the synchronous playback command of the synchronization module to obtain the second version information used to represent the update time of the key to be synchronized in the target database, and if the update time represented by the first version information of the key to be synchronized is later than the update time represented by the second version information, the data to be synchronized is written to the target database.

[0018] Optionally, the first version information and the second version information are globally incremented version numbers; the system also includes: a comparison and determination module, used to compare the sizes of the first version information and the second version information to determine the update time of the key to be synchronized in the source database and the target database.

[0019] Optionally, the system further includes: a synchronization implementation module: a custom implementation of the synchronization module; wherein the synchronization module includes the implementation logic of the synchronization write command, the implementation logic of the synchronization read command and the implementation logic of the synchronization playback command.

[0020] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0021] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a data synchronization method according to an embodiment of the present invention.

[0022] To achieve the above objective, according to another aspect of the embodiments of the present invention, a computer readable medium is provided.

[0023] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a data synchronization method according to an embodiment of the present invention is implemented.

[0024] An embodiment of the above invention has the following advantages or beneficial effects: data synchronization of the database is realized based on a predefined synchronization module, and there is no need to rewrite the database source code, and it can be upgraded along with the open source database version; the key to be synchronized will not be written into the target database in the implementation logic of the synchronous playback command, thereby avoiding the circular synchronization problem during two-way synchronization; by setting a globally incremented version number and comparing the version number size of the key to be synchronized in the source database and the target database, it is possible to conveniently know whether the update time of the key to be synchronized in the source database is earlier or later than that in the target database, so as to synchronize the data to be synchronized in the source database to the target database; when defining the synchronization module, the synchronous write command, the synchronous read command and the synchronous playback command are realized, and the data synchronization can be completed by calling these three commands subsequently, which is convenient for batch data synchronization and playback, and the development and maintenance cost is low; the synchronization command is obtained by adding a prefix to the native command, which is simple to implement and easy to develop and maintain.

[0025] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0027] Figure 1 is a schematic diagram of main steps of a data synchronization method according to an embodiment of the present invention;

[0028] Figure 2 is a system architecture diagram of a data synchronization method according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of the main flow of a data synchronization method according to an embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of main modules of a data synchronization system according to an embodiment of the present invention;

[0031] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0032] Figure 6 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] Figure 1 FIG. 1 is a schematic diagram of the main steps of the data synchronization method according to an embodiment of the present invention. Figure 1 As shown, the data synchronization method of the embodiment of the present invention mainly includes the following steps:

[0035] Step S101: Receive a synchronous write command from the client, load a predefined synchronization module, and save the current key in the synchronous write command and the first version information used to represent the update time of the current key in the source database to the source database according to the implementation logic of the synchronous write command in the synchronization module. A synchronization module is pre-implemented, and the module includes the implementation logic of the synchronous write command, the implementation logic of the synchronous read command, and the implementation logic of the synchronous playback command. Among them, the synchronous write command is used to write the current key and the first version information corresponding to the current key into the source database according to the set data type. The synchronous read command is used to uniformly obtain the key to be synchronized, the corresponding key value and the first version information. The synchronous playback command is used to obtain the second version information of the key to be synchronized in the target database, and compare it with the first version information, and perform data synchronization according to the comparison result.

[0036] The client corresponding to the source database adds the set prefix to the native write command to obtain the synchronous write command, and then sends the synchronous write command to the source database. After receiving the synchronous write command from the corresponding client, the source database loads the synchronization module to support the synchronous write command, synchronous read command and synchronous playback command. After the source database has loaded the synchronization module, it saves the current key and the corresponding first version information in the synchronous write command according to the implementation logic of the synchronous write command in the synchronization module.

[0037] Step S102: Call the synchronization read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the synchronization key of the data to be synchronized from the source database; wherein the synchronization key includes at least one current key. The data to be synchronized includes the synchronization key and the key value corresponding to the synchronization key. By calling the synchronization read command, a specified number of synchronization keys, the key values ​​corresponding to the synchronization keys and the first version information are obtained from the source database.

[0038] Step S103: Call the synchronous playback command of the synchronization module to obtain the second version information used to indicate the update time of the key to be synchronized in the target database. If the update time indicated by the first version information of the key to be synchronized is later than the update time indicated by the second version information, the data to be synchronized is written into the target database. In an embodiment, the first version information and the second version information are globally incremented version numbers. For example, the first version information of the key to be synchronized is 4.0, and the second version information of the key to be synchronized is 3.0, indicating that the update time of the key to be synchronized in the source database is later than the update time in the target database. By calling the synchronous playback command, the second version information of the key to be synchronized in the target database is obtained, and then the size of the first version information and the second version information are compared. If the first version information is larger than the second version information, the native write command is called to write the data to be synchronized into the target database.

[0039] Figure 2 is a system architecture diagram of a data synchronization method according to an embodiment of the present invention. Figure 2 As shown, the data synchronization method of the embodiment of the present invention is implemented by a source client, a source database, a synchronization extraction proxy node, a synchronization playback proxy node, a target database and a target client. The specific implementation principle is:

[0040] (1) The source client adds a prefix to the native write command to rewrite the native write command into a synchronous write command, and then sends the synchronous write command to the source database.

[0041] (2) The source database and the target database load synchronization modules respectively so that the database supports synchronization write commands, synchronization read commands, and synchronization playback commands. After receiving the synchronization write command, the source database executes the synchronization write command to save the current key in the synchronization write command and the first version information of the current key in the source database. In the embodiment, the current key is saved in the synchronization list of the source database.

[0042] (3) The synchronous extraction proxy node calls the synchronous read command to obtain the data to be synchronized and the first version information corresponding to the synchronization key of the data to be synchronized from the docked source database. A synchronous channel is established between the synchronous extraction proxy node and the synchronous playback proxy node for data transmission.

[0043] (4) The synchronous playback proxy node obtains the first version information corresponding to the data to be synchronized and the key to be synchronized from the synchronization channel, and then calls the synchronous playback command to obtain the second version information of the key to be synchronized in the target database, compares the size of the first version information and the second version information, and writes the data to be synchronized to the target database when the first version information is larger than the second version information. The target client can then obtain the data to be synchronized from the target database through the native read command.

[0044] The synchronization module in the embodiment of the present invention needs to be pre-defined and implemented so that it can be loaded into the database as an external extension function of the database. When performing data synchronization, only the synchronization module needs to be developed, and there is no need to change the database source code. It can be upgraded at any time with the open source version; it is convenient to synchronize and replay batch data; because the synchronization playback command will not write the synchronization key to be synchronized into the synchronization list of the target database, there is no circular synchronization problem during two-way synchronization; the development and maintenance cost is low. In the embodiment, the database is a key-value type database, such as a redis database.

[0045] The number of databases in the embodiment may be greater than 2, and each database may be placed in a remote computer room. When performing data synchronization, any one of the databases may be selected as the source database, and the other databases may be selected as the corresponding target databases.

[0046] Figure 3 FIG. 1 is a schematic diagram of the main flow of the data synchronization method according to an embodiment of the present invention. Figure 3 As shown, the data synchronization method of the embodiment of the present invention mainly includes the following steps:

[0047] Step S301: Based on the module function of the database, customize the synchronization module. Redis 4.0 and above versions support module functions (module), so that users can functionally expand the redis database according to their own needs. In an embodiment, the function to be expanded can be implemented through a synchronization module. The specific implementation of the synchronization module is: adding a synchronous write command corresponding to the native write command, adding a synchronous read command, and a synchronous playback command corresponding to the native write command. For example, the synchronous write command corresponding to the native set command is the sync.set command, and the corresponding synchronous playback command is the repl.set command; the synchronous read command is, for example, the sync.get command.

[0048] The implementation logic of the synchronous write command, synchronous read command, and synchronous playback command is as follows:

[0049] (1) The implementation logic of the synchronous write command is:

[0050] (11) Call the native write command. Redis supports five basic data types: string, hash, list, set, and zset. Each data type has a corresponding native write command. In the embodiment, by calling the native write command, the current key and the corresponding key value of the native write command are written to the source database. Taking the set command of the string type as an example, after calling set k1 v1, k1 and v1 will be recorded in the Redis database.

[0051] (12) The current key in the native write command is added to the synchronization list through the rpush command. The rpush command is used to insert one or more values ​​into the tail (rightmost) of the list. It should be noted that the data type of the current key written to the source database can be customized according to the requirements. Due to the orderliness of the list, it is convenient to write and read data, so the current key is written to the source database in the form of a list in the embodiment.

[0052] (13) Write the first version information corresponding to the current key in the native write command into the source database in a string type or an ordered set type. Preferably, the version information may be a globally incremented version number. In an embodiment, in addition to the version number, the current timestamp may also be used as the version information of the current key, and the value of the current timestamp is the system time of the client corresponding to the source database (i.e., the source client).

[0053] (2) The implementation logic of the synchronous read command is:

[0054] (21) Execute the lrange command to obtain a specified number of keys to be synchronized from the synchronization list. In the embodiment, the lrange command is used to obtain one or more current keys from the synchronization list as keys to be synchronized.

[0055] (22) Call the native read command to obtain the key value corresponding to the key to be synchronized. Each data type also has a corresponding native read command. Taking the string type as an example, you can call the native get command to obtain the key value corresponding to the key to be synchronized.

[0056] (23) Call the native read command to obtain the first version information corresponding to the key to be synchronized from the source database. If it is a string type, call the native get command to obtain the first version information corresponding to the key to be synchronized; if it is an ordered set type, call the native zrange command to obtain the first version information corresponding to the key to be synchronized.

[0057] (24) Return the key to be synchronized, the key value corresponding to the key to be synchronized, and the first version information.

[0058] (3) The implementation logic of the synchronous playback command is:

[0059] (31) Obtain the first version information V1 corresponding to the key to be synchronized, and the second version information V2 of the key to be synchronized in the target database, and determine the size of the first version information V1 and the second version information V2. If the first version information V1>the second version information V2, execute (32); otherwise, return directly. If the first version information V1>the second version information V2, it means that the source database corresponding to the first version information V1 is updated, and the target database corresponding to the second version information V2 needs to be updated. If the first version information V1<the second version information V2, it means that the target database corresponding to the second version information V2 has been updated, and there is no need to process the target database. In the embodiment, in addition to the version number, the current timestamp can also be used as the second version information of the key to be synchronized, and the value of the current timestamp is the system time of the client corresponding to the target database (i.e., the target client).

[0060] (32) Call the native write command. Call the native write command corresponding to the current data type. For example, if the current data type is string type, call the native set command. After this step is completed, the key to be synchronized and the corresponding key value will be written into the target database, and the entire data synchronization process is completed. In the embodiment, each database corresponds to a synchronization list. Since the synchronous playback command will not write the key to be synchronized into the synchronization list of the target database, when the data of the target database is reversely synchronized to the source database, the key to be synchronized will not be obtained from the synchronization list. Therefore, without adding special processing such as database instance identification or other identification, the circular synchronization problem during two-way synchronization can be avoided.

[0061] Step S302: The source client adds a set prefix to the native write command to obtain a synchronous write command, and sends the synchronous write command to the source database. The source client rewrites the native write command that needs to be synchronized. Assuming that sync. is the prefix of the rewritten synchronous write command, taking the set command as an example, the set command can be rewritten into the sync.set command through the above steps.

[0062] The basic syntax of the set command is: redis>set key value. The basic syntax of the sync.set command after rewriting is: redis>sync.set key value.

[0063] Step S303: The source database loads the synchronization module to execute the synchronization write command according to the implementation logic of the synchronization write command in the synchronization module. By executing the synchronization write command, the source database writes the current key and the corresponding key value in the native write command to the source database, adds the current key to the synchronization list of the source database, and writes the first version information corresponding to the current key to the source database in the form of a string type or an ordered set type.

[0064] Step S304: The synchronous extraction proxy node calls the synchronous read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the key to be synchronized from the source database. In an embodiment, the data to be synchronized includes the key to be synchronized and the key value corresponding to the key to be synchronized. The synchronous extraction proxy node executes the synchronous read command according to the implementation logic of the synchronous read command in the synchronization module to obtain the key to be synchronized from the synchronization list, and obtains the key to be synchronized, the key value corresponding to the key to be synchronized, and the first version information.

[0065] Step S305: The target database loads the synchronization module. The embodiment does not limit the execution order of this step, and it can be executed after step S301 and before step S306.

[0066] Step S306: The synchronous playback proxy node obtains the data to be synchronized from the synchronization channel, and then calls the synchronous playback command of the synchronization module to write the data to be synchronized into the target database. Among them, the synchronization channel is essentially a TCP connection for data transmission, and the message channel of the message system can be reused. According to the implementation logic of the synchronous playback command in the synchronization module, the synchronous playback proxy node uses the key to be synchronized, the corresponding key value and version information as input parameters, obtains the serialized information from the source database, and deserializes it in the synchronous playback proxy node to obtain the first version information V1; then obtains the second version information V2 of the key to be synchronized in the target database, compares the size of the first version information V1 and the second version information V2, and if the first version information V1>the second version information V2, the data to be synchronized is written into the target database. If the first version information V1<the second version information V2, no processing is performed.

[0067] It can be seen from the data synchronization method of the embodiment of the present invention that the data synchronization of the database is realized based on the predefined synchronization module, and there is no need to rewrite the database source code, and it can be upgraded along with the open source database version; the key to be synchronized will not be written into the target database in the implementation logic of the synchronous playback command, thereby avoiding the circular synchronization problem during two-way synchronization; by setting a globally incremented version number and comparing the version numbers of the key to be synchronized in the source database and the target database, it is possible to conveniently know whether the update time of the key to be synchronized in the source database is earlier or later than that in the target database, so as to synchronize the data to be synchronized in the source database to the target database; when defining the synchronization module, the synchronous write command, the synchronous read command and the synchronous playback command are realized, and the data synchronization can be completed by calling these three commands subsequently, which is convenient for batch data synchronization and playback, and the development and maintenance cost is low; the synchronization command is obtained by adding a prefix to the native command, which is simple to implement and easy to develop and maintain.

[0068] Figure 4 Schematic diagram of the main modules of the data synchronization system according to an embodiment of the present invention. Figure 4As shown, the data synchronization system 400 of the embodiment of the present invention mainly includes:

[0069] The loading and saving module 401 is used to receive a synchronous write command from the client, load a predefined synchronization module, and save the current key in the synchronous write command and the first version information used to represent the update time of the current key in the source database to the source database according to the implementation logic of the synchronous write command in the synchronization module. The synchronization module is pre-implemented, and the module includes the implementation logic of the synchronous write command, the implementation logic of the synchronous read command, and the implementation logic of the synchronous playback command. Among them, the synchronous write command is used to write the current key and the first version information corresponding to the current key into the source database according to the set data type. The synchronous read command is used to uniformly obtain the key to be synchronized, the corresponding key value and the first version information. The synchronous playback command is used to obtain the second version information of the key to be synchronized in the target database, and compare it with the first version information, and synchronize data according to the comparison result.

[0070] The client corresponding to the source database adds the set prefix to the native write command to obtain the synchronous write command, and then sends the synchronous write command to the source database. After receiving the synchronous write command from the corresponding client, the source database loads the synchronization module to support the synchronous write command, synchronous read command and synchronous playback command. After the source database has loaded the synchronization module, it saves the current key and the corresponding first version information in the synchronous write command according to the implementation logic of the synchronous write command in the synchronization module.

[0071] The acquisition module 402 is called to call the synchronous read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the key to be synchronized of the data to be synchronized from the source database; wherein the key to be synchronized includes at least one of the current keys. In an embodiment, the module is implemented in a synchronous extraction agent node. The data to be synchronized includes the key to be synchronized and the key value corresponding to the key to be synchronized. By calling the synchronous read command, a specified number of keys to be synchronized, the key values ​​corresponding to the keys to be synchronized and the first version information are obtained from the source database.

[0072] Call the write module 403, which is used to call the synchronous playback command of the synchronization module to obtain the second version information used to represent the update time of the key to be synchronized in the target database. If the update time represented by the first version information of the key to be synchronized is later than the update time represented by the second version information, the data to be synchronized is written into the target database. In an embodiment, this module is implemented in a synchronous playback proxy node. The first version information and the second version information are globally incremented version numbers. For example, the first version information of the key to be synchronized is 4.0, and the second version information of the key to be synchronized is 3.0, indicating that the update time of the key to be synchronized in the source database is later than the update time in the target database. By calling the synchronous playback command, the second version information of the key to be synchronized in the target database is obtained, and then the size of the first version information and the second version information is compared. If the first version information is larger than the second version information, the native write command is called to write the data to be synchronized into the target database.

[0073] In addition, the data synchronization system 400 of the embodiment of the present invention may also include: a comparison determination module and a synchronization implementation module ( Figure 4 (not shown). The comparison determination module is used to compare the size of the first version information and the second version information to determine the update time of the key to be synchronized in the source database and the target database. Synchronization implementation module: custom implementation of the synchronization module; wherein the synchronization module includes the implementation logic of the synchronization write command, the implementation logic of the synchronization read command and the implementation logic of the synchronization playback command.

[0074] From the above description, it can be seen that the data synchronization of the database is realized based on the predefined synchronization module, and there is no need to rewrite the database source code, and it can be upgraded along with the open source database version; the implementation logic of the synchronous playback command will not write the key to be synchronized to the target database, avoiding the circular synchronization problem during two-way synchronization; by setting the globally incremented version number and comparing the version number of the key to be synchronized in the source database and the target database, it is convenient to know the update time of the key to be synchronized in the source database compared with that in the target database, so as to synchronize the data to be synchronized in the source database to the target database; when defining the synchronization module, the synchronous write command, synchronous read command and synchronous playback command are implemented, and the data synchronization can be completed by calling these three commands subsequently, which is convenient for batch data synchronization and playback, and the development and maintenance cost is low; the synchronization command is obtained by adding a prefix to the native command, which is simple to implement and easy to develop and maintain.

[0075] Figure 5 An exemplary system architecture 500 is shown to which a data synchronization method or a data synchronization system according to an embodiment of the present invention can be applied.

[0076] like Figure 5As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0077] The user can use the terminal devices 501, 502, 503 to interact with the server 505 through the network 504 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 501, 502, 503.

[0078] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0079] The server 505 may be a server that provides various services, such as a background management server that processes the synchronous write command sent by the user using the terminal devices 501, 502, and 503. After receiving the synchronous write command, the background management server may load the synchronization module, call the command of the synchronization module to perform data synchronization and other processing, and feed back the processing result (such as the data synchronization result) to the terminal device.

[0080] It should be noted that the data synchronization method provided in the embodiment of the present application is generally executed by the server 505 , and accordingly, the data synchronization system is generally set in the server 505 .

[0081] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0082] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0083] The electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a data synchronization method of an embodiment of the present invention.

[0084] The computer-readable medium of the present invention stores a computer program, and when the program is executed by a processor, a data synchronization method of an embodiment of the present invention is implemented.

[0085] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing an electronic device of an embodiment of the present invention. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0086] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0087] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0088] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the main step diagram. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0089] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0090] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, may be described as: a processor includes a loading and saving module, a calling and acquiring module, and a calling and writing module. The names of these modules do not constitute limitations on the modules themselves in certain cases. For example, the loading and saving module may also be described as "receiving a synchronous write command from a client, loading a predefined synchronous module, and saving the current key in the synchronous write command and the first version information indicating the update time of the current key in the source database to the module of the source database according to the implementation logic of the synchronous write command in the synchronous module".

[0092] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: receiving a synchronous write command from a client, loading a predefined synchronization module, and saving the current key in the synchronous write command and the first version information used to indicate the update time of the current key in the source database to the source database according to the implementation logic of the synchronous write command in the synchronization module; calling the synchronous read command of the synchronization module to obtain the data to be synchronized from the source database, and the first version information corresponding to the key to be synchronized of the data to be synchronized; wherein the key to be synchronized includes at least one current key; calling the synchronous playback command of the synchronization module to obtain the second version information used to indicate the update time of the key to be synchronized in the target database, and if the update time indicated by the first version information of the key to be synchronized is later than the update time indicated by the second version information, the data to be synchronized is written into the target database.

[0093] From the above description, it can be seen that the data synchronization of the database is realized based on the predefined synchronization module, and there is no need to rewrite the database source code, and it can be upgraded along with the open source database version; the implementation logic of the synchronous playback command will not write the key to be synchronized to the target database, avoiding the circular synchronization problem during two-way synchronization; by setting the globally incremented version number and comparing the version number of the key to be synchronized in the source database and the target database, it is convenient to know the update time of the key to be synchronized in the source database compared with that in the target database, so as to synchronize the data to be synchronized in the source database to the target database; when defining the synchronization module, the synchronous write command, synchronous read command and synchronous playback command are implemented, and the data synchronization can be completed by calling these three commands subsequently, which is convenient for batch data synchronization and playback, and the development and maintenance cost is low; the synchronization command is obtained by adding a prefix to the native command, which is simple to implement and easy to develop and maintain.

[0094] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0095] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data synchronization method, characterized in that: include: Receive a synchronous write command from a client, load a predefined synchronization module, and save the current key in the synchronous write command and the first version information indicating the update time of the current key in the source database to the source database according to the implementation logic of the synchronous write command in the synchronization module; wherein the synchronous write command is used to write the current key and the first version information corresponding to the current key into the source database according to the set data type; Calling the synchronization read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the synchronization key of the data to be synchronized from the source database; wherein the synchronization key includes at least one of the current keys; Call the synchronous playback command of the synchronization module to obtain the second version information used to represent the update time of the key to be synchronized in the target database; if the update time represented by the first version information of the key to be synchronized is later than the update time represented by the second version information, write the data to be synchronized into the target database.

2. The method according to claim 1, characterized in that: The first version information and the second version information are globally incremented version numbers; After the step of obtaining the second version information indicating the update time of the key to be synchronized in the target database, the method further includes: The sizes of the first version information and the second version information are compared to determine whether the to-be-synchronized key is updated earlier or later in the source database than in the target database.

3. The method according to claim 1, characterized in that The method further comprises: The synchronization module is implemented by customizing the synchronization module, wherein the synchronization module includes the implementation logic of the synchronization write command, the implementation logic of the synchronization read command and the implementation logic of the synchronization playback command.

4. The method according to claim 3, characterized in that The implementation logic of the synchronous write command is: Invoking a native write command corresponding to the data type of the current key to write the key value corresponding to the current key into the source database; According to the set data type, the current key and the first version information of the current key are written into the source database.

5. The method according to claim 3, characterized in that: The implementation logic of the synchronous read command is: respectively calling native read commands corresponding to the data types of the keys to be synchronized, the key values ​​corresponding to the keys to be synchronized, and the first version information, so as to obtain a specified number of the keys to be synchronized, the key values ​​corresponding to the keys to be synchronized, and the first version information from the source database; The key to be synchronized, the key value corresponding to the key to be synchronized, and the first version information are returned.

6. The method according to claim 1, characterized in that The synchronous write command of the client is obtained by adding a set prefix to the native write command.

7. The method according to any one of claims 1 to 6, characterized in that: The source database and the target database are both redis databases, and the synchronous write command, the synchronous read command and the synchronous playback command are implemented in the module module of the redis database.

8. A data synchronization system, characterized in that: include: A loading and saving module is used to receive a synchronous write command from a client, load a predefined synchronous module, and save the current key in the synchronous write command and the first version information indicating the update time of the current key in the source database to the source database according to the implementation logic of the synchronous write command in the synchronous module; wherein the synchronous write command is used to write the current key and the first version information corresponding to the current key into the source database according to the set data type; A calling acquisition module is used to call the synchronization read command of the synchronization module to obtain the data to be synchronized and the first version information corresponding to the synchronization key of the data to be synchronized from the source database; wherein the synchronization key includes at least one current key; A write module is called to call the synchronous playback command of the synchronization module to obtain the second version information used to represent the update time of the key to be synchronized in the target database; if the update time represented by the first version information of the key to be synchronized is later than the update time represented by the second version information, the data to be synchronized is written into the target database.

9. The system according to claim 8, characterized in that The first version information and the second version information are globally incremented version numbers; the system further comprises: a comparison and determination module for The sizes of the first version information and the second version information are compared to determine whether the to-be-synchronized key is updated earlier or later in the source database than in the target database.

10. The system according to claim 8, characterized in that The system also includes: a synchronization implementation module: The synchronization module is implemented by customizing the synchronization module, wherein the synchronization module includes the implementation logic of the synchronization write command, the implementation logic of the synchronization read command and the implementation logic of the synchronization playback command.

11. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images