Data synchronization method and system

Abstract

The invention provides a data synchronization method and system, and the method comprises the steps: a data synchronization service monitors database data to obtain target data, and pushes the target data to a synchronization client; the synchronization client filters the target data, determines a synchronization vernier of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization vernier; if the target data is not the outdated data, the synchronization client inserts an SQL statement for the target data; and the synchronization client calls an interface for the synchronization server through the http request, and sends the target data encrypted through the private key to the synchronization server, so that the synchronization server performs data synchronization based on the target data. In the mode, the database can be prevented from being repeatedly and continuously modified, so that the data consistency of the cloud end and the local end is realized, and the finally reserved data must be updated at the latest time.

Description

Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a data synchronization method and system. Background Art

[0002] At present, for bonded supervision parks, operation and maintenance services can be deployed on a server in each bonded supervision park, which is in the local area network and the data is also stored on the server; or, the operation and maintenance service cloud platform can also be deployed on the cloud server, and the data is stored on the cloud server. Operation and maintenance personnel can be stationed on site in the park, and one operation and maintenance personnel can also be responsible for multiple parks. If the operation and maintenance personnel are in the park, they can open the equipment operation and maintenance service in the same local area network to process the operation and maintenance work order; if the operation and maintenance personnel are not in the park, they cannot directly access the equipment operation and maintenance service in the park, and need to process the work orders of different parks through the cloud platform.

[0003] The same data is stored both locally and in the cloud server, which involves the issue of data synchronization, which requires synchronization in both directions. In one case, the operation and maintenance personnel process the work order through the cloud service, then the cloud service data is modified first, and then the cloud server synchronizes the data to a certain park, and the local data in the park is subsequently modified; in another case, the operation and maintenance personnel process the work order through the equipment operation and maintenance service in the park, first modify the local data in the park, and then synchronize it to the cloud server, and the cloud server data is modified accordingly, so that the local data and the cloud server data are consistent.

[0004] However, since the modification will trigger synchronization, the database of the existing data synchronization solution will be modified repeatedly, which consumes a lot of resources. In extreme cases, the cycle will continue forever, resulting in synchronization errors and waste of resources. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a data synchronization method and system to determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor, so as to prevent the database from being modified repeatedly, thereby achieving data consistency between the cloud and the local, and the data that is ultimately retained must be the data with the latest update time.

[0006] In the first aspect, an embodiment of the present invention provides a data synchronization method, which is applied to a local server, wherein the local server is deployed with a data synchronization service and a synchronization client, and the database of the local server stores a synchronization schedule; the method includes: the data synchronization service monitors the database data of the local server to obtain target data, and pushes the target data to the synchronization client; the synchronization client filters the target data, determines the synchronization cursor of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization cursor; if the target data is not outdated data, the synchronization client inserts an SQL statement for the target data; wherein the SQL statement indicates that when inserting data, if there is no data with the same primary key in the table, new data is added, and if there is data with the same primary key, the data is modified to a new value; the synchronization client calls an interface to the synchronization server of the cloud server through an http request, and sends the target data encrypted by a private key to the synchronization server, so that the synchronization server synchronizes data based on the target data.

[0007] In an optional embodiment of the present invention, the above-mentioned step of determining whether the target data is outdated data based on the update time of the target data and the synchronization cursor includes: if the update time of the target data is earlier than the synchronization cursor of the target data, determining that the target data is outdated data; if the update time of the target data is not earlier than the synchronization cursor of the target data, determining that the target data is not outdated data.

[0008] In an optional embodiment of the present invention, the steps for the synchronization server to perform data synchronization based on the target data include: the synchronization server receives an http request, decrypts the target data through a public key to obtain an SQL statement; the synchronization server executes the SQL statement in the database to perform data synchronization, and returns information on successful data synchronization to the synchronization client; the synchronization client updates the synchronization cursor of the target data in the synchronization schedule based on the information on successful data synchronization.

[0009] In an optional embodiment of the present invention, the step of the synchronization client calling the interface of the synchronization server of the cloud server through an http request includes: the local server establishes communication with the cloud server based on the public network; the synchronization client calls the interface of the synchronization server of the cloud server through an http request based on the public network.

[0010] In a second aspect, an embodiment of the present invention further provides a data synchronization method, which is applied to a cloud server, wherein the cloud server is deployed with a data synchronization service and a synchronization client, and the database of the cloud server stores a synchronization schedule; the method comprises: the data synchronization service monitors the database data of the cloud server to obtain the target data, and pushes the target data to the synchronization client; the synchronization client filters the target data, determines the synchronization cursor of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization cursor; if the target data is not outdated data, the synchronization client inserts an SQL statement for the target data; wherein the SQL statement indicates that when inserting data, if there is no data with the same primary key in the table, new data is added, and if there is data with the same primary key, the data is modified to a new value; the synchronization client calls an interface to the synchronization server of the local server through an http request, and sends the target data encrypted by a private key to the synchronization server of the local server, so that the synchronization server of the local server performs data synchronization based on the target data.

[0011] In an optional embodiment of the present invention, the step of the synchronization client calling the interface of the synchronization server of the local server through an http request includes: the cloud server establishes communication with at least one local server based on intranet penetration; the synchronization client determines the target local server corresponding to the target data based on the synchronization schedule; the synchronization client calls the interface of the synchronization server of the target local server through an http request based on intranet penetration.

[0012] In an optional embodiment of the present invention, the above-mentioned synchronization progress table stores the regional ID of each local server; the step of the synchronization client determining the target local server corresponding to the target data based on the synchronization progress table includes: the synchronization client determines the target regional ID corresponding to the target data based on the synchronization progress table; the synchronization client determines the target local server corresponding to the target regional ID.

[0013] In a third aspect, an embodiment of the present invention further provides a data synchronization system, the data synchronization system comprising: a local server and a cloud server; the local server is used to execute the above-mentioned data synchronization method; the cloud server is used to execute the above-mentioned data synchronization method.

[0014] In an optional embodiment of the present invention, the database of the above-mentioned local server and the database of the cloud server both store a synchronization schedule; the synchronization schedule stored in the database of the local server includes: database name, table name and synchronization cursor; the synchronization schedule stored in the database of the cloud server includes: database name, table name, synchronization cursor and area id.

[0015] In an optional embodiment of the present invention, the target data to be synchronized includes: an update time field, an area id field, and a logical deletion field; the update time field does not automatically update the timestamp as the data is modified; the target data uses the original primary key and the area id as a joint primary key, and the target data has no other unique index except the primary key.

[0016] The embodiments of the present invention bring the following beneficial effects: The embodiments of the present invention provide a data synchronization method and system, which can determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor, prevent the database from being modified repeatedly, thereby achieving data consistency between the cloud and the local, and the data that is ultimately retained must be the data with the latest update time.

[0017] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0018] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A structural diagram of a data synchronization system provided by an embodiment of the present invention; Figure 2 A schematic diagram of a data synchronization system provided by an embodiment of the present invention; Figure 3 A flowchart of a data synchronization method provided by an embodiment of the present invention; Figure 4 A flowchart of a data synchronization method provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] At present, for bonded supervision parks, operation and maintenance services can be deployed on a server in each bonded supervision park, which is in the local area network and the data is also stored on the server; or, the operation and maintenance service cloud platform can also be deployed on the cloud server, and the data is stored on the cloud server. Operation and maintenance personnel can be stationed on site in the park, and one operation and maintenance personnel can also be responsible for multiple parks. If the operation and maintenance personnel are in the park, they can open the equipment operation and maintenance service in the same local area network to process the operation and maintenance work order; if the operation and maintenance personnel are not in the park, they cannot directly access the equipment operation and maintenance service in the park, and need to process the work orders of different parks through the cloud platform.

[0023] The same data is stored both locally and in the cloud server, which involves the issue of data synchronization, which requires synchronization in both directions. In one case, the operation and maintenance personnel process the work order through the cloud service, then the cloud service data is modified first, and then the cloud server synchronizes the data to a certain park, and the local data in the park is subsequently modified; in another case, the operation and maintenance personnel process the work order through the equipment operation and maintenance service in the park, first modify the local data in the park, and then synchronize it to the cloud server, and the cloud server data is modified accordingly, so that the local data and the cloud server data are consistent.

[0024] However, since the modification will trigger synchronization, the database of the existing data synchronization solution will be modified repeatedly, which consumes a lot of resources. In extreme cases, the cycle will continue forever, resulting in synchronization errors and waste of resources.

[0025] Based on this, a data synchronization method and system provided by an embodiment of the present invention specifically provides a method for synchronizing cloud and local data based on MySQL (a relational database management system). It can determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor, thereby preventing the database from being repeatedly modified, thereby achieving consistency in cloud and local data, and the data that is ultimately retained must be the data with the latest update time.

[0026] To facilitate understanding of this embodiment, a data synchronization system disclosed in an embodiment of the present invention is first introduced in detail.

[0027] Embodiment 1: The embodiment of the present invention provides a data synchronization system, see Figure 1A structural schematic diagram of a data synchronization system is shown, which includes: a local server and a cloud server; a data synchronization method executed by the local server; and a data synchronization method executed by the cloud server.

[0028] Among them, the data synchronization method executed by the local server is that the local server synchronizes data with the cloud server, and the data synchronization method executed by the cloud server is that the cloud server synchronizes data with the local server.

[0029] In some embodiments, the database of the local server and the database of the cloud server both store a synchronization schedule; the synchronization schedule stored in the database of the local server includes: database name, table name and synchronization cursor; the synchronization schedule stored in the database of the cloud server includes: database name, table name, synchronization cursor and area id.

[0030] The database of the local server can be referred to as the local database. The local database maintains a synchronization schedule table, which stores the database name, table name, and synchronization cursor. The database of the cloud server also maintains a synchronization schedule table, which stores the database name, table name, synchronization cursor, and region ID. The region ID can also be called the campus ID.

[0031] In some embodiments, the target data to be synchronized includes: an update time field, an area id field, and a logical deletion field; the update time field does not automatically update the timestamp as the data is modified; the target data uses the original primary key and the area id as a joint primary key, and the target data has no other unique index except the primary key.

[0032] The target data to be synchronized can be set with: update time field update_time, zone id field zone_id, logical deletion field del_flag. Among them, the automatic update of timestamp can be turned off. If it is not turned off, it will cause cyclic update problems. When there is a logical deletion field, the deleted data will not be truly deleted, but soft deleted. By changing del_flag to 1, it is indicated that the data is deleted. When querying, add del_flag is not 1, so that the deleted data will not be queried. In this embodiment, the original primary key and zone id can also be used as a joint primary key, and the target data does not have other unique indexes except the primary key.

[0033] In this embodiment, each park can be equipped with a local server in the intranet, each park can have a pair of RSA (Rivest-Shamir-Adleman, an asymmetric encryption algorithm) public key and private key, and the cloud server also has a pair of RSA public key and private key; the cloud server has a public fixed IP (Internet Protocol), so the local server can easily establish communication with the cloud server.

[0034] However, the local server does not have a fixed public IP address, and the cloud server cannot communicate with the local server. Therefore, this embodiment can open up the communication between the cloud server and the local server through intranet penetration. The intranet penetration server is deployed on the cloud server, and the intranet penetration client is deployed on the local server. The client configuration address points to the cloud server's intranet penetration server to establish an intranet penetration tunnel. The port where the local server's synchronization server is located can be mapped to a port on the cloud server. The cloud server can access this port, which is equivalent to accessing the synchronization server of the local server.

[0035] Embodiment 2: The embodiment of the present invention provides a data synchronization method, which is implemented on the basis of the above embodiment and can be applied to the local server provided by the above embodiment, see Figure 2 A schematic diagram of a data synchronization system is shown, in which a local server is deployed with a data synchronization service (canal-server service) and a synchronization client, and a database of the local server stores a synchronization schedule.

[0036] Based on the above description, see Figure 3 A flow chart of a data synchronization method is shown, the data synchronization method comprises the following steps: Step S302: The data synchronization service monitors the database data of the local server to obtain target data, and pushes the target data to the synchronization client.

[0037] like Figure 2 As shown, in this embodiment, the canal-server service of the local server can monitor the database data changes of the local server to obtain the target data, and push all the information of the target data to the synchronization client of the local server.

[0038] Step S304: the synchronization client filters the target data, determines the synchronization cursor of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization cursor.

[0039] In this embodiment, when the synchronization client of the local server receives the target data, it can first filter the target data through a table to filter out some data in the target data that does not need to be synchronized, and then find the synchronization cursor of the table where the target data is located in the synchronization progress table, and determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor.

[0040] In some embodiments, if the update time of the target data is earlier than the synchronization cursor of the target data, the target data is determined to be outdated data; if the update time of the target data is not earlier than the synchronization cursor of the target data, the target data is determined not to be outdated data.

[0041] If the update time of the target data is earlier than the synchronization cursor of the target data, it is determined that the target data is outdated data, so it does not need to be synchronized to the cloud server; if the update time of the target data is not earlier than the synchronization cursor of the target data, it is determined that the target data is not outdated data, so it needs to be synchronized to the cloud server.

[0042] Step S306, if the target data is not outdated data, the synchronization client inserts an SQL statement for the target data; wherein the SQL statement indicates that when inserting data, if there is no data with the same primary key in the table, new data is added; if there is data with the same primary key, the data is modified to a new value.

[0043] If the target data is not outdated, the synchronization client of the local server can insert SQL (Structured Query Language) statements for the target data, for example, manually assemble a structured language (i.e. SQL statement) of insert into XXX on duplicate key update XXX for the target data. The function of the SQL statement is: when inserting data, if there is no data with the same primary key in the table, then add the data; if there is data with the same primary key, then modify the data to the new value.

[0044] Step S308, the synchronization client calls the interface of the synchronization server of the cloud server through an http request, and sends the target data encrypted by the private key to the synchronization server, so that the synchronization server performs data synchronization based on the target data.

[0045] In some embodiments, the local server establishes communication with the cloud server based on the public network; the synchronization client calls the interface of the synchronization server of the cloud server through http request based on the public network.

[0046] like Figure 2 As shown, the synchronization client of the local server can call the interface of the synchronization server of the cloud server through http request based on the public network.

[0047] In some embodiments, the synchronization server receives an http request, decrypts the target data using a public key, and obtains an SQL statement; the synchronization server executes the SQL statement in the database to synchronize data, and returns information about successful data synchronization to the synchronization client; the synchronization client updates the synchronization cursor of the target data in the synchronization schedule based on the information about successful data synchronization.

[0048] like Figure 2 As shown, the synchronization client of the local server calls the interface of the synchronization server of the cloud server through an http request, and the parameters of the target data are encrypted by the park private key to prevent information leakage.

[0049] like Figure 2 As shown, after receiving the http request, the synchronization server of the cloud server decrypts the target data through the public key of the park to obtain the SQL statement. Among them, the cloud server stores the public keys of all parks, its own public key and private key; the local server stores the public key of the cloud server, its own public key and private key.

[0050] like Figure 2 As shown, the synchronization server of the cloud server executes the SQL statement in its own database to synchronize data, and returns information that the data synchronization is successful to the synchronization client of the local server; the synchronization client of the local server updates the synchronization cursor of the target data in the synchronization schedule based on the information that the data synchronization is successful, and modifies the time of the synchronization cursor to the update_time update time of the target data.

[0051] An embodiment of the present invention provides a data synchronization method, which can realize data synchronization from a local server to a cloud server, determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor, prevent the database from being repeatedly modified, thereby achieving data consistency between the cloud and the local, and the data that is ultimately retained must be the data with the latest update time.

[0052] Embodiment three: The embodiment of the present invention provides another data synchronization method, which is implemented on the basis of the above embodiment and can be applied to the cloud server provided by the above embodiment, such as Figure 2 As shown, the cloud server is deployed with a data synchronization service (canal-server service) and a synchronization client, and the database of the cloud server stores a synchronization schedule.

[0053] Based on the above description, see Figure 4 A flow chart of a data synchronization method is shown, the data synchronization method comprises the following steps: Step S402: The data synchronization service monitors the database data of the local server to obtain target data, and pushes the target data to the synchronization client.

[0054] like Figure 2 As shown, in this embodiment, the canal-server service of the cloud server can monitor the database data changes of the cloud server to obtain the target data, and push all the information of the target data to the synchronization client of the cloud server.

[0055] Step S404: the synchronization client filters the target data, determines the synchronization cursor of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization cursor.

[0056] In this embodiment, when the synchronization client of the cloud server receives the target data, it can first filter the target data through a table to filter out some data in the target data that does not need to be synchronized, and then find the synchronization cursor of the table where the target data is located in the synchronization progress table, and determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor.

[0057] If the update time of the target data is earlier than the synchronization cursor of the target data, it is determined that the target data is outdated data, so it does not need to be synchronized to the local server; if the update time of the target data is not earlier than the synchronization cursor of the target data, it is determined that the target data is not outdated data, so it needs to be synchronized to the local server.

[0058] Among them, if the update time is later than the synchronization cursor of the target data, the target data has not been synchronized and needs to be synchronized to the local server; if the update time is equal to the synchronization cursor of the target data, the target data may not have been synchronized or may have been synchronized, and this embodiment will still perform synchronization operations on the target data.

[0059] It should be noted here that if synchronization is performed only when the update time is later than the synchronization cursor of the target data, and synchronization is not performed when the update time is equal to the synchronization cursor of the target data, then if someone happens to modify the target data at the same time, the changes to the target data will not be synchronized.

[0060] Therefore, in this embodiment, for the case where the update time of the synchronization cursor is equal to the target data, the target data will still be synchronized, but such operation will not cause cyclic synchronization. The principle is: relying on the characteristics of MySQL, if the data you want to modify is consistent with the original data, MySQL will return no modification, so the canal-server service will not monitor the data changes, thereby interrupting the cycle, that is, no cyclic synchronization will be caused.

[0061] Step S406, if the target data is not outdated data, the synchronization client inserts an SQL statement for the target data; wherein the SQL statement indicates that when inserting data, if there is no data with the same primary key in the table, new data is added; if there is data with the same primary key, the data is modified to a new value.

[0062] If the target data is not outdated, the synchronization client of the cloud server can insert SQL statements for the target data, for example, manually splicing an SQL statement of insert into XXX on duplicate key update XXX for the target data. The function of the SQL statement is: when inserting data, if there is no data with the same primary key in the table, add the data; if there is data with the same primary key, modify the data to the new value.

[0063] Step S408, the synchronization client calls the interface of the synchronization server of the local server through an http request, and sends the target data encrypted by the private key to the synchronization server, so that the synchronization server performs data synchronization based on the target data.

[0064] In some embodiments, the cloud server establishes communication with at least one local server based on intranet penetration; the synchronization client determines the target local server corresponding to the target data based on the synchronization schedule; the synchronization client calls the synchronization server interface of the target local server through http request based on intranet penetration.

[0065] like Figure 2 As shown, the synchronization client of the cloud server can call the interface of the synchronization server of the local server through HTTP request based on the intranet penetration method.

[0066] In some embodiments, the synchronization client determines a target area ID corresponding to the target data based on the synchronization schedule; the synchronization client determines a target local server corresponding to the target area ID.

[0067] In this embodiment, by penetrating the intranet, the synchronization server of each park will map a port on the cloud server. Through the zone_id (i.e., the target zone id) of the target data itself, it can be known to which park's local server (i.e., the target local server) the target data should be synchronized to, and thus which port to access.

[0068] like Figure 2 As shown, the synchronization client of the cloud server calls the synchronization server interface of the local server through an http request, and the parameters of the target data are encrypted by the cloud server private key to prevent information leakage.

[0069] like Figure 2 As shown in the figure, after receiving the http request, the synchronization server of the local server decrypts the target data through the public key of the cloud server to obtain the SQL statement. Among them, the cloud server stores the public keys of all parks, its own public key and private key; the local server stores the public key of the cloud server, its own public key and private key.

[0070] like Figure 2 As shown, the synchronization server of the local server executes the SQL statement in its own database to synchronize data, and returns information that the data synchronization is successful to the synchronization client of the cloud server; the synchronization client of the cloud server updates the synchronization cursor of the target data in the synchronization schedule based on the information that the data synchronization is successful, and modifies the time of the synchronization cursor to the update_time update time of the target data.

[0071] An embodiment of the present invention provides another data synchronization method, which can realize data synchronization from a cloud server to a local server, determine whether the target data is outdated data based on the update time of the target data and the synchronization cursor, prevent the database from being repeatedly modified, thereby achieving data consistency between the cloud and the local, and the data that is ultimately retained must be the data with the latest update time.

[0072] The above data synchronization method provided by the embodiment of the present invention can be set with a synchronization cursor. For the update time lower than the cursor, the synchronization will be skipped, and the update time equal to or greater than the cursor time will continue to be synchronized: Assuming that the cloud and the local have modified the data in a short period of time, due to the uncertainty of the time consumption of the entire synchronization process and the existence of network fluctuations, the following situations may exist, which are steps 1-8 in the order of the timeline: Step 1: Modify the cloud data to a in the cloud; Step 2: Locally modify the local data to b; Step 3: The data synchronized from the cloud to the local computer is a, and the cloud cursor time is a; Step 4: The local data synchronized to the cloud is b, and the local cursor time is b; Step 5: The local computer receives a request to modify the cloud data to a, executes SQL, and modifies the local database to a; Step 6: The cloud receives the request to change the local data to b, executes SQL, and changes the cloud database to b; Step 7: Since the local database changes from b to a, the local data synchronized to the cloud is changed to a; Step 8: Since the cloud database changes from a to b, the cloud synchronizes data to the local database and changes it to b.

[0073] It can be seen that since synchronization will be triggered after the modification, the local database will be repeatedly modified between a and b, which consumes a lot of resources and will continue in the cycle forever in extreme cases.

[0074] In the embodiment of the present invention, the problem can be solved by the cursor time. Step 2 changes the cursor time to b. In step 7, since the update time of a is earlier than the cursor time b, the local data will not continue to be synchronized to the cloud. At this time, the local server data is a; after step 8, the local server data is b, and the cloud server data is also b. At this time, there is step 9, and the local server synchronizes data b to the cloud (because the cursor time is b and the update time is also b at this time. If the update time is equal to or greater than the cursor time, synchronization will continue). At this time, the cloud database is b, and SQL is executed to change the data to b.

[0075] Due to the characteristics of MySQL (if the original data is consistent with the data to be modified, nothing happens), the result of executing SQL is that no data has changed, so the canal-server will not be triggered to monitor data changes, thereby interrupting the link, and ultimately achieving data consistency between the cloud and local, and the data that is finally retained must be the data with the latest update time.

[0076] Setting the timestamp to update automatically will fill the update_time with the time when the modification SQL is executed each time a certain data is modified. It will cause the update time to be the latest dynamic time after the SQL is executed during synchronization, which will always be greater than the cursor time, and will also cause the synchronization logic to loop. Therefore, the above data synchronization method provided by the embodiment of the present invention can turn off the automatic update of the timestamp in the update_time field, thereby preventing the synchronization logic from looping and achieving data consistency between the cloud and the local.

[0077] The above data synchronization method provided by the embodiment of the present invention can be set with zone_id. Through zone_id, it can be clearly determined to which park's local server the target data will be synchronized, and from which park's local server the target data is synchronized. The reason why the embodiment of the present invention uses the original primary key and zone_id as the joint primary key is that the primary key is unique only in one table. For the same table in multiple parks, the primary key may be repeated, but adding zone_id as the joint primary key will definitely make it non-repeated.

[0078] In the above-mentioned data synchronization system provided by the embodiment of the present invention, the canal-server service can be deployed on a local server and a cloud server to monitor the respective MySQL data changes and push them to the respective synchronization clients.

[0079] The above-mentioned data synchronization system provided by the embodiment of the present invention, the synchronization client can be deployed on the local server and the cloud server, and is used to detect whether the synchronization server of the other party is online, accept the push of the canal-server, splice it into SQL after business processing, and complete the data synchronization through RSA encryption signature, call the synchronization server interface of the other end, maintain the synchronization schedule, and the data with update time lower than the synchronization schedule will be discarded.

[0080] The above-mentioned data synchronization system provided by the embodiment of the present invention, the synchronization server can be deployed on the local server and the cloud server, store the necessary public key and private key information, and after receiving the request, it will undergo RSA decryption and signature verification to confirm that the information has not been tampered with. After the source is correct, the SQL statement is executed to complete the synchronization and achieve the final consistency of the data on both ends.

[0081] In the above-mentioned data synchronization system provided by the embodiment of the present invention, the intranet penetration client can be deployed on a local server, and is used for the local server to connect to the cloud intranet penetration server and establish a VPN tunnel, so that when the cloud synchronizes data to the park, it can access the park's synchronization server.

[0082] In the above-mentioned data synchronization system provided by the embodiment of the present invention, the intranet penetration server can be deployed on a cloud server to establish an intranet penetration connection.

[0083] Embodiment 4: The embodiment of the present invention also provides an electronic device for executing the above data synchronization method; see Figure 5 A structural diagram of an electronic device is shown, the electronic device includes a memory 100 and a processor 101, wherein the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned data synchronization method.

[0084] Further, Figure 5 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .

[0085] The memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0086] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and completes the steps of the method of the above embodiment in combination with its hardware.

[0087] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned data synchronization method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0088] The computer program product of the data synchronization method and system provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the previous method embodiments. The specific implementation can be found in the method embodiments and will not be repeated here.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0090] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0091] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0092] In the description of the present invention, "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set composed of A, B, and C. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0093] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A data synchronization method, characterized in that: Applied to a local server, the local server is deployed with a data synchronization service and a synchronization client, and the database of the local server stores a synchronization schedule; the method includes: The data synchronization service monitors the database data of the local server to obtain target data, and pushes the target data to the synchronization client; The synchronization client filters the target data, determines the synchronization cursor of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization cursor; If the target data is not outdated data, the synchronization client inserts an SQL statement for the target data; wherein the SQL statement indicates that when inserting data, if there is no data with the same primary key in the table, the data is added, and if there is data with the same primary key, the data is modified to a new value; The synchronization client calls the interface of the synchronization server of the cloud server through an http request, and sends the target data encrypted by the private key to the synchronization server, so that the synchronization server performs data synchronization based on the target data.

2. The method according to claim 1, characterized in that The step of determining whether the target data is outdated data based on the update time of the target data and the synchronization cursor comprises: If the update time of the target data is earlier than the synchronization cursor of the target data, determining that the target data is outdated data; If the update time of the target data is not earlier than the synchronization cursor of the target data, it is determined that the target data is not outdated data.

3. The method according to claim 1, characterized in that The step of performing data synchronization on the synchronization server based on the target data includes: The synchronization server receives the http request and decrypts the target data using the public key to obtain the SQL statement; The synchronization server executes the SQL statement in the database to synchronize data, and returns information that the data synchronization is successful to the synchronization client; The synchronization client updates the synchronization cursor of the target data in the synchronization progress table based on the information of successful data synchronization.

4. The method according to claim 1, characterized in that: The step of the synchronization client calling the interface of the synchronization server of the cloud server through an http request includes: The local server establishes communication with the cloud server based on the public network; The synchronization client calls an interface of the synchronization server of the cloud server through an http request based on the public network.

5. A data synchronization method, characterized in that: Applied to a cloud server, the cloud server is deployed with a data synchronization service and a synchronization client, and the database of the cloud server stores a synchronization schedule; the method comprises: The data synchronization service monitors the database data of the cloud server to obtain target data, and pushes the target data to the synchronization client; The synchronization client filters the target data, determines the synchronization cursor of the target data from the synchronization schedule, and determines whether the target data is outdated data based on the update time of the target data and the synchronization cursor; If the target data is not outdated data, the synchronization client inserts an SQL statement for the target data; wherein the SQL statement indicates that when inserting data, if there is no data with the same primary key in the table, the data is added, and if there is data with the same primary key, the data is modified to a new value; The synchronization client calls the interface of the synchronization server of the local server through an http request, and sends the target data encrypted by the private key to the synchronization server of the local server, so that the synchronization server of the local server performs data synchronization based on the target data.

6. The method according to claim 5, characterized in that The step of the synchronization client calling the interface of the synchronization server of the local server through an http request includes: The cloud server establishes communication with at least one local server based on intranet penetration; The synchronization client determines the target local server corresponding to the target data based on the synchronization schedule; The synchronization client calls the interface of the synchronization server of the target local server through an http request based on the intranet penetration method.

7. The method according to claim 6, characterized in that The synchronization progress table stores the region IDs of the local servers; and the step of the synchronization client determining the target local server corresponding to the target data based on the synchronization progress table includes: The synchronization client determines the target area ID corresponding to the target data based on the synchronization schedule; The synchronization client determines a target local server corresponding to the target area ID.

8. A data synchronization system, characterized in that: The data synchronization system includes: a local server and a cloud server; The local server is used to execute the data synchronization method according to any one of claims 1 to 4; The cloud server is used to execute the data synchronization method described in any one of claims 5-7.

9. The data synchronization system according to claim 8, characterized in that: The database of the local server and the database of the cloud server both store a synchronization schedule; The synchronization schedule stored in the database of the local server includes: database name, table name and synchronization cursor; The synchronization progress table stored in the database of the cloud server includes: database name, table name, synchronization cursor and area id.

10. The data synchronization system according to claim 8, characterized in that: The target data to be synchronized includes: update time field, area id field, logical deletion field; the update time field does not automatically update the timestamp as the data is modified; The target data uses the original primary key and the region ID as a joint primary key, and the target data does not have any other unique index except the primary key.

Images