Data synchronization method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a data synchronization method and device, electronic equipment, a storage medium and a program product. Relates to the computer field. The method comprises the steps of determining whether changed data meet a synchronization requirement of a target database or not in response to monitoring that data change occurs in a source database; if it is determined that the synchronization requirement is met, generating a corresponding change record according to the change data and storing the change data in a change record table; determining a corresponding target synchronization strategy based on the change records which are not synchronized in the change record table and preset target data synchronization time; the target synchronization strategy is a single-thread execution data synchronization strategy or a multi-thread parallel execution data synchronization strategy; and synchronizing the change data into the target database according to the target synchronization strategy and based on the change record. According to the method, the change data is synchronized into the target database through different target synchronization strategies and based on the change records, and real-time synchronization of the change data can be achieved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a data synchronization method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of mobile Internet, there are scenarios in banking that require multiple functions to work together. For example, mobile banking calls customer channel services to conduct transactions in private systems to implement functions such as SMS and mobile phone contracts. The transaction data is stored in the electronic banking database, and the transaction volume of the electronic banking database is relatively large. Real-time processing of huge transaction data is an urgent problem to be solved.

[0003] In order to facilitate the management of transaction data, the transaction information that needs to be processed in real time is separated into the message information library. The current method of real-time update of the message information library is to synchronize the entire table of real-time transaction information to the message information library, but this method of full-table synchronization has a slow data synchronization speed and is difficult to achieve real-time synchronization of changed data, which cannot meet the usage needs of customers and results in a poor customer experience. Summary of the invention

[0004] The embodiments of the present application provide a data synchronization method, device, electronic device, storage medium and program product to solve the problem in the prior art that data update speed is slow and it is difficult to achieve real-time synchronization.

[0005] In a first aspect, an embodiment of the present application provides a data synchronization method, including:

[0006] In response to monitoring a data change in a source database, determining whether the changed data meets a synchronization requirement of a target database;

[0007] If it is determined that the synchronization requirements are met, a corresponding change record is generated according to the change data and the change data is stored in a change record table, wherein the change record includes change type identification information, change data type and change data identification information; and there is at least one change record in the change record table;

[0008] Determine a corresponding target synchronization strategy based on the change records in the change record table that have not been synchronized and the preset target data synchronization time; the target synchronization strategy is a single-thread execution data synchronization strategy or a multiple-thread parallel execution data synchronization strategy;

[0009] The changed data is synchronized to the target database according to the target synchronization strategy and based on the change record.

[0010] In a possible design, determining whether the changed data meets the synchronization requirement of the target database includes:

[0011] Determine the target business type corresponding to the changed data, and obtain the preset business type for data synchronization of the target database;

[0012] If it is determined that the target business type exists in the preset business type, determining that the changed data meets the synchronization requirement of the target database;

[0013] If it is determined that the target business type does not exist in the preset business types, it is determined that the changed data does not meet the synchronization requirement of the target database.

[0014] In a possible design, determining the change record in the change record table that has not been synchronized includes:

[0015] Obtaining the change type identification information corresponding to each change record in the change record table;

[0016] If it is determined that the change type identification information corresponding to the change record is the first preset value, the second preset value or the third preset value, then the change record determined to be the first preset value, the second preset value or the third preset value is a change record that has not been synchronized, the first preset value represents identification information for adding data, the second preset value represents identification information for modifying data, and the third preset value represents identification information for deleting data.

[0017] In a possible design, determining a corresponding target synchronization strategy based on the change records in the change record table that have not been synchronized and a preset target data synchronization time includes:

[0018] Determine the number of target synchronization tasks based on the unsynchronized change records and the preset target data synchronization time, so that the time for executing the target synchronization tasks is less than or equal to the preset target data synchronization time;

[0019] If the number of the target synchronization tasks is one, determining that the corresponding target synchronization strategy is a single-thread execution data synchronization strategy;

[0020] If the number of the target synchronization tasks is multiple, then the corresponding target synchronization strategy is determined to be a data synchronization strategy in which multiple threads execute data in parallel; the number of the multiple threads is the same as the number of the target synchronization tasks.

[0021] In a possible design, determining the target synchronization task quantity based on the unsynchronized change records and the preset target data synchronization time includes:

[0022] Determine, according to the unsynchronized change record, a single data synchronization time corresponding to a changed data type of the same user that has not been synchronized, wherein the single data synchronization time is determined based on the changed data type and change type identification information of the corresponding changed data;

[0023] Determine the total synchronization time of the changed data that has not been synchronized according to the single data synchronization time;

[0024] The total synchronization time is divided by the preset target data synchronization time and the result is rounded up, and the result of the division of the total synchronization time by the preset target data synchronization time and the result of rounding up is determined as the target synchronization task quantity.

[0025] In a possible design, synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes:

[0026] Generate a corresponding data change operation statement according to the target synchronization strategy and based on the change record;

[0027] The data operation statement is used to perform operations corresponding to the changed data in the target database to synchronize the changed data to the target database.

[0028] In a possible design, the target synchronization strategy is a single-threaded execution data synchronization strategy, and synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes:

[0029] A single thread is used to synchronize the changed data to the target database in the order of the change records in the change record table.

[0030] In a possible design, the target synchronization strategy is a data synchronization strategy executed in parallel by multiple threads, and the change record also includes account information to which the changed data belongs;

[0031] The step of synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes:

[0032] Get pre-built data tables corresponding to multiple threads;

[0033] Dividing the change records in the change record table according to the account information in the change data, and storing the divided change data in a corresponding data table in a chronological order;

[0034] Multiple threads are used to execute in parallel the step of synchronizing the changed data to the target database based on the changed records in the corresponding data table.

[0035] In a possible design, after synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, the method further includes:

[0036] The change type identification information in the synchronized change record is adjusted to a fourth preset value, where the fourth preset value indicates identification information that data synchronization has been completed.

[0037] In one possible design, the method further includes:

[0038] Periodically delete the change records in the change record table whose change type identification information is a fourth preset value.

[0039] In a second aspect, an embodiment of the present application provides a data synchronization device, including:

[0040] A determination module, configured to determine whether the changed data meets the synchronization requirements of the target database in response to monitoring that the source database has a data change;

[0041] A generation module is used to generate a corresponding change record according to the change data if it is determined that the synchronization requirements are met;

[0042] A storage module is used to store the change data in a change record table, wherein the change record includes change type identification information and change data identification information; and there is at least one change record in the change record table;

[0043] The determination module is further used to determine a corresponding target synchronization strategy based on the change record in the change record table; the target synchronization strategy is a single-thread execution data synchronization strategy or a multiple-thread parallel execution data synchronization strategy;

[0044] A synchronization module is used to synchronize the changed data to a target database according to the target synchronization strategy and based on the change record.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0046] The memory stores computer-executable instructions;

[0047] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement various possible implementation methods of the first aspect above.

[0049] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0050] The data synchronization method, device, electronic device, storage medium and program product provided by the embodiment of the present application, in response to monitoring the data change of the source database, determine whether the change data meets the synchronization requirements of the target database; if it is determined that the synchronization requirements are met, then generate a corresponding change record according to the change data and store the change data in the change record table, the change record includes change type identification information, change data type and change data identification information; there is at least one change record in the change record table; based on the change record in the change record table that does not perform synchronization and the preset target data synchronization time, determine the corresponding target synchronization strategy; the target synchronization strategy is a single-thread execution data synchronization strategy or a plurality of threads execute data synchronization strategy in parallel; according to the target synchronization strategy and based on the change record, synchronize the change data to the target database. By analyzing the change information of the change record table, it is possible to quickly locate the synchronization position of the change data in the target database. In addition, it is possible to flexibly select whether to apply a single-thread execution data synchronization strategy or a plurality of threads execute data synchronization strategy in parallel by analyzing the synchronization that does not perform synchronization and the preset target data synchronization time in the change record table, so as to realize the change records of different situations to be quickly synchronized to the target database according to the determined target synchronization strategy. Therefore, the method of recording change information based on the change record table and synchronizing the change data in a single thread or multiple threads can solve the problem of slow data update in full table synchronization and realize real-time synchronization of change data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] Figure 1 An application scenario diagram of the data synchronization method provided in an embodiment of the present application;

[0053] Figure 2 A flowchart of a data synchronization method provided in one embodiment of the present application;

[0054] Figure 3 A flowchart of a data synchronization method provided by another embodiment of the present application;

[0055] Figure 4 A schematic diagram of a data synchronization device provided in an embodiment of the present application;

[0056] Figure 5 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0059] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0060] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0061] In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail.

[0062] When there is a change in banking business and the data in the message information library needs to be updated in real time, if the table structure is modified directly, for example, a synchronization field is added to the contract library table structure. The contract library table structure is usually fixed, which will cause abnormal access to the contract library table structure by the channel service private system and the electronic banking system, resulting in abnormal functions of the channel service private system and the electronic banking system. Therefore, the update of the changed data can only be achieved through full table synchronization. The full table synchronization method is to synchronize all the data in the table where the changed business data is located to the table in the message information library to achieve data update. However, this full table synchronization method has a slow data synchronization speed and it is difficult to achieve real-time synchronization of the changed data.

[0063] Therefore, in the face of the existing technical problems, in order to achieve real-time synchronization of the changed data, the changed data can be updated. Specifically, a change record table can be used to store change records, wherein the changed data can be quickly located by identifying the change record identifier; by identifying the change type identifier, the change type can be quickly located whether it is adding data, modifying data or deleting data. Threads have the characteristics of small resource usage, fast task processing speed and support for parallel processing. Therefore, when facing different change record situations, it can be flexibly determined whether to apply a single-thread execution data synchronization strategy or a multi-thread parallel execution data synchronization strategy based on the change record situation that has not been synchronized in the change record table and combined with the preset target data synchronization time, so as to achieve the rapid synchronization of the change number to the target database according to the determined target synchronization strategy. Therefore, the change data can be synchronized to the target database according to the target synchronization strategy and based on the change record to achieve real-time synchronization of the change data.

[0064] Figure 1 An application scenario diagram of the data synchronization method provided in the embodiment of the present application, in which the server device 101 corresponding to the source database, the server device 102 corresponding to the target database, the data synchronization device 103 and the terminal device 104 may be included. Among them, the terminal device 104 is connected to the server device 101 corresponding to the source database in communication, and the data synchronization device 103 is connected to the server device 101 corresponding to the source database and the server device 102 corresponding to the target database in communication. Among them, the server device 101 corresponding to the source database may be a server of the electronic banking system, and the source database may be an electronic banking database. The server device 102 corresponding to the target database may be a server of the channel service private system, and the target server may be a message information library. Among them, the target information library may be a transaction data storing real-time processing business. Among them, the source database may be integrated into the server device 102 corresponding to the target database, or may be independent of the server device 102 corresponding to the target database. Similarly, the target database may also be integrated into the server device 102 corresponding to the target database, or may be independent of the server device 102 corresponding to the target database, which is not limited in this embodiment. The data synchronization device 103 may be a data synchronization server, a server cluster or other electronic device, or may be integrated in a server-side device corresponding to the target database, which is not limited in this embodiment.

[0065] Specifically, the user performs business operations through the financial system webpage or the financial system APP on the terminal device 104; the server device 101 corresponding to the source database receives the business data sent by the terminal device 104; wherein the business data may include transaction data. The server device 101 corresponding to the source database performs data changes on the data in the source database based on the business data. The data synchronization device 103 detects that the data in the source database has changed and determines whether the changed data meets the requirements for synchronizing the changed data to the target database. If it does, the data synchronization device 103 generates a corresponding change record according to the changed data and stores the change record in the change record table; the data synchronization device 103 determines whether the current target synchronization strategy is a single-thread execution data synchronization strategy or a plurality of threads executing data synchronization strategy in parallel according to the change record that has not been synchronized in the change record and the preset target data synchronization time; and synchronizes the changed data to the target database according to the target synchronization strategy and based on the change record, so as to realize real-time synchronization of the changed data. The server device 102 corresponding to the target database performs corresponding business operations based on the synchronized changed data. If the business operation is a transfer business, a short interest notification of the transfer business is performed.

[0066] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0067] FIG. 2 is a flow chart of a data synchronization method provided by an embodiment of the application. Figure 2 As shown, the execution subject of this embodiment is a data synchronization device, which can be implemented by a computer program; it can also be implemented by a medium storing relevant computer programs, such as a USB flash drive and / or an optical disk, or it can also be implemented by a physical device integrated or installed with relevant computer programs, such as a chip or a data synchronization device. The data synchronization device can be a server, a server cluster or other electronic equipment, etc. The data synchronization method provided in this embodiment includes the following steps:

[0068] Step 201: In response to monitoring data changes in a source database, determine whether the changed data meets the synchronization requirements of a target database.

[0069] The source database refers to the source database when data synchronization is performed. It can be used to obtain changed data from the source database.

[0070] The target database refers to the destination database for data synchronization, which can be used to receive the changed data to be synchronized.

[0071] Specifically, a monitoring thread may be configured in the server device corresponding to the source database, and when it is detected that there is data update in the data table of the source database, it is determined whether the changed data meets the synchronization requirement to the target database.

[0072] The data table may have data updated, which may be data addition, deletion or modification, etc., which is not limited in this embodiment.

[0073] Optionally, when determining whether the changed data meets the synchronization requirements of the target database, preset fields that need to be synchronized to the target database may be preconfigured, and if the changed data includes the preset fields, it is determined that the synchronization requirements of the target database are met. Conversely, if the changed data does not include the preset fields, it is determined that the synchronization requirements of the target database are not met.

[0074] The preset fields may be configured based on the fields included in the data table in the target database, or may be configured in other ways, which is not limited in this embodiment.

[0075] Step 202: If it is determined that the synchronization requirements are met, a corresponding change record is generated based on the change data and the change data is stored in a change record table. The change record includes change type identification information, change data type and change data identification information; there is at least one change record in the change record table.

[0076] The change record table refers to a table for storing change records. The change record table can store change records corresponding to change data, and can also store change data.

[0077] In addition to the change type identification information, the change data type and the change data identification information, the change record may also include the time of the change data, user information, etc.

[0078] The changed data type is the data type corresponding to the changed data, which can be, for example, a long type, a float type, etc., and can be set according to the requirements for the data type, which is not limited in this embodiment.

[0079] The change type identification information is information used to identify the change type, and different change type identifications can be set for different data change operations. For example, when the data change operation is a delete operation, the change type identification can be "del"; when the data change operation is an add operation, the change type identification can be: "add"; when the data change operation is a modify operation, the change type identification can be: "mod".

[0080] The changed data identification information refers to information identifying the changed data. For example, the changed data identification information may be composed of the ID of the changed user, the table name corresponding to the changed business type, the changed field name, and the changed time. This is only for illustrative reference and is not limited in this embodiment.

[0081] Specifically, when the synchronization requirements to the target database are met, the change data in the source table in the source database and its corresponding user information, change time, type of data change operation and other information are obtained. The change data type can be directly obtained through the preset program code; the corresponding change type identification information is generated according to the operation form of the change data; the change data identification information is generated according to the user information, change time, etc. corresponding to the change data; and the change data, change type identification information, change data type and change data identification information are associated and stored in the change record table. If the data is deleted, the change data field can be marked as NULL or marked as other, which is not limited in this embodiment. Among them, there must be at least one change record in the change record table.

[0082] Step 203: Determine a corresponding target synchronization strategy based on the unsynchronized change records in the change record table and the preset target data synchronization time; the target synchronization strategy is a single-thread execution data synchronization strategy or a multi-thread parallel execution data synchronization strategy.

[0083] The preset target data synchronization time is to meet the real-time synchronization of the changed data, and the set execution change data synchronization time must be less than or equal to the threshold time. For example, it can be set to 10s, and can be set according to the customer's satisfaction with the processing of internal business. In this embodiment, there is no limitation.

[0084] The target synchronization strategy refers to a preset strategy for data synchronization, which can be a single-threaded execution of the data synchronization strategy or a multi-threaded parallel execution of the data synchronization strategy.

[0085] The single-thread execution data synchronization strategy is to set a strategy for executing data synchronization by one thread.

[0086] The strategy of executing data synchronization in parallel with multiple threads refers to setting two or more threads to execute data synchronization tasks in parallel.

[0087] Specifically, the synchronization time of the changed data in each change record that has not been synchronized can be obtained, the synchronization time of each change record that has not been synchronized can be summed up, and the calculation result of the sum of the synchronization time of each change record can be compared with the preset target data synchronization time to confirm whether it exceeds the preset target data synchronization time. If it does not exceed, the target synchronization strategy is determined to be a single-thread execution data synchronization strategy; if it exceeds, the target synchronization strategy is determined to be a multiple-thread parallel execution data synchronization strategy. Optionally, the target synchronization strategy can also be determined based on the frequency of data changes. Exemplarily, the change frequency of each business within 1 day can be calculated, and the average change frequency of all businesses can be calculated; if the change frequency of each business within 1 day is greater than the average call frequency of all businesses, it is determined to execute the thread parallel execution data synchronization strategy; if the change frequency of each business within 1 day is less than the average call frequency of all businesses, it is determined to execute the single-thread execution data synchronization strategy.

[0088] It is understandable that the target synchronization strategy can be set according to business requirements, system performance, etc., and is not specifically limited in this embodiment.

[0089] Step 204: Synchronize the changed data to the target database according to the target synchronization strategy and based on the change record.

[0090] Specifically, start the thread corresponding to the target synchronization strategy determined in step 203, parse the change data identifier in the change record, and query the data location to be changed in the target database based on the result of the change data identifier obtained by parsing; use the SQL statement of the corresponding change type to synchronize the change data in the change record to the corresponding change data location in the target database.

[0091] The SQL statements of the changed type may include added SQL statements, deleted SQL statements or modified SQL statements.

[0092] In this embodiment, in response to monitoring the data change of the source database, it is determined whether the changed data meets the synchronization requirements of the target database; if it is determined that the synchronization requirements are met, a corresponding change record is generated according to the changed data and the changed data is stored in the change record table, and the change record includes change type identification information, change data type and change data identification information; there is at least one change record in the change record table; the corresponding target synchronization strategy is determined based on the change record in the change record table that has not been synchronized and the preset target data synchronization time; the target synchronization strategy is a single-thread execution data synchronization strategy or a plurality of threads execute data synchronization strategy in parallel; the changed data is synchronized to the target database according to the target synchronization strategy and based on the change record. By recording the change record identifier, the location of the changed data in the target database can be quickly located; then the corresponding target synchronization strategy is determined based on the change record in the change record table that has not been synchronized and the preset target data synchronization time, and the corresponding target synchronization strategy can be flexibly selected according to the change record in different situations; finally, the change data is synchronized according to the determined thread single-thread execution data synchronization strategy or the plurality of threads execute data synchronization strategy in parallel, which can significantly improve the data synchronization speed; in this way, the real-time synchronization of the changed data can be achieved to meet the user's usage needs.

[0093] As an optional implementation, based on the above embodiment, determining whether the changed data meets the synchronization requirement of the target database includes:

[0094] Determine the target business type corresponding to the changed data, and obtain the preset business type for data synchronization of the target database; if it is determined that the target business type exists in the preset business type, determine that the changed data meets the synchronization requirements of the target database; if it is determined that the target business type does not exist in the preset business type, determine that the changed data does not meet the synchronization requirements of the target database.

[0095] The target business type refers to the business type corresponding to the changed data when the data is changed. It can be any banking business that supports user operations.

[0096] The preset business type is a business type that needs to be synchronized according to the preset rules, such as transfer, deposit, withdrawal, contract signing, etc. The preset business type can be stored in the preset business type storage table.

[0097] Specifically, the server can parse the change data identification information to obtain the fields corresponding to the change data, determine the corresponding target business type based on the corresponding fields, then obtain the preset business type storage table for data synchronization of the target database, and then query whether the target business type belongs to the preset business type in the preset business type storage table; if it does, it can be determined that the current change data meets the synchronization requirements of the target database. On the contrary, if it does not, it can be determined that the current change data does not meet the synchronization requirements of the target database.

[0098] In this embodiment, determining whether the changed data meets the synchronization requirement of the target database includes:

[0099] Determine the target business type corresponding to the changed data, and obtain the preset business type for data synchronization of the target database; if it is determined that the target business type exists in the preset business type, determine that the changed data meets the synchronization requirements of the target database; only synchronize the changed data that is determined to meet the synchronization requirements of the target database, which can avoid unnecessary data synchronization, save system resources, and further improve the efficiency of changing data synchronization, and use matching business types to determine whether the synchronization requirements of the database are met, which can quickly determine whether the synchronization requirements of the database are met.

[0100] As an optional implementation manner, based on the above embodiment, determining the change record in the change record table that has not been synchronized includes:

[0101] Obtain change type identification information corresponding to each change record in the change record table; if it is determined that the change type identification information corresponding to the change record is a first preset value, a second preset value, or a third preset value, then the change record determined to be the first preset value, the second preset value, or the third preset value is a change record that has not been synchronized, the first preset value represents identification information for adding data, the second preset value represents identification information for modifying data, and the third preset value represents identification information for deleting data.

[0102] Specifically, the change record in the change record table is read, and the change type identification information in the change record is queried. If the change type identification information is a first preset value, a second preset value, or a third preset value, the change record containing the first preset value, the second preset value, or the third preset value is used as a change record that has not been synchronized. The identification information of the first preset value, the second preset value, or the third preset value can be used for application 1, 2, 3 identification, or other identification symbols, which are not limited in this embodiment.

[0103] In this embodiment, the change type identification information corresponding to each change record in the change record table is obtained; if the change type identification information corresponding to the change record is determined to be the first preset value, the second preset value, or the third preset value, then the change record determined to be the first preset value, the second preset value, or the third preset value is a change record that has not been synchronized, the first preset value represents identification information for adding data, the second preset value represents identification information for modifying data, and the third preset value represents identification information for deleting data. Determining the change record that has not been synchronized by using the change record of the first preset value, the second preset value, or the third preset value can eliminate the interference of the change record that has been synchronized, and can more accurately determine the change record.

[0104] As an optional implementation, based on the above embodiment, the corresponding target synchronization strategy is determined based on the change records that have not been synchronized in the change record table and the preset target data synchronization time, including:

[0105] The number of target synchronization tasks is determined based on the change records that have not been synchronized and the preset target data synchronization time, so that the time to execute the target synchronization task is less than or equal to the preset target data synchronization time; if the number of target synchronization tasks is one, the corresponding target synchronization strategy is determined to be a single-thread execution data synchronization strategy; if the number of target synchronization tasks is multiple, the corresponding target synchronization strategy is determined to be a multiple-thread parallel execution data synchronization strategy; the number of multiple threads is the same as the number of target synchronization tasks.

[0106] The target synchronization task quantity is the sum of all synchronization tasks that need to be executed, and one thread corresponds to one task. One thread can execute synchronization operations on multiple changed data; similarly, arranging a task is to preset the synchronization of multiple changed data in one thread.

[0107] Specifically, the data synchronization time of the change data corresponding to all the change records that have not been synchronized can be queried and summed up, and then the total data synchronization time of the change data corresponding to the change records that have not been synchronized can be obtained; the above total data synchronization time is compared with the time of the target synchronization task. If it is less than or equal to the preset target data synchronization time, it means that the change data corresponding to the change records that have not been synchronized can be synchronized in real time in one task, and the number of target synchronization tasks can be preset as one, then the target synchronization strategy is determined to be a single-thread execution data synchronization strategy; if it is greater than the preset target data synchronization time, it means that the demand for real-time synchronization of the change data corresponding to the change records that have not been synchronized cannot be met in one task, and the number of target synchronization tasks needs to be set to multiple, then the target synchronization strategy is determined to be a multiple-thread parallel execution data synchronization strategy.

[0108] Optionally, a preset quantity threshold can also be determined based on the change record, and then the target synchronization strategy can be directly determined. For example, the response time of a single thread executing one data synchronization can be calculated, and statistics are made once for different types of data structures. The number of different data structures processed by a single thread within 1 second can be obtained, and the number of data that can be processed within 1 second is used as the preset quantity threshold. In this way, different preset quantity thresholds can be set for different data structures. If there are change records corresponding to change data of multiple different data structures in the change record table, the average of the corresponding quantity thresholds of the multiple data structures can be obtained, and the average is determined as the preset quantity threshold. In this embodiment, the preset quantity threshold determined is not limited.

[0109] Then, after determining the preset number threshold, the number of change records that have not been synchronized is determined by identifying the change type identification information in the change records that have not been synchronized. If the number of change records that have not been synchronized is less than or equal to the preset number threshold, the corresponding target synchronization strategy is determined to be a single-thread execution data synchronization strategy; when it is greater than the preset number threshold, the corresponding target synchronization strategy is determined to be a multiple-thread parallel execution data synchronization strategy.

[0110] In this embodiment, the number of target synchronization tasks is determined based on the change records that have not been synchronized and the preset target data synchronization time, so that the time to execute the target synchronization task is less than or equal to the preset target data synchronization time; if the number of target synchronization tasks is one, the corresponding target synchronization strategy is determined to be a single-thread execution data synchronization strategy; if the number of target synchronization tasks is multiple, the corresponding target synchronization strategy is determined to be a multiple-thread parallel execution data synchronization strategy; the number of multiple threads is the same as the number of target synchronization tasks. The time of the target synchronization task is less than or equal to the preset target data synchronization time. When the time of the target synchronization task is less than or equal to the preset target data synchronization time, the number of target tasks is set to 1 and the single-thread execution data synchronization strategy is preset. The single-thread target synchronization strategy can still achieve the effect of real-time data synchronization on the basis of saving computing resources. When the time of the target synchronization task is greater than the preset target data synchronization time, the number of target tasks is set to multiple and the single-thread execution data synchronization strategy is adopted, and the multiple threads are used to execute the data synchronization strategy in parallel. The multiple threads executing the data synchronization strategy in parallel can make full use of the advantages of multi-core processors, improve the concurrency of data synchronization, and thus speed up the data synchronization speed. In this way, it is helpful to achieve real-time synchronization of changed data.

[0111] As an optional implementation, determining the target synchronization task quantity based on the unsynchronized change records and the preset target data synchronization time includes:

[0112] Determine the single data synchronization time corresponding to the type of change data that has not been synchronized for the same user based on the change records that have not been synchronized; the single data synchronization time is determined based on the change data type and change type identification information of the corresponding change data; determine the total synchronization time of the change data that has not been synchronized based on the single data synchronization time; divide the total synchronization time by the preset target data synchronization time and round up, and determine the result of dividing the total synchronization time by the preset target data synchronization time and rounding up as the target synchronization task quantity.

[0113] The single data synchronization time corresponding to the unsynchronized changed data type of the same user refers to the time required to execute all unsynchronized changed data of the same user in a single thread. It can be the sum of the single data synchronization time corresponding to each changed data type of the same user. The single data synchronization time corresponding to the changed data type refers to the time required to synchronize data of one data type in a single thread.

[0114] Optionally, the single data synchronization time can be determined according to the changed data type and the changed type identification information. Specifically, the addition, modification, and deletion operations corresponding to different change type identification information can be performed on different data types, and the execution time of the addition, modification, and deletion operations performed on different data types can be recorded in the data synchronization schedule. Among them, the changed data types that occupy the same number of bytes can be regarded as the same data type, because in data synchronization, the execution efficiency of the data types with the same number of bytes is the same, so each data type only needs to be tested once. The changed data type that occupies the same number of bytes refers to the data type that stores the same bytes in the memory or storage medium. For example, the float type occupies 4 bytes and the long type also occupies 4 bytes, then the float type and the long type belong to the changed data type that occupies the same number of bytes. The data synchronization schedule is a table for storing the execution time of adding, modifying, and deleting operations on data. Exemplarily, the time required for adding long type data, the time required for deleting long type data, and the time required for deleting long type data can be recorded respectively. The data synchronization schedule can be stored in the server. The data storage form in the data synchronization schedule can be a combination field of the change type information and the number of bytes of the data, and the single data synchronization time associated with the storage. Exemplarily, del4, del represents a change type identifier, and 4 represents a change data type.

[0115] The total synchronization time of the unsynchronized changed data is the total synchronization time required to execute all the unsynchronized changed data.

[0116] Among them, rounding up is a rounding method towards positive infinity, rounding a real number to the smallest integer greater than or equal to the number. Exemplarily, when the total synchronization time of the change data that has not been synchronized is divided by the preset target data synchronization time to be 3.4, the result after rounding up is 4. Specifically, the change records of the same user in the change records that have not been synchronized are arranged; then the number of bytes corresponding to each change data type is obtained, the change type identification information is combined with the number of bytes corresponding to each corresponding change data type, and the single data synchronization time corresponding to the change data type corresponding to the above combination result is queried in the data synchronization timetable; the single data synchronization time corresponding to the change data type of the same user is added as the single data synchronization time corresponding to the change data type that has not been synchronized for the same user. Finally, the single data synchronization time corresponding to the change data type that has not been synchronized for each user is added as the total synchronization time of the change data that has not been synchronized; the total synchronization time of the change data that has not been synchronized is divided by the preset target data synchronization time and rounded up, and the result of the above operation is determined as the target synchronization task number.

[0117] In this embodiment, the single data synchronization time corresponding to the changed data type of the same user is determined based on the unsynchronized change record, and the single data synchronization time is determined based on the changed data type and the change type identification information of the corresponding changed data; the total synchronization time of the changed data that has not been synchronized is determined based on the single data synchronization time corresponding to the changed data type of the same user; the total synchronization time of the changed data that has not been synchronized is divided by the preset target data synchronization time and rounded up, and the total synchronization time of the changed data that has not been synchronized is divided by the preset target data synchronization time and rounded up as the target synchronization task number. Determining the target synchronization task number based on the total synchronization time of the changed data that has not been synchronized and the preset target data synchronization time can ensure that thread resources are reasonably used and that the unsynchronized changed data can be completed within the preset time, so that the unsynchronized changed data can be synchronized in real time, which can further improve customer satisfaction.

[0118] As an optional implementation, based on the above embodiment, after synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, the method further includes:

[0119] The change type identification information in the synchronized change record is adjusted to a fourth preset value, where the fourth preset value indicates identification information that data synchronization has been completed.

[0120] Specifically, after the data is changed successfully, the change record table is accessed through a database statement and the change type identifier is adjusted to a fourth preset value. The fourth preset value may be identified as 0, Y, or other, which is not limited in this embodiment.

[0121] Optionally, the amount of uncompleted synchronization change data may also be determined based on identifying a fourth preset value.

[0122] Optionally, in this embodiment, the change records in the change record table whose change type identification information is a fourth preset value may also be deleted periodically.

[0123] Optionally, a timer can be preset in the target synchronization strategy to periodically query the change record table and delete the change record whose change type identification information is a fourth preset value. The preset period of the timer can be 1 week, 1 month, 1 year, or other periods, which are not specifically limited here.

[0124] In this embodiment, after synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, the method also includes: adjusting the change type identification information in the synchronized change record to a fourth preset value, the fourth preset value indicating the identification information of the completed data synchronization. Periodically delete the change records in the change record table whose change type identification information is the fourth preset value. The change records that have completed data synchronization can be accurately distinguished according to the preset fourth preset value; the change records that have completed synchronization can also be periodically cleaned according to the four preset values; periodically cleaning the change records that have completed synchronization can provide more storage space for new change records; it also reduces the calculation of the amount of data, thereby improving the overall efficiency of data synchronization.

[0125] As an optional implementation, based on the above embodiment, synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes:

[0126] Generate corresponding data change operation statements according to the target synchronization strategy and based on the change records; use the data operation statements to perform operations corresponding to the changed data in the target database to synchronize the changed data to the target database.

[0127] The data change operation statement refers to the operation statement executed by the database when the data is changed, which can be the addition, modification, deletion, etc. of data.

[0128] Specifically, a thread task corresponding to the target synchronization strategy is started. During the thread task, a change type identifier containing a preset value, a first preset value, a second preset value, or a third preset value is obtained from the change record table. A data addition statement is generated according to the first preset value identifier; a data change statement is generated according to the second preset value identifier; a data deletion statement is generated according to the third preset value identifier, thereby generating a data change operation statement. If a data addition or modification statement is generated, the change data identifier of the change record is first read, and then the location of the change data in the target database is queried by parsing the change data identifier, and the corresponding change data is synchronously changed to the target database through the data addition or modification statement. If a data deletion statement is generated, the location of the change data in the target database is directly located according to the parsed change data identifier result and the database deletion statement is executed.

[0129] Optionally, if the change data is not stored in the change record table and the change type identifier is the first preset value identifier or the second preset value identifier, the location of the change data in the source database can be queried based on the parsed change data identifier to obtain the corresponding change data; then the location of the change data in the target database can be queried by parsing the change data identifier, and the corresponding change data can be synchronized to the target database through database operation statements.

[0130] In this embodiment, the changed data is synchronized to the target database according to the target synchronization strategy and based on the change record, including: generating corresponding data change operation statements according to the target synchronization strategy and based on the change record; using the data operation statements to perform operations corresponding to the changed data in the target database to synchronize the changed data to the target database. The corresponding data change operation statements are generated by the changed data identifier, and each unexecuted synchronization record corresponds to a data change operation, so that data omissions can be avoided when synchronizing large quantities of data; real-time response to the changed data can also be guaranteed; and thus it helps to achieve real-time synchronization of the changed data.

[0131] As an optional implementation, based on the above embodiment, the target synchronization strategy is a single-threaded execution data synchronization strategy, and the changed data is synchronized to the target database according to the target synchronization strategy and based on the change record, including:

[0132] A single thread is used to synchronize the changed data to the target database in the order of the change records in the change record table.

[0133] Optionally, when a single thread is used to perform data synchronization, the change data time corresponding to the change data is read and the change records are stored in the corresponding data table in the order of the change data time; until all change records are synchronized to the target database.

[0134] In this embodiment, the target synchronization strategy is a single-threaded execution data synchronization strategy, and the changed data is synchronized to the target database according to the target synchronization strategy and based on the change record, including: using a single thread to synchronize the changed data to the target database according to the order of the change records in the change record table. The single-threaded execution data synchronization strategy can ensure that each change record is processed according to the order in the change record table. The single-threaded synchronization execution logic is simple and can quickly respond to data synchronization, thereby helping to achieve real-time synchronization of the changed data.

[0135] As an optional implementation, based on the above embodiment, the target synchronization strategy is that multiple threads execute the data synchronization strategy in parallel, and the change record also includes the account information to which the changed data belongs;

[0136] Synchronize the changed data to the target database according to the target synchronization strategy and based on the change records, including: obtaining pre-built data tables corresponding to multiple threads; dividing the change records in the change record table according to the change data account information, and storing the divided change data in the corresponding data table in sequence; using multiple threads to execute in parallel to synchronize the changed data to the target database based on the change records in the corresponding data table.

[0137] The pre-built data table refers to a table used to store data structures, which can be used to store change records; the number of data tables can be set according to requirements. The data tables stored when data changes occur can be preset for each user and numbered. For example, the data changes of user A are stored in the first data table, which can be marked as 1.

[0138] Specifically, after the data table is built in the server, a change data synchronization thread is set for each data table; at the same time, a unique label between each data table and thread can be set. Thus, the data table corresponding to each thread can be obtained according to the unique label between each data table and thread. Among them, it is ensured that at least one change data synchronization thread is matched for each data table.

[0139] Specifically, when the change record of the change record table is updated, the account information in the change record is queried, and the label of the corresponding storage data table can be queried based on the account information; after completing the information confirmation of the storage data table of each account; read the change data time corresponding to the change data and store the change record in the corresponding data table in the order of the change data time.

[0140] Optionally, when a new account appears, the changed data can be inserted into the data table that stores the least amount of data.

[0141] Specifically, after the changed data is stored in the corresponding data table, the execution thread of each table is started, and the data can be synchronized in chronological order. When a user has multiple changes, the multiple pieces of information of the user can be directly synchronized to the target database in chronological order.

[0142] Optionally, threads can be created through a thread pool. When the number of changed data to be synchronized stored in the same data table exceeds a preset threshold, the thread pool can dynamically increase the number of threads to achieve rapid synchronization of more changed data; when the number of changed data is small, the thread pool can dynamically reduce the number of threads to save server resources.

[0143] In this embodiment, when the target synchronization strategy is a data synchronization strategy executed in parallel by multiple threads, the change record also includes account information to which the change data belongs; synchronizing the change data to the target database according to the target synchronization strategy and based on the change record includes: obtaining a pre-built data table corresponding to multiple threads; dividing the change records in the change record table according to the change data account information, and storing the divided change data in the corresponding data table in order; using multiple threads to execute in parallel based on the change records in the corresponding data table to synchronize the change data to the target database. By storing the change records in different data tables and presetting the corresponding thread synchronization mechanism for each data table, when a large number of data changes occur, the thread synchronization mechanism can be quickly assigned to the change data. Combined with multi-threaded parallel execution of data synchronization, the speed of change data synchronization is enhanced, and real-time synchronization of change data can be achieved.

[0144] Figure 3 This is a flow chart of a data synchronization method provided by another embodiment of the present application. The data synchronization method provided by this embodiment includes the following steps:

[0145] Step 301: Monitor in real time whether the changed data of the source database meets the synchronization requirements of the target database.

[0146] Step 302: If the data synchronization requirements are met, a change record is generated and stored in a change record table.

[0147] Step 303: Determine the number of target synchronization tasks based on the unsynchronized change records and the preset target data synchronization time, and determine the target synchronization strategy based on the number of target synchronization tasks.

[0148] Step 304: Start the thread corresponding to the target synchronization strategy; and generate the corresponding data operation statement according to the change type identification information.

[0149] Step 305: Use the thread corresponding to the target synchronization strategy and locate the position corresponding to the changed data in the target database according to the changed data identification information.

[0150] Step 306: The thread corresponding to the target synchronization strategy synchronizes the changed data in the change record table to the source database according to the data operation statement.

[0151] In this embodiment, the implementation method and technical effects of steps 301 to 306 are similar to the implementation methods of the corresponding solutions in the above embodiments, and will not be repeated here.

[0152] Figure 4 The data synchronization device 40 provided in this embodiment includes: a determination module 401, a generation module 402, a storage module 403, and a synchronization module 404.

[0153] The determination module 401 is used to determine whether the changed data meets the synchronization requirement of the target database in response to monitoring that the source database has data changed.

[0154] The generation module 402 is used to generate a corresponding change record according to the change data if it is determined that the synchronization requirement is met.

[0155] The storage module 403 is used to store the changed data in a change record table, wherein the change record includes change type identification information, change data type and change data identification information; there is at least one change record in the change record table.

[0156] The determination module 401 is also used to determine the corresponding target synchronization strategy based on the unsynchronized change records in the change record table and the preset target data synchronization time; the target synchronization strategy is a single-thread execution data synchronization strategy or a multi-thread parallel execution data synchronization strategy.

[0157] The synchronization module 404 is used to synchronize the changed data to the target database according to the target synchronization strategy and based on the change record.

[0158] Optionally, when determining whether the changed data meets the synchronization requirement of the target database, the determination module 401 is specifically configured to:

[0159] Determine the target business type corresponding to the changed data, and obtain the preset business type for data synchronization of the target database; if it is determined that the target business type exists in the preset business type, determine that the changed data meets the synchronization requirements of the target database; if it is determined that the target business type does not exist in the preset business type, determine that the changed data does not meet the synchronization requirements of the target database.

[0160] Optionally, when determining the change record in the change record table for which synchronization has not been performed, the determination module 401 is specifically configured to:

[0161] Obtain change type identification information corresponding to each change record in the change record table; if it is determined that the change type identification information corresponding to the change record is a first preset value, a second preset value, or a third preset value, then the change record determined to be the first preset value, the second preset value, or the third preset value is a change record that has not been synchronized, the first preset value represents identification information for adding data, the second preset value represents identification information for modifying data, and the third preset value represents identification information for deleting data.

[0162] Optionally, the determination module 401, when determining a corresponding target synchronization strategy based on the change records in the change record table that have not been synchronized and a preset target data synchronization time, is specifically used to:

[0163] The number of target synchronization tasks is determined based on the change records that have not been synchronized and the preset target data synchronization time, so that the time to execute the target synchronization task is less than or equal to the preset target data synchronization time; if the number of target synchronization tasks is one, the corresponding target synchronization strategy is determined to be a single-thread execution data synchronization strategy; if the number of target synchronization tasks is multiple, the corresponding target synchronization strategy is determined to be a multiple-thread parallel execution data synchronization strategy; the number of multiple threads is the same as the number of target synchronization tasks.

[0164] Optionally, the determination module 401, when determining the target synchronization task quantity based on the unsynchronized change records and the preset target data synchronization time, is specifically configured to:

[0165] Determine the single data synchronization time corresponding to the type of change data that has not been synchronized for the same user based on the change records that have not been synchronized, and the single data synchronization time is determined based on the type of change data and the change type identification information of the corresponding change data; determine the total synchronization time of the change data that has not been synchronized based on the single data synchronization time; divide the total synchronization time by the preset target data synchronization time and round up, and determine the result of dividing the total synchronization time by the preset target data synchronization time and rounding up as the target synchronization task quantity.

[0166] Optionally, the synchronization module 404, when synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, is specifically used to:

[0167] Generate corresponding data change operation statements according to the target synchronization strategy and based on the change records; use the data operation statements to perform operations corresponding to the changed data in the target database to synchronize the changed data to the target database.

[0168] Optionally, the target synchronization strategy is a single-thread execution data synchronization strategy.

[0169] Accordingly, when synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, the synchronization module 404 is specifically used to:

[0170] A single thread is used to synchronize the changed data to the target database in the order of the change records in the change record table.

[0171] Optionally, the target synchronization strategy is a data synchronization strategy executed in parallel by multiple threads.

[0172] Accordingly, when synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, the synchronization module 404 is specifically used to:

[0173] Obtain pre-built data tables corresponding to multiple threads; divide the change records in the change record table according to the change data account information, and store the divided change data in the corresponding data table in sequence; use multiple threads to execute in parallel the steps of synchronizing the change data to the target database based on the change records in the corresponding data table.

[0174] Optionally, the data synchronization device provided in this embodiment further includes: an adjustment module and a deletion module.

[0175] Correspondingly, the adjustment module is used to adjust the change type identification information in the synchronized change record to a fourth preset value, where the fourth preset value represents identification information of completed data synchronization.

[0176] Optionally, the deleting module is used to periodically delete the change records in the change record table whose change type identification information is a fourth preset value.

[0177] FIG5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: a processor 501, and a memory 502 in communication with the processor 501. In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0178] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0179] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0180] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0181] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0182] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0183] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0184] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0185] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0186] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0189] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0190] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0191] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A data synchronization method, characterized in that: include: In response to monitoring a data change in a source database, determining whether the changed data meets a synchronization requirement of a target database; If it is determined that the synchronization requirements are met, a corresponding change record is generated according to the changed data and the changed data is stored in a change record table, wherein the change record includes change type identification information, change data type and change data identification information; There is at least one change record in the change record table; Determine a corresponding target synchronization strategy based on the change records in the change record table that have not been synchronized and a preset target data synchronization time; The target synchronization strategy is a single-thread execution data synchronization strategy or a multiple-thread parallel execution data synchronization strategy; The changed data is synchronized to the target database according to the target synchronization strategy and based on the change record.

2. The method according to claim 1, characterized in that The step of determining whether the changed data meets the synchronization requirement of the target database includes: Determine the target business type corresponding to the changed data, and obtain the preset business type for data synchronization of the target database; If it is determined that the target business type exists in the preset business type, determining that the changed data meets the synchronization requirement of the target database; If it is determined that the target business type does not exist in the preset business types, it is determined that the changed data does not meet the synchronization requirement of the target database.

3. The method according to claim 1, characterized in that The determining the change records in the change record table that have not been synchronized includes: Obtaining the change type identification information corresponding to each change record in the change record table; If it is determined that the change type identification information corresponding to the change record is the first preset value, the second preset value or the third preset value, then the change record determined to be the first preset value, the second preset value or the third preset value is a change record that has not been synchronized, the first preset value represents identification information for adding data, the second preset value represents identification information for modifying data, and the third preset value represents identification information for deleting data.

4. The method according to claim 1, characterized in that: Determining a corresponding target synchronization strategy based on the change records in the change record table that have not been synchronized and a preset target data synchronization time includes: Determine the number of target synchronization tasks based on the unsynchronized change records and the preset target data synchronization time, so that the time for executing the target synchronization tasks is less than or equal to the preset target data synchronization time; If the number of the target synchronization tasks is one, determining that the corresponding target synchronization strategy is a single-thread execution data synchronization strategy; If the number of the target synchronization tasks is multiple, then the corresponding target synchronization strategy is determined to be a data synchronization strategy in which multiple threads execute data in parallel; the number of the multiple threads is the same as the number of the target synchronization tasks.

5. The method according to claim 4, characterized in that Determining the target synchronization task quantity based on the unsynchronized change records and the preset target data synchronization time includes: Determine, according to the unsynchronized change record, a single data synchronization time corresponding to a changed data type of the same user that has not been synchronized, wherein the single data synchronization time is determined based on the changed data type and change type identification information of the corresponding changed data; Determine the total synchronization time of the changed data that has not been synchronized according to the single data synchronization time; The total synchronization time is divided by the preset target data synchronization time and the result is rounded up, and the result of the division of the total synchronization time by the preset target data synchronization time and the result of rounding up is determined as the target synchronization task quantity.

6. The method according to claim 1, characterized in that Synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes: Generate a corresponding data change operation statement according to the target synchronization strategy and based on the change record; The data operation statement is used to perform operations corresponding to the changed data in the target database to synchronize the changed data to the target database.

7. The method according to claim 4, characterized in that The target synchronization strategy is a single-threaded execution data synchronization strategy, and synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes: A single thread is used to synchronize the changed data to the target database in the order of the change records in the change record table.

8. The method according to claim 4, characterized in that The target synchronization strategy is a data synchronization strategy executed in parallel by multiple threads, and the change record also includes account information to which the changed data belongs; The step of synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record includes: Get pre-built data tables corresponding to multiple threads; Dividing the change records in the change record table according to the account information in the change data, and storing the divided change data in a corresponding data table in a chronological order; Multiple threads are used to execute in parallel the step of synchronizing the changed data to the target database based on the changed records in the corresponding data table.

9. The method according to claim 1, characterized in that: After synchronizing the changed data to the target database according to the target synchronization strategy and based on the change record, the method further includes: The change type identification information in the synchronized change record is adjusted to a fourth preset value, where the fourth preset value indicates identification information that data synchronization has been completed.

10. The method according to claim 9, characterized in that The method further comprises: Periodically delete the change records in the change record table whose change type identification information is a fourth preset value.

11. A data synchronization device, characterized in that: include: A determination module, configured to determine whether the changed data meets the synchronization requirements of the target database in response to monitoring that the source database has a data change; A generation module is used to generate a corresponding change record according to the change data if it is determined that the synchronization requirements are met; A storage module, used for storing the change data in a change record table, wherein the change record includes change type identification information and change data identification information; There is at least one change record in the change record table; A determination module is further used to determine a corresponding target synchronization strategy based on the change record in the change record table; The target synchronization strategy is a single-thread execution data synchronization strategy or a multiple-thread parallel execution data synchronization strategy; A synchronization module is used to synchronize the changed data to a target database according to the target synchronization strategy and based on the change record.

12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images