Data synchronization method, data synchronization device, electronic equipment and storage medium

Abstract

The present disclosure provides a data synchronization method, a data synchronization device, an electronic device and a storage medium, which can be applied to the technical field of big data. The data synchronization method is applied to a first database and a second database. The data synchronization method applied to the first database comprises: in response to receiving a data synchronization request from the second database, analyzing the data synchronization request, determining a data table update condition of the second database, the data table update condition representing whether a table structure of the master database has changed; obtaining incremental data generated by the first database within a preset time period, the preset time period representing a time period during which the first database performs a transaction operation and the second database does not perform a transaction operation; determining a data synchronization mode according to the data table update condition; and synchronizing the incremental database to the second database based on the data synchronization mode.

Description

Technical Field

[0001] This disclosure relates to the field of big data technology, specifically to a data synchronization method, a data synchronization device, an electronic device, and a storage medium. Background Technology

[0002] To provide users with 24 / 7 continuous service, such as telephone loss reporting, banks and other enterprises typically provide services to users through a primary database and synchronize transaction data to a backup database. During primary database upgrades or maintenance, business transaction requests are routed to the backup database, which then provides services to users to ensure business continuity.

[0003] In related technologies, after the primary database has been upgraded or maintained, data synchronization between the standby database and the primary database relies on manual synchronization operations by business personnel, resulting in high data synchronization costs and long operation times. Furthermore, manual synchronization operations increase the risk of data synchronization errors due to human error, impacting business continuity. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a data synchronization method, a data synchronization device, an electronic device, and a storage medium.

[0005] According to a first aspect of this disclosure, a data synchronization method is provided, applied to a first database, the method comprising:

[0006] In response to receiving a data synchronization request from the second database, the data synchronization request is parsed to determine the update status of the data tables in the second database. The update status of the data tables indicates whether the table structure of the second database has changed.

[0007] Obtain incremental data generated by the first database within a preset time period, where the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations;

[0008] Based on the data table update status, determine the data synchronization method; and

[0009] Based on the data synchronization method, the incremental database is synchronized to the second database.

[0010] According to embodiments of this disclosure, determining the data synchronization method based on data table updates includes:

[0011] If it is determined that the table structure of the second database has changed, the data synchronization method is determined to be the first synchronization method. The first synchronization method is used to synchronize incremental data to the second database through a shared disk.

[0012] If the table structure of the second database remains unchanged, the data synchronization method is determined to be the second synchronization method, which is used to synchronize incremental data to the second database through a replication tool.

[0013] According to embodiments of this disclosure, synchronizing the incremental database to the second database based on a data synchronization method includes:

[0014] If the data synchronization method is determined to be the first synchronization method, the incremental data is stored in a temporary table using a trigger;

[0015] The temporary table is exported as a file to obtain a shared file, and the shared file is named in accordance with the preset format;

[0016] Shared files are written to a shared disk so that a second database can read the shared files from the shared disk.

[0017] According to embodiments of this disclosure, synchronizing the incremental database to the second database based on a data synchronization method further includes:

[0018] If the data synchronization method is determined to be the second synchronization method, obtain the transaction logs corresponding to the incremental data;

[0019] The transaction logs are sent to the replication tool so that the replication tool can generate data replay statements based on the transaction logs and transmit the data replay statements to the second database.

[0020] According to a second aspect of this disclosure, a data synchronization method is provided, applied to a second database, the method comprising:

[0021] Send a data synchronization request to the first database. The data synchronization request includes the data table update status of the second database. The data table update status indicates whether the table structure of the second database has changed.

[0022] Based on the data table update status, determine the data synchronization method; and

[0023] Based on the data synchronization method, incremental data generated by the first database is synchronized to the first data table.

[0024] According to embodiments of this disclosure, determining the data synchronization method based on data table updates includes:

[0025] If it is determined that the table structure of the second database has changed, the data synchronization method is determined to be the first synchronization method. The first synchronization method is used to synchronize incremental data to the first data table of the second database through a shared disk.

[0026] If the table structure of the second database remains unchanged, the data synchronization method is determined to be the second synchronization method. The second synchronization method is used to synchronize incremental data to the first data table of the second database through a replication tool.

[0027] According to embodiments of this disclosure, synchronizing incremental data generated by the first database to the first data table based on a data synchronization method includes:

[0028] If the data synchronization method is determined to be the first synchronization method, the shared disk is scanned and the shared file is obtained. The shared file is determined based on the incremental data generated by the first database within a preset time period. The preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations.

[0029] Retrieve database table transformation rules, which include transformation rules for multiple types of database operations;

[0030] Based on the database table conversion rules and shared files, determine the second data table that matches the table structure of the second database; and

[0031] Insert the second database table into the first database table.

[0032] According to embodiments of this disclosure, synchronizing incremental data generated by the first database to the first data table based on a data synchronization method further includes:

[0033] When the data synchronization method is determined to be the second synchronization method, a data playback statement is received from the replication tool. The data playback statement is used to replay the operation of generating incremental data in the first database.

[0034] Execute the data replay statement to generate a third data table; and

[0035] Import the third data table into the first data table.

[0036] A third aspect of this disclosure provides a data synchronization apparatus for use in a first database, the apparatus comprising:

[0037] The parsing module is used to respond to a data synchronization request received from the second database, parse the data synchronization request, determine the data table update status of the second database, and the data table update status indicates whether the table structure of the second database has changed;

[0038] The acquisition module is used to acquire incremental data generated by the first database within a preset time period, where the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations.

[0039] The first determining module is used to determine the data synchronization method based on the data table update status; and

[0040] The first synchronization module is used to synchronize the incremental database to the second database based on the data synchronization method.

[0041] A fourth aspect of this disclosure provides a data synchronization apparatus for use in a second database, the apparatus comprising:

[0042] The sending module is used to send a data synchronization request to the first database. The data synchronization request includes the data table update status of the second database, which indicates whether the table structure of the second database has changed.

[0043] The second determining module is used to determine the data synchronization method based on the data table update status; and

[0044] The second synchronization module is used to synchronize incremental data generated by the first database to the first data table based on the data synchronization method.

[0045] A fifth aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the data synchronization method described above.

[0046] A sixth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the data synchronization method described above.

[0047] The seventh aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described data synchronization method.

[0048] The embodiments of this disclosure, in response to a received data synchronization request from a second database, parse the data synchronization request to determine the update status of the data tables in the second database; obtain the incremental data generated by the first database within a preset time period; determine the data synchronization method based on the data table update status; and synchronize the incremental database to the second database based on the data synchronization method. This eliminates the need for manual synchronization operations by business personnel, reducing synchronization costs and time. Furthermore, it avoids data synchronization errors caused by human error, improving data synchronization efficiency and preventing disruptions to business continuity. Since the first database automatically determines the data synchronization method by assessing the table structure changes in the second database, and then performs data synchronization based on the appropriate method, no business personnel intervention is required regardless of whether the table structure has changed. Therefore, data synchronization efficiency is improved. Attached Figure Description

[0049] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0050] Figure 1 This illustration schematically depicts an application scenario of the data synchronization method according to embodiments of the present disclosure;

[0051] Figure 2 A flowchart illustrating a data synchronization method applied to a first database according to an embodiment of the present disclosure is shown schematically.

[0052] Figure 3 This illustration schematically depicts an application scenario for determining a data synchronization method according to embodiments of the present disclosure;

[0053] Figure 4 This illustration schematically depicts an application scenario in which data synchronization is performed using a first synchronization method according to an embodiment of the present disclosure;

[0054] Figure 5 This illustration schematically depicts an application scenario where data synchronization is performed using a second synchronization method according to an embodiment of the present disclosure;

[0055] Figure 6 A flowchart illustrating a data synchronization method applied to a second database according to an embodiment of the present disclosure is shown schematically.

[0056] Figure 7 This schematically illustrates a structural block diagram of a data synchronization apparatus applied to a first database according to an embodiment of the present disclosure;

[0057] Figure 8 A schematic block diagram of a data synchronization apparatus applied to a second database according to an embodiment of the present disclosure is shown; and

[0058] Figure 9 A block diagram of an electronic device suitable for a data synchronization method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0059] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0060] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0061] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0062] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0063] It should be noted that the data synchronization method and data synchronization device in the embodiments of this disclosure can also be used for transaction data synchronization in the database in the financial field, and can also be used in any field other than the financial field. This disclosure does not limit the application field of the data synchronization method and data synchronization device.

[0064] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of data (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0065] With changes in payment methods and the rapid development of internet finance, online banking apps can provide users with a variety of services. For example, for transactions requiring high timeliness, such as reporting lost or stolen items via telephone banking, bank apps can provide 24 / 7 continuous service.

[0066] For developers of application software such as banking clients, upgrading and deploying the database of already released application software is necessary to provide better service or fix functional defects. To minimize the business impact on customers during application and system version deployment, banks and other enterprises typically provide services using a combination of primary and backup databases.

[0067] During routine service provision, the primary database provides services to users and synchronizes transaction data to the standby database. During primary database upgrades or maintenance, business transaction requests are routed to the standby database, which then provides services to users, ensuring service continuity. After the primary database upgrade or maintenance is completed, data generated by the standby database during the upgrade or maintenance is synchronized back to the primary database to restore the primary database to normal operation.

[0068] In related technologies, since upgrading or maintaining the primary database may involve changes at the data structure level, after the primary database is upgraded or maintained, business personnel manually synchronize the data from the backup database to the primary database. However, because the transaction volume generated by the backup database during system upgrades or maintenance is enormous, business personnel need to manually perform a large number of synchronization operations to complete the data synchronization, resulting in high data synchronization costs and long operation times.

[0069] In addition, the excessive number of manual steps involved in the data synchronization process increases the risk of data synchronization errors due to human error, which affects data synchronization efficiency and the continuity of business services provided to users.

[0070] Embodiments of this disclosure provide a data synchronization method applied to a first database. The method includes: responding to receiving a data synchronization request from a second database, parsing the data synchronization request, determining the data table update status of the second database, wherein the data table update status indicates whether the table structure of the second database has changed; acquiring incremental data generated by the first database within a preset time period, wherein the preset time period indicates the period during which the second database performs transaction operations and the first database does not perform transaction operations; determining a data synchronization method based on the data table update status; and synchronizing the incremental data to the second database based on the data synchronization method.

[0071] Figure 1 The illustration depicts an application scenario of the data synchronization method according to an embodiment of the present disclosure.

[0072] like Figure 1 As shown, application scenario 100 according to this embodiment may include terminal device 101, first database 102, and second database 103. Terminal device 101 includes first terminal device 1011, second terminal device 1012, and third terminal device 1013.

[0073] A network is a medium used to provide a communication link between terminal device 101, first database 102, and second database 103. A network can include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0074] Users can use at least one of the first terminal device 1011, the second terminal device 1012, and the third terminal device 1013 to interact with the first database 102 or the second database 103 via a network to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 1011, the second terminal device 1012, and the third terminal device 1013, such as payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0075] The first terminal device 1011, the second terminal device 1012, and the third terminal device 1013 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0076] The first database 102 and the second database 103 can be servers that provide various services, such as a backend management server (for example only) that supports service functions initiated by users using the first terminal device 1011, the second terminal device 1012, and the third terminal device 1013. The backend management server can analyze and process the received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0077] For example, during normal operation, a user initiates a business transaction using a bank client installed on terminal device 101, such as first terminal device 1011, second terminal device 1012, or third terminal device 1013. Terminal device 101 routes the business transaction to second database 103, so that second database 103 responds to the business transaction and returns the execution result to terminal device 101. At the same time, second database 103 synchronizes the data generated during the execution of the business transaction to first database 102.

[0078] During the upgrade / maintenance of the second database, a user initiates a business transaction using a bank client installed on terminal device 101. Terminal device 101 routes the business transaction to the first database 102 so that the first database 102 can respond to the transaction and return the execution result to terminal device 101. At this time, because the second database 103 is undergoing upgrade / maintenance, the first database 102 cannot synchronize data to the second database 102.

[0079] During the data synchronization period after the upgrade / maintenance of the second database, the first database 102 can still receive business transactions from the terminal device 101 and return the execution results to the terminal device. At the same time, the first database 102 can synchronize the data generated during the execution of the above-mentioned business transactions to the second database 103; the first database 102 can also synchronize the data generated by the first database 102 to the second database 103 when the second database 103 is undergoing upgrade / maintenance.

[0080] It should be noted that the data synchronization method provided in this embodiment can generally be executed by a first database and a second database. Correspondingly, the data synchronization device provided in this embodiment can generally be located in the first database and the second database. The data synchronization method provided in this embodiment can also be executed by a server or server cluster similar to the first database and the second database. Correspondingly, the data synchronization device provided in this embodiment can also be located in a server or server cluster similar to the first database and the second database.

[0081] It should be understood that Figure 1 The number of terminal devices, networks, and databases shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and databases can be included.

[0082] The following will be based on Figure 1 The described scene, through Figures 2-5 A data synchronization method applied to a first database according to a disclosed embodiment will be described in detail.

[0083] Figure 2 A flowchart illustrating a data synchronization method applied to a first database according to an embodiment of the present disclosure is shown schematically.

[0084] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0085] According to embodiments of this disclosure, the first database can be a backup database for the primary database. During system upgrades or maintenance of the primary database, application software business transactions are routed to the first database, which provides services. The second database can be the primary database. During normal operation, application software business transactions are routed to the second database, which provides services.

[0086] According to embodiments of this disclosure, the first database and the second database include distributed databases, such as MySQL databases, Oracle databases, etc. The first database or the second database can be implemented as a single database or as a database cluster.

[0087] In operation S210, in response to receiving a data synchronization request from the second database, the data synchronization request is parsed to determine the update status of the data tables in the second database.

[0088] According to embodiments of this disclosure, when the second database completes system upgrades or maintenance, the second database can send a data synchronization request to the first database to synchronize transaction data generated by the first database during system upgrades / maintenance and data synchronization to the second database.

[0089] According to embodiments of this disclosure, the data synchronization request may include information on data table updates. After receiving a data synchronization request from the second database, the first database determines the data table update status of the second database by parsing the data synchronization request.

[0090] According to embodiments of this disclosure, the data table update status indicates whether the table structure of the second database has changed. Changes in table structure include adding or removing fields from a data table, altering the calling relationships between data tables, and changing the number of data tables within the same transaction type.

[0091] For example, during a system upgrade, the second database added data validation functionality, resulting in the addition of validation fields to its tables. Consequently, the table structure of the second database changed, making it impossible to directly synchronize data table information from the first database to the second database.

[0092] According to embodiments of this disclosure, the data synchronization request may further include the time when the second database starts system upgrade / maintenance and the time when system upgrade / maintenance ends, so that the first database can verify the time.

[0093] In operation S220, incremental data generated by the first database within a preset time period is obtained.

[0094] According to embodiments of this disclosure, the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations. The preset time period includes the period during which the second database undergoes system upgrades / maintenance and the period during which data synchronization occurs after the system upgrades / maintenance are completed.

[0095] According to embodiments of this disclosure, during system upgrades / maintenance, the second database is served by the first database. At this time, the first database receives transaction requests from the application software and executes transaction operations, while the second database neither receives transaction requests nor executes transaction operations. When data synchronization is performed after the system upgrade / maintenance is complete, the second database lacks transaction data from the system upgrade / maintenance period, resulting in discontinuous transaction data. Therefore, the second database cannot provide services, and the first server continues to provide services. After copying the transaction data generated by the first database during the system upgrade / maintenance period to the second database, the first database stops providing services, and the second database begins providing services.

[0096] According to embodiments of this disclosure, during data synchronization after system upgrades / maintenance, the first database can synchronize transaction data from the second database during system upgrades / maintenance, as well as transaction data generated by the first database during the current data synchronization, to the second database. After the transaction data in the second database and the first database are fully synchronized, transaction requests from the application software will be switched back to the second database.

[0097] According to embodiments of this disclosure, incremental data may be transaction data generated by a first database in response to transaction requests from application software within a preset time period.

[0098] When operating S230, determine the data synchronization method based on the data table update status.

[0099] According to embodiments of this disclosure, data table updates include both changes in table structure and no changes in table structure. Different data synchronization methods correspond to whether the table structure has changed. Specifically, a change in table structure can correspond to at least one data synchronization method, and no change in table structure can correspond to at least one data synchronization method; the data synchronization methods differ depending on whether the table structure has changed.

[0100] In operation S240, the incremental database is synchronized to the second database based on the data synchronization method.

[0101] According to embodiments of this disclosure, after determining the data synchronization method based on whether the table structure has changed, the first database synchronizes incremental data to the second database using the determined data synchronization method.

[0102] According to embodiments of this disclosure, synchronizing incremental data to the second database indicates that all transaction data in the standby database has been synchronized to the main database, and the transaction requests of the application software can be switched from the first database back to the second database, with the second database providing the service.

[0103] The embodiments of this disclosure, in response to a received data synchronization request from a second database, parse the data synchronization request to determine the update status of the data tables in the second database; obtain the incremental data generated by the first database within a preset time period; determine the data synchronization method based on the data table update status; and synchronize the incremental database to the second database based on the data synchronization method. This eliminates the need for manual synchronization operations by business personnel, reducing synchronization costs and time. Furthermore, it avoids data synchronization errors caused by human error, improving data synchronization efficiency and preventing disruptions to business continuity. Since the first database automatically determines the data synchronization method by assessing the table structure changes in the second database, and then performs data synchronization based on the appropriate method, no business personnel intervention is required regardless of whether the table structure has changed. Therefore, data synchronization efficiency is improved.

[0104] Figure 3 The illustration depicts an application scenario of determining a data synchronization method according to an embodiment of the present disclosure.

[0105] like Figure 3 As shown, application scenario 300 includes operations S330 to S332, which can be used as a specific embodiment of operation S230.

[0106] In operation S330, determine whether the table structure of the second database has changed.

[0107] In operation S331, the data synchronization mode is determined to be the first synchronization mode. The first synchronization mode is used to synchronize incremental data to the second database through a shared disk.

[0108] In operation S332, the data synchronization mode is determined to be the second synchronization mode. The second synchronization mode is used to synchronize incremental data to the second database through the replication tool.

[0109] According to embodiments of this disclosure, the first database can obtain the data table update status of the second database by parsing the data synchronization request from the second database. Based on the data table update status, it can be determined whether the table data in the second database has changed.

[0110] According to embodiments of this disclosure, the data table may include at least one of the following: upgrade logs, maintenance logs, configuration files, or documentation.

[0111] According to an embodiment of this disclosure, after operation S330, if it is determined that the table structure of the second database has changed, operation S331 is entered, and the data synchronization method is determined to be a first synchronization method. The first synchronization method is used to synchronize incremental data to the second database via a shared disk. If it is determined that the table structure of the second database has not changed, operation S332 is entered, and the data synchronization method is determined to be a second synchronization method. The second synchronization method is used to synchronize incremental data to the second database via a replication tool.

[0112] According to embodiments of this disclosure, a shared disk can be accessed by two or more databases / systems to obtain data within the shared disk.

[0113] According to embodiments of this disclosure, files on the shared disk are updated and written to only by a single database or system; other databases or systems only have read permissions and cannot write data. Real-time sharing of incremental data between a first database, a second database, or other databases can be achieved through the shared disk.

[0114] According to embodiments of this disclosure, when a first database sends data to a second database to enable data synchronization in the second database, the first database has the permission to update and write shared data on the shared disk, while the second database only has the permission to read shared data on the shared disk in real time.

[0115] According to embodiments of this disclosure, when the table structure changes and the table structures of the second database and the first database are inconsistent, a third database with a table structure consistent with the second database can be used as a backup database during the normal service period of the second database, so as to synchronize the transaction data generated by the second database during the normal service period.

[0116] According to embodiments of this disclosure, the third database can be a newly constructed database or a database after modifying the table structure of the first database.

[0117] According to the embodiments of this disclosure, before the second database undergoes system maintenance / upgrade, the second database provides normal services. At this time, the data synchronization method between the second database and the first database is similar to the second synchronization method, in which the first database copies data from the second database using a copying tool.

[0118] After the second database has completed system maintenance / upgrades and data synchronization, it will provide normal services. At this time, similar to the second synchronization method, the third database will copy data from the second database using a replication tool to reduce the waste of computing resources caused by inconsistent table structures during data synchronization.

[0119] According to embodiments of this disclosure, synchronizing an incremental database to a second database based on a data synchronization method includes the following steps.

[0120] If the data synchronization method is determined to be the first synchronization method, the incremental data is stored in a temporary table through a trigger.

[0121] The temporary table is exported as a file, resulting in a shared file whose name conforms to the preset format.

[0122] Shared files are written to a shared disk so that a second database can read the shared files from the shared disk.

[0123] According to embodiments of this disclosure, incremental data may include insertions, deletions, and updates. When the data synchronization method is determined to be a first synchronization method, the inserted, deleted, and updated data can be obtained through one or more triggers.

[0124] According to embodiments of this disclosure, triggers include multiple types. For example, insert triggers, delete triggers, and update triggers.

[0125] For example, you can use an insert trigger to retrieve inserted data, a delete trigger to retrieve deleted data, and an update trigger to retrieve updated data. Alternatively, you can combine the insert, delete, and update triggers into a single trigger to retrieve incremental data.

[0126] According to embodiments of this disclosure, the first database can establish a temporary table and a basic service update table. A trigger captures data related to basic services generated by the first database within a preset time period using its own trigger function and stores it in the basic service update table. Then, the data is inserted into the temporary table based on the updated and unupdated values ​​of the basic services in the basic service update table.

[0127] According to embodiments of this disclosure, incremental data includes transaction data generated by basic services, but excludes data generated by non-basic services.

[0128] According to embodiments of this disclosure, basic services include services oriented towards application software, such as transaction data, query data, and account creation operations generated by users through application software. Non-basic services include operations such as data rollback within the database.

[0129] According to embodiments of this disclosure, the first database sets the non-basic service update table to SOFTFENCE protection to ensure that application software cannot update the aforementioned non-basic service data.

[0130] According to embodiments of this disclosure, a first database can write shared files to a shared disk, while a second database can only read shared files from the shared disk.

[0131] According to embodiments of this disclosure, the preset format includes: the filename of the shared file begins with a fixed string.

[0132] For example, the identifier of the first database is B, and the identifier of the second database is A. The first database B can write shared files to the shared disk. The fixed string can be TRSBTOA, indicating that the file in the shared disk is written by the first database and read by the second database.

[0133] According to embodiments of this disclosure, in order to avoid data update errors in the shared disk, the shared files are configured with a first type of file and a second type of file in the first database. The first type of file is used to manage files with preset filenames, and the second type of file is used to manage the first type of file.

[0134] For example, the first type of file can be ALIAS, used to manage files whose filenames begin with TRSBTOA, and the second type of file can be a file in UCAT format.

[0135] According to embodiments of this disclosure, shared files are named in a preset format to ensure that shared files with a fixed string as the filename can be authorized by the Resource Access Control Facility (RACF), thus avoiding update errors in the shared disk and affecting development efficiency.

[0136] According to embodiments of this disclosure, the second type of files are stored together with the shared files on a shared disk, and the second type of files are connected to a second database to ensure accurate consistency of the logs of the first and second databases. For example, the second database can only perform read-only access to the shared disk, its UCAT is stored on the shared volume along with the files, and it is connected to system A to ensure accurate consistency of the CATALOG.

[0137] The above embodiments are for illustrative purposes only. The first synchronization method includes various implementations, as long as data synchronization is achieved through a shared disk. For example, a temporary table can be stored on a shared disk, and the temporary table can be retrieved from the shared disk for data synchronization.

[0138] Figure 4 This illustration depicts an application scenario where data synchronization is performed using a first synchronization method according to an embodiment of the present disclosure.

[0139] like Figure 4 As shown, the application scenario 400 includes incremental data 401, triggers 402, temporary tables 403, shared files 404, shared disks 405, and a second database 406.

[0140] According to an embodiment of this disclosure, the first database obtains incremental data 401, uses trigger 402 to store the incremental data 401 into a temporary table 403, and exports the temporary table as a file in a preset format to obtain a shared file 404.

[0141] According to an embodiment of this disclosure, after the shared file 404 is generated, the first database writes the shared file 404 to the shared disk 405 so that the second database 406 can read the shared file 404 from the shared disk.

[0142] The embodiments of this disclosure ensure that the same data is shared by two (or more) systems through a shared disk. Since the shared disk uses a single database responsible for writing and updating, while other databases are read-only, it can improve data synchronization efficiency and reduce the number of data interactions in data synchronization scenarios with high timeliness and frequent updates.

[0143] According to embodiments of this disclosure, synchronizing an incremental database to a second database based on a data synchronization method further includes the following steps.

[0144] If the data synchronization method is determined to be the second synchronization method, obtain the transaction logs corresponding to the incremental data.

[0145] The transaction logs are sent to the replication tool so that the replication tool can generate data replay statements based on the transaction logs and transmit the data replay statements to the second database.

[0146] According to embodiments of this disclosure, if the table structure remains unchanged, the replication tool can directly synchronize incremental data in the first database that has not undergone table structure changes to the second database without performing table data conversion during this process.

[0147] According to embodiments of this disclosure, the replication tool is essentially an asynchronous replication technology based on log reading and transaction replay. Thus, assuming the table structure remains unchanged, the first database determines the transaction log corresponding to the incremental data and transmits the transaction log to the replication tool. The replication tool can generate data replay statements based on the transaction logs and transmit these statements to the second database, enabling the second database to reproduce the transaction data based on the replay statements, thereby achieving data synchronization.

[0148] According to embodiments of this disclosure, the replication tool includes the Q-REP tool, which plays a very important role in the "active-active-in-the-city" system. The normal operation of Q-REP is one of the necessary conditions to ensure data consistency and integrity.

[0149] The above embodiments are for illustrative purposes only. The second synchronization method includes various implementation methods, as long as the data synchronization is achieved through data playback technology.

[0150] Figure 5 This illustration depicts an application scenario where data synchronization is performed using a second synchronization method according to an embodiment of the present disclosure.

[0151] like Figure 5 As shown, this application scenario 500 includes incremental data 501, transaction logs 502, replication tool 503, and a second database 504.

[0152] According to an embodiment of this disclosure, a first database acquires incremental data 501 and determines a transaction log 502 corresponding to the incremental data 501. After determining the transaction log 502, the transaction log 502 is transmitted to a replication tool 503. The replication tool 503 generates a data playback statement based on the transaction log and transmits the data playback statement to a second database 504.

[0153] According to an embodiment of this disclosure, the second database 504 performs asynchronous replication by executing a data playback statement.

[0154] The following will be based on Figure 1 The described scene, through Figure 6 The data synchronization method applied to the second database according to the disclosed embodiments will be described in detail.

[0155] Figure 6 A flowchart illustrating a data synchronization method applied to a second database according to an embodiment of the present disclosure is shown schematically.

[0156] like Figure 6 As shown, the method 600 includes operations S610 to S630.

[0157] In operation S610, a data synchronization request is sent to the first database. The data synchronization request includes updates to the data tables in the second database.

[0158] When operating S620, determine the data synchronization method based on the data table update status.

[0159] When operating S630, based on the data synchronization method, the incremental data generated by the first database is synchronized to the first data table.

[0160] According to embodiments of this disclosure, the data table update status indicates whether the table structure of the second database has changed.

[0161] According to embodiments of this disclosure, after completing system maintenance / upgrade, the second database can send a data synchronization request to the first database so that the first database can begin synchronizing data with the second database.

[0162] According to embodiments of this disclosure, the second database determines the data synchronization method based on data table updates generated during system maintenance / upgrades, in order to determine how to obtain incremental data from the first database.

[0163] According to embodiments of this disclosure, incremental data is generated by the first database within a preset time period. The preset time period represents the period during which the first database performs transaction operations while the second database does not. The preset time period includes the period during which the second database undergoes system upgrades / maintenance and the period during which data synchronization occurs after the system upgrades / maintenance are completed.

[0164] The embodiments of this disclosure send a data synchronization request to a first database. The data synchronization request includes updates to the data tables in a second database, indicating whether the table structure of the second database has changed. Based on the data table updates, a data synchronization method is determined. Based on the data synchronization method, incremental data generated by the first database is synchronized to the first data table. Data synchronization can be achieved without manual synchronization operations by business personnel, reducing synchronization costs and time. It avoids data synchronization errors caused by human error, improves data synchronization efficiency, and avoids impacting business continuity.

[0165] Furthermore, the second database automatically determines the data synchronization method based on changes to its own table structure, and then obtains incremental data from different sources based on the appropriate method to achieve data synchronization. Regardless of whether the table structure changes, data synchronization on the second database side does not require the intervention of business personnel, thus improving data synchronization efficiency.

[0166] According to embodiments of this disclosure, determining the data synchronization method based on data table updates includes the following steps.

[0167] If it is determined that the table structure of the second database has changed, the data synchronization method is determined to be the first synchronization method. The first synchronization method is used to synchronize incremental data to the first data table of the second database through a shared disk.

[0168] If the table structure of the second database remains unchanged, the data synchronization method is determined to be the second synchronization method. The second synchronization method is used to synchronize incremental data to the first data table of the second database through a replication tool.

[0169] As one specific embodiment, the shared disk can store shared files corresponding to the incremental data.

[0170] According to embodiments of this disclosure, a shared disk can be accessed by two or more database systems to retrieve data within the shared disk. Since the second database is the database to be synchronized, it can only retrieve the shared files corresponding to the incremental data from the shared disk.

[0171] According to embodiments of this disclosure, a shared disk can enable real-time sharing of 55KM files between two or more databases.

[0172] According to embodiments of this disclosure, synchronizing incremental data generated by a first database to a first data table based on a data synchronization method includes the following steps.

[0173] If the data synchronization method is determined to be the first synchronization method, the shared disk is scanned and the shared file is obtained. The shared file is determined based on the incremental data generated by the first database within a preset time period. The preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations.

[0174] Retrieve database table transformation rules, which include transformation rules for multiple types of database operations;

[0175] Based on the database table conversion rules and shared files, determine the second data table that matches the table structure of the second database; and

[0176] Insert the second database table into the first database table.

[0177] According to embodiments of this disclosure, when the data synchronization method is determined to be the first synchronization method via a shared disk, the second database can periodically scan the shared disk to determine whether shared files have been stored on the shared disk.

[0178] For example, the second database can periodically scan the shared disk using a polling thread, and if a shared file is found to exist on the shared disk, it can retrieve the shared file.

[0179] Alternatively, the second database can activate a detection component to detect shared disks in real time, and retrieve shared files if they are found to exist on the shared disk.

[0180] According to embodiments of this disclosure, the shared file is written to the shared disk by the first database through a first synchronization method. The shared file is determined based on incremental data generated by the first database within a preset time period; for example, the shared file may be incremental data in the form of a preset filename.

[0181] According to embodiments of this disclosure, incremental data is generated within a preset time period, which represents the period during which the first database performs transaction operations and the second database does not perform transaction operations. For example, the preset time period may be during the system upgrade / maintenance of the second database, and during the data synchronization period after the system upgrade / maintenance is completed.

[0182] According to embodiments of this disclosure, since the table structure of the second database has changed, the table results between the first database and the second database are different, and data synchronization cannot be performed directly. It is necessary to convert the table data through database table conversion rules.

[0183] According to embodiments of this disclosure, the data table transformation rules include transformation rules for multiple types of database operations. These database operation types include adding fields, deleting fields, modifying fields, and renaming fields.

[0184] For operations that modify fields, the field names of the corresponding data values ​​in the database table may change; for operations that add fields, the data tables in the first database remain unchanged, and there are no new fields or data values ​​for them; for operations that delete fields, the data tables in the first database remain unchanged, but the deleted fields still exist, and during data synchronization, the deleted fields cannot be inserted into the data tables of the second database; for operations that rename fields, although the renamed data values ​​in the data tables do not change, the first database cannot determine from its data tables which field was the same as the one modified by the second database.

[0185] According to embodiments of this disclosure, data table conversion rules can be determined based on data table updates in the second database. Data table updates include data types and the specific field names that have changed. Data table conversion rules can be invoked based on the data types and specific field names.

[0186] According to embodiments of this disclosure, the data table conversion rules include database operation types, which can be further subdivided into one or more processing types, each processing type corresponding to a processing method.

[0187] For example, when the operation type is to add a field, the processing types include adding a field to the table structure, assigning a constant value to the added field, and assigning values ​​to other fields in the table to the added field.

[0188] When the operation type is "delete field", the processing type includes deleting fields from the table structure.

[0189] When the operation type is "modify field", the processing type includes "modify field type in table structure", "modify field length in table structure", and "modify field length in table structure".

[0190] When the operation type is to rename a field, the processing options are: renaming the type field, assigning a constant value to the renamed field, and assigning a value to another field in the table to the renamed field.

[0191] According to embodiments of this disclosure, for processing types involving newly added fields in table structures, the processing method is as follows: when exporting the newly added field, directly write the WITH DEFAULT field. For example, for data table A in the first database, which only includes fields COL1 and COL2, when it is determined that the table structure change is the addition of a field COL3 CHAR(8) NOT NULL, when exporting the file, select "SELECT COL1, COL2, CHAR(REPEAT(", 8)) AS COL3 FROM A"; COL3 is equivalent to writing 8 spaces.

[0192] For fields that are newly added and assigned constant values, the processing method is as follows: when exporting the newly added field, directly assign a constant value. For example, if a new field COL3 is added with DECIMAL(7,0) NOT NULL WITH DEFAULT 0, and the query is "SELECT COL1,COL2,COL3FROM A", then all newly added COL3 fields will be assigned the default value of 0.

[0193] For processing types involving adding a new field and assigning values ​​to other fields within the table, the processing method is as follows: when exporting the new field, treat the other fields as the new field. For example, if a new field COL3 is added, assign the value of COL2 to COL3, and when exporting the file, select "SELECT COL1, COL2, COL2 AS COL3 FROM A".

[0194] For processing table-structured field type modifications, the method is to use database conversion functions to change the original field to the target field, such as using SQL functions. For example, to change COL2 to DATE type, use the formula: "SELECT COL1,DATE(COL2)AS COL2 FROM A".

[0195] For cases where the field length changes due to table structure modifications, the approach is to pad the left and right sides according to the required rules. For example, if COL2 becomes longer, pad the left side with 0s: "SELECT COL1,RIGHT('00000'+CAST(COL2)FROM A".

[0196] For cases where table structure modifications shorten field lengths, the approach is to use functions like `substr` to truncate the required fields. For example, to truncate COL2 from the third character, use the following formula: `SELECT COL1,SUBSTR(COL2,3,1)FROM A`.

[0197] For fields deleted from a table structure, the processing method is to discard the deleted field directly when selecting the record table field. For example, to delete COL2, the statement is "SELECT COL1 FROM A".

[0198] For fields that are renamed, the processing method is as follows: when exporting the old table, directly change the old field names to the new field names. For example, rename COL2 to COL3, assign COL2 to COL3, and select "SELECT COL1, COL2 AS COL3 FROM A" when exporting the file.

[0199] For fields that are renamed and assigned constant values, the processing method is as follows: when exporting the old table, directly modify the old field names to the new field names and assign constant values ​​directly. For example, rename COL2 to COL3 and assign 8 spaces: "SELECT COL1,CHAR(REPEAT(",8))AS COL3 FROM A".

[0200] For processing types involving renaming fields and assigning values ​​to other fields within the table, the approach is as follows: when exporting the old table, directly rename the old field names to the new field names and directly assign values ​​to other fields. For example, rename COL2 to COL3 and assign COL1 to COL3, then select "SELECT COL1,COL1 AS COL3 FROM A" when exporting the file.

[0201] According to embodiments of this disclosure, after obtaining the database table conversion rules and the shared file, the table structure of the shared file is converted using the processing method within the database table conversion rules, that is, a second database table matching the table structure of the second database is generated.

[0202] According to embodiments of this disclosure, after the second database undergoes table structure changes, it converts historical data into data adapted to the transformed table structure, generating a first data table. After generating the second database table, it is inserted into the first data table, thus achieving synchronization between historical data in the second database and incremental data during periods of trading halt in the second database.

[0203] According to embodiments of this disclosure, the incremental data generated by the first database is synchronized to the first data table based on a data synchronization method, and the method further includes the following steps.

[0204] When the data synchronization method is determined to be the second synchronization method, a data playback statement is received from the replication tool. The data playback statement is used to replay the operation of generating incremental data in the first database.

[0205] Execute the data replay statement to generate a third data table; and

[0206] Import the third data table into the first data table.

[0207] According to embodiments of this disclosure, when the data synchronization method is determined to be the second synchronization method via a replication tool, the second database can receive data playback statements generated by the replication tool through an interface.

[0208] According to embodiments of this disclosure, the data replay statement includes all operations of generating the incremental database from the first database. For example, the operation sequence for generating incremental data in the first database is "query - create user - transact with user S - transact with user T". Based on the above operation sequence, the generated data replay statement includes "database statement for query operation - database statement for creating user - database statement for transacting with user S - database statement for transacting with user T". After execution according to the above data replay statement sequence, a third data table corresponding to the above data replay statement is generated in the second database. Inserting the third data table into the first data table achieves data synchronization within the second database.

[0209] According to embodiments of this disclosure, since the amount of transaction data generated by the first database during system maintenance / upgrades is very large, in order to improve data synchronization efficiency, the second database begins to retrieve execution statements from the replication tool to initiate data synchronization after a predetermined time before switching the transaction request from the first database back to the second database. For example, the reverse replication tool is started 40 minutes in advance so that the second database can copy data from the first database. When the data catch-up exceeds the basic service transaction time point, the transaction request is switched from the first database back to the second database.

[0210] Figure 7 A schematic block diagram of a data synchronization device applied to a first database according to an embodiment of the present disclosure is shown.

[0211] like Figure 7 As shown, the data synchronization device 700 applied to the first database in this embodiment includes a parsing module 710, an acquisition module 720, a first determination module 730, and a first synchronization module 740.

[0212] The parsing module 710 is used to respond to a received data synchronization request from the second database, parse the data synchronization request, and determine the update status of the data tables in the second database. The data table update status indicates whether the table structure of the second database has changed. In one embodiment, the parsing module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0213] The acquisition module 720 is used to acquire incremental data generated by the first database within a preset time period, where the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations. In one embodiment, the acquisition module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0214] The first determining module 730 is used to determine the data synchronization method based on the data table update status. In one embodiment, the first determining module 730 can be used to execute the operation S230 described above, which will not be repeated here.

[0215] The first synchronization module 740 is used to synchronize the incremental database to the second database based on a data synchronization method. In one embodiment, the first synchronization module 740 can be used to perform the operation S240 described above, which will not be repeated here.

[0216] According to embodiments of this disclosure, the first determining module 730 includes a first determining unit and a second determining unit.

[0217] The first determining unit is used to determine the data synchronization method as a first synchronization method when it is determined that the table structure of the second database has changed. The first synchronization method is used to synchronize incremental data to the second database through a shared disk. In one embodiment, the first determining unit can be used to perform the operation S331 described above, which will not be repeated here.

[0218] The second determining unit is used to determine the data synchronization method as the second synchronization method when the table structure of the second database has not changed. The second synchronization method is used to synchronize incremental data to the second database through a replication tool. In one embodiment, the second determining unit can be used to perform the operation S332 described above, which will not be repeated here.

[0219] According to embodiments of this disclosure, the first synchronization module 740 includes a first synchronization unit, a second synchronization unit, and a third synchronization unit.

[0220] The first synchronization unit is used to store incremental data to a temporary table via a trigger when the data synchronization mode is determined to be the first synchronization mode.

[0221] The second synchronization unit is used to export the temporary table as a file to obtain a shared file, and the shared file is named in accordance with a preset format.

[0222] The third synchronization unit is used to write shared files to a shared disk so that the second database can read the shared files from the shared disk.

[0223] According to embodiments of this disclosure, the first synchronization module 740 includes a fourth synchronization unit and a fifth synchronization unit.

[0224] The fourth synchronization unit is used to obtain the transaction logs corresponding to the incremental data when the data synchronization method is determined to be the second synchronization method.

[0225] The fifth synchronization unit is used to send transaction logs to the replication tool, so that the replication tool can generate data playback statements based on the transaction logs and transmit the data playback statements to the second database.

[0226] According to embodiments of this disclosure, any and more modules among the parsing module 710, the acquisition module 720, the first determination module 730, and the first synchronization module 740 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module.

[0227] According to embodiments of this disclosure, at least one of the parsing module 710, the acquisition module 720, the first determination module 730, and the first synchronization module 740 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable method of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three methods. Alternatively, at least one of the parsing module 710, the acquisition module 720, the first determination module 730, and the first synchronization module 740 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0228] Figure 8 A schematic block diagram of a data synchronization device applied to a second database according to an embodiment of the present disclosure is shown.

[0229] like Figure 8 As shown, the data synchronization device 800 applied to the second database in this embodiment includes a sending module 810, a second determining module 820, and a second synchronization module 830.

[0230] The sending module 810 is used to send a data synchronization request to the first database. The data synchronization request includes information on the update status of the data tables in the second database, indicating whether the table structure of the second database has changed. In one embodiment, the sending module 810 can be used to perform the operation S610 described above, which will not be repeated here.

[0231] The second determining module 820 is used to determine the data synchronization method based on the data table update status. In one embodiment, the second determining module 820 can be used to perform the operation S620 described above, which will not be repeated here.

[0232] The second synchronization module 830 is used to synchronize incremental data generated by the first database to the first data table based on a data synchronization method. In one embodiment, the second synchronization module 830 can be used to perform the operation S630 described above, which will not be repeated here.

[0233] According to embodiments of this disclosure, the second determining module 820 includes a third determining unit and a fourth determining unit.

[0234] The third determining unit is used to determine the data synchronization method as the first synchronization method when it is determined that the table structure of the second database has changed. The first synchronization method is used to synchronize incremental data to the first data table of the second database through a shared disk.

[0235] The fourth determining unit is used to determine the data synchronization method as the second synchronization method when the table structure of the second database has not changed. The second synchronization method is used to synchronize incremental data to the first data table of the second database through a replication tool.

[0236] According to embodiments of this disclosure, the second synchronization module 830 includes a sixth synchronization unit, a seventh synchronization unit, an eighth synchronization unit, and a ninth synchronization unit.

[0237] The sixth synchronization unit is used to scan the shared disk and obtain the shared file when the data synchronization method is determined to be the first synchronization method. The shared file is determined based on the incremental data generated by the first database within a preset time period. The preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations.

[0238] The seventh synchronization unit is used to obtain database table transformation rules, which include transformation rules for multiple types of database operations.

[0239] The eighth synchronization unit is used to determine the second data table that matches the table structure of the second database based on the database table conversion rules and shared files.

[0240] The ninth synchronization unit is used to insert the second database table into the first data table.

[0241] According to embodiments of this disclosure, the second synchronization module 830 further includes a tenth synchronization unit, an eleventh synchronization unit, and a twelfth synchronization unit.

[0242] The tenth synchronization unit is used to receive data playback statements from the replication tool when the data synchronization mode is determined to be the second synchronization mode. The data playback statements are used to replay the operation of generating incremental data in the first database.

[0243] The eleventh synchronization unit is used to execute data playback statements and generate the third data table.

[0244] The twelfth synchronization unit is used to import the third data table into the first data table.

[0245] According to embodiments of this disclosure, any plurality of modules among the sending module 810, the second determining module 820, and the second synchronization module 830 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules may be combined with at least some of the functions of other modules and implemented in one module.

[0246] According to embodiments of this disclosure, at least one of the transmitting module 810, the second determining module 820, and the second synchronization module 830 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable method of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three methods. Alternatively, at least one of the transmitting module 810, the second determining module 820, and the second synchronization module 830 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0247] Figure 9 A block diagram of an electronic device suitable for a data synchronization method according to an embodiment of the present disclosure is shown schematically.

[0248] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0249] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0250] According to embodiments of this disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0251] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0252] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0253] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0254] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0255] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0256] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0257] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0258] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0259] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0260] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this disclosure. It should be understood that the above descriptions are merely specific embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A data synchronization method applied to a first database, the method comprising: In response to receiving a data synchronization request from a second database, the data synchronization request is parsed to determine the data table update status of the second database, wherein the data table update status indicates whether the table structure of the second database has changed; Acquire incremental data generated by the first database within a preset time period, wherein the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations; Based on the update status of the data table, determine the data synchronization method; as well as Based on the aforementioned data synchronization method, the incremental database is synchronized to the second database; The step of determining the data synchronization method based on the data table update includes: when it is determined that the table structure of the second database has changed, the data synchronization method is determined to be the first synchronization method. The first synchronization method is used to synchronize the incremental data to the second database through a shared disk. The first database has the permission to update and write the shared data in the shared disk, while the second database only has the permission to read the shared data in the shared disk in real time. After the incremental data is stored in a temporary table, the temporary table is exported in the form of a file to obtain a shared file. The shared file has a first type of file and a second type of file in the first database. The first type of file is used to manage files with preset file names, and the second type of file is used to manage the first type of file. The second type of file is connected to the second database to ensure that the logs of the first database and the second database are accurate and consistent. The step of synchronizing the incremental database to the second database based on the data synchronization method includes: when the data synchronization method is determined to be the first synchronization method, storing the incremental data to a temporary table through a trigger; exporting the temporary table as a file to obtain a shared file, wherein the naming of the shared file conforms to a preset format; and writing the shared file to a shared disk so that the second database can read the shared file from the shared disk.

2. The method according to claim 1, wherein, The step of determining the data synchronization method based on the data table update status includes: If the table structure of the second database remains unchanged, the data synchronization method is determined to be the second synchronization method, which is used to synchronize the incremental data to the second database through a replication tool.

3. The method according to claim 2, wherein, The step of synchronizing the incremental database to the second database based on the data synchronization method further includes: If the data synchronization method is determined to be the second synchronization method, the transaction log corresponding to the incremental data is obtained. The transaction logs are sent to the replication tool so that the replication tool can generate a data playback statement based on the transaction logs and transmit the data playback statement to the second database.

4. A data synchronization method applied to a second database, the method comprising: Send a data synchronization request to the first database. The data synchronization request includes the data table update status of the second database. The data table update status indicates whether the table structure of the second database has changed. Based on the update status of the data table, determine the data synchronization method; as well as Based on the data synchronization method, the incremental data generated by the first database is synchronized to the first data table; The step of determining the data synchronization method based on the data table update includes: when it is determined that the table structure of the second database has changed, the data synchronization method is determined to be the first synchronization method. The first synchronization method is used to synchronize the incremental data to the second database through a shared disk. The first database has the permission to update and write the shared data in the shared disk, while the second database only has the permission to read the shared data in the shared disk in real time. After the incremental data is stored in a temporary table, the temporary table is exported in the form of a file to obtain a shared file. The shared file has a first type of file and a second type of file in the first database. The first type of file is used to manage files with preset file names, and the second type of file is used to manage the first type of file. The second type of file is connected to the second database to ensure that the logs of the first database and the second database are accurate and consistent. The step of synchronizing incremental data generated by the first database to the first data table based on the data synchronization method includes: when the data synchronization method is determined to be the first synchronization method, scanning the shared disk and obtaining a shared file, wherein the shared file is determined based on the incremental data generated by the first database within a preset time period, wherein the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations; obtaining database table conversion rules, wherein the data table conversion rules include conversion rules for multiple types of database operations; determining a second data table that matches the table structure of the second database based on the database table conversion rules and the shared file; and inserting the second database table into the first data table.

5. The method according to claim 4, wherein, The step of determining the data synchronization method based on the data table update status includes: If it is determined that the table structure of the second database has not changed, the data synchronization method is determined to be the second synchronization method. The second synchronization method is used to synchronize the incremental data to the first data table of the second database through a replication tool.

6. The method according to claim 5, wherein, The step of synchronizing incremental data generated by the first database to the first data table based on the data synchronization method further includes: If the data synchronization method is determined to be the second synchronization method, a data playback statement is received from the replication tool. The data playback statement is used to replay the operation of the first database generating the incremental data. Execute the data playback statement to generate a third data table; and Import the third data table into the first data table.

7. A data synchronization device applied to a first database, the device comprising: The parsing module is used to respond to a data synchronization request received from a second database, parse the data synchronization request, and determine the data table update status of the second database, wherein the data table update status indicates whether the table structure of the second database has changed; The acquisition module is used to acquire incremental data generated by the first database within a preset time period, wherein the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations; The first determining module is used to determine the data synchronization method based on the update status of the data table; as well as The first synchronization module is used to synchronize the incremental database to the second database based on the data synchronization method. The first determining module is further configured to: determine the data synchronization method as the first synchronization method when it is determined that the table structure of the second database has changed. The first synchronization method is used to synchronize the incremental data to the second database through a shared disk. The first database has the permission to update and write the shared data in the shared disk, while the second database only has the permission to read the shared data in the shared disk in real time. After the incremental data is stored in a temporary table, the temporary table is exported in the form of a file to obtain a shared file. The shared file has a first type of file and a second type of file in the first database. The first type of file is used to manage files with preset file names, and the second type of file is used to manage the first type of file. The second type of file is connected to the second database to ensure that the logs of the first database and the second database are accurate and consistent. The first synchronization module is further configured to: when the data synchronization method is determined to be the first synchronization method, store the incremental data to a temporary table through a trigger; export the temporary table as a file to obtain a shared file, wherein the naming of the shared file conforms to a preset format; The shared file is written to the shared disk so that the second database can read the shared file from the shared disk.

8. A data synchronization device, applied to a second database, the device comprising: The sending module is used to send a data synchronization request to the first database. The data synchronization request includes the data table update status of the second database, and the data table update status indicates whether the table structure of the second database has changed. The second determining module is used to determine the data synchronization method based on the update status of the data table; as well as The second synchronization module is used to synchronize incremental data generated by the first database to the first data table based on the data synchronization method. The second determining module is further configured to: when it is determined that the table structure of the second database has changed, determine the data synchronization method as the first synchronization method, wherein the first synchronization method is used to synchronize the incremental data to the second database through the shared disk, the first database has the permission to update and write the shared data in the shared disk, and the second database only has the permission to read the shared data in the shared disk in real time; After the incremental data is stored in a temporary table, the temporary table is exported in the form of a file to obtain a shared file. The shared file has a first type of file and a second type of file in the first database. The first type of file is used to manage files with preset file names, and the second type of file is used to manage the first type of file. The second type of file is connected to the second database to ensure that the logs of the first database and the second database are accurate and consistent. The second synchronization module is further configured to: when the data synchronization method is determined to be the first synchronization method, scan the shared disk and obtain the shared file, wherein the shared file is determined based on the incremental data generated by the first database within a preset time period, wherein the preset time period represents the period during which the first database performs transaction operations and the second database does not perform transaction operations; Obtain database table conversion rules, which include conversion rules for multiple types of database operations; determine a second data table that matches the table structure of the second database based on the database table conversion rules and the shared file; And insert the second database table into the first data table.

9. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 6.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images