Synchronous database writing methods, methods for obtaining change information, and their respective devices

Abstract

This specification provides embodiments of a database synchronous writing method, a method for obtaining change information, and respective apparatuses. The database synchronous writing method includes: obtaining database change information generated in response to changes in a source database; extracting key information of a source table from the database change information; determining a target table corresponding to the key information in a preset table mapping rule based on the key information of the source table; and extracting data from the database change information and writing it into the target table of the target database based on the structure information of the target table.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method for synchronously writing databases. One or more embodiments of this specification also relate to a database synchronous writing apparatus, a method for obtaining change information, an apparatus for obtaining change information, a computing device, and a computer-readable storage medium. Background Technology

[0002] Synchronous database writing refers to the real-time synchronization of data from a source database to a target database. For example, synchronous writing from a source database to a data warehouse is a strong requirement. A typical enterprise may have hundreds to thousands of databases, each with dozens to hundreds of tables, involving complex table structures such as database sharding and partitioning.

[0003] When changes are made to the source database, users need to configure the write process for each table, and each table has a corresponding real-time write task. However, enterprises have many databases and tables, resulting in a huge number of tasks. Creation, configuration, operation, and monitoring are extremely complex, and the large number of real-time write tasks consumes a lot of resources. When accumulated, the resource consumption is very large, resulting in low synchronization efficiency. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a method for synchronously writing to a database. One or more embodiments of this specification also relate to a device for synchronously writing to a database, a method for obtaining change information, a device for obtaining change information, a computing device, and a computer-readable storage medium, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a method for synchronously writing to a database is provided, comprising: acquiring database change information generated in response to changes in a source database; extracting key information of a source table from the database change information; determining a target table corresponding to the key information in a preset table mapping rule based on the key information of the source table, wherein the table mapping rule includes a correspondence between different key information and each target table; and extracting data from the database change information and writing it to the target table in the target database based on the structure information of the target table.

[0006] Optionally, it further includes: in response to the write interface being called, obtaining the interface parameters passed in when the write interface is called, the interface parameters including information for determining the message format of the database change information; and determining the message format of the database change information according to the interface parameters.

[0007] Optionally, the interface parameters include type information of the data synchronization middleware; the database change information is a database change event generated by the data synchronization middleware in response to changes in the source database; determining the message format of the database change information according to the interface parameters includes: determining the message format of the database change event according to the type of the data synchronization middleware determined by the interface parameters.

[0008] Optionally, determining the target table corresponding to the key information in a preset table mapping rule based on the key information of the source table includes: determining the corresponding target table based on the table name of the source table and the mapping relationship between the table name of the source table and the table name of the target table contained in the table mapping rule, wherein the key information is the table name; or, determining the corresponding target table based on the mapping relationship between the key fields of the source table and the key fields contained in the table mapping rule and the table name of the target table, wherein the key information is the key field; or, determining the corresponding target table based on the mapping relationship between the key information of the source table and the writing conditions that the key information must satisfy contained in the table mapping rule and the table name of the target table.

[0009] Optionally, obtaining database change information in response to changes in the source database includes: obtaining database change events generated by the data synchronization middleware in response to changes in the source database; or, obtaining database change events from a message queue, wherein the database change events are delivered to the message queue by the data synchronization middleware in response to changes in the source database.

[0010] Optionally, it further includes: receiving a user's request to set table mapping rules; and writing the mapping relationship between the source table key information and the target table carried in the setting request into the table mapping rules.

[0011] Optionally, it also includes: in response to a failure to write data to the target database, writing the dirty data generated by the failure to the failure storage area.

[0012] Optionally, it also includes: in response to a failure in writing data to the target database, sending corresponding fault information to the user of the target database.

[0013] Optionally, it further includes: calculating the synchronization health index from the source database to the target database based on the relevant data of the fault in the target database.

[0014] Optionally, it also includes: triggering the user's user-side pop-up fault interaction interface to display the fault information and / or the synchronized health indicators.

[0015] According to a second aspect of the embodiments of this specification, a database synchronous writing apparatus is provided, comprising: a change information acquisition module configured to acquire database change information generated in response to changes in a source database; a key information acquisition module configured to extract key information of a source table from the database change information; a target table determination module configured to determine, based on the key information of the source table, a target table corresponding to the key information in a preset table mapping rule, wherein the table mapping rule includes a correspondence between different key information and each target table; and a data writing module configured to extract data from the database change information and write it to the target table in the target database based on the structure information of the target table.

[0016] According to a third aspect of the embodiments of this specification, a method for obtaining change information is provided, applied to a data synchronization middleware, comprising: responding to an update of a log file of a source database, generating database change information based on the update of the log file of the source database; sending the database change information to a database synchronization write side, enabling the database synchronization write side to obtain the database change information; extracting key information of a source table from the database change information; determining a target table corresponding to the key information in a preset table mapping rule based on the key information of the source table, wherein the table mapping rule contains the correspondence between different key information and each target table; and extracting data from the database change information and writing it into the target table of the target database based on the structure information of the target table.

[0017] According to a fourth aspect of the embodiments of this specification, an apparatus for obtaining change information is provided, configured in a data synchronization middleware, comprising: a log update response module configured to respond to an update of a log file of a source database, and generate database change information based on the update of the log file of the source database; and a change information sending module configured to send the database change information to a database synchronization write side, enabling the database synchronization write side to obtain the database change information, extract key information of a source table from the database change information, determine the target table corresponding to the key information in a preset table mapping rule based on the key information of the source table, the table mapping rule containing the correspondence between different key information and each target table, and extract data from the database change information and write it to the target table of the target database based on the structure information of the target table.

[0018] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: a memory and a processor; the memory being configured to store computer-executable instructions, and the processor being configured to execute the computer-executable instructions: acquiring database change information generated in response to changes in a source database; extracting key information of a source table from the database change information; determining a target table corresponding to the key information in a preset table mapping rule based on the key information of the source table, the table mapping rule including a correspondence between different key information and each target table; extracting data from the database change information and writing it into the target table of the target database based on the structure information of the target table; or, the processor... The processor executes the computer-executable instructions: in response to an update of the source database's log file, it generates database change information based on the update of the source database's log file; it sends the database change information to the database synchronous write side, enabling the database synchronous write side to obtain the database change information; it extracts key information of the source table from the database change information; based on the key information of the source table, it determines the target table corresponding to the key information in a preset table mapping rule, wherein the table mapping rule contains the correspondence between different key information and each target table; and based on the structure information of the target table, it extracts data from the database change information and writes it to the target table in the target database.

[0019] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the synchronous writing method of the database described in any embodiment of this specification, or the steps of the method for obtaining change information described in any embodiment of this specification.

[0020] This specification provides an embodiment of a database synchronous write method. This method acquires database change information in response to changes in the source database, extracts key information of the source table from the database change information, and determines the corresponding target table in a preset table mapping rule based on the key information of the source table. Furthermore, the table mapping rule contains the correspondence between different key information and each target table. Therefore, the target table can be dynamically routed using the table name information in the database table change event and the table mapping rule. Moreover, based on the structure information of the target table, the required data is extracted from the database change information and written to the target table in the target database. Therefore, this method eliminates the need for users to configure write processes for each table, using a unified write process externally. It dynamically selects the target table using the key information of the source table contained in the database change information, and only requires managing the table mapping rule as needed to achieve real-time data writing of the entire database or multiple databases. It is evident that this method significantly reduces the number of processes (tasks) that users need to manage, facilitating management, maintenance, and monitoring, greatly saving resources, and effectively improving synchronization efficiency. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a synchronous database writing method provided in one embodiment of this specification;

[0022] Figure 2 This is a flowchart illustrating the processing procedure of a database synchronous write method according to one embodiment of this specification;

[0023] Figure 3 This is a schematic diagram of the structure of a database synchronous writing device provided in one embodiment of this specification;

[0024] Figure 4 This is a schematic diagram of the structure of a database synchronous writing device provided in another embodiment of this specification;

[0025] Figure 5 This is a flowchart illustrating a method for obtaining change information according to one embodiment of this specification;

[0026] Figure 6 This is a schematic diagram of a device for obtaining change information according to one embodiment of this specification;

[0027] Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0028] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0029] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0031] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0032] A database is a collection of databases (DBs) within a single database instance. Each DB is a combination of tables.

[0033] A table is a relational entity used to store structured data.

[0034] A data warehouse is a system primarily focused on data analysis. The data source can be database data or other log-like data.

[0035] Database change information is information generated in response to real-time changes in the source database, such as database change events generated by data synchronization middleware in response to changes in the source database.

[0036] This specification provides a method for synchronously writing to a database, and also relates to a device for synchronously writing to a database, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0037] Figure 1 A flowchart of a synchronous write method for a database according to an embodiment of this specification is shown, including steps 102 to 108.

[0038] Step 102: Obtain database change information in response to changes in the source database.

[0039] Database change information can be database change events generated by data synchronization middleware. Data synchronization middleware collects real-time changes from the source database and then delivers them as events to a message queue or downstream system. Therefore, data synchronization middleware can interface with real-time database synchronization, such as from an OLTP database to an OLAP data warehouse. When delivering database change events to the message queue, the data synchronization middleware allows users to flexibly configure a topic to store real-time changes to a single source table or to store real-time changes to the entire database.

[0040] Step 104: Extract key information from the source table from the database change information.

[0041] The key information may include table name, fields, etc.

[0042] Step 106: Based on the key information of the source table, determine the target table corresponding to the key information in the preset table mapping rules. The table mapping rules include the correspondence between different key information and each target table.

[0043] The table mapping rules can be configured according to the needs of the implementation scenario. For example, they can be mapping rules between several table names in the source database and several table names in the target database, mapping rules between several fields in the source database and several table names in the target database, or mapping rules between different write conditions corresponding to fields or table names in the source database and several table names in the target database.

[0044] Step 108: Based on the structure information of the target table, extract data from the database change information and write it into the target table of the target database.

[0045] Understandably, database change information is generated in response to changes in the source database, and it contains change-related data, such as specific information about the changed tables and fields.

[0046] The target table's structure information, also known as the schema, describes the form's structure. Based on this structure information, it can be determined which data needs to be written to the target table, and then the corresponding data can be extracted from the database change information and written to the target table. Therefore, in the method provided in this specification's embodiments, the source database and the target database can be databases with different structures. For example, if the database change information contains data for fields A, C, and D, and according to the target table's structure information, the target table contains fields A and B, then field A can be extracted from the database change information, and the data for field A can be written to the target table.

[0047] This method acquires database change information in response to changes in the source database, extracts key information about the source tables from this information, and determines the corresponding target table in a pre-defined table mapping rule based on this key information. Specifically, it dynamically routes the target table using the table name information and table mapping rules from the database table change event. Furthermore, it extracts the necessary data from the database change information based on the target table's structure and writes the extracted data to the target table in the target database. Therefore, this method eliminates the need for users to configure write processes for each table. It uses a unified write process externally, dynamically selecting the target table using the key information from the source tables contained in the database change information. Users only need to manage the table mapping rules as needed, and a single interface can be used to achieve real-time data writing to the entire database or multiple databases. As can be seen, this method significantly reduces the number of processes and tasks that users need to manage, facilitating management, maintenance, and monitoring, greatly saving resources, and effectively improving synchronization efficiency.

[0048] It is understood that before extracting key information from the source table from the database change information, the message format of the database change information can be determined as needed to extract information from it. The method provided in the embodiments of this specification is not limited to the implementation of determining the message format. For example, the message format can be determined by analyzing the database change information using preset message format identification rules. Alternatively, the message format can be determined by receiving parameters input by the user.

[0049] To more efficiently determine the message format of database change information, in one or more embodiments of this specification, in response to a write interface being called, the interface parameters passed when the write interface is called can be obtained. These interface parameters contain information for determining the message format of the database change information; the message format of the database change information is then determined based on these interface parameters. In this embodiment, a unified write interface is used externally, and the message format of the incoming database change information is determined through the write interface parameters. This allows for the extraction of key information and data from the source table based on the message format.

[0050] Since database change information is a database change event generated by the data synchronization middleware in response to changes in the source database, the message format of the database change event generated can be determined based on the type of the data synchronization middleware. In one or more embodiments of this specification, the interface parameters include the type information of the data synchronization middleware, thereby determining the message format of the database change event based on the type of the data synchronization middleware determined by the interface parameters.

[0051] It should be noted that the specific implementation of the table mapping rules described in the embodiments of this specification is not limited. The mapping relationship between key information of the source table and the target table in the table mapping rules can be set according to the needs of the implementation scenario.

[0052] For example, determining the target table corresponding to the key information in a preset table mapping rule based on the key information of the source table may include: determining the corresponding target table based on the table name of the source table and the mapping relationship between the source table name and the target table name contained in the table mapping rule, where the key information is the table name. In this embodiment, the target table is dynamically routed through the table name information of the source table, thus eliminating the need for the user to configure the write process for each table. Only the mapping relationship between the source table name and the target table in the table mapping rule needs to be adjusted, effectively improving synchronization efficiency.

[0053] For example, determining the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table may include: determining the corresponding target table based on the mapping relationship between the key fields of the source table and the key fields contained in the table mapping rules and the table name of the target table, where the key information refers to the key fields. In this embodiment, dynamic routing to the target table is achieved through the key fields of the source table, thus eliminating the need for users to configure write processes for each table. Only the mapping relationship between the key fields of the source table and the target table in the table mapping rules needs to be adjusted, effectively improving synchronization efficiency.

[0054] For example, determining the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table may include: determining the corresponding target table based on the mapping relationship between the key information of the source table and the write conditions that the key information must meet, contained in the table mapping rules, and the table name of the target table. The write conditions can be a single condition or a combination of multiple conditions, specifically set according to the implementation scenario of the target database. In determining the target table, it can be first determined whether the key information of the source table meets a certain write condition. If it does, the corresponding target table is determined based on the met write condition. For example, assuming the table mapping rule contains a mapping relationship such as: "The product name field equals women's clothing in source table A, which corresponds to target table B in the target database," then according to this table mapping rule, assuming the key information extracted from the database change information is "women's clothing" and "source table A," the corresponding target table can be determined as "target table B." According to this embodiment, dynamic routing to the target table is achieved through the write conditions that the key information must meet, thus eliminating the need for users to configure write processes for each table. Only the mapping relationship between the write conditions in the table mapping rules and the target table needs to be adjusted, effectively improving synchronization efficiency.

[0055] In the embodiments described in this specification, database change information can be obtained directly from the data synchronization middleware, or it can be obtained through a message queue from database change events delivered by the data synchronization middleware. Accordingly, each source database can correspond to one or more tasks to complete the synchronization.

[0056] For example, in one or more embodiments, a source database may correspond to a real-time synchronization task and a real-time write task. The real-time synchronization task may be generated by data synchronization middleware and used to write database change events generated by the data synchronization middleware to a topic in a message queue. The real-time write task is used to retrieve database change events from the message queue topic according to the method provided in the embodiments of this specification, extract data from the database change events, and write it to the target database. Therefore, in this embodiment, obtaining database change information generated in response to changes in the source database may include: retrieving database change events from the message queue, where the database change events are delivered to the message queue by the data synchronization middleware in response to changes in the source database.

[0057] For example, in one or more embodiments, a source database may correspond to a real-time synchronization task, which may be generated by a data synchronization middleware. Therefore, in this embodiment, obtaining database change information in response to changes in the source database may include: obtaining database change events generated by the data synchronization middleware in response to changes in the source database.

[0058] To facilitate user adjustments to table mapping rules, the method provided in this embodiment can also receive user requests to set table mapping rules; and write the mapping relationship between the source table's key information and the target table carried in the setting request into the table mapping rules. For example, a setting entry for table mapping rules can be provided in the management interface, through which user requests to set table mapping rules can be received.

[0059] The following is in conjunction with the appendix Figure 2 The synchronous writing method for a database, which combines the above-described embodiments, will be further explained. Figure 2 The flowchart of a database synchronous write method according to an embodiment of this specification is shown, and the specific steps include steps 202 to 212.

[0060] Step 202: In response to the write interface being called, obtain the interface parameters passed when the write interface is called, the interface parameters including the type information of the data synchronization middleware.

[0061] Step 204: Obtain the database change event delivered to the data synchronization middleware from the message queue according to the interface parameters.

[0062] Step 206: Determine the message format of the database change event based on the interface parameters.

[0063] Step 208: Extract key information of the source table from the database change event according to the message format.

[0064] Step 210: Determine the corresponding target table based on the key information of the source table and the mapping relationship between the key information and the table name of the target table contained in the table mapping rules, as well as the writing conditions that the key information must meet.

[0065] Step 212: Based on the structure information of the target table, extract data from the database change event and write it into the target table of the target database.

[0066] The target database of the method provided in this embodiment can be a data warehouse, which can be provided with a dedicated write interface. This write interface can be based on multiple tables. Through this write interface, the user can specify different interface parameters to tell the data warehouse the type of the real-time data synchronization middleware connected upstream, thereby determining the message format of the database change event message body. Key information such as the source table name is extracted from the database change event. The data warehouse can dynamically determine the target table based on the mapping between the extracted key information and the target table name. Then, according to the structure information of the dynamically determined target table and the format requirements of the target table, the corresponding data in the message body is extracted and written to the corresponding fields of the target table. It can be seen that the method provided by this embodiment provides a mechanism for dynamically determining the target table for the data warehouse, using the same interface for writing to different tables, and implementing dynamic routing through the write interface. In this mode, real-time writing of the entire database can be realized, as well as database merging logic such as database sharding and table partitioning can be implemented.

[0067] In one or more embodiments of this specification, to facilitate troubleshooting of faults occurring during synchronous writing by users of the target database, the method further includes: in response to a fault in writing data to the target database, writing the dirty data generated by the fault to a fault storage area. Through this embodiment, users can correct the data in the target database based on the dirty data written to the fault storage area. For example, the fault storage area can be implemented in any way, such as a database or file storage.

[0068] Furthermore, to resolve synchronization failures as quickly as possible, in one or more embodiments of this specification, in response to a failure in writing data to the target database, a corresponding failure message is sent to the user of the target database. In this embodiment, the failure message can be sent to the user via any means such as telephone, SMS, email, instant messaging, or interactive interface pop-ups. The failure message may include descriptive information about the dirty data, such as the source database and source table information, the target table information of the target database to be written, and a description of the time the failure occurred, to facilitate user troubleshooting.

[0069] To enable users to understand the synchronization health indicators from the source database to the target database, in one or more embodiments of this specification, the synchronization health indicators from the source database to the target database are further calculated based on relevant fault data of the target database. For example, the relevant fault data may be the fault occurrence time, and the corresponding synchronization health indicator can be obtained by calculating the frequency of the fault occurrence within a certain period. As another example, the relevant fault data may be the amount of dirty data at the time of the fault, and the corresponding synchronization health indicator can be obtained by calculating the proportion of dirty data generated within a certain period to the total amount of data synchronously written within that period.

[0070] To facilitate users' intuitive viewing of fault information and / or synchronization health indicators, in one or more embodiments of this specification, a fault interaction interface is triggered on the user's side to display the fault information and / or the synchronization health indicators. For example, the fault information and / or synchronization health indicators can be highlighted in the pop-up fault interaction interface. Additionally, the interface can display a reminder to the user to retrieve dirty data from the fault storage area to salvage data synchronization.

[0071] Corresponding to the above method embodiments, this specification also provides embodiments of a database synchronous writing device. Figure 3 A schematic diagram of a database synchronous writing device according to one embodiment of this specification is shown. Figure 3 As shown, the device includes: a change information acquisition module 302, a key information acquisition module 304, a target table determination module 306, and a data writing module 308.

[0072] The change information acquisition module 302 can be configured to acquire database change information generated in response to changes in the source database.

[0073] The key information acquisition module 304 can be configured to extract key information of the source table from the database change information.

[0074] The target table determination module 306 can be configured to determine the target table corresponding to the key information in a preset table mapping rule based on the key information of the source table. The table mapping rule includes the correspondence between different key information and each target table.

[0075] The data writing module 308 can be configured to extract data from the database change information and write it into the target table of the target database based on the structure information of the target table.

[0076] This device acquires database change information in response to changes in the source database, extracts key information about the source tables from this information, and determines the corresponding target table in a preset table mapping rule based on this key information. Specifically, it dynamically routes the target table using the table name information and table mapping rules in the database table change event. Furthermore, it extracts the necessary data from the database change information based on the target table's structure and writes the extracted data to the target table in the target database. Therefore, this device eliminates the need for users to configure write procedures for each table. It uses a unified write procedure externally, dynamically selecting the target table using the key information of the source tables contained in the database change information. Users only need to adjust the table mapping rules as needed to achieve real-time data writing of the entire database or multiple databases using a single interface. As can be seen, this device significantly reduces the number of tasks that users need to manage. Not only are tasks easier to manage, maintain, and monitor, but the reduced number of tasks also greatly saves resources and effectively improves synchronization efficiency.

[0077] Figure 4 A schematic diagram of a database synchronous writing device according to another embodiment of this specification is shown. Figure 4 As shown, the device may further include: an interface parameter acquisition module 310 and a format determination module 312.

[0078] The interface parameter acquisition module 310 can be configured to acquire the interface parameters passed in when the write interface is called in response to the write interface being called. The interface parameters include information for determining the message format of database change information.

[0079] The format determination module 312 can be configured to determine the message format of database change information based on the interface parameters.

[0080] In this embodiment, a unified write interface is used externally. The message format of the incoming database change information is determined by the write interface parameters, so that the key information and data of the source table can be extracted from the database change information according to the message format.

[0081] In one or more embodiments of this specification, the interface parameters may include type information of the data synchronization middleware. The database change information may be a database change event generated by the data synchronization middleware in response to a change in the source database. According to this embodiment, the format determination module 312 may be configured to determine the message format of the database change event based on the type of the data synchronization middleware determined by the interface parameters.

[0082] The apparatus provided in the embodiments of this specification can provide a dedicated write interface for a target database, which can be based on multiple tables. Through this write interface, the user can specify different interface parameters to inform the target database of the type of real-time data synchronization middleware connected upstream, thereby determining the message format of the database change event message body.

[0083] It should be noted that the specific implementation of the table mapping rules described in the embodiments of this specification is not limited. The mapping relationship between key information of the source table and the target table in the table mapping rules can be set according to the needs of the implementation scenario.

[0084] In one or more embodiments of this specification, the target table determination module 306 can be configured to determine the corresponding target table based on the table name of the source table and the mapping relationship between the table name of the source table and the table name of the target table contained in the table mapping rules.

[0085] In one or more embodiments of this specification, the target table determination module 306 can be configured to determine the corresponding target table based on the mapping relationship between the key fields of the source table and the key fields contained in the table mapping rules and the table name of the target table.

[0086] In one or more embodiments of this specification, the target table determination module 306 can be configured to determine the corresponding target table based on the key information of the source table and the mapping relationship between the writing conditions that the key information must satisfy and the table name of the target table contained in the table mapping rules.

[0087] In the embodiments of this specification, database change events can be obtained directly from the data synchronization middleware, or they can be obtained through a message queue from database change events delivered by the data synchronization middleware. Accordingly, each source database can correspond to one or more tasks to complete synchronization. For example, in one or more embodiments of this specification, the change information acquisition module 302 can be configured to acquire database change events generated by the data synchronization middleware in response to changes in the source database. As another example, the change information acquisition module 302 can be configured to acquire database change events from a message queue, where the database change events are delivered to the message queue by the data synchronization middleware in response to changes in the source database.

[0088] In one or more embodiments of this specification, such as Figure 4 As shown, the device may further include a mapping receiving module 314 and a mapping setting module 316.

[0089] The mapping receiving module 314 can be configured to receive user requests for setting table mapping rules.

[0090] The mapping setting module 316 can be configured to write the mapping relationship between the source table key information and the target table carried in the setting request into the table mapping rules.

[0091] This embodiment makes it easier for users to adjust table mapping rules.

[0092] In one or more embodiments of this specification, such as Figure 4 As shown, the device may further include a dirty data writing module 340, which can be configured to write dirty data generated by the failure to write to the failure storage area in response to a failure to write to the target database.

[0093] For example, such as Figure 4 As shown, the device may further include: a fault information sending module 342, which can be configured to send corresponding fault information to the user of the target database in response to a fault in data writing to the target database.

[0094] For example, such as Figure 4 As shown, the device may further include a health indicator calculation module 344, which can be configured to calculate the synchronization health indicators from the source database to the target database based on the fault-related data of the target database.

[0095] For example, such as Figure 4 As shown, the device may further include: an interface display module 346, which can be configured to display the fault information and / or the synchronous health indicators by triggering a user-side fault interaction interface to pop up on the user's side.

[0096] The above is a schematic scheme of a database synchronous writing device according to this embodiment. It should be noted that the technical solution of this database synchronous writing device and the technical solution of the database synchronous writing method described above belong to the same concept. For details not described in detail in the technical solution of the database synchronous writing device, please refer to the description of the technical solution of the database synchronous writing method described above.

[0097] Corresponding to the above-described method embodiment for synchronous writing to the database, this specification also provides a method embodiment for obtaining change information, which is applied to data synchronization middleware. Figure 5 A flowchart illustrating a method for obtaining change information according to one embodiment of this specification is shown. Figure 5 As shown, the method includes steps 502 and 504.

[0098] Step 502: In response to an update to the log file of the source database, generate database change information based on the update to the log file of the source database.

[0099] Step 504: Send the database change information to the database synchronous write side, so that the database synchronous write side can obtain the database change information, extract the key information of the source table from the database change information, determine the target table corresponding to the key information in the preset table mapping rules according to the key information of the source table, the table mapping rules contain the correspondence between different key information and each target table, and extract data from the database change information and write it into the target table of the target database according to the structure information of the target table.

[0100] The above is an illustrative scheme of a method for obtaining change information according to this embodiment. It should be noted that the technical solution of this method for obtaining change information belongs to the same concept as the technical solution of the above-described database synchronous write method. For details not described in detail in the technical solution of the method for obtaining change information, please refer to the description of the technical solution of the above-described database synchronous write method.

[0101] Corresponding to the above-described method embodiments for obtaining change information, this specification also provides an embodiment of a device for obtaining change information, configured in a data synchronization middleware. Figure 6 A schematic diagram of a device for obtaining change information according to one embodiment of this specification is shown. Figure 6 As shown, the device for obtaining change information includes: a log update response module 602 and a change information sending module 604.

[0102] The log update response module 602 is configured to respond to updates to the log files of the source database and generate database change information based on the updates to the log files of the source database.

[0103] The change information sending module 604 is configured to send the database change information to the database synchronous write side, enabling the database synchronous write side to obtain the database change information, extract key information of the source table from the database change information, determine the target table corresponding to the key information in a preset table mapping rule based on the key information of the source table, the table mapping rule includes the correspondence between different key information and each target table, and extract data from the database change information and write it into the target table of the target database based on the structure information of the target table.

[0104] The above is an illustrative scheme of an apparatus for obtaining change information according to this embodiment. It should be noted that the technical solution of this apparatus for obtaining change information belongs to the same concept as the technical solution of the above-described database synchronous writing method. For details not described in detail in the technical solution of the apparatus for obtaining change information, please refer to the description of the technical solution of the above-described database synchronous writing method.

[0105] Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0106] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0107] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0108] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 700 can also be a mobile or stationary server.

[0109] The processor 720 is used to execute the following computer-executable instructions:

[0110] Retrieve database change information in response to changes in the source database;

[0111] Extract key information from the source table from the database change information;

[0112] Based on the key information of the source table, the target table corresponding to the key information in the preset table mapping rules is determined. The table mapping rules include the correspondence between different key information and each target table.

[0113] Based on the structure information of the target table, data is extracted from the database change information and written into the target table in the target database;

[0114] Alternatively, the processor is configured to execute the computer-executable instructions:

[0115] In response to an update to the log file of the source database, database change information is generated based on the update to the log file of the source database.

[0116] The database change information is sent to the database synchronous write side, enabling the database synchronous write side to obtain the database change information, extract key information of the source table from the database change information, determine the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table, the table mapping rules contain the correspondence between different key information and each target table, and extract data from the database change information and write it into the target table of the target database based on the structure information of the target table.

[0117] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above-described database synchronous writing method belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-described database synchronous writing method.

[0118] One embodiment of this specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, are used for:

[0119] Retrieve database change information in response to changes in the source database;

[0120] Extract key information from the source table from the database change information;

[0121] Based on the key information of the source table, the target table corresponding to the key information in the preset table mapping rules is determined. The table mapping rules include the correspondence between different key information and each target table.

[0122] Based on the structure information of the target table, data is extracted from the database change information and written into the target table in the target database;

[0123] Alternatively, the processor may be used to execute the computer-executable instructions:

[0124] In response to an update to the log file of the source database, database change information is generated based on the update to the log file of the source database.

[0125] The database change information is sent to the database synchronous write side, enabling the database synchronous write side to obtain the database change information, extract key information of the source table from the database change information, determine the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table, the table mapping rules contain the correspondence between different key information and each target table, and extract data from the database change information and write it into the target table of the target database based on the structure information of the target table.

[0126] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above-described database synchronous write method belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-described database synchronous write method.

[0127] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0128] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A synchronous write method for a database, comprising: Retrieve database change information in response to changes in the source database; Extract key information from the source table from the database change information; Based on the key information of the source table, the target table corresponding to the key information in the preset table mapping rules is determined. The table mapping rules include the correspondence between different key information and each target table. The key information includes the table name or key fields. Based on the structure information of the target table, data is extracted from the database change information and written into the target table of the target database.

2. The method according to claim 1, further comprising: In response to the writing interface being called, the interface parameters passed in when the writing interface is called are obtained, and the interface parameters include information for determining the message format of the database change information; The message format for database change information is determined based on the interface parameters.

3. The method according to claim 2, wherein the interface parameters include type information of the data synchronization middleware; the database change information is a database change event generated by the data synchronization middleware in response to a change in the source database; The message format for determining database change information based on interface parameters includes: The message format for database change events is determined based on the type of data synchronization middleware determined by the interface parameters.

4. The method according to claim 1, wherein determining the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table includes: Based on the source table name and the mapping relationship between the source table name and the target table name contained in the table mapping rules, the corresponding target table is determined, and the key information is the table name; or, Based on the mapping relationship between the key fields of the source table and the key fields contained in the table mapping rules and the table name of the target table, the corresponding target table is determined, and the key information refers to the key fields. or, Based on the key information of the source table and the mapping relationship between the key information's write conditions and the target table's name contained in the table mapping rules, the corresponding target table is determined.

5. The method according to claim 1, wherein obtaining database change information generated in response to changes in the source database includes: Obtain database change events generated by the data synchronization middleware in response to changes in the source database; or, Retrieve database change events from the message queue, whereby the database change events are delivered to the message queue by the data synchronization middleware in response to changes in the source database.

6. The method according to claim 1, further comprising: Receive user requests to set table mapping rules; Write the mapping relationship between the key information of the source table and the target table carried in the setting request into the table mapping rules.

7. The method according to claim 1, further comprising: In response to a failure in writing data to the target database, the dirty data generated by the failure is written to the failure storage area.

8. The method according to claim 7, further comprising: In response to a failure in writing data to the target database, a corresponding failure message is sent to the user of the target database.

9. The method according to claim 7, further comprising: Based on the relevant fault data of the target database, calculate the synchronization health indicators from the source database to the target database.

10. The method of claim 8, further comprising: The fault information is displayed by triggering a user-side fault interaction interface on the user's side.

11. The method of claim 9, further comprising: The synchronized health indicators are displayed by triggering a pop-up fault interaction interface on the user's side.

12. A database synchronous write device, comprising: The change information acquisition module is configured to acquire database change information in response to changes in the source database; The key information acquisition module is configured to extract key information of the source table from the database change information; The target table determination module is configured to determine the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table. The table mapping rules include the correspondence between different key information and each target table, wherein the key information includes the table name or key fields. The data writing module is configured to extract data from the database change information and write it into the target table in the target database based on the structure information of the target table.

13. A method for obtaining change information, applied to data synchronization middleware, comprising: In response to an update to the log file of the source database, database change information is generated based on the update to the log file of the source database. The database change information is sent to the database synchronous write side, enabling the database synchronous write side to obtain the database change information, extract key information of the source table from the database change information, and determine the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table. The table mapping rules contain the correspondence between different key information and each target table, wherein the key information includes table name or key fields. Based on the structure information of the target table, data is extracted from the database change information and written to the target table in the target database.

14. An apparatus for acquiring change information, configured in a data synchronization middleware, comprising: The log update response module is configured to respond to updates to the log files of the source database and generate database change information based on the updates to the log files of the source database. The change information sending module is configured to send the database change information to the database synchronous write side, enabling the database synchronous write side to obtain the database change information, extract key information of the source table from the database change information, determine the target table corresponding to the key information in a preset table mapping rule based on the key information of the source table, the table mapping rule includes the correspondence between different key information and each target table, wherein the key information includes table name or key field, and extract data from the database change information and write it into the target table of the target database based on the structure information of the target table.

15. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions: Retrieve database change information in response to changes in the source database; Extract key information from the source table from the database change information; Based on the key information of the source table, the target table corresponding to the key information in the preset table mapping rules is determined. The table mapping rules include the correspondence between different key information and each target table. The key information includes the table name or key fields. Based on the structure information of the target table, data is extracted from the database change information and written into the target table in the target database; Alternatively, the processor is configured to execute the computer-executable instructions: In response to an update to the log file of the source database, database change information is generated based on the update to the log file of the source database. The database change information is sent to the database synchronous write side, enabling the database synchronous write side to obtain the database change information, extract key information of the source table from the database change information, and determine the target table corresponding to the key information in the preset table mapping rules based on the key information of the source table. The table mapping rules contain the correspondence between different key information and each target table, wherein the key information includes table name or key fields. Based on the structure information of the target table, data is extracted from the database change information and written to the target table in the target database.

16. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the synchronous writing method of the database according to any one of claims 1 to 11, or the steps of the method for obtaining change information according to claim 13.

17. A computer program product comprising computer instructions which, when executed by a processor, implement the steps of the synchronous writing method of the database according to any one of claims 1 to 11, or the steps of the method for obtaining change information according to claim 13.

Images