A data replication method and apparatus

Abstract

The application provides a data replication method and device. The method comprises the following steps: in response to a user's data replication condition editing operation on a configuration interface for a first source database, a data replication device acquires a first SQL replication statement, and then the first target data in the first source database can be replicated into a first target database corresponding to the first source database according to the first SQL replication statement. In this way, the method enables the user to flexibly change or flexibly configure the data replication condition on the configuration interface, can meet different data replication needs of the user, helps to realize on-demand data replication, and does not require the user to restart the service or modify the configuration or modify the business code to change the data replication condition, so that the data replication will not be interrupted when the data replication condition changes. In addition, since the data replication condition is persisted in the corresponding SQL replication statement, even if the user restarts the service or the data replication appears an abnormal situation, etc., the data replication condition will not be lost.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a data replication method and apparatus. Background Technology

[0002] Currently, database (such as MySQL) master-slave replication is widely used in the industry for applications such as master-slave high availability, read-write separation, disaster recovery backup, masquerading as a standby database for row and column transformation, implementing hybrid transactional / analytic processing (HTAP) functions, or realizing online migration of heterogeneous databases.

[0003] For example, taking MySQL as an example, MySQL master-slave replication generally uses "full replication" + "incremental replication". Specifically, the slave first obtains a consistent snapshot and corresponding binary log / logical log (Binlog) points from the master; then, it reads fully consistent data from the master using this consistent snapshot and inserts it into the slave; finally, when full replication is complete, the slave pulls the Binlog through this Binlog point to start incremental replication. During MySQL master-slave replication, there may be filtering requirements, such as users selectively synchronizing the data they need. As business evolves, user filtering requirements will change accordingly, therefore, database master-slave replication needs to support dynamic data filtering conditions (which can also be understood as data replication conditions).

[0004] However, although MySQL itself provides the ability to filter on the source and target devices, there are still some problems in the process of changing data replication conditions (such as needing to restart the service to change data replication conditions, the inability to persist data replication conditions, or the loss of data replication conditions after data replication anomalies), which is not flexible enough and difficult to meet the needs of some scenarios. Summary of the Invention

[0005] This application provides a data replication method and apparatus to enable flexible changes in data replication conditions and ensure that data replication is not interrupted when data replication conditions change.

[0006] Firstly, embodiments of this application provide a data replication method. This method can be executed by a data replication device or a component (such as a chip system or circuit) capable of supporting the data replication device in implementing the functions required for the method. For example, the data replication device can be a database server with the functions required for implementing the method, a plug-in with the functions required for implementing the method, or any other device (such as a computing device) with the functions required for implementing the method. Optionally, taking the execution of the data replication method by a data replication device as an example, in this method, in response to a user's editing operation on the data replication conditions of the first source database in the configuration interface, the data replication device can obtain the first SQL replication statement corresponding to the first source database. Then, the data replication device can copy the first target data in the first source database to the first target database corresponding to the first source database according to the first SQL replication statement.

[0007] This method provides users with a configuration interface, allowing for flexible changes and configurations of data replication conditions for the source database. This greater flexibility meets diverse data replication needs and facilitates on-demand data replication from the source database. Furthermore, users can dynamically change data replication conditions simply by editing the configuration interface, without interrupting data replication. This effectively avoids data replication interruptions caused by requiring users to restart the service, modify configurations, or alter business code to implement these changes. The data replication conditions for each source database are persisted in the corresponding SQL replication statement, ensuring that these conditions are not lost even if the user restarts the service or encounters replication anomalies. For example, a user initially wants to copy data from source database A, but now wants to copy data from source database B. This means the data replication conditions have changed. The user can edit the replication conditions for the currently needed source database B in the configuration interface, allowing for parallel replication of data from multiple source databases. Furthermore, the replication process for the currently needed source database B will not affect the replication process for the source database A that was previously in the replication state. This achieves the goal of uninterrupted data replication when the replication conditions are changed. Additionally, during master-slave replication, the user can specify which source database to copy data from. For instance, by editing the replication conditions for the desired source database in the configuration interface, the user can generate an SQL replication statement for that source database, filtering out unnecessary source databases. This achieves database-level data filtering, where each database is independent, and their respective data replication processes do not affect each other.

[0008] In one possible design, the data replication device can copy target data from a first source database to a first target database corresponding to the first source database based on a first SQL replication statement, including:

[0009] After obtaining the first SQL replication statement, the data replication device can establish a first replication channel between the first source database and the first target database based on the first SQL replication statement. It can then fully replicate the first part of the data included in the first target data to the first target database. After the full replication of the first part of the data included in the first target data is completed, the data replication device can perform binlog incremental replication on the first source database through the first replication channel. The binlog incremental replication corresponding to the first source database is used to replicate the second part of the data included in the first target data. The time when the second part of the data in the first target data is written to the first source database is later than the start time of the full replication of the first part of the data in the first target data.

[0010] In the above design, for any source database that needs to be replicated, a replication channel can be established between the source database and the corresponding target database. New data added to the source database can then be replicated through this channel. This replication channel can have an independent binlog replication stream.

[0011] In one possible design, the method further includes:

[0012] In response to the user's editing operation on the configuration interface for the data replication conditions of the second source database, the data replication device can obtain the second SQL replication statement corresponding to the second source database. Then, the data replication device can establish a second replication channel between the second source database and the second target database according to the second SQL replication statement, and can fully replicate the first part of the data included in the second target data in the second source database to the second target database. Then, when the full replication of the first part of the data included in the second target data is completed, the data replication device can perform binlog incremental replication on the second source database through the second replication channel. The binlog incremental replication corresponding to the second source database is used to replicate the second part of the data included in the second target data. The time when the second part of the data in the second target data is written to the second source database is later than the start time of the full replication of the first part of the data in the second target data. In addition, when the binlog incremental replication stream progress of the second replication channel is greater than or equal to the binlog incremental replication stream progress of the first replication channel, the data replication device can merge the second replication channel with the first replication channel.

[0013] In the above design, when adding data to a source database, the user only needs to edit the data replication conditions of the new source database in the configuration interface. This generates the corresponding SQL replication statement for the new source database, and a new replication channel is created between the new source database and the target database by executing the SQL replication statement. In this way, this method can dynamically add data replication operations for a specific source database without requiring the user to restart the service, modify the configuration, or change the business code to change the data replication conditions to replicate the data from the newly added source database. Furthermore, the data replication process for the newly added source database runs in parallel with the data replication process for the previously replicated source database, without affecting each other. In addition, when the binlog incremental replication progress of the newly created replication channel catches up with the binlog incremental replication progress of the previously created replication channel, the newly created replication channel is merged with the previously created replication channel. This effectively reduces the pressure on the main computing device (such as the main database server) and reduces network bandwidth consumption.

[0014] In one possible design, the method further includes:

[0015] In response to the user's instruction editing operation on the configuration interface for the first source library, the data copying device can obtain the first instruction information, which is used to instruct the copying of the first source library to stop. After that, the data copying device can stop copying the first source library according to the first instruction information, and can delete or stop using the first target library.

[0016] In this method, when a user decides to stop copying data from a source database, the user simply needs to edit the stop copying instruction for that source database on the configuration interface. After receiving the stop copying instruction for that source database, the data replication device can generate an SQL delete statement for the corresponding target database to delete the target database, or terminate the data replication process for that source database, or simply stop using the target database altogether. This allows for dynamically reducing or terminating the data replication operation for a specific source database.

[0017] Secondly, embodiments of this application provide a data replication method, which can be executed by a data replication device or a component (such as a chip system or circuit) capable of supporting the data replication device in implementing the function required by the method. For example, the data replication device can be a database server with the function required to implement the method, a plug-in with the function required to implement the method, or any other device (such as a computing device) with the function required to implement the method. Optionally, taking the execution of the data replication method by a data replication device as an example, in this method, in response to a user's editing operation on the data replication conditions of a first table to be replicated in a configuration interface, the data replication device can obtain a first SQL replication statement corresponding to the first table to be replicated. Then, the data replication device can copy the first target data from the first table to be replicated to a first target table according to the first SQL replication statement. The first table to be replicated is stored in a first source database, and the first target table is stored in a first target database corresponding to the first source database.

[0018] In the above design, by providing a configuration interface, users can easily change or configure data replication conditions for a specific table in a source database, offering greater flexibility and meeting diverse data replication needs. This facilitates on-demand data replication. Furthermore, users can dynamically change data replication conditions simply by editing the configuration interface, without interrupting data replication. This effectively avoids data replication interruptions caused by requiring users to restart the service, modify configurations, or alter business code to change replication conditions. The data replication conditions for any table are persisted in the corresponding SQL replication statement, ensuring that these conditions are not lost even if the user restarts the service or encounters replication anomalies. For example, a user initially wants to copy data from Table 1 in source database A, and now wants to copy data from Table 2 in source database A. At this point, the data copying conditions will change. The user can then edit the data copying conditions for Table 2 that needs to be copied in the configuration interface. This allows for parallel copying of data from multiple source databases, and the data copying process for Table 2 that needs to be copied will not affect the data copying process for Table 1 that was previously in the copying state. In other words, the goal of ensuring that data copying is not interrupted when the data copying conditions are changed is achieved.

[0019] In one possible design, the data replication device can copy first target data from a first table to be replicated to a first target table according to a first SQL replication statement, including:

[0020] After obtaining the first SQL replication statement, the data replication device can establish a first replication channel between the first source database and the first target database based on the first SQL replication statement. It can then fully replicate the first part of the data included in the first target data to the first target table. After the full replication of the first part of the data included in the first target data is completed, the data replication device can perform binlog incremental replication on the first table to be replicated through the first replication channel. The binlog incremental replication corresponding to the first table to be replicated is used to replicate the second part of the data included in the first target data. The time when the second part of the data in the first target data is written to the first table to be replicated is later than the start time of the full replication of the first part of the data in the first target data.

[0021] In the above design, for any table that needs to be copied, a replication channel can be established between the source database of the table to be copied and the target database of the corresponding target table. New data added to the table can then be copied through this replication channel. This replication channel has an independent binlog replication stream.

[0022] In one possible design, the method further includes:

[0023] In response to the user's editing operation on the configuration interface regarding the data replication conditions of the second table to be replicated, the data replication device can obtain the second SQL replication statement. Then, based on the second SQL replication statement, the data replication device can establish a second target database with hidden attributes, establish a second replication channel between the first source database and the second target database, and fully replicate the first portion of data included in the second target data of the second table to be replicated to the second target table. Then, when the full replication of the first portion of data included in the second target data is complete, the data replication device can perform binlog incremental replication on the second table to be replicated through the second replication channel. The second table to be replicated is stored in the first source database, and the second target database includes the second target table. The binlog incremental replication corresponding to the second table to be replicated is used to replicate the second portion of data included in the second target data. The time when the second portion of data in the second target data is written to the second table to be replicated is later than the start time of the full replication of the first portion of data in the second target data. Furthermore, when the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the data replication device can also merge the second replication channel with the first replication channel and merge the first target database with the second target database.

[0024] In the above design, when adding data to a table to be copied from the first source database, the user only needs to edit the data replication conditions of the newly added table in the configuration interface. This generates the corresponding SQL replication statement for the newly added table, and a new replication channel is created between the first source database and the created hidden database (such as the second target database) by executing the SQL replication statement. In this way, this method can dynamically add data replication operations for a specific table without requiring the user to restart the service, modify the configuration, or change the business code to change the data replication conditions to replicate the newly added table. Moreover, the data replication process for the newly added table executes in parallel with the data replication process for the previously replicated table, without affecting each other. Furthermore, when the binlog incremental replication progress of the newly added replication channel catches up with the binlog incremental replication progress of the previously created replication channel, merging the newly created replication channel with the previously created replication channel can effectively reduce the pressure on the main computing device (such as the main database server) and reduce network bandwidth consumption.

[0025] In one possible design, the method further includes:

[0026] In response to the user's instruction to edit the first table to be copied in the configuration interface, the data copying device can obtain the first instruction information. The first instruction information is used to instruct the first table to be copied to stop copying. After that, the data copying device can stop copying the first table to be copied according to the first instruction information, and can delete or stop using the first target table.

[0027] In the above design, when a user decides to stop copying data from a table to be copied, the user only needs to edit the stop copying instruction for that table on the configuration interface. Afterwards, upon receiving the stop copying instruction for that table, the data copying device can generate an SQL delete statement for the target table corresponding to that table, thereby deleting the target table, terminating the data copying process for that table, or simply ceasing to use the target table. This allows for dynamically reducing or terminating the data copying operation for a specific table.

[0028] Thirdly, embodiments of this application provide a possible data replication apparatus. Optionally, the data replication apparatus may be a computing device with data replication functionality (e.g., from a database server) or a device (e.g., a plug-in, component, or functional part) capable of supporting the computing device in implementing the data replication method required by the data replication method. In one example, when the data replication apparatus is used to implement the functionality of the data replication apparatus in the data replication method embodiment provided in the first aspect, the beneficial effects can be found in the description of the first aspect, and will not be repeated here. The data replication apparatus has the functionality to implement the behavior in the method example of the first aspect described above. The functionality can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functionality. In one possible design, the data replication apparatus includes: a display module, an acquisition module, and a processing module. The display module is used to display a configuration interface. The acquisition module is used to acquire a first SQL replication statement corresponding to the first source database in response to a user's editing operation of the data replication conditions of the first source database in the configuration interface. The processing module is used to copy the first target data in the first source database to the first target database corresponding to the first source database according to the first SQL replication statement corresponding to the first source database. These modules can perform the corresponding functions in any of the possible implementations of the design described in the first aspect above. For details, please refer to the detailed description of the corresponding functions of the data replication device in the method example, which will not be repeated here.

[0029] In another example, when the data replication device is used to implement the functions of the data replication device in the data replication method embodiment provided in the second aspect, the beneficial effects can be found in the description of the second aspect, and will not be repeated here. The data replication device has the function of implementing the behavior in the method example of the second aspect described above. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the data replication device includes: a display module, an acquisition module, and a processing module. The display module is used to display a configuration interface. The acquisition module is used to acquire a first SQL replication statement corresponding to the first table to be replicated in response to a user's editing operation of the data replication conditions of the first table to be replicated in the configuration interface, wherein the first table to be replicated is stored in a first source database. The processing module is used to copy the first target data in the first table to be replicated to a first target table according to the first SQL replication statement corresponding to the first table to be replicated, wherein the first target table is stored in a first target database corresponding to the first source database. These modules can perform the corresponding functions in any possible implementation design of the second aspect described above. For details, please refer to the detailed description of the corresponding functions of the data replication device in the method example, and will not be repeated here.

[0030] Fourthly, embodiments of this application provide a computing device. In one example, the computing device has the function of implementing the behavior of the data copying device in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect, and will not be repeated here. In another example, the computing device has the function of implementing the behavior of the data copying device in the method example of the second aspect described above. The beneficial effects can be found in the description of the second aspect, and will not be repeated here. The computing device includes a display, a memory, and a processor. Optionally, the computing device further includes a communication interface for communicating with other devices. The memory is used to store computer programs or instructions. The processor is coupled to the display, the memory, and the communication interface. When the processor executes the computer program or instructions, it causes the computing device to execute the method in any possible design of the first aspect or any possible design of the second aspect described above.

[0031] Fifthly, embodiments of this application provide a computing device cluster. In one example, the computing device cluster has the functionality to implement the behavior of the data replication device in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect, and will not be repeated here. In another example, the computing device cluster has the functionality to implement the behavior of the data replication device in the method example of the second aspect described above. The beneficial effects can be found in the description of the second aspect, and will not be repeated here. The computing device cluster includes at least one computing device. Each computing device includes a display, a processor, and a memory. The processor in each computing device is configured to support the data replication device in performing some or all of the functions of the method in any possible design of the first aspect or any possible design of the second aspect described above. The memory is coupled to the processor and stores the computer program instructions and data necessary for the data replication device. Each computing device also includes a communication interface for communicating with other devices.

[0032] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing computer program instructions that, when executed by a computing device, cause the computing device to perform the method in any possible design of the first aspect or any possible design of the second aspect described above.

[0033] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when run on a computing device, cause the computing device to perform the method in any possible design of the first aspect or any possible design of the second aspect described above.

[0034] Eighthly, embodiments of this application also provide a computer chip coupled to a memory, the computer chip being used to read a computer program stored in the memory and execute the method in any possible design of the first aspect or the method in any possible design of the second aspect.

[0035] Ninthly, embodiments of this application also provide a chip system including a processor for supporting a computer device in implementing the methods in any possible design of the first aspect or any possible design of the second aspect described above. In one possible design, the chip system further includes a memory for storing programs and data necessary for the computer device. The chip system may be composed of computer chips or may include computer chips and other discrete devices.

[0036] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0037] Figure 1a This illustration shows a scenario diagram of a one-to-one master-slave database deployment provided by an embodiment of this application;

[0038] Figure 1b This illustration shows a scenario diagram of a many-to-one deployment of master-slave databases provided in an embodiment of this application;

[0039] Figure 1c This illustration shows a scenario diagram of a one-to-many master-slave database deployment provided by an embodiment of this application;

[0040] Figure 1d This illustration shows a scenario diagram of a many-to-many deployment of master-slave databases provided in an embodiment of this application;

[0041] Figure 2 An exemplary flowchart of a data copying method provided in an embodiment of this application is shown.

[0042] Figure 3 This illustration shows a schematic diagram of copying target data from a source library according to an embodiment of this application;

[0043] Figure 4 An exemplary flowchart of another data copying method provided in an embodiment of this application is shown;

[0044] Figure 5 This illustration shows a schematic diagram of copying target data from a table to be copied, provided by an embodiment of this application.

[0045] Figure 6This illustration shows a schematic diagram of a possible data copying apparatus provided in an embodiment of this application;

[0046] Figure 7 An exemplary schematic diagram of a possible computing device provided in an embodiment of this application is shown;

[0047] Figure 8 This illustration shows a possible structural diagram of a computing device cluster provided in an embodiment of this application. Detailed Implementation

[0048] Before introducing the technical solutions provided in this application, some of the terms used in this application will be explained in order to facilitate understanding by those skilled in the art.

[0049] (1) Logical Log: Also called Binary Log, it mainly records changes to the source database deployed on the master database server. Replaying this binary log from the database server can generate the same data as the source database. Since this log records the modification process of the source database deployed on the master database server, it is called a logical log.

[0050] (2) Full synchronization: During the initialization phase of master-slave replication, a read lock or backup lock is first acquired on a source database deployed on the master database server to obtain a consistent snapshot. Then, based on this snapshot, fully consistent data is read from the source database and inserted into the corresponding target database deployed on the slave database server. Once the data in the source database is fully replicated, the data in the target database is completely consistent with the data in the source database snapshot.

[0051] (3) Incremental synchronization: After the data in a source database deployed on the master database server is fully copied, the subsequent data changes in the source database can be copied to the slave database server through the logical log / binary log and replayed on the slave database server to complete the data modification of the target database corresponding to the source database.

[0052] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0053] The following describes possible application scenarios for the data copying method provided in this application. It should be noted that this description is for the convenience of those skilled in the art and does not constitute a limitation on the scope of protection claimed in this application.

[0054] Scenario 1: One-to-one deployment of a master database server (or master database instance or master database device, abbreviated as master database) and a slave database server (or slave database instance or slave database device, abbreviated as slave database).

[0055] Figure 1a This illustration demonstrates a scenario of a one-to-one master-slave database deployment according to an embodiment of this application. In this scenario, both a master database server (e.g., master database server 111) and a slave database server (e.g., slave database server 121) can deploy (or manage or create) at least one database. For example, at least one source database (e.g., source database A1, source database B1, etc.) is deployed on master database server 111, and at least one target database (e.g., target database a1, target database b1, etc.) is deployed on slave database server 121. Each source database or each target database is used to store corresponding information or data. Optionally, master database server 111 and slave database server 121 can communicate via a wireless network or a wired network.

[0056] Scenario 2: Many-to-one deployment of master and slave database servers.

[0057] Figure 1b This illustration demonstrates a scenario of a many-to-one master-slave database deployment according to an embodiment of this application. In this scenario, multiple master database servers (e.g., master database server 111, master database server 112, master database server 113, etc.) and one slave database server (e.g., slave database server 121) can each deploy at least one database. For example, master database server 111 can deploy at least one source database (e.g., source database A1, source database B1, etc.), master database server 112 can deploy at least one source database (e.g., source database A2, source database B2, etc.), master database server 113 can deploy at least one source database (e.g., source database A3, source database B3, etc.), and slave database server 121 can deploy at least one target database (e.g., target database a*, target database b*, etc.). Each source database or each target database is used to store corresponding information or data. Optionally, master database servers 111, 112, and 113 can communicate with slave database server 121 via wireless or wired networks.

[0058] Scenario 3: One-to-many deployment of master database server and slave database server.

[0059] Figure 1cThis illustration demonstrates a scenario of one-to-many master-slave database deployment according to an embodiment of this application. In this scenario, one master database server (e.g., master database server 111) and multiple slave database servers (e.g., slave database server 121, slave database server 122, slave database server 123, etc.) can each deploy at least one database. For example, at least one source database (e.g., source database A1, source database B1, etc.) is deployed on master database server 111, at least one target database (e.g., target database a1, target database b1, etc.) is deployed on slave database server 121, at least one target database (e.g., target database a2, target database b2, etc.) is deployed on slave database server 122, and at least one target database (e.g., target database a3, target database b3, etc.) is deployed on slave database server 123. Each source database or each target database is used to store corresponding information or data. Optionally, master database server 111 can communicate with slave database servers 121, 122, and 123 via wireless or wired networks.

[0060] Scenario 4: Many-to-many deployment of master and slave database servers.

[0061] Figure 1dThis illustration demonstrates a scenario of deploying master-slave databases in a many-to-many manner, as provided in an embodiment of this application. In this scenario, at least one database can be deployed on each of the multiple master database servers (e.g., master database server 111, master database server 112, master database server 113, etc.) and the multiple slave database servers (e.g., slave database server 121, slave database server 122, slave database server 123, slave database server 124, slave database server 125, slave database server 126, etc.). For example, at least one source database (e.g., source database A1, source database B1, etc.) is deployed on master database server 111, at least one source database (e.g., source database A2, source database B2, etc.) is deployed on master database server 112, at least one source database (e.g., source database A3, source database B3, etc.) is deployed on master database server 113, at least one target database (e.g., target database a1, target database b1, etc.) is deployed on slave database server 121, at least one target database (e.g., target database a2, target database b2, etc.) is deployed on slave database server 122, at least one target database (e.g., target database a3, target database b3, etc.) is deployed on slave database server 123, at least one target database (e.g., target database a4, target database b4, etc.) is deployed on slave database server 124, at least one target database (e.g., target database a5, target database b5, etc.) is deployed on slave database server 125, and at least one target database (e.g., target database a6, target database b6, etc.) is deployed on slave database server 126. Each source database or each target database is used to store corresponding information or data. Optionally, the master database server and the slave database server can communicate via a wireless network or a wired network. For example, taking master database server 111 and slave database server 121 as an example, master database server 111 and slave database server 121 can communicate via a wireless network or a wired network. Optionally, slave database servers can also communicate with each other via a wireless network or a wired network. For example, taking slave database server 121 and slave database server 124 as an example, slave database server 121 and slave database server 124 can communicate via a wireless network or a wired network. Optionally, data replication can also be performed between database server 121 and database server 124. For example, when a user needs to copy target data from a database (e.g., database 1) deployed on database server 121 to a corresponding database (e.g., database 1') on database server 124, the user can edit the data replication conditions for database 1 on the configuration interface. After obtaining the corresponding SQL replication statement for database 1 from database server 124, the user can perform the replication operation for the target data in database 1 based on the corresponding SQL replication statement for database 1.

[0062] For example, in the above scenarios, the master database server can provide read and write services, while the slave database server can only provide read services. In this way, the slave database server can share the access pressure of the master database server, or the slave database server can also provide disaster recovery functions to achieve high availability of database services.

[0063] Optionally, in one example, the master database server and the slave database server can be servers with the same database service software installed. For example, both the master and slave database servers can be servers with MySQL installed. In this case, the source database deployed on the master database server and the target database deployed on the slave database server are homogeneous databases. For instance, when a computing device (or computer or other device) has database service software (such as MySQL) installed, that computing device (or computer or other device) becomes a database service device (or database server). In another example, the master database server and the slave database server can be servers with different database service software installed. For example, the master database server is a server with MySQL installed, and the slave database server is a server with ClickHouse installed. In this case, the source database deployed on the master database server and the target database deployed on the slave database server are heterogeneous databases.

[0064] Optionally, in one example, when a user wants to copy corresponding data from a source database (e.g., source database A1), they can log in to the data replication condition configuration interface through a database client installed on a terminal device (e.g., tablet, desktop computer, computer with wireless transceiver capabilities (e.g., laptop), or in-vehicle device). The user can then edit and submit the data replication conditions for source database A1 on this interface. For example, the user can enter the identifier or name of source database A1, or the identifier or name of the table to be copied from source database A1. Afterwards, when the database server obtains the identifier or name of source database A1, or both the identifier or name of source database A1 and the identifier or name of the table to be copied, it can generate SQL replication statements for copying the corresponding data from source database A1. Then, the database server can copy the corresponding data from source database A1 to the target database (e.g., target database a1) by executing the SQL copy statement corresponding to source database A1.

[0065] In another example, when a user wants to copy data from a source database (e.g., source database A1), after logging into the data replication configuration interface through the database client installed on the terminal device, they can edit and submit the SQL replication statement for source database A1. Then, when the database server receives the SQL replication statement for source database A1, it can execute the statement to copy the corresponding data from source database A1 to the target database (e.g., target database a1).

[0066] Furthermore, during the execution of the SQL replication statement corresponding to source database A1 from the database server, if a user wants to replicate corresponding data from source database B1, the user can edit and submit the data replication conditions for source database B1 on the data replication condition configuration interface. For example, the user can enter the identifier or name of source database B1, or the identifier or name of the table to be replicated in source database B1. Afterwards, when the database server obtains the identifier or name of source database B1, or both the identifier or name of source database B1 and the identifier or name of the table to be replicated in source database B1, it can generate the SQL replication statement for replicating the corresponding data from source database B1. Then, the database server can execute the SQL replication statement corresponding to source database B1 to replicate the corresponding data from source database B1 to the target database (e.g., target database b1). It should be noted that the SQL copy statements for source database A1 executed from the database server and the SQL copy statements for source database B1 executed from the database server are executed in parallel and do not conflict.

[0067] Optionally, users can also edit and submit the SQL replication statement corresponding to the source database B1 on the data replication condition configuration interface. Afterwards, when the database server obtains the SQL replication statement corresponding to the source database B1, it can execute the SQL replication statement to copy the corresponding data from the source database B1 to the target database (e.g., target database b1).

[0068] It should be noted that the above application scenarios are just examples. The data replication method provided in this application can also be applied to a variety of other scenarios (such as cascaded master-slave database deployment or other forms of master-slave database deployment), and is not limited to the scenarios shown above.

[0069] As described in the background section, existing solutions suffer from problems such as requiring service restarts to change data replication conditions, the inability to persist data replication conditions, or loss of data replication conditions after an anomaly. These issues make existing solutions inflexible and unable to meet the needs of certain scenarios. Therefore, this application provides a data replication method to enable flexible changes to data replication conditions and ensure uninterrupted data replication when conditions change.

[0070] The following is based on Figures 1a to 1d The application scenarios shown are illustrated, and the specific implementation of the data copying method in the embodiments of this application will be described in detail.

[0071] Example 1

[0072] Figure 2 This illustration shows a flowchart of a data copying method provided in an embodiment of this application. The method is applicable to... Figures 1a to 1d The application scenario shown is illustrated. For example, Figure 2 The data replication method shown can be applied to library-level data replication. For example, by changing library-level filtering conditions, data replication can be achieved for one or more source libraries. This method can be executed by a data replication device or a component (such as a chip system or circuit) that supports the functionality required for this method. Optionally, the data replication device can be, for example... Figures 1a to 1d The illustrated data replication method can be derived from a database server (e.g., database server 121), or it can be a functional element (e.g., a plug-in or component) within the database server (e.g., database server 121) that implements the data replication method. Alternatively, it can be a computing device with data replication functionality (e.g., database server 121), an independent unit embedded in the computing device that implements the data replication method, or a functional component (e.g., a chip) encapsulated in the computing device that implements the data replication method. Exemplarily, the computing device can include, but is not limited to, smartphones, tablets, desktop computers, wearable devices with wireless communication capabilities (e.g., smartwatches, smart bracelets), or in-vehicle devices. It should be understood that this application does not limit the specific device form of the computing device. To facilitate the introduction of the technical solutions provided in the embodiments of this application, the following description uses a data replication device executing a data replication method as an example. Figure 2 As shown, the method includes:

[0073] Step 201: In response to the user's editing operation on the configuration interface for the data replication conditions of the first source database, the data replication device obtains the first SQL replication statement.

[0074] For example, the first source database can be any one of multiple source databases deployed on a computing device (which can be understood as the main computing device, such as a main database server); the data replication conditions of the first source database can refer to the database-level filtering conditions (or database-level change conditions) corresponding to the first source database, such as the identifier or name of the first source database to be replicated, or the identifier or name of a table or several tables to be replicated in the first source database to be replicated.

[0075] Optionally, in one example, when a user (e.g., user A) wants to copy target data from a source database (e.g., the first source database) deployed on a computing device (e.g., a master database server), user A can log in to the configuration interface (e.g., the data replication condition configuration interface) through the database client installed on their terminal device. They can edit and submit the data replication conditions for the first source database on this interface. Afterward, the database client can send the data replication conditions for the first source database to the data replication device. For example, taking source database A as the first source database, user A can edit the identifier or name of source database A on the data replication condition configuration interface. Alternatively, user A can edit which table or tables in source database A need to be copied, such as editing the identifier or name of table 1 to be copied, or the identifier or name of table 1, table 2, etc. Then, the data replication device can obtain the data replication conditions for the first source database and generate the first SQL replication statement corresponding to the first source database based on these conditions. For example, the data replication device can pre-store general SQL replication statements for replicating data from the source database. When the data replication device obtains the data replication conditions for the first source database (such as the database-level filter conditions corresponding to source database A), it can add these conditions to the general SQL replication statements to generate the first SQL replication statement for the first source database. Alternatively, the data replication device can directly generate the first SQL replication statement for the first source database based on its replication conditions.

[0076] In another example, continuing with user A and source database A, when user A wants to replicate target data from source database A deployed on the primary database service, user A can log in to the data replication condition configuration interface through the database client installed on their terminal device. User A can edit and submit the first SQL replication statement corresponding to source database A on the data replication condition configuration interface. The database client can then send this first SQL replication statement to the data replication device. For example, after user A edits the data replication conditions for source database A (such as the database-level filtering conditions for source database A) on the data replication condition configuration interface, the database client can automatically generate the first SQL replication statement for source database A based on the edited conditions and send it to the data replication device. Alternatively, user A can directly edit and submit the first SQL replication statement for source database A on the data replication condition configuration interface, after which the database client can send it to the data replication device.

[0077] Step 202: The data replication device copies the first target data in the first source database to the first target database corresponding to the first source database according to the first SQL replication statement.

[0078] For example, the first target library can be any one of multiple target libraries deployed on a computing device (which can be understood as a computing device, such as a database server).

[0079] Optionally, when the data replication device obtains the first SQL replication statement corresponding to the first source database, in one example, the data replication device can, according to the first SQL replication statement, fully replicate the first portion of data included in the first target data in the first source database to the first target database, and establish a first replication channel between the first source database and the first target database. In another example, the data replication device can, according to the first SQL replication statement, first establish a first replication channel between the first source database and the first target database, and then, through the first replication channel, fully replicate the first portion of data included in the first target data in the first source database to the first target database.

[0080] Then, when the full copy of the first portion of data included in the first target data in the first source database is completed, the data copying device can perform binlog incremental copying of the first source database through the first copying channel. Specifically, the binlog incremental copying corresponding to the first source database is used to copy the second portion of data included in the first target data in the first source database; the time when this second portion of data is written to the first source database is later than the start time (or full copying start time) of the full copying of the first portion of data included in the first target data in the first source database.

[0081] Optionally, during the execution of the first SQL replication statement by the data replication device, when a user wants to replicate the second target data from a second source database deployed on the primary database server, in one example, the user can edit and submit the data replication conditions for the second source database on a configuration interface (e.g., a data replication condition configuration interface). Afterward, the database client can send the data replication conditions for the second source database to the data replication device. Then, when the data replication device receives the data replication conditions for the second source database, it can generate a second SQL replication statement corresponding to the second source database based on those conditions. In another example, the user can edit and submit the second SQL replication statement corresponding to the second source database on a configuration interface. Afterward, the database client can send this second SQL replication statement to the data replication device. In this example, the first and second source databases are not the same database. Thus, this method allows for dynamic changes to database-level data replication conditions and the dynamic addition of data replication operations for a specific source database. Furthermore, the data replication process for the second source database and the data replication process for the first source database, which is already in the replication state, can be executed in parallel without affecting each other.

[0082] When the data replication device obtains the second SQL replication statement corresponding to the second source database, in one example, the data replication device can, according to the second SQL replication statement, fully replicate the first portion of data included in the second target data in the second source database to the second target database, and can establish a second replication channel between the second source database and the second target database. In another example, the data replication device can, according to the second SQL replication statement, first establish a second replication channel between the second source database and the second target database, and then, through the second replication channel, fully replicate the first portion of data included in the second target data in the second source database to the second target database.

[0083] Then, after the full copy of the first part of the data included in the second target data in the second source database is completed, the data replication device can perform binlog incremental copying on the second source database through the second replication channel to achieve incremental copying of newly added data (such as the second part of the data included in the second target data) included in the second source database. Specifically, the binlog incremental copying corresponding to the second source database is used to copy the second part of the data included in the second target data in the second source database. The time when this second part of the data is written to the second source database is later than the start time of the full copying of the first part of the data included in the second target data in the second source database.

[0084] Optionally, when it is determined that the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the data replication device can merge the second replication channel with the first replication channel. This can effectively reduce the pressure on the main computing device (such as the main database server) and reduce network bandwidth consumption. For example, the data replication device can merge the second replication channel with the first replication channel to form a single replication channel. Alternatively, the data replication device can select either the second or first replication channel as a reserved replication channel and delete the unselected replication channel. It should be understood that the above descriptions are merely illustrative examples of several implementation methods for merging the first and second replication channels in a data replication device. Other implementation methods can also be used to merge the first and second replication channels, which will not be listed in detail in this embodiment.

[0085] For example, the following describes the implementation process of the data replication device replicating target data in the source databases, taking the first source database deployed on the main database server as database (DB)1, which includes tables t1, t2, and t3, the second source database as DB2, which includes tables t1, t2, and t3, the first target database deployed from the database server as DB1', which includes tables t1' and t3', and the second target database as DB2', which includes table t2'.

[0086] Figure 3 This illustration shows a schematic diagram of copying target data from a source library, provided by an embodiment of this application. Figure 3In this example, consider a user (e.g., user A) logging into a configuration interface (e.g., a data replication condition configuration interface) through a database client and editing the database-level filter conditions corresponding to the first source database (e.g., source database DB1). For instance, when user A wants to replicate target data from DB1, user A can edit the database-level filter conditions corresponding to DB1 on the configuration interface (e.g., the identifier or name of DB1, or the identifier or name of tables t1 and t3 in DB1 to be replicated, or the identifier or name of table t2 in DB1 that does not need to be replicated) and submit it. Afterward, the database client sends the database-level filter conditions corresponding to DB1 to the data replication device. After obtaining the database-level filter conditions corresponding to DB1, the data replication device generates the corresponding SQL replication statement for DB1 based on these conditions. For example, the SQL replication statement is CREATE REPLICATION FOR 'DB1'('host:port','user','passwd','DB1')REPL_FILTER black_list='t2'. In this SQL replication statement, a blacklist of tables to be replicated is set, indicating that table t2 in DB1 is filtered out, and only the data in tables t1 and t3 in DB1 is replicated.

[0087] After obtaining the SQL replication statement corresponding to DB1, the data replication device can execute the SQL replication statement. Optionally, during the execution of the SQL replication statement, in one example, the data replication device can fully replicate the existing data in tables t1 and t3 in DB1 to DB1', and can create a replication channel (channel1) between DB1 and DB1'. In another example, the data replication device can first create the replication channel (channel1) between DB1 and DB1', and then use channel1 to fully replicate the existing data in tables t1 and t3 in DB1 to DB1'. Optionally, during the full replication of the existing data in tables t1 and t3 in DB1, the data replication device can also obtain the binlog point GTID1 corresponding to the full replication.

[0088] Then, after the full copying of existing data in tables t1 and t3 in DB1 is complete, if new data is added to tables t1 and / or t3 in DB1, the data replication device can perform incremental binlog copying through channel 1 to copy the new data from tables t1 and t3 in DB1 to DB1'. For example, the data replication device can obtain the logical logs / binary logs related to the new data in tables t1 and / or t3 through channel 1, and simultaneously replay the logical logs / binary logs related to the new data in tables t1 and / or t3, thus enabling the copying of the new data from tables t1 and / or t3 to DB1'.

[0089] Optionally, if a user wants to copy target data from DB2 while the data replication device is executing the SQL replication statement corresponding to DB1, the user can edit the database-level filter conditions corresponding to DB2 on the configuration interface so that the data replication device can obtain the SQL replication statement used to copy the target data from DB2.

[0090] For example, continuing with the scenario where user A logs into the configuration interface (e.g., the data replication condition configuration interface) through a database client and edits the database-level filter conditions corresponding to source database A, user A can edit the database-level filter conditions corresponding to DB2 (e.g., the identifier or name of DB2, or the identifier or name of table t2 in DB2, or the identifier or name of tables t1 and t3 in DB2 that do not need to be replicated) and submit it. Afterwards, the database client sends the database-level filter conditions corresponding to DB2 to the data replication device. Upon receiving the database-level filter conditions corresponding to DB2, the data replication device generates the corresponding SQL replication statement for DB2 based on these conditions. For example, the SQL replication statement might be `CREATE REPLICATION FOR 'DB2'('host:port','user','passwd','DB2')REPL_FILTER white_list='t2'`. This SQL replication statement sets a whitelist of tables to be replicated, indicating that tables t1 and t3 in DB2 are filtered out, and only the data in table t2 in DB2 is replicated.

[0091] After obtaining the corresponding SQL replication statement for DB2, the data replication device can execute the SQL replication statement. In one example, during the execution of this SQL replication statement, the data replication device can fully replicate the existing data in table t2 in DB2 to DB2', and can create a replication channel (channel2) between DB2 and DB2'. In another example, the data replication device can first create the replication channel (channel2) between DB2 and DB2', and then use channel2 to fully replicate the existing data in table t2 in DB2 to DB2'. Optionally, during the full replication of the existing data in table t2 in DB2, the data replication device can also obtain the binlog point GTID2 corresponding to the full replication.

[0092] Then, after the full copy of existing data in table t2 in DB2 is complete, if new data is added to table t2 in DB2, the data replication device can perform incremental binlog copying through channel 2 to copy the new data from table t2 in DB2 to DB2'. For example, the data replication device can obtain the logical log / binary log related to the new data in table t2 through channel 2, and simultaneously replay the logical log / binary log related to the new data in table t2, thus enabling the copying of the new data from table t2 to DB2'.

[0093] Furthermore, when the binlog replication progress (or incremental binlog replication progress) of channel2 is determined to be greater than or equal to the binlog replication progress of channel1, the data replication device can merge channel2 and channel1 so that newly added data in tables t1 and t3 in DB1, as well as newly added data in table t2 in DB2, are all replicated incrementally through the same channel. For example, the data replication device can merge channel2 and channel1 together to form a single replication channel. Alternatively, the data replication device can select either channel2 or channel1 as a reserved replication channel; for instance, the data replication device can select channel1 as a reserved replication channel and delete channel2.

[0094] Optionally, when a user does not want to copy data from a specific source database (e.g., the first source database), the user can log in to the configuration interface through the database client, edit the first instruction information for the first source database, and submit it. This first instruction information instructs the replication of the first source database to be stopped; it can also be understood as instructing the replication of the target data within the first source database to be stopped. The database client can then send this first instruction information to the data replication device. Upon receiving the first instruction information, the data replication device can stop the replication of the first source database accordingly. Optionally, to more effectively stop the replication of target data from the first source database to the first target database, the data replication device can delete the first target database after receiving the first instruction information, or it can simply stop using the first target database without deleting it.

[0095] For example, taking DB2 as the source database, when a user no longer wants to replicate data in DB2, the user can log in to the configuration interface through the database client, edit the first instruction information for DB2 on the configuration interface, and submit it. This first instruction information is used to indicate that the replication of data in DB2 should be stopped. The database client can then send this first instruction information to the data replication device. In one example, after receiving the first instruction information, the data replication device can terminate the data replication process for DB2 according to the first instruction information, thereby stopping the replication of data in DB2. In another example, after receiving the first instruction information, the data replication device can delete the corresponding SQL replication statement for DB2 according to the first instruction information, thereby stopping the replication of data in DB2. Optionally, in one possible implementation, the data replication device can also generate an SQL delete statement for deleting DB2' while stopping the replication of data in DB2, such as "Drop DataBase DB2'". The data replication device then executes this SQL delete statement to delete DB2', thereby more effectively stopping the replication of data in DB2. In another possible implementation, the data replication device can stop copying data in DB2 and stop using DB2' at the same time, thereby more effectively stopping the copying of data in DB2.

[0096] As can be seen from steps 201 and 202 above, when a user wants to copy data from a source database, the user can edit the data replication conditions for that source database in the configuration interface to generate the corresponding SQL replication statement, thus enabling data replication for that source database. When a user wants to add data to be copied from a new source database, the user only needs to edit the data replication conditions for the newly added source database in the configuration interface to generate the corresponding SQL replication statement, thus enabling data replication for that newly added source database. Based on this, this method does not require the user to restart the service, modify the configuration, or modify the business code to change the data replication conditions of the source database to be copied. Moreover, the data replication process for the newly added source database to be copied is executed in parallel with the data replication process for the source database that is already in the replication state, without affecting each other, thus ensuring that data replication is not interrupted when the data replication conditions change. In addition, the data replication conditions for each source database are persisted in the corresponding SQL replication statement for that source database, so even if the user restarts the service or data replication encounters an anomaly, the data replication conditions for that source database will not be lost.

[0097]

Example 2

[0098] Figure 4 This illustration shows a flowchart of another data copying method provided in an embodiment of this application. This method is applicable to... Figures 1a to 1d The application scenario shown is illustrated. For example, Figure 4 The data replication method shown can be applied to table-level data replication. For example, by changing the table-level filter conditions, data replication can be achieved for one or more tables to be replicated. This method can be executed by a data replication device or a component (such as a chip system or circuit) that supports the functionality required for this method. Optionally, the data replication device can be, for example,... Figures 1a to 1d The illustrated data replication method can be derived from a database server (e.g., database server 121), or it can be a functional element (e.g., a plug-in or component) within the database server (e.g., database server 121). This functional element has the function of implementing a data replication method. Alternatively, it can be a computing device with data replication functionality (e.g., database server 121), or it can be an independent unit embedded in the computing device that has the function of implementing a data replication method, or it can be a functional component (e.g., a chip) encapsulated in the computing device that has the function of implementing a data replication method. To facilitate the introduction of the technical solutions provided in the embodiments of this application, the following description uses a data replication device executing a data replication method as an example. Figure 4 As shown, the method includes:

[0099] Step 401: In response to the user's editing operation on the configuration interface for the data replication conditions of the first table to be replicated, the data replication device obtains the first SQL replication statement.

[0100] For example, the first table to be copied can be stored in the first source database; the first source database can be any one of multiple source databases deployed on a computing device (which can be understood as the main computing device, such as a main database server); the data copying conditions of the first table to be copied can refer to the table-level filtering conditions (or table-level change conditions) corresponding to the first table to be copied, such as the identifier or name of the first table to be copied.

[0101] Optionally, in one example, when a user (e.g., user B) wants to copy target data from a specific table (e.g., the first table to be copied) in a source database (e.g., the first source database), user B can log in to the configuration interface (e.g., the data replication condition configuration interface) through the database client installed on their terminal device. On the data replication condition configuration interface, user B can edit and submit the data replication conditions for the first table to be copied. Afterwards, the database client can send the data replication conditions for the first table to be copied to the data replication device. For example, taking source database A as the first source database and table A1 in source database A as the first table to be copied, user B can edit the identifier or name of table A1 in source database A, etc., on the data replication condition configuration interface. Then, the data replication device can obtain the data replication conditions for the first table to be copied and generate the first SQL replication statement corresponding to the first table to be copied based on these conditions. For example, the data replication device can pre-store general SQL replication statements for copying data from tables to be replicated. When the data replication device obtains the data replication conditions for the first table to be replicated (such as the table-level filter conditions corresponding to table A1), it can add these conditions to the general SQL replication statements to generate the first SQL replication statement for the first table to be replicated. Alternatively, the data replication device can directly generate the first SQL replication statement for the first table to be replicated based on its data replication conditions.

[0102] In another example, continuing with user B and table A1 in source database A as the first table to be replicated, when user B wants to replicate target data from table A1 in source database A deployed on the primary database service, user B can log in to the data replication condition configuration interface through the database client installed on their terminal device. They can edit and submit the first SQL replication statement corresponding to table A1 on the data replication condition configuration interface. The database client can then send this first SQL replication statement to the data replication device. For example, after user B edits the data replication conditions for table A1 (such as the database-level filter conditions for table A1) on the data replication condition configuration interface, the database client can automatically generate the first SQL replication statement for table A1 based on the edited data replication conditions and send it to the data replication device. As another example, user B can directly edit and submit the first SQL replication statement for table A1 on the data replication condition configuration interface, after which the database client can send this first SQL replication statement to the data replication device.

[0103] Step 402: The data replication device copies the first target data from the first table to be copied to the first target table according to the first SQL replication statement.

[0104] For example, the first target table can be stored in the first target library corresponding to the first source library. The first target library can be any one of multiple target libraries deployed on a computing device (which can be understood as a computing device, such as a database server).

[0105] Optionally, when the data replication device obtains the first SQL replication statement corresponding to the first table to be replicated, in one example, the data replication device can, according to the first SQL replication statement, fully replicate the first portion of data included in the first target data of the first table to be replicated to the first target table, and establish a first replication channel between the first source database and the first target database. In another example, the data replication device can, according to the first SQL replication statement, first establish a first replication channel between the first source database and the first target database, and then, through the first replication channel, fully replicate the first portion of data included in the first target data of the first table to be replicated to the first target table.

[0106] Then, when the full copy of the first portion of data included in the first target data of the first table to be copied is completed, the data copying device can perform binlog incremental copying of the first table to be copied through the first copying channel. Specifically, the binlog incremental copying corresponding to the first table to be copied is used to copy the second portion of data included in the first target data of the first table to be copied; the time when this second portion of data is written to the first table to be copied is later than the start time of the full copying of the first portion of data included in the first target data of the first table to be copied.

[0107] Optionally, during the execution of the first SQL replication statement by the data replication device, when a user wants to replicate the second target data from the second table to be replicated in the first source database, in one example, the user can edit and submit the data replication conditions for the second table to be replicated on a configuration interface (such as a data replication condition configuration interface). Afterwards, the database client can send the data replication conditions for the second table to be replicated to the data replication device. Then, when the data replication device receives the data replication conditions for the second table to be replicated, it can generate the second SQL replication statement corresponding to the second table to be replicated based on these conditions. In another example, the user can edit and submit the second SQL replication statement corresponding to the second table to be replicated on the configuration interface. Afterwards, the database client can send the second SQL replication statement to the data replication device. In this example, the first table to be replicated and the second table to be replicated are not in the same database. Thus, this method can dynamically change table-level data replication conditions and dynamically add data replication operations for a specific table to be replicated. Furthermore, the data replication process for the second table to be replicated and the data replication process for the first table to be replicated, which is already in the replication state, can be executed in parallel without affecting each other.

[0108] When the data replication device obtains the second SQL replication statement corresponding to the second table to be replicated, it can first establish a second target database with hidden attributes based on the second SQL replication statement. This second target database contains the second target table, or it can create a second target table within the second target database. Optionally, after the data replication device establishes the second target database, in one example, it can fully replicate the first portion of data included in the second target data of the second table to be replicated to the second target table, and can establish a second replication channel between the first source database and the second target database. In another example, the data replication device can first establish a second replication channel between the second source database and the second target database, and then fully replicate the first portion of data included in the second target data of the second table to be replicated to the second target table through the second replication channel.

[0109] Then, after the full copying of the first portion of data included in the second target data of the second table to be copied is completed, the data copying device can perform binlog incremental copying on the second table to be copied through the second copying channel to achieve incremental copying of newly added data (such as the second portion of data included in the second target data) included in the second target data of the second table to be copied. Specifically, the binlog incremental copying corresponding to the second table to be copied is used to copy the second portion of data included in the second target data of the second table to be copied. The time when this second portion of data is written to the second table to be copied is later than the start time of the full copying of the first portion of data included in the second target data of the second table to be copied.

[0110] Optionally, when it is determined that the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the data replication device can merge the second replication channel with the first replication channel. This can effectively reduce the pressure on the main computing device (such as the main database server) and reduce network bandwidth consumption. For example, the data replication device can merge the second replication channel with the first replication channel to form a single replication channel. Alternatively, the data replication device can select either the second or first replication channel as a reserved replication channel and delete the unselected replication channel.

[0111] Optionally, the data replication device can merge the first target database and the second target database simultaneously while merging the first replication channel and the second replication channel. For example, the data replication device can migrate or merge data included in the second target database into the first target database and delete the second target database, or the data replication device can migrate or merge data included in the first target database into the second target database and delete the first target database, thereby achieving the merger of the first target database and the second target database. As another example, the data replication device can modify the identifier (or name) of the second target database to the identifier (or name) of the first target database, or the data replication device can modify the identifier (or name) of the first target database to the identifier (or name) of the second target database, thereby achieving the merger of the first target database and the second target database. It should be understood that the above content is only an exemplary description of several implementation methods for merging the first target database and the second target database using a data replication device. The data replication device can also use other implementation methods to merge the first target database and the second target database, which will not be listed in detail in this application embodiment.

[0112] For example, the following describes the implementation process of the data replication device replicating the target data in the table to be replicated, taking the first source database deployed on the main database server as database (DB)1, DB1 including tables t1, t2 and t3, the first target database deployed from the database server as DB1', DB1' including the target table t1', and the second target database (i.e., a hidden database that is not visible to the user, for example, a hidden database that can be used to copy only the data in the newly added table to be replicated) as DB1", DB1" including the target table t2'.

[0113] Figure 5 This illustration shows a schematic diagram of copying target data from a table to be copied, provided by an embodiment of this application. Figure 5 In this example, consider a user (e.g., user B) logging into a configuration interface (e.g., a data replication condition configuration interface) through a database client and editing the table-level filter conditions corresponding to the first table to be replicated (e.g., table t1 in the source database DB). For instance, when user B wants to replicate target data from table t1 included in DB1, user B can edit the table-level filter conditions corresponding to table t1 on the configuration interface (e.g., the identifier or name of table t1, or the identifiers or names of tables t2 and t3 that do not need to be replicated) and submit. Afterward, the database client sends the table-level filter conditions corresponding to table t1 to the data replication device.

[0114] After obtaining the table-level filter conditions corresponding to table t1, the data replication device generates the corresponding SQL replication statement for table t1 based on the table-level filter conditions. For example, the SQL replication statement is CREATE REPLICATION FOR 'DB1'('host:port','user','passwd','DB1')REPL_FILTER white_list='t1'. This SQL replication statement sets the whitelist of tables to be replicated, indicating that tables t2 and t3 included in DB1 are filtered out, and only the data in table t1 included in DB1 is replicated.

[0115] After obtaining the SQL replication statement corresponding to table t1, the data replication device can execute the SQL replication statement. In one example, during the execution of this SQL replication statement, the data replication device can fully replicate the existing data in table t1 to the target table t1' included in DB1', and can create a replication channel (channel1) between DB1 and DB1'. In another example, the data replication device can first create the replication channel (channel1) between DB1 and DB1', and then use channel1 to fully replicate the existing data in table t1 to the target table t1'. Optionally, during the full replication of the existing data in table t1, the data replication device can also obtain the binlog point GTID1 corresponding to the full replication.

[0116] Then, after the existing data in table t1 has been fully copied, if new data is added to table t1, the data replication device can perform incremental binlog replication through channel 1 to copy the new data from table t1 to the target table t1'. For example, the data replication device can obtain the logical log / binary log related to the new data in table t1 through channel 1, and simultaneously replay the logical log / binary log related to the new data in table t1, thus enabling the copying of the new data from table t1 to the target table t1'.

[0117] Optionally, if a user wants to copy target data from table t2, which is included in DB1, during the execution of the SQL copy statement corresponding to table t1 by the data replication device, the user can edit the table-level filter conditions corresponding to table t2 on the configuration interface so that the data replication device can obtain the SQL copy statement used to copy the target data in table t2.

[0118] For example, continuing with the scenario where user B logs into the configuration interface (e.g., the data replication condition configuration interface) through a database client and edits the database-level filter conditions corresponding to source database A, user B can edit the table-level filter conditions (e.g., the identifier or name of table t2) for table t2 on the configuration interface and submit it. Afterwards, the database client sends the table-level filter conditions for table t2 to the data replication device. Upon obtaining the table-level filter conditions for table t2, the data replication device generates the corresponding SQL replication statement for table t2 based on these conditions. For example, this SQL replication statement might be `ALTER DATABASE 'DB1' MODIFY REPL_FILTER white_list = 't1,t2'`. This SQL replication statement sets a whitelist of tables to be replicated, enabling the replication of target data from table t2, which is included in DB1.

[0119] After obtaining the SQL replication statement corresponding to table t2, the data replication device can execute the SQL replication statement. When executing the SQL replication statement, a hidden database DB1" can be created first, containing the target table t2'. Optionally, after creating DB1", in one example, the data replication device can fully replicate the existing data in table t2 to the target table t2', and can create a replication channel 2 between DB1 and DB1". In another example, the data replication device can first create the replication channel 2 between DB1 and DB1", and then use channel2 to fully replicate the existing data in table t2 to the target table t2'. Optionally, during the full replication of the existing data in table t2, the data replication device can also obtain the binlog point GTID2 corresponding to the full replication.

[0120] Then, after the existing data in table t2 has been fully copied, if new data is added to table t2, the data replication device can perform incremental binlog replication through channel 2 to copy the new data from table t2 to the target table t2'. For example, the data replication device can obtain the logical log / binary log related to the new data in table t2 through channel 2, and simultaneously replay the logical log / binary log related to the new data in table t2, thus enabling the copying of the new data from table t2 to the target table t2'.

[0121] Furthermore, when the binlog replication progress (or incremental binlog replication progress) of channel2 is determined to be greater than or equal to the binlog replication progress of channel1, the data replication device can merge channel2 and channel1 so that newly added data in table t1 and table t2 in DB1 are both incrementally replicated through the same channel. For example, the data replication device can merge channel2 and channel1 together to form a single replication channel. Alternatively, the data replication device can select either channel2 or channel1 as a reserved replication channel; for instance, the data replication device can select channel1 as a reserved replication channel and delete channel2.

[0122] Optionally, the data replication device can merge DB1' and DB1" simultaneously with channel2 and channel1. For example, the data replication device can migrate or merge all the data included in DB1" into DB1' and delete DB1" to form DB1'. Alternatively, the data replication device can modify the identifier (or name) of DB1' to the identifier (or name) of DB1" or vice versa, thus forming DB1'. For instance, when a user's data query request is received from the database server, the corresponding query results in DB1' can be returned to the user.

[0123] Optionally, when a user does not want to copy data from a specific table (such as the first table to be copied), the user can log in to the configuration interface through the database client, edit the first instruction information for the first table to be copied, and submit it. This first instruction information is used to instruct the copying of the first table to be copied to stop; it can also be understood as instructing the copying of the target data in the first table to be copied to stop. The database client can then send this first instruction information to the data replication device. After receiving the first instruction information, the data replication device can stop the copying of the first table to be copied according to the first instruction information. Optionally, to more effectively stop the copying of target data from the first table to be copied to the first target table, the data replication device can delete the first target table after receiving the first instruction information, or it can simply stop using the first target table without deleting it.

[0124] For example, taking table t2, which is included in DB1, as an example, when a user no longer wants to copy data from table t2, the user can log in to the configuration interface through the database client, edit the first instruction information for table t2 on the configuration interface, and submit it. This first instruction information is used to indicate that the copying of data in table t2 should be stopped. Afterwards, the database client can send this first instruction information to the data replication device. In one example, after receiving the first instruction information, the data replication device can terminate the data replication process for table t2 according to the first instruction information, thereby stopping the copying of data in table t2. In another example, after receiving the first instruction information, the data replication device can generate an SQL statement to stop the copying of data in table t2, such as the SQL statement ALTER DATABASE 'DB1' MODIFY REPL_FILTER white_list='t1' or ALTER DATABASE 'DB1' MODIFY REPL_FILTER black_list='t2'. In another example, after receiving the first instruction, the data replication device can modify the SQL replication statement corresponding to table t2 based on the first instruction to obtain the SQL statement used to stop replicating data in table t2. For example, the SQL statement could be `ALTER DATABASE 'DB1' MODIFY REPL_FILTER white_list = 't1'` or `ALTER DATABASE 'DB1' MODIFY REPL_FILTER black_list = 't2'`. Furthermore, while stopping the replication of data in DB2, the data replication device can also delete the target table t2', or it can choose not to delete the target table t2' but simply stop using it, thus more effectively stopping the replication of data in table t2.

[0125] As can be seen from steps 401 and 402 above, when a user wants to copy data from a first table to be copied in a source database, the user can edit the data copying conditions for that first table in the configuration interface to generate the corresponding SQL copying statement, thus enabling data copying for that first table. When a user wants to add a second table to be copied from the source database, the user only needs to edit the data copying conditions for the newly added second table in the configuration interface to generate the corresponding SQL copying statement, thus enabling data copying for that newly added second table. Based on this, this method does not require the user to restart the service, modify the configuration, or modify the business code to change the data copying conditions for the tables to be copied. Furthermore, the data copying process for the newly added second table to be copied executes in parallel with the data copying process for the previously copied first table, without affecting each other, thus ensuring that data copying is not interrupted when data copying conditions change. Furthermore, the data replication conditions for each table to be replicated are persisted in the corresponding SQL replication statement for that table. Therefore, even if the user restarts the service or an abnormal situation occurs during data replication, the data replication conditions for that table will not be lost.

[0126] It should be noted that in the description of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, "at least one of A, B, and C" includes A, B, C, AB, AC, BC, or ABC. Furthermore, unless otherwise specified, the ordinal numbers "first," "second," "third," etc., mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, sequence, priority, or importance of multiple objects. In addition, the terms "including," "comprising," "having," and their variations appearing in this application all mean "including but not limited to," unless otherwise specifically emphasized.

[0127] Furthermore, it should be noted that each step in the above embodiments can be executed by the corresponding device, or by components such as chips, processors, or chip systems within that device. This application does not limit the scope of these steps. The above embodiments are only illustrated by examples of execution by the corresponding device.

[0128] It should be noted that in the above embodiments, some steps may be selected for implementation, and the order of the steps in the figures may be adjusted. This application does not limit this. It should be understood that performing some of the steps in the figures, adjusting the order of the steps, or combining them in a specific implementation all fall within the protection scope of this application.

[0129] It is understood that, in order to achieve the functions described in the above embodiments, each device involved in the above embodiments includes a hardware structure and / or software module corresponding to perform each function. Those skilled in the art should readily recognize that, based on the units and method steps of the various examples described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0130] It should be noted that the "steps" in the embodiments of this application are merely illustrative and are intended to better understand one method of presentation used in the embodiments. They do not constitute a substantial limitation on the execution of the solution of this application. For example, the "step" can also be understood as a "feature". Furthermore, the steps do not constitute any limitation on the execution order of the solution of this application. Any changes to the order of steps, or the merging or splitting of steps made on this basis without affecting the overall solution implementation, resulting in a new technical solution, are also within the scope of disclosure of this application.

[0131] Based on the same concept, embodiments of this application also provide a possible data replication apparatus, which is suitable for... Figures 1a to 1d In the illustrated application scenario, the data replication device can optionally be a computing device with data replication functionality (e.g., from a database server) or a device (e.g., a plug-in, component, or functional part) capable of supporting the computing device in implementing the data replication method. Optionally, in one example, when the data replication device is for implementing library-level data replication, the data replication device is used to implement the above... Figure 2 The technical solutions involved in the data replication device in the method embodiments shown, or the modules (such as chips) of the data replication device used to implement the above. Figure 2 The technical solution involving the data replication device in the method embodiment shown can therefore also achieve the above-mentioned... Figure 2 The illustrated method embodiment demonstrates the beneficial effects of the data replication apparatus. In another example, when the data replication apparatus is used to implement table-level data replication, the data replication apparatus is used to achieve the above-described... Figure 4 The technical solutions involved in the data replication device in the method embodiments shown, or the modules (such as chips) of the data replication device used to implement the above. Figure 4The technical solution involving the data replication device in the method embodiment shown can therefore also achieve the above-mentioned... Figure 4 The beneficial effects of the data replication apparatus in the illustrated method embodiments are shown. For example, in embodiments of this application, the data replication apparatus may be as follows: Figures 1a to 1d The intended data comes from the database server.

[0132] See Figure 6 The data copying device 600 includes an acquisition module 601 (or a transceiver module, communication module, used for sending and receiving data), a processing module 602, and a display module 603. The data copying device 600 is used to implement the above-mentioned... Figure 2 The data copying device in the method embodiment shown above functions, or is used to implement the above. Figure 4 The method embodiment shown illustrates the function of the data copying device.

[0133] When the data copying device 600 is used to implement Figure 2 In the illustrated method embodiment, the data replication device functions as follows: the display module 603 is used to display a configuration interface. The acquisition module 601 is used to acquire the first SQL replication statement corresponding to the first source database in response to the user's editing operation of the data replication conditions of the first source database in the configuration interface. The processing module 602 is used to copy the first target data in the first source database to the first target database corresponding to the first source database according to the first SQL replication statement.

[0134] Wherein, when the data copying device 600 is used to implement Figure 2 For a more detailed description of the data copying device in the method embodiment shown, regarding the acquisition module 601, processing module 602, and display module 603, please refer to the above. Figure 2 The description of the data copying device in the method embodiment shown will not be repeated here.

[0135] When the data copying device 600 is used to implement Figure 4 In the illustrated method embodiment, the data replication device functions as follows: The display module 603 is used to display a configuration interface. The acquisition module 601 is used to acquire the first SQL replication statement corresponding to the first table to be replicated in response to the user's editing operation on the data replication conditions of the first table to be replicated in the configuration interface, wherein the first table to be replicated is stored in the first source database. The processing module 602 is used to copy the first target data in the first table to be replicated to the first target table according to the first SQL replication statement corresponding to the first table to be replicated, wherein the first target table is stored in the first target database corresponding to the first source database.

[0136] Wherein, when the data copying device 600 is used to implement Figure 4For a more detailed description of the data copying device in the method embodiment shown, regarding the acquisition module 601, processing module 602, and display module 603, please refer to the above. Figure 4 The description of the data copying device in the method embodiment shown will not be repeated here.

[0137] Optionally, the above Figure 6 The acquisition module, processing module, and display module shown can all be implemented in software or hardware. For example, the implementation of the processing module will be described below. Similarly, the implementation of the acquisition module and display module can refer to the implementation of the processing module, and will not be repeated here.

[0138] When implemented in software, a processing module, as an example of a software functional unit, can be an application or code block running on a computer device. The computer device can be at least one of a physical host, virtual machine, container, or other computing device. Furthermore, there can be one or more computer devices. For example, a processing module can be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application can be distributed within the same availability zone (AZ) or in different AZs. Similarly, the multiple hosts / virtual machines / containers used to run the application can be distributed within the same region or in different regions. Typically, a region can include multiple AZs.

[0139] Similarly, multiple hosts / virtual machines / containers used to run the application can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple Availability Zones (AZs).

[0140] When implemented in hardware, the processing module, as an example of a hardware functional unit, may include at least one computing device, such as a server. Alternatively, the processing module may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0141] The processing module includes multiple computing devices that can be distributed within the same Availability Zone (AZ) or in different AZs. Similarly, the processing module can be distributed within the same region or in different regions. Likewise, the processing module can be distributed within the same Virtual Private Cloud (VPC) or multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0142] It should be noted that the steps implemented by the acquisition module, processing module, and display module can be specified as needed. By implementing different execution steps of the data copying method through the acquisition module, processing module, and display module, the full functionality of the data copying device can be achieved.

[0143] Furthermore, it should be noted that the module division in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The functional modules in the embodiments of this application can be integrated into one module, or each module can exist physically separately, or two or more modules can be integrated into one module. For example, the acquisition module and the processing module can be integrated into one module, or the acquisition module and the processing module can be the same module. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0144] Based on the same concept, this application also provides a possible computing device for executing the data copying method shown in the above method embodiments. Related features can be found in the above method embodiments, and will not be repeated here. Figure 7As shown, the computing device 700 includes: a bus 701, a processor 702, a communication interface 703, a memory 704, and a display 705. The processor 702, the communication interface 703, and the memory 704 communicate with each other via the bus 701. The computing device 700 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 700.

[0145] The 701 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus 701 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 701 may include a path for transmitting information between various components of the computing device 700 (e.g., memory 704, processor 702, communication interface 703, display 705).

[0146] Processor 702 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0147] Display 705 is used to present a configuration interface (such as a data copying condition configuration interface) to enable human-computer interaction. For example, display 705 can display the data copying condition configuration interface, showing information input by the user or information provided to the user.

[0148] Memory 704 may include volatile memory, such as random access memory (RAM). Processor 702 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0149] The memory 704 stores executable program code, which is executed by the processor 702 to implement the functions of the aforementioned data copying apparatus (such as the acquisition module and the processing module), and the display 705 to implement the functions of the display module in the aforementioned data copying apparatus, thereby realizing the data copying method provided in this application embodiment. That is, the memory 704 stores computer program instructions for executing the data copying method.

[0150] Alternatively, the memory 704 stores executable code, which the processor 702 executes to implement the functions of the aforementioned data copying apparatus (such as the acquisition module and the processing module), and the display 705 is used to implement the functions of the display module in the aforementioned data copying apparatus, thereby realizing the data copying method provided in the embodiments of this application. That is, the memory 704 stores computer program instructions for the data copying apparatus to execute the data copying method provided in the embodiments of this application.

[0151] The communication interface 703 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0152] Based on the same concept, this application also provides a possible computing device cluster for executing the methods shown in the above method embodiments. Related features can be found in the above method embodiments and will not be repeated here. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a desktop computer, a laptop computer, or a smartphone, or other terminal device.

[0153] like Figure 8 As shown, the computing device cluster 800 includes at least one computing device 700. The memory 704 of one or more computing devices 700 in the computing device cluster 800 may store the same instructions for executing the data copying method provided in the embodiments of this application.

[0154] In some possible implementations, the memory 704 of one or more computing devices 700 in the computing device cluster 800 may also store partial instructions for executing the data copying method. In other words, a combination of one or more computing devices 700 can jointly execute the instructions for executing the data copying method.

[0155] It should be noted that the memory 704 in the different computing devices 700 within the computing device cluster 800 can store different instructions, each used to execute a portion of the functions of the data copying device. That is, the instructions stored in the memory 704 of the different computing devices 700 can implement the functions of one or more modules, such as the acquisition module and the processing module. Optionally, the display 705 in the different computing devices 700 can be used to implement the functions of the display module.

[0156] In some possible implementations, one or more computing devices in the computing device cluster 800 can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc.

[0157] Based on the same concept, embodiments of this application also provide a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the data copying method provided in embodiments of this application.

[0158] Based on the same concept, embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to execute the data copying method provided in embodiments of this application.

[0159] Based on the same concept, this application also provides a computer chip coupled to a memory, which is used to read a computer program stored in the memory and execute the data copying method provided in this application.

[0160] Based on the same concept, embodiments of this application also provide a chip system including a processor for supporting a computer device in implementing the data copying method provided in embodiments of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. The chip system may be composed of computer chips or may include computer chips and other discrete devices.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A data copying method, characterized in that, include: In response to the user's edit operation on the data replication conditions of the first source database in the configuration interface, obtain the first structured query language SQL replication statement; According to the first SQL copy statement, the first target data in the first source database is copied to the first target database corresponding to the first source database; The method further includes: In response to the user's edit operation on the configuration interface regarding the data replication conditions of the second source database, a second SQL replication statement is obtained; According to the second SQL copy statement, a second copy channel is established between the second source database and the second target database, and the first part of the data included in the second target data in the second source database is copied to the second target database in its entirety; When the full copy of the first part of the data included in the second target data is completed, the binlog incremental copy of the second source library is performed through the second copy channel. The binlog incremental copy of the second source library is used to implement the copy of the second part of the data included in the second target data. The time when the second part of the data in the second target data is written to the second source library is later than the start time of the full copy of the first part of the data in the second target data. Specifically, when the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the second replication channel and the first replication channel are merged. The first replication channel is the replication channel between the first source database and the first target database, and the first replication channel is established based on the first SQL replication statement.

2. The method as described in claim 1, characterized in that, According to the first SQL copy statement, the target data in the first source database is copied to the first target database corresponding to the first source database, including: Copy the first portion of data included in the first target data to the first target library in its entirety; When the full copy of the first part of the data included in the first target data is completed, the binary log / logical log binlog incremental copy is performed on the first source database through the first copy channel. The binlog incremental copy of the first source database is used to implement the copy of the second part of the data included in the first target data. The time when the second part of the data in the first target data is written to the first source database is later than the start time of the full copy of the first part of the data in the first target data.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: In response to the user's instruction editing operation on the configuration interface for the first source library, first instruction information is obtained, which is used to instruct the copying of the first source library to be stopped; Based on the first instruction information, the copying of the first source library is stopped, and the first target library is deleted or its use is stopped.

4. A data copying method, characterized in that, include: In response to the user's editing operation on the data copying conditions of the first table to be copied in the configuration interface, obtain the first structured query language SQL copying statement; The first table to be copied is stored in the first source database; Based on the first SQL copy statement, copy the first target data from the first table to be copied to the first target table; The first target table is stored in the first target library corresponding to the first source library; The method further includes: In response to the user's edit operation on the data replication conditions of the second table to be replicated in the configuration interface, the second SQL replication statement is obtained, and the second table to be replicated is stored in the first source database; According to the second SQL copy statement, a second target database with hidden attributes is established, and a second copy channel is established between the first source database and the second target database. The first part of the data included in the second target data in the second table to be copied is copied to the second target table in its entirety. The second target database includes the second target table. When the full copy of the first part of the data included in the second target data is completed, the binlog incremental copy of the second table to be copied is performed through the second copy channel. The binlog incremental copy of the second table to be copied is used to realize the copy of the second part of the data included in the second target data. The time when the second part of the data in the second target data is written to the second table to be copied is later than the start time of the full copy of the first part of the data in the second target data. Specifically, when the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the second replication channel is merged with the first replication channel, and the first target database is merged with the second target database. The first replication channel is a replication channel between the first source database and the first target database, and the first replication channel is established based on the first SQL replication statement.

5. The method as described in claim 4, characterized in that, According to the first SQL copy statement, the first target data in the first table to be copied is copied to the first target table, including: Copy the first portion of data included in the first target data to the first target table in its entirety; When the full copy of the first part of the data included in the first target data is completed, the first table to be copied is incrementally copied using binary log / logical log (binlog) through the first copy channel. The incremental copy of the binlog corresponding to the first table to be copied is used to copy the second part of the data included in the first target data. The time when the second part of the data in the first target data is written to the first table to be copied is later than the start time of the full copy of the first part of the data in the first target data.

6. The method as described in claim 4 or 5, characterized in that, The method further includes: In response to the user's instruction to edit the first table to be copied in the configuration interface, first instruction information is obtained, which is used to instruct the copying of the first table to be copied to be stopped; Based on the first instruction information, the copying of the first table to be copied is stopped, and the first target table is deleted or its use is stopped.

7. A data copying device, characterized in that, include: The display module is used to display the configuration interface; The acquisition module is used to acquire the first SQL copy statement in response to the user's editing operation on the data copy conditions of the first source database in the configuration interface; The processing module is used to copy the first target data in the first source database to the first target database corresponding to the first source database according to the first SQL copy statement; The processing module is also used for: In response to the user's edit operation on the data replication conditions of the second source database in the configuration interface, obtain the second SQL replication statement; According to the second SQL copy statement, a second copy channel is established between the second source database and the second target database, and the first part of the data included in the second target data in the second source database is copied to the second target database in its entirety; When the full copy of the first part of the data included in the second target data is completed, the binlog incremental copy of the second source library is performed through the second copy channel. The binlog incremental copy of the second source library is used to implement the copy of the second part of the data included in the second target data. The time when the second part of the data in the second target data is written to the second source library is later than the start time of the full copy of the first part of the data in the second target data. Specifically, when the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the second replication channel and the first replication channel are merged. The first replication channel is the replication channel between the first source database and the first target database, and the first replication channel is established based on the first SQL replication statement.

8. The apparatus as claimed in claim 7, characterized in that, When the processing module copies the first target data from the first source database to the first target database corresponding to the first source database according to the first SQL copy statement, it is specifically used for: Copy the first portion of data included in the first target data to the first target library in its entirety; When the full copy of the first part of the data included in the first target data is completed, the binlog incremental copy of the first source library is performed through the first copy channel. The binlog incremental copy of the first source library is used to implement the copy of the second part of the data included in the first target data. The time when the second part of the data in the first target data is written to the first source library is later than the start time of the full copy of the first part of the data in the first target data.

9. The apparatus as claimed in claim 7 or 8, characterized in that, The processing module is also used for: In response to the user's instruction editing operation on the configuration interface for the first source library, first instruction information is obtained, which is used to instruct the copying of the first source library to be stopped; Based on the first instruction information, the copying of the first source library is stopped, and the first target library is deleted or its use is stopped.

10. A data copying device, characterized in that, include: The display module is used to display the configuration interface; The acquisition module is used to acquire the first SQL copy statement in response to the user's editing operation on the data copying conditions of the first table to be copied in the configuration interface, wherein the first table to be copied is stored in the first source database; The processing module is used to copy the first target data from the first table to be copied to the first target table according to the first SQL copy statement, wherein the first target table is stored in the first target database corresponding to the first source database; The processing module is also used for: In response to the user's edit operation on the data replication conditions of the second table to be replicated in the configuration interface, the second SQL replication statement is obtained, and the second table to be replicated is stored in the first source database; According to the second SQL copy statement, a second target database with hidden attributes is established, and a second copy channel is established between the first source database and the second target database. The first part of the data included in the second target data in the second table to be copied is copied to the second target table in its entirety. The second target database includes the second target table. When the full copy of the first part of the data included in the second target data is completed, the binlog incremental copy of the second table to be copied is performed through the second copy channel. The binlog incremental copy of the second table to be copied is used to realize the copy of the second part of the data included in the second target data. The time when the second part of the data in the second target data is written to the second table to be copied is later than the start time of the full copy of the first part of the data in the second target data. Specifically, when the binlog incremental replication progress of the second replication channel is greater than or equal to the binlog incremental replication progress of the first replication channel, the second replication channel is merged with the first replication channel, and the first target database is merged with the second target database. The first replication channel is established based on the first SQL replication statement.

11. The apparatus as claimed in claim 10, characterized in that, When the processing module copies the first target data from the first table to be copied to the first target table according to the first SQL copy statement, it is specifically used for: Copy the first portion of data included in the first target data to the first target table in its entirety; When the full copy of the first part of the data included in the first target data is completed, the binlog incremental copy of the first table to be copied is performed through the first copy channel. The binlog incremental copy of the first table to be copied is used to implement the copy of the second part of the data included in the first target data. The time when the second part of the data in the first target data is written to the first table to be copied is later than the start time of the full copy of the first part of the data in the first target data.

12. The apparatus as claimed in claim 10 or 11, characterized in that, The processing module is also used for: In response to the user's instruction to edit the first table to be copied in the configuration interface, first instruction information is obtained, which is used to instruct the copying of the first table to be copied to be stopped; Based on the first instruction information, the copying of the first table to be copied is stopped, and the first target table is deleted or its use is stopped.

13. A computing device, characterized in that, The computing device includes a display, a processor, and a memory; The display is used to show the configuration interface; The memory is used to store computer program instructions; The processor invokes computer program instructions stored in the memory to execute the method as described in any one of claims 1-3 or the method as described in any one of claims 4-6.

14. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, and each computing device includes a display, a processor, and a memory; the display in the at least one computing device is used to display a configuration interface. The memory in the at least one computing device is used to store computer program instructions; The processor in the at least one computing device invokes computer program instructions stored in the memory to execute the method as described in any one of claims 1-3 or the method as described in any one of claims 4-6.

15. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, enable the computing device to perform the method as described in any one of claims 1-3 or the method as described in any one of claims 4-6.

16. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the method as described in any one of claims 1-3 or the method as described in any one of claims 4-6.

Images