A database modification method and apparatus

Abstract

The application discloses a database changing and matching method and device, and relates to the technical field of databases. A specific implementation of the method comprises the following steps: a first database cluster is newly established, the first database cluster comprises a first master database and a first slave database; data of a second database cluster is copied to the first database cluster by using a specified database copying mode or a database copying mode determined based on a preset strategy; a read-only mode is started for the first database cluster and the second database cluster, and identity switching is performed on the second database cluster and the first database cluster; and the read-only mode of the first database cluster is closed. The implementation can solve the technical problems of long full data copying time and long incremental data synchronization time.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a database configuration method and apparatus. Background Technology

[0002] When you need to change the configuration of a database cluster, taking a public cloud MySQL database (also applicable to Percona and MariaDB) as an example, if you want to reduce the storage space of a database cluster with 2TB of storage space (actual data volume of 1.2TB) to 1.5TB (the user wants to downsize), you need to create a new database cluster (including a master database and a slave database). Then, copy the data from the slave database of the original database cluster to the master database of the new cluster, and then copy the data from the master database of the new cluster to the slave database of the new cluster. After the data of the master database of the new cluster and the master instance of the original cluster are consistent, switch over to complete the configuration change.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0004] 1) The need to fetch full data twice results in a long full data copy time, which prolongs the configuration change time and wastes computing resources and network bandwidth; 2) The delay between the master and slave databases results in a long incremental data synchronization time. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a database configuration method and apparatus to solve the technical problems of long full data copying time and long incremental data synchronization time.

[0006] To achieve the above objectives, according to one aspect of the present invention, a database configuration method is provided, comprising:

[0007] A new first database cluster is established, which includes a first master database and a first slave database.

[0008] The data of the second database cluster is copied to the first database cluster using a specified database copying method or a database copying method determined based on a preset strategy.

[0009] Enable read-only mode for the first database cluster and the second database cluster, and switch the identity of the second database cluster and the first database cluster.

[0010] Disable read-only mode for the first database cluster.

[0011] Optionally, data from the second database cluster is copied to the first database cluster using a specified database copying method or a database copying method determined based on a preset strategy, including:

[0012] Determine whether a data copy method has been specified;

[0013] If so, the data of the second database cluster is copied to the first database cluster according to the specified data copying method; wherein, the second database cluster includes a second master database and a second slave database;

[0014] If not, based on the latency between the second master database and the second slave database in the second database cluster, the data of the second database cluster is copied to the first database cluster using either a parallel copy method for the slave database or a serial copy method for the master database.

[0015] Optionally, based on the latency between the second master database and the second slave database in the second database cluster, data from the second database cluster is copied to the first database cluster using either a parallel copy method for the slave database or a serial copy method for the master database, including:

[0016] Determine whether the latency between the second master database and the second slave database in the second database cluster is greater than or equal to the first time threshold.

[0017] If so, the data from the second database cluster will be copied to the first database cluster using the primary database serial copy method.

[0018] If not, then the data from the second database cluster is copied to the first database cluster using a parallel copy method.

[0019] Optionally, the specified data copying method is one of the following: serial copying from the database, parallel copying from the database, serial copying from the master database, and parallel copying from the master database.

[0020] Optionally, if the specified data copying method is a serial copy from the database,

[0021] Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0022] Copy all data from the second slave database to the first master database, and start the database service of the first master database;

[0023] Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0024] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0025] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0026] Optionally, if the specified data copying method is a parallel copy from the database,

[0027] Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0028] Copy all data from the second slave database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0029] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0030] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0031] Optionally, if the specified data copying method is the primary database serial copying method,

[0032] Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0033] Copy all data from the second primary database to the first primary database and start the database service of the first primary database;

[0034] Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0035] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0036] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0037] Optionally, if the specified data copying method is the primary database parallel copying method,

[0038] Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0039] Copy all data from the second master database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0040] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0041] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0042] Optionally, the entire dataset of the second slave database is copied in parallel to the first master database and the first slave database, including:

[0043] Using the Xtrabackup and tee tools, the full data of the second slave database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

[0044] Optionally, the entire dataset of the second master database is copied in parallel to the first master database and the first slave database, including:

[0045] Using the Xtrabackup and tee tools, the full data of the second master database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

[0046] Optionally, read-only mode is enabled for both the first and second database clusters, and identity switching is performed between the second and first database clusters, including:

[0047] Determine whether the latency between the second primary database and the first primary database is less than a second time threshold.

[0048] If so, then read-only mode is enabled for the first database cluster and the second database cluster until the latency between the second primary database and the first primary database is zero.

[0049] Switch the identity of the second database cluster to the identity of the first database cluster.

[0050] Additionally, according to another aspect of the present invention, a database configuration device is provided, comprising:

[0051] A new module is created to create a first database cluster, which includes a first master database and a first slave database.

[0052] The copy module is used to copy data from the second database cluster to the first database cluster using a specified database copy method or a database copy method determined based on a preset strategy.

[0053] The switching module is used to enable read-only mode for the first database cluster and the second database cluster, and to switch the identity of the second database cluster and the first database cluster.

[0054] The shutdown module is used to disable the read-only mode of the first database cluster.

[0055] Optionally, the copy module is further configured to:

[0056] Determine whether a data copy method has been specified;

[0057] If so, the data of the second database cluster is copied to the first database cluster according to the specified data copying method; wherein, the second database cluster includes a second master database and a second slave database;

[0058] If not, based on the latency between the second master database and the second slave database in the second database cluster, the data of the second database cluster is copied to the first database cluster using either a parallel copy method for the slave database or a serial copy method for the master database.

[0059] Optionally, the copy module is further configured to:

[0060] Determine whether the latency between the second master database and the second slave database in the second database cluster is greater than or equal to the first time threshold.

[0061] If so, the data from the second database cluster will be copied to the first database cluster using the primary database serial copy method.

[0062] If not, then the data from the second database cluster is copied to the first database cluster using a parallel copy method.

[0063] Optionally, the specified data copying method is one of the following: serial copying from the database, parallel copying from the database, serial copying from the master database, and parallel copying from the master database.

[0064] Optionally, if the specified data copying method is a serial copy from the database,

[0065] The copy module is also used for:

[0066] Copy all data from the second slave database to the first master database, and start the database service of the first master database;

[0067] Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0068] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0069] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0070] Optionally, if the specified data copying method is a parallel copy from the database,

[0071] The copy module is also used for:

[0072] Copy all data from the second slave database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0073] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0074] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0075] Optionally, if the specified data copying method is the primary database serial copying method,

[0076] The copy module is also used for:

[0077] Copy all data from the second primary database to the first primary database and start the database service of the first primary database;

[0078] Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0079] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0080] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0081] Optionally, if the specified data copying method is the primary database parallel copying method,

[0082] The copy module is also used for:

[0083] Copy all data from the second master database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0084] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0085] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0086] Optionally, the copy module is further configured to:

[0087] Using the Xtrabackup and tee tools, the full data of the second slave database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

[0088] Optionally, the copy module is further configured to:

[0089] Using the Xtrabackup and tee tools, the full data of the second master database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

[0090] Optionally, the switching module is further configured to:

[0091] Determine whether the latency between the second primary database and the first primary database is less than a second time threshold.

[0092] If so, then read-only mode is enabled for the first database cluster and the second database cluster until the latency between the second primary database and the first primary database is zero.

[0093] Switch the identity of the second database cluster to the identity of the first database cluster.

[0094] According to another aspect of the present invention, an electronic device is also provided, comprising:

[0095] One or more processors;

[0096] Storage device for storing one or more programs.

[0097] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0098] According to another aspect of the present invention, a computer-readable medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0099] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0100] One embodiment of the above invention has the following advantages or beneficial effects: By employing a specified database copying method or a database copying method determined based on a preset strategy to copy data from the second database cluster to the first database cluster, enabling read-only mode for both the first and second database clusters, switching the identity of the second and first database clusters, and disabling read-only mode, the technical problems of long full data copying time and long incremental data synchronization time in the prior art are overcome. This invention can shorten the full data copying time and incremental data synchronization time; moreover, by utilizing the characteristics of Xtrabackup streaming backup, it supports parallel data copying, and by combining it with the tee tool for multi-destination distribution, the process of repeated data reading and compression is reduced; furthermore, by selecting the data copying method based on the latency between the master and slave databases, the reconfiguration time is shortened and the reconfiguration efficiency is improved.

[0101] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0102] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0103] Figure 1 This is a schematic diagram of the main flow of the database configuration method according to an embodiment of the present invention;

[0104] Figures 2a-2d This is a flowchart illustrating four data copying methods according to embodiments of the present invention;

[0105] Figure 3a This is a flowchart illustrating the process of executing two data copy commands serially in the prior art;

[0106] Figure 3b This is a flowchart illustrating the parallel execution of two data copy commands according to an embodiment of the present invention;

[0107] Figure 4 This is a schematic diagram of the main flow of a database configuration method according to a possible embodiment of the present invention;

[0108] Figure 5 This is a schematic diagram of the main modules of the database configuration device according to an embodiment of the present invention;

[0109] Figure 6 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0110] Figure 7 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0111] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0112] Figure 1 This is a schematic diagram of the main flow of the database configuration method according to an embodiment of the present invention. As one embodiment of the present invention, such as... Figure 1 As shown, the database configuration method may include:

[0113] Step 101: Create a new first database cluster, which includes a first master database and a first slave database.

[0114] First, create a new database cluster, and then configure the database cluster according to the configuration information, such as storage space, CPU, memory, etc.

[0115] It should be noted that, in order to distinguish between the newly built database cluster and the original database cluster, this embodiment of the invention refers to the newly built database cluster as the first database cluster, and correspondingly, the master and slave databases in the newly built database cluster are referred to as the first master database and the first slave database, respectively; the original database cluster is referred to as the second database cluster, and correspondingly, the master and slave databases in the original database cluster are referred to as the second master database and the second slave database, respectively.

[0116] Step 102: Copy the data from the second database cluster to the first database cluster using a specified database copy method or a database copy method determined based on a preset strategy.

[0117] Next, the data on the original database cluster is copied to the newly created database cluster using the specified database copy method or the database copy method determined based on a preset strategy.

[0118] In embodiments of the present invention, a data copying method can be specified. The specified data copying methods include serial copying from the database, parallel copying from the database, serial copying from the main database, and parallel copying from the main database. One of these four data copying methods can be specified, and then the specified data copying method is used to copy the data from the second database cluster to the first database cluster.

[0119] Optionally, step 102 may include: determining whether a data copying method has been specified; if yes, copying the data of the second database cluster to the first database cluster according to the specified data copying method; wherein the second database cluster includes a second master database and a second slave database; if no, copying the data of the second database cluster to the first database cluster based on the delay time between the second master database and the second slave database in the second database cluster, using either a parallel copying method for the slave database or a serial copying method for the master database. First, it determines whether a data copying method has been specified. If a data copying method has been specified, then the data of the second database cluster is copied to the first database cluster according to the specified data copying method. If no data copying method has been specified, then an adaptive data copying method is selected to copy the data of the second database cluster to the first database cluster, making the data copying method very flexible and able to meet the configuration requirements of various read / write scenarios.

[0120] Optionally, based on the delay time between the second master database and the second slave database in the second database cluster, data from the second database cluster is copied to the first database cluster using either a parallel copy method for the slave database or a serial copy method for the master database. This includes: determining whether the delay time between the second master database and the second slave database in the second database cluster is greater than or equal to a first time threshold; if so, then the data from the second database cluster is copied to the first database cluster using the serial copy method for the master database; if not, then the data from the second database cluster is copied to the first database cluster using the parallel copy method for the slave database. This embodiment of the invention selects a suitable data copying method based on the delay time between the second master database and the second slave database in the second database cluster. If the delay time is greater than or equal to a preset first time threshold, then the data from the second database cluster is copied to the first database cluster using the serial copy method for the master database to shorten the incremental data synchronization time between the first master database and the second master database; otherwise, the data from the second database cluster can be copied to the first database cluster using the parallel copy method for the slave database.

[0121] The following describes four data copying methods in detail:

[0122] Optionally, such as Figure 2a As shown, if the specified data copying method is a serial copy from the database, then the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0123] Step 1) Copy all data from the second slave database to the first master database and start the database service of the first master database;

[0124] Step 2) Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0125] Step 3) Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0126] Step 4) Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0127] In step 1), the full data of the second slave database can be copied to the first master database using a streaming method. After the full data copy is completed, the database service (such as MySQL service) of the first master database is started, i.e., path ① in the figure.

[0128] It should be noted that during the data copying process, the second database cluster is writing data. After the copy is completed, the first primary database takes a relatively long time to start. During this window period, the data is not copied to the first database cluster. Therefore, the data in the first database cluster and the second database cluster are inconsistent.

[0129] In step 2), the full data of the first master database can be copied to the first slave database using a streaming method. After the full data copy is completed, the database service (such as MySQL service) of the first slave database is started. At this time, the data of the first slave database is consistent with the data of the first master and slave database, i.e., path ② in the figure.

[0130] In step 3), a master-slave replication command is executed on the first slave database to synchronize incremental data from the first master database, i.e., path ③ in the diagram. Since no data has been written to the first master database, the latency between the first master and the first slave database is zero at this time. It should be noted that after the master-slave replication command is executed on the first slave database, a TCP communication connection is established between the first slave database and the first master database, and the connection is maintained. Once the first master database has incremental data, it will trigger the synchronization operation of the incremental data, thereby ensuring data consistency between the master and slave databases.

[0131] In step 4), a master-slave replication command is executed on the first master database to synchronize incremental data from the second master database, i.e., path ④ in the diagram. There is a significant delay at this point because the second master database is continuously writing data during steps 2) and 3). It should be noted that after the master-slave replication command is executed on the first master database, a TCP communication connection is established between the first and second master databases and maintained. Once the second master database receives incremental data, it triggers the synchronization operation for that incremental data, thus ensuring data consistency between the first and second master databases.

[0132] It should also be noted that the master and slave in the master-slave replication command are relative. In step 3), master refers to the first master database and slave refers to the first slave database; however, in step 4), master refers to the second master database and slave refers to the first master database.

[0133] As can be seen from the above process description, the entire data needs to be copied twice, and the content copied in both copies is the same, resulting in repeated hard drive read operations and CPU calculations. Furthermore, the longer the second data copy takes, the greater the delay in step ④ between the first master database and the second master database, thus delaying the configuration change time. The serial copy method from the database takes the longest and is generally no longer used in production, only in very rare scenarios.

[0134] Optionally, such as Figure 2b As shown, if the specified data copying method is a parallel copy from the database, then the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0135] Step 1) Copy all data from the second slave database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0136] Step 2): Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database.

[0137] Step 3) Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0138] Unlike the serial copy method from the database, the parallel copy method from the database distributes the data to multiple destinations on the database to achieve the purpose of parallel copying.

[0139] In step 1), the full data of the second slave database can be copied in parallel to the first master database and the first slave database using a streaming method, i.e., path ① in the figure. After the full data copy is completed, the database services (such as MySQL services) of the first master database and the first slave database are started. At this time, the data of the first master database and the data of the first slave database are consistent.

[0140] Copying the full data from the second slave database to the first master and first slave databases in parallel includes: using the Xtrabackup and tee tools to copy the full data from the second slave database into two copies, and then simultaneously copying these two copies to the first master and first slave databases. During this parallel copying process, the command is not executed twice. Figure 3a As shown, instead of using Linux pipe technology, it distributes data streams, such as... Figure 3b As shown. Figure 3b As shown, with Figure 3a The difference is that only one Xtrabackup tool needs to be started, which reduces CPU usage and hard drive read operations. At the same time, combined with the tee tool, it can easily copy a stream into two streams and distribute these two streams separately, thereby significantly improving data copying efficiency.

[0141] Xtrabackup is a MySQL physical backup tool. Tee is a Linux command used to read data from standard input and output its contents.

[0142] In step 2), a master-slave replication command is executed on the first slave database to synchronize incremental data from the first master database, i.e., path ② in the diagram. Since no data has been written to the first master database, the latency between the first master and the first slave database is zero at this time. It should be noted that after the master-slave replication command is executed on the first slave database, a TCP communication connection is established between the first slave database and the first master database, and the connection is maintained. Once the first master database has incremental data, it will trigger the synchronization operation of incremental data, thereby ensuring data consistency between the master and slave databases.

[0143] In step 3), a master-slave replication command is executed on the first master database to synchronize incremental data from the second master database, i.e., path ③ in the diagram. There is a significant delay at this point because the second master database is continuously writing data during steps 1) and 2). It should be noted that after the master-slave replication command is executed on the first master database, a TCP communication connection is established between the first and second master databases and maintained. Once the second master database receives incremental data, it triggers the synchronization operation for that incremental data, thus ensuring data consistency between the first and second master databases.

[0144] As can be seen from the above process description, the data only needs to be copied once to complete the copying of two copies, and the configuration time can be shortened to 50-60% of the original, which is applicable to most scenarios.

[0145] Optionally, such as Figure 2c As shown, if the specified data copying method is the primary database serial copying method, then the data of the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0146] Step 1) Copy all data from the second primary database to the first primary database and start the database service of the first primary database;

[0147] Step 2) Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0148] Step 3) Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0149] Step 4) Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0150] This method is similar to the serial copy method from the database, so it will not be elaborated further. The difference is that the data copy is initiated from the second master database, which is suitable for scenarios where the master-slave latency of the second database cluster is large. For example, if the latency between the second master and slave databases is 2 hours, if data is still copied from the second slave database, then the time required for path ④ to synchronize incremental data will be 2 hours longer, which will increase the overall reconfiguration time by 2 hours. In addition, if there are services running on the second master database, from the perspective of ensuring service stability, adopting the serial method can reduce network bandwidth usage.

[0151] Optionally, such as Figure 2dAs shown, if the specified data copying method is the primary database parallel copying method, then the data of the second database cluster is copied to the first database cluster according to the specified data copying method, including:

[0152] Step 1) Copy all data from the second master database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0153] Step 2): Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database.

[0154] Step 3) Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0155] Optionally, copying the full data of the second master database to the first master database and the first slave database in parallel includes: using the Xtrabackup tool and the tee tool to copy the full data of the second master database into two copies of data to be copied, thereby copying the two copies of data to be copied to the first master database and the first slave database simultaneously.

[0156] This method is the fastest way to change configurations. It combines the advantages of parallel copying from the secondary database and serial copying from the primary database, reducing data copying time and the time for synchronizing incremental data. However, because it initiates parallel copying on the secondary primary database, it consumes more network bandwidth. It is suitable for emergency situations where configuration changes need to be completed quickly to solve resource shortages.

[0157] Step 103: Enable read-only mode for the first database cluster and the second database cluster, and switch the identity of the second database cluster and the first database cluster.

[0158] After copying the data from the second database cluster to the first database cluster, determine whether the latency between the second primary database and the first primary database is less than a second time threshold. If so, enable read-only mode for both the first and second database clusters. After enabling read-only mode, stop write operations on the second database cluster until the latency between the second primary database and the first primary database is zero. Then, switch the identity of the second database cluster and the first database cluster (e.g., ID and / or IP).

[0159] Step 104: Disable read-only mode of the first database cluster.

[0160] After switching identities, the first database cluster becomes the second database cluster. Therefore, disabling read-only mode is equivalent to disabling read-only mode on the newly created first database cluster, allowing read and write operations on the newly created first database cluster.

[0161] Based on the various embodiments described above, it can be seen that the embodiments of the present invention solve the technical problems of long full data copying time and long incremental data synchronization time in the prior art by adopting a parallel copying method from the primary database to the secondary database or a serial copying method from the primary database to the secondary database, enabling read-only mode for both the primary and secondary database clusters, switching the identity of the secondary and primary database clusters, and disabling the read-only mode of the primary database cluster. The embodiments of the present invention can shorten the full data copying time and the incremental data synchronization time; moreover, by utilizing the characteristics of Xtrabackup streaming backup, it supports parallel data copying, and by combining it with the tee tool for multi-destination distribution, it reduces the process of repeated data reading and compression; furthermore, by selecting the data copying method based on the latency between the primary and secondary databases, it shortens the configuration change time and improves configuration change efficiency.

[0162] Figure 4 This is a schematic diagram of the main flow of a database configuration method according to a possible embodiment of the present invention. As another embodiment of the present invention, such as... Figure 4 As shown, the database configuration method may include:

[0163] Step 401: Create a new first database cluster, which includes a first master database and a first slave database.

[0164] Step 402: Determine whether a data copying method has been specified; if yes, proceed to step 403; if no, proceed to step 404.

[0165] Step 403: Copy the data from the second database cluster to the first database cluster according to the specified data copying method; wherein, the second database cluster includes a second master database and a second slave database.

[0166] The specified data copying method is one of the following: serial copying from the database, parallel copying from the database, serial copying from the master database, and parallel copying from the master database.

[0167] Optionally, a copy strategy can be pre-configured, and then the data copy method can be matched according to the request parameters. Different request parameters can be matched with different data copy methods.

[0168] Step 404: Determine whether the delay time between the second master database and the second slave database in the second database cluster is greater than or equal to the first time threshold; if yes, proceed to step 405; if no, proceed to step 406.

[0169] Step 405: Using the master database serial copy method, copy the data from the second database cluster to the first database cluster.

[0170] Step 406: Copy the data from the second database cluster to the first database cluster using a parallel copy method.

[0171] Step 407: Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database; execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0172] Step 408: Determine whether the delay time between the second primary database and the first primary database is less than the second time threshold; if yes, proceed to step 409; if no, wait until it is less than the second time threshold.

[0173] A second time threshold can be pre-configured, such as 120 seconds, 100 seconds, 200 seconds, etc.

[0174] Step 409: Enable read-only mode for the first database cluster and the second database cluster until the latency between the second primary database and the first primary database is zero.

[0175] When the latency between the second primary database and the first primary database is zero, check that EXEC_GTID is consistent.

[0176] GTID: Global Transaction IDentifier is a globally unique identifier, with one GTID corresponding to one transaction.

[0177] EXEC_GTID: Records which GTIDs the current instance is executing.

[0178] Step 410: Switch the identity of the second database cluster with the identity of the first database cluster.

[0179] For example, swapping the IDs of the second and first database clusters ensures that the IDs used by upstream and downstream systems such as transaction billing remain unchanged; swapping the LB IPs of the second and first database clusters ensures that the LB IPs used by business users remain unchanged.

[0180] Step 411: Disable read-only mode for the first database cluster.

[0181] Once the switch is complete, data writing can be enabled. After switching identities, the first database cluster becomes the second database cluster. Therefore, disabling read-only mode is equivalent to disabling read-only mode on the newly created first database cluster, allowing read and write operations on the newly created first database cluster.

[0182] Step 412: Clean up the second database cluster.

[0183] You can first back up the data of the second database cluster, and then clean up the resources of the second database cluster, such as releasing resources like CPU, memory, storage space, and IP, thereby completing the configuration change.

[0184] Furthermore, the specific implementation details of the database configuration method in one of the reference embodiments of the present invention have been described in detail in the database configuration method described above, so the details will not be repeated here.

[0185] Figure 5 This is a schematic diagram of the main modules of a database configuration device according to an embodiment of the present invention. Figure 5 As shown, the database configuration device 500 includes a creation module 501, a copy module 502, a switching module 503, and a shutdown module 504. The creation module 501 creates a first database cluster, which includes a first master database and a first slave database. The copy module 502 copies data from a second database cluster to the first database cluster using a specified database copy method or a database copy method determined based on a preset strategy. The switching module 503 enables read-only mode for both the first database and the second database cluster and switches the identity between the second database cluster and the first database cluster. The shutdown module 504 disables the read-only mode for the first database cluster.

[0186] Optionally, the copy module 502 is further configured to:

[0187] Determine whether a data copy method has been specified;

[0188] If so, the data of the second database cluster is copied to the first database cluster according to the specified data copying method; wherein, the second database cluster includes a second master database and a second slave database;

[0189] If not, based on the latency between the second master database and the second slave database in the second database cluster, the data of the second database cluster is copied to the first database cluster using either a parallel copy method for the slave database or a serial copy method for the master database.

[0190] Optionally, the copy module 502 is further configured to:

[0191] Determine whether the latency between the second master database and the second slave database in the second database cluster is greater than or equal to the first time threshold.

[0192] If so, the data from the second database cluster will be copied to the first database cluster using the primary database serial copy method.

[0193] If not, then the data from the second database cluster is copied to the first database cluster using a parallel copy method.

[0194] Optionally, the specified data copying method is one of the following: serial copying from the database, parallel copying from the database, serial copying from the master database, and parallel copying from the master database.

[0195] Optionally, if the specified data copying method is a serial copy from the database,

[0196] The copy module 502 is further used for:

[0197] Copy all data from the second slave database to the first master database, and start the database service of the first master database;

[0198] Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0199] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0200] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0201] Optionally, if the specified data copying method is a parallel copy from the database,

[0202] The copy module 502 is further used for:

[0203] Copy all data from the second slave database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0204] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0205] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0206] Optionally, if the specified data copying method is the primary database serial copying method,

[0207] The copy module 502 is further used for:

[0208] Copy all data from the second primary database to the first primary database and start the database service of the first primary database;

[0209] Copy all data from the first master database to the first slave database and start the database service of the first slave database;

[0210] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0211] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0212] Optionally, if the specified data copying method is the primary database parallel copying method,

[0213] The copy module 502 is further used for:

[0214] Copy all data from the second master database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database.

[0215] Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database;

[0216] Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

[0217] Optionally, the copy module 502 is further configured to:

[0218] Using the Xtrabackup and tee tools, the full data of the second slave database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

[0219] Optionally, the copy module 502 is further configured to:

[0220] Using the Xtrabackup and tee tools, the full data of the second master database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

[0221] Optionally, the switching module 503 is further configured to:

[0222] Determine whether the latency between the second primary database and the first primary database is less than a second time threshold.

[0223] If so, then read-only mode is enabled for the first database cluster and the second database cluster until the latency between the second primary database and the first primary database is zero.

[0224] Switch the identity of the second database cluster to the identity of the first database cluster.

[0225] It should be noted that the specific implementation details of the database reconfiguration device described in this invention have been described in detail in the database reconfiguration method described above, so the details will not be repeated here.

[0226] Figure 6 An exemplary system architecture 600 is shown that can be applied to the database configuration method or database configuration apparatus of the present invention.

[0227] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, and 603, a network 604, and a server 605. Network 604 serves as the medium for providing communication links between terminal devices 601, 602, and 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0228] Users can use terminal devices 601, 602, and 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0229] Terminal devices 601, 602, and 603 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0230] Server 605 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 601, 602, and 603 (this is just an example). The backend management server can analyze and process data such as received item information query requests, and then feed the processing results back to the terminal devices.

[0231] It should be noted that the database configuration method provided in this embodiment of the invention is generally executed by server 605, and correspondingly, the database configuration device is generally set in server 605.

[0232] It should be understood that Figure 6 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0233] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing a terminal device of the present invention. Figure 7 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0234] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0235] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0236] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of this invention.

[0237] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0238] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer programs according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0239] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be located in a processor; for example, a processor can be described as including a new creation module, a copy module, a switching module, and a shutdown module. The names of these modules do not necessarily limit the functionality of the module itself.

[0240] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device implements the following method: creating a first database cluster, the first database cluster including a first master database and a first slave database; copying data from a second database cluster to the first database cluster using a specified database copying method or a database copying method determined based on a preset strategy; enabling read-only mode for both the first and second database clusters, and switching the identity of the second database cluster from that of the first database cluster; and disabling read-only mode for the first database cluster.

[0241] In another aspect, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0242] According to the technical solution of the present invention, by employing a parallel copy method from the primary database to the secondary database cluster or a serial copy method from the primary database cluster to copy data from the secondary database cluster to the primary database cluster, enabling read-only mode for both the primary and secondary database clusters, switching the identity of the secondary database cluster from the primary database cluster to the primary database cluster, and disabling read-only mode for the primary database cluster, the technical problems of long full data copying time and long incremental data synchronization time in the prior art are overcome. The embodiments of the present invention can shorten the full data copying time and incremental data synchronization time; moreover, by utilizing the characteristics of Xtrabackup streaming backup, it supports parallel data copying, and by combining it with the tee tool for multi-destination distribution, it reduces the process of repeated data reading and compression; furthermore, by selecting the data copying method based on the latency between the primary and secondary databases, it shortens the configuration change time and improves configuration change efficiency.

[0243] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A database configuration method, characterized in that, include: A new first database cluster is established, which includes a first master database and a first slave database. The data of the second database cluster is copied to the first database cluster using a specified database copying method or a database copying method determined based on a preset strategy. The second database cluster is the original database cluster; Enable read-only mode for the first database cluster and the second database cluster, and switch the identity of the second database cluster and the first database cluster. Disable read-only mode for the first database cluster; Copying data from the second database cluster to the first database cluster using a specified database copying method or a database copying method determined based on a preset strategy includes: Determine whether a data copying method has been specified; If so, the data of the second database cluster is copied to the first database cluster according to the specified data copying method; wherein, the second database cluster includes a second master database and a second slave database; If not, based on the delay time between the second master database and the second slave database in the second database cluster, the data of the second database cluster is copied to the first database cluster using either the parallel copy method of the slave database or the serial copy method of the master database. The parallel copy method for the second slave database refers to copying the data from the second slave database to the first master database and the first slave database in parallel. The serial copy method for the master database refers to copying the data from the second slave database to the first master database, and then copying the data from the first master database to the first slave database.

2. The method according to claim 1, characterized in that, Based on the latency between the second master database and the second slave database in the second database cluster, data from the second database cluster is copied to the first database cluster using either a parallel copy method for the slave database or a serial copy method for the master database, including: Determine whether the latency between the second master database and the second slave database in the second database cluster is greater than or equal to the first time threshold. If so, the data from the second database cluster will be copied to the first database cluster using the primary database serial copy method. If not, then the data from the second database cluster is copied to the first database cluster using a parallel copy method.

3. The method according to claim 1, characterized in that, The specified data copying method is one of the following: serial copy from the database, parallel copy from the database, serial copy from the master database, and parallel copy from the master database.

4. The method according to claim 3, characterized in that, If the specified data copying method is serial copy from the database... Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including: Copy all data from the second slave database to the first master database, and start the database service of the first master database; Copy all data from the first master database to the first slave database and start the database service of the first slave database; Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database; Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

5. The method according to claim 3, characterized in that, If the specified data copying method is parallel copying from the database... Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including: Copy all data from the second slave database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database. Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database; Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

6. The method according to claim 3, characterized in that, If the specified data copying method is the primary database serial copy method. Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including: Copy all data from the second primary database to the first primary database and start the database service of the first primary database; Copy all data from the first master database to the first slave database and start the database service of the first slave database; Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database; Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

7. The method according to claim 3, characterized in that, If the specified data copying method is the primary database parallel copying method. Then, the data from the second database cluster is copied to the first database cluster according to the specified data copying method, including: Copy all data from the second master database to the first master database and the first slave database in parallel, and start the database service of the first master database and the database service of the first slave database. Execute a master-slave replication command on the first slave database to synchronize incremental data from the first master database to the first slave database; Execute a master-slave replication command on the first master database to synchronize incremental data from the second master database to the first master database.

8. The method according to claim 5, characterized in that, Copying all data from the second slave database to the first master database and the first slave database in parallel includes: Using the Xtrabackup and tee tools, the full data of the second slave database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

9. The method according to claim 7, characterized in that, Copying all data from the second master database to the first master database and the first slave database in parallel includes: Using the Xtrabackup and tee tools, the full data of the second master database is copied into two copies, which are then copied to the first master database and the first slave database simultaneously.

10. The method according to claim 1, characterized in that, Enabling read-only mode for both the first and second database clusters, and switching the identity of the second database cluster from that of the first database cluster, including: Determine whether the latency between the second primary database and the first primary database is less than a second time threshold. If so, then read-only mode is enabled for the first database cluster and the second database cluster until the latency between the second primary database and the first primary database is zero. Switch the identity of the second database cluster to the identity of the first database cluster.

11. A database configuration device, characterized in that, include: A new module is created to create a first database cluster, which includes a first master database and a first slave database. The copy module is used to copy data from the second database cluster to the first database cluster using a specified database copy method or a database copy method determined based on a preset strategy. The second database cluster is the original database cluster; The switching module is used to enable read-only mode for the first database cluster and the second database cluster, and to switch the identity of the second database cluster and the first database cluster. The shutdown module is used to disable the read-only mode of the first database cluster; The copy module is also used for: Determine whether a data copying method has been specified; If so, the data of the second database cluster is copied to the first database cluster according to the specified data copying method; wherein, the second database cluster includes a second master database and a second slave database; If not, based on the delay time between the second master database and the second slave database in the second database cluster, the data of the second database cluster is copied to the first database cluster using either the parallel copy method of the slave database or the serial copy method of the master database. The parallel copy method for the second slave database refers to copying the data from the second slave database to the first master database and the first slave database in parallel. The serial copy method for the master database refers to copying the data from the second slave database to the first master database, and then copying the data from the first master database to the first slave database.

12. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-10.

13. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-10.

Images