Restart method of data synchronization tool and related device
By determining the process liveness status of the data synchronization tool and obtaining and modifying the network address of the Redis cluster node, the problem of the data synchronization tool being unable to restart due to master-slave node switching in Redis cluster mode was solved, thus achieving stable data synchronization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE INTERNET CO LTD
- Filing Date
- 2024-07-24
- Publication Date
- 2026-04-28
AI Technical Summary
In existing technologies, when switching between master and slave nodes in a Redis cluster, data synchronization tools fail to restart successfully, especially monitoring tools that cannot identify and connect to the switched-off nodes.
By determining whether the data synchronization tool's process is alive, obtaining the network addresses of available nodes in the source and target database clusters, and modifying configuration items in the configuration file to ensure that the data synchronization tool can restart, including using custom files and preset placeholders to automatically identify and connect to new nodes.
Even after a master-slave switch in a Redis cluster, the data synchronization tool can identify and connect to the correct node, avoiding the problem of being unable to restart and ensuring the continuity of the data synchronization process.
Smart Images

Figure CN118972435B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data synchronization technology, and in particular to a method for restarting a data synchronization tool and related equipment. Background Technology
[0002] Redis data synchronization tools, such as RedisShake, are used to replicate and synchronize Redis data. However, network problems, program errors, and other issues may cause the Redis data synchronization tool's process to exit unexpectedly, thus interrupting the data synchronization process.
[0003] To address the above situation, existing technologies use monitoring tools such as Zabbix, Nagios, or Prometheus to monitor the process of the Redis data synchronization tool. When the Redis data synchronization tool's process is detected to have exited, it is restarted to ensure that data replication and synchronization can continue.
[0004] However, when a Redis data synchronization tool is used to synchronize data in a Redis cluster, if a master-slave node switch occurs in the Redis cluster where data synchronization is being performed, there will be a problem that even if the monitoring tool detects that the Redis data synchronization tool's process has exited, it will not be able to successfully restart the Redis data synchronization tool's process. Summary of the Invention
[0005] This application provides a method for restarting a data synchronization tool to solve the problem in the prior art where, when using a data synchronization tool to synchronize data in a Redis cluster mode, a master-slave node switch occurs in the Redis cluster, causing the data synchronization tool's process to fail to restart.
[0006] This application also provides a restart device for a data synchronization tool, an electronic device, a computer-readable storage medium, and a computer program product.
[0007] The embodiments of this application adopt the following technical solutions:
[0008] Firstly, this application provides a method for restarting a data synchronization tool, comprising:
[0009] Determine if the data synchronization tool's process is alive; the data synchronization tool is used to synchronize the data to be synchronized from the source database cluster to the target database cluster.
[0010] When the process is not alive, obtain the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster.
[0011] Based on the network addresses of the first and second nodes, the configuration items for data synchronization in the data synchronization tool's configuration file are modified to obtain the modified target configuration file. The configuration items include the network address of the source database and the network address of the target database.
[0012] The process of the data synchronization tool is restarted based on the modified target configuration file.
[0013] Optionally, the first node is the master node in the source database cluster; the second node is the master node in the target database cluster.
[0014] Optionally, obtain the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster, including:
[0015] Obtain the node labels of the first cluster node in the source database cluster and the node labels of the second cluster node in the target database cluster; the node labels are used to characterize the role and functional attributes of the first or second cluster node.
[0016] The first node and the second node are determined based on the node labels of the first cluster node and the second cluster node.
[0017] Based on the first node and the second node, obtain the network address of the first node and the network address of the second node.
[0018] Optionally, before obtaining the node labels of the first cluster nodes in the source database cluster and the node labels of the second cluster nodes in the target database cluster, the method further includes:
[0019] Based on a pre-configured custom file, a preset command line for connecting to the database cluster nodes is executed to establish communication connections with the first and second cluster nodes.
[0020] A pre-configured custom file contains the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster.
[0021] Optional, pre-configured custom files can be configured as follows:
[0022] Create a custom file for the data synchronization tool to manage the source database cluster and the target database cluster. The custom file includes the network address of the source database and the network address of the target database.
[0023] Configure the network address of the source database in the custom file to be the network address of all cluster nodes in the source database cluster;
[0024] Configure the network address of the target database in the custom file to be the network address of all cluster nodes in the target database cluster.
[0025] Optionally, obtain the node labels of the first cluster node in the source database cluster and the node labels of the second cluster node in the target database cluster, including:
[0026] Based on a pre-configured custom file, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster are read in a loop.
[0027] Based on the communication connection, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster, a preset command line is executed to obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
[0028] Optionally, the method also includes:
[0029] By pre-selecting and replacing the data synchronization configuration items in the data synchronization tool's configuration file with preset placeholders, then...
[0030] Based on the network addresses of the first and second nodes, modify the configuration items for data synchronization in the data synchronization tool's configuration file to obtain the modified target configuration file, which includes:
[0031] Based on the network addresses of the first and second nodes, the preset placeholders in the configuration file of the data synchronization tool are modified to obtain the modified target configuration file.
[0032] Optionally, determine whether the data synchronization tool's process is alive, including:
[0033] Create a monitoring script to detect whether the process of the data synchronization tool is alive, and a scheduled task that can execute the monitoring script;
[0034] A monitoring script is executed based on a scheduled task to determine whether the data synchronization tool's process is alive.
[0035] Optionally, the network address of the first node includes the IP address and port address of the first node; the network address of the second node includes the IP address and port address of the second node.
[0036] Optionally, the first cluster node includes normal nodes in the source database cluster that did not fail after the master-slave node switch; the second cluster node includes normal nodes in the target database cluster that did not fail after the master-slave node switch.
[0037] Secondly, this application provides a restart device for a data synchronization tool, comprising a judgment module, an acquisition module, a modification module, and a restart module, wherein:
[0038] The judgment module is used to determine whether the process of the data synchronization tool is alive; the data synchronization tool is used to synchronize the data to be synchronized from the source database cluster to the target database cluster.
[0039] The acquisition module is used to acquire the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster, when the process is not alive.
[0040] The modification module is used to modify the configuration items for data synchronization in the configuration file of the data synchronization tool based on the network addresses of the first node and the second node, so as to obtain the modified target configuration file. The configuration items include the network address of the source database and the network address of the target database.
[0041] The restart module is used to perform an operation that causes the process of the data synchronization tool to restart based on the modified target configuration file.
[0042] Optionally, the first node is the master node in the source database cluster; the second node is the master node in the target database cluster.
[0043] Optional, the acquisition module includes:
[0044] The tag acquisition unit is used to acquire the node tags of the first cluster node in the source database cluster and the node tags of the second cluster node in the target database cluster; the node tags are used to characterize the role attributes and functional attributes of the first cluster node or the second cluster node.
[0045] The node determination unit is used to determine the first node and the second node based on the node label of the first cluster node and the node label of the second cluster node.
[0046] The connection address unit is used to obtain the network address of the first node and the network address of the second node based on the first node and the second node.
[0047] Optionally, the device is also used for:
[0048] Based on a pre-configured custom file, a preset command line for connecting to the database cluster nodes is executed to establish communication connections with the first and second cluster nodes.
[0049] A pre-configured custom file contains the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster.
[0050] Optional, pre-configured custom files can be configured as follows:
[0051] Create a custom file for the data synchronization tool to manage the source database cluster and the target database cluster. The custom file includes the network address of the source database and the network address of the target database.
[0052] Configure the network address of the source database in the custom file to be the network address of all cluster nodes in the source database cluster;
[0053] Configure the network address of the target database in the custom file to be the network address of all cluster nodes in the target database cluster.
[0054] Optional, the tag acquisition unit is used for:
[0055] Based on a pre-configured custom file, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster are read in a loop.
[0056] Based on the communication connection, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster, a preset command line is executed to obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
[0057] Optionally, the device is also used for:
[0058] By pre-selecting preset placeholders and replacing the data synchronization configuration items in the data synchronization tool's configuration file with these placeholders, the module can be modified for...
[0059] Based on the network addresses of the first and second nodes, the preset placeholders in the configuration file of the data synchronization tool are modified to obtain the modified target configuration file.
[0060] Optional, a decision module, used for:
[0061] Create a monitoring script to detect whether the process of the data synchronization tool is alive, and a scheduled task that can execute the monitoring script;
[0062] A monitoring script is executed based on a scheduled task to determine whether the data synchronization tool's process is alive.
[0063] Optionally, the network address of the first node includes the IP address and port address of the first node; the network address of the second node includes the IP address and port address of the second node.
[0064] Optionally, the first cluster node includes normal nodes in the source database cluster that did not fail after the master-slave node switch; the second cluster node includes normal nodes in the target database cluster that did not fail after the master-slave node switch.
[0065] Thirdly, this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the restart method of the data synchronization tool as described above.
[0066] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the restart method for the data synchronization tool as described above.
[0067] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the restart method of the data synchronization tool as described above.
[0068] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:
[0069] The inventors discovered that in existing technologies, if a master-slave node switch occurs in a Redis cluster that is performing data synchronization, the main reason why the Redis data synchronization tool cannot be successfully restarted even if the monitoring tool detects that the Redis data synchronization tool's process has exited is that after a master-slave node switch occurs in the Redis cluster, the Redis data synchronization tool cannot identify and connect to the nodes in the Redis cluster that are responsible for reading synchronization data and those that are responsible for writing synchronization data.
[0070] By using the method provided in this application embodiment, even if a master-slave node switch occurs in the Redis cluster performing data synchronization, it can be ensured that the data synchronization tool can identify and connect to the network address of the first node and the network address of the second node from the modified target configuration file. This avoids the problem in the prior art where the data synchronization tool cannot identify the first node in the source database cluster and the second node in the target database cluster after the master-slave switch, thereby solving the problem in the prior art where the process of the data synchronization tool cannot be successfully restarted in this situation. Attached Figure Description
[0071] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0072] Figure 1aA schematic diagram illustrating the implementation process of a restart method for a data synchronization tool provided in this application embodiment;
[0073] Figure 1b A schematic diagram illustrating the implementation process of a method for determining whether the process of a data synchronization tool is alive, provided in an embodiment of this application;
[0074] Figure 1c A schematic diagram illustrating the implementation flow of a method for obtaining the network address of a first node and the network address of a second node, provided in an embodiment of this application;
[0075] Figure 2 A schematic diagram illustrating an application process of the method provided in the embodiments of this application in practice;
[0076] Figure 3 This application provides a schematic diagram of the specific structure of a restart device for a data synchronization tool.
[0077] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0078] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0079] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0080] Example 1
[0081] To address the issue in existing technologies where a master-slave node switch occurs when synchronizing data in a Redis cluster using a data synchronization tool, the process of the data synchronization tool cannot be successfully restarted, this application provides a method for restarting a data synchronization tool.
[0082] The execution subject of this method can be various types of computing devices, or it can be an application or app installed on the computing device. The computing device can be a user terminal such as a mobile phone, tablet computer, or smart wearable device, or it can be a server.
[0083] For ease of description, this application uses a server as the execution subject of the method in its embodiments to illustrate the method. Those skilled in the art will understand that this embodiment uses a server as an example to describe the method, which is merely an illustrative example and does not limit the scope of protection of the corresponding claims.
[0084] Specifically, the implementation flow of the method provided in this application embodiment is as follows: Figure 1a As shown, it includes the following steps:
[0085] Step 11: Determine if the data synchronization tool's process is alive.
[0086] Data synchronization tools are used to synchronize data to be synchronized from a source database cluster to a target database cluster.
[0087] In this embodiment of the application, the data synchronization tool may be, for example, RedisShake, redis-port, RedisSyncer, or Redis Data Migration Tool.
[0088] The source database cluster refers to the source / source end of the data to be synchronized. Data to be synchronized can be obtained from the source database cluster.
[0089] Correspondingly, the target database cluster refers to the target end of the data to be synchronized. The data to be synchronized obtained from the source database cluster can be synchronized to the target database cluster.
[0090] When determining whether the process of a data synchronization tool is alive, the following methods can be used, for example:
[0091] (1) You can use the default command line to view the process list of the system to which the data synchronization tool belongs and check whether the process of the data synchronization tool is alive.
[0092] For example, if the data synchronization tool is running on a Linux system, you can use the `ps` command in conjunction with `grep` to find the process of the data synchronization tool. Specifically, if the command returns the process information of the data synchronization tool, it means that the process is still running, that is, the data synchronization tool is alive. Conversely, if the command does not return the process information of the data synchronization tool, it means that the process of the data synchronization tool is not alive.
[0093] (2) Secondly, considering that data synchronization tools, such as RedisShake, will listen to a specific port when running, in this embodiment of the application, it is also possible to check whether the specific port that the data synchronization tool listens to is occupied in order to determine whether the data synchronization tool is alive.
[0094] (3) In addition, you can check the log files of the data synchronization tool to determine whether the process of the data synchronization tool is alive.
[0095] For example, you can check the last few lines of log information in the data synchronization tool's log file (because the last few lines reflect the latest running status of the data synchronization tool). Based on these last few lines, you can determine whether the data synchronization tool's process is still running, or whether it encountered an error and exited. If it is still running, it means the data synchronization tool's process is alive; otherwise, it means the data synchronization tool's process is not alive.
[0096] It should be noted that the above-described methods for determining whether the process of the data synchronization tool is alive are merely illustrative examples of the embodiments of this application and do not impose any limitations on the embodiments of this application.
[0097] In one alternative implementation, to automate the checking of the data synchronization tool's process and ensure the continuity and stability of the checking process, while reducing manual intervention and maintenance costs, such as... Figure 1b As shown in the embodiments of this application, the survival of the data synchronization tool process can also be determined in the following way:
[0098] Step 111: Create a monitoring script to detect whether the process of the data synchronization tool is alive, and create a scheduled task that can execute the monitoring script;
[0099] Step 112: Execute a monitoring script based on a scheduled task to determine whether the data synchronization tool's process is alive.
[0100] For example, taking a Linux system as the system on which the data synchronization tool is running, you can use the Linux crontab tool to set up a scheduled task that executes a monitoring script (for example, this task can be set to execute the monitoring script every minute). Then, the created monitoring script is executed based on this scheduled task to achieve periodic monitoring of the data synchronization tool's process. Specifically, you can open an editor by executing the `crontab -e` command and edit the cron task list. Then, add a new scheduled task configuration line in the opened editor. This configuration can specify the execution frequency of the created monitoring script (for example, once per minute) and the specific path of the created monitoring script. Finally, save and exit the editor to complete the cron task settings, so that the Linux system can automatically execute the created script at the specified time intervals, thereby achieving periodic monitoring of the data synchronization tool's process.
[0101] Step 12: When the process is not alive, obtain the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster.
[0102] The network address of the first node includes the IP address and port address of the first node; the network address of the second node includes the IP address and port address of the second node.
[0103] In this embodiment, considering that when a data synchronization tool is used to synchronize data in a Redis cluster mode, it is typically necessary to configure a first node in the source database cluster so that data to be synchronized can be read from the source database cluster based on the first node; and a second node is also required in the target database cluster so that the data to be synchronized obtained from the source database cluster can be written to the target database cluster based on the second node. However, due to certain reasons (such as failures, network problems, etc.), the first and second nodes may become unable to continue providing services. In this case, the source and target database clusters will select new first and second nodes from other cluster nodes within the cluster (i.e., a master-slave node switch occurs in the Redis cluster performing data synchronization). Thus, because the data synchronization tool cannot recognize the new first and second nodes, the process of the data synchronization tool may fail and exit.
[0104] To avoid this technical problem, in this embodiment of the application, when the process is not alive, the network address of the first node in the source database cluster that is currently able to read the data to be synchronized, and the network address of the second node in the target database cluster that is currently able to write the data to be synchronized to the target database cluster can be obtained.
[0105] Among them, the first node in the source database cluster that can currently read the data to be synchronized is the new first node in the source database cluster after a master-slave node switch occurs in the Redis cluster.
[0106] Correspondingly, the second node in the target database cluster that can currently write the data to be synchronized into the target database cluster, that is, the new second node in the target database cluster after the master-slave node switch occurs in the Redis cluster.
[0107] In one alternative implementation, considering that when a data synchronization tool is used to synchronize data in a Redis cluster mode, typically one master node is configured in both the source and target database clusters for reading and writing data. Therefore, in this embodiment, the first node can be the master node in the source database cluster; and the second node can be the master node in the target database cluster.
[0108] In one alternative implementation, such as Figure 1c As shown, step 12 can be achieved in the following way:
[0109] Step 121: Obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
[0110] Among them, the node label is used to characterize the role and functional attributes of the first cluster node or the second cluster node.
[0111] For example, in a Redis cluster, there are master nodes and slave nodes. The master node can be labeled as Master Node, which indicates that the node is responsible for handling write operations and some read operations. The slave node can be labeled as Slave Node, which indicates that the node is responsible for replicating the data of the master node and is usually only used to handle read operations to reduce the load on the master node.
[0112] The first cluster node refers to a normal node in the source database cluster that has not failed or crashed after a master-slave switch occurs in the Redis cluster that is synchronizing data. These normal nodes can respond to requests normally and participate in the data processing tasks of the source database cluster.
[0113] Correspondingly, the second cluster node in the target database cluster refers to a normal node in the target database cluster that has not failed or crashed after a master-slave node switch occurs in the Redis cluster that is performing data synchronization. These normal nodes can respond to requests normally and participate in the data processing tasks of the target database cluster.
[0114] In this embodiment of the application, the node labels of the first cluster node in the source database cluster and the node labels of the second cluster node in the target database cluster can be viewed using the CLUSTER NODES command of the database cluster.
[0115] The `CLUSTER NODES` command is used in database cluster mode to retrieve detailed information about all nodes in the database cluster, including node ID, IP address, port number, role (master or slave), and node connection status (whether online). After executing this command, the database cluster returns a list containing the current status and configuration information of each node in the cluster.
[0116] Step 122: Determine the first node and the second node based on the node labels of the first cluster node and the second cluster node.
[0117] In this embodiment, the first node in the source database cluster that can currently read the data to be synchronized can be determined from the first cluster nodes of the source database cluster based on the node label of the first cluster node; and the second node in the target database cluster that can currently write the data to be synchronized can be determined from the second cluster nodes of the target database cluster based on the node label of the second cluster node.
[0118] In an optional implementation, if the first node is the master node in the source database cluster and the second node is the master node in the target database cluster, then in step 122, the current master node in the source database cluster is determined from the first cluster node of the source database cluster based on the node label of the first cluster node of the source database cluster; and the current master node in the target database cluster is determined from the second cluster node of the target database cluster based on the node label of the second cluster node of the target database cluster.
[0119] Step 123: Obtain the network address of the first node and the network address of the second node based on the first node and the second node.
[0120] In this embodiment of the application, after determining the first node and the second node, the network address of the first node and the network address of the second node can be obtained based on the first node and the second node.
[0121] In an optional implementation, before performing step 121 above, a communication connection can be established with the first cluster node and the second cluster node in the following manner, so that the node tags of the first cluster node in the source database cluster and the node tags of the second cluster node in the target database cluster can be obtained through the communication connection, as follows:
[0122] Based on a pre-configured custom file, a preset command line for connecting to the database cluster nodes is executed to establish communication connections with the first and second cluster nodes, so as to obtain the node labels of the first cluster node in the source database cluster and the node labels of the second cluster node in the target database cluster through the communication connection.
[0123] The pre-configured custom file contains the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster.
[0124] In this embodiment, the preset command line for connecting to the database cluster node may include, for example, "redis-cli -h$ip -p$port". Here, "redis-cli" in the preset command line is the connection tool, which allows connection to the specified database cluster node; secondly, "-h$ip -p$port" in the preset command line is used to specify the IP address and port address of the database cluster node to be connected using the -h and -p parameters.
[0125] Optionally, a custom file can be created before executing a preset command line for connecting to the database cluster nodes, based on a pre-configured custom file, and before establishing communication connections with the first and second cluster nodes.
[0126] This custom file can be similar to the configuration file of the data synchronization tool. The only difference is that the configuration items used for data synchronization in this custom file (e.g., source.address of the source database cluster) are configured to the network addresses (including IP addresses and port addresses) of all cluster nodes of the source database cluster; and the configuration items used for data synchronization in this custom file (e.g., target.address of the target database cluster) are configured to the network addresses (including IP addresses and port addresses) of all cluster nodes of the target database cluster.
[0127] Optionally, in one implementation, a pre-configured custom file can be created in the following manner:
[0128] (1) Create a custom file for the data synchronization tool to manage the source database cluster and the target database cluster. The custom file includes the network address of the source database and the network address of the target database.
[0129] (2) Configure the network address of the source database in the custom file to be the network address of all cluster nodes in the source database cluster.
[0130] (3) Configure the network address of the target database in the custom file to be the network address of all cluster nodes in the target database cluster.
[0131] For example, in practical applications, a custom file named `monitor-redis-shake.conf` can be created for data synchronization tools to manage the source and target database clusters. Then, the network address of the source database (e.g., `source.address`) in the `monitor-redis-shake.conf` file can be configured to the network addresses (including IP addresses and port addresses) of all cluster nodes in the source database cluster. Similarly, the network address of the target database (e.g., `target.address`) in the `monitor-redis-shake.conf` file can be configured to the network addresses (including IP addresses and port addresses) of all cluster nodes in the target database cluster.
[0132] It should be noted that, in this embodiment of the application, the configuration items used for data synchronization in the custom file are configured as the network addresses of all cluster nodes of the source database cluster and the network addresses of all cluster nodes of the target database cluster, respectively. This is mainly to ensure that the data synchronization tool can subsequently connect to each cluster node in the source database cluster and the target database cluster.
[0133] In one alternative implementation, after establishing a communication connection, the node labels of the first cluster nodes in the source database cluster and the node labels of the second cluster nodes in the target database cluster can be obtained as follows:
[0134] First, based on a pre-configured custom file, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster are read in a loop.
[0135] Secondly, based on the communication connection, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster, a preset command line is executed to obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
[0136] Specifically, after cyclically reading the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster, a communication connection can be established between the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster. If it is confirmed that the connection with the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster is successful, a preset command line for obtaining database cluster node information, such as the cluster nodes command, can be executed to obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
[0137] The `cluster nodes` command can be used to obtain node information (including node tags) of the first cluster node in the source database cluster and the node information (including node tags) of the second cluster node in the target database cluster. Optionally, in addition to obtaining node tags, the `cluster nodes` command can also obtain detailed information such as node identification information, IP address, port address, role information (such as master node or slave node), and slot allocation of the first and second cluster nodes.
[0138] Step 13: Modify the data synchronization configuration items in the data synchronization tool's configuration file according to the network addresses of the first node and the second node to obtain the modified target configuration file.
[0139] The configuration items include: the network address of the source database and the network address of the target database.
[0140] In this embodiment, the network address of the source database for data synchronization in the configuration file of the data synchronization tool can be modified according to the network address of the first node; and the network address of the target database for data synchronization in the configuration file of the data synchronization tool can be modified according to the network address of the second node, thereby obtaining the modified target configuration file.
[0141] For example, assuming the configuration information for data synchronization in the configuration file of the data synchronization tool is source.address = 10.26.31.172:6379; target.address = 10.26.29.235:6389; and the network address of the first node obtained in step 12 is 192.168.1.10:6379 (where the IP address of the first node is 192.168.1.10 and the port number is 6379), and the network address of the second node is 192.168.1.11:6379 (where the IP address of the second node is 192.168.1.11 and the port number is 6379), then when executing step 13 above, source.address can be modified to 192.168.1.10:6379; and target.address can be modified to 192.168.1.11:6379.
[0142] Step 14: Perform an operation to restart the data synchronization tool's process based on the modified target configuration file.
[0143] In this embodiment of the application, after obtaining the target configuration file, an operation can be performed to restart the process of the data synchronization tool based on the modified target configuration file.
[0144] In one alternative implementation, to ensure continuous data replication and synchronization, whenever the network address of the first node in the source database cluster or the network address of the second node in the target database cluster changes, it is necessary to first open the configuration file of the data synchronization tool, then locate the configuration items used for data synchronization in the configuration file, and finally modify the values of these configuration items one by one to the changed network addresses of the first and second nodes. This approach is prone to operational errors and suffers from high maintenance costs and a lack of centralized management.
[0145] To avoid the aforementioned problems, in this embodiment, preset placeholders can be used to represent data synchronization configuration items in the configuration file of the data synchronization tool. These placeholders are then referenced where the network addresses of the source and target databases need to be used. The specific implementation steps are as follows:
[0146] (1) Replace the configuration items for data synchronization in the configuration file of the data synchronization tool with the preset placeholders in advance.
[0147] In this context, a placeholder refers to a marker or symbol. In the configuration file of a data synchronization tool, a specific string or marker can be used as a placeholder to represent a dynamic value in a configuration item. For example, the network address of the source database ({source_address}) and the network address of the target database ({target_address}) in the configuration file of a data synchronization tool are placeholders. In actual use, they will be replaced by specific network addresses (such as 10.26.31.172:6379, 10.26.29.235:6389, etc.).
[0148] (2) Modify the preset placeholders in the configuration file of the data synchronization tool according to the network address of the first node and the network address of the second node to obtain the modified target configuration file.
[0149] The method provided in this application embodiment allows for the modification of specific values corresponding to preset placeholders when the network addresses of the first node in the source database cluster and the second node in the target database cluster change. This eliminates the need to modify every reference in the configuration file of the data synchronization tool, thereby reducing operational errors, lowering maintenance costs, and enabling centralized management.
[0150] By using the method provided in this application embodiment, even if a master-slave switch occurs in the master node of the Redis cluster, it can be ensured that the data synchronization tool can identify and connect to the network address of the first node and the network address of the second node from the modified target configuration file. This avoids the problem in the prior art that the data synchronization tool cannot identify the first node in the source database cluster and the second node in the target database cluster after the master-slave switch, thereby solving the problem that the process of the data synchronization tool cannot be successfully restarted in the prior art under this situation.
[0151] Example 2
[0152] The following example, using RedisShake as a data synchronization tool, illustrates how the method provided in this application is applied in practice.
[0153] Before describing how the methods provided in the embodiments of this application are applied in practice, the preparation operations involved in the embodiments of this application will be explained first:
[0154] A. Rename the original RedisShake configuration file redis-shake.conf to have the configuration items source.address (source Redis IP:Port address value) and target.address (target Redis IP:Port address value) represented by special placeholders as {source_address} and {target_address} respectively, and then rename it to redis-shake-template.conf.
[0155] Optionally, other configurations can be added according to actual needs in practical applications. Here, we only use the replacement of configuration items in the redis-shake.conf configuration file as an example.
[0156] B. Create a new configuration file monitor-redis-shake.conf. For the configuration file monitor-redis-shake.conf, you can use the same method as the preparation condition (1) above. Use special placeholders to represent the configuration items of the configuration file as {source_address} and {target_address} respectively. In addition, configure the value of source.address to the IP:Port address of all nodes in the source Redis cluster, and configure the value of target.address to the IP:Port address of all nodes in the target Redis cluster.
[0157] C. Create a monitoring script monitor-redis-shake.sh to work with Linux scheduled tasks or monitoring functions such as Zabbix, Nagios, and Prometheus to periodically execute the monitor-redis-shake.sh script to check whether the RedisShake process is alive.
[0158] After completing the above preparations, please refer to Figure 2 This is a schematic diagram illustrating an application process of the method provided in this application embodiment. The process specifically includes the following steps:
[0159] (1) Call the Linux ps command to check if the RedisShake process is alive; if it is not alive, proceed to step (2); or, if it is alive, end the operation.
[0160] (2) Read the values of source.address and target.address from the monitor-redis-shake.conf configuration file;
[0161] In the monitor-redis-shake.conf configuration file, the value of source.address is configured to the IP:Port address of all nodes in the source Redis cluster; and the value of target.address is configured to the IP:Port address of all nodes in the target Redis cluster.
[0162] In this embodiment of the application, reading the values of source.address and target.address can be understood as reading the IP:Port addresses of all nodes in the source Redis cluster and reading the IP:Port addresses of all nodes in the target Redis cluster.
[0163] (3) Based on the IP:Port addresses of all nodes in the source Redis cluster obtained in step (2) above, and using the preset command line `redis-cli -h $ip -p $port cluster nodes`, loop through and read the IP:Port value of each `source.address` (i.e., ...). Figure 2 In the code, i = 0, L = the number of source address values, and then check whether ++i is less than the relevant content of L, until the cluster node information of the source Redis cluster is obtained;
[0164] (4) Select the IP:Port of the master node from the obtained cluster node information of the source Redis cluster, and set source_address = master node IP:Port list.
[0165] (5) Similarly, for the IP:Port addresses of all nodes in the target Redis cluster obtained in step (2) above, the target.addressIP:Port value can be read and connected to in a loop based on the preset command line redis-cli -h $ip -p $port cluster nodes (i.e., ...). Figure 2 In the process, set i=0, L=the number of target address values, and then check whether ++i is less than the relevant content of L, until the cluster node information of the target Redis cluster is obtained;
[0166] (6) Select the IP:Port of the master node from the cluster node information of the target Redis cluster obtained, and set target_address = list of master node IP:Port.
[0167] (7) Based on the source_address and target_address obtained in steps (4) and (6) above, replace the values of the placeholders {source_address} and {target_address} in the redis-shake-template.conf file with source_address and target_address, and generate the redis-shake.conf configuration file.
[0168] (8) Use the redis-shake.conf configuration file to start RedisShake.
[0169] By using the method provided in this application embodiment, even if a master-slave switch occurs in the master node of the Redis cluster, it can be ensured that the data synchronization tool can identify and connect to the network address of the first node and the network address of the second node from the modified target configuration file. This avoids the problem in the prior art that the data synchronization tool cannot identify the first node in the source database cluster and the second node in the target database cluster after the master-slave switch, thereby solving the problem that the process of the data synchronization tool cannot be successfully restarted in the prior art under this situation.
[0170] Example 3
[0171] To address the issue in existing technologies where a master-slave switch occurs during data synchronization in a Redis cluster, preventing the data synchronization tool from restarting, this application provides a restart device for the data synchronization tool. A schematic diagram of the device's specific structure is shown below. Figure 3 As shown, it includes a judgment module 31, an acquisition module 32, a modification module 33, and a restart module 34. The functions of each module are as follows:
[0172] The judgment module 31 is used to determine whether the process of the data synchronization tool is alive; the data synchronization tool is used to synchronize the data to be synchronized from the source database cluster to the target database cluster.
[0173] The acquisition module 32 is used to acquire the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster, when the process is not alive.
[0174] Optionally, the network address of the first node includes the IP address and port address of the first node; the network address of the second node includes the IP address and port address of the second node.
[0175] Modification module 33 is used to modify the configuration items for data synchronization in the configuration file of the data synchronization tool according to the network addresses of the first node and the second node, so as to obtain the modified target configuration file; the configuration items include: the network address of the source database and the network address of the target database;
[0176] Restart module 34 is used to perform an operation that causes the data synchronization tool's process to restart based on the modified target configuration file.
[0177] Optionally, the first node is the master node in the source database cluster; the second node is the master node in the target database cluster.
[0178] Optionally, module 32 includes:
[0179] The tag acquisition unit is used to acquire the node tags of the first cluster node in the source database cluster and the node tags of the second cluster node in the target database cluster; the node tags are used to characterize the role attributes and functional attributes of the first cluster node or the second cluster node.
[0180] Optionally, the first cluster node includes normal nodes in the source database cluster that did not fail after the master-slave node switch; the second cluster node includes normal nodes in the target database cluster that did not fail after the master-slave node switch.
[0181] The node determination unit is used to determine the first node and the second node based on the node label of the first cluster node and the node label of the second cluster node.
[0182] The connection address unit is used to obtain the network address of the first node and the network address of the second node based on the first node and the second node.
[0183] Optionally, the device is also used for:
[0184] Based on a pre-configured custom file, a preset command line for connecting to the database cluster nodes is executed to establish communication connections with the first and second cluster nodes.
[0185] A pre-configured custom file contains the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster.
[0186] Optional, pre-configured custom files can be configured in the following way:
[0187] Create a custom file for the data synchronization tool to manage the source database cluster and the target database cluster. The custom file includes the network address of the source database and the network address of the target database.
[0188] Configure the network address of the source database in the custom file to be the network address of all cluster nodes in the source database cluster;
[0189] Configure the network address of the target database in the custom file to be the network address of all cluster nodes in the target database cluster.
[0190] Optional, the tag acquisition unit is used for:
[0191] Based on a pre-configured custom file, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster are read in a loop.
[0192] Based on the communication connection, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster, a preset command line is executed to obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
[0193] Optionally, the device is also used for:
[0194] The configuration items for data synchronization in the configuration file of the data synchronization tool are replaced with preset placeholders in advance;
[0195] Based on the network addresses of the first and second nodes, the preset placeholders in the configuration file of the data synchronization tool are modified to obtain the modified target configuration file.
[0196] Optionally, the judgment module 31 is used for:
[0197] Create a monitoring script to detect whether the process of the data synchronization tool is alive, and a scheduled task that can execute the monitoring script;
[0198] A monitoring script is executed based on a scheduled task to determine whether the data synchronization tool's process is alive.
[0199] Using the apparatus provided in this application embodiment, even if a master-slave switch occurs in the Redis cluster, the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized, can be obtained first. Then, the configuration items for data synchronization in the configuration file of the data synchronization tool are modified according to the network addresses of the first and second nodes to obtain the modified target configuration file. Finally, the process of the data synchronization tool is restarted based on the modified target configuration file, thereby solving the problem that the prior art cannot successfully restart the process of the data synchronization tool in this situation.
[0200] Example 4
[0201] Figure 4 To illustrate the hardware structure of an electronic device according to various embodiments of this application, the electronic device 400 includes, but is not limited to, components such as: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, a processor 410, and a power supply 411. Those skilled in the art will understand that... Figure 4 The electronic device structures shown are not intended to limit the electronic device. An electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. In the embodiments of this application, the electronic device includes, but is not limited to, mobile phones, tablets, laptops, PDAs, in-vehicle terminals, wearable devices, and pedometers.
[0202] The processor 410 is used to determine whether the process of the data synchronization tool is alive. The data synchronization tool is used to synchronize the data to be synchronized from the source database cluster to the target database cluster. When the process is not alive, it obtains the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster. Based on the network addresses of the first and second nodes, it modifies the configuration items for data synchronization in the configuration file of the data synchronization tool to obtain the modified target configuration file. The configuration items include the network address of the source database and the network address of the target database. It then executes an operation to restart the process of the data synchronization tool based on the modified target configuration file.
[0203] The memory 409 is used to store a computer program that can run on the processor 410, which, when executed by the processor 410, implements the aforementioned functions implemented by the processor 410.
[0204] It should be understood that, in this embodiment, the radio frequency unit 401 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink data from the base station and processes it with the processor 410; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 401 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. Furthermore, the radio frequency unit 401 can also communicate with networks and other devices through a wireless communication system.
[0205] The electronic device provides users with wireless broadband internet access through network module 402, such as helping users send and receive emails, browse web pages, and access streaming media.
[0206] The audio output unit 403 can convert audio data received by the radio frequency unit 401 or the network module 402 or stored in the memory 409 into audio signals and output them as sound. Furthermore, the audio output unit 403 can also provide audio output related to specific functions performed by the electronic device 400 (e.g., call signal reception sound, message reception sound, etc.). The audio output unit 403 includes a speaker, a buzzer, and a receiver, etc.
[0207] Input unit 404 is used to receive audio or video signals. Input unit 404 may include a graphics processing unit (GPU) 4041 and a microphone 4042. The GPU 4041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on display unit 406. The image frames processed by GPU 4041 can be stored in memory 409 (or other storage medium) or transmitted via radio frequency unit 401 or network module 402. Microphone 4042 can receive sound and process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via radio frequency unit 401 in telephone call mode.
[0208] The electronic device 400 also includes at least one sensor 405, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 4061 according to the ambient light level, and the proximity sensor can turn off the display panel 4061 and / or backlight when the electronic device 400 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used to identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. The sensor 405 may also include a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described in detail here.
[0209] The display unit 406 is used to display information input by the user or information provided to the user. The display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
[0210] User input unit 407 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of electronic devices. Specifically, user input unit 407 includes a touch panel 4071 and other input devices 4072. Touch panel 4071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 4071). Touch panel 4071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 410, which receives and executes commands from the processor 410. In addition, touch panel 4071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. Besides touch panel 4071, user input unit 407 may also include other input devices 4072. Specifically, other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.
[0211] Furthermore, the touch panel 4071 can cover the display panel 4061. When the touch panel 4071 detects a touch operation on or near it, it transmits the information to the processor 410 to determine the type of touch event. Subsequently, the processor 410 provides corresponding visual output on the display panel 4061 based on the type of touch event. Although in Figure 4 In this embodiment, the touch panel 4071 and the display panel 4061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 4071 and the display panel 4061 can be integrated to realize the input and output functions of the electronic device. The specific implementation is not limited here.
[0212] Interface unit 408 serves as an interface for connecting external devices to electronic device 400. For example, external devices may include a wired or wireless headphone port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 408 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within electronic device 400, or it can be used to transmit data between electronic device 400 and external devices.
[0213] The memory 409 can be used to store software programs and various data. The memory 409 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 409 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0214] The processor 410 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 409, and by calling data stored in the memory 409, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 410 may include one or more processing units; preferably, the processor 410 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 410.
[0215] The electronic device 400 may also include a power supply 411 (such as a battery) for supplying power to various components. Preferably, the power supply 411 can be logically connected to the processor 410 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.
[0216] In addition, the electronic device 400 includes some functional modules not shown, which will not be described in detail here.
[0217] Preferably, this application embodiment also provides an electronic device, including a processor 410, a memory 409, and a computer program stored in the memory 409 and executable on the processor 410. When the computer program is executed by the processor 410, it implements the various processes of the above-described data synchronization tool restart method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0218] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described data synchronization tool restart method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0219] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0220] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.
[0221] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0222] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0223] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0224] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0225] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0226] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0227] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.
Claims
1. A method for restarting a data synchronization tool, characterized in that, include: Determine whether the process of the data synchronization tool is alive; the data synchronization tool is used to synchronize the data to be synchronized from the source database cluster to the target database cluster; When the process is not alive, obtain the network address of the first node in the source database cluster that can currently read the data to be synchronized, and the network address of the second node in the target database cluster that can currently write the data to be synchronized to the target database cluster. Based on the network addresses of the first node and the second node, the configuration items for data synchronization in the configuration file of the data synchronization tool are modified to obtain the modified target configuration file; the configuration items include: the network address of the source database and the network address of the target database; The process of the data synchronization tool is restarted based on the modified target configuration file. The method further includes: The configuration items for data synchronization in the configuration file of the data synchronization tool are replaced with preset placeholders in advance; then, The step of modifying the configuration items for data synchronization in the configuration file of the data synchronization tool based on the network addresses of the first node and the second node to obtain the modified target configuration file includes: Based on the network addresses of the first node and the second node, the preset placeholders in the configuration file of the data synchronization tool are modified to obtain the modified target configuration file. The preset placeholders refer to tags or symbols.
2. The method as described in claim 1, characterized in that, include: The first node is the master node in the source database cluster; The second node is the master node in the target database cluster.
3. The method as described in claim 1 or 2, characterized in that, The step of obtaining the network address of the first node in the source database cluster that is currently capable of reading the data to be synchronized, and the network address of the second node in the target database cluster that is currently capable of writing the data to be synchronized to the target database cluster, includes: Obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster; the node label is used to characterize the role attributes and functional attributes of the first cluster node or the second cluster node; The first node and the second node are determined based on the node label of the first cluster node and the node label of the second cluster node; Based on the first node and the second node, obtain the network address of the first node and the network address of the second node.
4. The method as described in claim 3, characterized in that, Before obtaining the node labels of the first cluster nodes in the source database cluster and the node labels of the second cluster nodes in the target database cluster, the method further includes: Based on a pre-configured custom file, a preset command line for connecting to the database cluster nodes is executed to establish a communication connection with the first cluster node and the second cluster node; The pre-configured custom file contains the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster.
5. The method as described in claim 4, characterized in that, The pre-configured custom file is configured in the following way: A custom file is created for the data synchronization tool to manage the source database cluster and the target database cluster. The custom file includes the network address of the source database and the network address of the target database. Configure the network address of the source database in the custom file to be the network address of all cluster nodes in the source database cluster; Configure the network address of the target database in the custom file to be the network address of all cluster nodes in the target database cluster.
6. The method as described in claim 5, characterized in that, The step of obtaining the node labels of the first cluster nodes in the source database cluster and the node labels of the second cluster nodes in the target database cluster includes: Based on the pre-configured custom file, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster are read in a loop. Based on the communication connection, the network addresses of all cluster nodes in the source database cluster and the network addresses of all cluster nodes in the target database cluster, a preset command line for obtaining database cluster node information is executed to obtain the node label of the first cluster node in the source database cluster and the node label of the second cluster node in the target database cluster.
7. The method as described in claim 1, characterized in that, The determination of whether the data synchronization tool's process is alive includes: Create a monitoring script to detect whether the process of the data synchronization tool is alive, and a scheduled task that can execute the monitoring script; The monitoring script is executed based on the scheduled task to determine whether the process of the data synchronization tool is alive.
8. The method according to any one of claims 1-7, characterized in that, include: The network address of the first node includes the IP address and port address of the first node; The network address of the second node includes the IP address and port address of the second node.
9. The method as described in claim 3, characterized in that, include: The first cluster node includes normal nodes in the source database cluster that did not fail after the master-slave node switch; The second cluster node includes a normal node in the target database cluster that did not fail after the master-slave node switch.
Citation Information
Patent Citations
Fault processing method and device, server and storage medium
CN112199356A
Data table synchronization method and device, computer equipment and storage medium
CN115481198A