Data synchronization method, apparatus, device, and medium

Abstract

The application discloses a data synchronization method and device, equipment and medium, and relates to the technical field of distributed file storage clusters, comprising: establishing a connection relationship between a local cluster and a remote cluster to be synchronized based on the secret key, the Internet protocol address and the link name corresponding to the remote cluster to be synchronized; recording the local directory of the local cluster and the remote directory of the remote cluster to be synchronized through the connection relationship to obtain a replication relationship between the local cluster and the remote cluster to be synchronized; performing directory snapshot level data synchronization on the remote cluster to be synchronized based on the replication relationship, the local directory and the data resources in the local directory to obtain a synchronized remote cluster corresponding to the local cluster; and judging whether the preset disaster recovery data synchronization condition is met or not, and if yes, obtaining the data resources corresponding to the remote directory in the synchronized remote cluster through the replication relationship to perform directory snapshot level data synchronization. The data synchronization process does not affect other businesses.

Description

Technical Field

[0001] This invention relates to the field of distributed file storage cluster technology, and particularly to data synchronization methods, apparatus, devices, and media. Background Technology

[0002] Distributed storage clusters are increasingly used today. With the advancement of digitalization, data is gradually becoming the core of operations for enterprises and institutions, such as banks and financial companies. Users are also placing higher and higher demands on the stability of storage systems that carry data. Disaster recovery systems refer to establishing two or more identical IT (Internet Technology) systems in geographically dispersed locations. These systems can monitor each other's health status and switch functions. When one system stops working due to an accident, such as a fire or earthquake, the entire application system can switch to another location, allowing the system to continue functioning normally. Disaster recovery technology is a component of high availability technology. Disaster recovery systems place greater emphasis on handling the impact of external environments on the system, especially the impact of catastrophic events on the entire IT node, providing node-level system recovery capabilities. To ensure the continuity, recoverability, and high availability of data storage, remote disaster recovery solutions have emerged. Remote replication technology is one of the key technologies in remote disaster recovery solutions. Its core idea is to synchronously replicate the corresponding data multiple times to multiple locations, thereby minimizing the possibility of data loss caused by natural disasters or human-caused damage. However, data synchronization requires a lot of computing resources, which will reduce the operating speed of other services, or even stop other services.

[0003] In summary, how to ensure that data synchronization does not disrupt other business operations is a problem that needs to be solved in this field. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a data synchronization method, apparatus, device, and medium that does not affect other services during data synchronization. The specific solution is as follows:

[0005] In a first aspect, this application discloses a data synchronization method applied to a local cluster, comprising:

[0006] Based on the key, Internet protocol address, and link name corresponding to the remote cluster to be synchronized, establish a connection between the local cluster and the remote cluster to be synchronized;

[0007] The local directory of the local cluster and the remote directory of the remote cluster to be synchronized are recorded through the connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized.

[0008] Based on the replication relationship, the local directory, and the data resources in the local directory, the remote cluster to be synchronized is synchronized at the directory snapshot level to obtain a synchronized remote cluster corresponding to the local cluster.

[0009] Determine whether the preset disaster recovery data synchronization conditions are met. If they are met, use the replication relationship to obtain the data resources corresponding to the remote directory in the synchronized remote cluster, and perform data synchronization at the directory snapshot level.

[0010] Optionally, establishing the connection between the local cluster and the remote cluster to be synchronized based on the key, Internet Protocol address, and link name corresponding to the remote cluster to be synchronized includes:

[0011] Obtain the key corresponding to the remote cluster to be synchronized and verify the key. If the verification result meets the first preset condition, obtain the Internet protocol address bound to the corresponding monitoring process and the corresponding verification value.

[0012] An interface corresponding to the link name is added using the Internet Protocol address and the checksum, and the connection between the local cluster and the remote cluster to be synchronized is established using the interface.

[0013] Optionally, the step of adding an interface corresponding to the link name using the Internet Protocol address and the checksum, and establishing a connection between the local cluster and the remote cluster to be synchronized using the interface, includes:

[0014] Add an interface corresponding to the link name using the Internet Protocol address and the verification value;

[0015] The interface and remote connection are invoked in the remote cluster to be synchronized via the Secure Shell protocol, so as to add the interface in the remote cluster to be synchronized and establish the connection relationship between the local cluster and the remote cluster to be synchronized.

[0016] Optionally, the step of performing directory snapshot-level data synchronization on the remote cluster to be synchronized based on the replication relationship, the local directory, and the data resources in the local directory to obtain a synchronized remote cluster corresponding to the local cluster includes:

[0017] Based on the replication relationship, the local directory deletes the remote directory in the remote cluster to be synchronized that meets the preset deletion conditions to obtain the first remote directory;

[0018] The differences between the local directory and the first remote directory are compared, and a second remote directory is created using the differences to obtain a third remote directory containing the first and second remote directories. Then, based on the replication relationship, the local directory, and the data resources in the local directory, the third remote directory in the remote cluster to be synchronized is synchronized at the directory snapshot level to obtain a synchronized remote cluster corresponding to the local cluster.

[0019] Optionally, the step of deleting the remote directories in the remote cluster to be synchronized that meet preset deletion conditions based on the replication relationship and the local directory to obtain the first remote directory includes:

[0020] Based on the replication relationship and the local directory, it is determined whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions. If it does, the current remote directory is deleted and the current remote directory is updated to obtain the current first remote directory.

[0021] Optionally, the step of determining whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions based on the replication relationship and the local directory, and deleting the current remote directory if the conditions are met, includes:

[0022] Based on the replication relationship and the local directory, it is determined whether the current remote directory in the remote cluster to be synchronized exists in the local directory. If it does not exist, the current remote directory is deleted.

[0023] Alternatively, based on the copy relationship and the local directory, determine whether the file name in the current remote directory is consistent with the file name in the corresponding local directory. If they are consistent, determine whether the file type in the current remote directory is consistent with the file type in the corresponding local directory. If they are inconsistent, delete the current remote directory.

[0024] Optionally, the step of performing directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized, based on the replication relationship, the local directory, and the data resources in the local directory, to obtain a synchronized remote cluster corresponding to the local cluster, includes:

[0025] Modify the metadata of the current third remote directory and determine whether the current third remote directory meets the preset data synchronization conditions. If it does, determine the preset data synchronization type corresponding to the current third remote directory.

[0026] Based on the replication relationship, the local directory, the data resources in the local directory, and the preset data synchronization type, the current third remote directory is synchronized at the directory snapshot level to obtain the current synchronized remote cluster corresponding to the local cluster, and the metadata of the current synchronized remote cluster is modified.

[0027] Secondly, this application discloses a data synchronization device applied to a local cluster, comprising:

[0028] The connection establishment module is used to establish a connection between the local cluster and the remote cluster to be synchronized based on the key, Internet protocol address and link name corresponding to the remote cluster to be synchronized.

[0029] The replication relationship acquisition module is used to record the local directory of the local cluster and the remote directory of the remote cluster to be synchronized through the connection relationship, so as to obtain the replication relationship between the local cluster and the remote cluster to be synchronized.

[0030] The first synchronization module is used to perform directory snapshot-level data synchronization on the remote cluster to be synchronized based on the replication relationship, the local directory, and the data resources in the local directory, so as to obtain a synchronized remote cluster corresponding to the local cluster.

[0031] The second synchronization module is used to determine whether the preset disaster recovery data synchronization conditions are met. If they are met, the module uses the replication relationship to obtain the data resources corresponding to the remote directory in the synchronized remote cluster and performs data synchronization at the directory snapshot level.

[0032] Thirdly, this application discloses an electronic device, including:

[0033] Memory, used to store computer programs;

[0034] A processor for executing the computer program to implement the steps of the aforementioned disclosed data synchronization method.

[0035] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed data synchronization method.

[0036] As can be seen, based on the key, Internet Protocol address, and link name corresponding to the remote cluster to be synchronized, a connection relationship is established between the local cluster and the remote cluster to be synchronized. The local directory of the local cluster and the remote directory of the remote cluster to be synchronized are recorded through this connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized. Based on the replication relationship, the local directory, and the data resources in the local directory, directory snapshot-level data synchronization is performed on the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster. It is determined whether the preset disaster recovery data synchronization conditions are met. If so, the data resources corresponding to the remote directory in the synchronized remote cluster are obtained using the replication relationship, and directory snapshot-level data synchronization is performed. Therefore, this application establishes a connection relationship between the local cluster and the remote cluster to be synchronized, which allows for the replication relationship between the local cluster and the remote cluster to be synchronized. This enables subsequent directory snapshot-level data synchronization because directory snapshots are point-in-time backup images, reducing computing resources and thus not affecting other business operations. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a flowchart of a data synchronization method disclosed in this application;

[0039] Figure 2 A schematic diagram illustrating the process of establishing a specific connection relationship disclosed in this application;

[0040] Figure 3 This is a specific data synchronization diagram disclosed in this application;

[0041] Figure 4 This application discloses a specific data synchronization method;

[0042] Figure 5 This is a schematic diagram of a specific Pair operating state disclosed in this application;

[0043] Figure 6 This is a schematic diagram of the structure of a data synchronization device disclosed in this application;

[0044] Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0046] Distributed storage clusters are increasingly used today. With the advancement of digitalization, data is gradually becoming the core of operations for enterprises and institutions, such as banks and financial companies. Users are placing increasingly higher demands on the stability of storage systems that hold this data. Disaster recovery systems refer to establishing two or more identical IT systems in geographically dispersed locations. These systems can monitor each other's health status and switch functions. When one system fails due to an unexpected event, such as a fire or earthquake, the entire application system can switch to another location, allowing the system to continue functioning normally. Disaster recovery technology is a component of high availability technology. Disaster recovery systems place greater emphasis on handling the impact of external environments on the system, especially the impact of catastrophic events on the entire IT node, providing node-level system recovery capabilities. To ensure the continuity, recoverability, and high availability of data storage, remote disaster recovery solutions have emerged. Remote replication technology is one of the key technologies in remote disaster recovery solutions. Its core idea is to synchronously replicate the corresponding data multiple times to multiple locations, thereby minimizing the possibility of data loss due to natural disasters or human-caused damage. However, data synchronization requires significant computing resources, which can slow down the operation of other services or even cause them to stop.

[0047] Therefore, this application provides a data synchronization solution that can ensure that other business operations are not affected during the data synchronization process.

[0048] See Figure 1 As shown in the figure, this application discloses a data synchronization method applied to a local cluster, including:

[0049] Step S11: Based on the key, Internet Protocol address and link name corresponding to the remote cluster to be synchronized, establish the connection relationship between the local cluster and the remote cluster to be synchronized.

[0050] In this embodiment, establishing the connection between the local cluster and the remote cluster to be synchronized based on the key, Internet Protocol address, and link name corresponding to the remote cluster to be synchronized specifically includes: obtaining the key corresponding to the remote cluster to be synchronized and verifying the key; if the verification result meets a first preset condition, obtaining the Internet Protocol address bound to the corresponding monitoring process and the corresponding verification value; adding an interface corresponding to the link name using the Internet Protocol address and the verification value, and establishing the connection between the local cluster and the remote cluster to be synchronized using the interface.

[0051] In this embodiment, the step of adding an interface corresponding to the link name using the Internet Protocol address and the checksum, and establishing a connection between the local cluster and the remote cluster to be synchronized using the interface, specifically includes: adding an interface corresponding to the link name using the Internet Protocol address and the checksum; and calling the interface and remote connection in the remote cluster to be synchronized via Secure Shell protocol, so as to add the interface in the remote cluster to be synchronized and establish a connection between the local cluster and the remote cluster to be synchronized. For example... Figure 2 The diagram illustrates a specific connection establishment process. The local cluster sends a key retrieval request to the remote cluster to be synchronized, obtaining the key returned by the remote cluster. The local cluster then verifies the key to ensure its security. If the verification fails, a preset verification error message is returned to the target server for further instructions. If the verification succeeds, the local cluster sends the IP address (Internet Protocol version number) bound to the monitor process (mon) to the remote cluster to be synchronized. The local cluster sends a request to the remote cluster to obtain a keyring (checksum) after the remote cluster returns the IP address bound to the mon. The local cluster then sends a keyring request to the remote cluster to be synchronized, so that the remote cluster can return the keyring. Using the internet protocol address and checksum, the local cluster adds an interface corresponding to the connection name. If adding fails, a preset failure message is returned to the target server for further instructions. If adding succeeds, the local cluster calls the interface and establishes a remote connection via Secure Shell (SSH) to add the interface to the remote cluster, establishing a connection between the local and remote clusters. The local cluster then returns the processing result to the target server.

[0052] Step S12: Record the local directory of the local cluster and the remote directory of the remote cluster to be synchronized through the connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized.

[0053] In this embodiment, a Pair is a link from the local cluster to the remote cluster to be synchronized, and the Pair specifies the remote replication directory. Creating a Pair requires operation on the primary cluster, and involves selecting the remote cluster, the local directory, and the remote directory. After creation, the local directory can be the primary directory, and the remote directory can be the secondary directory. This means that data is synchronized from the primary directory to the secondary directory.

[0054] Step S13: Based on the replication relationship, the local directory, and the data resources in the local directory, perform directory snapshot-level data synchronization on the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster.

[0055] In this embodiment, for example Figure 3 The diagram illustrates a specific data synchronization process. After a remote replication relationship is established between the main directory and the slave directory, synchronization is initiated first. Before synchronization, a snapshot of the main directory is created, and the main directory data (snap1) at that snapshot time is copied to the slave directory to ensure that the slave directory data (snap2) is consistent with the main directory data. For example, if the main directory snapshot time is 13:24 on July 15, 2022, then the slave directory data will be synchronized to 13:24 on July 15, 2022.

[0056] In this embodiment, remote replication is based on directory-level snapshot technology to achieve asynchronous data replication between the master and slave storage systems. A directory snapshot is a backup image of the directory at a specific point in time. Remote replication synchronizes data based on snapshots, which does not affect business operations. Snapshot data is asynchronously replicated to one or more slave clusters. After replication is complete, a snapshot with the same name as the synchronized snapshot is created, ensuring that data in the slave clusters can still be used normally when the master cluster fails.

[0057] Step S14: Determine whether the preset disaster recovery data synchronization conditions are met. If they are met, use the replication relationship to obtain the data resources corresponding to the remote directory in the synchronized remote cluster, and perform data synchronization at the directory snapshot level.

[0058] In this embodiment, it can be understood that the main directory and the slave directory will perform data synchronization every certain period of time. If an abnormal disconnection occurs during a data synchronization process, the internal I / O (Input / Output) failure of the remote replication or the replication link needs to be eliminated or restored. At this time, the Pair enters the "waiting to recover" state, and the remote replication Pair needs to be manually synchronized to synchronize the resource data of the main directory and the slave directory, thereby realizing the data synchronization function after abnormal disconnection during asynchronous disaster recovery based on massive distributed file system.

[0059] As can be seen, based on the key, Internet Protocol address, and link name corresponding to the remote cluster to be synchronized, a connection relationship is established between the local cluster and the remote cluster to be synchronized. The local directory of the local cluster and the remote directory of the remote cluster to be synchronized are recorded through this connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized. Based on the replication relationship, the local directory, and the data resources in the local directory, directory snapshot-level data synchronization is performed on the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster. It is determined whether the preset disaster recovery data synchronization conditions are met. If so, the data resources corresponding to the remote directory in the synchronized remote cluster are obtained using the replication relationship, and directory snapshot-level data synchronization is performed. Therefore, this application establishes a connection relationship between the local cluster and the remote cluster to be synchronized, which allows for the replication relationship between the local cluster and the remote cluster to be synchronized. This enables subsequent directory snapshot-level data synchronization because directory snapshots are point-in-time backup images, reducing computing resources and thus not affecting other business operations.

[0060] See Figure 4 As shown in the illustration, this application discloses a specific data synchronization method applied to a local cluster, including:

[0061] Step S21: Based on the key, Internet Protocol address and link name corresponding to the remote cluster to be synchronized, establish the connection relationship between the local cluster and the remote cluster to be synchronized.

[0062] In this embodiment, disaster recovery backup scenarios are divided into point-to-point disaster recovery and several types of disaster recovery. Point-to-point disaster recovery involves deploying one production site and one disaster recovery site, with the disaster recovery site serving as a data backup for the production site. Centralized disaster recovery involves deploying multiple production sites and one disaster recovery site, with data being copied and backed up from the main directories within different production sites to different slave directories on the disaster recovery site. Data distribution scenarios refer to the periodic copying of data from the main site to different slave sites, primarily used in scenarios similar to headquarters periodically distributing data to branch offices.

[0063] Step S22: Record the local directory of the local cluster and the remote directory of the remote cluster to be synchronized through the connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized.

[0064] Step S23: Based on the replication relationship and the local directory, delete the remote directories in the remote cluster to be synchronized that meet the preset deletion conditions to obtain the first remote directory.

[0065] In this embodiment, the step of deleting the remote directory in the remote cluster to be synchronized that meets the preset deletion conditions based on the replication relationship and the local directory to obtain the first remote directory specifically includes: determining whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions based on the replication relationship and the local directory; if it does, deleting the current remote directory and updating the current remote directory to obtain the current first remote directory.

[0066] In this embodiment, determining whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions based on the replication relationship and the local directory, and deleting the current remote directory if it meets the conditions, specifically includes: determining whether the current remote directory in the remote cluster to be synchronized exists in the local directory based on the replication relationship and the local directory; if it does not exist, deleting the current remote directory; or, determining whether the file name in the current remote directory is consistent with the file name in the corresponding local directory based on the replication relationship and the local directory; if they are consistent, determining whether the file type in the current remote directory is consistent with the file type in the corresponding local directory; if they are inconsistent, deleting the current remote directory. It can be understood that current remote directories that exist in the remote directory but not in the local directory will be deleted, and current remote directories whose file names are consistent with the file names in the local directory but whose file types are inconsistent will be deleted.

[0067] Step S24: Compare the differences between the local directory and the first remote directory, and use the differences to create a second remote directory to obtain a third remote directory containing the first remote directory and the second remote directory. Then, based on the replication relationship, the local directory and the data resources in the local directory, perform directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster.

[0068] In this embodiment, the step of performing directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized, based on the replication relationship, the local directory, and the data resources in the local directory, to obtain a synchronized remote cluster corresponding to the local cluster, specifically includes: modifying the metadata of the current third remote directory and determining whether the current third remote directory meets preset data synchronization conditions; if it does, determining the preset data synchronization type corresponding to the current third remote directory; performing directory snapshot-level data synchronization on the current third remote directory based on the replication relationship, the local directory, the data resources in the local directory, and the preset data synchronization type, to obtain a currently synchronized remote cluster corresponding to the local cluster, and modifying the metadata of the currently synchronized remote cluster. It is understood that the metadata of the third remote directory can be the directory mode, the directory modification time (mtime), and the directory access time (atime). The system determines whether the current third-party remote directory meets the preset data synchronization conditions. These conditions include, for example, if the corresponding file does not exist in the current third-party remote directory, then both the data and metadata corresponding to that file are synchronized; if the file types are inconsistent, then both the corresponding data and metadata are synchronized; if the file sizes or modification times are inconsistent, then both the corresponding data and metadata need to be synchronized; and if the permission attribute change times are inconsistent, then the metadata needs to be synchronized. Data synchronization is based on the preset data synchronization type corresponding to the current third-party remote directory. For example, if the preset data synchronization type for the current third-party remote directory is a regular file or a hard link, then a maximum of 8*8MB of data is read from the source file each time and written to the destination file, where the source file is a file in the local directory and the destination file is the corresponding file in the current third-party remote directory; if the preset data synchronization type for the current third-party remote directory is a soft link, then the original link is released and a new link is created. During data synchronization, metadata also needs to be synchronized. This includes basic metadata such as mode, mtime, atime, uid (User Identification), and gid (Group Identification), as well as extended metadata attributes like ACL (Access Control Lists), WORM (Directory Quantity Limit), and directory configuration. After data synchronization is complete, a snapshot with the same name can be created in the synchronized remote cluster, and the snapshot ID (Identity document) from the local directory can be stored in the snapshot's metadata.

[0069] Step S25: Determine whether the preset disaster recovery data synchronization conditions are met. If they are met, use the replication relationship to obtain the data resources corresponding to the remote directory in the synchronized remote cluster, and perform data synchronization at the directory snapshot level.

[0070] In this embodiment, if an abnormal disconnection occurs during a data synchronization process, it is necessary to eliminate the internal I / O fault of the remote replication or restore the replication link. At this time, the Pair enters a "pending recovery" state, requiring manual data synchronization of the remote replication Pair to synchronize the master and slave resource data, thereby realizing the disaster recovery function for abnormal disconnections. The command obtained by the Pair can be: `icfs daemon client.mirror Pair ls`, so that data synchronization can be performed based on this command. The running status of the Pair is shown in Table 1 and... Figure 5 As shown:

[0071] Table 1

[0072]

[0073]

[0074] Therefore, this application, based on directory snapshot-level data synchronization, allows selection of snapshot time points for corresponding data synchronization, reducing the required computing resources and significantly minimizing the impact on other services. It also ensures that when local resources fail, the corresponding data resources in the remote directory of the synchronized remote cluster can be used normally for data synchronization.

[0075] See Figure 6 As shown in the figure, this application discloses a data synchronization device applied to a local cluster, including:

[0076] The connection relationship establishment module 11 is used to establish a connection relationship between the local cluster and the remote cluster to be synchronized based on the key, Internet protocol address and link name corresponding to the remote cluster to be synchronized.

[0077] The replication relationship acquisition module 12 is used to record the local directory of the local cluster and the remote directory of the remote cluster to be synchronized through the connection relationship, so as to obtain the replication relationship between the local cluster and the remote cluster to be synchronized.

[0078] The first synchronization module 13 is used to perform directory snapshot-level data synchronization on the remote cluster to be synchronized based on the replication relationship, the local directory and the data resources in the local directory, so as to obtain a synchronized remote cluster corresponding to the local cluster.

[0079] The second synchronization module 14 is used to determine whether the preset disaster recovery data synchronization conditions are met. If they are met, the data resources corresponding to the remote directory in the synchronized remote cluster are obtained using the replication relationship, and the data synchronization at the directory snapshot level is performed.

[0080] As can be seen, based on the key, Internet Protocol address, and link name corresponding to the remote cluster to be synchronized, a connection relationship is established between the local cluster and the remote cluster to be synchronized. The local directory of the local cluster and the remote directory of the remote cluster to be synchronized are recorded through this connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized. Based on the replication relationship, the local directory, and the data resources in the local directory, directory snapshot-level data synchronization is performed on the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster. It is determined whether the preset disaster recovery data synchronization conditions are met. If so, the data resources corresponding to the remote directory in the synchronized remote cluster are obtained using the replication relationship, and directory snapshot-level data synchronization is performed. Therefore, this application establishes a connection relationship between the local cluster and the remote cluster to be synchronized, which allows for the replication relationship between the local cluster and the remote cluster to be synchronized. This enables subsequent directory snapshot-level data synchronization because directory snapshots are point-in-time backup images, reducing computing resources and thus not affecting other business operations.

[0081] In some specific embodiments, the connection relationship establishment module 11 includes:

[0082] The verification value acquisition unit is used to acquire the key corresponding to the remote cluster to be synchronized and verify the key. If the verification result meets the first preset condition, the corresponding Internet protocol address bound to the monitoring process and the corresponding verification value are acquired.

[0083] The first relationship establishment unit is used to add an interface corresponding to the link name using the Internet Protocol address and the check value, and to establish a connection relationship between the local cluster and the remote cluster to be synchronized using the interface.

[0084] In some specific embodiments, the first relationship establishment unit includes:

[0085] An interface adding unit is used to add an interface corresponding to the link name using the Internet protocol address and the verification value;

[0086] The second relationship establishment unit is used to call the interface and remote connection in the remote cluster to be synchronized through the Secure Shell protocol, so as to add the interface in the remote cluster to be synchronized and establish the connection relationship between the local cluster and the remote cluster to be synchronized.

[0087] In some specific embodiments, the first synchronization module 13 includes:

[0088] The first remote directory acquisition unit is used to delete the remote directories in the remote cluster to be synchronized that meet the preset deletion conditions based on the replication relationship and the local directory, so as to obtain the first remote directory.

[0089] The synchronized remote cluster acquisition unit is used to compare the differences between the local directory and the first remote directory, and use the differences to create a second remote directory to obtain a third remote directory containing the first remote directory and the second remote directory. Then, based on the replication relationship, the local directory and the data resources in the local directory, the unit performs directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster.

[0090] In some specific embodiments, the first remote directory acquisition unit includes:

[0091] The current first remote directory acquisition unit is used to determine whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions based on the replication relationship and the local directory. If it meets the conditions, the current remote directory is deleted and the current remote directory is updated to obtain the current first remote directory.

[0092] In some specific embodiments, the current first remote directory acquisition unit includes:

[0093] The first deletion unit is used to determine whether the current remote directory in the remote cluster to be synchronized exists in the local directory based on the replication relationship and the local directory; if it does not exist, the current remote directory is deleted.

[0094] The second deletion unit is used to determine, based on the copy relationship and the local directory, whether the file name in the current remote directory is consistent with the file name in the corresponding local directory. If they are consistent, it determines whether the file type in the current remote directory is consistent with the file type in the corresponding local directory. If they are inconsistent, it deletes the current remote directory.

[0095] In some specific embodiments, the first synchronization module 13 includes:

[0096] The metadata modification unit is used to modify the metadata of the current third remote directory and determine whether the current third remote directory meets the preset data synchronization conditions. If it does, it determines the preset data synchronization type corresponding to the current third remote directory. Based on the replication relationship, the local directory, the data resources in the local directory, and the preset data synchronization type, it performs directory snapshot-level data synchronization on the current third remote directory to obtain the current synchronized remote cluster corresponding to the local cluster, and modifies the metadata of the current synchronized remote cluster.

[0097] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data synchronization method performed by the electronic device disclosed in any of the foregoing embodiments.

[0098] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0099] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0100] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0101] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the data synchronization method executed by the electronic device as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0102] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the method steps performed during the data synchronization process disclosed in any of the foregoing embodiments.

[0103] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0104] The above provides a detailed description of the data synchronization method, apparatus, device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data synchronization method, characterized in that, Applied to local clusters, including: Based on the key, Internet protocol address, and link name corresponding to the remote cluster to be synchronized, establish a connection between the local cluster and the remote cluster to be synchronized; The local directory of the local cluster and the remote directory of the remote cluster to be synchronized are recorded through the connection relationship to obtain the replication relationship between the local cluster and the remote cluster to be synchronized. Based on the replication relationship, the local directory, and the data resources in the local directory, the remote cluster to be synchronized is synchronized at the directory snapshot level to obtain a synchronized remote cluster corresponding to the local cluster. Determine whether the preset disaster recovery data synchronization conditions are met. If they are met, use the replication relationship to obtain the data resources corresponding to the remote directory in the synchronized remote cluster, and perform data synchronization at the directory snapshot level. The process of establishing a connection between the local cluster and the remote cluster to be synchronized, based on the key, internet protocol address, and link name corresponding to the remote cluster to be synchronized, includes: A key retrieval request is sent to the remote cluster to be synchronized, and the key returned by the remote cluster is obtained. If the key fails verification, the target server returns a preset verification error message. If the key passes verification, the Internet Protocol address bound to the monitoring process returned by the remote cluster is obtained, and a verification value retrieval request is sent to the remote cluster to be synchronized, and the verification value returned by the remote cluster is obtained. An interface corresponding to the link name is added using the Internet Protocol address and the verification value. The interface is called and a remote connection is established in the remote cluster to be synchronized via the Secure Shell protocol, so that the interface is added in the remote cluster to be synchronized, and a connection relationship is established between the local cluster and the remote cluster to be synchronized. The step of performing directory snapshot-level data synchronization on the remote cluster to be synchronized based on the replication relationship, the local directory, and the data resources in the local directory to obtain a synchronized remote cluster corresponding to the local cluster includes: Based on the replication relationship, the local directory deletes the remote directories in the remote cluster to be synchronized that meet the preset deletion conditions to obtain a first remote directory; compare the differences between the local directory and the first remote directory, and use the differences to create a second remote directory to obtain a third remote directory containing the first and second remote directories; then, based on the replication relationship, the local directory, and the data resources in the local directory, perform directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster.

2. The data synchronization method according to claim 1, characterized in that, The step of deleting the remote directories in the remote cluster to be synchronized that meet the preset deletion conditions based on the replication relationship and the local directory to obtain the first remote directory includes: Based on the replication relationship and the local directory, it is determined whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions. If it does, the current remote directory is deleted and the current remote directory is updated to obtain the current first remote directory.

3. The data synchronization method according to claim 2, characterized in that, The step of determining whether the current remote directory in the remote cluster to be synchronized meets the preset deletion conditions based on the replication relationship and the local directory, and deleting the current remote directory if it meets the conditions, includes: Based on the replication relationship and the local directory, it is determined whether the current remote directory in the remote cluster to be synchronized exists in the local directory. If it does not exist, the current remote directory is deleted. Alternatively, based on the copy relationship and the local directory, determine whether the file name in the current remote directory is consistent with the file name in the corresponding local directory. If they are consistent, determine whether the file type in the current remote directory is consistent with the file type in the corresponding local directory. If they are inconsistent, delete the current remote directory.

4. The data synchronization method according to claim 2, characterized in that, The step of performing directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized, based on the replication relationship, the local directory, and the data resources in the local directory, to obtain a synchronized remote cluster corresponding to the local cluster, includes: Modify the metadata of the current third remote directory and determine whether the current third remote directory meets the preset data synchronization conditions. If it does, determine the preset data synchronization type corresponding to the current third remote directory. Based on the replication relationship, the local directory, the data resources in the local directory, and the preset data synchronization type, the current third remote directory is synchronized at the directory snapshot level to obtain the current synchronized remote cluster corresponding to the local cluster, and the metadata of the current synchronized remote cluster is modified.

5. A data synchronization device, characterized in that, Applied to local clusters, including: The connection establishment module is used to establish a connection between the local cluster and the remote cluster to be synchronized based on the key, Internet protocol address and link name corresponding to the remote cluster to be synchronized. The replication relationship acquisition module is used to record the local directory of the local cluster and the remote directory of the remote cluster to be synchronized through the connection relationship, so as to obtain the replication relationship between the local cluster and the remote cluster to be synchronized. The first synchronization module is used to perform directory snapshot-level data synchronization on the remote cluster to be synchronized based on the replication relationship, the local directory, and the data resources in the local directory, so as to obtain a synchronized remote cluster corresponding to the local cluster. The second synchronization module is used to determine whether the preset disaster recovery data synchronization conditions are met. If they are met, the data resources corresponding to the remote directory in the synchronized remote cluster are obtained using the replication relationship, and the data synchronization at the directory snapshot level is performed. The data synchronization device is specifically used for: A key retrieval request is sent to the remote cluster to be synchronized, and the key returned by the remote cluster is obtained. If the key fails verification, the target server returns a preset verification error message. If the key passes verification, the Internet Protocol address bound to the monitoring process returned by the remote cluster is obtained, and a verification value retrieval request is sent to the remote cluster to be synchronized, and the verification value returned by the remote cluster is obtained. An interface corresponding to the link name is added using the Internet Protocol address and the verification value. The interface is called and a remote connection is established in the remote cluster to be synchronized via the Secure Shell protocol, so that the interface is added in the remote cluster to be synchronized, and a connection relationship is established between the local cluster and the remote cluster to be synchronized. The first synchronization module is specifically used for: Based on the replication relationship, the local directory deletes the remote directories in the remote cluster to be synchronized that meet the preset deletion conditions to obtain a first remote directory; compare the differences between the local directory and the first remote directory, and use the differences to create a second remote directory to obtain a third remote directory containing the first and second remote directories; then, based on the replication relationship, the local directory, and the data resources in the local directory, perform directory snapshot-level data synchronization on the third remote directory in the remote cluster to be synchronized to obtain a synchronized remote cluster corresponding to the local cluster.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data synchronization method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the steps of the data synchronization method as described in any one of claims 1 to 4.

Images