Data synchronization method for distributed cluster and related device therefor
By using a master-slave state switching mechanism based on storage management unit groups and a finite state machine mechanism in a distributed cluster, real-time data synchronization between different sites is achieved, solving the service unavailability problem caused by downtime migration and realizing real-time synchronization of incremental data.
Patent Information
- Application Number
- PCT/CN2024/092720
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-19
- Filing Date
- 2024-05-11
- Publication Date
- 2024-11-28
AI Technical Summary
In existing technologies, data synchronization between isolated sites requires downtime for migration, resulting in service unavailability and the inability to achieve real-time synchronization, especially for incremental data synchronization.
By determining the synchronization time slice based on the master-slave state switching of storage management unit groups in a distributed cluster, generating the data operation log of the master storage management unit, and synchronizing it to the slave storage management unit, the synchronization of the data operation log is realized by using a finite state machine mechanism, ensuring real-time data synchronization between different sites.
It enables seamless data migration and synchronization between isolated sites, ensures real-time synchronization of incremental data, solves the problem of service unavailability, and is suitable for data synchronization of any type.
Smart Images

Figure CN2024092720_28112024_PF_FP_ABST
Abstract
Description
Data synchronization method of distributed cluster and related equipment
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 19, 2023, with application number 202310576529.7 and application name “Data synchronization method for distributed clusters and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of computer technology, and in particular to a data synchronization method for a distributed cluster and related equipment. Background Art
[0003] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that what is described herein is prior art by its inclusion in this section.
[0004] Artifact warehouses require redundant backups at different sites and synchronized artifacts between them. This allows for master-slave failover to achieve high availability in the event of regional issues. Network latency due to regional factors can also be addressed through multi-site deployment. Furthermore, artifact synchronization is required between different deployment environments. Unverified artifacts must first be verified in the development environment and then, once verified, synchronized to the stable production environment using site synchronization.
[0005] Because different sites in the product warehouse are isolated from each other, conventional online data synchronization methods cannot achieve data synchronization between different sites in the product warehouse. Data migration between isolated sites in related technologies typically uses a downtime migration method. However, downtime migration results in temporary service unavailability and cannot achieve real-time synchronization of incremental data.
[0006] Summary of the Invention
[0007] The distributed cluster data synchronization method and related devices provided by the embodiments of the present invention at least solve the problem of service unavailability caused by shutdown migration between isolated different sites.
[0008] A distributed cluster data synchronization method, comprising:
[0009] Determine the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster;
[0010] Generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice;
[0011] The data operation log is synchronized to a slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log.
[0012] In some embodiments, the method further comprises:
[0013] generating metadata of the master storage management unit so that when synchronizing the data operation log to a slave storage management unit in the storage management unit group, a data operation log to be synchronized among the data operation logs is determined based on the metadata, wherein the metadata includes a synchronization status of the data operation log;
[0014] The metadata is updated based on the synchronization result of the data operation log.
[0015] In some embodiments, generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice includes:
[0016] Persisting data operation information of the cluster in the distributed cluster according to the execution order of the data operations, wherein the data operation information includes the operation object, operation content, and information of the storage management unit that performs the data operation;
[0017] According to the data operation information of the cluster, a data operation log of the primary storage management unit in the synchronization time slice is obtained, wherein the data operation log is sorted according to the execution order of the data operations.
[0018] In some embodiments, determining the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster includes:
[0019] A master-slave state switching logic clock of the storage management unit group is obtained, and the synchronization time slice and the sequence number of the synchronization time slice are determined according to the master-slave state switching logic clock.
[0020] In some embodiments, the method further comprises:
[0021] When the master-slave status of the storage management unit of the cluster in the distributed cluster is switched, the cluster reports the master-slave status switching logical clock to the storage system, and the storage system sequentially increments the sequence number of the synchronization time slice according to the master-slave status switching logical clock and sends it to the distributed cluster.
[0022] In some embodiments, the method further comprises:
[0023] The distributed cluster periodically obtains the sequence number of the synchronization time slice from the storage system.
[0024] In some embodiments, synchronizing the data operation log to a slave storage management unit in the storage management unit group includes:
[0025] According to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the serial number of the synchronization time slice, the data operation log to be synchronized in the main storage management unit is determined, and the data operation log to be synchronized is synchronized to the slave storage management unit.
[0026] In some embodiments, when the sequence numbers of the synchronization time slices of the primary storage management unit and the secondary storage management unit are the same, the data operation logs to be synchronized are determined and synchronized.
[0027] In some embodiments, in an initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, and the storage management units in the slave clusters in the distributed cluster are all slave storage management units, and the master-slave switching of the storage management unit group is managed by a preset controller.
[0028] In some of the embodiments, the preset controller sets the storage management unit with the identifier of the storage management unit as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.
[0029] In some embodiments, when there is data to be written into the storage management unit group, the primary storage management unit in the storage management unit group is determined by querying the preset controller, and the data is written into the primary storage management unit.
[0030] In some embodiments, the method further comprises:
[0031] The data storage state of the storage management unit is used as a finite state machine. A leader node and at least one follower node are respectively selected in the main storage management unit and the slave storage management unit. The leader node synchronizes the copies of the finite state machine within the storage management unit group based on the data operation log.
[0032] A distributed cluster system includes multiple clusters and a storage system, wherein the multiple clusters include a master cluster and at least one slave cluster, and the storage system is connected to the multiple clusters respectively;
[0033] The storage system is used to store the sequence number of the synchronization time slice; the multiple clusters all perform data synchronization through the above-mentioned data synchronization method.
[0034] In some embodiments, the distributed cluster system further includes a preset controller, which is used to control the master-slave state switching of the storage management unit and record the master-slave state of the storage management unit.
[0035] An electronic device includes: a processor and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the above-mentioned data synchronization method.
[0036] A non-transitory machine-readable medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the above-mentioned data synchronization method.
[0037] The data synchronization method and related equipment of a distributed cluster provided by an embodiment of the present invention determine the synchronization time slice by switching the master-slave status of the storage management unit group in the distributed cluster; generate a data operation log of the master storage management unit in the storage management unit group within the synchronization time slice; synchronize the data operation log to the slave storage management unit in the storage management unit group, so that the slave storage management unit synchronizes data based on the data operation log, thereby solving the problems of service unavailability and incremental data synchronization consistency caused by shutdown migration between different isolated sites, realizing non-stop data migration between isolated sites, and realizing incremental data synchronization between isolated sites.
[0038] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be derived from these drawings without inventive effort.
[0040] FIG1 is a flow chart of a distributed cluster data synchronization method according to an embodiment of the present invention.
[0041] FIG2 is a schematic diagram of generating a data operation log and metadata of a storage management unit according to an embodiment of the present invention.
[0042] FIG3 is a flow chart of a storage management unit sending a data operation log request according to an embodiment of the present invention.
[0043] FIG4 is a flow chart of a storage management unit receiving a data operation log request according to an embodiment of the present invention.
[0044] FIG5 is a flowchart of a storage management unit serving as a Master sending a data snapshot according to an embodiment of the present invention.
[0045] FIG6 is a schematic diagram of synchronous control of multiple storage management units according to an embodiment of the present invention.
[0046] FIG7 is a schematic diagram of synchronous task processing based on a distributed consensus algorithm according to an embodiment of the present invention.
[0047] FIG8 is a schematic diagram of the structure of a distributed cluster according to an embodiment of the present invention.
[0048] FIG9 is a schematic structural diagram of the electronic device of this embodiment. DETAILED DESCRIPTION
[0049] The following describes embodiments of the present invention in more detail with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.
[0050] As the name suggests, an artifact repository is a warehouse for artifacts. Artifacts are the tangible results of software delivery, typically in runnable binary form. Therefore, an artifact repository is often also referred to as a binary artifact repository. During the development phase, the artifact repository provides a single entry point for dependency resolution for microservice developers using various development languages. During the build phase, it provides a single dependency resolution source and a unified artifact management library for build tools in various languages. During the testing phase, all test environment deployment tools pull artifacts that meet test conditions from the artifact repository for deployment. After testing, the test results are fed back to the artifact repository and associated with the artifacts. During the deployment phase, artifacts are checked against quality checkpoints to ensure they meet deployment conditions. If they do, the deployment tool pulls the artifacts from the artifact repository and deploys them to the environment.
[0051] To ensure data security, real-time data synchronization requires a unified, synchronized clock between different devices to avoid synchronization errors and ensure data rollback. However, the product warehouse is isolated across multiple regions. Since these isolated regions lack synchronized clocks, conventional real-time data synchronization methods cannot be used. Instead, a downtime migration approach is required.
[0052] For example, in the related art, Jforg's repository replication data migration solution uses HTTP calls to upload products by calling an interface that does not exist in the existing product system. However, this method cannot avoid the security and stability of data synchronization under non-Byzantine problems. It only supports product migration but not data migration in other formats, and cannot isolate data in multiple regions.
[0053] The open source data migration solution Rsync uses an SSH connection to transfer data to the target system, invoking the data interface on the remote system. This method requires manual migration and cannot automatically synchronize incremental data. Furthermore, it is only a simple file synchronization and cannot establish a connection between files and systems.
[0054] The distributed cluster system of this embodiment includes a master cluster and at least one slave cluster. Each cluster includes multiple nodes, which are further divided into multiple isolated storage management units according to regions. The storage management units are also called regions. The master storage management unit and the slave storage management unit for data synchronization are a storage management unit group. There is one and only one master storage management unit in the storage management unit group, and at least one or more slave storage management units. The master-slave status of the storage management units in the storage management unit group can be switched. In the initial state, the regions in the master cluster are all in the master role, that is, in the Master state, and the regions in the slave cluster are all in the slave role, that is, in the Slaver state. The master cluster acts as the cluster that initiates data synchronization during the data synchronization process; the slave cluster acts as the cluster that receives data synchronization during the data synchronization process. However, after the master-slave status of the regions in the master-slave cluster is switched, data synchronization may also be initiated by the slave cluster and received by the master cluster.
[0055] For example, during data synchronization, initially, data is written to the master region in the master cluster and then synchronized to the slave region in the slave cluster. When the master-slave status of two or more regions switches, data is written to the master region in the slave cluster and then synchronized to the slave region in the master cluster.
[0056] To achieve data synchronization between isolated sites and avoid service downtime and unavailability, an embodiment of the present invention provides a distributed cluster data synchronization method. Figure 1 is a flow chart of the distributed cluster data synchronization method according to an embodiment of the present invention. As shown in Figure 1, the process includes the following steps:
[0057] Step S101 : determining a synchronization time slice based on the master-slave state switching of a storage management unit group in a distributed cluster.
[0058] Step S102: Generate a data operation log of the primary storage management unit in the storage management unit group within the synchronization time slice.
[0059] Step S103: Synchronize the data operation log to the slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log.
[0060] Through the above steps, synchronization time slices are determined based on the master-slave state switching of storage management unit groups in a distributed cluster. The data storage states of the storage management units within the synchronization time slices are synchronized based on data operation logs, solving the problem of real-time data synchronization between multiple isolated storage management units. This data synchronization method not only enables real-time data synchronization of existing data, but also of incremental data, without requiring downtime for migration and is applicable to any type of data synchronization.
[0061] The synchronization status of the data operation log, for example, the synchronized data operation log, the data operation log submitted to the slave storage management unit, and other information, can be stored in metadata and updated according to the synchronization result of the data operation log to determine the data storage status of each storage management unit when synchronizing data. In some embodiments, the method further includes: generating metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes the synchronization status of the data operation log; and updating the metadata based on the synchronization result of the data operation log.
[0062] In some embodiments, the data storage state of the storage management unit is used as a finite state machine, so that data synchronization of the storage management unit can be achieved based on a replica synchronization mechanism of the finite state machine. In the replica synchronization mechanism of the finite state machine, replica synchronization is performed based on a synchronization log. In this embodiment, the data operation log is used as the synchronization log for the replica synchronization of the finite state machine.
[0063] In this embodiment, the predefined data objects include cluster objects, node objects and finite state machines. Among them, the cluster object includes the status of each cluster node. The node object includes the metadata of the node, which is used to record the status information of the cluster during the data synchronization process, including the serial number (term) of the synchronization time slice of the storage management unit, the index (logIndex) of the data operation log used to mark the location information of the data operation log, the index number (snapshotIndex) of the last data operation log contained in the snapshot, the serial number (snapshotTerm) of the synchronization time slice of the last data operation log contained in the snapshot, and the index number (preLogIndex) of the largest log that has been applied to the slave. The node object also includes node location information (endpoint), which is used to represent the coordinates of the node, which is generally the IP address of the terminal in a network environment. The node object can also include the status of the master node that initiates the data synchronization cluster during the data synchronization process. The finite state machine includes the executed data objects and the executed actions (i.e., data operations). In this embodiment, the storage management unit with a master-slave relationship is regarded as a finite state machine.
[0064] The data storage status includes both existing data and data operations such as additions, deletions, and modifications performed on existing data. Existing data is typically stored as snapshots, and data operations are recorded in operation logs. If the existing data between storage management units in a master-slave relationship is synchronized, meaning that the existing data of each storage management unit is identical, the data storage status of the storage management unit can be represented solely by data operations on the existing data.
[0065] A finite state machine has a finite number of states. The state machine begins in a given start state (e.g., the initial state, or a state with the same inventory data). Each input received by the state machine generates a new state and corresponding output through transition equations and output equations. This new state remains until the next input arrives, and the generated output is delivered to the corresponding recipient. Multiple copies of the same state machine can be synchronized across multiple storage management units using a replicated state machine approach. Multiple copies of the same state machine begin in a start state and, receiving the same inputs in the same order, reach the same state with the same output generated. This is known as a replicated state machine.
[0066] The term of the synchronization time slice is used as the synchronization clock for all clusters in the distributed cluster. The term of the synchronization time slice is strictly incremented from a set value (e.g., 0). The term of the synchronization time slice is cached in a storage system with high-performance storage media.
[0067] In the above steps, the synchronization time slice of the finite state machine is updated based on the master-slave state switch of any group of storage management units. Since the update of the synchronization time slice sequence number is marked by the master-slave state switch of the storage management unit, the state of the finite state machine of any storage management unit after the master-slave state switch falls within at least one synchronization time slice, that is, a correspondence can be established between the finite state machine of the storage management unit and the sequence number of the synchronization time slice. Therefore, by obtaining the master-slave state switch logical clock of any group of storage management units in the cluster, the cluster can determine the synchronization time slice sequence number of the synchronization time slice that the storage management unit enters starting from the master-slave state switch logical clock. Since the synchronization time slice sequence number is also affected by the master-slave state switch of storage management units in other groups in the distributed cluster and is automatically incremented, for a particular storage management unit, the state change of the finite state machine may span multiple synchronization time slices and correspond to the sequence numbers of multiple synchronization time slices. In this embodiment, the data operation log and metadata generated in each synchronization time slice are marked with the corresponding synchronization time slice sequence number.
[0068] When the master / slave status of any group of storage management units in any cluster in a distributed cluster switches, the cluster reports the master / slave switchover logical clock to the storage system. The synchronization time slice sequence number in the storage system increments by one. The storage system then sequentially increments the synchronization time slice sequence number based on the master / slave switchover logical clock and distributes it to the distributed cluster. Furthermore, clusters in the distributed cluster periodically retrieve synchronization time slice sequence numbers to prevent failures in synchronization time slice sequence number distribution.
[0069] In this embodiment, the input data for the finite state machine is data operation information. To meet the input consistency requirement of the replicated state machine, the data operation information must not only have identical operation content but also the same execution order. Therefore, in each cluster of the distributed cluster, the data operation information of the cluster is persisted according to the execution order of the data operations. The data operation information includes the operation object, operation content, and information about the storage management unit that performs the data operation.
[0070] Specifically, in the cluster that receives data writes (usually the Master cluster), a point record can be attached to each write operation to record the operation object, operation content, and information about the storage management unit that performs the data operation. After encapsulation, it is used as the incremental operation log (pendingLog) generated by the data synchronization initiator, and then the incremental operation log is persisted to the database. The database is used to ensure that the incremental operation log is stored in the order of execution.
[0071] Because in the cluster that receives data writes, the incremental operation logs executed by all storage management units are stored together in the order of execution. In order to synchronize data in units of storage management units, the incremental operation logs can be further grouped and processed according to the storage management units. For example, as shown in Figure 2, in the cluster that receives data writes, a node for processing synchronization tasks is selected to start a thread to read incremental operation logs in batches from the database, and then divide them according to the storage management units. In the same storage management unit, each log is marked with the serial number of the synchronization time slice and the index number (logIndex) of the data operation log to generate a data operation log for data synchronization.
[0072] If the current storage management unit generates a synchronization log (log) for the first time, the index number of the data operation log will automatically increase in order starting from 0, and the metadata (metadata) of the storage management unit will be initialized at the same time. The data snapshot index number (snapshotIndex) in the metadata is a random value n (for example, it can be any integer between 1-100). The metadata of the storage management unit also includes data used for data synchronization management, for example, the index number (lastAppliedIndex) of the last data operation log of the storage management unit that has been applied from the node, and the index number (committedIndex) of the data operation log that has been submitted by the storage management unit. During initialization, lastAppliedIndex and committedIndex are set to the same value n as snapshotIndex. The serial number of the synchronization time slice in the metadata is set to the serial number of the current synchronization time slice. By persisting the data operation log and metadata, the data preparation work for data synchronization is completed.
[0073] If this is not the first time the current storage management unit is generating a synchronization log (log), that is, if historical data operation logs already exist (including data operation logs that have been synchronized, as well as data operation logs that have not been synchronized), the index number of the newly generated data operation log is incremented by one from the index number of the last data operation log generated last time. The snapshotIndex, lastAppliedIndex, and committedIndex in the metadata are also incremented by one from the snapshotIndex, lastAppliedIndex, and committedIndex generated last time, respectively. The sequence number of the synchronization time slice is also set to the sequence number of the current synchronization time slice. Persisting the data operation logs and metadata completes the data preparation for data synchronization.
[0074] During data synchronization, data synchronization is achieved through replica synchronization using a finite state machine. For the storage management unit, the cluster receiving the data write will determine the data operation log to be synchronized in the primary storage management unit based on the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata, and the sequence number of the synchronization time slice, and then synchronize the data operation log to be synchronized to the secondary storage management unit.
[0075] Referring to Figure 3, the primary storage management unit selects a node for handling synchronization tasks. This node then handles the synchronization tasks and selects the storage management units that require synchronization. During the synchronization task, the next log index number to be synchronized is first retrieved from the cache. If it does not exist, the index number of the last committed log is queried in the metadata. The index number of the previous data operation log, preLogIndex, is then set to the committed index number minus one. The decision to send a snapshot (a snapshot of the current data in the finite state machine) is made by comparing the size of preLogIndex and snapshotIndex. If preLogIndex is not less than snapshotIndex, it indicates that the existing data has been migrated. Otherwise, a batch of data operation logs that require synchronization are retrieved from the database and sent to the secondary storage management unit for processing.
[0076] The process of processing data operation logs from the storage management unit is shown in FIG4 , and a node for processing data synchronization is also selected from the storage management unit. This node is responsible for processing data operation logs.
[0077] In some of the embodiments, in the same storage management unit group, when the serial numbers of the synchronization time slices of the master storage management unit and the slave storage management unit are the same, the data operation logs to be synchronized are determined and the synchronization of the data operation logs to be synchronized is performed. If the serial numbers of the synchronization time slices are different, a retry is performed.
[0078] If the sequence number of the synchronization time slice of the received data operation log is inconsistent with the sequence number of the synchronization time slice of the current node, it may be that the sequence number of the synchronization time slice of the current node has not been updated or the synchronization request of the data operation log has expired. At this time, the node will return the sequence number of the synchronization time slice of the current node to the main storage management unit. If the sequence numbers of the synchronization time slices are consistent, it means that the synchronization request of the data operation log is valid. The node will check again whether the current storage management unit is in the slave role (Slaver) state in the cluster. If it is in the slave state, the log verification will be performed.
[0079] The current FSM status of the slave storage management unit is determined by the committedIndex of the submitted data operation log and the sequence number of the synchronization time slice. If it can match the synchronization request, it only needs to directly apply the log sent by the main storage management unit, update the current FSM status and metadata information, and return the result to the main storage management unit. Otherwise, it is necessary to return the current FSM status to the main storage management unit in the hope that the main storage management unit will send a matching log in the next synchronization request. The main storage management unit may not receive the return value from the slave storage management unit due to a response timeout. In this case, it will retry. If it receives the return value from the slave storage management unit, it will update the next synchronization request sent according to the status returned by the slave storage management unit.
[0080] The above synchronization method can be used for the synchronization of incremental data, and can also be used for the synchronization of stock data. Continuing to refer to Figure 3, when the data operation log request is sent for the first time, there is no data operation log written to the storage management unit from the storage management unit, and the conflict position returned to the main storage management unit is 0. The main storage management unit continues to send the data operation log after updating the nextLogIndex. At this time, the preLogIndex of the data operation log must be less than the randomly generated snapshotIndex, so the main storage management unit will send the snapshot to migrate the stock data. Snapshot is a snapshot of the FSM of the current storage management unit generated by the main storage management unit. The snapshot stores the indexes of all data (such as products) and related data. In some embodiments, the snapshot sent by the main storage management unit can be the URL of the snapshot, and the actual generated snapshot will be uploaded to the file storage system, which can reduce the data size of the snapshot request. The process of sending snapshots by the main storage management unit is shown in Figure 5. First, the storage management unit that needs to migrate inventory is determined in the set of snapshots that need to be sent, and an FSM snapshot of the storage management unit is generated. After sending an installation snapshot request to the slave storage management unit, the installation status of the slave storage management unit is judged. If the sequence number of the request and the synchronization time slice of the slave storage management unit do not match or the installation of the snapshot from the slave storage management unit fails, the installation snapshot request will be added to the set of snapshots to be sent and retried.
[0081] In some embodiments, in an initial state, all storage management units in a master cluster of a distributed cluster are master storage management units, and all storage management units in slave clusters of the distributed cluster are slave storage management units. Master-slave switching of the storage management units is managed by a preset controller. The preset controller adds an identifier of the storage management unit to a gray list and sets the storage management unit with the identifier as the master storage management unit, thereby implementing master-slave switching of the storage management units in the distributed cluster.
[0082] For example, referring to Figure 6, the master-slave relationship of different storage management units in different clusters is set and saved in a queryable master-slave controller. A unified access master-slave controller is set. When the storage management unit is not in the gray list, the storage management units of cluster 1 (for example, the master cluster) are all in the Master state, and the storage management units of cluster 2 (for example, the slave cluster) are all in the Slaver state. At this time, the data flow is from cluster 1 to cluster 2. If the storage management unit is in the gray list, the storage management unit of cluster 2 is in the Master state, and data flows from cluster 2 to cluster 1. Due to bidirectional data writing, the finite state machine (FSM) of the master storage management unit and the slave storage management unit in the same storage management unit group always maintains a consistent state in a safe state. At this time, by controlling whether the storage management unit is in the gray list, the roles of different storage management units in different clusters are controlled, thereby controlling the flow of data synchronization. The data added by the user in any cluster will not be lost. The new data will be synchronized in cluster 1 and cluster 2 in the form of a synchronization log. The above-mentioned safe state means that the FSM states of the primary storage management unit and the secondary storage management unit in the same storage management unit are consistent, and there is no data operation log that has been applied to the primary storage management unit but not yet applied to the secondary storage management unit.
[0083] In some embodiments, when a preset controller is provided for querying the master / slave status of a storage management unit, when new data is written to a storage management unit, the preset controller is queried to determine the master storage management unit, and the new data is written to the master storage management unit. For example, when a user needs to perform a data operation, the user can directly access cluster 1 or cluster 2; the user can query the storage management unit identifier stored in the master / slave controller to confirm whether the access is to the master node or the slave node.
[0084] In order to ensure the reliability of data synchronization and avoid data synchronization failure caused by single node failure, in some embodiments, a leader node and at least one follower node are elected in the main storage management unit and the slave storage management unit respectively, and the leader node in each storage management unit performs the copy synchronization of the finite state machine of the storage management unit with a master-slave relationship. Referring to Figure 7, in this embodiment, based on raft (a distributed consensus algorithm), a leader node can be elected in each cluster or storage management unit to handle synchronization tasks; at the same time, multiple follower nodes are configured to take over the leader node to continue processing synchronization tasks after the leader node fails, ensuring that data will not be lost and can be synchronized to the slave storage management unit. When the leader node of the main storage management unit crashes or other abnormal situations occur, the data in the main storage management unit has been persisted and will not be lost. After selecting a new leader in the follower node, it can continue to synchronize data to the slave storage management unit based on the persisted log and metadata, so the slave storage management unit can still be consistent with the main storage management unit.
[0085] The embodiment of the present invention, based on raft log synchronization technology, can achieve stock data migration and incremental data synchronization of multiple storage management units. It can be applied to files of any format and is not limited to Jforg product files. At the same time, raft is used to ensure the security and reliability of data synchronization under non-Byzantine problems. All synchronized data is applied to the system through FSM, and the associations between data can also be synchronized between different systems. By isolating data through multiple storage management units, the synchronization direction of different storage management units at different sites can be controlled.
[0086] This embodiment also provides a distributed cluster system. Figure 8 is a schematic diagram of the distributed cluster system of this embodiment. As shown in Figure 8, the system includes a cluster 81, at least one cluster 82, and a storage system 83. Cluster 81 acts as a master control, cluster 82 acts as a slave control, and storage system 83 is connected to each of the aforementioned clusters.
[0087] The storage system 83 is used to store the serial numbers of the synchronization time slices; the clusters all perform data synchronization through the above-mentioned distributed cluster data synchronization method.
[0088] In some of the embodiments, the cluster determines the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster; generates a data operation log of the master storage management unit in the storage management unit group within the synchronization time slice; and synchronizes the data operation log to the slave storage management unit in the storage management unit group so that the slave storage management unit performs data synchronization based on the data operation log.
[0089] In some embodiments, the cluster generates metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes the synchronization status of the data operation log; based on the synchronization result of the data operation log, the metadata is updated.
[0090] In some of the embodiments, the cluster persists data operation information of the cluster in the distributed cluster according to the execution order of the data operations, wherein the data operation information includes the operation object, the operation content and the information of the storage management unit that performs the data operation; based on the data operation information of the cluster, the data operation log of the main storage management unit within the synchronization time slice is obtained, wherein the data operation log is sorted according to the execution order of the data operations.
[0091] In some embodiments, the cluster obtains a master-slave state switching logical clock of the storage management unit group, and determines a synchronization time slice and a sequence number of the synchronization time slice according to the master-slave state switching logical clock.
[0092] In some of the embodiments, when the master-slave status of the storage management unit of the cluster in the distributed cluster is switched, the cluster reports the master-slave status switching logical clock to the storage system, and the storage system sequentially increments the sequence number of the synchronization time slice according to the master-slave status switching logical clock and sends it to the distributed cluster.
[0093] In some embodiments, the distributed cluster periodically obtains the sequence number of the synchronization time slice from the storage system.
[0094] In some of the embodiments, the cluster determines the data operation log to be synchronized in the primary storage management unit based on the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata, and the sequence number of the synchronization time slice, and synchronizes the data operation log to be synchronized to the secondary storage management unit.
[0095] In some embodiments, the cluster determines the data operation logs to be synchronized and synchronizes the data operation logs to be synchronized when the sequence numbers of the synchronization time slices of the primary storage management unit and the secondary storage management unit are the same.
[0096] In some embodiments, the distributed cluster system further includes a preset controller, which is used to control the master-slave state switching of the storage management unit and record the master-slave state of the storage management unit.
[0097] In some of the embodiments, in the initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, and the storage management units in the slave clusters in the distributed cluster are all slave storage management units, and the master-slave switching of the storage management unit group is managed by a preset controller.
[0098] In some of the embodiments, the preset controller sets the storage management unit with the identifier of the storage management unit as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.
[0099] In some embodiments, when data is to be written into the storage management unit group, a primary storage management unit in the storage management unit group is determined by querying a preset controller, and the data is written into the primary storage management unit.
[0100] In some of the embodiments, the data storage state of the storage management unit is used as a finite state machine, and a leader node and at least one follower node are respectively elected in the main storage management unit and the slave storage management unit. The leader node synchronizes the copies of the finite state machine within the storage management unit group based on the data operation log.
[0101] An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, wherein the computer program, when executed by the at least one processor, causes the electronic device to perform the method of an embodiment of the present invention.
[0102] An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform the method of the embodiment of the present invention.
[0103] An embodiment of the present invention further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to enable the computer to perform the method of the embodiment of the present invention.
[0104] With reference to Figure 9, the structural block diagram of the electronic device that can be used as the server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0105] As shown in Figure 9, the electronic device includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device can also be stored. The computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0106] Multiple components within the electronic device are connected to the I / O interface 905, including an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 can be any type of device capable of inputting information into the electronic device. The input unit 906 can receive input numeric or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 907 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 908 may include, but is not limited to, a magnetic disk or an optical disk. The communication unit 909 allows the electronic device to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.
[0107] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via the ROM 902 and / or the communication unit 909. In some embodiments, the computing unit 901 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).
[0108] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0109] In the context of an embodiment of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0110] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open-ended, i.e., "including but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; and the term "some embodiments" means "at least some embodiments." The modifications of "one" and "a plurality of" mentioned in the embodiments of the present invention are illustrative and non-restrictive. Those skilled in the art should understand that, unless the context clearly indicates otherwise, they should be understood as "one or more."
[0111] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0112] The various steps described in the method implementation scheme provided in the embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method implementation scheme may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.
[0113] The term "embodiment" in this specification refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments are referenced to each other. In particular, for the embodiments of the device, equipment, and system, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts are referred to the partial description of the method embodiment.
[0114] The above-described embodiments merely illustrate several implementations of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that variations and modifications are possible within the scope of the present invention, as would be apparent to one skilled in the art. These variations and modifications fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.
Claims
1. A distributed cluster data synchronization method, comprising: Determine the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster; Generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice; The data operation log is synchronized to a slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log.
2. The method according to claim 1, wherein: The method further comprises: Generate metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes a synchronization status of the data operation log; Based on the synchronization result of the data operation log, the metadata is updated.
3. The method according to claim 1, wherein: Generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice includes: Persisting data operation information of the cluster in the distributed cluster according to the execution order of the data operation, wherein the data operation information includes the operation object, the operation content and the information of the storage management unit that performs the data operation; According to the data operation information of the cluster, a data operation log of the primary storage management unit in the synchronization time slice is obtained, wherein the data operation log is sorted according to the execution order of the data operation.
4. The method according to claim 1, wherein: Based on the master-slave state switching of the storage management unit group in the distributed cluster, the synchronization time slice is determined including: The master-slave state switching logic clock of the storage management unit group is obtained, and the synchronization time slice and the sequence number of the synchronization time slice are determined according to the master-slave state switching logic clock.
5. The method according to claim 1, wherein: The method further comprises: When the master-slave state of the storage management unit of the cluster in the distributed cluster switches, the cluster reports the master-slave state switching logical clock to the storage system, and the storage system sequentially increases the sequence number of the synchronization time slice according to the master-slave state switching logical clock and sends it to the distributed cluster.
6. The method according to claim 5, wherein: The method further comprises: The distributed cluster periodically obtains the sequence number of the synchronization time slice from the storage system.
7. The method according to claim 2, wherein: Synchronizing the data operation log to the slave storage management unit in the storage management unit group includes: According to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the sequence number of the synchronization time slice, the data operation log to be synchronized in the main storage management unit is determined, and the data operation log to be synchronized is synchronized to the slave storage management unit.
8. The method according to claim 7, wherein: In the case that the sequence numbers of the synchronization time slices of the primary storage management unit and the secondary storage management unit are the same, the data operation logs to be synchronized are determined and the data operation logs to be synchronized are synchronized.
9. The method according to claim 1, wherein: In the initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, and the storage management units in the slave clusters in the distributed cluster are all slave storage management units. The master-slave switching of the storage management unit group is managed by a preset controller.
10. The method according to claim 9, wherein: The preset controller sets the storage management unit with the identifier as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.
11. The method according to claim 9, wherein: In the case where there is data to be written into the storage management unit group, the primary storage management unit in the storage management unit group is determined by querying the preset controller, and the data is written into the primary storage management unit.
12. The method according to claim 1, wherein: The method further comprises: The data storage state of the storage management unit is used as a finite state machine. A leader node and at least one follower node are respectively selected in the main storage management unit and the slave storage management unit. The leader node synchronizes the copies of the finite state machine within the storage management unit group based on the data operation log.
13. A distributed cluster system, comprising a plurality of clusters and a storage system, wherein the plurality of clusters comprises a master cluster and at least one slave cluster, and the storage system is connected to the plurality of clusters respectively; The storage system is used to store the serial number of the synchronization time slice; the multiple clusters all perform data synchronization through the method described in any one of claims 1 to 12.
14. The distributed cluster system according to claim 13, wherein: The distributed cluster system further comprises a preset controller, which is used to control the master-slave state switching of the storage management unit and record the master-slave state of the storage management unit.
15. An electronic device, comprising: A processor, and a memory storing a program, wherein the program comprises instructions which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 12.
16. A non-transitory machine-readable medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.
Citation Information
Patent Citations
Big data storage and access system and method based on distributed file system
CN104320401A
Data processing method and device based on database cluster and electronic equipment
CN112612853A
Metadata synchronization method, system and equipment for distributed storage system
CN115599747A
Data synchronization method of distributed cluster and related equipment thereof
CN117112690A
Data disaster recovery method, device and system
US20190095293A1
