Data synchronization methods, devices, electronic equipment, storage media, and software products

Abstract

This application discloses a data synchronization method, apparatus, electronic device, storage medium, and program product, relating to the field of centralized storage device technology. It includes determining the corresponding mirror pair management structure based on the structure identifier contained in the request when responding to a backup data access request, eliminating the need for complex configuration traversal. Simultaneously, based on the preset mirror pairing rules in the structure, the mirror relationship between mirror sub-regions with the same name on different nodes is clarified, enabling rapid location of the mirror target sub-region on the target node. This shortens the target mirror location time, reduces the resource consumption of the central processing unit (CPU) in the configuration query and relationship matching stages, and alleviates CPU overload problems during synchronous replication. Furthermore, it employs an end-to-end link between nodes to directly send data transmission commands to the target node, eliminating the need for data to be relayed through the cluster's shared network, thus enabling direct transmission of backup data to the mirror sub-region with the same name on the target node.

Description

Technical Field

[0001] This application relates to the field of centralized storage device technology, and more particularly to a data synchronization method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Node-level data replication technology is a key technology for ensuring data synchronization in clustered storage systems. Its core principle is to build a highly reliable data redundancy system by synchronizing data mirrors across multiple storage nodes, thereby addressing the risk of data loss due to the failure of a single node.

[0003] In related technologies, the main implementation methods for node-level data replication are synchronous replication and asynchronous replication. Among them, synchronous replication requires both the master and slave nodes to complete the processing before returning the result after the business initiates an input / output (IO) request. It can achieve replica takeover in seconds in case of failure, but because real-time mirroring consumes a lot of IO bandwidth and central processing unit (CPU) resources, it seriously affects the overall performance of the storage system. Summary of the Invention

[0004] This application provides a data synchronization method, apparatus, electronic device, storage medium, and program product to at least solve the problems of high IO bandwidth and CPU resource consumption and impact on system performance in traditional centralized storage systems during synchronous mirror copying.

[0005] This application provides a data synchronization method, comprising: responding to an access request for backup data, determining a mirror pair management structure based on a structure identifier contained in the access request; determining a mirror sub-region with the same name for a target node based on mirror pairing rules included in the mirror pair management structure; the mirror pairing rules are used to configure the mirror relationship between mirror sub-regions with the same name between different nodes; sending a data transmission instruction to a target node through a target link to control the target node to receive backup data in response to the data transmission instruction and store the backup data in the mirror sub-region with the same name for the target node; wherein, the target link is an end-to-end link between the target node and the target node.

[0006] This application also provides a data synchronization device, including:

[0007] The determination module is used to respond to access requests for backup data, determine the mirror pair management structure based on the structure identifier contained in the access request, and determine the same-named mirror sub-region of the target node based on the mirror pair management structure. The mirror pairing rules are used to configure the mirror relationship between the same-named mirror sub-regions of different nodes.

[0008] The sending module is used to send data transmission instructions to the target node through the target link, so as to control the target node to receive backup data in response to the data transmission instructions and store the backup data in the mirror sub-region with the same name of the target node; wherein, the target link is an end-to-end link between the target node and the target node.

[0009] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the above-described data synchronization method when executing the computer program.

[0010] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described data synchronization method.

[0011] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described data synchronization method.

[0012] When responding to backup data access requests, this application determines the corresponding mirror pair management structure based on the structure identifier included in the request, eliminating the need for complex configuration traversal. Simultaneously, based on the pre-defined mirror pairing rules within the structure, it clarifies the mirror relationships between mirror sub-regions with the same name on different nodes, quickly locating the mirror sub-region with the same name on the target node. This shortens the target mirror location time, reduces CPU resource consumption during configuration queries and relationship matching, and alleviates CPU overload issues during synchronous replication. By employing end-to-end links between nodes to directly send data transmission commands to the target node, data does not need to be relayed through the cluster's shared network, enabling direct transmission of backup data to the mirror sub-region with the same name on the target node. Compared to the potential multi-node data forwarding and shared bandwidth contention in synchronous replication, end-to-end links reduce IO data transmission latency and bandwidth consumption, avoiding the impact of bandwidth congestion on the overall IO performance of the storage system and compensating for the significant IO performance degradation of synchronous replication. Attached Figure Description

[0013] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A schematic diagram of the specific hardware architecture on which the execution of a data synchronization method provided in this application depends;

[0015] Figure 2 A flowchart illustrating a data synchronization method provided for the implementation of this application;

[0016] Figure 3 A diagram illustrating the layered architecture of the cluster control terminal, client components, and data persistence components, as well as the collaborative relationships between these components;

[0017] Figure 4A This diagram illustrates the process of initial deployment of the cluster from the first node to the joining of the second node and the completion of mirror pair optimization. Figure 1 ;

[0018] Figure 4B This diagram illustrates the process of initial deployment of the cluster from the first node to the joining of the second node and the completion of mirror pair optimization. Figure 2 ;

[0019] Figure 5A This diagram illustrates the process of transitioning the cluster composition from a temporary three-replica to an optimal two-replica after the addition of a third node. Figure 1 ;

[0020] Figure 5B This diagram illustrates the process of transitioning the cluster composition from a temporary three-replica to an optimal two-replica after the addition of a third node. Figure 2 ;

[0021] Figure 6A This diagram illustrates the process of adding the fourth node to the cluster's steady-state mirror image to complete the assembly. Figure 1 ;

[0022] Figure 6B This diagram illustrates the process of adding the fourth node to the cluster's steady-state mirror image to complete the assembly. Figure 2 ;

[0023] Figure 7 This is a schematic diagram of the structure of a data synchronization device provided in an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0026] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0027] To more clearly illustrate the embodiments of this application, the technical terms used in the embodiments will be briefly introduced below:

[0028] A Data Persistence Component (DPC) is a software module or subsystem responsible for reliably transferring data from volatile storage to non-volatile storage.

[0029] The Memory Manager Component (MMC) is a core module in an operating system or virtualization platform responsible for allocating, reclaiming, and protecting memory resources.

[0030] The IO buffer reference manager (IOBM) is a component responsible for managing the mapping relationship between data blocks in the input / output buffer and the physical location of the storage device.

[0031] Mirror Address Management (MAM) is a component that maintains the physical location of data replicas in a distributed storage or high-availability system.

[0032] Mirror Couple Management (MCPM) is a mechanism for managing the lifecycle of storage entities (such as disks, volumes, or nodes) that are copies of each other.

[0033] The Cluster Management side (CMS) is the core component in a distributed system responsible for coordinating node status, resource allocation, and fault handling.

[0034] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The specific application environment architecture or specific hardware architecture on which the execution of the data synchronization method depends is described here.

[0036] like Figure 1 The diagram shows a specific hardware architecture upon which the data synchronization method depends.

[0037] The core hardware architecture consists of multiple cluster nodes with independent computing and storage capabilities. Each node must be equipped with: a central processing unit (CPU) for executing the core logic of the data persistence components, such as parsing access requests, matching structure identifiers, and invoking mirroring pairing rules; a memory module with dedicated memory partitions for deploying memory management components and storing various management structures, ensuring low latency for structure identifier queries and rule readings; local storage media, including solid-state drives (SSDs) or hard disk drives (HDDs), with independent physical partitions for storing received backup data in mirrored sub-regions, and these partitions must support pre-planning based on sub-region capacity to adapt to the storage resource allocation logic in the mirroring pairing rules; and CPU and memory to support dynamic calculation of mirroring pairing rules (such as primary / backup node allocation and load balancing logic) and real-time adjustment of domain configurations.

[0038] The cluster control node hardware requires at least one dedicated (or distributed) cluster control node to serve as the hardware carrier for the cluster control end. The hardware configuration must meet global management requirements. Redundant network interfaces are used to ensure stable communication with all cluster nodes and to issue control commands such as mirror pairing rules and synchronization structure identifiers, preventing method execution from stalling due to control link interruptions.

[0039] Node interconnect hardware is used to ensure low latency and high reliability of data transmission. Each cluster node must be equipped with a high-speed network interface card (NIC) that supports remote direct memory access technology. Direct, end-to-end links between nodes are established via dedicated fiber optic cables or high-speed network cables, rather than relying on shared Ethernet.

[0040] When a backup data access request is received, the node calls the memory management component to locate the mirror pair management structure. After determining the target node based on the mirror pairing rules in the mirror pair management structure, it sends a data transmission command through the network card and the direct link. After the target node receives the command, the storage controller writes the backup data into the mirror sub-region with the same name, and finally completes the data synchronization.

[0041] The embodiments of this application provide a data synchronization method, and the method is described in detail below in conjunction with the execution flow of the data synchronization method.

[0042] like Figure 2 As shown, Figure 2 A flowchart illustrating a data synchronization method provided for implementation of this application, the method comprising:

[0043] S201. In response to an access request for backup data, determine the mirror pair management structure based on the structure identifier contained in the access request.

[0044] Backup data refers to the data that the current node needs to synchronize to the target node (i.e., the slave / standby node). Access requests include the context ID of the mirror pair management structure, the mirror address offset, and the backup data size. The mirror pair management structure is obtained through the data persistence component (DPC) during node initialization in the cluster creation phase.

[0045] After receiving a request to access backup data, the current node responds to the request by determining the mirror pair management structure of the structure identifier index based on the structure identifier contained therein, without the need for complex configuration traversal.

[0046] S202. Based on the mirror pairing rules included in the mirror pair management structure, determine the same-named mirror sub-region of the target node.

[0047] Among them, the mirror pairing rule is used to configure the mirror relationship between mirrored sub-regions with the same name between different nodes. It clarifies the mirror relationship between mirrored sub-regions with the same name between different nodes and can quickly locate the mirrored sub-region with the same name of the target node.

[0048] The mirror pairing rules are retrieved from the mirror pair management structure. Then, based on the correspondence between the mirror sub-regions of the current node and the mirror sub-regions of the target node indicated by the mirror pairing rules, it is determined which mirror sub-region of which node the backup data needs to be synchronized to. The mirror pairing rules implicitly assume that the current node has a master-slave relationship with at least one other node (such as the target node), and the backup data is stored in a certain mirror sub-region of the current node. In this case, the backup data needs to be synchronized to a mirror sub-region with the same name as this mirror sub-region on the target node.

[0049] S203. Send a data transmission command to the target node through the target link to control the target node to respond to the data transmission command, receive the backup data, and store the backup data in the mirror sub-region of the target node with the same name.

[0050] The target link is the end-to-end link between the current node and the target node. Compared to the multi-node data forwarding and shared bandwidth contention that may occur in synchronous replication, the end-to-end link can reduce the transmission latency and bandwidth consumption of IO data, avoid the impact of bandwidth congestion on the overall IO performance of the storage system, and make up for the large IO performance loss of synchronous replication.

[0051] The current node sends a data transmission command for synchronizing backup data to the target node via an end-to-end link. The target node, in response to the received command, first receives the backup data to be synchronized, and then stores it in a mirrored sub-region with the same name as the current node. By directly sending data transmission commands to the target node via the end-to-end link, data does not need to be relayed through the cluster's shared network, enabling direct transmission of backup data to the target node's mirrored sub-region.

[0052] In some embodiments, before performing step S201 (in response to a backup data access request, determining the mirror pair management structure based on the structure identifier contained in the access request), specifically during the cluster creation phase and node initialization process, the current node requests management structure resources from the memory management component MMC through the data persistence component, initializes the resource space of the management structure resources, and allocates corresponding structure identifiers (Context IDs) to each management structure. The requested management structure resources include at least one of the following: a cache index identifier management IOBM structure, a mirror address management MAM structure, a region management structure, and a mirror pair management MCPM structure.

[0053] MMC can allocate dedicated resource space and a unique structure identifier (Context ID) for at least one of the following structures, such as the cache index identifier management IOBM structure, the image address management MAM structure, the region management structure, and the image pair management MCPM structure, according to the actual needs of the nodes, such as the number of image sub-regions and the configuration of slave nodes, so as to avoid configuration conflicts caused by resource overlap between different structures.

[0054] In the above embodiment, during the node initialization phase of cluster creation, the current node requests and completes initialization of management structure resources from the memory management component MMC through the data persistence component DPC. This produces multi-dimensional key technical effects, laying a core foundation for subsequent cluster image pairing and data synchronization. First, this operation achieves standardized allocation and control of management structure resources. Through a unified ContextID, precise location and management of each structure are achieved, significantly reducing resource query costs during subsequent component interactions.

[0055] DPC pre-initializes the management structure, enabling it to handle subsequent node additions and domain configuration updates without requiring temporary resource requests. It can directly use ContextID to call IOBM to manage IO cache identifiers, leverage MAM to cache slave node addresses, and establish mirror pair relationships based on MCPM, improving component response speed and preventing image processing delays caused by resource request latency. Furthermore, the reserved resource space and standardized structure identifiers during initialization adapt to subsequent cluster expansion scenarios. Regardless of any domain adjustments or image reassemblies triggered by new nodes, DPC can quickly respond based on the pre-initialized management structure, ensuring the stability of core operations such as image data transmission, address lookup, and cache management. This mitigates the risk of data loss and service interruption due to uninitialized structures or insufficient resources at the underlying resource level.

[0056] Then, a buffer space can be requested from the memory management component MMC. This buffer space is used to store the cache index identifier and the mirror physical address. The cache index identifier is the IO cache index identifier (IOB). Each managed resource structure has a unique structure identifier, used to quickly locate and access the storage address of the corresponding managed structure. The unique identifier corresponding to each managed resource structure (such as IOBM, MAM, MCPM) is equivalent to establishing a memory address index for the structure. When DPC interacts with MMC later, there is no need to perform complex memory scans or configuration matching to query the structure's storage address; the target structure can be directly located using this identifier, significantly reducing the latency of resource calls between components.

[0057] MMC allocates independent buffer spaces for the IO cache index identifier (IOB) and the mirror physical address. This avoids conflicts between these two types of critical data and other memory resources, reducing the risk of IO cache identifier corruption and mirror address loss due to data overwriting or interference. Furthermore, the dedicated buffer can optimize data read / write speeds through memory partitioning. Compared to shared memory, IOB lookups and mirror physical address calls do not require traversing redundant memory areas, shortening data access time and laying an efficient data foundation for subsequent mirror data transfer and IO request responses. When processing mirror requests, DPC can quickly locate the corresponding management structure through the structure identifier and then retrieve the IOB and mirror address from the dedicated buffer. This improves data processing efficiency within a single node and provides stable underlying data support for subsequent inter-node data synchronization and mirror pair switching, further reducing the risk of service interruption due to data storage chaos or location delays.

[0058] Optionally, the data persistence component can request 1024 management structure resources from the memory management component (MMC) based on business needs. These 1024 structures are suitable for medium-to-large-scale storage operations, meeting requirements such as dividing into 1024 mirror sub-regions, building 1024 mirror pairs, or managing 1024 node cache indexes, avoiding resource shortages or redundancy. In scenarios with high-frequency I / O requests, each structure can independently carry management logic, such as corresponding to an I / O cache identifier or a set of mirror addresses, preventing resource contention and ensuring rapid business response.

[0059] Optionally, this application determines a baseline management structure from the management structure resources through a data persistence component. Specifically, the size of each management structure is calculated and compared, and the largest management structure is selected as the baseline management structure. Setting a uniform resource allocation threshold based on the largest structure can avoid uneven resource allocation among structures, simplify logic, and reduce management complexity. The largest structure can carry complex logic, and using it as a baseline ensures sufficient resources when expanding functionality, reduces DPC adaptation adjustments, and improves component interaction stability.

[0060] In some embodiments, before performing step S201 (in response to a backup data access request, determining the image pair management structure based on the structure identifier contained in the access request), specifically during the cluster creation phase and node initialization process, the current node divides the image area into multiple image sub-regions through the client component to allocate and reserve image storage space for business data. The information of the image sub-regions includes an IO buffer reference and the sub-region capacity. Then, the client component registers the information of the image sub-regions with the data persistence component (DPC).

[0061] For example, the four-part division method is used to divide the mirrored region into four mirrored sub-regions, such as Zone0, Zone1, Zone2, and Zone3.

[0062] It should be noted that each mirrored sub-region corresponds one-to-one with a domain in the Cluster Management Side (CMS). For example, the domains in the Cluster Management Side may include domain0, domain1, domain2, and domain3, and the aforementioned four mirrored sub-regions correspond to these four domains. The domains in the Cluster Management Side can determine node pairing rules and mirroring rules based on the mirrored sub-regions of each node. Node pairing rules are used to configure the master-slave relationship between different nodes, while mirroring rules are used to configure the mirroring relationship between mirrored sub-regions with the same name between master and slave nodes.

[0063] In the above embodiment, during the node initialization phase of cluster creation, the current node first divides the mirror region into multiple mirror sub-regions using a four-part partitioning method through the client component. Each mirror sub-region clearly defines the IO cache reference identifier and capacity. This not only reserves dedicated space for business operations, avoiding storage conflicts, but also accurately matches business needs and facilitates the data persistence component to quickly locate the IO cache, improving read and write efficiency. The client component synchronizes key information of the mirror sub-regions to the data persistence component, enabling it to understand the distribution of storage resources and directly access resources in the mirror sub-regions, shortening the response latency of mirror operations. Simultaneously, the data persistence component can pre-initialize the management structure, avoiding information gaps between components. The correspondence between the mirror sub-regions and the cluster control domain facilitates the cluster control terminal in formulating node-mirror pairing rules, ensuring that the rules effectively apply to the storage mirror sub-regions. When a new node joins, the configuration can be quickly updated based on this, ensuring accurate and efficient mirror reassembly and supporting the stable operation of the cluster mirror system.

[0064] In some embodiments, during the process of storing the information of the mirrored sub-region to the data persistence component, the cache index identifier management structure is obtained through the data persistence component, and then the cache index identifier management structure is divided into multiple sub-structures, and the information of the mirrored sub-region is stored in the corresponding sub-structures.

[0065] Each substructure corresponds to a domain in the cluster control terminal. Each substructure records information about the mirrored sub-region, and then allocates free buffer space to store the IO cache reference identifier based on its size, in order to uniformly manage the mirrored sub-regions registered by the clients.

[0066] For example, the cache index identifier management structure is divided into four substructures: IOBM0, IOBM1, IOBM2, and IOBM3. These four substructures correspond to the four domains domain0, domain1, domain2, and domain3 of the cluster control terminal, respectively.

[0067] Each IOBM substructure corresponds to a mirror pair management MCPM structure, and each mirror pair management MCPM structure is associated with three mirror address management MAM structures. The mirror address management MAM structure is used to store and manage the mirror physical addresses of the mirror sub-regions of the slave node, and to request free buffer space based on the size of the sub-region to store the mirror physical addresses.

[0068] like Figure 3 As shown, Figure 3This diagram illustrates the layered architecture and collaborative relationships of the Cluster Controller (CMS), Client component, and Data Persistence Component (DPC) in a clustered storage system. It showcases the association and interaction logic between the CMS, Client component, and Data Persistence Component (DPC) at the domain and zone levels.

[0069] Figure 3 The CMS manages four domains, from domain0 to domain3, which are logical units for dividing storage resources in the clustered storage system. The client side contains four mirrored sub-regions, from Zone0 to Zone3. Each Zone corresponds to a domain in the CMS and is used to hold the client's storage data; it is the basic logical area for data storage.

[0070] Within the data persistence component, the IO cache index identifier management structure has substructures 0 to 3, corresponding to Zones 0 to 3 respectively. These substructures manage the IO cache-related identifiers for each mirrored sub-region and serve as the entry point identifier management structure for data IO operations. The Mirror Pair Management MCPM structure also has four substructures, corresponding one-to-one with substructures 0 to 3. These manage the mirror pair relationships for each mirrored sub-region, such as determining the pairing of mirrored sub-regions between primary and backup nodes, and are the core structure for mirror relationship management. The Mirror Address Management MAM structure, from MAM0 to MAM2, is associated with each Mirror Pair Management MCPM structure and is used to cache the physical addresses of the mirrored sub-regions of the slave node. When the primary node needs to synchronize data with the slave node or perform mirroring operations, it first queries the slave node's address information through the Mirror Address Management MAM structure to ensure address reachability of mirrored data transmission.

[0071] Optionally, in the process of storing the information of the mirrored sub-region into the sub-structure, the information of the mirrored sub-region includes the cache index identifier and the sub-region capacity. First, the storage space size is calculated based on the cache index identifier and the sub-region capacity. Then, the information of the mirrored sub-region is stored into the buffer space of the sub-structure based on the storage space size.

[0072] The size of the buffer space used to store information of the mirror sub-region can be calculated according to the following formula (1): Buffer = A*N*M / P + B*M / P (1). In formula (1), M is the capacity of the sub-region, P is the page size, A is the size of the bytes occupied by each page address, N is the number of slave nodes corresponding to each node, and B is the size of the bytes occupied by the IO cache index identifier. All parameters in the formula are strongly related to the actual business scenario. The calculated buffer space can accurately carry the mirror physical address and IO cache index identifier without the need for subsequent dynamic adjustment of the space size, thus reducing the performance loss caused by frequent space adaptation operations of the sub-structure.

[0073] For example, assuming a sub-region capacity of 40GB, using 4KB pages as units, with each page address occupying 8 bytes, and each node corresponding to the address mapping of 3 potential slave nodes, the buffer space occupied by the mirrored physical address is calculated. Following the maximum rule of reserving one IOB per page, with each IOB occupying 4 bytes, the buffer space occupied by the IOB is calculated. The buffer space size is the sum of the above two parts: 40GB / 4KB * 8 * 3 + 40GB / 4KB * 4 = 270MB.

[0074] The above embodiments, when storing mirrored sub-region information into the substructure, use the formula Buffer=A*N*M / P+B*M / P, combined with parameters such as sub-region capacity and page size, to accurately calculate the buffer space. For example, under conditions such as a 40G sub-region and 4K pages, the calculated buffer size is 270M, avoiding space overflow and waste. This accurately calculated buffer prevents information truncation and misalignment, and also reserves adaptation space for cluster expansion and data synchronization, stably supporting subsequent operations and reducing the risk of business interruption due to storage issues.

[0075] In some embodiments, based on the application for a mirror address management structure, the data synchronization method provided in this application further includes: first querying the mirror physical address of the same-named mirror sub-region of the target node through the target link, obtaining the pre-approved mirror address management structure, and storing the mirror physical address of the same-named mirror sub-region of the target node into the mirror address management structure.

[0076] The above embodiments, based on the pre-application of a mirror address management structure, query the mirror physical address of the mirror sub-region with the same name on the target node through the target link and store it. Utilizing a dedicated structure to store the address eliminates the need for repeated queries when the master node performs data synchronization operations; it can directly call the structure to obtain the address, reducing link overhead and latency. For example, when the master node (node0) synchronizes Zone0 data with the backup node (node1), it reads the structure address, avoiding IO performance loss and improving response speed. Structured storage prevents address information corruption, and unified management ensures that valid addresses are called. When the target node's sub-region address changes, updating the structure ensures accurate data synchronization and avoids inconsistency risks.

[0077] When a node joins the cluster, or when mirror pair assembly, switching, or other operations are performed, refer to... Figure 3 The CMS first updates the configuration information such as the node composition of domain0 to domain3. The client's Zones 0 to 3 serve as data carriers, and their data mirroring logic is handled by the DPC. The DPC identifies the corresponding Zone's IO operations through IOB, then the MCPM determines the mirrored sub-region of the target slave node according to the mirroring pairing rules. Finally, using the slave node address cached in the MAM, data transmission and other mirroring-related operations are completed through inter-node links, ensuring reliable data mirroring and persistence across cluster nodes.

[0078] In some embodiments, after the mirror region is divided into multiple mirror sub-regions by the client component during the node initialization phase, the process further includes: responding to a domain pair configuration change event from the cluster control terminal via a data persistence component, refreshing the domain pair configuration information, and initializing the mirror pair management structure for each mirror sub-region, wherein the domain pair configuration change event from the cluster control terminal indicates that other nodes have joined the cluster. Then, it checks whether the mirror physical address of the mirror sub-region with the same name of the other node is stored in the mirror address management structure. If not, the current node queries the mirror physical address of the other node through the end-to-end link between the node and other nodes. The mirror physical address of the other node is then stored in the buffer space of the mirror address management structure, completing the mirror preparation.

[0079] The domain pair configuration information includes the node composition information of multiple domains. It should be noted that whenever a new node is added to the cluster, the cluster control system (CMS) will respond to the node addition event immediately and update the domain pair configuration information. For example, the node composition of domain0 will be updated from (0) to (0,1).

[0080] In the above embodiments, when other nodes join the cluster, the CMS triggers a domain pair configuration change event, and the DPC can immediately detect and refresh the domain pair configuration information, avoiding mirror logic errors caused by configuration lag. At the same time, the DPC initializes the corresponding mirror pair management structure for each mirror sub-region, providing a dedicated management carrier for the subsequent primary and backup node mirror relationship. The basic mirror structure can be built without waiting for other processes, shortening the preparation time of the mirror system after a node joins and improving the response efficiency of cluster expansion.

[0081] By querying and storing addresses via end-to-end links between nodes, the efficiency and reliability of obtaining mirrored physical addresses are enhanced. The DPC first checks if the physical address of a sub-region with the same name exists on another node in the mirrored address management structure. If it is missing, it queries directly via end-to-end links. Compared to querying via the cluster's shared network, end-to-end links avoid address query delays caused by network congestion and reduce the number of intermediate nodes involved in address transmission, lowering the risk of address loss or tampering. For example, when the DPC of node0 queries the physical address of Zone0 of node1, it directly obtains and stores it in the MAM's buffer space via a dedicated link, ensuring that a valid address can be quickly accessed during subsequent data synchronization, avoiding synchronization failures due to missing or incorrect addresses.

[0082] DPC not only initializes the image management structure but also ensures that the image address management structure fully stores the physical addresses of other nodes' images. This combination prevents image relationships from failing to be established due to uninitialized structures or missing addresses. Subsequent image construction can be completed simply by triggering data transfer commands, without requiring additional preparation steps, thus laying a complete technical foundation for rapid data redundancy among cluster nodes.

[0083] In scenarios where nodes are added to the cluster one by one, this application can dynamically update the domain pair configuration information, i.e., the node composition information, based on the current number of nodes in the cluster through the cluster control system (CMS). Changes in node composition information will trigger mirror pair reassembly, for example, changing from a single node to a two-node pair, and then to a temporary three-node pair. The ultimate goal is to allow the mirror pairs to gradually converge from a temporary transitional state to a load-balanced steady state.

[0084] For example, assuming the first node (node0) is added, the domain pair configuration information indicates that each domain is in a single-node state, with no node pairing rules, no mirror pairing rules, and the mirror relationship remains silent. When the second node (node1) is added, the domain pair configuration information indicates a two-node pairing state, such as (0,1), meaning node0 is the master node and node1 is the slave node. When the third node (node2) and the fourth node (node3) are added, the domain pair configuration information indicates a temporary three-replica state, such as (1,0,2) and (1,2,3), respectively. After the backup data synchronization is complete, the domain pair configuration information is adjusted to a two-node pairing state. The domain pair configuration information is then refreshed to ensure consistency with CMS updates.

[0085] The current node queries the mirror physical address of other nodes through the end-to-end link between the nodes. This is done by first querying the IO cache index identifier of other nodes through the end-to-end link between the nodes, and then determining the mirror physical address mapped by the IO cache index identifier.

[0086] For example, suppose a configuration change event for a domain pair indicates that another node (node1) has joined the cluster. The current node (node0) checks whether the mirror physical address of the mirror sub-region with the same name as the other node (node1) is stored in the mirror address management structure. If it is not stored, the current node (node0) actively queries the mirror physical address of the other node (node1) through the end-to-end link between the other node (node1) and the current node (node0).

[0087] Optionally, during the process of storing the mirrored physical addresses of other nodes into the buffer space of the mirrored address management structure, the management table identifier corresponding to the mirrored physical address of the other node is first determined based on the node identifier of the other node. Then, the mirrored physical addresses of the other nodes are stored into the buffer space corresponding to this management table identifier.

[0088] Specifically, the Management Table Identifier (MAM) corresponding to the mirrored physical address of other nodes is determined according to the following formula (2). tableid :

[0089] (2)

[0090] In the current node's node identifier (owner) nodeID ) is greater than other nodes (backup) nodeID When determining the node identifier of a node, the Management Table Identifier (MAM) corresponding to the mirrored physical address of other nodes is determined. tableid ) equals other nodes (backup) nodeID The node identifier of the current node (owner). nodeID ) smaller than other nodes (backup) nodeIDWhen determining the node identifier of a node, the Management Table Identifier (MAM) corresponding to the mirrored physical address of other nodes is determined. tableid ) equals other nodes (backup) nodeID Subtract a preset value from the node identifier, such as MAM. tableid =backup nodeID -1. Then store the mirror physical addresses of other nodes into the buffer space corresponding to this management table identifier.

[0091] In the above embodiments, formula (2) directly maps a unique MAM based on the size relationship between the current node identifier and other node identifiers. tableid This calculation logic does not require traversing the buffer; it can be located by substituting the formula in reverse during the call. For example, when 0 < 1, MAM... tableid =0, and subsequent calls to calculate "1-1=0" can find the address, greatly improving the real-time performance of calls in multi-node scenarios. Formula (2) assigns a unique and fixed MAM to the target node address. tableid To avoid address conflicts, taking a 4-node cluster as an example, owner_nodeID=1 stores different backups. nodeID When using addresses, each corresponds to a different MAM. tableid This facilitates partition management, reduces the risk of address overlay, and simplifies the operation and maintenance troubleshooting process. Formula (2) only depends on the relationship between node identifier size, and does not depend on the number of nodes or business scenarios, supporting cluster expansion.

[0092] In some embodiments, when other nodes are added to the cluster, the current node synchronizes backup data to the newly added nodes. The current node synchronizes backup data to other nodes through end-to-end links, so that the other nodes store the backup data in a mirrored sub-region with the same name on the other nodes.

[0093] For example, when the current node (node0) synchronizes the backup data of Zone1 to the newly added node (node2), the end-to-end link can directly transmit the data from Zone1 of node0 to Zone1 of node2 without passing through other nodes. Compared with the transmission over a shared network, this can significantly shorten the data transmission time. Especially in scenarios with a large amount of backup data (such as GB level), data synchronization can be completed quickly, allowing the newly added node to have data service capabilities more quickly.

[0094] When new nodes are added to the cluster, the current node synchronizes and backs up data to them via an end-to-end inter-node link. This ensures that the new node stores its data in a mirrored sub-region with the same name, improving data synchronization efficiency and enhancing data reliability and consistency. Specifically, the current node and the new node transmit data directly via a dedicated end-to-end link, without needing to go through the cluster's shared network. This avoids bandwidth contention and congestion issues associated with multi-node data transmission in the shared network, reducing data synchronization latency.

[0095] New nodes store synchronized backup data in a mirrored sub-region with the same name as the current node (e.g., Zone0 of node0 corresponds to Zone0 of node2), ensuring complete data consistency between the primary and backup nodes under the same mirror logic, thus creating data redundancy. In the event of a failure of the current node, the new node can quickly switch to providing services as the primary node using the complete backup data stored in the mirrored sub-region, avoiding data loss. Simultaneously, this synchronized mirrored sub-region model ensures that the data logic of each node remains consistent during subsequent mirror pair switching and load balancing adjustments, preventing data corruption due to mismatched sub-region data and guaranteeing cluster data consistency from a data storage perspective.

[0096] During the process of adding a new node, while the existing node synchronizes data via an end-to-end link, it can still handle business I / O requests normally without pausing services to wait for data synchronization to complete. Simultaneously, during data synchronization, the new node can gradually build a complete mirrored sub-region of data. Once synchronization is complete, it can seamlessly integrate into the cluster mirroring system, assuming the role of a backup or master node. For example, when node3 is added to the cluster, while node2 is synchronizing data to node3, node2 can still provide data read and write services for the business. After node3 completes synchronization, it can directly participate in domain pairing and mirror pair formation, avoiding business interruptions caused by node expansion. This ensures that the cluster maintains stable service capabilities throughout the expansion process, providing reliable technical support for the dynamic growth of the cluster size.

[0097] In some embodiments, if incremental data is received when backup data has not been synchronized to other nodes, the incremental data is synchronized to the mirror sub-region with the same name on other nodes.

[0098] For example, assuming three additional nodes are added to the cluster, if the backup data is not fully synchronized to these three nodes, the incremental data received by the current node will be synchronized to at least two nodes to avoid data incompleteness due to node anomalies. This strategy of synchronizing to at least two nodes strengthens the multi-redundancy against node anomaly risks. Compared to synchronizing to only one new node, synchronizing to at least two nodes provides multi-replica protection. If one of the nodes receiving incremental data experiences anomalies such as going offline or hardware failure, the other node still holds complete incremental data and can continue to participate in subsequent backup synchronization and business services, avoiding incremental data loss due to a single node anomaly. For instance, if the current node synchronizes incremental data to the same-named zones of the newly added nodes node2 and node3, and node2 suddenly goes offline, the same-named zone of node3 still contains incremental data. Subsequent backup synchronization can continue based on the incremental data from node3 without retransmitting the incremental data, significantly reducing the risk of data incompleteness caused by node anomalies and improving the cluster's fault tolerance.

[0099] In the above embodiments, when backup data has not been synchronized to other nodes, the current node synchronizes the received incremental data to the mirrored sub-regions of other nodes with the same name, achieving real-time storage of incremental data and avoiding data loss during synchronization intervals. The synchronized connection between backup data and incremental data reduces subsequent retransmission costs.

[0100] When backup data has not been fully synchronized to new nodes, relying solely on subsequent incremental data transmissions may result in data loss due to current node failures, link interruptions, or other issues. However, synchronizing incremental data in real-time to mirrored sub-regions of other nodes with the same name effectively creates an instant redundant copy of the incremental data. For example, when the current node synchronizes backup data to three new nodes, if only 50% of the backup synchronization is completed, the received incremental data is synchronized to at least two of the new nodes' corresponding zones. Even if the current node experiences a sudden failure, the complete incremental data will already be stored in the corresponding sub-regions of these two nodes, eliminating the need to rely on the failed node for recovery.

[0101] After incremental data is synchronized to the mirrored sub-region with the same name in real time, once the backup data has completed basic synchronization, new nodes do not need to retrieve incremental data from the current node again. They only need to integrate the stored incremental data with the basic backup data to quickly form complete mirrored sub-region data. For example, while the current node synchronizes basic backup data to node2, it also synchronizes incremental data to node2's Zone1 in real time. After the basic backup synchronization is complete, node2 only needs to merge the basic data and incremental data without requesting additional incremental data. This shortens the total time for new nodes to integrate into the cluster mirror system, reduces the repeated transmission pressure on the current node, ensures efficient connection of the synchronization process, and does not affect its normal business processing capabilities.

[0102] In some embodiments, after backup data is synchronized to other nodes, in response to a mirror pair switchover command initiated by the cluster control terminal, a mirror pair switchover operation is performed on the mirror sub-regions of the same name on other nodes. Establishing a stable state for the cluster mirror system and clarifying the primary / backup collaboration relationship can ensure seamless continuity of business services and avoid the risk of switchover interruptions.

[0103] After backup data synchronization is complete, the mirrored sub-regions of each node have a complete data foundation. At this point, performing a mirror pair switch essentially means that the CMS, based on preset rules (such as node load and failure risk), clarifies the primary and backup roles for the mirrored sub-regions. For example, the current node's Zone0 is the primary, and the newly added node's Zone0 is the backup. For instance, after a CMS command triggers a switch between the Zone0-Zone3 mirrored pairs of node0 and node1, the primary-slave relationship of each sub-region with the same name is fixed. Subsequent data reads and writes only need to follow the logic of primary node writing and slave node synchronization, avoiding data write conflicts or synchronization disorders caused by ambiguous primary and backup roles. At the same time, after the switch is completed, the cluster mirrored pairs officially enter a steady state, providing a clear collaborative framework for subsequent node failure switching and data redundancy backup, and solidifying the foundation for cluster data reliability.

[0104] Because the failover operation is performed only after backup data has been synchronized, and only the primary and backup roles of the same-named mirror sub-regions are logically adjusted, there is no need to pause services at the business level. For example, if the current node is processing business I / O requests through Zone1, and the backup data is synchronized to Zone1 of node2, when the failover operation is performed to transfer the primary role to node2, only the primary and backup address identifiers in the mirror address management structure need to be updated. Business requests can seamlessly switch from Zone1 of the current node to Zone1 of node2, without any noticeable delay or interruption for the user. This logic of switching only after data is ready completely avoids failover failures due to incomplete data or business interruptions caused by the failover, ensuring the continuity and stability of cluster services.

[0105] Furthermore, by switching mirror pairs, the primary role of a high-load node is transferred to a mirrored sub-region with the same name on a low-load node, achieving balanced resource utilization. For example, if node0 experiences excessive IO load due to consistently holding the primary role of multiple sub-regions, after backup data is synchronized to node1 and node2, CMS commands switch the primary role of node0's Zone2 and Zone3 to the corresponding sub-regions of node1 and node2. This reduces the load on node0 while fully utilizing the idle storage resources of node1 and node2. This load-based switching adjustment avoids overloading a single node due to too many primary roles and prevents other nodes from having idle resources, improving the resource utilization efficiency of the entire cluster and providing hardware-level support for long-term stable service.

[0106] From the perspective of the current node, the current node triggers corresponding operations through the Data Persistence Component (DPC) when domain pairs change in each round. First, for each mirrored sub-region, the IO Cache Index Identifier Management (IOBM) structure is initialized to build the basic mirroring framework. The domain pair configuration information in its own storage is refreshed in real time to ensure consistency with updates on the cluster control system (CMS). The physical address of the mirrored sub-region with the same name as the slave node is cached through the Mirror Address Management (MAM) structure. Specifically, if the master node ID > slave node ID, then the address is cached through the MAM. tableid =backup nodeID If the master node ID is less than the slave node ID, then MAM... tableid =backup nodeID -1. If the mirror address management MAM structure does not cache the mirror physical address of the mirror sub-region with the same name on the slave node, the slave node will be actively queried through the end-to-end link between nodes to fill in the address and complete the mirror preparation. When a new node is added, the current node acts as the master node to synchronize and back up data to the mirror sub-region with the same name on the new node. Before the data synchronization is complete, the current node, as the master node, needs to synchronize incremental data to the slave node to avoid data incompleteness due to node abnormalities. After all mirror pairs are ready and the data synchronization is completed, the CMS initiates the mirror pair switchover to achieve load balancing. For example, after node1 is added, domain1 / 2 is adjusted to (1,0), and after node3 is added, the optimal combination of (0,1), (1,2), (2,3), and (3,0) is finally formed. It should be noted that the switchover adjustment is only performed on the domains where the master and backup roles are swapped (such as domain1 / 2). The mirror pairs of other domains provide services normally to reduce the risk of business interruption.

[0107] For example, suppose four new nodes are added to the cluster. After node1 joins the cluster, its domain pair configuration information is (1,0). After the backup data is synchronized to the new node, the cluster control terminal initiates a mirror pair switch command. The current node responds to the command and performs a mirror pair switch operation, forming a cyclic optimal combination of (0,1), (1,2), (2,3), and (3,0).

[0108] For example, after the first node (node0) joins the cluster, the cluster control terminal (CMS) updates the node composition of domains 0 to 3 to a single node (0), meaning each domain contains only node0. The data persistence component (DPC) of node0 responds to domain changes by not only initializing the MCPM structure for each mirror sub-region, but also reserving and requesting MCPM structure resources for the corresponding mirror sub-region (Zone), laying the groundwork for subsequent node additions. At this time, the mirror relationships remain silent, waiting for the next node to join.

[0109] The cluster control system (CMS) responds to the second node (node1) joining event, synchronously updating the node pairing from domain0 to domain3 to a (0,1) pairing state, indicating that node0 is the master node and node1 is the slave node. The data persistence components (DPCs) of node0 and node1 detect the domain configuration change, immediately refresh their stored domain pair configuration information, and manage the IOBM structure for each mirror sub-region's associated IO cache index, completing the initialization of the corresponding mirror pair management MCPM structure and establishing the mirror pair relationship.

[0110] As the master node, node0 first checks whether the associated Mirror Address Management (MAM) structure has cached the physical address of the corresponding mirror sub-region of node1. If not cached, it queries the mirror physical address mapped to node1's IOB through the high-speed data link between nodes, and then processes it according to the rules in the owner node. nodeID >backup nodeID Save to MAM tableid =backup nodeID Conversely, store in MAM tableid =backup nodeID -1 is stored in the mirror address to manage the MAM structure buffer space.

[0111] Basic operations include, but are not limited to: configuration refresh, initialization of the MCPM structure for image pair management, and address caching. After completion, in order to achieve load balancing, the cluster control terminal CMS will optimize and adjust the domain pairs to a combination mode of (0,1), (1,0), (1,0), and (0,1). Here, only the primary and backup roles of domain1 and domain2 are swapped, with node1 as the primary node and node0 as the backup node.

[0112] The Data Persistence Component (DPC) for nodes 0 and 1 only performs switching adjustments for the mirror pairs of domain 1 and domain 2. Node 1, as the master node, repeatedly checks the mirror address to manage the MAM structure cache, then queries the node 0 address, and then populates the mirror address to manage the MAM structure. The mirror pairs for the other domains (0, 3) maintain their original state and continue to provide services. The cluster control system (CMS) waits for the DPC to complete all responses before performing subsequent operations to avoid configuration confusion.

[0113] When the third node (node2) is added, the cluster control terminal CMS temporarily updates the node composition of domain0 to domain3 to (0,1), (1,0,2), (1,0,2), (0,1,2). Here, only domain1, 2, and 3 enter the three-replica state, while domain0 maintains two nodes. In domain1 and 2, node1 is the primary node and node0 and node2 are the backup nodes. In domain3, node0 is the primary node and node1 and node2 are the backup nodes.

[0114] The Data Persistence Component (DPC) responds to domain changes and reassembles the mirror pair. The new master node (such as node1 and node0) needs to check the physical addresses of the mirror sub-regions of the two slave nodes (node0 and node2, node1 and node2) at the same time. If the mirror address management MAM structure is not cached, the physical address mapping query process is triggered until all slave node addresses are cached, and the mirror pair reassembly is not considered to be officially completed.

[0115] The master node synchronizes data to the mirror sub-region of the new slave node node2. If data synchronization is not complete, the mirror pair provides services in a three-replica configuration. When incremental data arrives, the master node synchronizes it to both slave nodes simultaneously, such as node1 synchronizing to node0 and node2, to prevent data incompleteness caused by node2 leaving abnormally.

[0116] After all mirror data is synchronized, the cluster control terminal CMS will adjust the domain pairs to the optimal dual-replica combination (0,1), (1,2), (0,2), (2,0), and perform mirror pair switching according to the established rules of primary and backup address verification and mirror address management MAM structure filling, thus completing the transition from temporary three replicas to steady-state dual replicas.

[0117] When the fourth node (node3) is added, the cluster control terminal CMS temporarily updates the node composition of domain1 to domain3 to (1,2,3), (0,2,3), and (2,0,3). domain0 maintains the original two nodes. The new master node (such as node1, node0, and node2) reuses the three-replica processing flow when node2 was added: check the mirror address management MAM structure address cache of the two slave nodes. If it is not cached, query and fill it. During data synchronization, the three-replica form ensures data reliability.

[0118] After the mirror data synchronization and reassembly are completed, the cluster control terminal CMS will adjust the domain pairs to the optimal combination of (0,1), (1,2), (2,3), (3,0) in a circular manner, and perform mirror pair switching according to the previous rules. After the switching is completed, the stable mirror pair of the cluster is completed, and each domain runs in a dual-node master-slave mode, taking into account both load balancing and data reliability.

[0119] The following will provide a detailed explanation with reference to illustrations, such as... Figure 4A and Figure 4B The diagram illustrates the process of the cluster being initially deployed from the first node (node0) to the second node (node1) joining and completing image pair optimization. Figure 3 The process involves the CMS-led update of domain node composition, the DPC-driven initialization and address management of the image structure, and finally, the load balancing-driven image pair switching. This can be explained in two stages:

[0120] The first stage is the initial state of node0 as a single node. At this time, the data persistence component DPC of node0 reflects its core actions in response to domain changes through the silent state label: initialize the MCPM structure for managing mirror pairs for each domain's mirror sub-region, but because there are no slave nodes, all mirror relationships are in a silent state waiting for subsequent nodes, with no data transmission link or mirror pairing identifier.

[0121] The second phase is the domain update and mirror pair construction phase after node1 joins. First, the interactive arrows in the cluster control terminal (CMS) demonstrate that the node composition of domain0 to domain3 is synchronously adjusted to a (0,1) pairing, identifying node0 as the master node and node1 as the slave node, distinguishing the roles of the two nodes in the domain association. Next, the connection between DPC and IOBM / MCPM shows the process of MCPM initialization for each mirror sub-region after the DPC of node0 and node1 synchronously refreshes the domain configuration. At the same time, the core operation of node0 as the master node is highlighted: through bidirectional arrows pointing to node1, combined with the buffer space identifier of the mirror address management MAM structure, it shows that it first checks the address cache, and if it is missing, it queries the IOB physical address of node1 and then stores it to the MAM according to the rules. Finally, the switching command arrows and the composition changes of domain1 and domain2 indicate the load balancing optimization actions. At this point, only the master and slave roles of domain1 and domain2 are swapped, with node1 as the master node and node0 as the slave node. The DPC of the corresponding region marked node1 repeats the operation of "first check the address cache, if it is missing, query the IOB physical address of node1, and then store it to MAM according to the rules". Meanwhile, domain0 and domain3 remain (0,1) paired and marked as normal service, clearly distinguishing the mirror pair that needs to be switched and adjusted from the mirror pair that continues to provide service.

[0122] When node2 joins the cluster, the node composition of domains 1 through 3 will be temporarily updated to (1,0,2), (1,0,2), and (0,1,2), resulting in a brief three-replica state for these domains, with one master node and two slave nodes. When a domain pair changes, DPC responds to the domain change and reassembles the mirror pair. As the new master node, it must first check whether the physical addresses of the mirror sub-regions of the two slave nodes are cached. If not, it triggers a physical address mapping query process. The reassembly of the relevant mirror pair is only considered complete after the address caching is completed. The master node synchronizes data to the mirror sub-regions of the newly joined slave node (node2). If the mirror data of the corresponding mirror sub-region has not yet been synchronized, the mirror pair will provide services in a three-replica configuration. That is, when incremental data arrives, the master node must simultaneously synchronize the data to the two backup mirror regions to avoid data incompleteness caused by abnormal node departure during data synchronization. Once all mirror data is synchronized, a mirror pair switching operation is performed. The CMS adjusts the domain pairs to the optimal combination (0,1), (1,2), (0,2), (2,0), and performs the mirror pair switching according to the operation rules in step two.

[0123] like Figure 5A and 5B This illustrates the process of the domain node composition transitioning from a temporary three-replica structure to an optimal two-replica structure after the addition of the third node (node2) to the cluster. This can be explained in two stages:

[0124] The first phase involves node2 initially entering a temporary three-replica state. The nodes from the three domains are temporarily updated to the form (1,0,2), (1,0,2), and (0,1,2). Within each domain, roles are clearly defined by master and slave node labels. For example, in domains 1 and 2, node1 is the master node, and node0 and node2 are slave nodes; in domain 3, node0 is the master node, and node1 and node2 are slave nodes, all marked with a temporary three-replica status identifier. The new master node1, for the two slave nodes (e.g., node0 and node2), first checks if the physical addresses of their mirror sub-regions are cached in the MAM using the Data Persistence Component (DPC). If not cached, it triggers the address lookup process by pointing to the slave node through an end-to-end link between nodes. Once the address is cached, it marks the mirror pair reassembly as complete. Furthermore, the master node points to node2 to synchronize mirror data. If data synchronization is not complete, incremental data is synchronized to the two slave nodes to prevent data loss.

[0125] The second phase is the optimal combination switchover phase after mirror data synchronization. The cluster control system (CMS) issues a switchover command. Domain 0 retains its original configuration, while domains 1 through 3 are updated to the optimal dual-node combination of (0,1), (1,2), (0,2), and (2,0), respectively. Within each domain, the primary and backup roles are re-marked, such as domain 1 being the primary node (node0) and the secondary node (node1); and domain 2 being the primary node (node1) and the secondary node (node2). The temporary three-replica identifier is removed, and it is changed to a stable two-replica configuration. Simultaneously, the mirror pair switchover operation follows the established rules from when the second node was added, requiring only the domains whose composition has changed to complete the switchover; domains that remain unchanged are still marked for continued service.

[0126] When node3 joins the cluster, the node composition of domain1 to domain3 will be temporarily updated to (1,2,3), (0,2,3), and (2,0,3), resulting in a temporary three-replica state. Following the procedure for handling temporary three-replicas, the mirror pairs at the time of node joining are reassembled. After completion, a mirror pair switching operation is performed. CMS adjusts the domains to the optimal combination (0,1), (1,2), (2,3), and (3,0), and performs the mirror pair switching according to the rules in step two. After the switching is complete, the stable mirror pair formation of the cluster is finished.

[0127] like Figure 6A and Figure 6B The diagram illustrates the process of the fourth node (node3) joining the cluster to complete the assembly of the steady-state image. This can be explained in two stages:

[0128] The first phase involves the initial temporary three-replica state and mirror pair reassembly phase upon node3's addition. The nodes node0, node1, node2, and node3 of the three domains (domain1, domain2, and domain3) are temporarily updated to the forms (1,2,3), (0,2,3), and (2,0,3). Within each domain, the roles are clearly defined by master and slave node labels. For example, domain1 uses node1 as the master, with node2 and node3 as backups; domain2 uses node0 as the master, with node2 and node3 as backups; and domain3 uses node2 as the master, with node0 and node3 as backups. All nodes are uniformly labeled with a temporary three-replica state identifier. Simultaneously, referencing the three-replica processing logic from when the third node (node2) was added, the new master node1 checks whether the physical addresses of the mirror sub-regions are cached in the MAM for the two slave nodes (e.g., node2 and node3). If not cached, it triggers a physical address mapping query process by pointing to the corresponding slave node through an end-to-end link between nodes. Once all slave node addresses are cached, the mirror pair reassembly is marked as complete. The core task of this phase is to reuse mature processes to complete the basic mirror preparation after the new node's addition.

[0129] The second phase involves mirror pair switching and achieving stable cluster status. The CMS (Cluster Control System) issues a switchover command, adjusting the node combinations from domain0 to domain3 to a cyclical optimal dual-replica configuration of (0,1), (1,2), (2,3), and (3,0). Within each domain, the master and slave node roles are redefined: domain0 is master node0 and slave node1; domain1 is master node1 and slave node2; domain2 is master node2 and slave node3; and domain3 is master node3 and slave node0. Temporary triple-replica identifiers are removed and replaced with stable dual-replica status labels. Simultaneously, the switchover operation follows the established logic from when the second node was added, such as the master node checking the MAM cache and querying the slave node's replica address if it's missing. The switchover only applies to domains with changed composition, without interrupting services in other domains. Finally, the stable mirror pair is established, indicating that the cluster has built a stable mirror system that balances load balancing and reliability.

[0130] In summary, the data synchronization method provided in this application solves the technical problems of high IO bandwidth and CPU resource consumption and impacted system performance during synchronous replication by accurately locating mirror sub-regions and optimizing data transmission links. In comparison, one source of performance degradation in synchronous replication is the need for the master node to traverse multiple configuration information sets to determine the mirror target, consuming additional CPU resources. In contrast, this method allows the master node to directly determine the corresponding mirror pair management structure based on the structure identifier contained in the request when responding to backup data IO requests, eliminating the need for complex configuration traversal. Furthermore, based on the preset mirror pairing rules within the structure, the mirror relationship between mirror sub-regions with the same name on different nodes is clearly defined, quickly locating the mirror sub-region with the same name on the target node. This significantly shortens the mirror target location time, reduces CPU resource consumption during configuration query and relationship matching, and alleviates CPU overload issues during synchronous replication. Another core issue with synchronous replication is that real-time mirroring relies on the cluster's shared network, which can easily lead to network bandwidth congestion. This method, on the other hand, uses end-to-end links between nodes to send data transmission commands directly to the target node. Data does not need to be relayed through the cluster's shared network, enabling direct transmission of backup data to the target node's mirrored sub-region. Compared to the potential multi-node data forwarding and shared bandwidth contention in synchronous replication, end-to-end links can reduce IO data transmission latency and bandwidth consumption, avoiding the impact of bandwidth congestion on the overall IO performance of the storage system and compensating for the high IO performance loss of synchronous replication.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0132] like Figure 7 As shown, embodiments of this application also provide a data synchronization device, which includes:

[0133] The determination module 701 is used to respond to the access request for backup data, determine the mirror pair management structure according to the structure identifier contained in the access request, and determine the same-named mirror sub-region of the target node based on the mirror pairing rules included in the mirror pair management structure; the mirror pairing rules are used to configure the mirror relationship between the same-named mirror sub-regions of different nodes.

[0134] The sending module 702 is used to send a data transmission instruction to the target node through the target link, so as to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in the mirror sub-area of ​​the same name of the target node; wherein, the target link is an end-to-end link between the target node and the target node.

[0135] As an optional implementation provided in this application embodiment, the determining module 701, in response to the backup data access request, before determining the mirror pair management structure based on the structure identifier contained in the access request, is further configured to: during the node initialization phase, request management structure resources from the memory management component through the data persistence component; wherein, the management structure resources include at least one of the following: cache index identifier management structure, mirror address management structure, region management structure, and mirror pair management structure.

[0136] As an optional implementation provided in this application, the determining module 701 is further configured to, during the node initialization phase, divide the mirror region into multiple mirror sub-regions through the client component, the information of the mirror sub-regions including the cache index identifier and the sub-region capacity size; and register the information of the mirror sub-regions to the data persistence component.

[0137] As an optional implementation provided in this application, the determining module 701 is specifically used for: obtaining the cache index identifier management structure through the data persistence component; dividing the cache index identifier management structure into multiple substructures; and storing the information of the mirrored sub-regions into the corresponding substructures.

[0138] As an optional implementation provided in this application, the determining module 701 is specifically used to: calculate the size of the storage space to be stored based on the cache index identifier of the mirrored sub-region and the capacity size of the sub-region; and store the information of the mirrored sub-region into the buffer space of the sub-structure according to the size of the storage space to be stored.

[0139] As an optional implementation provided in this application, the determining module 701 is further configured to: query the mirror physical address of the mirror sub-region of the target node with the same name through the target link; obtain the mirror address management structure through the data persistence component; and store the mirror physical address of the mirror sub-region of the target node with the same name into the mirror address management structure.

[0140] As an optional implementation provided in this application, the determining module 701, during the node initialization phase, after dividing the mirror region into multiple mirror sub-regions through the client component, is further configured to: respond to the domain pair configuration change event of the cluster control terminal through the data persistence component, refresh the domain pair configuration information, and initialize the mirror pair management structure of each mirror sub-region; the configuration change event indicates that other nodes have joined the cluster; check whether the mirror physical address of the mirror sub-region with the same name of other nodes is stored in the mirror address management structure; if not, query the mirror physical address of other nodes through the end-to-end link between nodes; and store the mirror physical address of other nodes in the buffer space of the mirror address management structure.

[0141] As an optional implementation provided in this application, the determining module 701 stores the mirror physical addresses of other nodes into the buffer space of the mirror address management structure, specifically for: determining the management table identifier corresponding to the mirror physical address of other nodes according to the node identifier of other nodes; and storing the mirror physical addresses of other nodes into the buffer space corresponding to the management table identifier.

[0142] As an optional implementation provided in this application, after the determining module 701 stores the mirror physical address of other nodes into the buffer space of the mirror address management structure, it is further used to: synchronize backup data to other nodes through the node-to-node end-to-end link with other nodes, so that the other nodes can store the backup data into the mirror sub-region of the same name of other nodes.

[0143] As an optional implementation provided in this application, the sending module 702 is further configured to: if incremental data is received when the backup data has not been synchronized to other nodes, synchronize the incremental data to the mirror sub-region of the same name on other nodes.

[0144] As an optional implementation provided in this application, the determining module 701 is further configured to: after the backup data is synchronized to other nodes, in response to the mirror pair switching instruction initiated by the cluster control terminal, perform a mirror pair switching operation on the mirror sub-regions of the same name on other nodes.

[0145] For a description of the features in the embodiment corresponding to the data synchronization device, please refer to the relevant description in the embodiment corresponding to the data synchronization method, which will not be repeated here.

[0146] like Figure 8 As shown, embodiments of this application also provide an electronic device, including a memory 801 and a processor 802, wherein the memory 801 stores a computer program and the processor 802 is configured to run the computer program to perform the steps in any of the above-described data synchronization method embodiments.

[0147] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data synchronization method embodiments at runtime.

[0148] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0149] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data synchronization method embodiments.

[0150] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data synchronization method embodiments.

[0151] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] The foregoing has provided a detailed description of a data synchronization method, apparatus, electronic device, storage medium, and program product provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data synchronization method, characterized in that, include: In response to an access request for backup data, the mirror pair management structure is determined based on the structure identifier contained in the access request; Based on the mirror pairing rules included in the mirror pair management structure, the mirror sub-regions with the same name of the target node are determined; the mirror pairing rules are used to configure the mirror relationship between mirror sub-regions with the same name of different nodes. A data transmission instruction is sent to the target node via the target link to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in a mirror sub-region with the same name as the target node; wherein, the target link is an end-to-end link between the target node and the target node.

2. The method according to claim 1, characterized in that, Before determining the mirror pair management structure based on the structure identifier contained in the access request in response to the backup data access request, the method further includes: During the node initialization phase, the data persistence component requests management structure resources from the memory management component; The management structure resources include at least one of the following: cache index identifier management structure, mirror address management structure, region management structure, and mirror pair management structure.

3. The method according to claim 2, characterized in that, The method further includes: During the node initialization phase, the mirror region is divided into multiple mirror sub-regions by the client component. The information of the mirror sub-regions includes the cache index identifier and the sub-region capacity size. Register the information of the mirrored sub-region to the data persistence component.

4. The method according to claim 3, characterized in that The step of storing the information of the mirrored sub-region to the data persistence component includes: The cache index identifier management structure is obtained through the data persistence component; The cache index identifier management structure is divided into multiple sub-structures; The information of the mirrored sub-region is stored in the corresponding sub-structure.

5. The method according to claim 4, characterized in that, The step of storing the information of the mirrored sub-region into the corresponding sub-structure includes: The size of the storage space to be calculated is based on the cache index identifier of the mirrored sub-region and the capacity of the sub-region; According to the size of the storage space to be stored, the information of the mirrored sub-region is stored in the buffer space of the sub-structure.

6. The method according to claim 2, characterized in that, The method further includes: The mirror physical address of the mirror sub-region with the same name of the target node is queried through the target link; The image address management structure is obtained through the data persistence component; Store the mirror physical address of the mirror sub-region with the same name of the target node into the mirror address management structure.

7. The method according to claim 3, characterized in that, In the node initialization phase, after the mirror region is divided into multiple mirror sub-regions by the client component, the method further includes: The data persistence component responds to configuration change events of domain pairs in response to the cluster control terminal, refreshes the domain pair configuration information, and initializes the mirror pair management structure of each mirror sub-region; the configuration change event indicates that other nodes have joined the cluster. Check whether the mirror address management structure stores the mirror physical address of the mirror sub-region with the same name of the other node; If not, then query the mirror physical address of the other nodes through the node-to-node end-to-end link with the other nodes; Store the mirror physical addresses of the other nodes into the buffer space of the mirror address management structure.

8. The method according to claim 7, characterized in that, The step of storing the mirrored physical addresses of the other nodes into the buffer space of the mirrored address management structure includes: Based on the node identifiers of the other nodes, determine the management table identifier corresponding to the mirror physical address of the other nodes; Store the mirror physical addresses of the other nodes into the buffer space corresponding to the management table identifier.

9. The method according to claim 7, characterized in that, After storing the mirrored physical addresses of the other nodes into the buffer space of the mirrored address management structure, the method further includes: The backup data is synchronized to the other nodes via an end-to-end link, so that the other nodes store the backup data in a mirror sub-region with the same name.

10. The method according to claim 9, characterized in that, The method further includes: If the backup data is not synchronized to the other nodes, and incremental data is received, the incremental data will be synchronized to the mirror sub-region with the same name on the other nodes.

11. The method according to claim 9, characterized in that, The method further includes: After the backup data is synchronized to the other nodes, in response to the mirror pair switching command initiated by the cluster control terminal, a mirror pair switching operation is performed on the mirror sub-regions with the same name on the other nodes.

12. A data synchronization device, characterized in that, include: The determination module is used to determine the mirror pair management structure based on the structure identifier contained in the access request in response to an access request for backup data. Based on the mirror pairing rules included in the mirror pair management structure, the mirror sub-regions with the same name of the target node are determined; the mirror pairing rules are used to configure the mirror relationship between mirror sub-regions with the same name of different nodes. The sending module is used to send a data transmission instruction to the target node via the target link, so as to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in a mirror sub-region with the same name of the target node; wherein, the target link is an end-to-end link between the target node and the target node.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data synchronization method as described in any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data synchronization method as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the data synchronization method as described in any one of claims 1 to 11.

Images