Data synchronization method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses a data synchronization method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of centralized storage device.The data synchronization method comprises the steps that when a backup data access request is responded, a corresponding mirror image pair management structural body is determined according to a structural body identifier contained in the request, and complex configuration traversal is not needed; meanwhile, based on a preset mirror image pairing rule in the structural body, the mirror image relationship of the same-name mirror image sub-regions of different nodes is defined, the same-name mirror image sub-regions of the target node can be quickly locked, the mirror image target positioning time is shortened, the resource consumption of a central processing unit in configuration query and relationship matching links is reduced, and the user experience is improved. And the overload problem of the central processing unit in synchronous replication is relieved. An end-to-end link between nodes is adopted to directly send a data transmission instruction to a target node, data does not need to be transferred through a cluster sharing network, and direct transmission of backup data to a same-name mirror image sub-region of the target node can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of centralized storage devices, and in particular to a data synchronization method, device, electronic device, storage medium, and program product. Background Art

[0002] Node-level data replication technology is a key technical means to ensure data synchronization in cluster storage systems. Its core principle is to build a highly reliable data redundancy system by synchronizing data mirrors between multiple storage nodes, thereby addressing the risk of data loss caused by a single node failure.

[0003] In related technologies, the main implementation methods for node-level data replication are synchronous replication and asynchronous replication. Synchronous replication requires that after a business end initiates an input / output (IO) request, both the master and slave nodes must complete the processing simultaneously before returning the result. This allows replicas to take over in seconds in the event of a failure. However, real-time mirroring consumes a large amount of IO bandwidth and central processing unit (CPU) resources, severely impacting the overall performance of the storage system. Summary of the Invention

[0004] The present application provides a data synchronization method, device, electronic device, storage medium and program product to at least solve the problem of large IO bandwidth and CPU resource consumption and impact on system performance in synchronous mirror replication in traditional centralized storage systems.

[0005] The present application provides a data synchronization method, comprising: in response to an access request for backup data, determining a mirror pair management structure according to a structure identifier included in the access request; determining a mirror sub-region of a target node with the same name based on a mirror pairing rule included in the mirror pair management structure; the mirror pairing rule is used to configure a mirror relationship between mirror sub-regions with the same name between different nodes; sending a data transmission instruction to a target node through a target link to control the target node to receive backup data in response to the data transmission instruction and store the backup data in the mirror sub-region of the target node with the same name; wherein the target link is an end-to-end link between nodes of the target node.

[0006] The present application also provides a data synchronization device, comprising: a determination module configured to, in response to a request for accessing backup data, determine a mirror pair management structure based on a structure identifier included in the access request; and determine a mirror sub-region of the target node with the same name based on a mirror pairing rule included in the mirror pair management structure; the mirror pairing rule being used to configure a mirror relationship between mirror sub-regions with the same name between different nodes; A sending module is used to send a data transmission instruction to a target node through a target link to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in the mirror sub-area of ​​the target node with the same name; wherein the target link is an end-to-end link between nodes and the target node.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of the above-mentioned data synchronization method when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program implements the steps of the above-mentioned data synchronization method when executed by a processor.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data synchronization method when executed by a processor.

[0010] When responding to a backup data access request, the present application determines the corresponding mirror pair management structure based on the structure identifier contained in the request, without the need for complex configuration traversal; at the same time, based on the preset mirror pairing rules in the structure, the mirror relationship between the mirror sub-regions of the same name on different nodes is clarified, and the mirror sub-region of the same name on the target node can be quickly locked, shortening the positioning time of the mirror target, reducing the resource consumption of the CPU in the configuration query and relationship matching links, and alleviating the CPU overload problem in synchronous replication. An end-to-end link between nodes is used to directly send data transmission instructions to the target node. The data does not need to be transferred through the cluster shared network, and the backup data can be directly transmitted to the mirror sub-region of the same name on the target node; compared with the multi-node data forwarding and shared bandwidth competition that may exist in synchronous replication, the end-to-end link can reduce the transmission delay and bandwidth occupancy of IO data, avoid the impact of bandwidth congestion on the overall IO performance of the storage system, and make up for the defect of large IO performance loss in synchronous replication. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A schematic diagram of a specific hardware architecture on which the execution of a data synchronization method provided in an embodiment of the present application relies; Figure 2 A flowchart of a data synchronization method provided for the implementation of this application; Figure 3A diagram showing the layered architecture of the cluster control terminal, client components, and data persistence components, as well as the collaborative relationships between the components; Figure 4A Schematic diagram of the process from the initial deployment of the first node to the addition of the second node and the completion of the image pair optimization Figure 1 ; Figure 4B Schematic diagram of the process from the initial deployment of the first node to the addition of the second node and the completion of the image pair optimization Figure 2 ; Figure 5A This is a diagram showing the transition from a temporary three-copy configuration to an optimal two-copy configuration after the third node joins the cluster. Figure 1 ; Figure 5B This is a diagram showing the transition from a temporary three-copy configuration to an optimal two-copy configuration after the third node joins the cluster. Figure 2 ; Figure 6A This is a diagram showing the process of adding the fourth node of the cluster to the stable mirror pair of the cluster. Figure 1 ; Figure 6B This is a diagram showing the process of adding the fourth node of the cluster to the stable mirror pair of the cluster. Figure 2 ; Figure 7 A schematic diagram of the structure of a data synchronization device provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0015] In order to more clearly illustrate the embodiments of the present application, the following briefly introduces the technical terms used in the embodiments: A Data Persistence Component (DPC) is a software module or subsystem responsible for reliably converting data from volatile storage to non-volatile storage.

[0016] The Memory Manager Component (MMC) is the core module in an operating system or virtualization platform that is responsible for allocating, reclaiming, and protecting memory resources.

[0017] The IO buffer reference manager (IOBM) is a component responsible for managing the mapping between data blocks in the input and output buffers and the physical locations of storage devices.

[0018] Mirror Address Management (MAM) is a component that maintains the physical location of data copies in distributed storage or high-availability systems.

[0019] Mirror Couple Management (MCPM) is a mechanism for managing the lifecycle of storage entities (such as disks, volumes, or nodes) that are replicas of each other.

[0020] The Cluster Management Side (CMS) is the core component in a distributed system responsible for coordinating node status, resource allocation, and fault handling.

[0021] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the data synchronization method depends, the specific application environment architecture or specific hardware architecture is described here.

[0023] like Figure 1 As shown, a schematic diagram of the specific hardware architecture on which the execution of the data synchronization method depends.

[0024] The core of the hardware architecture consists of multiple cluster nodes with independent computing and storage capabilities. Each node must be equipped with: a central processing unit (CPU) to execute the core logic of the data persistence component, such as parsing access requests, matching structure identifiers, and invoking image pairing rules; a dedicated memory partition for deploying memory management components and storing various management structures, ensuring low latency for structure identifier queries and rule reads; and local storage media, such as solid-state drives or mechanical hard drives, must be physically partitioned into independent image sub-regions for storing received backup data. These partitions must support pre-planning based on sub-region capacity to adapt to the storage resource allocation logic in the image pairing rules. The CPU and memory support dynamic calculation of image pairing rules (such as primary and backup node allocation and load balancing logic) and real-time adjustment of domain configurations.

[0025] Cluster control node hardware requires the deployment of at least one independent (or distributed) cluster control node. This node serves as the hardware carrier for the cluster control end, and its hardware configuration must meet global management requirements. Redundant network interfaces ensure stable communication with all cluster nodes, issuing control instructions such as image pairing rules and synchronization structure identifiers to prevent method execution stalls due to control link interruptions.

[0026] Node interconnect hardware ensures low latency and high reliability of data transmission. Each cluster node must be equipped with a high-speed network card that supports Remote Direct Memory Access (RDMA). Dedicated fiber or high-speed network cables are used to build end-to-end direct links between nodes, rather than relying on shared Ethernet. When receiving a request to access backup data, the node calls the memory management component to locate the mirror pair management structure; after determining the target node based on the mirror pairing rules in the mirror pair management structure, it sends a data transmission instruction through the network card and the direct link; after the target node receives the instruction, the storage controller writes the backup data into the mirror sub-area with the same name, finally completing data synchronization.

[0027] An embodiment of the present application provides a data synchronization method, and the method is described in detail in conjunction with the execution process of the data synchronization method.

[0028] like Figure 2 As shown, Figure 2 A flowchart of a data synchronization method provided for the implementation of this application, the method comprising: S201 : In response to a request for accessing backup data, determine a mirror pair management structure according to a structure identifier included in the access request.

[0029] Backup data refers to data that the current node needs to synchronize with the target node (i.e., the secondary / backup node). The access request includes the context ID of the mirror pair management structure, the mirror address offset, and the backup data size. The mirror pair management structure is obtained through the data persistence component (DPC) when initializing nodes during cluster creation.

[0030] After receiving the access request for the backup data, the current node determines the mirror pair management structure indexed by the structure identifier in response to the access request, without complicated configuration traversal.

[0031] S202: Determine the mirror sub-region of the target node with the same name based on the mirror pairing rule included in the mirror pair management structure.

[0032] Among them, the mirror pairing rule is used to configure the mirror relationship between mirror sub-regions with the same name between different nodes, clarifying the mirror relationship between mirror sub-regions with the same name on different nodes, and quickly locking the mirror sub-region with the same name on the target node.

[0033] Retrieve the mirror pairing rule from the mirror pair management structure. Then, based on the correspondence between the current node's mirror sub-regions and the target node's mirror sub-regions as indicated by the mirror pairing rule, determine which node's mirror sub-region to synchronize the backup data to. The mirror pairing rule implies that if the current node has a master-slave relationship with at least one other node (such as the target node), and the backup data is stored in a mirror sub-region of the current node, then synchronize the backup data to the mirror sub-region of the target node with the same name as that mirror sub-region.

[0034] S203 : Send a data transmission instruction to the target node through the target link to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in the mirror sub-area of ​​the target node with the same name.

[0035] The target link is an end-to-end link between the current node and the target node. Compared to the multi-node data forwarding and shared bandwidth competition that can occur in synchronous replication, an end-to-end link reduces I / O data transmission latency and bandwidth usage, preventing bandwidth congestion from impacting the overall I / O performance of the storage system and compensating for the significant I / O performance loss inherent in synchronous replication.

[0036] The current node sends a data transmission instruction for synchronizing backup data to the target node via an end-to-end internode link. The target node responds to the data transmission instruction by first receiving the backup data to be synchronized and then storing the backup data in a mirrored sub-region with the same name as the current node. By using the end-to-end internode link to directly send data transmission instructions to the target node, data does not need to be transferred through the cluster's shared network, enabling direct transmission of backup data to the mirrored sub-region of the target node with the same name.

[0037] In some embodiments, before executing step S201 (responding to a request for access to backup data and determining a mirror pair management structure based on a structure identifier included in the access request), specifically during the cluster creation phase and node initialization process, the current node requests management structure resources from the memory management component (MMC) via the data persistence component, initializes the resource space for the management structure resources, and assigns corresponding structure identifiers (Context IDs) to each management structure. The requested management structure resources include at least one of a cache index identifier management (IOBM) structure, a mirror address management (MAM) structure, a region management structure, and a mirror pair management (MCPM) structure.

[0038] Among them, MMC can allocate exclusive resource space and unique structure identifier (Context ID) to at least one structure such as cache index identification management IOBM structure, image address management MAM structure, region management structure, and mirror pair management MCPM structure according to the actual needs of the node, such as the number of mirror sub-regions and slave node configuration, to avoid configuration conflicts caused by resource overlap between different structures.

[0039] In the above embodiment, during the node initialization phase of cluster creation, the current node requests management structure resources from the memory management component MMC through the data persistence component DPC and completes initialization. This can produce multi-dimensional key technical effects and lay the core foundation for the subsequent cluster mirror pair formation and data synchronization. First, this operation realizes the standardized allocation and control of management structure resources. Through a unified ContextID, each structure can be accurately located and managed, significantly reducing the resource query cost during subsequent component interaction.

[0040] DPC completes the initialization of the management structure in advance, so that when subsequent nodes join or domain configurations are updated, DPC does not need to temporarily apply for resources. Instead, it can directly call IOBM to manage IO cache identifiers through ContextID, use MAM to cache slave node addresses, and rely on MCPM to establish mirror pair relationships, thereby improving component response speed and avoiding mirror process stalls caused by resource application delays. In addition, the resource space and standardized structure identifiers reserved during initialization can adapt to subsequent cluster expansion scenarios. Regardless of the domain adjustment or mirror reorganization triggered by the newly added nodes, DPC can respond quickly based on the initialized management structure, ensuring the stability of core operations such as mirror data transmission, address query, and cache management, and avoiding the risk of data loss and service interruption caused by uninitialized structures or insufficient resources at the underlying resource level.

[0041] Then, you can also apply for buffer space from the memory management component MMC. The buffer space is used to store the cache index identifier and the mirror physical address. The cache index identifier is the IO cache index identifier (IOB). Each management resource structure has a unique structure identifier, which is used to quickly locate and access the storage address of the corresponding management structure. The unique identifier corresponding to each management resource structure (such as IOBM, MAM, MCPM) is equivalent to establishing a memory address index for the structure. When the DPC interacts with the MMC subsequently, there is no need to query the structure storage address through complex memory scanning or configuration matching. The target structure can be directly located through the identifier, which greatly reduces the delay in resource calls between components. MMC allocates independent buffer space for IO cache index identifiers (IOBs) and mirror physical addresses. On the one hand, this avoids conflicts between the two types of critical data and other memory resources, reducing the risk of IO cache identifier confusion and mirror address loss due to data overwriting or interference. On the other hand, dedicated buffers can improve data read and write speeds through memory partition optimization. Compared with shared memory, IOB queries and mirror physical address calls do not need to traverse redundant memory areas, shortening data access time and laying an efficient data foundation for subsequent mirror data transmission and IO request responses. When processing mirror requests, DPC can quickly find the corresponding management structure through the structure identifier, and then retrieve the IOB and mirror address from the dedicated buffer. This not only improves the data processing efficiency within a single node, but also provides stable underlying data support for subsequent data synchronization between nodes and mirror pair switching, further reducing the risk of service interruption due to data storage confusion or positioning delays.

[0042] Optionally, the data persistence component requests 1024 management structure resources from the memory management component (MMC) based on business needs. These 1024 structures are suitable for medium- and large-scale storage applications, meeting requirements such as partitioning 1024 mirror sub-regions, building 1024 mirror pairs, and managing 1024 node cache indexes, avoiding resource shortages or redundancy. In scenarios with high I / O requests, each structure can independently carry management logic, such as corresponding to an I / O cache identifier or a group of mirror addresses, to prevent resource contention and ensure rapid business response.

[0043] Optionally, this application determines a baseline management structure from management structure resources through a data persistence component. Specifically, the sizes of each management structure are calculated and compared, and the largest management structure is selected as the baseline management structure. Setting a unified resource allocation threshold based on the largest structure can avoid uneven resource allocation between structures, simplify logic, and reduce management complexity. The largest structure can carry complex logic, and using this as a benchmark can ensure sufficient resources when expanding functions, reduce DPC adaptation adjustments, and improve component interaction stability.

[0044] In some embodiments, before executing step S201 (responding to a request to access backup data and determining a mirror pair management structure based on a structure identifier included in the access request), specifically during the cluster creation phase and node initialization process, the current node, through the client component, divides the mirror area into multiple mirror sub-areas to partition and reserve mirror storage space for business data. Information about the mirror sub-areas includes an IO buffer reference identifier and the sub-area capacity. The client component then registers this information with the data persistence component (DPC).

[0045] Illustratively, the quartering method is used to divide the mirror area into four mirror sub-areas, such as Zone0, Zone1, Zone2, and Zone3.

[0046] It should be noted that mirror sub-regions correspond one-to-one with domains on the Cluster Management System (CMS). For example, the CMS domains may include domain0, domain1, domain2, and domain3. The aforementioned four mirror sub-regions correspond to the four domains here. The CMS domain can determine node pairing rules and mirror pairing rules based on the mirror sub-regions of each node. Node pairing rules configure the master-slave relationship between different nodes, while mirror pairing rules configure the mirror relationship between master and slave nodes with the same-named mirror sub-regions.

[0047] In the above embodiment, during the node initialization phase of cluster creation, the current node first divides the mirror area into multiple mirror sub-areas using the four-partition method through the client component, and each mirror sub-area clearly defines the IO cache reference identifier and capacity. This not only reserves exclusive space for the business to avoid storage conflicts, but also accurately matches business needs, and facilitates the data persistence component to quickly locate the IO cache and improve read and write efficiency. The client component synchronizes the key information of the mirror sub-area to the data persistence component, so that it can grasp the distribution of storage resources and can directly call the mirror sub-area resources to shorten the response delay of the mirror operation. At the same time, the data persistence component can initialize the management structure in advance to avoid information faults between components. The correspondence between the mirror sub-area and the cluster control end domain facilitates the cluster control end to formulate node and mirror pairing rules to ensure that the rules are effectively applied to the storage mirror sub-area. When a new node joins, the configuration can be quickly updated based on this to ensure the accuracy and efficiency of the mirror reorganization and support the stable operation of the cluster mirror system.

[0048] In some embodiments, in the process of storing the information of the mirror sub-area in the data persistence component, the cache index identification management structure is obtained through the data persistence component, and then the cache index identification management structure is divided into multiple sub-structures, and then the information of the mirror sub-area is stored in the corresponding sub-structures.

[0049] Each substructure corresponds to a domain on the cluster controller. Each substructure records information about the mirror subregion. Based on the size of the IO cache reference identifier, free buffer space is allocated to store the IO cache reference identifier, allowing for unified management of the mirror subregions registered by the client.

[0050] For example, the cache index identification management structure is divided into four sub-structures, IOBM0, IOBM1, IOBM2, and IOBM3, which correspond to the four domains, domain0, domain1, domain2, and domain3, of the cluster control terminal, respectively.

[0051] Each IOBM substructure corresponds to a mirror pair management MCPM structure, and each mirror pair management MCPM structure is associated with three mirror address management MAM structures. The mirror address management MAM structure is used to store and manage the mirror physical addresses of the mirror subregions of the slave node and to apply for free buffer space based on the subregion capacity of the mirror subregion to store the mirror physical addresses.

[0052] like Figure 3 As shown, Figure 3This diagram shows the layered architecture and component collaboration relationship of the cluster control terminal (CMS), client components (Client), and data persistence components. It presents the association and component interaction logic of the cluster control terminal CMS, client components, and data persistence components (DPC) at the domain and mirror sub-zone levels in the cluster storage system.

[0053] Figure 3 The CMS manages four domains, domain0 through domain3, which are logical divisions of storage resources within the clustered storage system. The client side consists of four mirrored sub-zones, Zone0 through Zone3. Each zone corresponds to a domain in the CMS and carries client data, serving as the fundamental logical area for data storage.

[0054] Within the data persistence component, the substructures of the IO cache index identification management structure, from substructure 0 to substructure 3, correspond to Zone0 to Zone3 respectively, and are used to manage the IO cache related identification of each mirror sub-area. They are the entry identification management structure for data IO operations. There are also four mirror pair management MCPM structures, which correspond one-to-one to substructure 0 to substructure 3. They are responsible for managing the mirror pair relationship corresponding to each mirror sub-area, such as determining the pairing of mirror sub-areas of the master and backup nodes, etc., and are the core structure for mirror relationship management. The mirror address management MAM structures, from MAM0 to MAM2, are associated with each mirror pair management MCPM structure and are used to cache the physical address of the slave node mirror sub-area. When the master node needs to synchronize data with the slave node or perform a mirror operation, it will first query the address information of the slave node through the mirror address management MAM structure to ensure the address accessibility of the mirror data transmission.

[0055] Optionally, in the process of storing the information of the mirror sub-region corresponding to the sub-structure, the information of the mirror sub-region includes the cache index identifier and the sub-region capacity size. First, the size of the space to be stored is calculated based on the cache index identifier and the sub-region capacity size, and then the information of the mirror sub-region is stored in the buffer space of the sub-structure based on the size of the space to be stored.

[0056] The buffer space size used to store the mirror sub-region information can be calculated using the following formula (1): Buffer = A*N*M / P + B*M / P (1). In formula (1), M is the sub-region capacity, P is the page size, A is the byte size of each page address, N is the number of slave nodes corresponding to each node, and B is the byte size of the IO cache index identifier. Each parameter in the formula is strongly related to the actual business scenario. The calculated buffer space can accurately carry the mirror physical address and IO cache index identifier, eliminating the need for subsequent dynamic space resizing, reducing the performance loss caused by frequent space adaptation operations in the sub-structure.

[0057] For example, assuming a 40GB sub-region capacity, using 4K pages as units, with each page address occupying 8 bytes and each node corresponding to the address mapping of three potential slave nodes, the size of the buffer space occupied by the mirrored physical address is calculated. Based on the maximum rule of one IOB per page, each IOB occupies 4 bytes, and the size of the buffer space occupied by the IOB is calculated. The buffer size is the sum of the above two parts: 40GB / 4K*8*3+40GB / 4K*4=270MB.

[0058] When storing mirrored sub-region information in a substructure, the above embodiment uses the formula Buffer = A*N*M / P + B*M / P, taking into account parameters such as sub-region capacity and page size, to accurately calculate the buffer space. For example, under conditions such as a 40GB sub-region and a 4KB page, the calculated buffer size is 270MB, avoiding space overflow and waste. This precisely calculated buffer prevents information truncation and misalignment, and also reserves adaptive space for cluster expansion and data synchronization, providing stable support for subsequent operations and reducing the risk of business interruption due to storage issues.

[0059] In some embodiments, based on applying for a mirror address management structure, the data synchronization method provided by the present application also includes: first querying the mirror physical address of the mirror sub-area with the same name of the target node through the target link, obtaining the pre-applied mirror address management structure, and storing the mirror physical address of the mirror sub-area with the same name of the target node in the mirror address management structure.

[0060] The above embodiment, based on the pre-application of the mirror address management structure, queries and stores the mirror physical address of the mirror sub-area of ​​the target node with the same name through the target link, and uses a dedicated structure to store the address, so that the master node does not need to repeatedly query when performing operations such as data synchronization, and directly calls the structure to obtain the address, thereby reducing link overhead and delay. For example, when the master node node0 synchronizes Zone0 data to the backup node node1, it reads the structure address to avoid IO performance loss and improve response speed. Structured storage can avoid address information confusion, and unified management can ensure that a valid address is called. When the address of the sub-area of ​​the target node changes, updating the structure can ensure accurate data synchronization and avoid the risk of inconsistency.

[0061] When a node joins the cluster, or when mirror pair formation, switching, etc. are performed, refer to Figure 3 The CMS will first update configuration information, including the node composition of domains 0 to 3. Client Zones 0 to 3 serve as data carriers, and the mirroring logic for their data is handled by the DPC. The DPC uses the IOB to identify I / O operations for the corresponding zone. The MCPM then determines the target slave node's mirrored sub-region with the same name based on the mirror pairing rules. Finally, using the slave node addresses cached in the MAM, mirroring operations such as data transmission are completed through inter-node links, ensuring reliable data mirroring and persistence across cluster nodes.

[0062] In some embodiments, after the mirror area is divided into multiple mirror sub-areas by the client component during the node initialization phase, the process also includes: responding to the configuration change event of the domain pair of the cluster control end through the data persistence component, refreshing the domain pair configuration information, and initializing the mirror pair management structure of each mirror sub-area, wherein the configuration change event of the domain pair of the cluster control end indicates that other nodes have joined the cluster. Then, it is checked whether the mirror physical address of the mirror sub-area of ​​the same name of the other node is stored in the mirror address management structure. If not, the current node queries the mirror physical address of the other node through the end-to-end link between the nodes and the other nodes. Then, the mirror physical address of the other node is stored in the buffer space of the mirror address management structure to complete the mirror preparation.

[0063] The domain pair configuration information includes the node composition information of multiple domains. It should be noted that every time a new node is added to the cluster, the cluster control system (CMS) will immediately respond to the node joining event and update the domain pair configuration information. For example, the node composition of domain0 is updated from (0) to (0,1).

[0064] In the above embodiment, when other nodes join the cluster, CMS triggers a domain pair configuration change event, and DPC can immediately perceive and refresh the domain pair configuration information to avoid mirror logic confusion caused by configuration delays; at the same time, DPC initializes the corresponding mirror pair management structure for each mirror sub-area, providing a dedicated management carrier for the subsequent master-slave node mirror relationship, and can complete the basic construction of the mirror structure without waiting for other processes, shortening the preparation time of the mirror system after the node joins, and improving the response efficiency of cluster expansion.

[0065] The address is queried and stored through the end-to-end link between nodes to enhance the efficiency and reliability of obtaining the mirror physical address. The DPC first checks whether there is a physical address of the sub-area with the same name on other nodes in the mirror address management structure. If it is missing, it is directly queried through the end-to-end link. Compared with the query method that relies on the cluster shared network, the end-to-end link avoids the address query delay caused by network congestion, and at the same time reduces the forwarding of intermediate nodes during the address transmission process, reducing the risk of address loss or tampering. For example, when the DPC of node0 queries the physical address of Zone0 of node1, it directly obtains and stores it in the buffer space of MAM through a dedicated link, ensuring that the valid address can be quickly called during subsequent data synchronization, avoiding synchronization failures due to missing or incorrect addresses.

[0066] DPC not only initializes the mirror pair management structure but also ensures that the mirror address management structure fully stores the mirror physical addresses of other nodes. This combination prevents mirror relationship establishment failures due to uninitialized structures or missing addresses. Subsequent image building can be completed by simply triggering a data transfer instruction, eliminating the need for additional preparatory steps. This provides a complete technical foundation for rapidly achieving data redundancy between cluster nodes.

[0067] In the scenario where nodes are added to the cluster one by one, this application allows the cluster controller (CMS) to dynamically update the domain pair configuration information, i.e., the node composition information, based on the current number of nodes in the cluster. Changes in node composition information trigger mirror pair reorganization, for example, from a single node to two nodes, and then to a temporary three-node state. The ultimate goal is to gradually converge the mirror pair from the temporary transition state to a load-balanced steady state.

[0068] For example, assuming that when the first node node0 is added, the domain pair configuration information indicates that each domain is in a single-node state, there is no node pairing rule, no mirror pairing rule, and the mirror relationship remains silent. When the second node node1 is added, the domain pair configuration information indicates a dual-node pairing state such as (0,1), indicating that node0 is the master node and node1 is the slave node. When the third node node2 and the fourth node node3 are added, the domain pair configuration information indicates a temporary three-copy state such as (1,0,2) and (1,2,3). After the backup data synchronization is completed, the domain pair configuration information is adjusted to a dual-node pairing state. Refresh your own domain pair configuration information to ensure consistency with the CMS update.

[0069] The current node queries the mirror physical address of other nodes through the inter-node end-to-end link with other nodes. The current node first queries the IO cache index identifier of other nodes through the inter-node end-to-end link with other nodes, and then determines the mirror physical address mapped by the IO cache index identifier.

[0070] For example, assuming a domain pair configuration change event indicates that another node, node1, has joined the cluster, the current node, node0, checks whether the mirror address management structure stores the mirror physical address of the mirror subregion with the same name as the other node, node1. If not, the current node, node0, proactively queries the mirror physical address of the other node, node1, through the end-to-end link between the current node, node0, and the other node, node1.

[0071] Optionally, in the process of storing the mirrored physical addresses of other nodes in the buffer space of the mirrored address management structure, the management table identifier corresponding to the mirrored physical addresses of other nodes is first determined based on the node identifier of the other node, and then the mirrored physical addresses of other nodes are stored in the buffer space corresponding to the management table identifier.

[0072] Specifically, the management table identifier MAM corresponding to the mirror physical address of other nodes is determined according to the following formula (2): tableid : (2) In the node identifier of the current node (owner nodeID ) is greater than other nodes (backup nodeID ) node identifier, determine the management table identifier (MAM) corresponding to the mirror physical address of other nodes tableid ) is equal to other nodes (backup nodeID ) node identifier. In the node identifier of the current node (owner nodeID ) is smaller than other nodes (backup nodeID ) node identifier, determine the management table identifier (MAM) corresponding to the mirror physical address of other nodestableid ) is equal to other nodes (backup nodeID ) minus a preset value, such as MAM tableid =backup nodeID -1. Then the mirror physical addresses of other nodes are stored in the buffer space corresponding to this management table identifier.

[0073] Formula (2) in the above embodiment directly maps the unique MAM based on the size relationship between the current node identifier and other node identifiers. tableid The calculation logic does not need to traverse the buffer, and the reverse substitution formula can be used to locate it when calling. For example, when 0<1, MAM tableid =0, the subsequent call calculation "1-1=0" can find the address, which greatly improves the real-time performance of the call in multi-node scenarios. Formula (2) assigns a unique fixed MAM to the target node address tableid , to avoid address conflicts. Taking a 4-node cluster as an example, owner_nodeID=1 stores different backup nodeID When addressing, each corresponds to a different MAM tableid , facilitating partition management, reducing address coverage risks, and simplifying the operation and maintenance troubleshooting process. Formula (2) relies only on the size relationship of node identifiers, not on the number of nodes or business scenarios, and supports cluster expansion.

[0074] In some embodiments, when a new node is added to the cluster, the current node synchronizes backup data with the newly added node. The current node synchronizes backup data with the other node via an inter-node end-to-end link with the other node, so that the other node stores the backup data in the mirror sub-area of ​​the same name of the other node.

[0075] For example, when the current node (node0) synchronizes Zone1 backup data with the newly added node (node2), the end-to-end link can directly transmit the data from Zone1 of node0 to Zone1 of node2 without forwarding it through other nodes. Compared with shared network transmission, this can significantly shorten data transmission time. Especially in scenarios with large amounts of backup data (such as GB level), data synchronization can be completed quickly, allowing the newly added node to quickly acquire data service capabilities.

[0076] When additional nodes are added to the cluster, the current node synchronizes its backup data with them via an end-to-end link between nodes, ensuring that the new node stores its data in the mirrored sub-region with the same name. This improves data synchronization efficiency and enhances data reliability and consistency. Specifically, the current and new nodes transmit data directly over a dedicated end-to-end link, bypassing the cluster's shared network. This avoids bandwidth competition and congestion associated with multi-node data transmission on the shared network, and reduces data synchronization latency.

[0077] The newly added node stores synchronized backup data in the mirrored sub-region with the same name as the current node (e.g., Zone0 for node0 corresponds to Zone0 for node2), ensuring that data under the same mirror logic is completely consistent between the primary and backup nodes, thus achieving data redundancy. If the current node fails, the newly added node can quickly switch to providing services as the primary node, relying on the complete backup data stored in the mirrored sub-region with the same name, thus avoiding data loss. Furthermore, this synchronization of mirrored sub-regions with the same name ensures that the data logic of each node remains unified when the cluster performs subsequent mirror pair switching and load balancing adjustments, preventing data confusion caused by sub-region data mismatches. This ensures cluster data consistency from a data storage perspective.

[0078] During the node addition process, while the current node synchronizes data via an end-to-end link, it can still process business I / O requests normally, without suspending service to wait for data synchronization to complete. Furthermore, during data synchronization, the newly added node can gradually build complete mirrored sub-region data. Once synchronization is complete, it can seamlessly integrate into the cluster mirroring system and assume the role of backup or master node. For example, when a cluster adds node3, while node2 synchronizes data with node3, node2 can still provide data read and write services for the business. After node3 synchronization is complete, it can directly participate in domain pairing and mirror pair formation, avoiding business interruptions caused by node expansion, ensuring that the cluster maintains stable service capabilities during expansion, and providing reliable technical support for the dynamic growth of cluster scale.

[0079] In some embodiments, when the backup data is not synchronized to other nodes, if incremental data is received, the incremental data is synchronized to the mirror sub-region of the same name of the other nodes.

[0080] For example, assuming that three other nodes are added to the cluster, when the backup data is not fully synchronized to these three other nodes, the incremental data received by the current node is synchronized to at least two nodes to avoid incomplete data due to node anomalies. The strategy of synchronizing to at least two nodes strengthens the multiple redundancy of node anomaly risks. Compared with synchronizing to only one newly added node, synchronizing to at least two nodes can form multi-copy protection. If one of the nodes receiving incremental data encounters an anomaly such as being offline or having a hardware failure, the other node still holds the complete incremental data and can continue to participate in subsequent backup synchronization and business services, avoiding the loss of incremental data due to a single node anomaly. For example, the current node synchronizes incremental data to the Zone with the same name of the newly added nodes node2 and node3. If node2 suddenly goes offline, there is still incremental data in the Zone with the same name of node3. The backup synchronization can be continued based on the incremental data of node3 without retransmitting the incremental data, which greatly reduces the risk of incomplete data caused by node anomalies and improves the cluster's ability to resist failures.

[0081] In the above embodiment, if the backup data has not yet been synchronized to other nodes, the current node synchronizes the received incremental data to the mirrored sub-region of the same name on the other node, enabling immediate storage of the incremental data and preventing data loss during synchronization gaps. The synchronous connection between the backup data and the incremental data reduces the cost of subsequent retransmissions.

[0082] If backup data hasn't been fully synchronized to the newly added nodes, relying solely on subsequent incremental data transmission could result in data loss due to issues like a current node failure or a link outage. However, synchronizing incremental data to mirrored sub-zones of the same name on other nodes in real time creates an instant redundant copy of the incremental data. For example, if the current node is synchronizing backup data to three newly added nodes and only 50% of the backup synchronization is complete, the incremental data received will have been synchronized to the same-name zones of at least two of the newly added nodes. Even if the current node suddenly fails, the complete incremental data will already be stored in the same-name sub-zones of these two nodes, eliminating the need to rely on the failed node for recovery.

[0083] After the incremental data is synchronized to the mirror sub-region of the same name in real time, once the backup data has completed basic synchronization, the newly added node no longer needs to re-acquire the incremental data from the current node. Instead, it only needs to integrate the stored incremental data with the basic backup data to quickly form the complete mirror sub-region data. For example, while the current node is synchronizing the basic backup data to node2, it also synchronizes the incremental data to node2's Zone1 in real time. After the basic backup synchronization is complete, node2 only needs to merge the basic data and incremental data without the need for additional incremental data requests. This shortens the total time it takes to integrate the new node into the cluster mirror system, while reducing the pressure of repeated transmission on the current node, ensuring efficient synchronization process without affecting its normal business processing capabilities.

[0084] In some embodiments, after the backup data is synchronized to other nodes, a mirror pair switch operation is performed on the mirror sub-regions of the same name on other nodes in response to a mirror pair switch instruction initiated by the cluster control terminal. This establishes a stable cluster mirroring system and clarifies the active-standby collaboration relationship, ensuring seamless service continuity and avoiding the risk of switchover interruptions.

[0085] After backup data synchronization is complete, the mirror sub-regions of each node with the same name have a complete data foundation. At this point, the mirror pair switch is executed. Essentially, CMS specifies the master and backup roles for the mirror pair sub-regions based on preset rules (such as node load and failure risk), such as the current node Zone0 being the master and the newly added node Zone0 being the backup. For example, after a CMS instruction triggers the switch of the Zone0-Zone3 mirror pair between node0 and node1, the master-slave relationship of each sub-region with the same name is fixed. Subsequent data reads and writes only need to follow the logic of master node writing and slave node synchronization, avoiding data write conflicts or synchronization disorders caused by ambiguous master and backup roles. At the same time, after the switch is complete, the cluster mirror pair officially enters a steady state, providing a clear collaborative framework for subsequent node failover and data redundancy backup, and consolidating the foundation of cluster data reliability.

[0086] Because the switching operation is performed under the premise that the backup data has been synchronized, and only the primary and backup roles of the mirror sub-regions with the same name are logically adjusted, there is no need to suspend services at the business level. For example, if the current node is processing business IO requests through Zone1, and the backup data is synchronized to Node2's Zone1, when the switching operation is performed to transfer the primary role to Node2, only the primary and backup address identifiers in the mirror address management structure need to be updated. Business requests can be seamlessly switched from the current node Zone1 to Node2's Zone1, and users will not perceive any delays or interruptions. This logic of switching after data is ready completely avoids switching failures caused by incomplete data or business interruptions caused by switching, ensuring the continuity and stability of cluster services.

[0087] In addition, mirror pair switching operations transfer the master role of a high-load node to the mirror sub-zone of the same name on a low-load node, achieving balanced resource utilization. For example, if node0 has been assuming the master role for multiple sub-zones for a long time, resulting in excessive IO load, after the backup data is synchronized to node1 and node2, the CMS command will switch the master roles of Zone2 and Zone3 on node0 to the sub-zones of the same name on node1 and node2. This reduces the load on node0 while fully utilizing the idle storage resources of node1 and node2. This load-based switching adjustment avoids a single node being overloaded due to too many master roles and prevents other node resources from being idle, improving the resource utilization efficiency of the entire cluster and providing hardware-level support for long-term stable services.

[0088] From the perspective of the current node, the current node triggers corresponding operations in each round of domain pair changes through the data persistence component DPC. First, for each mirror sub-region, initialize the IO cache index identification management (IOBM) structure to build the mirror basic framework. Refresh the domain pair configuration information stored in itself in real time to ensure consistency with the updates of the cluster control end CMS. The mirror address management MAM structure caches the mirror physical address of the mirror sub-region with the same name from the slave node. Specifically, if the master node ID is greater than the slave node ID, it goes to MAM. tableid =backup nodeID ; If the master node ID < slave node ID, then MAM tableid =backup nodeID -1. If the mirror address management (MAM) structure doesn't cache the mirror physical address of the same-named mirror subregion of the slave node, it proactively queries the slave node via the end-to-end link between nodes, populates the address, and completes mirror preparation. When a new node is added, the current node, acting as the master, synchronizes backup data to the new node's mirror subregion of the same name. Before data synchronization is complete, the current master node must synchronize incremental data to the slave node to prevent data incompleteness due to node anomalies. After all mirror pairs are ready and data synchronization is complete, the CMS initiates a mirror pair switch to achieve load balancing. For example, after node1 joins, domain1 / 2 is adjusted to (1,0). After node3 joins, the optimal cyclical combination of (0,1), (1,2), (2,3), and (3,0) is formed. It should be noted that switchover is only performed for the domains where the master / slave roles are swapped (such as domain1 / 2). The remaining domain mirror pairs continue to provide normal services, reducing the risk of service interruption.

[0089] For example, assume that four new nodes are added to the cluster. After node 1 joins the cluster, the domain pair configuration information is (1,0). After the backup data is synchronized to the new node, the cluster control terminal issues a mirror pair switch instruction. The current node responds to the instruction and executes the mirror pair switch operation, forming a cyclic optimal combination of (0,1), (1,2), (2,3), and (3,0).

[0090] For example, after the first node (node0) joins the cluster, the cluster controller CMS updates the node composition of domain0 to domain3 to a single node (0), that is, each domain only contains node0. In response to the domain change, node0's data persistence component DPC not only initializes the mirror pair management MCPM structure corresponding to each mirror sub-region, but also reserves and applies for the mirror pair management MCPM structure resources of the corresponding mirror sub-region Zone, paving the way for subsequent nodes to join. At this time, the mirror relationship remains silent, waiting for the next node to join.

[0091] In response to the second node (node1) joining, the cluster controller (CMS) synchronizes the node pairings of domains 0 through 3 to a (0,1) pairing state, indicating that node 0 is the master node and node 1 is the slave node. The data persistence components (DPCs) for nodes 0 and 1 detect the domain configuration change and immediately refresh their stored domain pair configuration information. They also initialize the corresponding mirror pair management MCPM structures for each mirror sub-region's associated IO cache index identifiers and establish the mirror pair relationship.

[0092] Node0, as the master node, first checks whether the associated mirror address management (MAM) structure has cached the physical address of the mirror sub-region corresponding to node1; if not, it queries the mirror physical address mapped by node1's IOB through the high-speed data link between nodes, and nodeID >backup nodeID Save to MAM tableid =backup nodeID Otherwise, save to MAM tableid =backup nodeID -1 is stored in the mirror address to manage the MAM structure buffer space.

[0093] After basic operations include but are not limited to: configuration refresh, image pair management MCPM structure initialization, and address caching, the cluster control end CMS optimizes the domain pair to a combination mode of (0,1), (1,0), (1,0), (0,1) to achieve load balancing. Here, only domain1 and domain2 swap the active and standby roles, with node1 as the active node and node0 as the standby node.

[0094] The data persistence component (DPC) on node0 and node1 only switches and adjusts the mirror pairs for domain1 and domain2. Node1, acting as the master node, repeatedly checks the mirror address management (MAM) structure cache, then queries the node0 address and repopulates the mirror address management (MAM) structure. The mirror pairs for the remaining domains (0 and 3) maintain their original state and continue to provide services. The cluster control system (CMS) must wait for the data persistence component (DPC) to complete all responses before performing subsequent operations to avoid configuration confusion.

[0095] When the third node (node2) joins, the cluster control end CMS temporarily updates the node composition of domain0 to domain3 to (0,1), (1,0,2), (1,0,2), (0,1,2). Here, only domain1, 2, and 3 enter the three-copy state. Domain0 maintains two nodes, among which domain1 and 2 use node1 as the active node and node0 and node2 as the backup nodes. Domain3 uses node0 as the active node and node1 and node2 as the backup nodes.

[0096] The data persistence component DPC responds to domain changes to reorganize the mirror pair. The new master node (such as node1, node0) needs to check the physical addresses of the mirror sub-regions of the two slave nodes (node0 and node2, node1 and node2) at the same time. If the mirror address management MAM structure is not cached, the physical address mapping query process is triggered. The mirror pair reorganization is officially completed until all slave node addresses are cached.

[0097] The master node synchronizes data to the mirrored subregion of the new slave node, node2. If data synchronization is not complete, the mirror pair provides services as three replicas. When incremental data arrives, the master node synchronizes it to both slave nodes simultaneously, such as node1 synchronizing to node0 and node2, to prevent data incompleteness caused by the abnormal departure of node2.

[0098] After all mirror data is synchronized, the cluster control end CMS adjusts the domain pairs to the optimal dual-copy combinations (0,1), (1,2), (0,2), (2,0), and executes mirror pair switching based on the established rules for primary and backup address verification and mirror address management MAM structure filling, completing the transition from temporary three copies to stable two copies.

[0099] When the fourth node (node3) joins, the cluster control end CMS temporarily updates the node composition of domain1 to domain3 to (1,2,3), (0,2,3), (2,0,3), domain0 maintains the original two nodes, and the new master node (such as node1, node0, node2) reuses the three-copy processing flow when node2 joins: check the mirror address management MAM structure address cache of the two slave nodes, and query and fill it if it is not cached. During data synchronization, three copies are used to ensure data reliability.

[0100] After the mirror data synchronization and reorganization are completed, the cluster control end CMS adjusts the domain pairs to the cyclic optimal combination (0,1), (1,2), (2,3), (3,0), and executes the mirror pair switching according to the previous rules; after the switch is completed, the cluster's stable mirror pair is formed, and each domain runs in a dual-node active-standby mode, taking into account load balancing and data reliability.

[0101] The following is a detailed introduction with diagrams, such as Figure 4A and Figure 4B As shown in the figure, the process diagram of the cluster from the initial deployment of the first node (node0) to the joining of the second node (node1) and the completion of the image pair optimization is shown. Figure 3 The process involves updating the domain node composition led by the cluster control system (CMS), initializing the mirror structure and managing addresses driven by the data persistence component (DPC), and finally switching mirror pairs guided by load balancing. This can be explained in two stages: The first stage is the initial state of node0. At this time, the data persistence component DPC of node0 reflects its core action in response to domain changes through the silent state label: the mirror pair management MCPM structure is initialized for the mirror sub-area of ​​each domain. However, since there is no slave node, all mirror relationships are in a silent state waiting for subsequent nodes, and there is no data transmission link or mirror pairing identifier.

[0102] The second stage is the domain update and mirror pair construction stage after node1 joins. First, through the interactive arrows of the cluster control end CMS, it is shown that it synchronously adjusts the node composition of domain0 to domain3 to a (0,1) pairing form, identifies node0 as the master node and node1 as the slave node, and uses the master-slave identifier to distinguish the roles of the two nodes in the domain association; then, through the associated lines of DPC, IOBM, and MCPM, it reflects the process of completing MCPM initialization for each mirror sub-area after the DPC of node0 and node1 synchronously refreshes the domain configuration, and at the same time focuses on the core operation of node0 as the master node: through the bidirectional arrow pointing to node1, combined with the buffer space identifier of the mirror address management MAM structure, it shows that it first checks the address cache, and if it is missing, queries the IOB physical address of node1, and then stores it in the MAM according to the rules; finally, the switch instruction arrow and the composition change of domain1 and domain2 represent the load balancing optimization action. At this time, only the master and slave node roles of domain1 and domain2 are swapped. Node1 becomes the master node and node0 becomes the slave node. The DPC marked in the corresponding area for node1 repeats the operation of "first checking the address cache, and if it is missing, querying the IOB physical address of node1, and then storing it in the MAM according to the rules." Domain0 and domain3 still maintain the (0,1) pairing and are marked as normal service, clearly distinguishing between the mirror pairs that need to be switched and adjusted and the mirror pairs that continue to provide service.

[0103] When node2 joins the cluster, the node composition of domains 1 through 3 is temporarily updated to (1,0,2), (1,0,2), and (0,1,2), resulting in a temporary three-replica state for these domains: one master node and two slave nodes. When domain pairs change, the DPC reorganizes the mirror pairs in response to the domain change. The new master node first checks whether the physical addresses of the mirror sub-regions of the two slave nodes are cached. If not, a physical address mapping query process is triggered. Only after the address cache is complete is the reorganization of the mirror pair officially completed. The master node synchronizes data from the mirror sub-region of the newly joined slave node (node2). If the mirror data for the corresponding mirror sub-region has not yet been synchronized, the mirror pair will provide services in a three-replica state. That is, when incremental data arrives, the master node synchronizes the data to both backup mirror regions simultaneously to prevent data incompleteness caused by abnormal node departures during the data synchronization process. After all mirror data is synchronized, the mirror pair switching operation is performed. The CMS adjusts the domain pairs to the optimal combination of (0,1), (1,2), (0,2), (2,0), and performs the mirror pair switching according to the operation rules in the second step.

[0104] like Figure 5A and 5B The figure shows the transition from a temporary three-copy replica to an optimal two-copy replica after the third node (node2) joins the cluster. This can be explained in two stages: In the first phase, node2 joins the initial temporary three-copy state. The node composition of the three domains is temporarily updated to the form of (1,0,2)(1,0,2)(0,1,2). The roles of each domain are clearly defined by the master node and slave node labels. For example, domain1 and domain2 use node1 as the master node and node0 and node2 as the backup nodes; domain3 uses node0 as the master node and node1 and node2 as the backup nodes, and the temporary three-copy state identifier is marked. The new master node node1 first checks whether the physical address of its mirror sub-region is cached in the MAM for the two slave nodes (such as node0 and node2) through the data persistence component DPC. If it is not cached, it points to the slave node through the end-to-end link between nodes, triggering the address query process. After the address cache is completed, the mirror pair reorganization is marked as complete. In addition, the master node points to node2 to synchronize the mirror data. When the data synchronization is not complete, the incremental data is synchronized to the two slave nodes to prevent data loss.

[0105] The second phase is the optimal combination switching phase after mirror data synchronization. The cluster control center (CMS) issues a switch command, domain0 maintains its original configuration, and domains 1 through 3 are updated to the optimal dual-node configuration of (0,1)(1,2)(0,2)(2,0), respectively. The master and backup roles within each domain are relabeled, such as domain1 becomes master node0 and slave node1; domain2 becomes master node1 and slave node2. The temporary three-replica identifier is removed, replacing it with a stable dual-replica identifier. At the same time, the mirror pair switching operation follows the established rules for the second node joining. Switching only needs to be completed for the domains whose composition has changed, while the unchanged domains are still labeled as continuing service.

[0106] When node3 joins the cluster, the node composition of domains 1 through 3 is temporarily updated to (1, 2, 3), (0, 2, 3), and (2, 0, 3), resulting in a temporary three-replica state. Following the procedure for handling temporary three-replica states, the mirror pairs created when the node joins are reorganized. After this is complete, a mirror pair switchover is performed. The CMS adjusts the domains to the optimal combinations of (0, 1), (1, 2), (2, 3), and (3, 0), and switches the mirror pairs according to the rules in step 2. After the switchover is complete, the cluster's stable mirror pairs are established.

[0107] like Figure 6A and Figure 6B The figure shows the process of adding the fourth node (node3) of the cluster to the stable mirror pair of the cluster. The process can be explained in the following two stages: The first phase involves the initial temporary three-replica state and mirror pair reorganization of node3 upon its addition. The nodes node0, node1, node2, and node3 in the three domains (domain1, domain2, and domain3) are temporarily updated to the (1, 2, 3) (0, 2, 3) (2, 0, 3) configuration. Within each domain, the roles are clearly defined using the master and slave node labels. For example, domain1 uses node1 as the master, with nodes2 and node3 as the slaves; domain2 uses node0 as the master, with nodes2 and node3 as the slaves; and domain3 uses node2 as the master, with nodes0 and node3 as the slaves. These nodes are uniformly labeled with the temporary three-replica state. Furthermore, in accordance with the three-replica processing logic used when the third node (node2) joined, the new master node node1 first checks whether the physical addresses of the mirror sub-regions of the two slave nodes (e.g., node2 and node3) are cached in the MAM. If not, the new master node node1 uses the end-to-end link between nodes to point to the corresponding slave nodes, triggering the physical address mapping query process. Once all slave node addresses are cached, the mirror pair reorganization is marked as complete. This clearly states that the core task of this phase is to reuse established processes to complete the basic mirror preparations after the new node joins.

[0108] The second phase involves mirror pair switching and achieving cluster stability. The CMS (cluster control system) issues a switchover command, adjusting the node combination from domain0 to domain3 to a cyclically optimized dual-replica configuration of (0,1)(1,2)(2,3)(3,0). The master and slave node roles within each domain are relabeled, such as domain0 with master node0 and slave node1; domain1 with master node1 and slave node2; domain2 with master node2 and slave node3; and domain3 with master node3 and slave node0. The temporary three-replica identifier is removed and replaced with a stable dual-replica status label. The switchover operation follows the established logic from before (when the second node joins). For example, the master node checks the MAM cache; if missing, it queries the slave node for the node address. The switchover is completed only for the domain whose composition has changed, without interrupting services in other domains. Finally, the cluster's stable mirror pair is established, indicating that the cluster has built a stable mirror system that balances load balancing and reliability.

[0109] In summary, the data synchronization method provided by this application solves the technical problems of large IO bandwidth and CPU resource consumption and affected system performance of synchronous replication by accurately locating the mirror sub-area and optimizing the data transmission link. In comparison, one of the performance losses of synchronous replication comes from the fact that the master node needs to traverse multiple configuration information to determine the mirror target, which occupies additional CPU resources; while in this method, when the master node responds to the backup data IO request, it directly determines the corresponding mirror pair management structure based on the structure identifier contained in the request, without the need for complex configuration traversal; at the same time, based on the preset mirror pairing rules in the structure, the mirror relationship of the same-name mirror sub-areas of different nodes is clarified, and the same-name mirror sub-area of ​​the target node is quickly locked, which greatly shortens the positioning time of the mirror target, reduces the resource consumption of the CPU in the configuration query and relationship matching links, and alleviates the CPU overload problem in synchronous replication. Another core issue with synchronous replication is that real-time mirroring relies on a cluster shared network, which can easily lead to network bandwidth congestion. This method uses an end-to-end link between nodes to directly send data transmission instructions to the target node. Data does not need to be transferred through the cluster shared network, and backup data can be directly transmitted to the mirror sub-region of the target node with the same name. Compared with the multi-node data forwarding and shared bandwidth competition that may exist in synchronous replication, the end-to-end link can reduce the transmission delay and bandwidth usage of IO data, avoid the impact of bandwidth congestion on the overall IO performance of the storage system, and make up for the large IO performance loss of synchronous replication.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0111] like Figure 7 As shown, an embodiment of the present application further provides a data synchronization device, the device comprising: Determination module 701 is configured to, in response to a request for accessing backup data, determine a mirror pair management structure based on a structure identifier included in the access request; determine a mirror sub-region of the target node with the same name based on a mirror pairing rule included in the mirror pair management structure; the mirror pairing rule is used to configure a mirror relationship between mirror sub-regions of the same name on different nodes; The sending module 702 is used to send a data transmission instruction to the target node through the target link to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in the mirror sub-area of ​​the target node with the same name; wherein the target link is an end-to-end link between nodes and the target node.

[0112] As an optional implementation provided by an embodiment of the present application, the determination module 701, in response to an access request for backup data, before determining the mirror pair management structure based on the structure identifier included in the access request, is further used to: during the node initialization phase, request a management structure resource from the memory management component through the data persistence component; wherein the management structure resource includes at least one of a cache index identifier management structure, a mirror address management structure, a region management structure, and a mirror pair management structure.

[0113] As an optional implementation provided in an embodiment of the present application, the determination module 701 is also used to divide the mirror area into multiple mirror sub-areas through the client component during the node initialization phase, and the information of the mirror sub-areas includes the cache index identifier and the sub-area capacity size; and register the information of the mirror sub-areas to the data persistence component.

[0114] As an optional implementation provided in an embodiment of the present application, the determination module 701 is specifically used to: obtain a cache index identification management structure through a data persistence component; divide the cache index identification management structure into multiple substructures; and store the information of the mirror sub-area in the corresponding substructure.

[0115] As an optional implementation provided in an embodiment of the present application, the determination module 701 is specifically used to: calculate the size of the space to be stored based on the cache index identifier of the mirror sub-region and the capacity size of the sub-region; and store the information of the mirror sub-region in the buffer space of the sub-structure according to the size of the space to be stored.

[0116] As an optional implementation provided in an embodiment of the present application, the determination module 701 is further used to: query the mirror physical address of the mirror sub-region of the target node with the same name through the target link; obtain the mirror address management structure through the data persistence component; and store the mirror physical address of the mirror sub-region of the target node with the same name in the mirror address management structure.

[0117] As an optional implementation provided by an embodiment of the present application, the determination module 701, in the node initialization phase, after dividing the mirror area into multiple mirror sub-areas through the client component, is also used to: respond to the configuration change event of the domain pair of the cluster control end through the data persistence component, refresh the domain pair configuration information, and initialize the mirror pair management structure of each mirror sub-area; the configuration change event indicates that other nodes have joined the cluster; check whether the mirror physical address of the mirror sub-area with the same name of other nodes is stored in the mirror address management structure; if not, query the mirror physical address of other nodes through the end-to-end link between nodes and other nodes; and store the mirror physical address of other nodes in the buffer space of the mirror address management structure.

[0118] As an optional implementation provided in an embodiment of the present application, the determination module 701 stores the mirror physical addresses of other nodes in the buffer space of the mirror address management structure, specifically for: determining the management table identifier corresponding to the mirror physical address of other nodes based on the node identifier of other nodes; and storing the mirror physical address of other nodes in the buffer space corresponding to the management table identifier.

[0119] As an optional implementation provided by an embodiment of the present application, after determining that the mirror physical address of other nodes is stored in the buffer space of the mirror address management structure, the module 701 is also used to: synchronize backup data to other nodes through the end-to-end link between nodes and other nodes, so that the other nodes store the backup data in the mirror sub-area of ​​the same name of the other nodes.

[0120] As an optional implementation provided by an embodiment of the present application, the sending module 702 is further used to: when the backup data is not synchronized to other nodes, if incremental data is received, synchronize the incremental data to the mirror sub-area of ​​the same name of other nodes.

[0121] As an optional implementation provided in an embodiment of the present application, the determination module 701 is further used to: after the backup data is synchronized to other nodes, respond to the mirror pair switching instruction initiated by the cluster control end, perform a mirror pair switching operation on the mirror sub-area with the same name of other nodes.

[0122] For the description of the features in the embodiment corresponding to the data synchronization device, please refer to the relevant description of the embodiment corresponding to the data synchronization method, and no further details will be given here.

[0123] like Figure 8 As shown, an embodiment of the present application further provides an electronic device, including a memory 801 and a processor 802, wherein the memory 801 stores a computer program, and the processor 802 is configured to run the computer program to execute the steps in any of the above data synchronization method embodiments.

[0124] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data synchronization method embodiments when running.

[0125] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0126] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above data synchronization method embodiments are implemented.

[0127] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned data synchronization method embodiments are implemented.

[0128] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] The above is a detailed introduction to a data synchronization method, device, electronic device, storage medium and program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only applicable to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data synchronization method, characterized in that: include: In response to a request for accessing backup data, determining a mirror pair management structure according to a structure identifier included in the access request; Determining a mirror sub-region of the target node with the same name based on a mirror pairing rule included in the mirror pair management structure; the mirror pairing rule is used to configure a mirror relationship between mirror sub-regions with the same name between different nodes; A data transmission instruction is sent to the target node via a target link to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in the mirror sub-area of ​​the target node with the same name; wherein the target link is an end-to-end link between nodes and the target node.

2. The method according to claim 1, characterized in that Before determining the mirror pair management structure in response to the access request for backup data according to the structure identifier included in the access request, the method further includes: During the node initialization phase, the data persistence component requests the memory management component to manage structure resources; The management structure resource includes at least one of a cache index identification management structure, a mirror address management structure, a region management structure, and a mirror pair management structure.

3. The method according to claim 2, characterized in that The method further comprises: In the node initialization phase, the mirror area is divided into a plurality of mirror sub-areas by the client component, and the information of the mirror sub-areas includes a cache index identifier and a sub-area capacity size; Register the information of the mirror sub-area to the data persistence component.

4. The method according to claim 3, characterized in that The storing the information of the mirror sub-area to the data persistence component includes: Obtaining a cache index identification management structure through the data persistence component; Dividing the cache index identification management structure into multiple substructures; The information of the mirror sub-region is stored in the sub-structure accordingly.

5. The method according to claim 4, characterized in that The storing the information of the mirror sub-region into the sub-structure includes: Calculating the size of the space to be stored according to the cache index identifier of the mirror sub-region and the capacity of the sub-region; According to the size of the space to be stored, the information of the mirror sub-region is stored in the buffer space of the sub-structure.

6. The method according to claim 2, characterized in that The method further comprises: Querying the mirror physical address of the mirror sub-region with the same name of the target node through the target link; Obtaining the mirror address management structure through the data persistence component; The mirror physical address of the mirror sub-region with the same name of the target node is stored in the mirror address management structure.

7. The method according to claim 3, characterized in that In the node initialization phase, after the mirror area is divided into a plurality of mirror sub-areas by the client component, the method further includes: Responding to a domain pair configuration change event on the cluster control side through the data persistence component, the domain pair configuration information is refreshed, and the mirror pair management structure of each mirror sub-region is initialized; the configuration change event indicates that another node has joined the cluster; Check whether the mirror address management structure stores the mirror physical address of the mirror sub-region with the same name of the other node; If not, querying the mirror physical address of the other node through the inter-node end-to-end link with the other node; The mirror physical addresses of the other nodes are stored in the buffer space of the mirror address management structure.

8. The method according to claim 7, characterized in that The storing of the mirror physical addresses of the other nodes into the buffer space of the mirror address management structure includes: Determining, according to the node identifier of the other node, a management table identifier corresponding to the mirror physical address of the other node; The mirror physical addresses of the other nodes are stored in the buffer space corresponding to the management table identifier.

9. The method according to claim 7, characterized in that After storing the mirror physical addresses of the other nodes in the buffer space of the mirror address management structure, the method further includes: The backup data is synchronized to the other node via the inter-node end-to-end link with the other node, so that the other node stores the backup data in the mirror sub-area of ​​the other node with the same name.

10. The method according to claim 9, characterized in that The method further comprises: In the case where the backup data is not synchronized to the other nodes, if incremental data is received, the incremental data is synchronized to the mirror sub-area of ​​the other nodes with the same name.

11. The method according to claim 9, characterized in that The method further comprises: After the backup data is synchronized to the other nodes, in response to a mirror pair switching instruction initiated by the cluster control end, a mirror pair switching operation is performed on the mirror sub-region with the same name of the other nodes.

12. A data synchronization device, characterized in that: include: a determination module, configured to respond to a request for accessing backup data and determine a mirror pair management structure according to a structure identifier included in the access request; Determining a mirror sub-region of the target node with the same name based on a mirror pairing rule included in the mirror pair management structure; the mirror pairing rule is used to configure a mirror relationship between mirror sub-regions with the same name between different nodes; A sending module is used to send a data transmission instruction to the target node through a target link to control the target node to receive the backup data in response to the data transmission instruction and store the backup data in the mirror sub-area of ​​the target node with the same name; wherein the target link is an end-to-end link between nodes and the target node.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the data synchronization method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data synchronization method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data synchronization method according to any one of claims 1 to 11 are implemented.

Images