Cross-cluster active-active construction methods, data writing methods, devices, media and products

Abstract

This invention provides a method for constructing a cross-cluster active-active architecture, a data writing method, a device, a medium, and a product, applied in the field of cloud platform technology. The method uses a storage performance development kit (SDK) to register virtualized logical units created for two clusters as block device abstractions (BADs). Then, it establishes connections between the internal systems of two nodes in different clusters. Finally, it constructs redundant mirrored BADs for the two connected nodes in different BADs and sets them on the corresponding nodes. This allows data to be written to the corresponding BAD based on the data sent by the client and the default data writing path. Therefore, this invention, by leveraging the SSD, ensures compatibility and adaptation between heterogeneous storage resources and user-space input / output stacks. The construction of redundant mirrored BADs ensures real-time data consistency across clusters, achieving standardized, high-performance, and highly reliable cross-cluster active-active operation.

Description

Technical Field

[0001] This invention relates to the field of cloud platform technology, and in particular to a cross-cluster active-active construction method, data writing method, device, medium and product. Background Technology

[0002] As enterprises deepen their digital transformation, data has become a core asset, thus placing increasingly higher demands on the reliability, availability, and performance of storage systems. Active-active storage architecture, by enabling real-time data synchronization and parallel application access between two sites (clusters), provides automatic failover capabilities for site-level failures and is a key technology and solution for ensuring business continuity.

[0003] Currently, traditional active-active storage solutions are mainly divided into two categories: one is based on specific high-end storage hardware, which has the advantage of stability and reliability, but has problems such as high cost, vendor lock-in, and poor scalability; the other is based on data replication solutions at the host layer or application layer (such as asynchronous / synchronous replication based on database log replication or specific file systems). Although this type of solution is more flexible, it usually incurs significant performance loss, increases the processing burden on the host, and makes configuration management and fault handling complex.

[0004] The advent of SPDK (Storage Performance Development Kit) and NVMe-oF (NVMe over Fabric) technologies has opened up new paths for building high-performance software-defined storage. SPDK, with its high-performance user-space I / O (Input / Output Stack) and internal BDEV (Block Device, the block device abstraction layer in the SPDK framework) module, enables unified management of various storage backends. The NVMe-oF protocol extends the high-performance storage access capabilities of NVMe (Non-Volatile Memory express) to the network layer, laying a solid technological foundation for low-latency distributed storage. However, current SPDK-based solutions mostly focus on performance optimization within a single cluster, and there is still a lack of effective solutions for deep integration of SPDK and NVMe-oF capabilities, making it difficult to support the deployment of high-performance, standardized cross-cluster active-active systems under a pure software architecture.

[0005] It is evident that finding a pure software-based, high-performance, and standardized cross-cluster active-active solution is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a cross-cluster active-active construction method, data writing method, device, medium and product, which can solve the problem of deeply integrating the capabilities of SPDK and NVMe-oF, and provide a pure software, high-performance, standardized cross-cluster active-active solution.

[0007] To address the aforementioned technical problems, in one aspect, embodiments of the present invention provide a cross-cluster active-active construction method, comprising:

[0008] Based on the storage performance development kit, the virtualization logical units created by the first cluster are registered as the first device abstraction layer. After the first device abstraction layer is exposed to the second cluster based on the network storage protocol, the virtualization logical units created by the second cluster are registered as the second device abstraction layer. Here, the virtualization logical unit is the block storage resource formed by the first / second cluster through virtualization technology, and the first / second device abstraction layer is the storage resource abstraction instance managed by the storage performance development kit.

[0009] The correspondence between each node in the first cluster and each node in the second cluster is determined so as to establish a connection between the internal systems corresponding to the two nodes in different clusters; wherein the first device abstraction layer is mounted on the internal system corresponding to each node in the first cluster, and the second device abstraction layer is mounted on the internal system corresponding to each node in the second cluster.

[0010] Based on the first / second device abstraction layer, corresponding redundant mirrored device abstraction layers are constructed for the two nodes that establish a connection, and are set in the corresponding nodes respectively, so that the data to be written is written to the corresponding first and second device abstraction layers based on the data to be written sent by the client and the default data writing path.

[0011] In some embodiments, registering the virtualized logical units created by the first cluster as the first device abstraction layer based on the storage performance development kit includes:

[0012] Obtain the virtualized logical unit pre-created in the first cluster;

[0013] The storage performance development kit (SDK) uses the block device abstraction and management module to initiate an access request to the virtualization logical unit so that the SSD can establish a communication connection with the first cluster.

[0014] The storage attributes of virtualized logical units are adapted through the block device abstraction management module;

[0015] Register the adapted virtualization logic unit as the first device abstraction layer.

[0016] In some embodiments, the storage attributes of virtualized logical units are adapted through the block device abstraction management module, including:

[0017] Convert the storage attributes in the virtualization logical unit into the standard attribute format supported by the storage performance development kit.

[0018] In some embodiments, after adapting the storage attributes of the virtualized logical units through the block device abstraction management module, the method further includes:

[0019] Generate block device abstraction layer information compatible with the user-space input / output stack of the storage performance development kit, so that the virtualization logic unit can access the user-space input / output processing flow of the storage performance development kit and be uniformly identified, scheduled and controlled by the block device abstraction management module.

[0020] In some embodiments, registering the adapted virtualization logical unit as the first device abstraction layer includes:

[0021] The block device abstraction management module assigns a unique block device abstraction layer identifier to the adapted virtualization logical unit.

[0022] Write the block device abstraction layer identifier and the corresponding block device abstraction layer information into the block device abstraction layer device list of the storage performance development kit;

[0023] The user-space input / output stack of the storage performance development kit is triggered to initialize the adapted virtualized logic unit, so that the adapted virtualized logic unit has the ability to receive and process input / output requests.

[0024] In some embodiments, exposing the first device abstraction layer to the second cluster based on a network storage protocol includes:

[0025] Start a remote access service based on the network storage protocol in the storage performance development kit;

[0026] The parameter information corresponding to the first device abstraction layer is synchronized to the second cluster through the remote access service;

[0027] A data transmission channel is established between the first cluster and the second cluster through a network storage protocol, so that the second cluster can access the first device abstraction layer through the data transmission channel, thereby exposing the first device abstraction layer to the second cluster.

[0028] In some embodiments, after initiating the remote access service in the storage performance development kit based on a network storage protocol, the method further includes:

[0029] Configure network communication parameters and access permission policies for remote access services; the network communication parameters include: transmission protocol type, port number, queue depth and data block transmission unit size, and the access permission policies include the authentication rules and read / write operation permissions of the second cluster.

[0030] In some embodiments, after exposing the first device abstraction layer to the second cluster based on a network storage protocol, the method further includes:

[0031] The storage performance development kit is used to monitor the access status of the second cluster to the first device abstraction layer in real time.

[0032] When the access status indicates abnormal access, an access permission freeze or data transmission channel disconnection operation is triggered.

[0033] In some embodiments, based on the first / second block device abstraction layer, corresponding redundant mirror block device abstraction layers are constructed for the two nodes establishing a connection, including:

[0034] Retrieve the node device abstraction layers of the two nodes that have established a connection in the first and second device abstraction layers respectively;

[0035] Call the Redundancy-Device module in the Storage Performance Development Kit and create an interface;

[0036] Enter the primitive device parameter list of the two node device abstraction layers into the creation interface;

[0037] Configure the first and second device abstraction layers in mirror mode by creating an interface;

[0038] Based on the primitive device parameter list and mirror mode, a redundant mirror block device abstraction layer is created so that the redundant mirror block device abstraction layer has the characteristic of synchronously writing to the first device abstraction layer and the second device abstraction layer after receiving input and output requests.

[0039] In some embodiments, after configuring the first and second device abstraction layers to mirror mode using the creation interface, the method further includes:

[0040] Configure data dual-write synchronization rules and consistency verification strategies for the first and second device abstraction layers.

[0041] In some embodiments, setting the redundant mirror block device abstraction layer in the corresponding nodes includes:

[0042] According to the set order, it is preferentially set on the corresponding node in the first cluster or the second cluster.

[0043] In some embodiments, it also includes:

[0044] Based on the new requirement of dual-active volumes, a new virtualization logical unit is created on any node of the first cluster, and a new virtualization logical unit is created on any node of the second cluster; the new virtualization logical unit serves as the underlying raw storage resource of the dual-active volume.

[0045] On the other hand, the present invention also provides a data writing method, comprising:

[0046] The system retrieves the data to be written sent by the client and inputs it into the cross-cluster active-active construction method. This allows the system to write the data to be written to the first device abstraction layer corresponding to the first cluster and the second device abstraction layer corresponding to the second cluster in the completed cross-cluster active-active architecture based on the default data writing path.

[0047] In some embodiments, writing the data to be written to the first device abstraction layer corresponding to the first cluster and the second device abstraction layer corresponding to the second cluster based on the default data write path includes:

[0048] Write the data to be written to the redundant mirror block device abstraction layer in any node of the first cluster or the second cluster;

[0049] Based on the node device abstraction layers corresponding to two nodes in the redundant mirror block device abstraction layer, the data to be written is written to the first and second device abstraction layers corresponding to the two node device abstraction layers.

[0050] In some embodiments, it also includes:

[0051] When any node in the first cluster disconnects from the node connected to the second cluster, and both the current node in the first cluster and the current node in the second cluster are continuously communicating with the arbitration service module, then based on the cluster priority policy, either cluster is granted the permission to continue direct write service, and the other cluster is instructed to enter the permission to indirectly write service.

[0052] In some embodiments, it also includes:

[0053] When any node in the first cluster disconnects from the node connected to the second cluster, the communication between the current node in the first cluster and the arbitration service module is interrupted, while the current node in the second cluster continues to communicate with the arbitration service module, the current node in the first cluster will be instructed to enter the indirect write service permission, and the remaining nodes in the first cluster and each node in the second cluster will be granted the direct write / indirect write service permission.

[0054] In some embodiments, it also includes:

[0055] When the first cluster and / or the second cluster are in the degraded operation phase, obtain the data to be written corresponding to the degraded operation phase.

[0056] Based on the range of storage data blocks corresponding to the data to be written, corresponding markings are made in the corresponding dirty bitmap so that when the first cluster and / or the second cluster resume normal operation, the data synchronization process is started, the dirty data blocks marked in the dirty bitmap are read, and the dirty data blocks are written into the corresponding first device abstraction layer and / or second device abstraction layer.

[0057] On the other hand, the present invention also provides an electronic device, comprising:

[0058] Memory, used to store computer programs;

[0059] A processor is used to execute computer programs to implement the steps of the cross-cluster active-active construction method described above, or to implement the steps of the data writing method described above.

[0060] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the cross-cluster dual-active construction method described above, or implements the steps of the data writing method described above.

[0061] On the other hand, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the cross-cluster dual-active construction method described above, or implements the steps of the data writing method described above.

[0062] As can be seen from the above technical solution, in the cross-cluster active-active construction method provided by the present invention, the virtualized logical units created for the two clusters based on the storage performance development kit are registered as block device abstractions (BADs) respectively. Then, a connection is established between the internal systems corresponding to the two nodes in different clusters. Finally, redundant mirrored BADs are constructed for the two connected nodes in different BADs and set in the corresponding nodes respectively, so that the data to be written is written to the corresponding first BAD and second BADs based on the data to be written sent by the client and the default data write path. Therefore, in the process of registering the virtualized logical units created for the second cluster as the second BAD, the present invention exposes the first BAD to the second cluster before registration based on the network storage protocol. This design retains the high-performance characteristics of the user-space input / output stack of the storage performance development kit while achieving secure interoperability and standardized management of cross-cluster storage resources. Registering the first device abstraction layer completes the adaptation of heterogeneous storage resources and user-space input / output stacks, ensuring that the core advantages of low latency and zero copy access to the first cluster's storage are not lost. The "exposing to the second cluster" step essentially establishes an authorized communication channel through network storage protocols, which not only avoids the security risks of the second cluster directly accessing the private resources of the first cluster, but also provides a unified resource description basis for the second cluster's adaptation. Finally, "registering the virtualized logical units created by the second cluster as the second device abstraction layer" enables cross-cluster storage resources to be managed uniformly by the storage performance development kits of both ends, and to support high-performance operations, achieving the dual goals of "local high-performance management and control" and "secure cross-cluster interoperability". In other words, this invention leverages a storage performance development kit to achieve standardized registration of virtualized logical units to the block device abstraction layer. This ensures compatibility and adaptation between heterogeneous storage resources and user-mode input / output stacks, while fully utilizing the advantages of low latency and zero-copy high-performance access. By establishing secure and stable cross-cluster connections for the internal systems of different cluster nodes, physical isolation between clusters is broken, laying a reliable foundation for storage resource interoperability. Furthermore, redundant mirrored block device abstraction layers are constructed on both ends of the block device abstraction layer and deployed on the corresponding nodes. Combined with the default data write path, synchronous writing of data to be written to both ends is achieved. This ensures real-time consistency and high availability of cross-cluster data, while the unified management capabilities of the storage performance development kit support the efficiency and stability of client data writing. This effectively solves the problems of performance loss, data synchronization lag, and decentralized resource management in traditional cross-cluster storage solutions, achieving standardized, high-performance, and highly reliable operation of cross-cluster active-active storage. Attached Figure Description

[0063] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A flowchart illustrating a cross-cluster active-active construction method provided in an embodiment of the present invention;

[0065] Figure 2 This is a diagram of a cross-cluster active-active architecture provided in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the first default data write path provided in an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of the second default data write path provided in an embodiment of the present invention;

[0068] Figure 5 This is a diagram of the organs provided in an embodiment of the present invention;

[0069] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0071] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0072] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Next, we will describe in detail the cross-cluster active-active construction method, data writing method, device, medium and product provided by the embodiments of the present invention.

[0074] Figure 1A flowchart of a cross-cluster active-active construction method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes the following steps:

[0075] S10: Based on the storage performance development kit, the virtualization logical unit created by the first cluster is registered as the first device abstraction layer, and after the first device abstraction layer is exposed to the second cluster based on the network storage protocol, the virtualization logical unit created by the second cluster is registered as the second device abstraction layer.

[0076] S11: Determine the correspondence between each node in the first cluster and each node in the second cluster in order to establish a connection between the internal systems corresponding to the two nodes in different clusters.

[0077] S12: Based on the first / second device abstraction layer, construct corresponding redundant mirror block device abstraction layers for the two nodes that establish the connection, and set them in the corresponding nodes respectively, so that the data to be written can be written to the corresponding first and second device abstraction layers based on the data to be written sent by the client and the default data writing path.

[0078] It should be noted that the virtualization logical unit is the block storage resource formed by the first / second cluster through virtualization technology; the first / second block device abstraction layer is an abstract instance of storage resources managed by the storage performance development kit; the first block device abstraction layer is mounted on the internal system corresponding to each node in the first cluster, and the second device abstraction layer is mounted on the internal system corresponding to each node in the second cluster.

[0079] In an embodiment, such as Figure 2 As shown, the current cross-cluster active-active construction method is specifically applied to scenarios involving one client and two clusters, specifically cluster A and cluster B. Each cluster contains multiple nodes. Figure 2 For example, each cluster consists of three nodes. The first cluster (Cluster-A) includes nodes A1, A2, and A3, while the second cluster (Cluster-B) includes nodes B1, B2, and B3.

[0080] exist Figure 2In the framework shown, the first virtualized logical unit (LUN-A) created by the first cluster (Cluster-A) is registered as the first device abstraction layer (BDEV-A) based on the Storage Performance Development Kit (SPDK). Then, after exposing the first device abstraction layer (BDEV-A) to the second cluster (Cluster-B) through the Network Storage Protocol (NVMe-oF), the second virtualized logical unit (LUN-B) created by the second cluster (Cluster-B) is registered as the second device abstraction layer (BDEV-B).

[0081] Based on this, since both Cluster A and Cluster B contain three nodes, the corresponding relationships between these three nodes are determined. Node A1 corresponds to Node B1, Node A2 corresponds to Node B2, and Node A3 corresponds to Node B3. Furthermore, connections are established between the subsystems of nodes in different clusters; that is, the subsystem of node A1 is connected to the subsystem of node B1, the subsystem of node A2 is connected to the subsystem of node B2, and the subsystem of node A3 is connected to the subsystem of node B3. Meanwhile, the first device abstraction layer (BDEV-A) is mounted on the internal systems of each node in the first cluster (Cluster-A), and the second device abstraction layer (BDEV-B) is mounted on the internal systems of each node in the second cluster (Cluster-B). Since the first device abstraction layer (BDEV-A) and the second device abstraction layer (BDEV-B) are storage resource abstraction instances managed by the Storage Performance Development Kit (SPDK), it is equivalent to the first virtualization logical unit (LUN-A) created by the first cluster (Cluster-A) being mounted on the internal systems of each node in the first cluster (Cluster-A), and the second virtualization logical unit (LUN-B) created by the second cluster (Cluster-B) being mounted on the internal systems of each node in the second cluster (Cluster-B).

[0082] Then, based on the first device abstraction layer (BDEV-A) and the second device abstraction layer (BDEV-B), corresponding redundant mirror block device abstraction layers (Raid-bdev) are constructed for the two nodes establishing the connection, and set in the corresponding nodes respectively. That is, for nodes A1 and B1, based on the first device abstraction layer (BDEV-A) and the second device abstraction layer (BDEV-B), a corresponding first redundant mirror block device abstraction layer (Raid-bdev-1) is set for nodes A1 and B1. This first redundant mirror block device abstraction layer (Raid-bdev-1) includes the node device abstraction layer (Local-BDEV-A1) for node A1 and the node device abstraction layer (Local-BDEV-B1) for node B1; for nodes A2 and B2, based on the first device abstraction layer (BDEV-A) and the second device abstraction layer (BDEV-B), a corresponding second redundant mirror block device abstraction layer (Raid-bdev-A1) is set for nodes A2 and B2. 2) The second redundant mirror block device abstraction layer (Raid-bdev-2) includes the node device abstraction layer (Local-BDEV-A2) corresponding to node A2 and the node device abstraction layer (Local-BDEV-B2) corresponding to node B2; For nodes A3 and B3, based on the first device abstraction layer (BDEV-A) and the second device abstraction layer (BDEV-B), a corresponding third redundant mirror block device abstraction layer (Raid-bdev-3) is set for nodes A3 and B3, which includes the node device abstraction layer (Local-BDEV-A3) corresponding to node A3 and the node device abstraction layer (Local-BDEV-B3) corresponding to node B3.

[0083] In addition, it is easy to understand that each node has a corresponding subsystem-pulic.

[0084] The cross-cluster active-active architecture is constructed in the above steps. Then, the data to be written sent by the client enters the currently completed cross-cluster active-active architecture. Based on the default data writing path, the data to be written is written to the corresponding first device abstraction layer (BDEV-A) and second device abstraction layer (BDEV-B), that is, written to the first virtualization logical unit (LUN-A) corresponding to the first device abstraction layer (BDEV-A) and the second virtualization logical unit (LUN-B) corresponding to the second device abstraction layer (BDEV-B).

[0085] Specifically, when the first default data write path is: Client - Cluster A - Cluster B, such as... Figure 3 As shown (taking a single node A1 and node B1 as an example), the data to be written sent by the client enters the first redundant mirror block device abstraction layer (Raid-bdev-1) in node A1. Then, it is first written to the first virtualization logical unit (LUN-A) based on the node A1 device abstraction layer (Local-BDEV-A1) corresponding to node A1 in the first redundant mirror block device abstraction layer (Raid-bdev-1). Finally, it is written to the second virtualization logical unit (LUN-B) based on the node B1 device abstraction layer (Local-BDEV-B1) corresponding to node B1 in the first redundant mirror block device abstraction layer (Raid-bdev-1).

[0086] When the second default data write path is: Client - Second Cluster (Cluster-B) - First Cluster (Cluster-A), such as Figure 4 As shown (taking a single node A1 and node B1 as an example), the data to be written sent by the client enters the first redundant mirror block device abstraction layer (Raid-bdev-1) in node B1. Then, it is first written to the second virtualization logical unit (LUN-B) based on the node B1 device abstraction layer (Local-BDEV-B1) corresponding to node B1 in the first redundant mirror block device abstraction layer (Raid-bdev-1). Finally, it is written to the first virtualization logical unit (LUN-A) based on the node A1 device abstraction layer (Local-BDEV-A1) corresponding to node A1 in the first redundant mirror block device abstraction layer (Raid-bdev-1).

[0087] The process of writing data to the first virtualization logical unit (LUN-A) and the second virtualization logical unit (LUN-B) also includes a fault handling and recovery mechanism. First, an independent arbitration service is deployed at a third site within the overall architecture. All nodes in the first cluster (Cluster-A) and the second cluster (Cluster-B) periodically send heartbeats to the arbitration service, allowing the arbitration service to determine the status of the first cluster (Cluster-A), the second cluster (Cluster-B), and each node based on the heartbeats.

[0088] like Figure 3 and Figure 4As shown, when the inter-cluster network is interrupted, and neither node A1 nor node B1 is aware of the other, but both can connect to the arbitration service, they will compete for write permissions. Therefore, the arbitration service grants one cluster (e.g., the first cluster (Cluster-A)) the permission to continue directly writing to the service according to a strategy (e.g., first-come, first-served, cluster priority), and instructs the other cluster (e.g., the second cluster (Cluster-B)) to enter the permission to indirectly write to the service, thereby avoiding data conflicts. In other words, when neither node A1 nor node B1 is aware of the other, but both can connect to the arbitration service, their default data write path can be either the first or the second path. For the first path, the first cluster (Cluster-A), i.e., node A1, has the permission to directly write to the service as a client, while the second cluster (Cluster-B), i.e., node B1, has the permission to indirectly write to the service, mainly through the first redundant mirror block device abstraction layer (Raid-bdev-1) in node A1. Regarding the second scenario, the second cluster (Cluster-B), i.e., node B1, has direct write permissions for the client, while the first cluster (Cluster-A), i.e., node A1, has indirect write permissions, primarily through the first redundant mirror block device abstraction layer (Raid-bdev-1) in node B1. When neither node A1 nor node B1 is aware of the other, and node A1 cannot connect to the arbitration service, but node B1 can connect to the arbitration service, then only the second default data write path can be selected.

[0089] However, in practical applications, such as Figure 2As shown, both Cluster A and Cluster B have multiple nodes. Therefore, when any node in Cluster A (e.g., node A1) disconnects from a connected node in Cluster B (e.g., node B1), communication between the current node (node ​​A1) in Cluster A and the arbitration service module is interrupted, while the current node (node ​​B1) in Cluster B continues to communicate with the arbitration service module, the current node (node ​​A1) in Cluster A will be instructed to enter the indirect write service, granting the remaining nodes in Cluster A (nodes A2 and A3) and the nodes in Cluster B (nodes B1, B2, and B3) of Cluster B the right to directly / indirectly write. In other words, the path between nodes A1 and B1 can only default to the second path, but the paths between nodes A2 and B2, and between nodes A3 and B3, can choose either the first or second path. Its sophisticated arbitration mechanism can reduce the overall handover caused by local network failures.

[0090] It's easy to understand that this invention employs Redundant Array of Independent Disks (RAID) technology, meaning that redundant mirrored block device abstraction layers are set up in both connected nodes. This design ensures that in the event of a failure, it can tolerate the failure of one base_bdev (a core concept in the SPDK storage framework, referring to the underlying basic block device, which is the physical or logical foundation for building higher-level virtual block devices). When one base_bdev fails, the other healthy base_bdev can still continue to provide data services, without affecting normal read and write operations for clients, thus achieving the dual-active goal of the storage cluster.

[0091] In addition, a lazy bitmap synchronization mechanism to aid recovery has been added during the data writing process, such as... Figure 5As shown. In the event of a failure, the normally functioning first cluster (Cluster-A) and / or second cluster (Cluster-B) will enter a degraded operation phase. During this phase, a dirty bitmap is started in memory and on disk. All newly written data to be written will have its corresponding data block range on the storage volume marked as 1 in the dirty bitmap. This bitmap is synchronously persisted to disk as it is updated in memory to prevent metadata loss. Once the inter-cluster network recovers, the synchronization process of the first cluster (Cluster-A) and / or the second cluster (Cluster-B) will read the dirty bitmap, only writing the data blocks marked as "dirty" to the first device abstraction layer (BDEV-A) and / or the second device abstraction layer (BDEV-B). Specifically, this means writing to the first virtualization logical unit (LUN-A) corresponding to the first device abstraction layer (BDEV-A) and the second virtualization logical unit (LUN-B) corresponding to the second device abstraction layer (BDEV-B). After synchronization is complete, the dirty bit is cleared, and the system returns to active-active mode. This mechanism avoids a full system scan, greatly reducing recovery time.

[0092] As can be seen from the above technical solution, in the cross-cluster active-active construction method provided by the present invention, the virtualized logical units created for the two clusters based on the storage performance development kit are registered as block device abstractions (BADs) respectively. Then, a connection is established between the internal systems corresponding to the two nodes in different clusters. Finally, redundant mirrored BADs are constructed for the two connected nodes in different BADs and set in the corresponding nodes respectively, so that the data to be written is written to the corresponding first BAD and second BADs based on the data to be written sent by the client and the default data write path. Therefore, in the process of registering the virtualized logical units created for the second cluster as the second BAD, the present invention exposes the first BAD to the second cluster before registration based on the network storage protocol. This design retains the high-performance characteristics of the user-space input / output stack of the storage performance development kit while achieving secure interoperability and standardized management of cross-cluster storage resources. Registering the first device abstraction layer completes the adaptation of heterogeneous storage resources and user-space input / output stacks, ensuring that the core advantages of low latency and zero copy access to the first cluster's storage are not lost. The "exposing to the second cluster" step essentially establishes an authorized communication channel through network storage protocols, which not only avoids the security risks of the second cluster directly accessing the private resources of the first cluster, but also provides a unified resource description basis for the second cluster's adaptation. Finally, "registering the virtualized logical units created by the second cluster as the second device abstraction layer" enables cross-cluster storage resources to be managed uniformly by the storage performance development kits of both ends, and to support high-performance operations, achieving the dual goals of "local high-performance management and control" and "secure cross-cluster interoperability". In other words, this invention leverages a storage performance development kit to achieve standardized registration of virtualized logical units to the block device abstraction layer. This ensures compatibility and adaptation between heterogeneous storage resources and user-mode input / output stacks, while fully utilizing the advantages of low latency and zero-copy high-performance access. By establishing secure and stable cross-cluster connections for the internal systems of different cluster nodes, physical isolation between clusters is broken, laying a reliable foundation for storage resource interoperability. Furthermore, redundant mirrored block device abstraction layers are constructed on both ends of the block device abstraction layer and deployed on the corresponding nodes. Combined with the default data write path, synchronous writing of data to be written to both ends is achieved. This ensures real-time consistency and high availability of cross-cluster data, while the unified management capabilities of the storage performance development kit support the efficiency and stability of client data writing. This effectively solves the problems of performance loss, data synchronization lag, and decentralized resource management in traditional cross-cluster storage solutions, achieving standardized, high-performance, and highly reliable operation of cross-cluster active-active storage.

[0093] In some embodiments, the specific implementation of registering the virtualization logical unit created by the first cluster as the first device abstraction layer based on the storage performance development toolkit in step S10 above is as follows:

[0094] Obtain the virtualized logical unit pre-created in the first cluster;

[0095] The storage performance development kit (SDK) uses the block device abstraction and management module to initiate an access request to the virtualization logical unit so that the SSD can establish a communication connection with the first cluster.

[0096] The storage attributes of virtualized logical units are adapted through the block device abstraction management module;

[0097] Register the adapted virtualization logic unit as the first device abstraction layer.

[0098] The specific implementation method of adapting the storage attributes of virtualized logical units through the block device abstraction management module is as follows: converting the storage attributes in the virtualized logical units into the standard attribute format supported by the storage performance development kit.

[0099] Furthermore, after the storage attributes of the virtualized logical unit are adapted through the block device abstraction management module, the module also includes generating block device abstraction layer information that is compatible with the user-mode input / output stack of the storage performance development kit, so that the virtualized logical unit can access the user-mode input / output processing flow of the storage performance development kit and be uniformly identified, scheduled and controlled by the block device abstraction management module.

[0100] The specific implementation method for registering the adapted virtualization logic unit as the first device abstraction layer is as follows:

[0101] The block device abstraction management module assigns a unique block device abstraction layer identifier to the adapted virtualization logical unit.

[0102] Write the block device abstraction layer identifier and the corresponding block device abstraction layer information into the block device abstraction layer device list of the storage performance development kit;

[0103] The user-space input / output stack of the storage performance development kit is triggered to initialize the adapted virtualized logic unit, so that the adapted virtualized logic unit has the ability to receive and process input / output requests.

[0104] In a specific embodiment, the process of registering the virtualized logical unit created by the first cluster (Cluster-A) as the first device abstraction layer (LUN-A) includes five steps: the first step is to obtain all the native attributes of the virtualized logical unit; the second step is to load the adaptation plugin and standardize the attribute conversion; the third step is to enter the device information and mark the status; the fourth step is to perform multi-dimensional availability verification; and the fifth step is to complete the registration and access the resources.

[0105] For the first step, the first cluster (Cluster-A) pre-creates the corresponding first virtualization logical unit (LUN-A), and then obtains all the native attribute information of the first virtualization logical unit (LUN-A). Its native attribute information includes two categories: basic parameters and specific characteristic parameters. Basic parameters include the total storage capacity of the first virtualization logical unit (LUN-A), the minimum data block read / write granularity, the maximum input / output queue depth, read / write permission control policies, and the access path of the physical storage medium. Specific characteristic parameters include whether snapshot functionality is supported, the data redundancy protection level, the reserved metadata partition size, and input / output request priority rules. By obtaining all attributes, a comprehensive understanding of the characteristics of the underlying storage resources is ensured, laying the foundation for subsequent standardized adaptation.

[0106] For the second step, the block device abstraction management module registration interface of the Storage Performance Development Kit (SPDK) is invoked, and a custom attribute adaptation plugin developed for the current first virtualization logical unit (LUN-A) is loaded. Then, after the Storage Performance Development Kit (SPDK) establishes a communication connection with the first cluster (Cluster-A), the adaptation plugin initiates a standardization conversion process, mapping the non-standard native attributes of the first virtualization logical unit (LUN-A) one by one to the standard format supported by the user-space input / output stack of the Storage Performance Development Kit (SPDK). The specific conversion rules are as follows: The custom data block size of the first virtualization logical unit (LUN-A) is mapped to the standard block size supported by the Storage Performance Development Kit (SPDK); the kernel-mode system call access interface of the first virtualization logical unit (LUN-A) is converted to the standardized user-mode read / write interface of the Storage Performance Development Kit (SPDK); the custom permission policy of the first virtualization logical unit (LUN-A) is converted to three permission models recognizable by the Storage Performance Development Kit (SPDK): read-only, read-write, and no access; the custom input / output queue model of the first virtualization logical unit (LUN-A) is adapted to the multi-queue, asynchronous input / output model supported by the Storage Performance Development Kit (SPDK). Based on the above conversion results, complete and corresponding block device abstraction layer information is generated, which includes a device unique identifier field, attribute adaptation result field, interface call rule field, performance benchmark parameter field, etc. Among them, the design of generating complete and corresponding block device abstraction layer information is so that the corresponding first virtualization logical unit (LUN-A) can access the user-space input / output processing flow of the storage performance development kit (SPDK) and be uniformly identified, scheduled and controlled by the block device abstraction management module.

[0107] In the third step, the generated block device abstraction layer information is entered into the global block device abstraction management list of the Storage Performance Development Kit (SPDK). Simultaneously, a block device abstraction layer identifier (which can also be understood as a globally unique device number) is assigned to the first virtualization logical unit (LUN-A) to facilitate unified identification and scheduling by the SPDK. After the entry is completed, the SPDK marks the initial state of the first virtualization logical unit (LUN-A) as pending verification and synchronizes the block device abstraction layer identifier and state information to the cluster storage management system, achieving visualization of resource status at the cluster level.

[0108] For the fourth step, the built-in verification module of the Storage Performance Development Kit (SPDK) triggers the availability verification process, performing four core checks sequentially according to preset rules: Attribute format compliance check: Verifying whether the converted attributes fully comply with the SPDK specifications, checking for issues such as missing fields and format errors; Local access path connectivity check: Testing the communication link with the underlying virtualization logical unit by simulating read and write requests to ensure uninterrupted data transmission; Read and write permission validity check: Simulating access requests at different permission levels to verify whether they can be correctly responded to or intercepted, ensuring that the permission control policy is effective; Performance benchmark compliance check: Testing whether input and output latency and throughput meet preset performance thresholds to ensure that it can leverage the high-performance advantages of the SPDK. Only when all verification items pass will the status of the first virtualization logical unit (LUN-A) be updated to available; if any verification item fails, specific error information is returned, and the attribute re-conversion process is triggered.

[0109] In the fifth step, after completing all verification processes, the first virtualization logical unit (LUN-A) is officially converted and registered as the first device abstraction layer (BDEV-A). At this point, the first device abstraction layer (BDEV-A) is fully integrated into the user-space input / output processing flow of the Storage Performance Development Kit (SPDK). All input / output requests for this device do not need to be scheduled by the operating system kernel; data transmission is completed directly in user space, fully leveraging the core performance advantages of the SPDK: zero copy and low latency. Simultaneously, the first device abstraction layer (BDEV-A) is incorporated into the unified management system of the SPDK, supporting its use by subsequent redundant device modules, providing a standardized, high-performance storage resource carrier for cross-cluster connection establishment and redundant mirror block device abstraction layer construction.

[0110] It should be noted that the embodiments provided by the present invention are only one possible implementation method, but are not limited to this only implementation method. Users can set their own implementation methods according to their needs.

[0111] This invention provides a specific implementation method for registering the virtualized logical unit created by the first cluster as the first block device abstraction layer. Through a complete process of full attribute acquisition, standardized conversion, and multi-dimensional verification, it achieves deep adaptation between heterogeneous virtualized logical units and the user-space input / output stack of the storage performance development kit. This not only retains the extreme performance advantages of the storage performance development kit, but also ensures the availability and standardization of the block device abstraction layer, providing reliable underlying resource support for the construction of cross-cluster dual-active storage systems.

[0112] In some embodiments, the specific implementation of exposing the first device abstraction layer to the second cluster based on the network storage protocol in step S10 above is as follows:

[0113] Start a remote access service based on the network storage protocol in the storage performance development kit;

[0114] The parameter information corresponding to the first device abstraction layer is synchronized to the second cluster through the remote access service;

[0115] A data transmission channel is established between the first cluster and the second cluster through a network storage protocol, so that the second cluster can access the first device abstraction layer through the data transmission channel, thereby exposing the first device abstraction layer to the second cluster.

[0116] After starting the remote access service in the storage performance development kit based on the network storage protocol, the process also includes configuring network communication parameters and access permission policies for the remote access service. The network communication parameters include the transmission protocol type, port number, queue depth, and data block transmission unit size. The access permission policies include the authentication rules and read / write operation permissions of the second cluster.

[0117] Furthermore, after exposing the first device abstraction layer to the second cluster based on the network storage protocol, the system also includes: monitoring the access status of the second cluster to the first device abstraction layer in real time through a storage performance development kit; and triggering access permission freezing or data transmission channel disconnection when the access status indicates abnormal access.

[0118] In this embodiment, exposing the first device abstraction layer to the second cluster based on the network storage protocol includes three steps: the first step is pre-exposure preparation; the second step is standardized data synchronization; and the third step is exposure service startup and accessibility diagnosis.

[0119] For the first step, the Block Device Abstraction Management module of the Storage Performance Development Kit (SPDK) is invoked to reconfirm that the status of the first Block Device Abstraction Layer (BDEV-A) is "available." Simultaneously, its core attributes, such as storage capacity, data block size, and input / output performance, are verified to be normal, ensuring the exposed resource has stable service capabilities. If the status is abnormal, the repair mechanism of the aforementioned registration process is triggered until the resource is restored to availability. At the same time, network storage protocol parameters and internal subsystem association configurations are also required. The network storage protocol parameter configuration includes: setting the transport layer to remote direct memory access, configuring the target network port (e.g., port number 4420), setting the maximum number of concurrent connections (preset to 64-256 based on the cluster's business scale), configuring the authentication policy (using the CHAP (Challenge Handshake Authentication Protocol) authentication mechanism, with preset authentication username and key), and binding the physical network card port to ensure the exclusivity and stability of the network communication link. The internal subsystem association configuration configures the access control rules for the internal subsystems. This is mainly to allow designated nodes (such as node B1) of the second cluster (Cluster-B) to access the first device abstraction layer (BDEV-A) through the configured authentication information, while rejecting access requests from unauthorized nodes to ensure resource security.

[0120] The second step involves extracting parameter information from the first device abstraction layer (BDEV-A), which includes the device's unique identifier, standard data block size, storage capacity, read / write permissions, interface call rules, performance benchmark parameters, etc. This information is then converted into a standardized metadata format supported by the Network Storage Protocol (NVMe-oF) using an adapter plugin, ensuring the second cluster (Cluster-B) can accurately identify resource characteristics. The standardized metadata is then pushed to the second cluster (Cluster-B) via a pre-established inter-cluster control channel. Upon receiving the metadata, the second cluster (Cluster-B) verifies its format compliance and completeness. If verification passes, it returns a "successful reception" confirmation signal; if verification fails, it reports specific error fields and regenerates and pushes the metadata again until synchronization is complete.

[0121] In the third step, the remote access service initiated based on the Network Storage Protocol (NVMe-oF) exposes the access point of the first Device Abstraction Layer (BDEV-A) to the outside world. Simultaneously, the exposure status is synchronized to the storage management system of the first cluster (Cluster-A), marked as "Exposed - Awaiting Access". Meanwhile, the first cluster (Cluster-A) responds to the access request from the second cluster (Cluster-B), completes CHAP authentication, and establishes a data transmission channel. Then, by simulating the reading of a small amount of test data from the first Device Abstraction Layer (BDEV-A), the transmission rate, latency, and data integrity of the data transmission channel are verified to ensure its availability. After the test passes, the exposure status of the first Device Abstraction Layer (BDEV-A) is updated to "Exposed - Accessible", completing the entire exposure process.

[0122] Furthermore, after the exposure process is completed, the access status of the second cluster (Cluster-B) to the first device abstraction layer (BDEV-A) is monitored in real time; when the access status indicates abnormal access, the access permission is frozen or the data transmission channel is disconnected to ensure the security of access.

[0123] It is easy to understand that after exposing the first device abstraction layer (BDEV-A) to the second cluster (Cluster-B) based on the network storage protocol, the second cluster (Cluster-B) creates a corresponding second virtualization logical unit (LUN-B) and registers it as the second device abstraction layer (BDEV-B). The implementation of registering the second virtualization logical unit (LUN-B) of the second cluster (Cluster-B) as the second device abstraction layer (BDEV-B) is the same as the implementation of registering the first virtualization logical unit (LUN-A) of the first cluster (Cluster-A) as the first device abstraction layer (BDEV-A), so it will not be elaborated further here.

[0124] It should be noted that the embodiments provided by the present invention are only one possible implementation method, but are not limited to this only implementation method. Users can set their own implementation methods according to their needs.

[0125] This invention provides a specific implementation method for exposing a first device abstraction layer (DIP) to a second cluster based on a network storage protocol. This method employs a network storage protocol to achieve low-latency, high-bandwidth exposure of the first DIP across clusters, maximizing the high-performance advantages of the user-space input / output stack of the storage performance development toolkit while avoiding the performance degradation of traditional network storage protocols. Furthermore, through pre-processing state verification, identity authentication configuration, and access control rule settings, a multi-layered security protection system is constructed while breaking down physical isolation between clusters, effectively mitigating the risks of unauthorized access and data leakage. A standardized metadata synchronization mechanism ensures that the second cluster accurately understands the characteristics of the first DIP, providing a unified and standardized resource description basis for the subsequent registration of the remote block device abstraction layer by the second cluster. Simultaneously, the entire exposure process seamlessly integrates with the aforementioned first DIP registration process, and connectivity testing ensures the accessibility of exposed resources, laying a stable and reliable foundation for resource interoperability in the construction of redundant mirrored block device abstractions across clusters. Ultimately, this supports the high-performance and high-availability operation of the cross-cluster active-active storage system.

[0126] In some embodiments, the specific implementation of step S12, which involves constructing corresponding redundant mirror block device abstraction layers for the two nodes establishing a connection based on the first / second block device abstraction layers, is as follows:

[0127] Retrieve the node device abstraction layers of the two nodes that have established a connection in the first and second device abstraction layers respectively;

[0128] Call the Redundancy-Device module in the Storage Performance Development Kit and create an interface;

[0129] Enter the primitive device parameter list of the two node device abstraction layers into the creation interface;

[0130] Configure the first and second device abstraction layers in mirror mode by creating an interface;

[0131] Based on the primitive device parameter list and mirror mode, a redundant mirror block device abstraction layer is created so that the redundant mirror block device abstraction layer has the characteristic of synchronously writing to the first device abstraction layer and the second device abstraction layer after receiving input and output requests.

[0132] The method includes configuring the first and second device abstraction layers in mirror mode after creating the interface, and also includes configuring data dual-write synchronization rules and consistency verification strategies for the first and second device abstraction layers.

[0133] The redundant image block device abstraction layer is set in the corresponding nodes, including: according to the set order, it is preferentially set in the corresponding nodes in the first cluster or the second cluster.

[0134] In this embodiment, the implementation is based on the first / second device abstraction layer. The construction of corresponding redundant mirror block device abstraction layers for the two nodes establishing the connection includes four steps: the first step is: precondition verification; the second step is: creating the node device abstraction layer; the third step is: creating the corresponding redundant mirror block device abstraction layer; and the fourth step is: two-end linkage test.

[0135] In the first step, the two connected nodes respectively call the Block Device Abstraction Management module of the Storage Performance Development Kit (SPDK) to verify that the status of the Block Device Abstraction Layer corresponding to both nodes is "available." Simultaneously, they confirm that core attributes are consistent, mainly including data block size, storage capacity, and read / write permissions, to avoid image build failure due to attribute differences. If attribute inconsistencies exist, an attribute adaptation and repair process is triggered until the attributes on both ends are unified. Then, a cross-cluster connection status verification is performed to verify that the communication channel between the two nodes is unobstructed and meets the image synchronization requirements. At the same time, the authentication status is verified to be "passed," ensuring the security and stability of the connection. Finally, initialization is performed to ensure image build and synchronization capabilities are available.

[0136] The second step involves calling the Redundancy-Device module in the Storage Performance Development Kit (SPDK) and creating an interface. Then, the node in the first cluster (Cluster-A) (e.g., node A1) corresponding to the node device abstraction layer (Local-BDEV-A1) in the first device abstraction layer (BDEV-A), and the node in the second cluster (Cluster-B) (e.g., node B1) corresponding to the node device abstraction layer (Local-BDEV-B1) in the second device abstraction layer (BDEV-B), are entered into the primitive device parameter list of the creation interface. A unique identifier is assigned to each of these primitive devices for subsequent module management and scheduling. Simultaneously, the interface is used to configure the first device abstraction layer (BDEV-A) and the second device abstraction layer (BDEV-B) in mirror mode and to set data dual-write synchronization rules and consistency verification strategies. Specifically, this includes: 1. Dual-write rule: All input / output requests must be written synchronously to two primitive devices, and a "write complete" signal is only returned to the business layer after both ends have successfully written; 2. Synchronization priority rule: The write operation of the first device abstraction layer (BDEV-A) has a higher priority than the second device abstraction layer (BDEV-B) to ensure low latency for business access; 3. Timeout retry rule: The second device abstraction layer (BDEV-B) automatically retryes after a timeout (timeout threshold set to 100μs), and a fault alarm is triggered if the retry fails after 3 attempts; 4. Real-time verification: CRC (Cyclic Redundancy Check 32) checksum is calculated for each piece of data to be written, and the checksums are compared after both ends have been written to ensure data consistency; 5. Periodic verification: A full data verification is triggered every hour, comparing the checksums of all data blocks at both ends, and an automatic repair process is initiated when inconsistent data is found.

[0137] The third step involves calling the Redundancy-Device module in the Storage Performance Development Kit (SPDK) to generate corresponding description information (including a list of primitive devices, synchronization rules, and verification strategies). The device abstraction layers for the two nodes (Local-BDEV-A1 and Local-BDEV-B1) are entered into the primitive device parameter list of the creation interface. Then, based on the primitive device parameter list and the mirror mode, a redundant mirror block device abstraction layer is created. It should be noted that this is prioritized on the corresponding nodes in either the first cluster (Cluster-A) or the second cluster (Cluster-B). In other words, there is a specific order in which the corresponding redundant mirror block device abstraction layers are set on different nodes.

[0138] For the fourth step, the two corresponding nodes simulate business input and output requests to verify whether the redundant mirrored block device abstraction layer can achieve data dual-write synchronization, and whether the data on both primitive devices is completely consistent after the write is completed. Next, a single primitive device failure is simulated (e.g., disabling the first device abstraction layer of node A1) to verify that the mirrored device can automatically switch to the backup device to handle requests alone, and that incremental synchronization can be automatically started after the failure recovers. Simultaneously, the input / output latency and throughput at both ends can be tested to ensure that the performance meets the design standards of the Storage Performance Development Kit (SPDK) without significant performance loss. After the tests are passed, the entire construction process of the redundant mirrored block device abstraction layer is complete.

[0139] It should be noted that the embodiments provided by the present invention are only one possible implementation method, but are not limited to this only implementation method. Users can set their own implementation methods according to their needs.

[0140] This invention provides a specific implementation method for constructing corresponding redundant mirror block device abstraction layers for two nodes establishing a connection, based on a first / second device abstraction layer. This method achieves active-active mirrored storage of cross-cluster data by constructing symmetrical redundant mirror block device abstraction layers for the two cross-cluster nodes establishing a connection. This ensures that business data is not lost and services are not interrupted in the event of a failure of any node or cluster, significantly improving the high availability of the storage system. Furthermore, the mirroring based on the redundant-device module of the storage performance development kit retains the low latency and zero-copy advantages of the user-space input / output stack. Combined with mirror synchronization priority rules, this further guarantees high performance for business access. Simultaneously, configuring dual data consistency verification mechanisms and failover rules effectively solves the consistency and reliability issues of cross-cluster data synchronization. The symmetrical configuration and information exchange between the two mirrored devices lay the foundation for subsequent cross-cluster load-balanced access to business data and seamless failover, ultimately supporting the stable and efficient operation of the cross-cluster active-active storage system.

[0141] In addition, the cross-cluster active-active construction method provided by this invention can also respond to the new demand for active-active volumes. Specifically, based on the new demand for active-active volumes, a new virtualization logical unit is created on any node of the first cluster (Cluster-A), and a new virtualization logical unit is created on any node of the second cluster (Cluster-B); wherein, the new virtualization logical unit serves as the underlying raw storage resource of the active-active volume. The subsequent steps are the same as those described above, and will not be repeated here.

[0142] In summary, the cross-cluster active-active construction method is specifically divided into two embodiments. The first embodiment is the construction process of the cross-cluster active-active architecture, and the second embodiment is the data writing process in the cross-cluster active-active construction method.

[0143] Specifically, the first embodiment includes six steps, as follows:

[0144] The first step: Register the virtual volume as a local BDEV. On node A1 of the first cluster (Cluster-A), a specific SPDK BDEV plugin is developed or called. This plugin calls interfaces such as spdk_bdev_register() to register the storage cluster's virtual volume as a block device BDEV within the SPDK framework, denoted as Local-BDEV-A1 (the node device abstraction layer corresponding to node A1). Similarly, on node B1 of the second cluster (Cluster-A), a peer virtual volume is mounted as a block device BDEV and registered as Local-BDEV-B1 (the node device abstraction layer corresponding to node B1). Then, the above operations are performed on all nodes and their corresponding nodes in both clusters to ensure that the resources of each node in both clusters are symmetrical with those of its corresponding node.

[0145] The second step: Export the first Device Abstraction Layer (BDEV-A) as the target end via the NVMe-oF protocol. On node A1 of the first cluster (Cluster-A), call SPDK to create an internal system, add Local-BDEV-A1 to the corresponding internal system, and add the specified network interface card (NIC) IP address as the exposed port. You can specify whether to use RDMA (Remote Direct Memory Access) or TCP (Transmission Control Protocol). Similarly, on node B1 of the second cluster (Cluster-B), call SPDK to create an internal system, add the corresponding Local-BDEV-B1 to the internal system, and add the specified NIC IP address as the exposed port. You can specify whether to use RDMA or TCP.

[0146] Perform the same operation on the other nodes of the first cluster (Cluster-A) and the other nodes of the corresponding second cluster (Cluster-B).

[0147] The third step: Connecting both ends. SPDK has the ability to directly connect to remote NVMe devices. By specifying the TCP / RDMA protocol, the remote node IP, and the port, it connects to the remote NVMe device as a local BDEV device. Specifically, on node A1 of the first cluster (Cluster-A), by calling the internal system connected to node B1 of the second cluster (Cluster-B), Local-BDEV-B1 corresponding to node B1 in the second cluster (Cluster-B) is added as Remote-BDEV-B1 (the remote device from node A1's perspective), and the corresponding internal network connection is specified to avoid impacting client performance. On node B1 of the second cluster (Cluster-B), by calling the internal system connected to node A1 of the first cluster (Cluster-A), Local-BDEV-A1 corresponding to node A1 in the first cluster (Cluster-A) is added as Remote-BDEV-A1 (the remote device from node B1's perspective), and the corresponding internal network connection is specified to avoid impacting client performance. At this point, node A1 in the first cluster (Cluster-A) and node B1 in the second cluster (Cluster-B) can both see the LUN device created by the peer cluster. Similarly, each node in the first cluster (Cluster-A) and the second cluster (Cluster-B) performs the above steps to connect to the peer's BDEV device.

[0148] The fourth step: Building a cross-cluster RAID mirror: Node A1 calls the SPDK's redundancy-device module (RAID BDEV module, e.g., spdk_bdev_raid_create) to create a RAID-type virtual block device (first redundant mirror block device abstraction layer) using Local-BDEV-A1 and Remote-BDEV-B1 as two primitive devices (base_bdev), denoted as Raid-bdev-1. This RAID module ensures that all data written to Raid-bdev-1 is synchronously written to both Local-BDEV-A1 and Remote-BDEV-A1. A symmetric operation is performed on node B1 to create an identical Raid-bdev-1.

[0149] Similarly, each node in the first cluster (Cluster-A) and the second cluster (Cluster-B) performs the above steps, and each node can see the Raid-bdev.

[0150] Furthermore, during the creation of Raid-bdev, a super_block (a core data structure in the storage system used to store global metadata, usually located at the beginning of a volume or device) containing metadata such as RAID level, stripe size, member list, and bitmap is written to the beginning of the volume. The creation process will first check this area. If it finds that RAID information consistent with the current request already exists, it is determined to be a duplicate creation and the write operation is skipped. This ensures that when the first cluster (Cluster-A) completes RAID creation first, the second cluster (Cluster-B) will not overwrite the existing metadata and bitmap when performing the same configuration creation on the same set of underlying block devices, thus avoiding damage to the running array and achieving idempotency protection and data consistency across cluster environments.

[0151] The fifth step: Provide Raid-bdev services to the outside world through SPDK. The first cluster (Cluster-A) and the second cluster (Cluster-B) each create a system that provides IO and is independent of internal synchronization, which can be mounted by clients. At the same time, clients can mount two clusters. Multi-path technology is used to aggregate multiple paths, and in the event of a failure, the path is automatically switched to continue providing IO.

[0152] The sixth step: When adding new volumes, both clusters create corresponding LUNs, register them as their respective BDEV devices, and mount them onto the internal systems. Each cluster then creates a Raid-bdev device. Finally, depending on requirements, these devices are mapped to new external targets or existing targets.

[0153] The second embodiment specifically includes two steps, as follows:

[0154] The first step: Data input. Obtain the data to be written sent by the client and input it into the cross-cluster active-active architecture described above. This ensures that, in the completed cross-cluster active-active architecture, the data is written to the first device abstraction layer (BDEV-A) corresponding to the first cluster (Cluster-A) and the second device abstraction layer (BDEV-B) corresponding to the second cluster (Cluster-B) based on the default data write path. In other words, it is written to the first virtualization logical unit (LUN-A) and the second virtualization logical unit (LUN-B).

[0155] The second step: fault handling and recovery, which involves introducing arbitration and lazy synchronization mechanisms during the data writing process. This mechanism includes three aspects:

[0156] First, the arbitration mechanism prevents split-brain attacks. Its specific implementation is as follows: An independent arbitration service is deployed at a third site within the overall architecture. All nodes in the first cluster (Cluster-A) and the second cluster (Cluster-B) periodically send heartbeats to the arbitration service, allowing the service to determine the status of the first cluster (Cluster-A), the second cluster (Cluster-B), and each node based on these heartbeats. When any node in the first cluster (Cluster-A) disconnects from its connected node in the second cluster (Cluster-B), and both the current node in the first cluster (Cluster-A) and the current node in the second cluster (Cluster-B) are continuously communicating with the arbitration service module, then, based on a cluster priority policy, either cluster is granted the right to continue directly writing to the service, while the other cluster is instructed to enter the right to indirectly write to the service, thereby avoiding data conflicts. When any node in Cluster A loses connection with a node in Cluster B, communication between the current node in Cluster A and the arbitration service module is interrupted, while communication between the current node in Cluster B and the arbitration service module continues, the current node in Cluster A will be instructed to enter the indirect write service, and the remaining nodes in Cluster A and each node in Cluster B will be granted the direct / indirect write service. This sophisticated arbitration mechanism reduces the overall failover caused by local network failures.

[0157] Secondly, it has fault tolerance, meaning it can tolerate the failure of one base_bdev. When one base_bdev fails, the other normal base_bdev can still continue to provide data services without affecting the normal read and write operations of the client, thus achieving the dual-active goal of the storage cluster.

[0158] Thirdly: Lazy bitmap synchronization aids recovery. In the event of a failure, the normally functioning first cluster (Cluster-A) and / or second cluster (Cluster-B) will enter a degraded operation phase. During this phase, a dirty bitmap is initiated in memory and on disk. All newly written data to be written will have its corresponding data block range on the storage volume marked as 1 in the dirty bitmap. This bitmap is synchronously persisted to disk as it is updated in memory to prevent metadata loss. Once the inter-cluster network is restored, the synchronization process of the first cluster (Cluster-A) and / or the second cluster (Cluster-B) will read the dirty bitmap, only retrieving those data blocks marked as "dirty," and writing them to the first device abstraction layer (BDEV-A) and / or the second device abstraction layer (BDEV-B). Specifically, this means writing them to the first virtualization logical unit (LUN-A) corresponding to the first device abstraction layer (BDEV-A) and the second virtualization logical unit (LUN-B) corresponding to the second device abstraction layer (BDEV-B). After synchronization is complete, the dirty bit is cleared, and the system returns to active-active mode. This mechanism avoids a full system scan, greatly reducing recovery time.

[0159] Therefore, the cross-cluster active-active construction method provided by this invention has the following advantages:

[0160] 1. High performance: Fully utilizes the user-space, lock-free, polling I / O architecture of SPDK and the low-latency characteristics of NVMe-oF to provide I / O performance close to that of local NVMe hard drives for dual-active storage.

[0161] 2. Strong consistency: By using the SPDK RAID module to synchronize at the block level and combining it with a fine-grained arbitration mechanism, the data split-brain caused by network partitions is effectively avoided, ensuring strong data consistency.

[0162] 3. Fast recovery: Through the lazy bitmap mechanism, incremental synchronization after fault recovery is achieved, which greatly reduces recovery time and cross-cluster network bandwidth consumption.

[0163] 4. Flexible and decoupled architecture: Through the BDEV plugin framework, the backend can be adapted to various distributed storage systems, and the dual-active logic of the frontend is implemented by SPDK, which is decoupled from the storage engine itself, enhancing the versatility and scalability of the solution.

[0164] 5. Pure software implementation: Built on open source components (SPDK), avoiding dependence on specific hardware and reducing costs and deployment complexity.

[0165] On the other hand, the present invention also provides a data writing method, comprising:

[0166] The data to be written sent by the client is obtained and input into the cross-cluster active-active construction method described above, so that in the completed cross-cluster active-active architecture, the data to be written is written to the first device abstraction layer corresponding to the first cluster and the second device abstraction layer corresponding to the second cluster based on the default data writing path.

[0167] This involves writing the data to be written to the first device abstraction layer corresponding to the first cluster and the second device abstraction layer corresponding to the second cluster, based on the default data write path.

[0168] Write the data to be written to the redundant mirror block device abstraction layer in any node of the first cluster or the second cluster;

[0169] Based on the node device abstraction layers corresponding to two nodes in the redundant mirror block device abstraction layer, the data to be written is written to the first and second device abstraction layers corresponding to the two node device abstraction layers.

[0170] The method also includes:

[0171] When any node in the first cluster is disconnected from the node connected to the second cluster, and both the current node in the first cluster and the current node in the second cluster are continuously communicating with the arbitration service module, then based on the cluster priority policy, either cluster is granted the permission to continue direct write service, and the other cluster is instructed to enter the permission to indirectly write service.

[0172] When any node in the first cluster disconnects from the node connected to the second cluster, the communication between the current node in the first cluster and the arbitration service module is interrupted, while the current node in the second cluster continues to communicate with the arbitration service module, the current node in the first cluster will be instructed to enter the indirect write service permission, and the other nodes in the first cluster and each node in the second cluster will be granted the direct write / indirect write service permission.

[0173] When the first cluster and / or the second cluster are in the degraded operation phase, the data to be written corresponding to the degraded operation phase is obtained; based on the storage data block range corresponding to the data to be written, the corresponding dirty bitmap is marked accordingly, so that when the first cluster and / or the second cluster resumes the normal operation phase, the data synchronization process is started, the dirty data blocks marked in the dirty bitmap are read, and the dirty data blocks are written into the corresponding first device abstraction layer and / or second device abstraction layer.

[0174] Since the embodiments corresponding to the data writing method are the same as those provided above, the present invention will not be described in detail here.

[0175] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 6As shown, the electronic device includes: a memory 60 for storing computer programs;

[0176] The processor 61 is used to execute a computer program to implement the steps of the cross-cluster dual-active construction method as described in the above embodiments, or to implement the steps of the data writing method described above.

[0177] The electronic devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0178] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 61 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 61 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0179] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 60 is used to store at least the following computer program 601, which, after being loaded and executed by the processor 61, is capable of implementing the relevant steps of the cross-cluster active-active construction method or data writing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. The operating system 602 may include Windows, Unix, Linux, etc.

[0180] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64, a power supply 65, and a communication bus 66.

[0181] Those skilled in the art will understand that Figure 6 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0182] It is understood that if the cross-cluster active-active construction method or data writing method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk, and other media capable of storing program code.

[0183] Based on this, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the cross-cluster dual-active construction method described above, or implements the steps of the data writing method described above.

[0184] Based on this, embodiments of the present invention also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the cross-cluster dual-active construction method described above, or implements the steps of the data writing method described above.

[0185] The foregoing has provided a detailed description of a cross-cluster active-active construction method, data writing method, device, medium, and product provided by embodiments of the present invention. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0186] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0187] The present invention has provided a detailed description of a cross-cluster active-active construction method, data writing method, device, medium, and product. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for constructing a cross-cluster active-active architecture, characterized in that, include: Based on the storage performance development kit, the virtualization logical units created by the first cluster are registered as the first device abstraction layer. After the first device abstraction layer is exposed to the second cluster based on the network storage protocol, the virtualization logical units created by the second cluster are registered as the second device abstraction layer. The virtualization logical units are block storage resources formed by the first / second cluster through virtualization technology, and the first / second device abstraction layers are storage resource abstraction instances managed by the storage performance development kit. The correspondence between each node in the first cluster and each node in the second cluster is determined so as to establish a connection between the internal systems corresponding to the two nodes in different clusters; wherein the first device abstraction layer is mounted on the internal system corresponding to each node in the first cluster, and the second device abstraction layer is mounted on the internal system corresponding to each node in the second cluster. Based on the first / second device abstraction layer, corresponding redundant mirror block device abstraction layers are constructed for the two nodes that establish a connection, and are respectively set in the corresponding nodes, so that the data to be written is written to the corresponding first device abstraction layer and second device abstraction layer based on the data to be written sent by the client and the default data writing path.

2. The cross-cluster active-active construction method according to claim 1, characterized in that, The storage performance development toolkit registers the virtualized logical units created in the first cluster as the first device abstraction layer, including: Obtain the virtualized logical unit pre-created in the first cluster; The storage performance development kit (SDK) initiates an access request to the virtualization logical unit through the block device abstraction and management module, so that the SSD can establish a communication connection with the first cluster. The storage attributes of the virtualization logical unit are adapted through the block device abstraction management module; The adapted virtualization logic unit is registered as the first device abstraction layer.

3. The cross-cluster active-active construction method according to claim 2, characterized in that, The process of adapting the storage attributes of the virtualized logical unit through the block device abstraction management module includes: The storage attributes in the virtualization logical unit are converted into the standard attribute format supported by the storage performance development kit.

4. The cross-cluster active-active construction method according to claim 2, characterized in that, After the block device abstraction management module performs the storage attribute adaptation processing on the virtualization logical unit, the method further includes: Generate block device abstraction layer information compatible with the user-space input / output stack of the storage performance development kit, so that the virtualization logic unit can access the user-space input / output processing flow of the storage performance development kit and be uniformly identified, scheduled and controlled by the block device abstraction management module.

5. The cross-cluster active-active construction method according to claim 4, characterized in that, Registering the adapted virtualization logical unit as the first device abstraction layer includes: The block device abstraction management module assigns a unique block device abstraction layer identifier to the adapted virtualization logical unit. Write the block device abstraction layer identifier and the corresponding block device abstraction layer information into the block device abstraction layer device list of the storage performance development kit; The user-space input / output stack of the storage performance development kit is triggered to initialize the adapted virtualization logic unit, so that the adapted virtualization logic unit has the ability to receive and process input / output requests.

6. The cross-cluster active-active construction method according to claim 1, characterized in that, Exposing the first device abstraction layer to the second cluster based on the network storage protocol includes: Based on the network storage protocol, a remote access service is initiated in the storage performance development kit; The parameter information corresponding to the first device abstraction layer is synchronized to the second cluster through the remote access service; The network storage protocol establishes a data transmission channel between the first cluster and the second cluster, so that the second cluster can access the first device abstraction layer through the data transmission channel, thereby exposing the first device abstraction layer to the second cluster.

7. The cross-cluster active-active construction method according to claim 6, characterized in that, After the remote access service is started in the storage performance development kit based on the network storage protocol, the following is also included: Configure network communication parameters and access permission policies for the remote access service; wherein, the network communication parameters include: transmission protocol type, port number, queue depth and data block transmission unit size, and the access permission policy includes the authentication rules and read / write operation permissions of the second cluster.

8. The cross-cluster active-active construction method according to claim 6, characterized in that, After exposing the first device abstraction layer to the second cluster based on the network storage protocol, the method further includes: The storage performance development kit is used to monitor the access status of the second cluster to the first device abstraction layer in real time. When the access status indicates abnormal access, the access permission is frozen or the data transmission channel is disconnected.

9. The cross-cluster active-active construction method according to claim 1, characterized in that, The step of constructing corresponding redundant mirror block device abstraction layers for the two nodes establishing a connection, based on the first / second block device abstraction layer, includes: Obtain the node device abstraction layer of the two nodes that have established a connection in the first / second device abstraction layer respectively; Call the redundancy-device module in the storage performance development kit and create an interface; The two node device abstraction layers are entered into the primitive device parameter list of the creation interface; The first device abstraction layer and the second device abstraction layer are configured in mirror mode through the creation interface; Based on the primitive device parameter list and the mirror mode, the redundant mirror block device abstraction layer is created so that the redundant mirror block device abstraction layer has the characteristic of synchronously writing to the first block device abstraction layer and the second block device abstraction layer after receiving input / output requests.

10. The cross-cluster active-active construction method according to claim 9, characterized in that, After configuring the first device abstraction layer and the second device abstraction layer to mirror mode using the creation interface, the method further includes: Configure data dual-write synchronization rules and consistency verification strategies for the first device abstraction layer and the second device abstraction layer.

11. The cross-cluster active-active construction method according to claim 9, characterized in that, Setting the redundant mirror block device abstraction layer in the corresponding nodes includes: According to the set order, it is preferentially set in the corresponding node in the first cluster or the second cluster.

12. The cross-cluster active-active construction method according to any one of claims 1-11, characterized in that, Also includes: Based on the new requirement of dual-active volumes, a new virtualization logical unit is created on any of the nodes in the first cluster, and a new virtualization logical unit is created on any of the nodes in the second cluster; wherein, the new virtualization logical unit serves as the underlying raw storage resource of the dual-active volume.

13. A data writing method, characterized in that, include: Obtain the data to be written sent by the client, and input the data to be written into the cross-cluster dual-active construction method according to any one of claims 1-12, so that in the completed cross-cluster dual-active architecture, the data to be written is written to the first device abstraction layer corresponding to the first cluster and the second device abstraction layer corresponding to the second cluster based on the default data writing path.

14. The data writing method according to claim 13, characterized in that, Based on the default data write path, the data to be written is written to the first device abstraction layer corresponding to the first cluster and the second device abstraction layer corresponding to the second cluster, including: The data to be written is written into the redundant mirror block device abstraction layer in any node of the first cluster or the second cluster; Based on the node device abstraction layers corresponding to the two nodes in the redundant mirror block device abstraction layer, the data to be written is written into the first block device abstraction layer and the second block device abstraction layer corresponding to the two node device abstraction layers.

15. The data writing method according to claim 13, characterized in that, Also includes: When any node in the first cluster is disconnected from the node connected to the second cluster, and both the current node in the first cluster and the current node in the second cluster are continuously communicating with the arbitration service module, then based on the cluster priority policy, either cluster is granted the permission to continue direct write service, and the other cluster is instructed to enter the permission to indirect write service.

16. The data writing method according to claim 13, characterized in that, Also includes: When any node in the first cluster disconnects from the node connected to the second cluster, the communication between the current node in the first cluster and the arbitration service module is interrupted, while the current node in the second cluster continues to communicate with the arbitration service module, the current node in the first cluster will be instructed to enter the indirect write service permission, and the remaining nodes in the first cluster and each node in the second cluster will be granted the direct write / indirect write service permission.

17. The data writing method according to claim 13, characterized in that, Also includes: When the first cluster and / or the second cluster is in a degraded operation phase, obtain the data to be written corresponding to the degraded operation phase; Based on the range of storage data blocks corresponding to the data to be written, corresponding markings are made in the corresponding dirty bitmap so that when the first cluster and / or the second cluster resume normal operation, the data synchronization process is started, the dirty data blocks marked with the markings in the dirty bitmap are read, and the dirty data blocks are written into the corresponding first device abstraction layer and / or second device abstraction layer.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the cross-cluster active-active construction method as described in any one of claims 1 to 12, or to implement the steps of the data writing method as described in any one of claims 13 to 17.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the cross-cluster dual-active construction method as described in any one of claims 1 to 12, or the steps of the data writing method as described in any one of claims 13 to 17.

20. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the cross-cluster dual-active construction method according to any one of claims 1 to 12, or the steps of the data writing method according to any one of claims 13 to 17.

Images