Hard disk sharing method based on NVMe protocol and dual-control array

Abstract

The present invention relates to a hard disk sharing method based on NVMe protocol and dual-control array. The method comprises S1, constructing a basic storage unit; S2, two sharing end controllers in a universal dual-control storage array used as a sharing end create a Target subsystem and add all hard disks of a back-end hard disk group; S3, the sharing end controller configures an NVMe Target mapping port according to a link type, and associates the Target subsystem with the NVMe Target mapping port; S4, two access end controllers in a universal dual-control storage array used as an access end identify and connect the corresponding NVMe Target according to the specified corresponding port and link type; S5, the access end controller executes a multipath device management command, identifies the unique hard disk corresponding to the dual-path synthesis, and establishes a connection and access. The method utilizes the NVMe mechanism of the Linux kernel to realize storage resource sharing between multiple universal dual-control storage arrays; the method does not need to design a non-universal hardware module, shortens the system construction cycle, and saves human and material resources.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data storage, and in particular to a hard disk sharing method based on NVMe protocol and dual-control array. Background Art

[0002] Storage arrays with multiple controllers are widely used in data centers, cloud computing platforms, large enterprise IT systems and other storage scenarios that require high reliability, high performance and easy management. In the prior art, there are many ways to build storage arrays with multiple controllers, such as using multiple general dual-controller storage arrays. Due to the actual data usage requirements, multiple general dual-controller storage arrays need to be interconnected and shared at the controller level to achieve data interoperability.

[0003] However, in the prior art, multiple general dual-control storage arrays often require customized PCIe switches or customized high-speed Ethernet switches to achieve interconnection in specific data center application scenarios. On this basis, they also need to rely on dedicated conversion boards, switching boards, backplanes and other components, which are costly and complex in design, and lack a general controller interconnection and sharing method at the software level. Summary of the invention

[0004] In view of the above problems existing in the prior art, the present invention provides a hard disk sharing method based on the NVMe protocol and a dual-control array. The method can utilize the NVMe mechanism of the Linux kernel to realize storage resource sharing between multiple general dual-control storage arrays; so that in the entire multi-controller storage array network, each controller can access the hard disk owned by any other dual-control storage array; the method does not require specially designed non-general hardware modules, shortens the system construction cycle, and saves manpower and material resources.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is specifically as follows:

[0006] The hard disk sharing method based on the NVMe protocol and the dual-control array is characterized by comprising the steps of:

[0007] S1. Construct a basic storage unit, each of which includes two universal dual-controller storage arrays; each universal dual-controller storage array includes two controllers and a back-end hard disk group connected to all controllers;

[0008] S2. The two shared-end controllers in the universal dual-control storage array used as the shared end load the pre-configured module of the Linux kernel, and create a Target subsystem in the NVMe subsystems; add all the hard disks of the backend hard disk group in the namespace of the Target subsystem; for the same hard disk, configure the same uuid number in the namespace of the NVMe Target of the two shared-end controllers;

[0009] S3, the shared end controller configures the NVMe Target mapping port according to the link type, and associates the Target subsystem with the NVMe Target mapping port;

[0010] S4, two access end controllers in the universal dual-control storage array used as the access end load the pre-configured module; create an Initiator subsystem in the NVMe subsystems; identify and connect the corresponding NVMeTarget according to the specified corresponding port and link type;

[0011] S5. The access end controller executes the multipath device management command, identifies the unique hard disk corresponding to the dual-path synthesis, and establishes connection and access.

[0012] Therefore, the basic storage unit includes two universal dual-control storage arrays that can share storage resources with each other. Each universal dual-control storage array includes two controllers that execute Linux. According to the different data flows in a data sharing process, the two universal dual-control storage arrays are divided into a universal dual-control storage array at the sharing end and a universal dual-control storage array at the access end according to the executed actions. The two can be interchangeable according to actual needs, and different modules only need to be loaded according to the corresponding method.

[0013] The two controllers in the general dual-control storage array used as the shared end are defined as shared end controllers. Their main function is to use the Linux kernel NVMe mechanism to map all the hard disks in the back-end hard disk group connected to the two to the two access end controllers in the general dual-control storage array used as the access end through the corresponding two links. The Linux kernel NVMe mechanism is a kernel module provided by the kernel that supports NVMe Target and NVMe Initiator, realizes the mapping of block devices such as hard disks based on the NVMe protocol, and supports three link forms: fc, ordinary tcp network, and rdma.

[0014] First, the two shared-end controllers in the universal dual-control storage array used as the shared end load the pre-configured module of the Linux kernel. For the shared-end controller, this pre-configured module is the nvmet module, which is a software module of the Linux kernel. After loading the nvmet module, the entire NVMe software mechanism can be initialized and work normally, which is the first step of all configurations. Next, different link kernel modules are loaded according to different links. Ordinary TCP network links load the nvmet-tcp module; fc link environments load the nvmet-fc module; rdma network links load the nvmet-rdma module. Then create a Target subsystem in NVMe subsystems; add all hard disks of the backend hard disk group in the namespace of the Target subsystem, such as adding / dev / sdb, / dev / sdc, / dev / sdd, etc. For the same hard disk, the same uuid number is configured in the namespace of the NVMeTarget of the two shared end controllers. The purpose here is to use the Linux Multipath mechanism. Because the same uuid number is configured for the same hard disk, the multipath mechanism will merge the two paths into one path to form a block device. The failure of one link or the abnormality of any controller in a general dual-controller storage array will not affect the normal function. The uuid number is a 32-bit hexadecimal digital identifier used to uniquely identify a hard disk or partition.

[0015] Then the shared-end controller configures the NVMe Target mapping port according to the link type, and associates the Target subsystem with the NVMe Target mapping port. There are many types of links here depending on the actual situation. Since NVMe Target supports three modes: NVMe-tcp, NVMe-fc, and NVMe-rdma, the link types used for mapping can correspond to ordinary tcp networks, fc, or rdma, respectively. At this time, the two shared-end controllers in the universal dual-control storage array used as the shared end are ready, waiting for the two access-end controllers in the universal dual-control storage array used as the access end to establish an identification link with it through the corresponding link to achieve subsequent shared access.

[0016] The two access controllers in the universal dual-control storage array used as the access end load pre-configured modules, which are specifically NVMe core, NVMe fabrics and other modules. Then create the Initiator subsystem in NVMe subsystems; then load different link kernel modules under the Initiator subsystem according to the different links, load the nvme-tcp module for ordinary tcp network links; load the nvme-fc module for fc link environments; and load the nvme-rdma module for rdma network links. The link type selected here must be the same as the link type matched when configuring the NVMe Target mapping port on the shared end controller mentioned above.

[0017] Then, the access end controller executes the NVMediscover command to discover the corresponding NVMe Target according to the specified NVMe Target mapping port and link type, executes the NVMe connect command to the above discovered NVMeTarget, and establishes a connection to complete the hard disk identification.

[0018] Finally, the access controller executes the multipath device management command multipath-ll, and the access controller "sees" the NVMe device synthesized by the two link paths. By accessing the NVMe device, the corresponding hard disk in the universal dual-controller storage array used as the shared end can be accessed.

[0019] In addition, the two sharing-end controllers of the universal dual-control storage array used as the sharing end can also create an Initiator subsystem; the two access-end controllers in the universal dual-control storage array used as the access end can also create a Target subsystem; by executing the steps of this method in reverse, the two universal dual-control storage arrays in a basic storage unit can recognize and access all the hard disks mounted on each other, that is, the function of two-way sharing is realized; in other words, the same controller supports NVMe Target to share its own hard disk and supports NVMe Initiatot to access other hard disks, and only needs to swap the roles of the two and reconfigure them.

[0020] In addition, since the same hard disk is configured with the same uuid number in the namespace of the NVMe Target of the two shared-end controllers, the effect of link path merging is formed, so that the access-end controller can identify the unique hard disk corresponding to the dual-path synthesis and establish connection and access by executing multipath device management commands. At this time, the failure of one link or the abnormality of one controller of a universal dual-control storage array will not affect the normal function. The security and stability of interconnection sharing are greatly improved.

[0021] In addition, the NVMe Target and NVMe Initiator software mechanisms of the Linux kernel support FC, TCP networks and RDMA, and the link form is more extensive and more versatile. By connecting multiple basic storage units using a universal switch, storage resource sharing between multiple universal dual-control storage arrays can be achieved. This method uses the NVMe mechanism of the Linux kernel to achieve storage resource sharing between multiple universal dual-control storage arrays; so that in the entire multi-controller storage array network, each controller can access the hard disk owned by any other dual-control storage array. This method does not rely on too many hardware restrictions, does not require special hardware such as PCIe switch and smart network card, and can build a multi-controller storage array at a lower cost, which is more versatile. As a preferred embodiment of the present invention, the shared end controller configures the NVMe Target mapping port according to the NVMe-tcp link, and configures the IP and port number.

[0022] Therefore, NVMe-tcp allows NVMe commands to be transmitted over a standard TCP network. The advantage of this approach is that it can leverage the existing widely deployed TCP network infrastructure, and is suitable for cost-sensitive environments that already have a mature TCP network infrastructure. Specifically, the shared-side controller needs to load the nvmet-tcp module and set addr_trtype to tcp.

[0023] As a preferred embodiment of the present invention, the shared end controller configures the NVMe Target mapping port according to the NVMe-fc link and configures the WWN number.

[0024] Therefore, NVMe-fc encapsulates NVMe commands in the Fibre Channel protocol and transmits them over the Fibre Channel network. Fibre Channel is a high-performance, connection-based serial data communication protocol that is widely used in enterprise-level storage networks. Specifically, the shared-end controller needs to load the nvmet-fc module and set addr_trtype to fc.

[0025] As a preferred embodiment of the present invention, the shared end controller configures the NVMe Target mapping port according to the NVMe-rdma link.

[0026] NVMe-rdma is a more efficient transmission method. It uses rdma technology to directly transmit data between the memory of the target and the initiator, bypassing the network stack of the operating system, thereby reducing CPU consumption and latency. This method is usually better than NVMe-tcp and NVMe-fc in performance. Specifically, the shared end controller needs to load the nvmet-rdma module and set addr_trtype to rdma.

[0027] As a preferred embodiment of the present invention, in step S2, when two shared-end controllers in a universal dual-control storage array used as a shared end create a Target subsystem in NVMe subsystems, it also includes a step of setting a cntlid number range, and the cntlid number ranges set by the two shared-end controllers are different.

[0028] The cntlid number in the NVMe Target, i.e. the controller identifier, is a number used to identify a controller in the NVMe protocol, and is used to distinguish and manage different controllers. Each time the NVMe Initiator corresponding to the access end controller discovers the NVMe Target corresponding to the shared end controller and establishes a session connection, it will be assigned a cntlid number to distinguish the two links formed by the two different shared end controllers. In addition, the cntlid number ranges set for the two shared end controllers are different, which can ensure that at any time, the two links of the two shared end controllers will not overlap, ensuring that the multipath mechanism can work normally.

[0029] As a preferred embodiment of the present invention, in step S4, when identifying and connecting the corresponding NVMe Target according to the specified corresponding port and link type, the specific operation steps of identification and connection are: the two access end controllers in the general dual-control storage array used as the access end execute the NVMediscover command to discover the corresponding NVMe Target, and execute the NVMe connect command to establish a connection with the discovered NVMe Target.

[0030] As a preferred embodiment of the present invention, the universal dual-control storage array is a Feiteng 4U 24-disk dual-control storage array.

[0031] A multi-controller storage array, characterized in that it comprises a general switch and a plurality of basic storage units obtained based on the hard disk sharing method.

[0032] As a preferred embodiment of the present invention, the general switch has an Ethernet switching function.

[0033] As a preferred embodiment of the present invention, the general switch has a FC protocol switching function.

[0034] In summary, the present invention has the following beneficial effects:

[0035] 1. This method can use the NVMe mechanism of the Linux kernel to realize storage resource sharing among multiple universal dual-controller storage arrays; so that in the entire multi-controller storage array network, each controller can access the hard disk directly connected and controlled by any other dual-controller storage array; this method does not require the special design of non-universal hardware modules, shortens the system construction cycle, and saves manpower and material resources.

[0036] 2. This method does not rely on too many hardware restrictions and does not require dedicated hardware such as PCIe switch and smart network card. It can build a multi-controller storage array at a lower cost and is more versatile. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A method flow chart of the hard disk sharing method;

[0038] Figure 2 Schematic diagram of the basic storage unit in this embodiment. DETAILED DESCRIPTION

[0039] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Anyone can implement the present disclosure in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0040] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment".

[0041] The basic storage unit includes two universal dual-control storage arrays that can share storage resources with each other. Each universal dual-control storage array includes two controllers configured with Linux. According to the different data flows during a data sharing process, the two universal dual-control storage arrays are divided into a universal dual-control storage array at the sharing end and a universal dual-control storage array at the access end according to the actions performed. The two can be interchangeable according to actual needs, and different modules only need to be loaded according to the corresponding methods.

[0042] In this embodiment 1, Figure 2As shown, a basic storage unit is taken as an example, which includes machine 1 and machine 2, wherein machine 2 is an access-side universal dual-control storage array that performs access actions, and machine 1 is a sharing-side universal dual-control storage array that performs sharing actions.

[0043] The whole machine 1 includes controller A and controller B, and the whole machine 2 includes controller C and controller D. Each controller is provided with a network communication interface, which can support any one of the tcp / fc / rdma network protocols. Each controller is equipped with a Linux kernel, and the NVMe mechanism of the Linux kernel is used to realize data sharing between the whole machine 1 and the whole machine 2.

[0044] like Figure 1 As shown, first, controller A and controller B in the whole machine 1 load the pre-configuration module of the Linux kernel. This pre-configuration module is the nvmet module. After loading the nvmet module, the entire NVMe software mechanism can be initialized and work normally. It is the first step of all configurations. Next, different link kernel modules are loaded according to different links. Ordinary TCP network links load the nvmet-tcp module; fc link environment loads the nvmet-fc module; rdma network links load the nvmet-rdma module. Then create a Target subsystem in NVMe subsystems; add all hard disks of the backend hard disk group to the namespace of the Target subsystem, such as adding / dev / sdb, / dev / sdc, / dev / sdd, etc. After adding successfully, check the UUID number of each hard disk in the namespaces. For the same hard disk, configure the same uuid number in the namespace of the NVMe Target of the two shared end controllers. Since the same hard disk is configured with the same uuid number in the namespace of the NVMeTarget of the two shared-end controllers, the link path is merged, so that the access-end controller can identify the unique hard disk corresponding to the dual-path synthesis and establish connection and access by executing multipath device management commands. At this time, the failure of one link or the abnormality of one controller of a universal dual-control storage array does not affect the normal function, which greatly improves the security and stability of interconnection sharing.

[0045] Next, controller A and controller B in the whole machine 1 configure the NVMe Target mapping port according to the link type, and associate the Target subsystem with the NVMe Target mapping port. Since NVMe Target supports three modes: NVMe-tcp, NVMe-fc, and NVMe-rdma, the link type used for mapping can correspond to ordinary tcp network, fc, or rdma respectively. For example, controller A and controller B load the nvmet-tcp module and set addr_trtype to tcp. At this time, the selected link type is ordinary tcp network.

[0046] Controller C and controller D in the whole machine 2 execute the NVMe discover command to discover the corresponding NVMe Target according to the specified NVMe Target mapping port and link type, and execute the NVMe connect command to establish a connection with the discovered NVMeTarget.

[0047] Controller C and controller D in machine 2 execute the multipath device management command multipath-ll, and the NVMe device synthesized by the two link paths is identified. By accessing the NVMe device, the hard disk corresponding to machine 1 can be accessed.

[0048] In another embodiment 2, controller A and controller B of whole machine 1 create an Initiator subsystem; controller C and controller D in whole machine 2 create a Target subsystem; by executing the above steps in reverse, two general dual-control storage arrays in a basic storage unit can recognize each other and access all hard disks mounted on each other, that is, realize the function of two-way sharing; in other words, the same controller supports NVMe Target to share the hard disk controlled by itself, and also supports access to other hard disks through NVMe Initiatot, and only needs to swap the roles of the two and reconfigure.

[0049] In another embodiment 3, two Feiteng 4U 24-disk dual-control storage arrays are used to form the basic storage unit in this method.

[0050] In another embodiment 4, the present invention further provides a multi-control storage array, which includes a general switch and multiple basic storage units obtained based on the above method. The general switch has an Ethernet switching function, which is used to connect the multiple basic storage units in the above embodiment, and multiple Feiteng 4U 24-disk dual-control storage arrays can realize mutual sharing functions without the need for specially customized hardware. In addition, according to different application scenarios, a general switch with fc protocol switching function can also be used.

[0051] A number of embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A hard disk sharing method based on NVMe protocol and dual-control array, characterized in that: Includes steps: S1. Construct a basic storage unit, each of which includes two universal dual-controller storage arrays; each universal dual-controller storage array includes two controllers and a back-end hard disk group connected to all controllers; S2. The two shared-end controllers in the universal dual-control storage array used as the shared end load the pre-configured module of the Linux kernel, and create a Target subsystem in the NVMe subsystems; add all the hard disks of the backend hard disk group in the namespace of the Target subsystem; for the same hard disk, configure the same uuid number in the namespace of the NVMe Target of the two shared-end controllers; S3, the shared end controller configures the NVMe Target mapping port according to the link type, and associates the Target subsystem with the NVMe Target mapping port; S4, two access-end controllers in the universal dual-control storage array used as access ends load the preconfiguration module; the two access-end controllers create an Initiator subsystem in the NVMe subsystems, and identify and connect the NVMe Target created in the shared-end controller according to the NVMe Target mapping port configured by the shared-end controller in S3 and the selected link type; S5. The access end controller executes the multipath device management command, identifies the unique hard disk corresponding to the dual-path synthesis, and establishes connection and access.

2. The hard disk sharing method based on NVMe protocol and dual-control array according to claim 1 is characterized in that: The shared end controller configures the NVMe Target mapping port according to the NVMe-tcp link, and configures the IP and port number.

3. The hard disk sharing method based on NVMe protocol and dual-control array according to claim 1 is characterized in that: The shared end controller configures the NVMe Target mapping port according to the NVMe-fc link and configures the WWN number.

4. The hard disk sharing method based on NVMe protocol and dual-control array according to claim 1, characterized in that: The shared end controller configures the NVMe Target mapping port according to the NVMe-rdma link.

5. The hard disk sharing method based on NVMe protocol and dual-control array according to claim 2 is characterized in that: In step S2, when two shared-end controllers in a universal dual-control storage array used as a shared end create a Target subsystem in NVMe subsystems, a step of setting a cntlid number range is also included, and the cntlid number ranges set by the two shared-end controllers are different.

6. The hard disk sharing method based on NVMe protocol and dual-control array according to claim 5 is characterized in that: In step S4, when the corresponding NVMe Target is identified and connected according to the specified corresponding port and link type, the specific operation steps of identification and connection are: the two access end controllers in the general dual-control storage array used as the access end execute the NVMe discover command to discover the corresponding NVMe Target, and execute the NVMe connect command to establish a connection with the discovered NVMe Target.

7. The hard disk sharing method based on NVMe protocol and dual-control array according to claim 1 is characterized in that: The universal dual-control storage array is a Feiteng 4U 24-disk dual-control storage array.

8. A multi-control storage array, characterized in that: It comprises a general switch and a plurality of basic storage units obtained by a hard disk sharing method based on the NVMe protocol and a dual-control array as described in any one of claims 1 to 7 above.

9. A multi-control storage array according to claim 8, characterized in that: The general switch has an Ethernet switching function.

10. The multi-control storage array according to claim 8, characterized in that: The general switch has the fc protocol switching function.

Images