Storage system and method for controlling the same

Abstract

This invention provides a storage system and a control method thereof that improve communication performance while suppressing controller memory consumption. [Solution] The storage system is provided with a network interface equipped with a processor having multiple cores, each processing commands that involve communication with a target outside the node, and a controller that issues such commands and has the network interface process them. The network interface connects multiple first sessions with the target and assigns a different processor core to each first session. The multiple first sessions with the target are managed as a single virtual second session, and commands issued by the controller are processed using the second session, which uses one of the multiple first sessions.

Description

Technical Field

[0001] The present invention relates to a storage system and a control method thereof, and is suitable for application to a storage system equipped with, for example, a Smart Network Interface Card (SmartNIC).

Background Art

[0002] In recent years, as a network form connecting a storage system and a host server, a Storage Area Network (SAN) has become widespread. A SAN is a network configured by combining a network cable such as an optical fiber and a switch. With such a SAN, sharing of storage resources among a plurality of host servers becomes possible.

[0003] A storage system connected to a host server and a SAN often performs remote replication to replicate data to a storage system at a remote site connected via a Wide Area Network (WAN) for service continuity in case of a disaster.

[0004] In remote replication, data replication is performed in units of volumes provided by the storage system to the host. Thus, even when the primary site goes down, services can be continued using the data stored in the volumes replicated to the remote site. Here, the "volume" refers to a logical storage area (logical volume). The same applies hereinafter.

[0005] In order to smoothly operate remote replication of volumes, which are increasing in capacity in recent years, the data transfer speed between the storage system at the remote replication source (hereinafter, appropriately referred to as the initiator storage system) and the storage system at the remote replication destination (hereinafter, appropriately referred to as the target storage system) is important.

[0006] In this regard, for example, Patent Document 1 discloses a smart NIC-utilizing storage system (hereinafter referred to as the smart NIC-utilizing storage system) that incorporates a smart NIC into the storage system and offloads remote replication communication processing to the smart NIC.

[0007] Here, "smart NIC" refers to a network interface equipped with a processor and memory, capable of running general-purpose operating systems (OS) and open-source software (OSS) protocol servers directly. By offloading block protocol processing from the storage system controller to the smart NIC, the load on the controller can be reduced, improving the performance of the storage system.

[0008] In a storage system equipped with a smart NIC, the smart NIC establishes a connection and session with the target storage system at a remote site, and then transfers data and control information between the smart NIC and the target storage system.

[0009] In this case, the iSCSI (Internet Small Computer System Interface) protocol, which is widely used in general-purpose storage systems, can be used as the communication protocol between the smart NIC and the target storage system. By using the iSCSI protocol, it becomes possible to use various types of storage systems as the target storage system. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] U.S. Patent Application Publication No. 2022 / 0400982 [Overview of the project] [Problems that the invention aims to solve]

[0011] By the way, in a storage system that utilizes smart NICs and is equipped with smart NICs, there are two problems when offloading communication processing to the smart NICs.

[0012] The first issue is that communication between the smart NIC and the target storage system is typically performed in a single session per target storage system, and that session is processed by a single CPU (Central Processing Unit) core within the smart NIC. Therefore, the performance of that CPU core becomes a bottleneck in terms of communication performance.

[0013] Smart NICs typically have multiple CPU cores, and by distributing the sessions to be handled by each CPU core, overhead such as lodge contention due to resource exclusion between CPU cores can be reduced. Here, "CPU core" refers to the number of processors built into that CPU. A CPU can perform the same number of processes in parallel as the number of CPU cores.

[0014] However, since a smart NIC establishes only one session per target storage system and handles the communication processing for that session with only one CPU core, the communication performance will depend on the performance of the CPU core handling that session, even if other CPU cores are available.

[0015] The second issue is that while increasing the number of sessions per target storage system could be considered to solve the problems mentioned above, this method would increase the amount of session information data managed by both the smart NIC and the controller.

[0016] In particular, when a storage system is equipped with a large number of smart NICs, the amount of session information that the controller must maintain becomes enormous. However, the controller has a predetermined maximum number of session connections based on its memory capacity. Therefore, when increasing the number of sessions per target storage system, the number of target storage systems that can be connected decreases inversely proportional to the increase in the number of sessions.

[0017] This invention was made in consideration of the above points, and aims to propose a storage system and a control method thereof that can improve communication performance while suppressing the memory consumption of the controller. [Means for solving the problem]

[0018] To solve these problems, the present invention provides a storage system comprising nodes and a storage area for storing data in a higher-level device, wherein the node is provided with a network interface having a plurality of processors, each processing commands involving communication with a target outside the node, and a controller that controls the reading and writing of the data and issues the commands involving communication with the target, causing the network interface to process them. The network interface connects a plurality of first sessions with the target and assigns a different core of the processor to each of the first sessions, manages the plurality of first sessions with the target as a single virtual second session, and uses the second session, which uses one of the plurality of first sessions, to process the commands involving communication with the target issued by the controller.

[0019] Furthermore, the present invention provides a control method for a storage system comprising nodes and providing a storage area for storing data in a higher-level device, wherein the node has a network interface provided with a plurality of processors, each having a plurality of cores that process commands involving communication with a target outside the node, and a controller that controls the reading and writing of the data and issues the commands involving communication with the target, causing the network interface to process them, and the network interface comprises a first step of connecting a plurality of first sessions with the target and assigning a different core of the processor to each of the first sessions, and managing the plurality of first sessions with the target as a single virtual second session, and a second step of processing the commands involving communication with the target issued by the controller using the second session which uses one of the plurality of first sessions.

[0020] According to the storage system and control method of the present invention, commands from the controller to the network interface can be processed by multiple processor cores within the network interface, thus preventing the performance of the processor cores from becoming a bottleneck in communication performance. Furthermore, since the controller manages multiple first sessions as a single virtual second session, it is possible to prevent an increase in the amount of session information that the controller must manage in memory. [Effects of the Invention]

[0021] According to the present invention, a storage system and a control method thereof can be realized that can improve communication performance while suppressing the memory consumption of the controller. [Brief explanation of the drawing]

[0022] [Figure 1] This is a block diagram for illustrating the outline of the first embodiment. [Figure 2] It is a block diagram showing the overall configuration of the information processing system according to the first embodiment. [Figure 3] It is a block diagram showing a configuration example of the host server. [Figure 4] It is a block diagram showing a configuration example of the management server. [Figure 5] It is a block diagram showing a configuration example of the controller of the initiator storage system. [Figure 6] It is a block diagram showing a configuration example of the front-end interface of the initiator storage system. [Figure 7] It is a chart showing a configuration example of the port management table. [Figure 8] It is a chart showing a configuration example of the logical device management table. [Figure 9] It is a chart showing a configuration example of the remote path management table. [Figure 10] It is a chart showing a configuration example of the front-end interface management table. [Figure 11] It is a chart showing a configuration example of the session management table. [Figure 12] It is a chart showing a configuration example of the virtual session management table. [Figure 13] It is a chart for explaining the command format of the IO command. [Figure 14] It is a sequence diagram showing the flow of session establishment processing. [Figure 15] It is a sequence diagram showing the flow of write IO command processing. [Figure 16] It is a sequence diagram showing the flow of abort command processing. [Figure 17] It is a sequence diagram showing the flow of session disconnection processing. [Figure 18] It is a sequence diagram showing the flow of session monitoring processing. [Figure 19] It is a block diagram for explaining the outline of the second embodiment. [Figure 20]This is a block diagram for illustrating the outline of the third embodiment. [Modes for carrying out the invention]

[0023] An embodiment of the present invention will be described in detail below with reference to the drawings.

[0024] It should be noted that the embodiments described below are not intended to limit the scope of the claims of the present invention, and not all of the elements described in each embodiment are necessarily necessary for solving the problems of the present invention.

[0025] Furthermore, in the following, where necessary for convenience, the explanation will be divided into multiple sections or embodiments. Unless otherwise specified, these are not unrelated, and one may be a modification, detail, or supplementary explanation of part or all of the other.

[0026] Furthermore, when referring to the number of elements (including count, numerical value, quantity, range, etc.) below, unless specifically stated or clearly limited in principle to a particular number, it is not limited to that particular number, and may be greater than or less than that number.

[0027] (1) First embodiment (1-1) Overview of this embodiment In this embodiment, we assume a smart NIC-utilizing storage system that is equipped with a smart NIC as the initiator storage system and offloads communication processing from the storage system controller that controls remote replication to the smart NIC.

[0028] Furthermore, in this embodiment, the controller of the initiator storage system is assumed to connect to one target storage system in one session using a communication protocol such as iSCSI, and to perform remote replication to the target storage system.

[0029] In this process, the smart NIC of the initiator storage system establishes multiple iSCSI sessions (hereinafter referred to as "actual sessions") with the target storage system internally and uses them for data communication. This allows the load to be distributed among the CPU cores within the smart NIC, thereby improving communication performance.

[0030] Furthermore, by virtualizing these multiple real sessions and presenting them to the controller as a single session, the amount of session information that the controller needs to manage is reduced, thereby suppressing the controller's resource consumption.

[0031] Furthermore, the following description of smart NICs can be applied not only to interface devices whose functions can be programmed by software executed by a processor, but also to interface devices with programmable logic circuit configurations, such as FPGAs (Field Programmable Gate Arrays). An FPGA may include logic circuits that realize each of the functions implemented by the program, and cache memory used in calculations.

[0032] Figure 1 shows an overview of how a storage system 3 located at primary site 2 ("Site A" in Figure 1) performs remote replication to a storage system 6 located at secondary site 5 ("Site B" in Figure 1) via a network 4 such as a WAN, using the communication method described above.

[0033] Each controller 10 of the storage system (initiator storage system) 3 installed at primary site 2 is equipped with a front-end interface 11 consisting of a smart NIC. The front-end interface 11 then establishes a session with the storage system (target storage system) 6 installed at secondary site 5 and performs data transfer such as replication data and control information.

[0034] Furthermore, the remote replication control program 12 implemented in the controller 10 of the initiator storage system 3 replicates the data of the volume PVOL within the initiator storage system 3 (hereinafter referred to as the primary volume) to a specific volume SVOL located within the target storage system 6 (hereinafter referred to as the secondary volume), and reflects any subsequent updates to the primary volume PVOL in the secondary volume SVOL.

[0035] The controller 10 of the initiator storage system 3 requests the front-end interface 11 to establish a communication path with the target storage system 6 when remote replication is initiated.

[0036] Upon receiving this request, the front-end interface 11 establishes a TCP / IP connection with the target storage system 6 by, for example, sending a TCP / IP (Transmission Control Protocol / Internet Protocol) connection request to the target storage system 6.

[0037] Furthermore, the front-end interface 11 then establishes a real session with its target storage system 6 in order to transfer data over this TCP / IP connection. At this time, the front-end interface 11 establishes the same number of TCP / IP connections as the number of CPU cores 13 assigned to the ports used for the session, and the real sessions 14 on these TCP / IP connections.

[0038] Furthermore, the front-end interface 11 assigns each CPU core 13 a different real session 14 from the real sessions 14 established as described above, and then evenly distributes the processing of commands such as I / O commands to each CPU core 13. This enables load balancing among the CPU cores 13.

[0039] Furthermore, the front-end interface 11 manages the session information of each of these actual sessions 14 using the virtual session management table 91, which will be described later in Figure 12, while presenting these multiple actual sessions 14 to the controller 10 as a single virtual session. In other words, the front-end interface 11 notifies the controller 10 of the session information of these multiple actual sessions 14 as session information of a single virtual session (hereinafter referred to as a virtual session).

[0040] The controller 10 then registers and manages the session information of the virtual session provided by the front-end interface 11 in the session management table 72, which will be described later in relation to Figure 11. This reduces the total number of session information entries that the controller 10 needs to manage in the session management table 72.

[0041] Through the methods described above, it is possible to improve communication performance during data transfer while balancing the load between the CPU cores 13 of the front-end interface 11 during remote replication of the smart NIC-utilizing storage system (initiator storage system 3). Furthermore, even when the number of sessions per target storage system 6 is increased, the resource consumption of the controller 10 of the initiator storage system 3 can be kept equivalent to that when there is one session, and the controller 10 can establish session connections with the same number of target storage systems as before.

[0042] The following describes an information processing system according to this embodiment that applies the above method. Note that the following example illustrates the case where a smart NIC is used as the front-end interface 11, but this is only one form. For example, the present invention can also be applied to a storage system that does not have a smart NIC and where the controller 10 performs control processing related to communication with other storage systems.

[0043] Furthermore, while the above description focused on the application of the present invention to remote replication from the storage system 3 at the primary site 2 to the storage system 6 at the secondary site 5, the present invention is not limited to this case and can also be applied to remote replication from the storage system 6 at the secondary site 5 to the storage system 3 at the primary site 2.

[0044] (1-2) Configuration of the information processing system according to this embodiment In Figure 2, 20 represents the information processing system according to this embodiment as a whole. In this information processing system 20, a storage system 22 installed at the primary site 21 and a storage system 24 installed at the secondary site 23 are connected via a network 25 such as a WAN, and can communicate with each other.

[0045] Here, we assume that the storage system 22 at the primary site 21 and the storage system 24 at the secondary site 23 have the same configuration, and common configurations will be described as the configuration of storage system 22. While it is possible to use a different configuration for the storage system 24 at the secondary site 23 than that of the storage system 22 at the primary site 21, this will be omitted for simplicity.

[0046] The storage system 22 at the primary site 21 is connected to the host server 27 and the management server 28 via a network 26 such as a LAN (Local Area Network) or WAN.

[0047] The host server 27 is a higher-level device that reads and writes data to the storage system 22, and issues I / O (Input / Output) commands (read commands and write commands) to the storage system 22. The host server 27 sends block-level I / O commands to the storage system 22 according to the block protocol.

[0048] In this context, "block protocol" refers to a data communication protocol that reads and writes data in fixed-length block units. A storage system that provides a block protocol divides and manages the physical storage area into multiple logical volumes (LUs: Logical Units), and provides a data access service to the host server in units of predetermined block sizes defined within the logical volumes.

[0049] The management server 28 is a management device used by users or operators to control and monitor the storage system 22, and is equipped with user interfaces such as a GUI (Graphical User Interface) and a CLI (Command Line Interface). Users or operators can use these user interfaces to give various instructions to the storage system 22 and monitor the storage system 22.

[0050] The storage system 22 comprises one or more memory control devices (compute nodes) 30 and one or more memory device units (drive nodes) 31.

[0051] The storage control device 30 comprises one or more controllers 32. To improve the availability of the storage system 22, a dedicated power supply may be provided for each controller 32, and power may be supplied to each controller 32 from its dedicated power supply. Alternatively, multiple storage control devices 30 may be provided, and the controllers 32 may be connected via an HCA (Host Channel Adapter) network.

[0052] The controller 32 provides the storage area offered by the storage device 35 in the storage device unit 31 (described later) to the host server 27 as a logical volume for reading and writing data, and also has the function of processing data read / write requests from the host server 27 to the logical volume. The controller 32 also has the function of performing remote replication to copy such volume to the storage system 24 at the sub-site 23.

[0053] The controller 32 includes one or more front-end interfaces 33 and one or more back-end interfaces 34. The front-end interfaces 33 communicate with the host server 27, the management server 28, and the storage system 24 of the sub-site 23 via the network 26, and the back-end interfaces 34 communicate with the storage device unit 31.

[0054] The storage device unit 31 comprises multiple physical storage devices (PDEVs) 35. These storage devices 35 consist of non-volatile, high-capacity storage devices such as hard disk drives or SSDs (Solid State Drives). Different types of storage devices can be used as storage devices 35 within the same storage device unit 31. Furthermore, multiple identical storage devices 35 may be configured to form a RAID (Redundant Array of Inexpensive Disks) group. Data is stored in the RAID group according to a predetermined RAID level.

[0055] Next, we will describe the specific configurations of the host server 27, the management server 28, and the storage system 22.

[0056] Figure 3 shows an example configuration of the host server 27. As shown in Figure 3, the host server 27 is configured with a CPU 41, memory 42, storage devices 43, and a network interface 44 that are interconnected via an internal bus 40 consisting of a PCIe (Peripheral Component Interconnect-express) bus or the like.

[0057] The CPU 41 is the processor responsible for controlling the overall operation of the host server 27. The memory 42 consists of volatile semiconductor memory such as RAM (Read Only Memory) and is used as the working memory for the CPU 41.

[0058] The storage device 43 consists of non-volatile, high-capacity storage devices such as hard disk drives and SSDs, which store various programs and data that needs to be stored for a long period of time. The network interface 44 is an interface device for communicating with the storage system 22 and the management server 28.

[0059] In this embodiment, the memory 42 of the host server 27 stores the application program 45 and the storage connection program 46 that are read from the storage device 43 when the host server 27 is started up or when necessary.

[0060] The application program 45 is a program that has the function of reading and writing data to the logical volume provided by the storage system 22 via the storage connection program 46. The storage connection program 46 is a program that has the function of receiving various requests such as I / O (Input / Output) requests from the application program 45 and reading and writing data to the storage system 22.

[0061] On the other hand, Figure 4 shows an example of the configuration of the management server 28. As shown in Figure 4, the management server 28 is also configured with a CPU 51, memory 52, storage device 53, and network interface 54 that are interconnected via an internal bus 50 consisting of a PCIe bus or the like.

[0062] The CPU 51, memory 52, storage device 53, and network interface 54 are devices with the same configuration and functions as the CPU 41, memory 42, storage device 43, and network interface 44 of the host server 27, respectively, so their explanation is omitted here.

[0063] The management server 28's memory 52 stores the management server program 55. The management server program 55 is a program that provides a user interface such as a GUI, CLI, or REST (Representational State Transfer)ful API (Application Programming Interface), and provides functions for a user or operator to control and monitor the storage system 22. When the management server program 55 receives control or monitoring instructions from a user or operator to the storage system 22, it communicates with the storage system 22 to perform control and management.

[0064] On the other hand, Figure 5 shows an example configuration of the controller 32 of the storage system 22. The controller 32 is configured with a front-end interface 33, a back-end interface 34, a CPU 61, memory 62, and a cache 63 that are interconnected via an internal bus 60 consisting of a PCIe bus or the like.

[0065] The front-end interface 33 is a programmable network interface and consists of components such as a smart NIC. In this embodiment, block protocol processing is performed on the front-end interface 33.

[0066] The backend interface 34 is an interface device for the controller 32 to communicate with the storage device unit 31 (Figure 2). The backend interface 34 stores the data written by the host server 27 to the logical volume in the storage device 35 (Figure 2) within the storage device unit 31 that is associated with that logical volume.

[0067] The CPU 61 and memory 62 have the same configuration and functions as the CPU 41 and memory 42 of the host server 27, so their description is omitted here. The cache 63 consists of volatile semiconductor memory and is used to temporarily store data to be written provided by the host server 27 or the front-end interface 33, as well as data read from the storage device unit 31.

[0068] In this embodiment, the memory 62 stores a block storage control program 64, a session control program 65, a front-end interface integrated control program 66, a remote replication control program 67, a port management table 68, a logical device management table 69, a remote path management table 70, a front-end interface management table 71, and a session management table 72. However, these programs and management tables may also be stored in the storage device unit 31 (Figure 2).

[0069] The block storage control program 64 is a program that has the function of configuring a logical volume by combining multiple logical devices (LDEVs), which are logical storage areas of a predetermined size associated with the physical storage area provided by each storage device unit 31, and providing the configured logical volume to the front-end interface 33. The front-end interface 33 can access any logical device by specifying the logical volume. This allows the front-end interface 33 to use the logical device as the storage destination for data stored in the logical volume provided to the host server 27. The data stored in the logical device is then stored in the corresponding storage device 35 within the storage device unit 31 associated with that logical device by the back-end interface 34.

[0070] The session control program 65 is a program that has the function of controlling the session between the host server 27 and the front-end interface 33. The session control program 65 also has the function of instructing the front-end interface 33 to connect, disconnect, and retrieve information from the session with the target storage system 24 (Figure 2), in accordance with the instructions of the remote replication control program 67, which will be described later.

[0071] The front-end interface integrated control program 66 is a program that has the function of controlling all front-end interfaces 33 within the controller 32 in which it is implemented (hereinafter referred to as "the controller"). The front-end interface integrated control program 66 powers on the necessary front-end interfaces 33, starts them up, and initializes them via the internal bus, thereby making them ready to accept data access from the host server 27.

[0072] The remote replication control program 67 is a program that has the function of controlling the remote replication of data from the storage system 22 where the controller 32 is located to the storage system 24 at the secondary site 23.

[0073] Details of the port management table 68, logical device management table 69, remote path management table 70, front-end interface management table 71, and session management table 72 will be described later. These management tables, each maintained by a controller 32 within the storage system 22, are all updated synchronously, and all controllers 32 always perform various control processes based on the information stored in the management tables with the same content.

[0074] Figure 6 shows an example of the configuration of the front-end interface 33 according to this embodiment. As shown in Figure 6, the front-end interface 33 is configured to include one or more network interfaces 81 interconnected via an internal bus 80 which is composed of a PCIe bus or the like, an internal interface 82, a CPU 83, memory 84, and a storage device 85.

[0075] The network interface 81 is an interface device for communicating with the host server 27 and the storage system 24 of the secondary site 23 (Figure 2), and is composed of, for example, a physical port. For this reason, the network interface 81 will be referred to as port 81 below. A unique IP (Internet Protocol) address is assigned to port 81. An IP address is a network identifier, and the host server 27 and the storage system 24 of the secondary site 23 use the IP address assigned to port 81 to communicate with their front-end interface 33.

[0076] The internal interface 82 is a device that serves as an interface for the CPU 83 to communicate with other devices within its controller 32 via the internal bus 80.

[0077] The CPU 83 is a processor that controls the operation of the entire front-end interface 33. In this embodiment, the CPU 83 of the front-end interface 33 comprises one or more CPU cores 83A. Each CPU core 83A can operate as an independent CPU and, for example, processes various commands that involve communication with the target storage system 24.

[0078] Memory 84 consists of semiconductor memory such as RAM, and is used as working memory for each CPU core 83A. Storage device 85 consists of non-volatile semiconductor memory such as flash memory, and is used to hold various programs and data necessary for the operation of the front-end interface 33.

[0079] In this embodiment, the memory 84 of the front-end interface 33 stores the front-end session control program 86, the front-end interface control program 87, the protocol control program 88, the block access program 89, the iSCSI initiator program 90, and the virtual session management table 91. Details of the virtual session management table 91 will be described later.

[0080] The front-end session control program 86 is a program that has the function of establishing connections and sessions for communication between the storage system (hereinafter referred to as the "own storage system") 22 on which the front-end interface (hereinafter referred to as the "own front-end interface") 33 is implemented and the host server 27, and between the own storage system 22 and the storage system 24 of the secondary site 23. In this embodiment, the type of such connection is assumed to be a TCP / IP connection, and the type of such session is assumed to be an iSCSI session.

[0081] Specifically, the front-end session control program 86 configures a TCP port for a Listen service that accepts connection requests from the host server 27. When the front-end session control program 86 receives a connection request for such a Listen service, it establishes a TCP / IP connection with the host server 27. Subsequently, the front-end session control program 86 accepts session connection requests from the host server 27 and establishes a session with the host server 27.

[0082] Furthermore, the front-end session control program 86 requests a connection to the storage system 24 at the sub-site 23 for each port 81 of its own front-end interface 33, and establishes a connection with the storage system 24. After this, the front-end session control program 86 requests a session connection to the storage system 24 and establishes a session with the said storage system 24.

[0083] The front-end interface control program 87 is the OS (Operating System) of the front-end interface 33, and is a program that communicates with the controller 32 and has functions such as initializing the front-end interface 33, managing resources, managing faults, and scheduling tasks.

[0084] Furthermore, the front-end interface control program 87 works in conjunction with the front-end interface integrated control program 66 (Figure 5) of its own controller 32 to synchronize various management tables (including the virtual session management table 91) stored in memory 84 with the corresponding management tables stored in memory 62 (Figure 5) of its own controller 32.

[0085] The protocol control program 88 is a program that processes the block access protocol received from the host server 27 and converts it into a block access command request to the controller 32. The block access program 89 also communicates with the controller 32 and reads and writes data to the logical devices that constitute the logical volume.

[0086] The iSCSI initiator program 90 is a program that has the function of connecting to an iSCSI target of a storage system other than itself, such as the storage system 24 of the secondary site 23, and reading and writing data to the logical volume provided by that iSCSI target. Each CPU core 83A of the CPU 83 can independently execute this iSCSI initiator program 90, thereby independently reading and writing data to the logical volume.

[0087] (1-3) Configuration of various management tables Next, we will describe the configuration of the various management tables held by the controller 32 and front-end interface 33 of the storage system 22.

[0088] Figure 7 shows the configuration of the port management table 68 stored in the memory 62 (Figure 5) of the controller 32. The port management table 68 is a table used by the controller 32 to manage the ports 81 (Figure 6) held by each front-end interface 33 within its storage system 22.

[0089] As shown in Figure 7, the port management table 68 is configured to include a port ID field 68A, a controller ID field 68B, a front-end interface ID field 68C, an IP address field 68D, and a protocol type field 68E. In the port management table 68, one record (row) corresponds to one port existing within the local storage system 22.

[0090] The port ID field 68A stores the port ID, which is unique to the port 81 within the local storage system 22 and assigned to that port 81. The front-end interface ID field 68C stores the front-end interface ID, which is unique to the front-end interface 33 within the local storage system 22 and assigned to the front-end interface 33 that has port 81.

[0091] Furthermore, the Controller ID field 68B stores the unique identifier (Controller ID) assigned to the controller 32 on which the front-end interface 33 is implemented within the local storage system 22, and the IP address field 68D stores the IP address set to the corresponding port 81.

[0092] Furthermore, the protocol type field 68E stores the protocol type set for port 81. The protocol type also includes information on whether it is a "target" which is the destination for data access from other storage systems, or an "initiator" which is the destination for reading and writing data to other storage systems. Examples of such protocol types include "iSCSI target," "iSCSI initiator," "NVMe / TCP target," and "NVMe / TCP initiator."

[0093] Therefore, in the example shown in Figure 7, for example, port 81, which is assigned the port ID "P0-A0-0", is a port located on the front-end interface 33 "FE0-A" mounted on the controller 32 "CTRL0", is assigned the IP address "192.0.10.1", and its protocol type is indicated as "iSCSI target".

[0094] Figure 8 also shows the configuration of the logical device management table 69 stored in the memory 62 of the controller 32. The logical device management table 69 is a table used by the controller 32 to manage each logical device defined within its own storage system 22.

[0095] Each logical device is assigned a controller 32 responsible for reading and writing data to that logical device. Each controller 32 refers to this logical device management table 69 to read and write data to the logical device assigned to it. The controller 32 responsible for a logical device can be changed by updating the logical device management table 69.

[0096] As shown in Figure 8, the logical device management table 69 is configured to include a logical device ID column 69A, a controller column 69B, a physical device used column 69C, a capacity column 69D, a public port ID column 69E, a LUN column 69F, and an authentication information column 69G. In the logical device management table 69, one record corresponds to one logical device present in the local storage system 22.

[0097] The Logical Device ID field 69A stores the unique identifier (LDEV ID) assigned to the corresponding logical device within the local storage system 22, and the Controller ID field 69B stores the identifier (Controller ID) of the controller 32 responsible for reading and writing data to that logical device.

[0098] Furthermore, the "Physical Device Used" field 69C stores an identifier (PDEV ID) unique to the storage device within the storage system 22, which is assigned to a physical storage area of ​​a predetermined size associated with the corresponding logical device (this is the physical storage area provided by the corresponding storage device, and hereinafter referred to as the physical device).

[0099] Furthermore, the capacity field 69D stores the capacity of the logical device, and the public port ID field 69E stores the port ID of the port 81 that exposes the logical device to the host server 27. Note that there may be multiple ports 81 associated with a single logical device.

[0100] The LUN field 69F stores the unique identifier (LUN) assigned to the logical volume composed of that logical device within the local storage system 22. Furthermore, the authentication information field 69G stores authentication information such as the account and password used by the host server 27 to access the corresponding logical device.

[0101] Therefore, in the example in Figure 8, the logical device "LDEV1" is associated with the physical device "PDEV1," and its capacity is shown to be "5TB." Figure 8 also shows that the logical device is associated with controller 32, "CTRL0," as its assigned controller, and constitutes a logical volume of LUN "0." Furthermore, Figure 8 shows that the logical device is exposed to the host server 27 via two ports 81, "P0-A0-0" and "P1-A0-0," and that the authentication information is "john@XXX."

[0102] Note that Figure 8 shows a case where one logical volume is associated with one storage device, but this is merely an example. For example, it is also possible to create a large capacity pool from one or more physical devices, such as with thin provisioning, and then virtually carve out the required amount of storage space to use as a logical device.

[0103] Figure 9 shows the configuration of the remote path management table 70 stored in the controller's memory. The remote path management table 70 is a table used by the controller 32 to manage the paths (hereinafter referred to as remote paths) established between the storage system 24 (Figure 2) of the sub-site 23 (Figure 2).

[0104] Remote paths are configured by the user through the interface of the management server 28 (Figure 2), and this information is stored in the remote path management table 70. The controller 32 of the storage system 22 at the primary site 21 (Figure 2) then refers to the information stored in this remote path management table 70 and sends the data stored in the logical device that is the source of the remote replication, as well as various control information, to the storage system 24 at the secondary site 23.

[0105] As shown in Figure 9, the remote path management table 70 is configured to include a remote path ID field 70A, a local LDEV ID field 70B, a local port ID field 70C, a remote IP field 70D, a remote port number field 70E, a target IQN field 70F, a remote LUN field 70G, an authentication information field 70H, and a usage field 70I. In the remote path management table 70, one record corresponds to one remote path set up between the local storage system 22 and the remote replication destination storage system 24.

[0106] The Remote Path ID field 70A stores the identifier (Remote Path ID) unique to the remote path within the local storage system 22 that is assigned to the corresponding remote path. The Local LDEV ID field 70B stores the LDEV ID of the logical device on the local side (the local storage system 22 side, and so on) that is the target of remote replication, and the Local Port ID field 70C stores the port ID of the local port 81 (Figure 6) used for that remote replication.

[0107] The Remote IP field 70D stores the IP address set to the port on the remote side (the storage system 24 on the remote replication destination side, and so on hereafter) to which the remote path is connected, and the Remote Port Number field 70E stores the port number of the communication protocol of the remote storage system 24. Here, "port number" refers to a number used in a communication protocol to identify a program (service) that a computer uses for communication.

[0108] Furthermore, the Target IQN field 70F stores the identifier (IQN: iSCSI Qualified Name) of the iSCSI service at the remote replication destination, and the Remote LUN field 70G stores the identifier within the corresponding IQN for accessing the logical volume at the remote replication destination.

[0109] Furthermore, the authentication information field 70H stores authentication information such as accounts and passwords for accessing the logical volume of the remote replication destination, and the purpose field 70I stores the purpose of the remote connection. Such purposes include remote replication, external storage connection, and tier control. The purpose is also set by the user through the user interface of the management server 28 (Figure 2).

[0110] Therefore, in the example in Figure 9, the remote path "RP1" is a path used to "remotely replicate" data stored in the logical device "LDEV1" on the local side, and it is shown to be a path that connects port 81, "P0-A0-1" on the local side, with a port on the remote side that is assigned the IP address "192.0.100.1" and port number "3260".

[0111] Figure 9 also shows that the LUN of the logical volume on the remote replication destination side is "1", and the authentication information for accessing that logical volume is "john@XXX". Furthermore, Figure 9 shows that the IQN of the iSCSI target is "inq.2024-05.com.hatachi.iscsi:remote-1".

[0112] Figure 10 shows the configuration of the front-end interface management table 71 stored in the memory 62 (Figure 5) of the controller 32. The front-end interface management table 71 is a table used by the controller 32 to manage the front-end interfaces 33 (Figure 6) that exist within its own storage system 22.

[0113] As shown in Figure 10, the front-end interface management table 71 is configured to include a front-end interface ID column 71A, an installed controller column 71B, a port count column 71C, and a CPU core count column 71D. In the front-end interface management table 71, one record corresponds to one front-end interface 33 present in the storage system 22.

[0114] The Frontend Interface ID field 71A stores the identifier (Frontend Interface ID) that is unique to the corresponding Frontend Interface 33 within the local storage system 22, which has been assigned to that Frontend Interface 33.

[0115] Furthermore, the controller ID of the controller 32 on which the front-end interface 33 is installed is stored in the mounted controller field 71B, and the number of ports field 71C is stored in the number of ports 81 that the corresponding front-end interface 33 has.

[0116] Furthermore, the CPU core count field 71D stores the number of CPU cores 83A (Figure 6) of the CPU 83 (Figure 6) of the front-end interface 33. In this embodiment, CPU cores 83A are allocated equally to each port 81. Therefore, for example, if there are 2 ports 81 and 8 CPU cores 83A, the number of CPU cores per port will be 4.

[0117] Therefore, in the example shown in Figure 10, the controller 32 named "CTRL0" is equipped with a front-end interface 33 named "FE0-A", and this front-end interface 33 has "2" ports 81 and "8" CPU cores 83A.

[0118] Figure 11 shows the configuration of the session management table 72 stored in the memory 62 (Figure 5) of the controller 32. The session management table 72 is a table used by the controller 32 to manage sessions established with other storage systems and host servers 27.

[0119] The controller 32 communicates with all front-end interfaces 33 and manages virtual sessions, which are bundles of multiple real sessions established by each front-end interface 33, and the real session established by a front-end interface 33, as separate sessions in the session management table 72. Since all real sessions constituting a single virtual session managed by this session management table 72 are established on the same remote path, the local IP address, remote IP address, remote port number, target IQN, session purpose, and authentication information of these real sessions are all the same.

[0120] As shown in Figure 11, the session management table 72 is configured to include a session ID field 72A, a local port ID field 72B, a local IP field 72C, a remote IP field 72D, a remote port number field 72E, a target IQN field 72F, a session status field 72G, a session purpose field 72H, and an authentication information field 72I. In the session management table 72, one record corresponds to one session (a real session or a virtual session).

[0121] The Session ID field 72A stores the session ID, which is a unique identifier assigned to the corresponding session within the local storage system 22. The Local Port ID field 72B stores the port ID of the local port 81 (Figure 6) used by that session, and the Local IP field 72C stores the IP address set for that port 81.

[0122] The Remote IP field 72D stores the IP address of the remote port 81 used by the session, and the Remote Port Number field 72E stores the port number of the remote storage system 24. Furthermore, the Target IQN field 72F stores the IQN of the iSCSI target to which the session is connected.

[0123] The session status field 72G stores the state of the session (hereinafter referred to as the session state). The session states include "Not connected," "TCP connected," "Normal," "Faulting," and "Disconnected," and the corresponding session state is stored in the session status field 72G. Furthermore, if the session is a virtual session, the session status field stores an abstract representation of the states of the multiple real sessions that make up that virtual session.

[0124] Furthermore, the session purpose field 72H stores the purpose of use for that session. There are two purposes: "discovery" and "IO," and one of the corresponding one is stored in the session purpose field 72H. "Discovery" refers to the process in which the initiator communicates with the target to find out the target's IQN, IP address, or port number, while "IO" refers to the process of reading and writing data. In addition, the authentication information field 72I stores the authentication information used when the session was connected.

[0125] Therefore, in the example shown in Figure 11, the session with session ID "S1" is a session established between port 81 on the local side, which has port ID "P0-A0-1" and IP address "192.0.10.1", and port 81 on the remote storage system 24, which has port number "3260" and IP address "192.0.100.1". The purpose is "IO", the current session state is "normal", and the authentication information for that session is "john@XXX". Figure 11 also shows that the iSCSI target's IQN is "inq.2024-05.com.hatachi.iscsi:remote-1".

[0126] On the other hand, Figure 12 shows the virtual session management table 91 stored in the memory 84 (Figure 6) of the front-end interface 33 (Figure 6). The virtual session management table 91 is a table used by the CPU 83 of the front-end interface 33 to manage the relationship between actual sessions it has established with other storage systems 24 and virtual sessions recognized by the controller 32 (managing multiple actual sessions as a single virtual session).

[0127] This virtual session management table 91 is configured to include a virtual session ID field 91A, a usage field 91B, an actual session ID field 91C, a CPU core number field 91D, a local port ID field 91E, a local IP field 91F, a local port number field 91G, a remote IP field 91H, a remote port number field 91I, a target IQN field 91J, and a session status field 91K.

[0128] In the virtual session management table 91, each row in the Virtual Session ID column 91A, Usage column 91B, Local Port ID column 91E, Local IP column 91F, Remote IP column 91H, Remote Port Number column 91I, and Target IQN column 91J corresponds to one virtual session, and each row in the Actual Session ID column 91C, CPU Core Number column 91D, Local Port Number column 91G, and Session Status column 91K corresponds to one actual session that constitutes the corresponding virtual session.

[0129] The virtual session ID field 91A stores the session ID of the corresponding virtual session, and the purpose field 91B stores the purpose of that virtual session. The actual session ID field 91C is divided into several sub-fields, each corresponding to an actual session that makes up the corresponding virtual session, and the session ID assigned to the actual session is stored in each of these sub-fields.

[0130] Furthermore, the CPU core number field 91D is divided into multiple sub-fields, each corresponding to a real session that makes up the virtual session. Within these sub-fields, the identifier (CPU core number) of the CPU core 83A (Figure 6) assigned to the corresponding real session and responsible for processing that session is stored.

[0131] The Local Port ID field 91E stores the port ID of local port 81 (Figure 6) used by each actual session that constitutes the corresponding virtual session, and the Local IP field 91F stores the IP address set for that port 81.

[0132] Furthermore, the local port number field 91G is divided into multiple sub-fields, each corresponding to a physical session that makes up the corresponding virtual session. Within these sub-fields, the port number of local port 81 used by the corresponding physical session is stored.

[0133] Furthermore, the Remote IP field 91H stores the IP address set on the remote port 81 to which each corresponding session is connected, and the Remote Port Number field 91I stores the port number of the remote port 81 to which each corresponding session is connected. Additionally, the Target IQN field 91J stores the IQN of the iSCSI target to which each corresponding session is connected.

[0134] Furthermore, the session status field 91K is divided into several sub-fields, each corresponding to a real session that makes up the corresponding virtual session. The session status of the corresponding real session is stored in each of these sub-fields. The session statuses include "Not connected," "TCP connected," "Normal," "Faulting," and "Disconnected," and one of these will be stored in the session status field 91K.

[0135] Therefore, in the example shown in Figure 12, the virtual session assigned the session ID "S1" is used for "IO" and virtualizes four actual sessions: "S1-1", "S1-2", "S1-3", and "S1-4". It is shown that the processing of each of these actual sessions is handled by CPU core 83A, which is assigned the CPU core number "1", "2", "3", or "4", respectively.

[0136] Figure 12 also shows that each actual session uses port 81 on the local side, which is assigned the port ID "P0-A0-1", has the IP address "192.0.30.1", and is assigned port numbers "40001" to "40004", while the remote side uses port 81 on the remote side, which is assigned the IP address "192.0.100.1" and is assigned port number "3260".

[0137] Furthermore, Figure 12 shows that the IQN of the iSCSI target to which each of these actual sessions is connected is "iqn.2024-05.com.hatachi.iSCSI:remote-1", and that the session status of these actual sessions is currently "normal".

[0138] (1-4) IO command format Figure 13 shows an example of the command format of an IO command used for communication between the controller 32 and the front-end interface 33 mounted on the controller 32.

[0139] This IO command has a session ID area CA1, a command handle area CA2, a SCSI CDB area CA3, a LUN area CA4, a cache address area CA5, and a data length area CA6.

[0140] The session ID area CA1 stores the session ID of the virtual session targeted by the IO command, and the command handle area CA2 stores the identifier (command handle) unique to that IO command. The SCSI CDB area CA3 is the CDB (Command Descriptor Block) in the SCSI protocol, and the contents of the IO command are written in this SCSI CDB area CA3.

[0141] Furthermore, the LUN area CA4 stores the LUN of the logical volume to be I / O, and the cache address area CA5 stores the starting address of the storage area in the cache 63 (Figure 5) in the controller 32 where the data to be read or written is stored or should be stored. In addition, the data length area CA6 stores the data length of the data to be read or written.

[0142] After establishing a session, the controller 32 sends an I / O command in the command format shown in Figure 13 to the front-end interface 33, instructing it to read and write data to the remote replication destination storage system 24.

[0143] The front-end interface 33, upon receiving this IO command, dynamically determines which CPU core 83A should process the IO command based on the command handle contained within it. The specific method for determining which CPU core 83A to process will be described later.

[0144] (1-5) Flow of various processes related to remote replication Next, the flow of various processes performed in relation to remote replication in the information processing system 1 according to the above-described embodiment will be explained. In the following, the processing entity of some processes may be described as a "program," but in practice, it goes without saying that the CPU 61 of the controller 32 of the storage system 22 (Figure 5) and the CPU core 83A of the front-end interface 33 (Figure 6) execute the processes based on that program.

[0145] (1-5-1) Session establishment process Figure 14 shows the flow of a series of processes (hereinafter referred to as the session establishment process) when establishing a session between the initiator storage system 22 installed at the primary site 21 and the target storage system 24 installed at the secondary site 23. The initiator storage system 22 establishes an iSCSI session with the target storage system 24 according to the flow shown in Figure 14 and uses it for communication of control information and data for remote replication.

[0146] In practice, in response to user instructions, the initiator storage system 22 initiates a session connection with the target storage system 24, and the following session connection preprocessing is performed first (S1).

[0147] Specifically, the remote replication control program 67 (Figure 5) of the controller 32 of the initiator storage system 22 instructs the session control program 65 (Figure 5) to connect to each remote path pre-configured by the user, which is stored in the remote path management table 70 (Figure 9). This instruction includes information from each record in the remote path management table 70 that corresponds to the remote path used by the session instructed by the user to connect.

[0148] Upon receiving this instruction, the session control program 65 stores the necessary information regarding the remote path used by the session instructed to connect by the user in the instruction in the session management table 72 (Figure 11) as session information for the session it is trying to connect to at that time, and sets the session status of that session to "Not connected".

[0149] Furthermore, if the number of sessions registered in the session management table 72 exceeds a predetermined threshold set in advance, the session control program 65 interrupts the session connection process, responds with an error to the remote replication control program 67, and terminates the session establishment process.

[0150] Next, the session control program 65 sends a TCP connection request to the front-end interface 33 (S2). This TCP connection request includes all the information registered in the session management table 72 for the corresponding session.

[0151] Then, the front-end session control program 86 (Figure 6) of the front-end interface 33, which received this TCP connection request, calculates the number of sessions to connect to the target storage system 24 (S3).

[0152] In this embodiment, for I / O sessions, the number of sessions per target storage system 24 is equal to the number of CPU cores 83A assigned to port 81 (Figure 6), allowing for load balancing among the CPU cores 83A. As mentioned above, for I / O sessions, CPU cores 83A are equally allocated to each port 81. For discovery sessions with low processing load, the number of sessions per target is set to "1". The above number of sessions is merely an example, and the number of sessions may be changed depending on the required performance and application.

[0153] Next, the front-end session control program 86 of the front-end interface 33 stores information on all actual sessions that are about to connect in the virtual session management table 91 (Figure 12). At this time, the status of each actual session is set to "Not connected" (S4).

[0154] In this case, if the number of virtual sessions registered in the virtual session management table 91 exceeds a predetermined threshold, the front-end session control program 86 interrupts the process, sends an error to the session control program 65 of the controller 32, and then terminates the session establishment process.

[0155] Subsequently, the front-end session control program 86 establishes TCP connections between the target storage system 24 and the required actual sessions by executing the processes in steps S5 and S6 in parallel. At this time, the front-end session control program 86 assigns a different CPU core 83A to each actual session.

[0156] Specifically, the front-end session control program 86 establishes a TCP connection with the target storage system 24 for each actual session by sending a TCP connection request to the target storage system 24 (S5).

[0157] In this case, the connection information used for such TCP connection requests is the information contained in the TCP connection request provided by the session control program 65 of the controller 32 in step S2, and this is used for all actual sessions. In other words, the front-end session control program 86 reuses this information for all actual sessions.

[0158] Next, the front-end session control program 86 updates the information registered in the virtual session management table 91 for each actual session that established a TCP connection with the target storage system 24 in step S5 (S6).

[0159] Specifically, the front-end session control program 86 changes the value of the session status column 91K (Figure 12) corresponding to each actual session in the virtual session management table 91 to "TCP connected". The front-end session control program 86 also stores the core number of the CPU core 83A assigned to each actual session in the CPU core number column 91D (Figure 12) corresponding to each actual session in the virtual session management table 91.

[0160] The front-end session control program 86 executes the series of processes in steps S5 and S6 in parallel, in a number of parallel operations equal to the number of actual sessions determined in step S3. This suppresses the increase in processing time for TCP connection processing due to an increase in the number of sessions.

[0161] Furthermore, if the processing in step S5 fails for even one actual session, the front-end session control program 86 interrupts the TCP connection processing for all actual sessions, sends an error to the session control program 65 of the controller 32, and then terminates this session establishment process. This is because, since each actual session shares a physical path, if even one actual session fails to establish a TCP connection, it can be determined that the other actual sessions are in the same state.

[0162] Meanwhile, when the front-end session control program 86 has finished executing steps S5 and S6 for all actual sessions and has finished establishing TCP connections for these actual sessions to the target storage system 24, it sends a response to that effect to the session control program 65 of the controller 32 (S7).

[0163] Upon receiving this response, the session control program 65 of the controller 32 updates the session status of the corresponding virtual session in the session management table 72 (Figure 11) to "TCP connected" (S8).

[0164] The session control program 65 also sends a login request to the front-end interface 33, the source of the response received in step S7, requesting login to the virtual session whose state was updated in step S8 (S9). This login request includes all the contents of the record corresponding to this virtual session in the session management table 72.

[0165] The front-end session control program 86 of the front-end interface 33, upon receiving this login request, then performs login to these real sessions (S10) by having the CPU cores 83A assigned to each real session execute the processes of steps S10 and S11 in parallel for each real session that constitutes the virtual session.

[0166] In practice, the front-end session control program 86 sends an iSCSI login request to the target storage system 24 for each CPU core 83A assigned to each actual session in step S5. This establishes a TCP connection for each actual session. At this time, the front-end session control program 86 uses the information received from the controller's 32 session control program 65 in step S9 as connection information for all actual sessions to be used in the iSCSI login request.

[0167] The front-end session control program 86 also updates the virtual session management table 91 to set the status of each actual session to "normal" (S11).

[0168] Then, once the front-end session control program 86 has completed the processing in steps S10 and S11 for all actual sessions, it terminates the iSCSI login process for each actual session.

[0169] In this case, if the iSCSI login process in step S10 fails for even one actual session, the front-end session control program 86 interrupts the iSCSI login process for all actual sessions, sends an error to the session control program 65 of the controller 32, and terminates this session establishment process.

[0170] Subsequently, the front-end session control program 86 responds to the session control program 65 of the controller 32 that the login for the virtual session requested in step S9 has been completed (S12).

[0171] Upon receiving this response, the session control program 65 updates the session management table 72 (S13). Specifically, the session control program 65 updates the value stored in the session status column 72G (Figure 11) of the record corresponding to that virtual session in the session management table 72 to "normal".

[0172] With this, the session establishment process is completed. Through this session establishment process, a different actual session is established for each CPU core 83A when a session connection for I / O is established in the front-end interface 33. This will distribute the load among the CPU cores 83A for subsequent I / O processing.

[0173] (1-5-2) Write IO command processing Figure 15 shows the flow of a series of processes (hereinafter referred to as write I / O command processing) when the initiator storage system 22 performs remote replication to the target storage system 24.

[0174] The remote replication control program 67 of the controller 32 issues control information and I / O commands for reading and writing replicated data to the target storage system 24 to the session control program 65 when remote replication is performed. The session control program 65 then creates a write I / O command according to these I / O commands (S20).

[0175] In this case, the session control program 65 uses a value obtained by recalculating the starting address on the cache 63 (Figure 5) where the write I / O command is stored using an arbitrary hash function as the handle value for the write I / O command. This prevents bias from occurring when calculating the CPU core 83A responsible for processing the write I / O command from the handle value, as will be described later.

[0176] The hash function referred to here is a function that calculates arbitrary data using a specific procedure and outputs a fixed-length random string (hash value). However, various other methods can be broadly applied to the calculation method of the handle value for write I / O commands. For example, a random number may be used as the handle value. Alternatively, a value other than a number may be used as the command handle value (command handle information).

[0177] The session control program 65 also stores the handle value of the created write I / O command and uses it for abort command processing, which will be described later with respect to Figure 16.

[0178] Returning to the explanation of Figure 15, when the session control program 65 of the controller 32 creates a write I / O command in step S20, it sends the write I / O command and the data to be written (hereinafter referred to as the write data) to the front-end interface 33 (S21).

[0179] The front-end session control program 86 of the front-end interface 33, upon receiving this write I / O command and write data, determines whether or not this write I / O command targets a virtual session composed of multiple real sessions (S22).

[0180] Then, when the front-end session control program 86 confirms that the write I / O command targets a virtual session composed of multiple real sessions, it calculates the core number of the CPU core 83A that should process the write I / O command (hereinafter referred to as the processing CPU core number) using the following formula (S23).

[0181]

number

[0182] In other words, the front-end session control program 86 calculates the processing CPU core number by adding the CPU core number offset of port 81 to the remainder (surplus) obtained by dividing the handle value of the write I / O command by the number of CPU cores 83A assigned to the corresponding port 81.

[0183] In this context, "corresponding port" refers to port 81 within the front-end interface 33, which is session-connected to the target storage system 24 to which the data is written using the write I / O command.

[0184] Furthermore, the "CPU core number offset" here refers to the CPU core number of the first CPU core 83A among the multiple CPU cores 83A assigned to each port 81. In this embodiment, CPU cores 83A with consecutive CPU core numbers are evenly allocated to each port 81.

[0185] For example, if there are two ports 81 and eight CPU cores, the first port 81 will be assigned four CPU cores 83A with CPU core numbers "0" to "3", and the second port 81 will be assigned four CPU cores 83A with CPU core numbers "4" to "7". Therefore, in this example, the "CPU core number offset" mentioned above will be "0" for the first port 81 and "4" for the second port 81.

[0186] Furthermore, if the write I / O command from the controller 32 in step S22 does not target a virtual session composed of multiple real sessions (i.e., such a virtual session consists of only one real session), the front-end session control program 86 determines the CPU core number of the CPU core 83A responsible for that real session as the processing CPU core number.

[0187] Next, the front-end session control program 86 enqueues the write I / O command obtained in step S21 to the I / O command queue managed by CPU core 83A, which has the processing CPU core number calculated in step S23 (S24).

[0188] As a result, the CPU core 83A executes the iSCSI initiator program 90 (Figure 6) and sends its write I / O command to the target storage system 24 (S25, S26).

[0189] Specifically, the iSCSI initiator program 90, which is executed by the CPU core 83A, creates an iSCSI_PDU (Protocol Data Unit) for the write I / O command (S25). Here, "iSCSI_PDU" is an information unit that encapsulates the aforementioned write I / O command for accessing the target storage system 24. The iSCSI initiator program 90 then sends the created iSCSI_PDU and the write data to the target storage system 24 (S26).

[0190] Meanwhile, when the target storage system 24 receives the iSCSI_PDU and write data, it performs an I / O operation (write operation) to store the write data in the corresponding logical device within the corresponding logical volume, in accordance with the write I / O command contained in the iSCSI_PDU (S27). When the target storage system 24 completes such I / O operation, it sends a response to that effect to the front-end interface 33 in the initiator storage system 22 that sent the iSCSI_PDU (S28).

[0191] Upon receiving this response, the iSCSI initiator program 90, which is being executed by the corresponding CPU core 83A of the front-end interface 33, responds to its own controller 32 that the data writing is complete (S29). The iSCSI initiator program 90 also dequeues the aforementioned write I / O command from the corresponding I / O command queue (S30), and then terminates the series of processes.

[0192] The above description explained the case where the IO command sent from the controller 32 to the front-end interface 33 is a write IO command, but the same procedure is followed when such an IO command is a read IO command that reads data from the target storage system 24.

[0193] Furthermore, the processing in steps S22 to S26 of Figure 15 is executed each time an IO command request is sent from the controller 32 to the front-end interface 33. In this embodiment, the front-end interface 33 can load distribute the IO commands from the controller 32 among the actual sessions, thereby enabling load distribution among the CPU cores 83A.

[0194] Furthermore, in this embodiment, the front-end interface 33 dynamically calculates the CPU cores 83A that process I / O commands from the controller 32 based on the content of those I / O commands, thereby enabling load balancing among the CPU cores 83A with minimal processing overhead.

[0195] (1-5-3) Processing Abort Commands Figure 16 shows the sequence of processes (hereinafter referred to as abort command processing) when the initiator storage system 22 interrupts the execution of an I / O command already issued to the target storage system 24.

[0196] The initiator storage system 22 can interrupt the execution of I / O commands already issued to the target storage system 24 by issuing an abort command to the target storage system 24.

[0197] In practice, when the controller 32 of the initiator storage system 22 times out during IO command processing or receives an abort instruction from the remote replication control program 67 (Figure 5), it starts processing abort commands for issued IO commands and first creates an abort command (S40).

[0198] The abort command created at this time includes the session ID of the virtual session used by the I / O command to be aborted and the handle value of the I / O command to be aborted. The controller 32 then sends the created abort command to the front-end interface 33 that issued the I / O command to be aborted (S41).

[0199] Upon receiving this abort command, the front-end session control program 86 of the front-end interface 33 refers to the virtual session management table 91 to determine whether the virtual session connected to the target storage system 24 to which the aborted IO command is sent is composed of multiple real sessions (S42).

[0200] If the front-end session control program 86 obtains a positive result from this determination, it determines the CPU core number (processing CPU core number) of the CPU core 83A that should process the abort command using the following formula (S43).

[0201]

number

[0202] In other words, the front-end session control program 86 calculates the processing CPU core number by adding the CPU core number offset of port 81 to the remainder (surplus) obtained by dividing the handle value of the I / O command to be aborted in the abort command by the number of CPU cores 83A assigned to port 81 from which the I / O command was issued. As a result, the CPU core number of the CPU core 83A that processed the command to be aborted is calculated as the CPU core number of the CPU core 83A that should process the abort command.

[0203] In response to this, the front-end session control program 86 determines the CPU core number of the CPU core 83A responsible for processing the abort command if the virtual session connected to the target storage system 24 to which the aborted IO command is sent consists of a single physical session. This is because, if there is only one physical session, there is only one CPU core assigned to that virtual session, and there is no need to distinguish CPU cores according to the command type.

[0204] Next, the front-end session control program 86 forwards an abort command to the iSCSI initiator program 90 (hereinafter referred to as the assigned iSCSI initiator program) being executed by the CPU core 83A to which the CPU core number calculated in step S43 has been assigned (S44).

[0205] The iSCSI initiator program 90 then first searches the command queue of the CPU core 83A on which it is running to determine the processing status of the I / O command to be aborted (S45).

[0206] Then, if the iSCSI initiator program 90 is processing the I / O command to be aborted and has already been sent to the target storage system 24, it proceeds to step S46.

[0207] Furthermore, if the iSCSI initiator program 90 is processing an I / O command to be aborted but has not yet been sent to the target storage system 24, it interrupts the processing of that I / O command, proceeds to step S50, and responds to the controller 32 that the abort was successful.

[0208] Furthermore, if the iSCSI initiator program 90 determines that the processing of the I / O command to be aborted has been completed on the target storage system 24, it proceeds to step S50 and reports the abort failure to the controller 32.

[0209] Meanwhile, the iSCSI initiator program 90, upon proceeding to step S46, creates a TMF (Task Management Function) Request PDU to abort the I / O command to be aborted (S46), and sends the created TMF Request PDU to the target storage system (S47). Here, "TMF Request" refers to a command request for managing tasks in the iSCSI protocol.

[0210] Upon receiving this TMF Request PDU, the target storage system 24 aborts the I / O command to be aborted if possible (S48) and responds to the initiator storage system 22 with the result (S49).

[0211] The iSCSI initiator program 90 of the initiator storage system 22, upon receiving this response, then responds to its controller 32 regarding the success or failure of the abort (S50). With this, the abort command processing is completed.

[0212] As described above, in this embodiment, the CPU core 83A responsible for processing the actual session used by the I / O command is identified from the handle value of the I / O command to be aborted. Therefore, even when a virtual session consists of multiple actual sessions, the actual session to be aborted can be identified within the front-end interface 33, and the I / O command to be aborted can be aborted.

[0213] (1-5-4) Session Disconnection Process Figure 17 shows the flow of a series of processes (hereinafter referred to as session disconnection processes) executed by the initiator storage system 22 to disconnect the session with the target storage system 24 when a communication failure occurs between the initiator storage system 22's front-end interface 33 and the target storage system 24.

[0214] As described later, if even one of the multiple real sessions that constitute the same virtual session established with the target storage system 24 is disconnected, the front-end interface 33 of the initiator storage system 22 disconnects all remaining real sessions established with the target storage system 24.

[0215] This is because all these real sessions communicate using the same IP address and therefore share the same physical communication path. For this reason, if a communication failure is detected in one real session, it can be assumed that a communication failure has also occurred in the remaining real sessions.

[0216] In this context, a communication failure refers to a failure in which communication with the target storage system 24 becomes impossible due to a malfunction in port 81 of the front-end interface 33 (Figure 6) or equipment such as a switch on the network path.

[0217] Then, when the iSCSI initiator program 90, which is being executed by the CPU core 83A, detects such a communication failure, it notifies the front-end session control program 86 that a failure has occurred in the actual session (S60). Here, as methods for detecting such communication failures, we assume monitoring of hardware failure information by the iSCSI initiator program 90 and reception of a forced disconnection from the target storage system 24.

[0218] Upon receiving this notification, the front-end session control program 86 executes a session disconnection process in parallel, disconnecting the actual session where the iSCSI initiator program 90 detected a communication failure, and all other actual sessions that constitute the same virtual session as that actual session (S61).

[0219] Specifically, the front-end session control program 86 instructs each iSCSI initiator program 90, which is executed by each CPU core 83A responsible for processing each actual session that constitutes the virtual session, to disconnect the actual session. Thus, each iSCSI initiator program 90 that receives this instruction forcibly disconnects the session with the corresponding target storage system 24 and releases the session-related resources within the front-end interface 33.

[0220] By performing the disconnection process for multiple actual sessions in parallel in this way, it is possible to suppress the increase in session disconnection time that occurs as the number of sessions increases.

[0221] Then, when the front-end session control program 86 has finished disconnecting all corresponding real sessions, it notifies its own controller 32 that the virtual sessions composed of those real sessions have been disconnected (S62).

[0222] Furthermore, upon receiving this notification, the session control program 65 (Figure 5) of the controller 32 updates the value stored in the session status column 72G of the record corresponding to the virtual session in the session management table 72 (Figure 11) to "Faulty," and releases all available resources associated with that virtual session.

[0223] This concludes the session termination process.

[0224] (1-5-5) Session monitoring process Figure 18 shows the flow of session monitoring processing performed between the initiator storage system 22 and the target storage system 24. The front-end interface 33 of the initiator storage system 22 monitors the status of its own controller 32 by ping (status monitoring notification).

[0225] Furthermore, the front-end interface 33 responds to pings from the target storage system 24 for monitoring the status of each actual session only when the controller 32 is in a normal state, and does not respond when there is no response to the ping from the controller 32.

[0226] As a result, in this information system 1, the target storage system 24 can only receive a ping response from the initiator storage system 22 if the initiator storage system 22 can communicate with the target storage system 24 successfully, including the status of the controller 32.

[0227] In this context, "ping" refers to a communication method used to check network connectivity. It involves sending a string to a specified recipient and verifying network connectivity based on whether or not a response is received. The same applies below.

[0228] Specifically, the front-end session control program 86 of the front-end interface 33 periodically issues a ping to its own controller 32 for monitoring its status (S70).

[0229] Then, the session control program 65 of the controller 32, upon receiving this ping, updates the status of the session offloaded to its front-end interface 33 in the session management table 72 to "normal" (updating the value stored in the corresponding session status column 72G in the session management table 72 to "normal") (S71), and then sends a ping response to its front-end interface 33 (S72).

[0230] Furthermore, upon receiving this response, the front-end session control program 86 of the front-end interface 33 updates the status of each actual session that constitutes each virtual session managed in the virtual session management table 91 (Figure 12) to "normal" (updating the values ​​stored in each session status column 91K in the virtual session management table 91 to "normal") (S73).

[0231] Furthermore, if the front-end session control program 86 does not receive a ping response from the controller 32 after a certain period of time has elapsed since sending a ping to the controller 32 in step S70, it updates the status of each actual session that constitutes each virtual session managed in the virtual session management table 91 to "failed".

[0232] Meanwhile, the target storage system 24 also periodically sends a ping for monitoring its status for each actual session established with the initiator storage system 22 (S74).

[0233] The front-end interface 33 of the initiator storage system 22, upon receiving this ping, determines whether or not a response to this ping is required (S75).

[0234] Specifically, in the front-end interface 33, each iSCSI initiator program 90 (Figure 6) executed by a CPU core 83A assigned to each actual session notifies the front-end session control program 86 that it has received a ping from the target storage system 24.

[0235] Furthermore, upon receiving this notification, the front-end session control program 86 refers to the virtual session management table 91 to determine whether the status of all corresponding real sessions with its initiator storage system 22 is "normal".

[0236] The front-end session control program 86 then does not send a ping response to the target storage system 24 if it obtains a negative result from the determination, and only sends a ping response to the target storage system 24 if it obtains a positive result from the determination (S76). The front-end session control program 86 performs this process in step S76 for all actual sessions.

[0237] With the above steps completed, the target storage system 24 completes monitoring the status of the initiator storage system 22, and thereafter, the process from step S70 onward is repeated periodically.

[0238] (1-6) Effects of this embodiment As described above, in the information processing system 1 of this embodiment, the front-end interface 33 of the initiator storage system 22 connects multiple actual sessions with the target storage system 24, and assigns a different CPU core 83A to each actual session.

[0239] The front-end interface 33 then distributes the I / O commands received from the controller 32 evenly among the CPU cores 83A assigned to each of the multiple actual sessions for processing.

[0240] Therefore, since the I / O commands given from the controller 32 to the front-end interface 33 can be processed by multiple CPU cores 83A, the load on the CPU cores 83A can be distributed, and it is possible to prevent the performance of the CPU cores 83A from becoming a bottleneck in communication performance.

[0241] Furthermore, in this information processing system 1, the controller 32 manages multiple actual sessions as a single virtual session. Therefore, the number of session information that the controller 32 needs to manage using the session management table 72 (Figure 11) stored in memory 62 (Figure 5) can be kept to the same level as before, thereby reducing the consumption of memory 62.

[0242] Therefore, according to the information processing system 1 of this embodiment, communication performance can be improved while suppressing the consumption of the memory 62 of the controller 32.

[0243] (2) Second embodiment Figure 19, which uses the same reference numerals as Figure 1 to indicate corresponding parts, shows an overview of the information processing system 100 according to the second embodiment. In this embodiment as well, remote replication is performed from the storage system (initiator storage system) 3 at the primary site 2 to the storage system (target storage system) 102 at the secondary site 5 via the network 4, just as in the first embodiment.

[0244] However, in this embodiment, the controller 103 of the target storage system 102 at sub-site 5 is also equipped with one or more front-end interfaces 104, and unlike the first embodiment, it offloads communication for remote replication from the controller 103 to the front-end interface 104, similar to the initiator storage system 3 (storage system 22 in Figure 2) in the first embodiment.

[0245] In this embodiment, when the front-end interface 104 of the target storage system 102 receives requests from the initiator storage system 3 (storage system 22 in Figure 2) to establish multiple real sessions 14 for remote replication, it presents these established real sessions 14 to the controller 103 as a single virtual session.

[0246] More specifically, the front-end interface 104 of the target storage system 102 registers and manages these multiple actual sessions 14 in a virtual session management table 105, which has a configuration similar to the virtual session management table 91 described above for Figure 12, and manages them internally.

[0247] Furthermore, the controller 103, which is equipped with the front-end interface 104, manages the virtual session consisting of these actual sessions 14 as a single session using a session management table 106 that has the same configuration as the session management table 72 described above in Figure 11.

[0248] This reduces the number of sessions that the controller 103 of the target storage system 102 needs to manage using the session management table 106. Furthermore, this method allows for the establishment of more actual sessions 14 between the initiator storage system 3 and the target storage system 102 without increasing the resource consumption of the controller 103 of the target storage system 102.

[0249] Therefore, according to this embodiment, the target storage system 102 can also improve communication performance while suppressing memory consumption of the controller 103.

[0250] In this embodiment, the target storage system 102 has the same hardware configuration, configuration of the various management tables it holds, and flow of various processes as the initiator storage system 3 (storage system 22 in Figure 2) according to the first embodiment, so its explanation will be omitted.

[0251] (3) Third Embodiment Figure 20, which uses the same reference numerals as Figure 1 to indicate corresponding parts, shows an overview of the information processing system 110 according to the third embodiment. In this embodiment as well, remote replication is performed from the storage system (initiator storage system) 111 at the primary site 2 to the storage system (target storage system) 112 at the secondary site 5 via the network 4, similar to the first embodiment.

[0252] However, in this embodiment, the difference from the first embodiment is that the front-end interface 121 in the initiator storage system 111 converts session connection requests and I / O commands from the controller 120 into a multi-core compatible protocol such as NVMe (Non-Volatile Memory Express) / TPC protocol and sends it to the target storage system 112.

[0253] The NVMe / TPC protocol enables load balancing across multiple cores by connecting multiple TCP / IP connections within a single session between the initiator and target, and configuring an independent queue for each connection. Therefore, even when using a single session, it is possible to create the same number of connections as the CPU cores, thereby enabling load balancing across the CPU cores within the front-end interface 121.

[0254] In this embodiment, the front-end interface 121 of the initiator storage system 111 presents multiple NVMe / TPC sessions with the initiator storage system 111 to the controller 120 as virtual sessions. This enables NVMe / TPC communication for remote replication with the same resource consumption of the controller 120 as in the conventional system. Furthermore, the front-end interface 121 of the initiator storage system 111 enables load balancing among the CPU cores 13, thereby improving communication performance.

[0255] Therefore, according to the information processing system 110 of this embodiment, similar to the information processing system 1 of the first embodiment, it is possible to improve communication performance while suppressing the memory consumption of the controller 120 of the initiator storage system 111.

[0256] In this embodiment, the initiator storage system 111 is identical to the initiator storage system 3 (storage system 22 in Figure 2) in the first embodiment in terms of hardware configuration, the configuration of various management tables it maintains, and the flow of various processes, except that the communication protocol with the target storage system 112 is the NVMe / TPC protocol. Therefore, a detailed explanation of this is omitted.

[0257] (4) Other embodiments In the first to third embodiments described above, we described cases where smart NICs are applied as front-end interfaces 11, 33, 104, and 121 of storage systems 3, 22, 24, 102, 110, and 111. However, the present invention is not limited to these cases, and can also be applied to storage systems in which a controller performs communication processing without using smart NICs.

[0258] Furthermore, while the first to third embodiments described above describe the case where the iSCSI protocol or the NVMe / TPC protocol is applied as the communication protocol between the host server 27 and the storage systems 3, 22, 24, 102, 110, and 111, the present invention is not limited to these, and communication protocols such as FC (Fibre Channel)-SCSI or FC-NVMe may also be applied.

[0259] Furthermore, while the first to third embodiments described above describe a case where a storage node connects multiple real sessions to a target storage system, and these real sessions are used for remote replication, the present invention is not limited to this. The present invention can also be applied when such real sessions are used as sessions for processing commands that involve some kind of communication other than remote replication, and the communication destination is some target node other than a storage system.

[0260] It should be noted that the present invention is not limited to the first to third embodiments described above or any other embodiments described above, and includes various modifications. For example, the first to third embodiments described above are described in detail for the purpose of explaining the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described.

[0261] Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0262] Furthermore, some or all of the configurations, functions, and / or processing units in the first to third embodiments may be implemented in hardware, for example, by designing them as integrated circuits. Alternatively, each such configuration or function may be implemented in software in which the processor executes a program that implements that function. Information such as programs, tables, and / or files that implement each function can also be stored in memory, a hard disk drive, an SSD, or other recording device, or in an integrated circuit card or SD card.

[0263] Furthermore, the control lines and information lines shown are those deemed necessary for explanation purposes and do not necessarily represent all control lines and information lines required for the product. In reality, it can be assumed that almost all components are interconnected. [Industrial applicability]

[0264] The present invention can be applied to a storage system configured in which the controller offloads communication processing to a network interface. [Explanation of Symbols]

[0265] 1, 20, 100, 110... Information processing system, 2... Primary site, 3, 6, 22, 24, 102, 110, 111... Storage system, 5... Secondary site, 10, 22, 103, 120... Controller, 11, 33... Front-end interface, 61, 83... CPU, 64... Block storage control program, 65... Session control program, 66... ​​Front-end interface integrated control program, 67... Remote replication control program, 68... Port management table, 69... Logical device management table, 70... Remote path management table, 71... Front-end interface management table, 72... Session management table, 81... Network interface (port), 83A... CPU core, 86... Front-end session control program, 87... Front-end interface control program, 88... Protocol control program, 89... Block access program, 90... iSCSI initiator program, 91... Virtual session management table.

Claims

1. In a storage system that includes nodes and provides storage space for storing data in a higher-level device, A network interface is provided with a processor having multiple cores, each processing commands that involve communication with an external target of the node, A controller that controls the reading and writing of the aforementioned data, and issues the aforementioned commands involving communication with the target, causing the network interface to process them, The node is provided with, The aforementioned network interface is Multiple first sessions are connected to the target, and a different core of the processor is assigned to each of the first sessions. Multiple first sessions with the target are managed as a single virtual second session. Using the second session which uses one of the multiple first sessions, the command issued from the controller, which involves communication with the target, is processed. A storage system characterized by the following features.

2. The aforementioned controller, A connection request for the second session, including information on the remote path established between the target and the network interface, is sent to the network interface. The aforementioned network interface is Based on the remote path information included in the connection request, the plurality of first sessions are connected to the target. The storage system according to feature 1.

3. The aforementioned network interface is The number of sessions of the first session connected to the target is determined according to the number of cores in the processor and the purpose of the session. The storage system according to feature 1.

4. The aforementioned core is associated with a port used for communication. The aforementioned command includes a command handle value, The aforementioned command handle value is associated with a port, The aforementioned network interface is Using the command handle value included in the command, the core that uses the port associated with the command handle value is determined as the core to which the processing of the command will be allocated. The storage system according to feature 1.

5. The aforementioned controller, The command handle information included in the command issued to the network interface is stored. When issuing an abort request to the network interface to abort the aforementioned command, include the command handle information of the command in the abort request. The aforementioned network interface is Based on the command handle information of the command to be aborted included in the abort request, the core responsible for the abort request is determined. The storage system according to feature 4.

6. The network interface monitors the status of the controller, Responding to a life-or-death monitoring notification from the aforementioned target. The storage system according to feature 1.

7. A control method for a storage system comprising nodes and providing storage areas for storing data in a higher-level device, The aforementioned storage system, A network interface is provided with a processor having multiple cores, each processing commands that involve communication with an external target of the node, A controller that controls the reading and writing of the aforementioned data, and issues the aforementioned commands involving communication with the target, causing the network interface to process them, The node has, The first step is that the network interface connects a plurality of first sessions to the target, assigns a different core of the processor to each of the first sessions, and manages the plurality of first sessions to the target as a single virtual second session. A second step of processing the command issued from the controller, which involves communication with the target, using a second session that uses one of the multiple first sessions; A method for controlling a storage system, characterized by comprising the following features.

8. In the first step described above, The aforementioned controller, A connection request for the second session, including information on the remote path established between the target and the network interface, is sent to the network interface. The aforementioned network interface is Based on the remote path information included in the connection request, the plurality of first sessions are connected to the target. A method for controlling a storage system according to feature 7.

9. In the first step described above, the network interface is The number of sessions of the first session connected to the target is determined according to the number of cores in the processor and the purpose of the session. A method for controlling a storage system according to feature 7.

10. The aforementioned core is associated with a port used for communication. The aforementioned command includes a command handle value, The aforementioned command handle value is associated with a port, In the second step described above, the network interface is Using the command handle value included in the command, the core that uses the port associated with the command handle value is determined as the core to which the processing of the command will be allocated. A method for controlling a storage system according to feature 7.

11. The aforementioned controller, The command handle information included in the command issued to the network interface is stored. When issuing an abort request to the network interface to abort the aforementioned command, include the command handle information of the command in the abort request. The aforementioned network interface is Based on the command handle information of the command to be aborted included in the abort request, the core responsible for the abort request is determined. A method for controlling a storage system according to feature 10.

12. The network interface monitors the status of the controller, Responding to a life-or-death monitoring notification from the aforementioned target. A method for controlling a storage system according to feature 7.

Images