Storage system and core allocation control method of network interface

Abstract

To realize core allocation control capable of suppressing influence of a process between ports and sufficiently exhibiting performance of the process of the port in a network interface mounted on a storage system.SOLUTION: A storage system includes a storage controller on which a network interface including a processor equipped with a plurality of cores, a memory, and a plurality of ports is mounted. The processor allocates at least one occupied core to each of the plurality of ports, controls the occupied core to execute a management process of the port to which the occupied core is allocated and a network protocol process of communication performed via the port to which the occupied core is allocated, and controls the cores other than the occupied core to execute the network protocol process of the communication performed via the port.SELECTED DRAWING: Figure 5C

Description

Technical Field

[0001] The present invention relates to a network interface mounted on a storage system.

Background Art

[0002] With the acceleration of networks and the emergence of new protocols, there is a demand for the acceleration of network interfaces and multi-protocol support in enterprise storage systems as well. Conventionally, a CPU on a storage controller has been used to support multiple protocols, or offloaded to protocol-specific hardware to support multiple protocols. For example, U.S. Patent No. 9,100,349 discloses a technique for supporting multiple protocols using a NIC (Network Interface Card) equipped with Ethernet and FC protocol chips.

[0003] However, when using the CPU of a storage controller for communication protocol processing, there is a problem that the CPU resources are consumed by protocol processing. When using a Channel Board (CHB) equipped with an ASIC dedicated to protocol processing, it is necessary to develop a dedicated ASIC for each protocol, and it is difficult to quickly support protocols.

[0004] On the other hand, SmartNICs having a general-purpose processor and memory and performing network protocol processing have emerged. In a SmartNIC, for example, the same operating system as that operating on a server system can be operated, and the software protocol stack and applications used therein can be operated. Since network protocol processing can be implemented in software, it is possible to quickly support multiple protocols and new protocols, and it is also possible to flexibly respond to updates of network protocol processing.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] A SmartNIC is equipped with a general-purpose processor having multiple cores, memory, and ports. In the core allocation control methods of the prior art, there are a method of allocating cores to each port at a predetermined ratio and a method of allocating cores to each port in a time-sharing manner.

[0007] In the case of the former allocation method, since the processing performance of one port depends on the number of allocated cores, there is a problem that the maximum performance of the SmartNIC cannot be exerted. On the other hand, if all cores are allocated to one port to exert the maximum performance, the network protocol processing of other ports cannot be executed.

[0008] In the case of the latter allocation method, if a management process such as resetting of other ports occurs during the execution of the network protocol processing of a certain port, the cores allocated to a certain port may be allocated to other ports, which may affect the network protocol processing of a certain port.

[0009] An object of the present invention is to realize core allocation control that suppresses the influence of processing between ports and can sufficiently exert the performance of network protocol processing of ports in a network interface mounted on a storage system.

Means for Solving the Problems

[0010] A typical example of the invention disclosed in the present application is as follows. That is, a storage system including at least one storage controller, wherein the storage controller is equipped with a processor having a plurality of cores, a memory connected to the processor, and a network interface having a plurality of ports connected to the processor. The processor assigns at least one occupied core to each of the plurality of ports. The occupied core is controlled to execute the management process of the port to which the occupied core is assigned and the network protocol process of the communication performed via the port to which the occupied core is assigned. Cores other than the occupied core are controlled to execute the network protocol process of the communication performed via the port.

Effect of the Invention

[0011] According to the present invention, in the network interface mounted on the storage system, it is possible to realize core allocation control that suppresses the influence of processing between ports and fully exhibits the performance of the network protocol processing of the port. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 6

Figure 7A

Figure 7B

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments will be described with reference to the drawings. Note that the following description and drawings are examples for explaining the present invention, and for the sake of clarity of explanation, appropriate omissions and simplifications are made. The present invention can be implemented in various other forms, and unless otherwise particularly limited, each component may be in a single or plural number.

[0014] Also, the embodiments described below do not limit the invention according to the claims, and not all combinations of elements described in the embodiments are essential for the solution means of the invention.

[0015] In the following description, various types of information may be described using expressions such as "table", "list", "queue", etc. However, various types of information may be represented by data structures other than these, and in order to indicate that they do not depend on the data structure, "table of xxx", "list of xxx", "queue of xxx", etc. may be referred to as "xxx information", etc. In the following description, when describing identification information, expressions such as "identification information", "identifier", "name", "ID", "number", etc. are used, but these can be mutually replaced.

[0016] In the following description, when there are a plurality of components having the same or similar functions, they are basically described with the same reference numerals. However, even if the functions are the same, the means for realizing the functions may be different. Furthermore, the embodiments described later may be implemented by software operating on a general-purpose computer, or may be implemented by dedicated hardware or a combination of software and hardware.

[0017] In the following description, the "program" may be used as the subject to explain the processing. However, the program is executed by a processor (e.g., CPU: Central Processing Unit), and for the defined processing, appropriate memory resources (e.g., memory) and / or interface devices (communication ports) are used as appropriate. Therefore, the subject of the processing may be explained as the processor.

[0018] The processing described with the program as the subject may also be the processing performed by a computer (e.g., a computing host, a storage device) having a processor. In the following description, the expression "controller" may refer to a processor or a hardware circuit that performs part or all of the processing performed by the processor.

[0019] The program may be installed on each computer from a program source (e.g., a program distribution server or a computer-readable storage medium). In this case, the program distribution server includes a CPU and memory resources. The memory resources further store the distribution program and the program to be distributed. By the CPU executing the distribution program, the CPU of the program distribution server may distribute the program to be distributed to other computers.

[0020] In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0021] In the following description, a memory drive or simply a drive means a physical storage device, and typically may be a non-volatile storage device (e.g., an auxiliary storage device). The drive may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Different types of drives may be mixed in the storage system.

[0022] Also, in the following description, a drive has a VOL, and "VOL" is an abbreviation for volume, which may be a physical storage device or a logical storage device. A VOL may be a real VOL (RVOL) or a virtual VOL (VVOL). "RVOL" may be a VOL based on the physical storage resources (e.g., one or more RAID groups) of the storage system having the RVOL.

[0023] "VVOL" may be any of an externally connected VOL (EVOL), a thin provisioning VOL (TPVOL), and a snapshot VOL. EVOL is based on the storage space (e.g., VOL) of an external storage system and may be a VOL that follows storage virtualization technology. TPVOL is composed of a plurality of virtual areas (virtual storage areas) and may be a VOL that follows capacity virtualization technology (typically Thin Provisioning).

[0024] Also, in the following description, a VOL recognized by a host (VOL provided to the host) is referred to as an "LDEV". In the following description, an LDEV is a TPVOL (or RVOL), and a pool is a TP pool. However, the features of the present disclosure can also be applied to a storage device that does not employ capacity expansion technology (Thin Provisioning).

[0025] A "pool (POOL)" is a logical storage area (e.g., a collection of a plurality of pool VOLs) and may be prepared for each use. For example, it may be a TP pool. A TP pool may be a storage area composed of a plurality of pages (physical storage areas). When a page is not allocated to the virtual area (virtual area of TPVOL) to which the address specified by the write request received from a host computer (hereinafter referred to as a host) belongs, a page is allocated from the TP pool to that virtual area (write destination virtual area) (even if a page has already been allocated to the write destination virtual area or a new page is allocated to the write destination virtual area). A "pool VOL" may be a VOL that is a component of a pool. A pool VOL may be an RVOL or an EVOL.

[0026] Also, in the following description, VOL may be either a "Logical Unit" (hereinafter referred to as LU) in SCSI or a "Name Space" (hereinafter referred to as NS) in NVMe.

[0027] Also, in the following description, "RAID" is an abbreviation for Redundant Array of Inexpensive Disks. A RAID group is composed of a plurality of drives (typically the same type of drive) and stores data according to the RAID level associated with the RAID group. The RAID group may be referred to as a parity group. The parity group may be, for example, a RAID group that stores parity.

[0028] The network interface device (hereinafter also simply referred to as a network interface) according to an embodiment of the present specification can be implemented in an information processing system including a server system and a storage system. The storage system can include a storage controller and a drive box. The network interface can include, for example, a general-purpose processor, a memory, a network controller, and an interface with a host system.

[0029] A configuration using a general-purpose processor and a memory capable of realizing software-based protocol processing solves the problem that hardware has to be updated every time the support for a new protocol such as NVMe / TCP in addition to iSCSI is increased. For example, in order to be able to quickly respond to changes such as switching from iSCSI to NVMe / TCP, changes due to specification updates of NVMe / TCP, or changes to an even newer protocol, software of the network interface is replaced using a general-purpose processor, a memory, etc. to keep up with the changes. The network interface according to an embodiment of the present specification supports a plurality of network protocol processes simultaneously.

Example

[0030] FIG. 1 is a diagram showing a configuration example of the information processing system according to the first embodiment.

[0031] The information processing system includes one or more server systems 100 and a storage system. The storage system includes one or more storage devices 101 and one or more drive boxes 103.

[0032] The storage device 101 includes one or more storage controllers 102. In FIG. 1, each storage device 101 includes two storage controllers 102. The storage controller 102 is connected to one or more server systems 100 via the front-end network 105.

[0033] The drive box 103 is connected to one or more storage controllers 102 via the back-end network 107. Also, the storage controller 102 is connected to another nearby storage controller 102 via the storage controller inter-network 106, and is connected to another storage controller 102 at a medium or long distance via the external network 108.

[0034] The front-end network 105 is a storage area network that connects between the storage controller 102 and the server system 100. For example, an IP network such as iSCSI or NVMe / TCP (NVMe over TCP) is used.

[0035] The back-end network 107 is a network that connects between the storage controller 102 and the drive box 103. For example, an IP network such as iSCSI or NVMe / TCP is used.

[0036] The storage controller network 106 is a network used for the redundancy of the storage controller 102 and is composed of a broadband interconnect. Using this network, data duplication of write data, sharing of metadata, etc. are performed, and even if one storage controller 102 is blocked due to maintenance, failure, etc., the other storage controller 102 can continue the storage process.

[0037] The external network 108 is a Wide Area Network (WAN) or a Local Area Network (LAN). For example, it is a network where the data link layer is Ethernet (registered trademark), the internet layer is the internet protocol, and the transport layer is TCP or UDP, etc., and communication is performed using the Protocol Data Unit (PDU) of iSCSI or NVMe / TCP. This network can take the form of an internet line or a dedicated line. Depending on the distance, the communication delay increases, and when the network equipment is not composed only of lossless devices, the occurrence rate varies depending on the above line type, but packet loss is assumed to occur.

[0038] The server system 100 is a host machine on which user applications, etc. operate, has one or more processors, and is configured to include one or more storage devices such as a memory and an auxiliary storage device. For example, a database or a web service operates, and the server system 100 writes and reads the data created by them to and from the storage controller 102 via the network interface 104. Also, the server system 100 has a network interface 104 as an interface device for connecting to the storage controller 102. The server system 100 may be composed of a plurality of server groups, each server group having a network interface 104 and connecting to the storage controller 102 and other server systems 100.

[0039] The storage controller 102 has one or more processors and memory. Assume that the storage controller 102 in the storage device 101 has the same configuration.

[0040] The processor has a plurality of cores, and the cores instruct the transfer of data stored in the corresponding drive box 103 according to read commands and write commands from the server system 100.

[0041] The memory is composed of, for example, a semiconductor memory such as SDRAM (Synchronous Dynamic Random Access Memory). The memory may be configured in combination with a volatile memory and a non-volatile memory such as SCM (Storage Class Memory). The memory stores an execution program (such as a storage control program) and a management table referred to by the processor as the main memory of the processor. In addition, the memory is also used as a disk cache (cache memory) of the storage controller 102.

[0042] The storage controller 102 has a network interface 104 as an interface device for connecting to the drive box 103. The network interface 104 communicates information related to storage processing such as data transfer and data copy instructed by the server system 100 with the drive box 103.

[0043] The drive box 103 mounts a plurality of drives such as SSDs (Solid State Drives) and HDDs (Hard Disk Drives). In addition, the drive box 103 includes an internal switch, as well as a processor and memory used for transfer processing, in order to connect the plurality of drives to the storage controller 102. The drive box 103 receives, stores, and holds the data generated in the server system 100 via the storage controller 102.

[0044] To ensure the availability of the data held by the drive box 103, RAID may be configured among the built-in drives, or RAID may be configured among multiple drive boxes 103. Also, the drive box 103 has a network interface 104 as an interface device for connecting to the storage controller 102.

[0045] The network interface 104 is mounted on each of the server system 100, the storage controller 102, and the drive box 103, and serves as an interface for connecting various devices to various networks.

[0046] The network interface 104 may be, for example, a SmartNIC. Various functions of the SmartNIC are implemented using a general-purpose processor mounted on the SmartNIC and a partial hardware offload engine. Also, the SmartNIC may be configured using an FPGA (Field Programmable Gate Array), in which case each function is realized on the FGPA. Further, as another form, it may be configured as dedicated interface hardware with the whole being hardware-implemented. Details of the network interface 104 will be described later.

[0047] Note that the information processing system and the storage system may include those other than those shown here. For example, network devices such as switches and routers may be connected in the middle of each network, or devices for monitoring and maintenance may be connected. Also, a configuration may be adopted in which a storage service on a public cloud is connected via the external network 109.

[0048] The network interface 104 of the server system 100 has initiator software in iSCSI or NVMe / TCP to read and write the data of the drive box 103 via the storage controller 102. In contrast, the network interface 104 of the storage controller has target software.

[0049] The network interface 104 of the storage controller 102 has initiator software in iSCSI or NVMe / TCP to read and write data from the drive box 103. In contrast, the network interface 104 of the drive box 103 has target software.

[0050] Furthermore, the network interface 104 of the storage controller 102 has initiator software in iSCSI or NVMe / TCP to read and write data from other storage devices 101. In contrast, the network interface 104 of other storage devices 101 has target software.

[0051] Note that a part of the network interface 104 may be a NIC that does not have the function of a SmartNIC. For example, the network interface 104 installed in the server system 100 is a general NIC that operates as an initiator, and the network interface 104 installed in the storage controller 102 is a SmartNIC that operates as a target.

[0052] FIG. 2 is a diagram showing a configuration example of the network interface (SmartNIC) 104 of the first embodiment.

[0053] The network interface 104 is connected via the host bus 201 to the internal configuration of the device on which the network interface 104 is mounted, for example, the storage controller 102, the server system 100, or the drive box 103. The network interface 104 is connected to other devices via the network path 200, for example, via an IP network.

[0054] Network interface 104 includes a data processing unit 202 and a memory 210. The data processing unit 202 includes a host interface 207 and a network controller 203. Further, the data processing unit 202 includes a processor 204, a memory controller 209, a DMA (Direct Memory Access) controller 206, and a cache 208. Also, the network interface 104 has one or more ports.

[0055] The host bus 201 is a bus that connects the network interface 104 to a storage controller 102, a server system 100, a drive box 103, etc. The host bus 201 is a broadband and high-speed interconnect, and for example, it is configured to be connected to a processor of the mounted device etc. via PCIe.

[0056] The host interface 207 is an interface for connecting the network interface 104 and a host system (hereinafter simply referred to as a host) via the host bus 201. For example, when the host bus 201 is PCIe, it can include, for example, a PCIe PHY.

[0057] The DMA controller 206 exchanges data between the memory 210 of the network interface 104 and the memory of the host.

[0058] The network path 200 is, for example, a path of an IP network and takes the network form of a WAN, a LAN, or a SAN (Storage Area Network). The network interface 104 communicates via one network path 200 or, considering redundancy, two or more network paths 200.

[0059] The network controller 203 is an interface for connecting the network interface 104 to the network path 200. The network controller 203 executes, for example, physical layer processing such as PHY, and stateless processing of the data link layer, internet layer, and transport layer. The network controller 203 performs, for example, checksum and frame processing.

[0060] The network controller 203 is compatible with, for example, Ethernet, IP, TCP, UDP, etc. Furthermore, it may include offload engines such as IPsec (Internet Security Protocol), TLS (Transport Layer Security), DIF (Data Integrity Field), etc. Also, it has a configuration compatible with connection to optical cables, copper cables, etc.

[0061] The processor 204 is, for example, a general-purpose processor and executes, for example, an operating system. The processor 204 further executes other software to perform processing such as protocol processing, command processing, and management of the network interface 104. The processor 204 can have any configuration, for example, includes one or more CPUs or MPUs (Micro Processing Unit), and can include a plurality of cores 205. Note that the processor 204 is hardware physically different from the processor of the storage controller 102, and each executes its own processing. The processor 204 executes network protocol processing in communication with a host (for example, the server system 100) connected via a port. The processor of the storage controller 102 executes storage processing.

[0062] The memory 210 is composed of, for example, a semiconductor memory such as an SDRAM, and may be configured in combination with a non-volatile memory such as an SCM. The memory 210 stores, as the main memory of the processor 204, an execution program (instruction codes for protocol processing and command processing) and a management table referred to by the processor. Further, the memory 210 is also used as a buffer for commands and data transmitted and received with the network. Furthermore, it takes a queuing interface with the network controller 203 and the host interface 207, and stores queue descriptors, indexes, and the like.

[0063] The memory controller 209 is an interface for controlling the reading and writing of data to and from the memory 210. The memory controller 209 may be built into, for example, the processor 204, the data processing unit 202, or the network interface 104.

[0064] The cache 208 temporarily stores data between the memory 210 and the processor 204. The processor 204 can access the cache 208 faster than the memory 210. The data read from the memory 210 is stored in the cache 208. The processor 204 accesses the cache 208 to read data (including commands). The cache 208 can have a hierarchical structure. From the hierarchy closer to the processor, it is called L1 cache, L2 cache, etc. The cache 208 may be within the processor 204. When having a hierarchical structure, only some hierarchies may be within the processor 204 and the remaining hierarchies may be outside the processor 204. The processor 204 and the DMA controller 206 ensure coherence (consistency) with respect to the cache 208.

[0065] Note that the information processing system and the storage system may include those other than those shown here. For example, modules and interfaces for monitoring and maintenance, and a non-volatile storage device storing an operating system and software programs operating on the network interface 104 may be added.

[0066] FIG. 3 is a diagram showing the program configuration of the network interface 104 of the first embodiment.

[0067] In order to realize a software-based changeable processing function, the network interface 104 operates the operating system 300 using the processor 204, the cache 208, and the memory 210, and operates software programs for various processes thereon.

[0068] The software programs 300 to 310 are expanded in the memory 210 and executed by the processor 204. Some instructions are stored in the cache 208. Note that for DIF (Data Integrity Field), CRC (Cyclic Redundancy Check), encryption, compression, hash, parity processing, etc., they may be implemented in dedicated hardware and controlled by software to improve efficiency.

[0069] The operating system 300 is a basic software program serving as a basis for operating the network interface 104 and manages the entire network interface 104. The operating system 300 provides a common usage environment for each software operating on the processor 204 of the network interface 104. The operating system 300 may be an embedded operating system or a general-purpose operating system that operates on a server, such as Linux (registered trademark), etc.

[0070] The network controller driver 301 is driver software for controlling the network controller 203. The network controller driver 301 outputs a parameter set for packet processing to be offloaded during packet generation or reception to the network controller 203. Further, the network controller driver 301 outputs to the network controller 203 the packets generated by the protocol processing 303 for transmission. Also, the network controller driver 301 outputs the packets received from the network controller 203 to the protocol processing 303.

[0071] The host interface driver 302 is driver software for controlling the host interface 207. Communication between the network interface 104 and the host is executed via the host interface driver 302.

[0072] The protocol processing 303, in cooperation with the command processing 305, the DMA control 307, and the network controller driver 301, generates transmission packets and performs the transmission processing thereof. Also, the protocol processing 303 processes the received packets, extracts information and data such as control information, iSCSI PDUs, and NVMe / TCP PDUs, and outputs the extracted information to the command processing 305.

[0073] The protocol processing 303 performs IP header processing in the Internet protocol layer, TCP header processing and UDP header processing in the transport layer, and iSCSI processing and NVMe / TCP processing. For example, the protocol processing 303 executes programs such as socket programs, iSCSI initiators or targets, and NVMe / TCP initiators or targets.

[0074] The host queue control 304 is software for controlling a queue interface for the communication between the host and commands. The host queue control 304 manages, within the network interface 104, a queue of commands to be sent to the host and a queue of commands received from the host. The host queue control 304 stores commands to or from the host in the queue. Further, the host queue control 304 controls the Head and Tail when the queue has, for example, a ring structure. The host queue control 304 controls the Head and Tail of the queue held by the host.

[0075] The command processing 305 receives a command for controlling the network interface 104 from the host and controls the network interface 104. The command processing 305 receives a processing request of a network protocol from the host, activates the protocol processing 303, and responds to the host with the processing result of the protocol processing 303. Further, the command processing 305 activates the DMA control 307 for data transfer with the memory secured at the host and performs response processing. Furthermore, the command processing 305 performs initial setting, setting change, software exchange of the network interface 104, and notification to the host at the time of failure.

[0076] The buffer control 306 is software for controlling a buffer for temporarily holding data in the memory 210. The buffer stores data received from the network and transferred to the host or data received from the host and transmitted to the network. The buffer control 306 secures, uses, and manages a group of buffers of different sizes. The buffer control 306 controls the buffer so as to improve the cache hit rate.

[0077] The DMA control 307 performs an interface process with the DMA hardware, for example, to control data transfer between the memory secured at the host side and the memory 210 on the network interface 104.

[0078] The core allocation control 308 controls the allocation of the core 205 to the port. The details of the core 205 allocation control will be described later.

[0079] The maintenance / fault processing 309 supports the update of the software of the network interface 104, the detection of hardware failures, and the notification to the host, etc.

[0080] The initialization processing 310 initializes the hardware that constitutes the network interface 104 and performs various software initialization processes.

[0081] Figure 4 is a diagram showing an example of the relationship of the programs of the storage interface of the first embodiment.

[0082] In the network interface 104, the operating system 300 operates, and each software program operates on the operating system 300. Further, by the initialization processing 310 and the maintenance / fault processing 309, the network interface 104 executes maintenance such as initial settings and software updates, and fault processing. The fault processing includes, for example, fault detection and collection of dump trace information such as statistical information and error information.

[0083] The core allocation control 308 controls the allocation of the core 205 that executes network protocol processing in communication with the host connected via the port.

[0084] The network controller driver 301 controls the network controller 203, stores the transmission packet in the packet buffer of the network controller 203, and acquires the received packet from the packet buffer. Further, settings are made to offload frame processing of the data link layer, the Internet protocol layer, and the transport layer, and stateless processing, for example, checksum calculation.

[0085] In response to the operation of the network controller driver 301, the protocol processing 303 performs network protocol processing such as IP, TCP, UDP, iSCSI PDU processing, NVMe / TCP PDU processing, etc. The protocol processing 303 processes the received packets received from the network controller driver 301, and extracts control information, information such as iSCSI PDU and NVMe / TCP PDU, and data. The received packets may not contain data. The protocol processing 303 outputs the extracted information to the command processing 305. The data can be transferred to the host via the buffer controlled by the buffer control 306 or without passing through the buffer.

[0086] The protocol processing 303 includes the information obtained from the command processing 305 and the data transmitted from the host in the transmission packets to the network. The data can be transferred to the network via the buffer controlled by the buffer control 306 or without passing through the buffer. The transmission packets may not contain data.

[0087] The command processing 305 executes command processing in cooperation with the host queue control 304, the buffer control 306, and the DMA control 307. The command processing 305 controls the host interface 207 via the host interface driver 302. The command processing 305 exchanges transmission commands or reception commands for network communication between the host system, for example, information for generating iSCSI or NVMe / TCP PDUs, analyzed information, or the PDUs themselves. The commands are exchanged with the host system using the queue interface. The host queue control 304 controls this queue interface.

[0088] Based on the result of the protocol processing 303, the command processing 305 generates a descriptor that can be processed by the host, and stores it in the queue via the host queue control 304. Also, the host system retrieves the descriptor generated by the host system from the queue, and configures and uses each function within the network interface 104, such as the protocol processing 303. Note that the descriptors in the queue can be exchanged between the host system and the network interface 104 using the DMA controller 206 by the DMA control 307.

[0089] FIG. 5A, FIG. 5B, and FIG. 5C are diagrams showing an example of the allocation mode of the core 205 of the network interface 104 in the first embodiment.

[0090] In the allocation mode of the core 205 shown in FIG. 5A, the core 205 is allocated to each port at a predetermined ratio. In FIG. 5A, the core 205 is evenly allocated to two ports. The core 205 allocated to the port executes network protocol processing for the data transmitted and received via the port. In the following description, the allocation mode of the core 205 shown in FIG. 5A is described as the first allocation mode.

[0091] In the allocation mode of the core 205 shown in FIG. 5B, all the cores 205 are allocated to one port. In the following description, the allocation mode of the core 205 shown in FIG. 5B is described as the second allocation mode.

[0092] In the allocation mode of the core 205 shown in FIG. 5C, one core 205 is exclusively allocated to one port, and the other cores 205 are allocated in a time-sharing manner. In FIG. 5C, the hatched core 205 indicates the core exclusively allocated to the port. Note that two or more cores 205 can be exclusively allocated. In the following description, the core 205 exclusively allocated is described as the exclusive core 205, and the core 205 allocated in a time-sharing manner is described as the shared core 205. Also, the allocation mode of the core 205 shown in FIG. 5C is described as the third allocation mode.

[0093] The occupancy core 205 executes network protocol processing for commands and data transmitted and received via the assigned ports and management processing for the assigned ports. The occupancy core 205 does not execute network protocol processing and management processing for other ports. Management processing related to a port includes, for example, initialization, abort, reset, etc. The shared core 205 executes network protocol processing for data transmitted and received via the ports.

[0094] In the third allocation mode, since the occupancy core 205 executes management processing, it is possible to reduce the influence between ports associated with the execution of the management processing. Also, since the core 205 can be allocated to one port as much as possible, the processing performance can be ensured.

[0095] In the first and second allocation modes, information (allocation information) associating the port ID with the core 205 is generated. In the third allocation mode, information (allocation information) associating the port ID with the occupancy core 205 is generated.

[0096] Note that the allocation mode of the core 205 can be selected by the user. Also, as another allocation mode, an allocation mode in which one occupancy core 205 is allocated for each port and cores 205 other than the occupancy core 205 are allocated to each port at a predetermined ratio can be considered.

[0097] FIG. 6 is a diagram showing the correspondence between the I / O queue 600 between the core 205 and the server system 100 in the network interface 104 of Example 1 and the queue 601 between the core 205 and the host. In FIG. 6, the correspondence between the I / O queue 600 between the core 205 and the server system in the third allocation mode and the queue 601 between the core 205 and the host is shown. The hatched core 205 represents the occupied core 205, and the white core 205 represents the shared core 205. In FIG. 6, the queue 601 is shown to be divided for each of Port0 and Port1. For the occupied core 205, only one port's worth of queue is set, and for the shared core 205, two queues 601 are allocated, one for Port0 and one for Port1, respectively. However, a configuration where each core has one queue 601 without having as many queues as the number of ports is also possible, or the host may have only one queue per port and distribute the queues to each core within the network interface.

[0098] For one port, as many I / O queues 600 as the maximum allocation number of the core 205 are set. In the example shown in FIG. 6, since a maximum of seven cores 205 are allocated to one port, seven I / O queues 600 are set. The occupied core 205 executes the processing of one I / O queue 600 of the allocated port, and the shared core 205 is controlled to execute the processing of the I / O queue 600 of each port.

[0099] The queue 601 responsible for the occupied core 205 is controlled such that commands and data for network protocol processing or data and commands for management processing related to the port are transferred. The queue 601 responsible for the shared core 205 is controlled such that data and commands for network protocol processing are transferred. The transfer control of commands and data to the queue 601 is performed by the host queue control 304 or the DMA control 307.

[0100] By adopting the third allocation mode, it is possible to minimize the impact on the processing of other ports while executing the management processing of one port.

[0101] The network interface 104 of this embodiment performs allocation control of the core 205 according to any one of three allocation methods. FIGS. 7A and 7B are diagrams showing an example of state transition in the allocation control of the core 205 of the network interface 104 of Embodiment 1. FIG. 8 is a diagram showing an example of allocation mode management information held by the network interface 104 of Embodiment 1. FIG. 9 is a sequence diagram showing an example of a procedure for changing the allocation mode of the network interface 104 of Embodiment 1.

[0102] FIG. 7A is a state transition diagram when the network interface 104 manages the allocation mode. In this case, the network interface 104 manages the allocation mode using the allocation mode management information 800 as shown in FIG. 8. The allocation mode management information 800 includes a current allocation mode 801 that stores information on the current allocation mode and a next allocation mode 802 that stores information on the allocation mode at the next startup.

[0103] When the power of the network interface 104 is turned on, it transitions from state 701 to state 702. In state 702, the network interface 104 starts allocation control of the core 205 in the allocation mode set in the next allocation mode 802. After starting the allocation control of the core 205, the network interface 104 transitions to state 703 and sets the allocation mode of the next allocation mode 802 in the current allocation mode 801.

[0104] When the network interface 104 receives an instruction to change the allocation mode, it transitions to state 704, updates the next allocation mode 802, and waits for a restart. In state 704, when an instruction to change the allocation mode is received, the network interface 104 transitions to state 704 again. When restarted in state 704, allocation control of the core 205 is performed in the changed allocation mode.

[0105] Here, the procedure for changing the allocation mode will be described with reference to FIG. 9.

[0106] The user instructs, via a user interface (not shown), to change the allocation mode to the storage controller 102 which is a network interface host (step S901).

[0107] The storage controller 102 generates a host command for changing the allocation mode (step S902) and issues it to the network interface 104 (step S903).

[0108] The network interface 104 updates the next allocation mode 802 of the allocation mode management information 800 (step S904). Thereafter, the network interface 104 responds with the completion of the host command (step S905).

[0109] The storage controller 102 instructs the network interface 104 to reset (power off) (step S906). Also, the storage controller 102 accesses the network interface 104 to confirm the completion of the reset (step S907).

[0110] The user instructs, via the user interface, the storage controller 102 to acquire the state of the network interface 104 (step S908). When receiving the instruction, the storage controller 102 responds with the state of the network interface 104 (step S909).

[0111] Figure 7B is a state transition diagram when the network interface 104 does not manage the allocation mode. In this case, the default allocation mode is set for the network interface 104.

[0112] When the power is turned on, the network interface 104 transitions from state 711 to state 712 and executes an initialization process. After the initialization process is completed, the network interface 104 starts core allocation control in the default allocation mode and transitions to state 713. When a process reboot occurs, the network interface 104 transitions to state 712.

[0113] When the network interface 104 receives an instruction to change the allocation mode, it transitions to state 714 and waits for a process reboot to change to the instructed allocation mode. When a process reboot occurs, the network interface 104 transitions from state 711 to state 712, starts the initialization process, starts core allocation control in the changed allocation mode, and then transitions to state 716. When a process reboot occurs, the network interface 104 transitions to state 715. When the network interface 104 receives an instruction to change the allocation mode, it transitions to state 714.

[0114] The procedure for changing the allocation mode is the same as that in FIG. 9. However, in step S904, the network interface 104 waits for a process reboot in the instructed allocation mode. Also, in step S905, the storage controller 102 instructs the network interface 104 to perform a process reboot.

[0115] FIG. 10 is a flowchart for explaining an example of the allocation mode setting process at startup executed by the network interface 104 of the first embodiment. In FIG. 10, the process executed by the network interface 104 adopting the method of FIG. 7A is explained.

[0116] When the power of the network interface 104 is turned on (step S1001), it executes an initialization process (step S1002).

[0117] After the initialization process of the network interface 104 is completed, it starts the process in the next allocation mode (step S1003).

[0118] The network interface 104 determines whether the current allocation mode 801 is different from the next allocation mode 802 by referring to the allocation mode management information 800 (step S1004).

[0119] If the current allocation mode 801 matches the next allocation mode 802, the network interface 104 proceeds to step S1006.

[0120] If the current allocation mode 801 is different from the next allocation mode 802, the network interface 104 updates the current allocation mode 801 (step S1005), and then proceeds to step S1006. Specifically, the value of the next allocation mode 802 is overwritten in the current allocation mode 801.

[0121] In step S1006, the network interface 104 determines whether it has received a reset instruction based on the result of the hardware check by the storage controller 102, which is the network interface host (step S1006).

[0122] If it has not received a reset instruction, the network interface 104 ends the allocation mode setting process and starts various processes.

[0123] If it has received a reset instruction, the network interface 104 shifts to power-off (step S1007), and then shifts to power-on (step S1001).

[0124] FIG. 11 is a flowchart for explaining an example of the allocation mode setting process at startup executed by the network interface 104 of the first embodiment. In FIG. 11, the process executed by the network interface 104 adopting the method of FIG. 7B is explained.

[0125] When the network interface 104 is powered on (step S1101), it executes an initialization process (step S1102).

[0126] After the initialization process of the network interface 104 is completed, it starts a process in the default allocation mode (step S1103).

[0127] The network interface 104 determines whether it has received a reset instruction based on the result of the hardware check by the storage controller 102 which is the network interface host (step S1104).

[0128] If it has received a reset instruction, the network interface 104 shifts to power-off (step S1105), and then shifts to power-on (step S1101).

[0129] If it has not received a reset instruction, the network interface 104 starts accepting change instructions for the allocation mode for a certain period. The network interface 104 monitors the change instructions for the allocation mode for a certain period and determines whether it has received the change instruction (step S1106).

[0130] If it has not received a change instruction for the allocation mode, the network interface 104 ends the allocation mode setting process and starts various processes.

[0131] If it has received a change instruction for the allocation mode, the network interface 104 changes the allocation mode (step S1107) and reboots the process (step S1108).

[0132] After the reboot of the process, the network interface 104 executes an initialization process (step S1109).

[0133] After the initialization process is completed, the network interface 104 starts the process in the changed allocation mode (step S1110). Thereafter, the network interface 104 ends the allocation mode setting process and starts various processes.

[0134] FIG. 12 is a flowchart for explaining an example of the allocation mode setting process during the execution of the network protocol process executed by the network interface 104 of the first embodiment. In FIG. 12, the process executed by the network interface 104 adopting the method of FIG. 7A is explained.

[0135] When the network interface 104 receives an instruction to change the allocation mode during the execution of the network protocol process, it starts the process described below.

[0136] The network interface 104 updates the next allocation mode 802 of the allocation mode management information 800 (step S1201), and responds to the storage controller 102 that the change is completed (step S1202).

[0137] When the network interface 104 receives a reset instruction from the storage controller 102 (step S1203), after shifting to the power-off state (step S1204), it shifts to the power-on state (step S1205).

[0138] The processes from step S1205 to step S1211 are the same as the processes from step S1001 to step S1007.

[0139] FIG. 13 is a flowchart for explaining an example of the allocation mode setting process during the execution of the network protocol process executed by the network interface 104 of the first embodiment. In FIG. 13, the process executed by the network interface 104 adopting the method of FIG. 7B is explained.

[0140] When the network interface 104 receives an instruction to change the allocation mode during the execution of network protocol processing, it starts the processing described below.

[0141] The network interface 104 changes the allocation mode (step S1301) and responds to the storage controller 102 that the change is completed (step S1302).

[0142] When the network interface 104 receives a reboot instruction from the storage controller 102 (step S1303), it reboots the process (step S1304).

[0143] The processing from step S1305 to step S1306 is the same as the processing from step S1109 to step S1110.

[0144] FIG. 14A and FIG. 14B are flowcharts for explaining an example of the core allocation process executed by the network interface 104 of the first embodiment.

[0145] After the network interface 104 establishes a connection with the device connected via the network interface 104 (step S1401), it determines the allocation mode of the core 205 (step S1402). When adopting the method of FIG. 7A, the allocation mode of the core 205 can be determined based on the allocation mode management information 800. When adopting the method of FIG. 7B, the allocation mode of the core 205 can be determined based on the allocation information.

[0146] When the allocation mode is the second allocation mode, the network interface 104 acquires the allocated core number (step S1403). The allocated core number is the identification number of the core 205.

[0147] The network interface 104 allocates a connection to the core 205 corresponding to the allocated core number (step S1404).

[0148] The network interface 104 updates the assigned core number (step S1405) and ends the core allocation process. For example, the assigned core number is updated so that core 205 is assigned in a round-robin manner.

[0149] When the allocation mode is the first allocation mode, the network interface 104 acquires the port number of the port for which the connection has been established (step S1406).

[0150] The network interface 104 acquires the assigned core number corresponding to the port number (step S1407). In the first allocation mode, a predetermined number of cores 205 are assigned to each port, so the assigned core number is managed for each port.

[0151] The network interface 104 assigns the connection to core 205 corresponding to the assigned core number (step S1408).

[0152] The network interface 104 updates the assigned core number (step S1409) and ends the core allocation process. For example, the assigned core number is updated so that core 205 is assigned in a round-robin manner.

[0153] When the allocation mode is the third allocation mode, the network interface 104 acquires the port number of the port for which the connection has been established (step S1410).

[0154] The network interface 104 acquires the assigned core number (step S1411).

[0155] The network interface 104 determines whether it is the occupied core of a port other than the port number acquired by core 205 corresponding to the assigned core number (step S1412).

[0156] When the core that occupies a port other than the port number acquired by core 205 corresponding to the assigned core number, network interface 104 updates the assigned core number (step S1413), and then returns to step S1411. For example, the assigned core number is updated so that core 205 is assigned in a round-robin manner.

[0157] When the core that occupies a port other than the port number acquired by core 205 corresponding to the assigned core number is not the core, network interface 104 assigns a connection to core 205 corresponding to the assigned core number (step S1414).

[0158] Network interface 104 updates the assigned core number (step S1415) and ends the core allocation process. For example, the assigned core number is updated so that core 205 is assigned in a round-robin manner.

[0159] FIG. 15 is a sequence diagram showing the flow of setting the queue of network interface 104 in the first embodiment.

[0160] Server system 100, which is the initiator, determines the number of queues to request from the target (step S1501) when establishing, for example, an NVMe / TCP connection, and transmits a queue setting request to storage device 101, which is the target (step S1502).

[0161] Network interface 104 receives the queue setting request and checks the number of requested queues (step S1503).

[0162] Network interface 104 checks the number of cores 205 that can be assigned to the port that has received the queue setting request (step S1504), and determines the number of queues to be set based on that number (step S1505). For example, an integer multiple of the number of cores 205 is determined as the number of queues. At this time, the number of requested queues may be considered.

[0163] After setting a determined number of queues, the network interface 104 transmits a response including the determined number of queues to the server system 100, which is the initiator (step S1506).

[0164] The server system 100 determines the number of queues to be used based on the response (step S1507).

[0165] When the number of queues handled by the core 205 is large, an overhead associated with queue switching occurs. The network interface 104 of this embodiment determines the number (maximum number) of queues based on the number of cores 205 that can be assigned to the port and notifies the server system 100, which is the initiator. Thereby, the occurrence of overhead can be reduced.

[0166] FIGS. 16 and 17 are diagrams showing an example of the distribution and operation state of network protocol processing in the network interface 104 of the first embodiment. Here, it is assumed that the network interface 104 has ports 0 and 1.

[0167] FIG. 16 shows the distribution and operation state of the processing in the network interface 104 operating in the first allocation mode. The dotted rectangular wave represents the network protocol processing of port 0, the hatched rectangle represents the network protocol processing of port 1, and the white rectangle represents the management processing of port 1.

[0168] In the first allocation mode, the cores 205 are allocated to the two ports at a predetermined ratio. Each core 205 executes the network protocol processing of the allocated port. When the core (4) receives the management processing of port 1, it executes the management processing. Until the management processing is completed, the network protocol processing of port 1 is not executed by the other cores (5), (6), (7).

[0169] FIG. 17 shows the distribution of processing and the operating states in the network interface 104 operating in the third allocation mode. The dotted rectangular wave represents the network protocol processing of port 0, the hatched rectangle represents the network protocol processing of port 1, and the white rectangle represents the management processing of port 1.

[0170] The dedicated core 205 executes the network protocol processing and the management processing of the allocated port. The shared core 205 executes the network protocol processing of the ports allocated on a time-division basis. When the dedicated core (4) receives a request to execute management processing, it executes the management processing. During the execution of the management processing, the network protocol processing of port 1 is not executed, but the network protocol processing of port 0 is executed using the shared core 205.

[0171] In addition, when the magnitude of the load of the network protocol processing is known in advance, the connections may be allocated in consideration of the core performance, the core utilization rate, and the like.

[0172] Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments are those in which the configuration has been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, for a part of the configuration of each embodiment, addition, deletion, or replacement with other configurations is possible.

[0173] In addition, each of the above-described configurations, functions, processing units, processing means, etc. may be implemented in hardware by designing part or all of them, for example, by means of an integrated circuit. Further, the present invention can also be realized by a program code of software that realizes the functions of the embodiments. In this case, a storage medium storing the program code is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the same constitute the present invention. As a storage medium for supplying such a program code, for example, a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, an SSD (Solid State Drive), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, etc. are used.

[0174] In addition, the program code for realizing the functions described in this embodiment can be implemented in a wide range of programs or script languages such as assembler, C / C++, perl, Shell, PHP, Python, Java (registered trademark), etc.

[0175] Furthermore, by distributing the program code of the software that realizes the functions of the embodiments via a network, it can be stored in a storage means such as a hard disk or memory of a computer or a storage medium such as a CD-RW or CD-R, and a processor included in the computer reads and executes the program code stored in the storage means or the storage medium.

[0176] In the above-described embodiments, the control lines and information lines show those considered necessary for explanation, and not necessarily all the control lines and information lines are shown on the product. All the components may be interconnected.

Explanation of Reference Numerals

[0177] 100 Server system 101 Storage device 102 Storage Controller 103 Drive Box 104 Network Interface 105 Front - End Network 106 Network between Storage Controllers 107 Back - End Network 108 External Network 109 External Network 200 Network Path 201 Host Bus 202 Data Processing Unit 203 Network Controller 204 Processor 205 Core 206 DMA Controller 207 Host Interface 208 Cache 209 Memory Controller 210 Memory 300 Operating System 301 Network Controller Driver 302 Host Interface Driver 303 Protocol Processing 304 Host Queue Control 305 Command Processing 306 Buffer Control 307 DMA Control 308 Core Allocation Control 309 Maintenance / Fault Handling 310 Initialization Processing 600 Queue 601 Queue 800 Allocation Mode Management Information

Claims

1. A storage system comprising at least one storage controller, wherein the storage controller is equipped with a processor having a plurality of cores, a memory connected to the processor, and a network interface having a plurality of ports connected to the processor, the processor allocates at least one occupied core to each of the plurality of ports, the occupied core is controlled to execute management processing of the port to which the occupied core is allocated and network protocol processing of communication performed via the port to which the occupied core is allocated, cores other than the occupied core are controlled to execute network protocol processing of communication performed via the port, a storage system characterized by this.

2. The storage system according to claim 1, wherein when the processor detects an opportunity to execute network protocol processing of one of the ports, the processor causes a core other than the occupied core or the occupied core of the port to execute the network protocol processing, a storage system characterized by this.

3. The storage system according to claim 1, wherein the processor allocates cores other than the occupied core to each of the plurality of ports at a predetermined ratio, when the processor detects an opportunity to execute network protocol processing of one of the ports, the processor causes a core allocated to the port other than the occupied core or the occupied core of the port to execute the network protocol processing, a storage system characterized by this.

4. The storage system according to claim 1, wherein the processor determines the number of queues for each of the plurality of ports based on the maximum number of core allocations including the occupied core of the port, a storage system characterized by this.

5. The storage system according to claim 1, wherein the processor provides an interface for selecting the core allocation mode according to claim 1, an allocation mode in which all the cores are allocated to one port, and an allocation mode in which the cores are allocated to each of the plurality of ports at a predetermined ratio, a storage system characterized by this.

6. A method for controlling the allocation of cores of a network interface having a processor with a plurality of cores, a memory connected to the processor, and a plurality of ports connected to the processor, which is mounted on a storage controller of a storage system, comprising: a first step in which the processor allocates at least one occupied core to each of the plurality of ports; a second step in which the processor controls the occupied core to execute management processing of the port to which the occupied core is allocated and network protocol processing of communication performed via the port to which the occupied core is allocated; a third step in which the processor controls the cores other than the occupied cores to execute network protocol processing of communication performed via the ports, the method for controlling the allocation of cores of a network interface being characterized by including the steps.

7. A method for controlling the allocation of cores of a network interface according to claim 6, the method for controlling the allocation of cores of a network interface being characterized by including a step of causing the cores other than the occupied cores or the occupied core of the port to execute the network protocol processing when the processor detects an opportunity to execute the network protocol processing of one of the ports.

8. A method for controlling the allocation of cores of a network interface according to claim 6, the first step including a step in which the processor allocates the cores other than the occupied cores to each of the plurality of ports at a predetermined ratio, the method for controlling the allocation of cores of a network interface being characterized by including a step of causing the cores allocated to the port other than the occupied cores or the occupied core of the port to execute the network protocol processing when the processor detects an opportunity to execute the network protocol processing of one of the ports.

9. A method for controlling the allocation of cores of a network interface according to claim 6, the method for controlling the allocation of cores of a network interface being characterized by including a step in which the processor determines the number of queues for each of the plurality of ports based on the maximum number of cores including the occupied core of the port.

10. A method for controlling the allocation of cores of a network interface according to claim 6, comprising: The step of presenting, by the processor, an interface for selecting the core allocation mode according to claim 6, an allocation mode of allocating all the cores to one of the ports, and an allocation mode of allocating the cores to each of the plurality of ports at a predetermined ratio. A method for controlling the allocation of cores of a network interface, characterized by including this step.

Images