Method for controlling the allocation of cores in a storage system and network interface.

Abstract

To realize core allocation control capable of suppressing influence of a process between ports and sufficiently exhibiting performance of the process of the port in a network interface mounted on a storage system.SOLUTION: A storage system includes a storage controller on which a network interface including a processor equipped with a plurality of cores, a memory, and a plurality of ports is mounted. The processor allocates at least one occupied core to each of the plurality of ports, controls the occupied core to execute a management process of the port to which the occupied core is allocated and a network protocol process of communication performed via the port to which the occupied core is allocated, and controls the cores other than the occupied core to execute the network protocol process of the communication performed via the port.SELECTED DRAWING: Figure 5C

Description

Technical Field

[0001] The present invention relates to a network interface mounted on a storage system.

Background Art

[0002] With the acceleration of the network and the emergence of new protocols, there is a demand for the acceleration of network interfaces and multi-protocol support in enterprise storage systems as well. Conventionally, a CPU on a storage controller has been used to support multiple protocols, or offloaded to protocol-specific hardware to support multiple protocols. For example, U.S. Patent No. 9,100,349 discloses a technique for supporting multiple protocols using a NIC (Network Interface Card) equipped with Ethernet and FC protocol chips.

[0003] However, when using the CPU of a storage controller for communication protocol processing, there is a problem that the CPU resources are consumed by protocol processing. When using a Channel Board (CHB) equipped with a dedicated ASIC for protocol processing, it is necessary to develop a dedicated ASIC for each protocol, and it is difficult to quickly support protocols.

[0004] On the other hand, SmartNICs having a general-purpose processor and memory and performing network protocol processing have emerged. In SmartNICs, for example, the same operating system as that operating on a server system can be operated, and the software protocol stack and applications used there can be operated. Since network protocol processing can be implemented in software, it is possible to quickly support multiple protocols and new protocols, and it is also possible to flexibly respond to updates of network protocol processing.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] U.S. Patent No. 9100349 [Overview of the project] [Problems that the invention aims to solve]

[0006] SmartNIC incorporates a general-purpose processor with multiple cores, memory, and ports. Conventional core allocation control methods include a method that allocates cores to each port at a predetermined ratio, and a method that allocates cores to each port using time-sharing.

[0007] In the former allocation method, the processing performance of a single port depends on the number of cores allocated, which means that the SmartNIC cannot achieve its maximum performance. On the other hand, if all cores are allocated to a single port to achieve maximum performance, network protocol processing for other ports becomes impossible.

[0008] In the latter allocation method, if management processes such as resetting occur on other ports while network protocol processing is running on one port, the cores assigned to one port may be reassigned to other ports, potentially affecting the network protocol processing on that port.

[0009] The present invention aims to realize core allocation control in a network interface installed in a storage system that minimizes the impact of processing between ports and allows the network protocol processing performance of the ports to be fully utilized. [Means for solving the problem]

[0010] A representative example of the invention disclosed in this application is as follows: a storage system comprising at least one storage controller, wherein the storage controller includes a processor having a plurality of cores, memory connected to the processor, and a network interface having a plurality of ports connected to the processor, the processor assigning at least one dedicated core to each of the plurality of ports, the dedicated core being controlled to perform management processing for the port to which it is assigned and network protocol processing for communications performed through the port to which it is assigned, and the cores other than the dedicated core being controlled to perform network protocol processing for communications performed through the ports. [Effects of the Invention]

[0011] According to the present invention, in a network interface installed in a storage system, it is possible to achieve core allocation control that suppresses the impact of processing between ports and allows the network protocol processing performance of the ports to be fully utilized. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example configuration of the information processing system in Example 1. [Figure 2] This figure shows an example of the network interface configuration in Example 1. [Figure 3] This figure shows the program configuration of the network interface in Example 1. [Figure 4] This figure shows an example of the program relationship for the network interface in Example 1. [Figure 5A] This figure shows an example of the core allocation mode for the network interface in Example 1. [Figure 5B] This figure shows an example of the core allocation mode for the network interface in Example 1. [Figure 5C] It is a diagram showing an example of the core allocation mode of the network interface in Example 1. [Figure 6] It is a diagram showing the correspondence between cores and queues in the network interface of Example 1. [Figure 7A] It is a diagram showing an example of state transition in the core allocation control of the network interface in Example 1. [Figure 7B] It is a diagram showing an example of state transition in the core allocation control of the network interface in Example 1. [Figure 8] It is a diagram showing an example of the allocation mode management information held by the network interface in Example 1. [Figure 9] It is a sequence diagram showing an example of the procedure for changing the allocation mode of the network interface in Example 1. [Figure 10] It is a flowchart explaining an example of the allocation mode setting process at startup executed by the network interface in Example 1. [Figure 11] It is a flowchart explaining an example of the allocation mode setting process at startup executed by the network interface in Example 1. [Figure 12] It is a flowchart explaining an example of the allocation mode setting process during the execution of network protocol processing executed by the network interface in Example 1. [Figure 13] It is a flowchart explaining an example of the allocation mode setting process during the execution of network protocol processing executed by the network interface in Example 1. [Figure 14A] It is a flowchart explaining an example of the core distribution process executed by the network interface in Example 1. [Figure 14B] It is a flowchart explaining an example of the core distribution process executed by the network interface in Example 1. [Figure 15] It is a sequence diagram showing the flow of queue setting of the network interface in Example 1. [Figure 16]This is a diagram showing an example of the distribution of processing and the operating state in the network interface of Example 1. [Figure 17] This is a diagram showing an example of the distribution of processing and the operating state in the network interface of Example 1.

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments will be described based on the drawings. Note that the following description and drawings are examples for explaining the present invention, and for the sake of clarity of explanation, appropriate omissions and simplifications have been made. The present invention can also be implemented in various other forms, and unless otherwise particularly limited, each component may be in a single or plural number.

[0014] Also, the embodiments described below do not limit the invention according to the claims, and not all combinations of elements described in the embodiments are essential for the solution means of the invention.

[0015] In the following description, various types of information may be described using expressions such as "table", "list", "queue", etc. However, the various types of information may be represented by other data structures, and in order to indicate that they do not depend on the data structure, "table of xxx", "list of xxx", "queue of xxx", etc. may be referred to as "xxx information", etc. In the following description, when explaining identification information, expressions such as "identification information", "identifier", "name", "ID", "number", etc. are used, but these are mutually replaceable.

[0016] In the following description, when there are a plurality of components having the same or similar functions, they are basically described with the same reference numerals. However, even if the functions are the same, the means for realizing the functions may be different. Furthermore, the embodiments described later may be implemented by software operating on a general-purpose computer, or may be implemented by dedicated hardware or a combination of software and hardware.

[0017] Furthermore, while the following explanation may use "program" as the subject to describe processing, a program is executed by a processor (e.g., CPU: Central Processing Unit) and performs defined processing using memory resources (e.g., memory) and / or interface devices (communication ports), etc. as appropriate. Therefore, it may also be described as the processor being the entity performing the processing.

[0018] The processes described using a program as the subject may also refer to processes performed by a computer with a processor (e.g., a computing host, a storage device). Furthermore, in the following description, the term "controller" may refer to a processor or a hardware circuit that performs some or all of the processes performed by the processor.

[0019] The program may be installed on each computer from the program source (for example, a program distribution server or a computer-readable storage medium). In this case, the program distribution server includes a CPU and memory resources, which further store the distribution program and the program to be distributed. The CPU of the program distribution server may distribute the program to other computers by executing the distribution program.

[0020] Furthermore, in the following explanation, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0021] Furthermore, in the following description, "storage drive" or simply "drive" means a physical storage device, which may typically be a non-volatile storage device (e.g., an auxiliary storage device). A drive may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Different types of drives may be mixed in a storage system.

[0022] Furthermore, in the following description, a drive has a VOL, where "VOL" is an abbreviation for volume, and may be a physical storage device or a logical storage device. A VOL may be a physical VOL (RVOL) or a virtual VOL (VVOL). An "RVOL" may be a VOL based on the physical storage resources (e.g., one or more RAID groups) of the storage system having that RVOL.

[0023] "VVOL" can be any of the following: External Attached Volume (EVOL), Capacity Expansion Volume (TPVOL), or Snapshot Volume. EVOL is based on the storage space (e.g., VOL) ​​of an external storage system and may be a VOL that conforms to storage virtualization technology. TPVOL consists of multiple virtual areas (virtual storage areas) and may be a VOL that conforms to capacity virtualization technology (typically Thin Provisioning).

[0024] Furthermore, in the following description, the VOL recognized by the host (the VOL provided to the host) is referred to as "LDEV". In the following description, the LDEV is a TPVOL (or RVOL), and the pool is a TP pool. However, the features of this disclosure can also be applied to storage devices that do not employ thin provisioning technology.

[0025] A "pool" is a logical storage area (for example, a collection of multiple pool VOLs), and may be prepared for different purposes. For example, a pool may be a TP pool. A TP pool may be a storage area composed of multiple pages (physical storage areas). If a storage controller receives a write request from a host computer (hereinafter referred to as the host) and the address specified by that address is not allocated a page in the virtual area (the virtual area of ​​the TP VOL), it allocates a page from the TP pool to that virtual area (the write destination virtual area) (even if a page is already allocated in the write destination virtual area, a new page may be allocated to the write destination virtual area). A "pool VOL" may be a VOL that is a component of the pool. A pool VOL may be an RVOL or an EVOL.

[0026] Furthermore, in the following explanation, VOL can refer to either a "Logical Unit (hereinafter referred to as LU)" in SCSI or a "Name Space (hereinafter referred to as NS)" in NVMe.

[0027] Furthermore, in the following explanation, "RAID" is an abbreviation for Redundant Array of Inexpensive Disks. A RAID group consists of multiple drives (typically of the same type) and stores data according to the RAID level associated with that RAID group. A RAID group may also be called a parity group. A parity group may be, for example, a RAID group that stores parity.

[0028] A network interface device (hereinafter also simply referred to as a network interface) according to one embodiment of this specification may be implemented in an information processing system including a server system and a storage system. The storage system may include a storage controller and a drive box. The network interface may include, for example, an interface with a general-purpose processor, memory, a network controller and a host system.

[0029] A configuration using a general-purpose processor and memory capable of software-based protocol processing solves the problem of having to update hardware every time support for a new protocol, such as NVMe / TCP in addition to iSCSI, is added. For example, to quickly respond to changes such as switching from iSCSI to NVMe / TCP, changes due to NVMe / TCP specification updates, or even changes to newer protocols, the system can keep up with changes by replacing the network interface software using a general-purpose processor and memory. One embodiment of the network interface described herein supports the simultaneous processing of multiple network protocols. [Examples]

[0030] Figure 1 shows an example of the configuration of the information processing system in Example 1.

[0031] The information processing system includes one or more server systems 100 and a storage system. The storage system includes one or more storage devices 101 and one or more drive boxes 103.

[0032] The storage device 101 includes one or more storage controllers 102. In Figure 1, each storage device 101 includes two storage controllers 102. The storage controllers 102 connect to one or more server systems 100 via a front-end network 105.

[0033] The drive box 103 connects to one or more storage controllers 102 via the backend network 107. The storage controllers 102 also connect to other nearby storage controllers 102 via the inter-storage controller network 106, and to other storage controllers 102 at medium or long distances via the external network 108.

[0034] The front-end network 105 is a storage area network connecting the storage controller 102 and the server system 100, and utilizes IP networks such as iSCSI and NVMe / TCP (NVMe over TCP).

[0035] The backend network 107 is a network connecting the storage controller 102 and the drive box 103, and uses an IP network such as iSCSI or NVMe / TCP.

[0036] The storage controller network 106 is a network used for redundancy of the storage controller 102 and consists of a high-bandwidth interconnect. This network is used for redundancy of write data and sharing of metadata, so that even if one storage controller 102 is blocked due to maintenance or failure, the other storage controller 102 can continue storage processing.

[0037] The external network 108 is a Wide Area Network (WAN) or Local Area Network (LAN), and for example, a network where the data link layer is Ethernet®, the internet layer is Internet Protocol, and the transport layer is TCP or UDP, and communication is performed using iSCSI or NVMe / TCP Protocol Data Units (PDUs). This network can take the form of an internet line or a dedicated line. Communication delay increases with distance, and if the network equipment is not composed solely of lossless equipment, packet loss is expected to occur, although the occurrence rate will differ depending on the type of line.

[0038] The server system 100 is a host machine on which user applications and the like run, and has one or more processors and is configured to include memory and one or more auxiliary storage devices. For example, a database or web service runs on it, and the server system 100 writes and reads the data created by them to the storage controller 102 via the network interface 104. The server system 100 also has a network interface 104 as an interface device for connecting to the storage controller 102. The server system 100 may be composed of multiple server groups, each server group having a network interface 104 and connecting to the storage controller 102 and other server systems 100.

[0039] The storage controller 102 has one or more processors and memory. The storage controller 102 within the storage device 101 is assumed to have the same configuration.

[0040] The processor has multiple cores, which instruct the transfer of data stored in the corresponding drive box 103 in response to read and write commands from the server system 100.

[0041] The memory consists of semiconductor memory such as SDRAM (Synchronous Dynamic Random Access Memory). The memory may also be configured in combination with volatile memory and non-volatile memory such as SCM (Storage Class Memory). The memory serves as the processor's main memory, storing executable programs (such as storage control programs) and management tables referenced by the processor. The memory is also used as the disk cache (cache memory) of the storage controller 102.

[0042] The storage controller 102 has a network interface 104 as an interface device for connecting to the drive box 103. The network interface 104 communicates with the drive box 103 regarding information related to storage processing such as data transfer and data copying instructed by the server system 100.

[0043] The drive box 103 is equipped with multiple drives such as SSDs (Solid State Drives) and HDDs (Hard Disk Drives). The drive box 103 also includes an internal switch, a processor, and memory for data transfer to connect the multiple drives to the storage controller 102. The drive box 103 receives, stores, and retains data generated by the server system 100 via the storage controller 102.

[0044] To ensure the availability of the data it holds, the drive box 103 may configure RAID among its internal drives, or it may configure RAID among multiple drive boxes 103. Furthermore, the drive box 103 has a network interface 104 as an interface device for connecting to the storage controller 102.

[0045] The network interface 104 is installed in the server system 100, the storage controller 102, and the drive box 103, and serves as a connection interface between various devices and various networks.

[0046] The network interface 104 may be, for example, a SmartNIC. The various functions of the SmartNIC are implemented using the general-purpose processor on the SmartNIC and a hardware offload engine. The SmartNIC may also be configured using an FPGA (Field Programmable Gate Array), in which case each function is implemented on the FPGA. Furthermore, it may be configured as a dedicated interface hardware with the entire system implemented in hardware. Details of the network interface 104 will be described later.

[0047] Note that the information processing system and storage system may include those other than those specified herein. For example, network devices such as switches and routers may be connected to each network, as may devices for monitoring and maintenance. Alternatively, the configuration may involve connecting to a storage service on a public cloud via an external network 109.

[0048] The network interface 104 of the server system 100 has initiator software in iSCSI or NVMe / TCP for reading and writing data to the drive box 103 via the storage controller 102. In contrast, the network interface 104 of the storage controller has target software.

[0049] The network interface 104 of the storage controller 102 has iSCSI or NVMe / TCP initiator software for reading and writing data to the drive box 103. In contrast, the network interface 104 of the drive box 103 has target software.

[0050] Furthermore, the network interface 104 of the storage controller 102 has iSCSI or NVMe / TCP initiator software for reading and writing data from other storage devices 101. In contrast, the network interface 104 of the other storage device 101 has target software.

[0051] Note that some of the network interfaces 104 may be NICs that do not have SmartNIC functionality. For example, the network interface 104 installed on the server system 100 is a general NIC and acts as an initiator, while the network interface 104 installed on the storage controller 102 is a SmartNIC and acts as a target.

[0052] Figure 2 shows an example configuration of the network interface (SmartNIC) 104 in Example 1.

[0053] The network interface 104 is connected via the host bus 201 to equipment that incorporates the network interface 104, such as the storage controller 102 and the internal configuration of the server system 100 or the drive box 103. The network interface 104 is also connected via the network path 200 to other equipment, for example, via an IP network.

[0054] The network interface 104 includes a data processing unit 202 and memory 210. The data processing unit 202 includes a host interface 207 and a network controller 203. Furthermore, the data processing unit 202 includes a processor 204, a memory controller 209, a DMA (Direct Memory Access) controller 206, and a cache 208. The network interface 104 also has one or more ports.

[0055] The host bus 201 is a bus that connects the network interface 104 to the storage controller 102, server system 100, drive box 103, etc. The host bus 201 is a high-bandwidth, high-speed interconnect and, for example, connects to the processor of the installed equipment via PCIe.

[0056] The host interface 207 is an interface for connecting the network interface 104 and the host system (hereinafter simply referred to as the host) via the host bus 201. For example, if the host bus 201 is PCIe, it may include a PCIe PHY.

[0057] The DMA controller 206 exchanges data between the memory 210 of the network interface 104 and the host's memory.

[0058] The network path 200 is, for example, an IP network path and can take the form of a WAN, LAN, or SAN (Storage Area Network). The network interface 104 communicates via one network path 200, or two or more network paths 200, taking redundancy into consideration.

[0059] The network controller 203 is an interface for connecting the network interface 104 to the network path 200. The network controller 203 performs physical layer processing such as PHY, as well as stateless processing at the data link layer, internet layer, and transport layer. The network controller 203 also performs checksum and frame processing, for example.

[0060] The network controller 203 supports, for example, Ethernet, IP, TCP, UDP, etc. Furthermore, it may include offload engines such as IPsec (Internet Security Protocol), TLS (Transport Layer Security), and DIF (Data Integrity Field). It also has a configuration that supports connections to optical cables, copper cables, etc.

[0061] The processor 204 is, for example, a general-purpose processor and runs an operating system. The processor 204 also runs other software to perform tasks such as protocol processing, command processing, and management of the network interface 104. The processor 204 can have any configuration and may include, for example, one or more CPUs or MPUs (Micro Processing Units) and multiple cores 205. Note that the processor 204 is physically different hardware from the processor in the storage controller 102, and each performs its own processing. The processor 204 performs network protocol processing in communication with a host (e.g., server system 100) connected via a port. The processor in the storage controller 102 performs storage processing.

[0062] Memory 210 is composed of semiconductor memory such as SDRAM, and may be configured in combination with non-volatile memory such as SCM. Memory 210 serves as the main memory of the processor 204, storing executable programs (instruction codes for protocol processing and command processing) and management tables referenced by the processor. Memory 210 is also used as a buffer for commands and data sent and received with the network. Furthermore, it has a queuing interface with the network controller 203 and the host interface 207, and stores queue descriptors and indexes.

[0063] The memory controller 209 is an interface for controlling the reading and writing of data to the memory 210. The memory controller 209 may be integrated into, for example, the processor 204, the data processing unit 202, or the network interface 104.

[0064] Cache 208 temporarily stores data between memory 210 and processor 204. Processor 204 can access cache 208 faster than memory 210. Data read from memory 210 is stored in cache 208. Processor 204 accesses cache 208 to read data (including commands). Cache 208 can have a hierarchical structure. Layers closer to the processor are called L1 cache, L2 cache, etc. Cache 208 may be located within processor 204, or, if it has a hierarchical structure, some layers may be within processor 204 while the remaining layers are outside of it. Processor 204 and DMA controller 206 ensure coherence (consistency) with respect to cache 208.

[0065] Furthermore, the information processing system and storage system may include other components not specified herein. For example, modules and interfaces for monitoring and maintenance, and non-volatile storage devices for storing operating systems and software programs running on the network interface 104 may be added.

[0066] Figure 3 shows the program configuration of the network interface 104 in Example 1.

[0067] To implement software-based modifiable processing functions, the network interface 104 uses the processor 204, cache 208, and memory 210 to run the operating system 300, and on top of that, runs software programs for various processes.

[0068] Software programs 300-310 are loaded into memory 210 and executed by processor 204. Some instructions are stored in cache 208. Note that processes such as DIF (Data Integrity Field), CRC (Cyclic Redundancy Check), encryption, compression, hashing, and parity processing may be implemented in dedicated hardware and controlled by software to improve efficiency.

[0069] The operating system 300 is the fundamental software program that forms the basis for operating the network interface 104 and manages the entire network interface 104. The operating system 300 provides a common user environment for all software running on the processor 204 of the network interface 104. The operating system 300 may be an embedded operating system or a general-purpose operating system that runs on a server, such as Linux®.

[0070] The network controller driver 301 is driver software for controlling the network controller 203. The network controller driver 301 outputs a parameter set for packet processing to be offloaded when a packet is generated or received to the network controller 203. Furthermore, the network controller driver 301 outputs to the network controller 203 for sending packets generated by the protocol processing unit 303. In addition, the network controller driver 301 outputs packets received from the network controller 203 to the protocol processing unit 303.

[0071] The host interface driver 302 is driver software for controlling the host interface 207. Communication between the network interface 104 and the host is performed via the host interface driver 302.

[0072] The protocol processor 303 works in conjunction with the command processor 305, DMA control 307, and network controller driver 301 to generate and transmit packets. The protocol processor 303 also processes received packets to extract control information, iSCSI PDU, NVMe / TCP PDU information and data, and outputs the extracted information to the command processor 305.

[0073] Protocol processor 303 performs IP header processing at the Internet Protocol layer, TCP header processing and UDP header processing at the transport layer, and iSCSI and NVMe / TCP processing. For example, protocol processor 303 executes programs such as socket programs, iSCSI initiators or targets, and NVMe / TCP initiators or targets.

[0074] The host queue control 304 is software for controlling the queue interface for exchanging commands with the host. Within the network interface 104, the host queue control 304 manages the queue for commands sent to the host and the queue for commands received from the host. The host queue control 304 stores commands to or from the host in the queue. It also controls the Head and Tail of the queue, for example, when the queue is structured as a ring. The host queue control 304 controls the Head and Tail of the queue held by the host.

[0075] Command processing unit 305 receives commands from the host to control the network interface 104 and controls the network interface 104. Command processing unit 305 receives network protocol processing requests from the host, starts protocol processing unit 303, and sends the processing results of protocol processing unit 303 back to the host. Command processing unit 305 also starts DMA control unit 307 for data transfer with memory allocated on the host and performs response processing. Furthermore, command processing unit 305 performs initial setup, configuration changes, software replacement of the network interface 104, and notification to the host in case of failure.

[0076] Buffer control 306 is software for controlling buffers that temporarily hold data in memory 210. The buffers store data received from the network and transferred to the host, or data received from the host and sent to the network. Buffer control 306 allocates, uses, and manages multiple buffer groups of different sizes. Buffer control 306 controls the buffers to improve the cache hit rate.

[0077] The DMA control 307 performs interface processing with the DMA hardware to control data transfer between, for example, memory allocated on the host side and memory 210 on the network interface 104.

[0078] Core allocation control 308 controls the allocation of core 205 to ports. Details of core 205 allocation control will be described later.

[0079] Maintenance / troubleshooting 309 is for network interface 104 software It supports updates, hardware failure detection, and notification to the host.

[0080] The initialization process 310 initializes the hardware constituting the network interface 104 and performs initialization processing for various software.

[0081] Figure 4 shows the results of Example 1. network This diagram shows an example of the relationship between interfaces and programs.

[0082] The network interface 104 operates the operating system 300, and various software programs run on the operating system 300. Furthermore, the network interface 104 performs initial setup, maintenance such as software updates, and fault handling through initialization processing 310 and maintenance / fault handling 309. Fault handling includes, for example, fault detection and collection of dump trace information such as statistical information and error information.

[0083] The core allocation control 308 controls the allocation of cores 205 that perform network protocol processing in communication with hosts connected via ports.

[0084] The network controller driver 301 controls the network controller 203, stores transmitted packets in the network controller 203's packet buffer, and retrieves received packets from that packet buffer. It also configures settings for offloading frame processing at the data link layer, internet protocol layer, and transport layer, as well as stateless processing such as checksum calculation.

[0085] In response to the operation of the network controller driver 301, the protocol processor 303 performs network protocol processing such as IP, TCP, UDP, iSCSI PDU processing, and NVMe / TCP PDU processing. The protocol processor 303 processes the received packets received from the network controller driver 301 and extracts control information, iSCSI PDU, NVMe / TCP PDU information, and data. Received packets may not contain data. The protocol processor 303 outputs the extracted information to the command processor 305. The data can be transferred to the host via a buffer controlled by the buffer control 306, or without using a buffer.

[0086] Protocol processing 303 includes information obtained from command processing 305 and data sent from the host in a packet to be sent to the network. The data may be transferred to the network via or without a buffer controlled by buffer control 306. The packet to be sent may not contain any data.

[0087] Command processing unit 305 executes command processing in cooperation with host queue control unit 304, buffer control unit 306, and DMA control unit 307. Command processing unit 305 controls the host interface 207 via the host interface driver 302. Command processing unit 305 exchanges transmission or reception commands for network communication with the host system, such as information for iSCSI or NVMe / TCP PDU generation, parsed information, or the PDU itself. Commands are exchanged with the host system using the queue interface. Host queue control unit 304 controls this queue interface.

[0088] Command processing 305 generates descriptors that the host can process based on the results of protocol processing 303 and stores them in the queue via host queue control 304. It also retrieves descriptors generated by the host system from the queue and uses them to configure and utilize various functions within the network interface 104, such as protocol processing 303. Note that the descriptors in the queue can be exchanged between the host system and the network interface 104 using the DMA controller 206 via DMA control 307.

[0089] Figures 5A, 5B, and 5C show an example of the allocation mode of the core 205 of the network interface 104 in Embodiment 1.

[0090] In the core 205 allocation mode shown in Figure 5A, cores 205 are allocated to each port in a predetermined ratio. In Figure 5A, cores 205 are allocated equally to two ports. The cores 205 allocated to a port perform network protocol processing on the data sent and received through the port. In the following description, the core 205 allocation mode shown in Figure 5A will be referred to as the first allocation mode.

[0091] The assignment mode for core 205 shown in Figure 5B is such that all core 205s are assigned to a single port. In the following explanation, Figure 5B The allocation mode for core 205 shown is referred to as the second allocation mode.

[0092] In the core 205 allocation mode shown in Figure 5C, one core 205 is exclusively allocated to each port, while other cores 205 are allocated on a time-sharing basis. In Figure 5C, the shaded cores 205 indicate the cores exclusively allocated to a port. Note that there may be two or more exclusively allocated cores 205. In the following explanation, the exclusively allocated cores 205 will be referred to as exclusive cores 205, and the time-sharing allocated cores 205 will be referred to as shared cores 205. The core 205 allocation mode shown in Figure 5C will be referred to as the third allocation mode.

[0093] The dedicated core 205 performs network protocol processing for commands and data sent and received through its assigned port, and also performs management processing for the assigned port. The dedicated core 205 does not perform network protocol processing or management processing for other ports. Port management processing includes, for example, port initialization, abort, and reset. The shared core 205 performs network protocol processing for data sent and received through the port.

[0094] In the third allocation mode, the dedicated core 205 performs management processing, thus reducing the impact between ports caused by the execution of management processing. Furthermore, since core 205 can be allocated to a single port as much as possible, processing performance can be ensured.

[0095] In the first and second allocation modes, information (allocation information) is generated that associates the port ID with core 205. In the third allocation mode, information (allocation information) is generated that associates the port ID with the occupied core 205.

[0096] Furthermore, the allocation mode for Core 205 can be selected by the user. Another possible allocation mode is one in which one dedicated Core 205 is allocated to each port, and the remaining Core 205s are allocated to each port in a predetermined ratio.

[0097] Figure 6 shows the correspondence between the I / O queue 600 between the core 205 and the server system 100, and the queue 601 between the core 205 and the host in the network interface 104 of Example 1. In Figure 6, the correspondence between the I / O queue 600 between the core 205 and the server system, and the queue 601 between the core 205 and the host is shown in the third allocation mode. The shaded core 205 represents the dedicated core 205, and the white core 205 represents the shared core 205. In Figure 6, the queue 601 is shown to be separated for each Port0 and Port1. The dedicated core 205 is given only one queue for one port, and the shared core 205 is allocated two queues, one for Port0 and one for Port1. However, it is also possible for each core to have one queue 601 instead of having a queue for each port, or for the host to have only one queue per port and distribute from that queue to each core within the network interface.

[0098] For each port, the maximum number of I / O queues 600 is configured, equal to the maximum number of cores 205 that can be allocated. In the example shown in Figure 6, a maximum of seven cores 205 can be allocated to one port, so seven I / O queues 600 are configured. Each dedicated core 205 processes one I / O queue 600 for its allocated port, while the shared cores 205 are controlled to process the I / O queues 600 for each port.

[0099] The queue 601 managed by the dedicated core 205 is controlled to receive commands and data for network protocol processing or data and commands for port management processing. The queue 601 managed by the shared core 205 is controlled to receive data and commands for network protocol processing. The transfer of commands and data to queue 601 is controlled by the host queue control 304 or the DMA control 307.

[0100] By adopting the third allocation mode, it is possible to perform management processing on one port while minimizing the impact on processing on other ports.

[0101] The network interface 104 in this embodiment controls the allocation of cores 205 according to one of three allocation methods. Figures 7A and 7B show an example of state transitions in the allocation control of cores 205 of the network interface 104 in Embodiment 1. Figure 8 shows an example of allocation mode management information held by the network interface 104 in Embodiment 1. Figure 9 is a sequence diagram showing an example of the procedure for changing the allocation mode of the network interface 104 in Embodiment 1.

[0102] Figure 7A is a state transition diagram when the network interface 104 manages the assignment mode. In this case, the network interface 104 manages the assignment mode using assignment mode management information 800 as shown in Figure 8. The assignment mode management information 800 includes current assignment mode 801, which stores information about the current assignment mode, and next assignment mode 802, which stores information about the assignment mode at the next startup.

[0103] When the power is turned on, the network interface 104 transitions from state 701 to state 702. In state 702, the network interface 104 starts the allocation control of core 205 in the allocation mode set to the next allocation mode 802. After starting the allocation control of core 205, the network interface 104 transitions to state 703 and sets the allocation mode of the next allocation mode 802 to the current allocation mode 801.

[0104] When the network interface 104 receives an instruction to change the allocation mode, it transitions to state 704, updates the next allocation mode 802, and waits for a restart. If an instruction to change the allocation mode is received while in state 704, the network interface 104 transitions back to state 704. If a restart occurs while in state 704, the allocation control of core 205 is performed in the changed allocation mode.

[0105] Here, we will explain the procedure for changing the assignment mode using Figure 9.

[0106] The user instructs the storage controller 102, which is a network interface host, to change the allocation mode via a user interface (not shown) (step S901).

[0107] The storage controller 102 generates a host command to change the allocation mode (step S902) and issues it to the network interface 104 (step S903).

[0108] The network interface 104 updates the next assignment mode 802 in the assignment mode management information 800 (step S904). Then, the network interface 104 responds with host command completion (step S905).

[0109] The storage controller 102 instructs the network interface 104 to reset (power off) (step S906). The storage controller 102 also accesses the network interface 104 to confirm that the reset is complete (step S907).

[0110] The user instructs the storage controller 102 to obtain the status of the network interface 104 via the user interface (step S908). Upon receiving this instruction, the storage controller 102 responds with the status of the network interface 104 (step S909).

[0111] Figure 7B is a state transition diagram when network interface 104 does not manage the assignment mode. In this case, the default assignment mode is set for network interface 104.

[0112] When the network interface 104 is powered on, it transitions from state 711 to state 712 and performs initialization. After the initialization is complete, the network interface 104 starts core allocation control in the default allocation mode and transitions to state 713. If a process reboot occurs, the network interface 104 transitions to state 712.

[0113] When the network interface 104 receives an instruction to change the allocation mode, it transitions to state 714 and waits for a process reboot to change to the instructed allocation mode. If a process reboot occurs, the network interface 104 transitions from state 711 to state 712, starts initialization processing, starts core allocation control in the changed allocation mode, and then transitions to state 716. If a process reboot occurs, the network interface 104 transitions to state 715. When an instruction to change the allocation mode is received, the network interface 104 transitions to state 714.

[0114] The procedure for changing the allocation mode is the same as in Figure 9. However, in step S904, the network interface 104 waits for a process reboot in the specified allocation mode. Also, in step S905, the storage controller 102 instructs the network interface 104 to reboot the process.

[0115] Figure 10 is a flowchart illustrating an example of the assignment mode setting process performed by the network interface 104 in Embodiment 1 during startup. Figure 10 illustrates the process performed by the network interface 104 using the method shown in Figure 7A.

[0116] When the network interface 104 is powered on (step S1001), it performs an initialization process (step S1002).

[0117] After the initialization process is complete, the network interface 104 starts a process in next assignment mode (step S1003).

[0118] The network interface 104 refers to the assignment mode management information 800 to determine whether the current assignment mode 801 is different from the next assignment mode 802 (step S1004).

[0119] If the current assignment mode 801 matches the next assignment mode 802, the network interface 104 proceeds to step S1006.

[0120] If the current assignment mode 801 is different from the next assignment mode 802, the network interface 104 updates the current assignment mode 801 (step S1005), and then proceeds to step S1006. Specifically, the value of the next assignment mode 802 is overwritten in the current assignment mode 801.

[0121] In step S1006, the network interface 104 determines whether or not it has received a reset instruction based on the results of a hardware check by the storage controller 102, which is the network interface host (step S1006).

[0122] If no reset command is received, the network interface 104 terminates the assignment mode setting process and starts various other processes.

[0123] When a reset command is received, the network interface 104 transitions to power OFF (step S1007) and then to power ON (step S1001).

[0124] Figure 11 is a flowchart illustrating an example of the assignment mode setting process performed by the network interface 104 in Embodiment 1 during startup. Figure 11 illustrates the process performed by the network interface 104 using the method shown in Figure 7B.

[0125] When the network interface 104 is powered on (step S1101), it performs an initialization process (step S1102).

[0126] After the initialization process is complete, the network interface 104 starts a process in the default assignment mode (step S1103).

[0127] The network interface 104 determines whether or not it has received a reset instruction based on the results of a hardware check by the storage controller 102, which is the network interface host (step S1104).

[0128] When a reset command is received, the network interface 104 transitions to power OFF (step S1105) and then to power ON (step S1101).

[0129] If no reset instruction has been received, the network interface 104 begins accepting instructions to change the assignment mode for a certain period of time. The network interface 104 monitors for instructions to change the assignment mode for a certain period of time and determines whether or not such an instruction has been received (step S1106).

[0130] If no instruction to change the assignment mode has been received, the network interface 104 terminates the assignment mode setting process and starts various other processes.

[0131] When an instruction to change the allocation mode is received, the network interface 104 changes the allocation mode (step S1107) and reboots the process (step S1108).

[0132] After the process reboots, the network interface 104 performs an initialization process (step S1109).

[0133] After the initialization process is complete, the network interface 104 starts a process in the changed assignment mode (step S1110). Subsequently, the network interface 104 finishes the assignment mode setting process and starts various processes.

[0134] Figure 12 is a flowchart illustrating an example of the assignment mode setting process during network protocol processing performed by the network interface 104 of Embodiment 1. Figure 12 illustrates the process performed by the network interface 104 employing the method shown in Figure 7A.

[0135] If the network interface 104 receives an instruction to change the assignment mode while performing network protocol processing, it will start the process described below.

[0136] The network interface 104 updates the next assignment mode 802 in the assignment mode management information 800 (step S1201) and responds to the storage controller 102 that the change is complete (step S1202).

[0137] When the network interface 104 receives a reset command from the storage controller 102 (step S1203), it transitions to power OFF (step S1204) and then to power ON (step S1205).

[0138] The processing from step S1205 to step S1211 is the same as the processing from step S1001 to step S1007.

[0139] Figure 13 is a flowchart illustrating an example of the assignment mode setting process during network protocol processing performed by the network interface 104 of Embodiment 1. Figure 13 illustrates the process performed by the network interface 104 employing the method shown in Figure 7B.

[0140] If the network interface 104 receives an instruction to change the assignment mode while performing network protocol processing, it will start the process described below.

[0141] The network interface 104 changes the allocation mode (step S1301) and responds to the storage controller 102 that the change is complete (step S1302).

[0142] If the network interface 104 receives a reboot instruction from the storage controller 102 (step S1303), it reboots the process (step S1304).

[0143] The processing from step S1305 to step S1306 is the same as the processing from step S1109 to step S1110.

[0144] Figures 14A and 14B are flowcharts illustrating an example of the core distribution process performed by the network interface 104 of Embodiment 1.

[0145] After establishing a connection with the device connected via the network interface 104 (step S1401), the network interface 104 determines the assignment mode of the core 205 (step S1402). If the method shown in Figure 7A is adopted, the assignment mode of the core 205 can be determined based on the assignment mode management information 800. If the method shown in Figure 7B is adopted, the assignment mode of the core 205 can be determined based on the assignment information.

[0146] If the allocation mode is second allocation mode, the network interface 104 obtains the allocated core number (step S1403). The allocated core number is the identification number of core 205.

[0147] The network interface 104 assigns a connection to core 205 corresponding to the assigned core number (step S1404).

[0148] The network interface 104 updates the assigned core number (step S1405) and terminates the core allocation process. For example, the assigned core number is updated so that core 205 is allocated using a round-robin method.

[0149] If the assignment mode is first assignment mode, the network interface 104 obtains the port number of the port on which the connection was established (step S1406).

[0150] The network interface 104 obtains the assigned core number corresponding to the port number (step S1407). In the first assignment mode, a predetermined number of cores 205 are assigned to each port, so the assigned core number is managed for each port.

[0151] The network interface 104 assigns a connection to core 205 corresponding to the assigned core number (step S1408).

[0152] The network interface 104 updates the assigned core number (step S1409) and terminates the core allocation process. For example, the assigned core number is updated so that core 205 is allocated using a round-robin method.

[0153] If the assignment mode is third assignment mode, the network interface 104 obtains the port number of the port on which the connection was established (step S1410).

[0154] Network interface 104 obtains the assigned core number (step S1411).

[0155] The network interface 104 determines whether the core 205 corresponding to the assigned core number is a core occupying a port other than the port number it acquired (step S1412).

[0156] If core 205, which corresponds to the assigned core number, is a core that occupies a port other than the acquired port number, the network interface 104 updates the assigned core number (step S1413), and then returns to step S1411. For example, the assigned core number is updated so that core 205 is assigned using a round-robin method.

[0157] If core 205 corresponding to the assigned core number is not a core occupying a port other than the acquired port number, the network interface 104 assigns a connection to core 205 corresponding to the assigned core number (step S1414).

[0158] The network interface 104 updates the assigned core number (step S1415) and terminates the core allocation process. For example, the assigned core number is updated so that core 205 is allocated using a round-robin method.

[0159] Figure 15 is a sequence diagram showing the queue configuration process for network interface 104 in Example 1.

[0160] The initiator server system 100, for example, when establishing an NVMe / TCP connection, determines the number of queues to request from the target (step S1501) and sends a queue configuration request to the target storage device 101 (step S1502).

[0161] The network interface 104 receives a queue configuration request and checks the number of queues requested (step S1503).

[0162] The network interface 104 checks the number of cores 205 that can be assigned to the port that received the queue configuration request (step S1504), and determines the number of queues to configure based on that number (step S1505). For example, the number of queues is determined to be an integer multiple of the number of cores 205. At this time, the number of requested queues may also be taken into consideration.

[0163] After setting the determined number of queues, the network interface 104 sends a response to the initiator server system 100, including the determined number of queues (step S1506).

[0164] The server system 100 determines the number of queues to use based on the response (step S1507).

[0165] When a large number of queues are handled by core 205, overhead occurs due to queue switching. In this embodiment, the network interface 104 determines the maximum number of queues based on the number of cores 205 that can be assigned to the port and notifies the initiator server system 100. This reduces the occurrence of overhead.

[0166] Figures 16 and 17 show an example of the distribution and operating state of network protocol processing in the network interface 104 of Embodiment 1. Here, the network interface 104 is assumed to have port 0 and port 1.

[0167] Figure 16 shows the distribution of processing and operating status in network interface 104 operating in first allocation mode. The dotted rectangular wave represents network protocol processing for port 0, the diagonal rectangle represents network protocol processing for port 1, and the white rectangle represents management processing for port 1.

[0168] In the first allocation mode, cores 205 are allocated to two ports in a predetermined ratio. Each core 205 performs network protocol processing for its assigned port. Core (4) performs management processing when it receives a management request for port 1. Until the management processing is complete, the other cores (5), (6), and (7) do not perform network protocol processing for port 1.

[0169] Figure 17 shows the distribution of processing and operating status in network interface 104 operating in third allocation mode. The dotted rectangular wave represents network protocol processing for port 0, the diagonal rectangle represents network protocol processing for port 1, and the white rectangle represents management processing for port 1.

[0170] The dedicated core 205 performs network protocol processing and management processing for the assigned port. The shared core 205 performs network protocol processing for the time-sharing assigned port. When the dedicated core (4) receives a request to execute management processing, it executes the said management processing. While management processing is being executed, network protocol processing for port 1 is not performed, but network protocol processing for port 0 is performed using the shared core 205.

[0171] Furthermore, if the load on network protocol processing is known in advance, connections may be allocated considering core performance and core utilization.

[0172] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. Furthermore, for example, the embodiments described above are detailed explanations of the configuration in order to clearly illustrate the present invention, and are not necessarily limited to those having all the configurations described. In addition, some of the configurations in each embodiment can be added to, deleted from, or replaced with other configurations.

[0173] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, in whole or in part, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0174] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, Python, and Java (registered trademark).

[0175] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0176] In the above-described embodiment, the control lines and information lines shown are those deemed necessary for explanation and do not necessarily represent all control lines and information lines in the actual product. All components may be interconnected. [Explanation of Symbols]

[0177] 100 Server Systems 101 Storage device 102 Storage Controllers 103 Drive Box 104 Network Interfaces 105 Front-end network 106 Inter-storage controller network 107 Backend Network 108 External Network 109 External Network 200 network paths 201 Host Bus 202 Data Processing Unit 203 Network Controller 204 Processors 205 cores 206 DMA Controller 207 Host Interface 208 Cache 209 Memory Controller 210 memory 300 Operating Systems 301 Network Controller Driver 302 Host Interface Driver 303 Protocol Processing 304 Host Queue Control 305 Command Processing 306 Buffer control 307 DMA control 308 Core Allocation Control 309 Maintenance / Troubleshooting 310 Initialization process 600 Queues 601 Queue 800 Assignment Mode Management Information

Claims

1. A storage system comprising at least one storage controller, The storage controller includes a processor having multiple cores, memory connected to the processor, and a network interface having multiple ports connected to the processor. The aforementioned processor, At least one dedicated core is assigned to each of the aforementioned multiple ports. The dedicated core is controlled to perform management processing for the port to which the dedicated core is assigned and network protocol processing for communications performed through the port to which the dedicated core is assigned. A storage system characterized in that the cores other than the dedicated core are controlled to perform network protocol processing for communications conducted via the port.

2. A storage system according to claim 1, The storage system is characterized in that, when the processor detects an opportunity to execute network protocol processing for one of the ports, it causes a core other than the dedicated core or the dedicated core of that port to execute the network protocol processing.

3. A storage system according to claim 1, The aforementioned processor, To each of the aforementioned multiple ports, the cores other than the occupied core are allocated in a predetermined ratio. A storage system characterized in that, when an opportunity to execute network protocol processing for one of the aforementioned ports is detected, the system causes a core assigned to that port other than the dedicated core, or the dedicated core of that port, to execute the said network protocol processing.

4. A storage system according to claim 1, A storage system characterized in that the processor determines the number of queues for each of the plurality of ports based on the maximum number of cores to be allocated, including the dedicated cores of the port.

5. A storage system according to claim 1, The storage system is characterized in that the processor provides an interface for selecting the core allocation mode described in claim 1, the allocation mode for all of the cores to a single port, and the allocation mode for all of the multiple ports in a predetermined ratio.

6. A method for controlling the allocation of cores of a storage controller in a storage system, comprising a processor having multiple cores, memory connected to the processor, and a network interface having multiple ports connected to the processor, The first step is for the processor to assign at least one dedicated core to each of the plurality of ports, The processor provides a second step of controlling the dedicated core to perform management processing for the port to which the dedicated core is assigned and network protocol processing for communications performed through the port to which the dedicated core is assigned. A method for controlling the allocation of cores for a network interface, characterized in that the processor controls the cores other than the dedicated core to perform network protocol processing for communications performed via the port.

7. A method for controlling the allocation of cores of a network interface according to claim 6, A method for controlling the allocation of cores in a network interface, characterized in that when the processor detects an opportunity to execute network protocol processing for one of the ports, it includes the step of causing a core other than the dedicated core or the dedicated core of the port to execute the network protocol processing.

8. A method for controlling the allocation of cores of a network interface according to claim 6, The first step includes the processor allocating the cores other than the occupied core to each of the plurality of ports in a predetermined ratio. The method for controlling the allocation of cores of the network interface is characterized in that, when the processor detects an opportunity to execute network protocol processing for one of the ports, it causes a core other than the dedicated core assigned to that port or the dedicated core of that port to execute the network protocol processing.

9. A method for controlling the allocation of cores of a network interface according to claim 6, A method for controlling the allocation of cores in a network interface, characterized in that the processor includes the step of determining the number of queues for each of the plurality of ports based on the maximum number of cores to be allocated, including the dedicated cores of the port.

10. A method for controlling the allocation of cores of a network interface according to claim 6, A method for controlling the allocation of cores in a network interface, characterized in that the processor presents an interface for selecting the core allocation mode described in claim 6, the allocation mode for all of the cores to be allocated to one port, and the allocation mode for all of the multiple ports to be allocated the cores in a predetermined ratio.

Images