Network interface and buffer control method therefor

Abstract

Suppresses performance degradation due to memory bottlenecks in network interfaces. [Solution] The network interface includes a processor, a memory, and a cache that is accessed by the processor before the memory, and the processor reserves multiple buffer groups in the memory, each of which is composed of multiple buffers of the same size and has different buffer sizes from one another, and when the transfer data size of first transfer data is equal to or smaller than the maximum buffer size of the different buffer sizes, the processor selects the buffer group with the smallest buffer size among the buffer sizes equal to or larger than the transfer data size to store the first transfer data.

Description

[Technical Field]

[0001] The present invention relates to a network interface. [Background technology]

[0002] For example, information systems and storage systems are composed of a server system on which user applications run, a storage controller that manages the stored data and improves reliability, and a drive box containing multiple drives that store and hold data used by the server system.

[0003] The storage system receives a request from the server system, and in the case of a read request, for example, the storage controller reads data from the drive in the drive box and transfers the data to the server system.

[0004] Traditionally, the front-end network of storage controllers, especially for enterprise products, has been dominated by FC (Fibre Channel) networks, while the back-end network has been dominated by SAS (Serial Attached SCSI) networks. Drive boxes have typically been configured as JBOD (Just a Bunch of Disks), with multiple SAS / SATA drives for storing data installed in slots in the drive box.

[0005] In recent years, the adoption of high-performance flash drives such as SSDs (Solid State Drives) has increased in order to improve the performance of storage systems, and NVMe (Non-Volatile Memory) technology, which is optimal for accessing flash drives, has become increasingly popular. The NVMe Express protocol was standardized, and the NVMe over Fabrics (NVMe-oF) standard was also introduced, which allows the NVMe protocol to be used over Internet Protocol (IP) for highly scalable drive connections in back-end networks.

[0006] The idea of ​​End-to-End-NVMe-oF, in which NVMe-oF is applied to front-end networks as well and all connections from the server system via the storage controller to the drive box are NVMe-oF, is becoming more widespread. Drive boxes equipped with NVMe-oF-compatible flash drives (FBOF: Fabric-attached Bunch of Flash) and Composable Storage, which connects multiple storage controllers via a network, have emerged, and the use of IP connections is becoming more widespread in the enterprise storage field.

[0007] With the increasing speed of networks and the emergence of new protocols (such as NVMe-oF), enterprise storage systems are also required to have faster network interfaces and support multiple protocols. Conventionally, multiple protocols have been supported by using the CPU on the storage controller or by offloading each protocol to dedicated hardware. For example, U.S. Patent No. 9,100,349 discloses technology that supports multiple protocols in a single device using a network interface card (NIC) equipped with both Ethernet and FC protocol chips.

[0008] However, the use of the CPU on the storage controller has the problem that CPU resources are consumed by protocol processing. The CHB requires the development of a dedicated ASIC for each protocol, which is timely. It was difficult to provide comprehensive protocol support.

[0009] In response to this, network interface devices are also changing, with the emergence of SmartNICs, which incorporate general-purpose processors and memory, run an operating system, and then run software on top of that to process network protocols. SmartNICs can run the same operating system as those that run on server systems, for example, and can also run the software protocol stacks and applications used there. Because processing can be implemented in software, they can flexibly support multiple protocols, quickly respond to new protocols, and update protocol processing. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] U.S. Patent No. 9,100,349 Summary of the Invention [Problem to be solved by the invention]

[0011] By implementing protocol processing in software using SmartNIC, a NIC equipped with a general-purpose CPU and memory, it is possible to flexibly support multiple protocols while offloading the protocol processing load from the storage controller.

[0012] However, due to the constraints of the board area required for mounting the SmartNIC on the storage controller and the power limitations of the storage controller interface, there are limitations on the type, number and wiring of the memory chips that can be mounted.

[0013] For example, protocols such as iSCSI and NVMe TCP transfer data in TCP / IP packets, temporarily store the packets in the SmartNIC memory, and require multiple memory access processes, so it may not be possible to achieve a configuration that achieves the required memory performance. As a result, memory may become a performance bottleneck and the desired performance may not be achieved. [Means for solving the problem]

[0014] A network interface according to a representative embodiment of the present invention includes a processor, a memory, and a cache that is accessed by the processor before the memory, wherein the processor reserves multiple buffer groups in the memory, each buffer group consisting of multiple buffers of the same size, the multiple buffer groups having different buffer sizes from one another, and when the transfer data size of first transfer data is equal to or smaller than the maximum buffer size of the different buffer sizes, the processor selects a buffer group with the smallest buffer size among buffer sizes equal to or larger than the transfer data size to store the first transfer data. [Effects of the Invention]

[0015] According to a representative embodiment of the present invention, it is possible to suppress performance degradation due to memory bottlenecks in a network interface. Problems, configurations, and effects other than those described above will become clear from the following description of the embodiment. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 illustrates an example of the configuration of an information processing system. [Figure 2] FIG. 2 illustrates an example of the configuration of a network interface. [Figure 3] FIG. 2 is a diagram showing a program configuration of a network interface. [Figure 4] FIG. 10 is a diagram illustrating an example of a program relationship of a storage interface. [Figure 5]FIG. 2 is a diagram illustrating an outline of a process in which a network interface transmits and receives commands and data to and from a host. [Figure 6A] FIG. 1 illustrates some aspects of data transfer. [Figure 6B] FIG. 1 illustrates some aspects of data transfer. [Figure 7] An example of the configuration of a buffer list is shown below. [Figure 8A] Indicates the buffer list management information contained in the buffer list. [Figure 8B] Indicates the information held by the buffer entry. [Figure 9] 10 shows an example of the configuration of a buffer management table. [Figure 10] 10 shows an example of a processing flow of a target network interface. [Figure 11] 10 shows an example of a processing flow of a network interface of an initiator. [Figure 12] 10 shows a flowchart of an example of a buffer control process. [Figure 13] 10 shows a flowchart of an example of a buffer list determination process. [Figure 14] 10 is a flowchart of an example of a buffer control process. [Figure 15] 10 shows a flowchart of an example of processing in a buffer list determination step. [Figure 16] 10 shows a flowchart of an example of a maximum active number determination process. [Figure 17] 10 shows a flowchart of another example of the maximum active number determination process. [Figure 18] 10 shows a flowchart of another example of the maximum active number determination process. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment will be described with reference to the drawings. Note that the following description and drawings are merely examples for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms, and unless otherwise specified, each component may be singular or plural.

[0018] Furthermore, the embodiments described below do not limit the invention according to the claims, and not all of the combinations of elements described in the embodiments are necessarily essential to the solution of the invention.

[0019] In the following explanation, various types of information may be described using expressions such as "table," "list," and "queue," but various types of information may be expressed in data structures other than these, and to indicate that they are independent of data structure, "xxx table," "xxx list," "xxx queue," etc. may be referred to as "xxx information," etc. In the following explanation, when describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0020] In the following description, when there are multiple components having the same or similar functions, they will generally be described using the same reference numerals, but even if the functions are the same, the means for realizing the functions may be different. Furthermore, the embodiments described below may be implemented as software running on a general-purpose computer, or as dedicated hardware or a combination of software and hardware.

[0021] Furthermore, in the following explanation, processing may be described using the "program" as the subject, but since a program is executed by a processor (e.g., CPU: Central Processing Unit) to perform a specified process using storage resources (e.g., memory) and / or interface devices (communication ports) as appropriate, the subject of the processing may also be described as the processor.

[0022] The processing described with a program as the subject may be processing performed by a computer having a processor (for example, a computing host or a storage device). The term "processor" may refer to a processor or a hardware circuit that performs some or all of the processing performed by a processor.

[0023] The program may be installed on each computer from a program source (e.g., a program distribution server or a computer-readable storage medium), in which case the program distribution server includes a CPU and storage resources, and the storage resources further store the distribution program and the program to be distributed, and the CPU of the program distribution server may distribute the program to be distributed to other computers by executing the distribution program.

[0024] In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0025] In the following description, a storage drive or simply a drive refers to a physical storage device, and may typically be a non-volatile storage device (e.g., an auxiliary storage device). A drive may be, for example, a hard disk drive (HDD) or a solid state drive (SSD). Different types of drives may be mixed in a storage system.

[0026] Furthermore, in the following explanation, a drive has a VOL, and "VOL" is an abbreviation for volume, which may be a physical storage device or a logical storage device. A VOL may be a real VOL (RVOL) or a virtual VOL (VVOL). An "RVOL" may be a VOL based on the physical storage resources (for example, one or more RAID groups) of the storage system that has that RVOL.

[0027] A "VVOL" may be any of an Externally Connected VOL (EVOL), a Capacity Expanded VOL (TPVOL), or a Snapshot VOL. An EVOL is based on the storage space (e.g., a VOL) ​​of an external storage system and may be a VOL that complies with storage virtualization technology. A TPVOL is composed of multiple virtual areas (virtual storage areas) and may be a VOL that complies with capacity virtualization technology (typically Thin Provisioning).

[0028] In the following explanation, a VOL recognized by a host (a VOL provided to a host) is referred to as an "LDEV." In the following explanation, an LDEV is a TPVOL (or RVOL), and a pool is a TP pool. However, the features of this disclosure can also be applied to storage devices that do not employ capacity expansion technology (Thin Provisioning).

[0029] A "pool (POOL)" is a logical storage area (for example, a collection of multiple pool VOLs) and may be prepared for each purpose. For example, the pool may be a TP pool. A TP pool may be a storage area made up of multiple pages (physical storage areas). If a page is not assigned to the virtual area (TPVOL virtual area) to which the address specified in a write request received from a host computer (hereinafter referred to as the host) belongs, the storage controller allocates a page from the TP pool to that virtual area (write-destination virtual area) (even if a page has already been allocated to the write-destination virtual area, a new page may be allocated to the write-destination virtual area). A "pool VOL" may be a VOL that is a component of a pool. A pool VOL may be an RVOL or an EVOL.

[0030] In the following description, a VOL may be a "Logical Unit (hereinafter, LU)" in SCSI or a "Name Space (hereinafter, NS)" in NVMe.

[0031] In the following explanation, "RAID" stands for Redundant Array of Independent Disks (RAID). RAID group is an abbreviation for Inexpensive Disks. A RAID group is made up of drives (typically of the same type) and stores data according to the RAID level associated with that RAID group. A RAID group may also be called a parity group. A parity group may, for example, be a RAID group that stores parity.

[0032] A network interface device (hereinafter simply referred to as a network interface) according to an embodiment of the present specification can be implemented in an information processing system including a server system and a storage system. The storage system can include a storage controller and a drive box. The network interface can include, for example, a general-purpose processor, a memory, a network controller, and an interface with a host system.

[0033] A configuration using a general-purpose processor and memory capable of software-based protocol processing solves the problem of having to update hardware every time support for a new protocol, such as NVMe-oF, is added in addition to iSCSI. For example, a general-purpose processor, memory, etc. can be used to quickly respond to changes such as switching from iSCSI to NVMe-oF, changes due to NVMe-oF specification updates, or changes to new protocols, by replacing the network interface software. The network interface of one embodiment of this specification supports multiple protocol processing simultaneously.

[0034] In one embodiment of the present specification, a network interface can employ some or all of the following methods for controlling a data buffer that stores data that is the dominant factor in memory access during protocol processing, in order to reduce memory access during protocol processing.

[0035] The network interface uses buffers of multiple sizes. The buffer size is set according to the transfer data size of the supported protocol. For example, multiple data buffers are allocated for each supported transfer data size. Data buffers are allocated in a list structure, for example, at initialization (startup).

[0036] For example, when a network interface starts up, it allocates multiple buffers of different sizes, such as 8KB and 256KB. This reduces the buffer allocation load during I / O and makes it possible to use buffers for short and long data lengths. When many short data length transfer requests are received, the amount of memory used is reduced, and when long data length transfer requests are received, the load of looking up the list is reduced by not using as many buffers.

[0037] A network interface according to an embodiment of this specification selects a recently used area when selecting a data buffer. Data buffers are managed in a list, and the most recently released buffer is used for the next process. This localizes the range of access to the data buffer, increasing the cache hit rate. By achieving cache hits for transferred data and reducing the number of memory accesses, the problem of performance degradation due to memory bottlenecks is resolved.

[0038] A network interface according to an embodiment of this specification controls the command processing that can be started based on the difference between the cache capacity and the total amount of active buffer usage. To minimize data overflow from the cache, the network interface limits the number of buffers used for simultaneously processing data transfers. This increases the cache hit frequency, reduces memory accesses, and prevents performance degradation due to memory bandwidth pressure.

[0039] 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment of the present specification. The information processing system includes one or more server systems 100 and a storage system. The storage system includes one or more storage devices 101 and one or more drive boxes 105.

[0040] The storage device 101 includes one or more storage controllers 102. In the configuration example of Fig. 1, each storage device 101 includes two storage controllers 102. The storage controllers 102 are connected to one or more server systems 100 via a front-end network 106.

[0041] A drive box 105 is equipped with one or more drives and is connected to one or more storage controllers 102 via a back-end network 107. The storage controller 102 is connected to other storage controllers 102 at medium or long distances via an external network 109, in addition to being connected to a short distance via an inter-storage controller network 108.

[0042] The server system 100 is a host machine on which user applications and the like run, and is configured to include one or more processors and one or more storage devices, such as memory and auxiliary storage devices. For example, a database or a web service runs on the server system 100, and the server system 100 writes and reads data created by these to and from the storage controller 102 via a network interface 104. The server system 100 is also connected to the storage controller 102 via a front-end network 106, and has a network interface 104 as its interface device. The server system 100 may be configured from a group of multiple servers, each of which may have a network interface 104 and be connected to the storage controller 102 or other server systems 100.

[0043] To provide storage functionality to the server system 100, the two storage controllers 102 in the storage device 101 configure a redundant controller. Each storage controller 102 includes one or more processors and one or more storage devices. The storage controllers 102 in the storage device 101 have the same configuration.

[0044] Each storage controller 102 has one or more processors, and the core of each processor instructs the transfer of data stored in the corresponding drive box 105 in response to a read command or a write command from the server system 100. The memory of the storage controller 102 is configured, for example, with semiconductor memory such as SDRAM (Synchronous Dynamic Random Access Memory). The memory may also be configured by combining volatile memory and non-volatile memory such as SCM (Storage Class Memory).

[0045] The memory serves as the processor's main memory and stores execution programs (storage control programs, etc.), management tables referenced by the processor, etc. The memory is also used as a disk cache (cache memory) for the storage controller 102. The storage controller 102 has a network interface 104 as an interface device for the drive box 105. The network interface 104 communicates with the drive box 105 information relating to storage processing such as data transfer and data copy instructed by the server system 100.

[0046] The drive box 105 is equipped with a plurality of drives such as SSDs and HDDs, and includes an internal switch and a transfer The storage controller 102 receives, stores, and holds data generated by the server system 100.

[0047] To ensure the availability of data stored in the drive box 105, a RAID may be configured between the drives built in the drive box 105, or a RAID may be configured between multiple drive boxes 105. The drive box 105 is also connected to the storage controller 102 via a back-end network 107, and has a network interface 104 as its interface device.

[0048] The network interface 104 is a device that is mounted on each of the server system 100, the storage controller 102, and the drive box 105, and serves as a connection interface between various devices and various networks.

[0049] The network interface 104 may be, for example, a SmartNIC. Various functions of the SmartNIC are implemented using a general-purpose processor equipped with the SmartNIC and a partial hardware offload engine. The SmartNIC may also be configured using an FPGA (Field Programmable Gate Array), in which case each function is realized on the FPGA. Alternatively, the SmartNIC may be configured as dedicated interface hardware that is entirely implemented in hardware. Details of the network interface 104 will be described later.

[0050] The front-end network 106 is a storage area network that connects the storage controller 102 and the server system 100, and uses, for example, an IP network such as iSCSI or NVMe-oF (NVMe over Fabrics). The back-end network 107 is a network that connects the storage controller 102 and the drive box 105, and uses, for example, an IP network such as iSCSI or NVMe-oF.

[0051] The inter-storage controller network 108 is a network that is used for redundancy of the storage controllers 102 and is configured with a wide-band interconnect. This network is used to duplicate write data and share metadata, so that even if one storage controller 102 is blocked due to maintenance, a fault, or the like, the other storage controller 102 can continue storage processing.

[0052] The external network 109 is a wide area network (WAN) or a local area network (LAN), for example, a network with a data link layer of Ethernet (registered trademark), an internet layer of Internet Protocol, and a transport layer of TCP or UDP, and performs communication using iSCSI or NVMe-oF Protocol Data Units (PDUs). The network can take the form of an internet line or a dedicated line. Communication delays increase depending on the distance, and if the network devices are not configured with only lossless devices, packet loss is expected, although the occurrence rate varies depending on the type of line.

[0053] The information system and storage system may include systems other than those shown here. For example, each network may be connected to network devices such as switches and routers, or to devices for monitoring and maintenance. Also, a configuration in which a storage service on a public cloud is connected via an external network 109 may be used.

[0054] The network interface 104 of the server system 100 uses iSCSI or other protocols to read and write data from and to the drive box 105 via the storage controller 102. In NVMe-oF, the network interface 104 of the storage controller has initiator software, whereas the network interface 104 of the storage controller has target software.

[0055] The network interface 104 of the storage controller 102 has initiator software for iSCSI or NVMe-oF to read and write data from and to the drive box 105. In contrast, the network interface 104 of the drive box 105 has target software.

[0056] Furthermore, the network interface 104 of the storage controller 102 has initiator software for iSCSI or NVMe-oF to read and write data from and to other storage devices 101. In contrast, the network interface 104 of the other storage devices 101 has target software.

[0057] Note that some of the network interfaces 104 may be NICs that do not have the functionality of a SmartNIC. For example, the NIC 104 in 100 is a NIC that operates as an initiator, and the NIC 104 in 102 is a SmartNIC that operates as a target.

[0058] 2 is a diagram illustrating an example of the configuration of a network interface 104 according to an embodiment of the present specification. The network interface 104 is connected to a device incorporating the network interface 104, such as the storage controller 102, the internal configuration of the server system 100, or a drive box 105, via a host bus 201. The network interface 104 is connected to other devices via a network path 200, for example, via an IP network connection.

[0059] The network interface 104 includes a data processing unit 202 and a memory 208. The data processing unit 202 includes a host interface 207 for connecting to the host bus 201, and a network controller 203 for connecting to the network path 200 and processing network protocols. The data processing unit 202 further includes a processor 204, a memory controller 209, a DMA (Direct Memory Access) controller 205, and a cache 206.

[0060] The host bus 201 is a bus that connects the network interface 104 to the storage controller 102, the server system 100, the drive box 105, etc. The host bus 201 is a broadband, high-speed interconnect, and is configured to connect to a processor of a device mounted on it via PCIe, for example.

[0061] The host interface 207 is an interface for connecting the network interface 104 to a host system (hereinafter also simply referred to as a host) via the host bus 201. For example, if the host bus 201 is PCIe, it may include a PCIe PHY. The DMA controller 205 exchanges data between the memory 208 in the network interface 104 and the memory of the host.

[0062] The network path 200 is, for example, an IP network path, and may take the form of a WAN, LAN, or SAN (Storage Area Network). The network interface 104 communicates via one network path 200, or two or more network paths 200 in consideration of redundancy.

[0063] The network controller 203 is an interface for connecting the network interface 104 to the network path 200. The network controller 203 processes, for example, a physical layer such as PHY, as well as a data link layer, an internet layer, and The network controller 203 performs stateless processing of the transport layer and performs checksum and frame processing, for example.

[0064] The network controller 203 supports, for example, Ethernet, IP, TCP, UDP, etc. Furthermore, it may include offload engines such as IPsec (Internet Security Protocol), TLS (Transport Layer Security), and DIF (Data Integrity Field). It also has a configuration that supports connection with optical cables, copper cables, etc.

[0065] The processor 204 is, for example, a general-purpose processor, and executes an operating system that is also used in the server system 100. The processor 204 also executes other software to perform processes such as protocol processing, command processing, and management of the network interface 104. The processor 204 may have any configuration, and may include, for example, one or more CPUs or MPUs (Micro Processing Units), and one or more cores 241.

[0066] The memory 208 is configured, for example, by a semiconductor memory such as an SDRAM, and may be configured in combination with a non-volatile memory such as an SCM. The memory 208 serves as the main memory of the processor 204, storing execution programs (instruction codes for protocol processing and command processing) and management tables referenced by the processor. The memory 208 is also used as a buffer for commands and data sent and received from the network. Furthermore, the memory 208 provides a queuing interface with the network controller 203 and host interface 207, and stores queue descriptors, indexes, etc.

[0067] The memory controller 209 is an interface for controlling the memory when reading and writing data from and to the memory 208. The memory controller 209 may be built into the processor 204, the data processing unit 202, or the network interface 104, for example.

[0068] The cache 206 temporarily stores data between the memory 208 and the processor 204. The processor 204 can access the cache 206 faster than the memory 208. Data read from the memory 208 is stored in the cache 206. The processor 204 accesses the cache 206 before the memory 208 to read data (including commands). The cache 206 can have a hierarchical structure. In descending order of hierarchy, caches are called L1 cache, L2 cache, etc. The processor 204 and the DMA controller 205 ensure coherency (consistency) with the cache 206.

[0069] It should be noted that the information system and storage system may include other components in addition to those shown here, such as modules and interfaces for monitoring and maintenance, and non-volatile storage devices for storing operating systems and software programs running on the network interface 104.

[0070] 3 is a diagram showing the program configuration of a network interface according to an embodiment of the present specification. To realize software-based, changeable processing functions, the network interface 104 uses a general-purpose processor 204, cache 206, and memory 208 to run an operating system 300, on which various processing software programs are run.

[0071] The software programs 300 to 308 are loaded into the memory 208 and executed by the processor 20 4 are processed. Some instructions are stored in the cache 206. Processing parts that are more efficient when implemented in hardware, such as a CRC (Cyclic Redundancy Check), encryption, compression, hashing, and parity processing, may be implemented in hardware and controlled by software.

[0072] The operating system 300 is a basic software program that serves as the foundation for operating the network interface and manages the entire network interface. The operating system 300 provides a common operating environment for each piece of software running on the processor of the network interface. The operating system 300 may be an embedded operating system or a general-purpose operating system that runs on a server, such as Linux (registered trademark).

[0073] The network controller driver 301 is driver software for controlling the network controller 203. The network controller driver 301 passes a parameter set for packet processing to be offloaded when a packet is generated or received to the network controller 203. Furthermore, the network controller driver 301 passes a packet generated by the protocol processing 303 to the network controller 203 for transmission. The network controller driver 301 also passes a packet received from the network controller 203 to the protocol processing 303.

[0074] The host interface driver 302 is driver software for controlling the host interface 207. Communication between the network interface 104 and the host is carried out via the host interface driver 302.

[0075] The protocol processing 303 generates transmission packets and performs transmission processing thereof in cooperation with the command processing 305, the DMA control 307, and the network controller driver 301. The protocol processing 303 also processes received packets to extract information and data such as control information, iSCSI PDU, and NVMe-oF PDU, and passes the extracted information to the command processing 305.

[0076] The protocol processing 303 performs IP header processing at the Internet Protocol layer, TCP header processing and UDP header processing at the transport layer, iSCSI processing, and NVMe-oF processing. For example, the protocol processing 303 executes programs such as a socket program, an iSCSI initiator or target, and an NVMe-oF initiator or target.

[0077] The host queue control 304 is software for controlling the queue interface for exchanging commands with the host. The host queue control 304 manages the internal queue of commands to be sent to the host and the internal queue of commands received from the host within the network interface 104. The host queue control 304 stores commands to the host or commands from the host in the internal queue. It also controls the head and tail when the internal queue has, for example, a ring structure. The host queue control 304 controls the head and tail of the command queue held by the host.

[0078] Command processing 305 receives commands for controlling the network interface 104 from the host and controls the network interface 104. It receives a network protocol processing request from the host, starts protocol processing 303, and responds to the host with the processing results of protocol processing 303. It also starts DMA control 307 to transfer data with memory allocated by the host and performs response processing. It also performs initial setting and setting changes, software replacement for the network interface 104, and notification to the host in the event of a failure.

[0079] The buffer control 306 is software for controlling buffers that temporarily hold data in the memory 208. The buffers store data received from the network and transferred to the host, or data received from the host and sent to the network. The buffer control 306 allocates, uses, and manages a group of buffers of different sizes. The buffer control 306 controls the buffers to improve the cache hit rate. This point will be described in more detail later.

[0080] The DMA control 307 performs interface processing with DMA hardware to control data transfer between, for example, memory secured on the host side and the memory 208 on the network interface 104 .

[0081] The initialization / maintenance / fault processing 308 initializes the hardware that constitutes the network interface 104 and initializes various pieces of software. It also supports software updates for the network interface 104, detection of hardware faults and notification to the host, etc.

[0082] Figure 4 is a diagram showing an example of the relationship between the programs of the storage interface shown in Figure 3. An operating system 300 runs on the network interface 104, and each software program operates based on this. Furthermore, the network interface 104 executes initial settings, maintenance such as software updates, and fault handling through initialization / maintenance / fault processing 308. Fault handling includes, for example, fault detection and the collection of dump trace information such as statistical information and error information.

[0083] The network controller driver 301 controls the network controller 203, stores transmission packets in the packet buffer of the network controller 203, and retrieves received packets from the packet buffer. It also performs settings for offloading frame processing of the data link layer, internet protocol layer, and transport layer, as well as stateless processing such as checksum calculation.

[0084] In response to the operation of the network controller driver 301, protocol processing 303 performs network protocol processing such as IP, TCP, UDP, iSCSI PDU processing, and NVMe-oF PDU processing. Protocol processing 303 processes packets received from the network controller driver 301 to extract information and data such as control information, iSCSI PDUs, and NVMe-oF PDUs. The received packets may not contain data. Protocol processing 303 passes the extracted information to command processing 305. Data can be transferred to the host via a buffer controlled by buffer control 306 or without using a buffer.

[0085] Protocol processing 303 includes the information obtained from command processing 305 and the data sent from the host in a packet sent to the network. The data can be transferred to the network either through a buffer controlled by buffer control 306 or without going through the buffer. The sent packet may not contain any data.

[0086] Command processing 305 executes command processing in cooperation with host queue control 304, buffer control 306, and DMA control 307. Command processing 305 controls the host interface 207 via the host interface driver 302. Command processing 305 exchanges send or receive commands for network communication with the host system, such as information for generating iSCSI or NVMe-oF PDUs, analyzed information, or the PDUs themselves. Commands are exchanged with the host system using a queue interface. Host queue control 304 controls this queue interface.

[0087] Command processing 305 generates a descriptor that can be processed by the host based on the result of protocol processing 303, and stores it in a queue via host queue control 304. It also obtains the descriptor generated by the host system from the queue and sets and uses each function in network interface 104, such as protocol processing 303. Note that the queue descriptor can be exchanged between the host system and network interface 104 using DMA controller 205 via DMA control 307.

[0088] 5 is a diagram illustrating an overview of the process in which the network interface 104 transmits and receives commands and data to and from a host, for example, the storage controller 102. Below, commands exchanged with the host are also referred to as host commands.

[0089] The network interface 104 receives a packet 351 from the network and assigns its processing to one of the cores 241. The assigned core 241 executes the protocol processing and extracts information and data. In the example of Figure 5, the extracted data is stored in a buffer 381. The core 241 generates a host command 355 to be sent to the host from the information extracted from the received packet 351.

[0090] Host commands are stored in an internal transmit queue 371. An internal transmit queue 371 is provided for each core 241, and in Fig. 5, one internal transmit queue is indicated by the reference numeral 371 as an example. In the example of Fig. 5, the internal transmit queue 371 is a ring buffer (FIFO), and new commands are stored in its tail area. The head and tail of each internal transmit queue 371 are managed.

[0091] The network interface 104 transfers the host command retrieved from the Head of the internal send queue 371 to the host send queue 401 of the host 400. The host 400 is, for example, the storage controller 102 or the server system 100. The host send queue 401 is a ring buffer in the example of Figure 5. The network interface 104 manages the Head and Tail of the host send queue 401 together with the host 400.

[0092] 5, host commands from multiple internal transmit queues 371 are transferred to one host transmit queue 401. In another example, multiple host transmit queues 401 may be provided, and for example, the same number of host transmit queues 401 may be used corresponding to each internal transmit queue 371.

[0093] The host 400 stores host commands to be transferred to the network interface 104 in a host receive queue 402. The host receive queue 402 is a ring buffer. The network interface 104, together with the host 400, manages the head and tail of the host receive queue 402.

[0094] The network interface 104 transfers commands from the host receive queue 402 to its internal receive queue 372. An internal receive queue 372 is provided for each core 241. In FIG. 5, the internal receive queue 372 checks the host command transferred from the host and selects one corresponding internal receive queue for transfer. At this time, the host command may be checked by the host queue control 304 or by DMA hardware. One internal receive queue is indicated by the reference numeral 372, for example. Multiple host receive queues 402 may be provided; for example, the same number of host receive queues 402 may be used as those corresponding to each internal receive queue 372, so that the process of allocating commands to the internal receive queues 372 is unnecessary.

[0095] The network interface 104 transfers data stored in the buffer 381 to the host 400 in response to a host command from the host 400, and also transfers data 354 stored in the memory of the host 400 to the buffer 382. As will be described later, the buffers 381 and 382 are controlled to improve the cache hit rate.

[0096] Upon receiving a host command 356 to transmit a packet containing data 354 to the network, the network interface 104 performs protocol processing of the data 354 in accordance with the host command 356 to generate a transmit packet 352. The network interface 104 transmits the transmit packet 352 to the network.

[0097] 6A and 6B are diagrams illustrating several aspects of data transfer. Fig. 6A shows several examples of data transfer using the cache 206 / memory 208 of the network interface 104, and Fig. 6B shows several examples of data transfer without using the cache 206 / memory 208. The data transfer is performed using the DMA controller 205. The network interface 104 may support all of the data transfers shown in Figs. 6A and 6B, or may support only some of the data transfers, for example, only the data transfer shown in Fig. 6A.

[0098] FIG. 6A shows a data transfer via the cache / memory of the network interface 104. For example, this corresponds to data transfer via iSCSI or NVMe TCP. In FIG. 6A, data transfer 501 is executed between the host 400 and the cache 206 / memory 208 of the network interface 104. No error check or guarantee code is assigned or removed during data transfer 501. Data transfer 501 corresponds to, for example, update data for a program running on the network interface 104 or the transfer of a descriptor transferred via a queue. Data transfer 511 is executed between the host 400 and the remote site 450 via the cache 206 / memory 208 of the network interface 104. No error check or guarantee code is assigned or removed during data transfer 511.

[0099] Data transfer 512 is performed from the remote site 450 to the host 400 via the cache 206 / memory 208 of the network interface 104. The DMA controller 205 adds a DIF (Data Integrity Field) to the data from the remote site 450 and transfers it to the host 400. Information for generating the DIF is obtained from the host 400.

[0100] Data transfer 513 is executed from host 400 to remote site 450 via cache 206 / memory 208 of network interface 104. DMA controller 205 checks the data by referencing the DIF attached to the data from host 400, and removes the DIF from the data. The data from which the DIF has been removed is transferred to remote site 450.

[0101] Data transfer 514 is performed between the host 400 and the remote site 450 via the cache 206 / memory 208 of the network interface 104. A DIF is attached to the data to be transferred. The DMA controller 205 performs an error check by referencing the DIF attached to the data.

[0102] Data transfer 515 is performed between host 400 and remote site 450 via cache 206 / memory 208 of network interface 104. A DIF is attached to the data to be transferred. DMA controller 205 performs an error check by referencing the DIF attached to the data, removes the DIF, and attaches a new DIF.

[0103] 6B shows data transfer that does not go through the cache / memory of the network interface 104. For example, this corresponds to RDMA (Remote Direct Memory Access).

[0104] Data transfer 521 is executed between the host 400 and the remote site 450. No error check or addition or removal of a guarantee code is performed in data transfer 521. Data transfer 522 is executed from the remote site 450 to the host 400. The DMA controller 205 adds a DIF (Data Integrity Field) to the data from the remote site 450 and transfers it to the host 400.

[0105] Data transfer 523 is executed from host 400 to remote site 450. DMA controller 205 checks the data by referencing the DIF attached to the data from host 400 and removes the DIF from the data. The data from which the DIF has been removed is transferred to remote site 450.

[0106] Data transfer 524 is performed between host 400 and remote site 450. A DIF is attached to the data to be transferred. DMA controller 205 performs an error check by referencing the DIF attached to the data. Data transfer 525 is performed between host 400 and remote site 450. A DIF is attached to the data to be transferred. DMA controller 205 performs an error check by referencing the DIF attached to the data, removes the DIF, and attaches a new DIF.

[0107] 7 shows an example of the configuration of a buffer list. The network interface 104 creates and manages buffer groups of different sizes in the memory 208. Each buffer group is made up of multiple buffers. The different sizes are set according to the supported data transfer size. For example, buffers of sizes such as 8 KB, 16 KB, 32 KB, and 256 KB are reserved when the network interface 104 is started.

[0108] Each buffer group is managed by a corresponding buffer list. Figure 7 shows, as an example, a buffer list 700A for a buffer group of size X, a buffer list 700B for a buffer group of size Y, and a buffer list 700C for a buffer group of size Z. The buffer lists are managed by the buffer control 306.

[0109] Each buffer list contains multiple buffer entries 703. In FIG. 7, one buffer entry is indicated by the symbol 703 as an example. The buffer entry 703 is represented by a rectangle, and the number inside indicates the entry number. For example, a buffer list with a size of 8 KB manages 8 KB buffers, which is the number of buffer entries. The buffer entry 703 indicates information about the corresponding buffer.

[0110] 8A shows buffer list management information 750 included in each buffer list. The buffer list management information shows a buffer entry head address 751 and a buffer active count 752. The buffer entry head address 751 indicates the address of the first entry in the buffer list. The buffer active count 752 indicates the number of buffer entries currently in use.

[0111] 8B shows information held by the buffer entry 703. The buffer entry 703 shows a buffer entry address 761, a buffer address 762, and a next buffer entry address 763. The buffer entry address 761 shows the address of the buffer entry. The buffer address 762 shows the address of the buffer entry managed by the buffer entry 703. The next buffer entry address 763 indicates the address of the buffer entry next to the buffer entry 703 in question.

[0112] The buffer entry 703 may manage whether the buffer is active or free in a field not shown. The buffer entry head address 751 indicates the first entry of the free buffer. The buffer entries of the free buffer are ordered according to the next buffer entry address 763. In this way, in the buffer list, a buffer entry is selected starting from the first buffer entry to store new data. In this way, the allocation order of free buffers is managed.

[0113] When the first buffer entry is selected, the buffer entry head address 751 is updated. Therefore, the entry of the in-use buffer is removed from the list of free buffer entries. As will be described later, a buffer entry that has been used in a data transfer and then released is returned to a predetermined position in the list of free buffer entries, the first position in the example described later.

[0114] The buffer list of the above multiple buffer sizes reduces the buffer allocation load during I / O, and by using buffers for short data lengths and long data lengths, the amount of memory used can be reduced when many short data length transfer requests are received, and the number of buffers used can be reduced when long data length transfer requests are received, thereby reducing the load of looking up the list.

[0115] 9 shows an example of the configuration of a buffer management table 900. The buffer management table 900 indicates information for managing buffers, and is generated by the storage controller 102. The buffer management table 900 allows the buffer group to be dynamically changed. Specifically, the buffer management table 900 indicates a buffer list 771, a buffer size 772, a number of buffers 773, a maximum active number 774, and a control type 775.

[0116] The buffer list 771 indicates the identifier of the buffer list. The buffer size 772 indicates the buffer size of the buffer group managed by the buffer list. The number of buffers 773 indicates the number of buffers managed by the buffer list, that is, the number of buffer entries. The number of buffers 773 indicates the number of buffers secured at initialization. The maximum active number 774 indicates the maximum number of buffers that can be used simultaneously in the buffer group managed by the buffer list. By changing the maximum active number, the maximum number of buffers that can be used simultaneously is dynamically controlled. The control type 775 indicates the buffer control type. The buffer control method is switched depending on the control type. For example, the method of calculating the allowable active number of buffers can be switched between performing the calculation for each buffer list, or taking into account the active numbers of all buffer lists. Furthermore, when transferring data based on the buffer size of the buffer list, it is possible to switch between using multiple buffers or reserving new space.

[0117] 10 shows an example of a processing flow of the target network interface 104. The network interface 104 executes processing for a communication packet from the network to the host of the network interface 104. For example, the host is the target storage controller 102, and the communication packet is transferred from the initiator server system 100 via the network 106.

[0118] The network controller 203 receives a communication packet from the network (S101). The network controller 203 distributes the protocol processing of the received communication packet to a core of the processor 204 (S102).

[0119] The core assigned with the protocol processing of the communication packet executes the protocol processing 303. The protocol processing 303 extracts information from the communication packet and passes it to the command processing 305 (S103).

[0120] The command processing 305 generates a host command for notifying the host of the communication protocol processing result according to the information extracted from the communication packet (S104). The notification of the processing result, for example, notifies the storage controller 102 that a read request or a write request has been received from the network. Furthermore, the command processing 305 loads the generated host command into the internal send queue 371 in the network interface 104 (S105). In response to the loading of the host command, the command processing 305 updates the Tail of the internal send queue 371 (S106).

[0121] Upon receiving the Tail update of the internal transmit queue 371 of the core it is responsible for, the host queue control 304 transfers the host command in the internal transmit queue 371 to the host transmit queue via the DMA control 307 and the DMA controller 205 (S107). In response to the transfer of the host command, the host queue control 304 updates the Tail of the host transmit queue 401 (S108). The network interface 104 and the host may refer to the same information for the Tail and Head information of the host transmit queue 401, or may transfer information held by each. Furthermore, the host queue control 304 updates the Head of the internal transmit queue (S109).

[0122] The host acquires a host command from the host transmission queue 401 and updates the Head of the host transmission queue (S110). The network interface 104 may receive and update the Head of the host transmission queue in accordance with the update on the host side, or may receive notification from the host at a different timing. The host references the acquired host command and determines whether it is necessary to send the command to the network interface 104 (S111).

[0123] If the host needs to send a host command to the network interface 104 (S111: YES), the host loads the host command into the host receive queue 402 (S112) and updates the Tail of the host receive queue 402 (S113). In response to the host updating the information in the host receive queue 402, the network interface 104 also updates the information in its host receive queue 402. For this update, the host and the network interface 104 may refer to common information, or the updated information may be transferred between them.

[0124] Upon receiving the Tail update of the host receive queue 402, the host queue control 304 of one of the cores uses the DMA controller 205 via the DMA control 307 to transfer the host command stacked in the host receive queue 402 to the network interface 104 (S114). For example, the cores may periodically obtain a lock to check whether the host receive queue has been updated, and if there is an update, perform a transfer process. Alternatively, a core may be provided that is dedicated to checking for and transferring updates to the host receive queue. Alternatively, multiple host receive queues may be created, and if there is an update to the receive queue that the core is responsible for, a transfer process may be performed. The host queue control 304 loads the host command from the host into the internal receive queue 372 (S115). For example, the contents of the host command may be checked to determine which internal receive queue 372 to load it into, or if there are multiple host receive queues, the command may be loaded into the corresponding internal receive queue 372.

[0125] In response to the host command being loaded into the internal receive queue 372, the host queue control 304 updates the Tail of the internal receive queue 372 (S116), and also updates the Head of the host receive queue 402 (S117). The method for updating the Head of the host receive queue 402 is the same as that described for updating the Tail of the host receive queue 402.

[0126] The command processing 305 acquires the host command from the internal receive queue 372 and executes the processing (S118). If data transfer using a buffer is required (S119: YES), the buffer control step S120 is executed. Details of the buffer control step S120 will be described later. The data transfer is a data transfer from the network to the host or a data transfer from the host to the network.

[0127] If the buffer allocation fails (S121: NO), the flow returns to step S118. If the buffer allocation is successful (S121: YES), the command processing 305 transfers data from the host to the buffer using the DMA controller 205 under the control of the DMA control 307, or stores data from the network controller 203 in the buffer and transfers it from the buffer to the host using the DMA controller 205 (S123). In response to the data transfer, the host queue control 304 updates the Head of the internal receive queue 372 (S124).

[0128] Returning to step S119, if data transfer using a buffer is not required (S119: NO), the command processing 305 determines whether or not data transfer is required (S122). If data transfer without using a buffer is required (S122: YES), the DMA controller 205 transfers data between the network controller 203 and the host without using a buffer (S123). If data transfer without using a buffer is not required (S122: NO), the next process is performed without transferring data. The host queue control 304 updates the Head of the internal receive queue 372 (S124).

[0129] Next, the command processing 305 determines whether it is necessary to send a communication packet to the network (S125). If it is not necessary (S125: NO), the flow returns to step S104. If it is necessary (S125: YES), the command processing 305 requests the protocol processing 303, the protocol processing 303 executes protocol processing to generate a communication packet, and the network controller 203 transmits the communication packet to the network (S126).

[0130] 11 shows an example of a processing flow of the initiator network interface 104. For example, the server system 100 that accesses the storage controller 102, or the network interface 104 implemented in the storage controller 102 of the initiator in inter-storage communication, operates according to FIG.

[0131] The host loads a host command into the host receive queue 402 (S151) and updates the Tail of the host receive queue 402 (S152). In response to the host updating the information in its host receive queue 402, the network interface 104 also updates the information in its host receive queue 402. The host and the network interface 104 may refer to common information, or may transfer updated information between them.

[0132] Upon receiving the Tail update of the host receive queue 402, the command processing 305 of one of the cores uses the DMA controller 205 via the DMA control 307 to transfer the host commands stacked in the host receive queue 402 to the network interface 104 (S153). For example, the cores may periodically take each other's locks to check whether there are any updates to the host receive queue and perform transfer processing if there are any updates, or a core may be provided that is dedicated to checking for updates to the host receive queue and transferring them, or multiple host receive queues may be created and transfer processing may be performed if there is an update to the receive queue that each core is responsible for.

[0133] The host queue control 304 queues host commands from the host in the internal receive queue 372. For example, if it is a new session, the internal receive queue 372 to load into may be selected in a round robin manner, or the contents of the host command may be checked to determine which internal receive queue 372 to load into, or if there are multiple host receive queues, the command may be loaded into the corresponding internal receive queue 372.

[0134] In response to the host command being loaded into the internal receive queue 372, the host queue control 304 updates the Tail of the internal receive queue 372 (S155), and also updates the Head of the host receive queue 402 (S156). The method for updating the Head of the host receive queue 402 is the same as that described for updating the Tail of the host receive queue 402.

[0135] The command processing 305 acquires the host command from the internal receive queue 372 and executes the processing (S157). If data transfer using a buffer is required (S158: YES), a buffer control step S159 is executed. Details of the buffer control step S159 will be described later. The data transfer is, for example, a data transfer from the host to the network or a data transfer from the host to the network.

[0136] If the buffer allocation fails (S160: NO), the flow returns to step S157. If the buffer allocation is successful (S160: YES), the command processing 305 uses the DMA controller 205 via the DMA control 307 to transfer the data to the buffer (S123). In response to the data transfer, the host queue control 304 updates the Head of the internal receive queue 372 (S163).

[0137] Returning to step S158, if data transfer using a buffer is not required (S119: NO), command processing 305 determines whether or not data transfer is required (S161). If data transfer without using a buffer is required (S161: YES), DMA controller 205 transfers data between network controller 203 and the host without using a buffer (S123). If data transfer without using a buffer is not required (S61: NO), host queue control 304 updates the Head of internal receive queue 372 (S163).

[0138] Next, the command processing 305 determines whether it is necessary to send a communication packet to the network (S164). If it is not necessary (S164: NO), the flow proceeds to step S166. If it is necessary (S164: YES), the command processing 305 requests the protocol processing 303, the protocol processing 303 executes protocol processing to generate a communication packet, and the network controller 203 transmits the communication packet to the network (S165).

[0139] Next, the command processing 305 generates a host command for notifying the host of the processing result of the host command from the host (S166). The notification of the processing result may, for example, notify the host that a communication packet has been sent to the network. Furthermore, the command processing 305 loads the generated host command into the internal transmission queue 371 in the network interface 104 (S167). In response to the loading of the host command, the command processing 305 updates the Tail of the internal transmission queue 371 (S168).

[0140] Upon receiving the Tail update of the internal transmit queue 371, the host queue control 304 transfers the host command in the internal transmit queue 371 to the host transmit queue via the DMA control 307 and the DMA controller 205 (S169). In response to the transfer of the host data, the host queue control 304 updates the Tail of the host transmit queue 401 (S170). The information on the Tail and Head of the host transmit queue 401 is stored in the same file as the network interface 104 and the host. The information may be referenced, or the information held by each may be transferred. Furthermore, the host queue control 304 updates the Head of the internal transmission queue (S171).

[0141] The host acquires the host command from the host transmission queue 401 and updates the Head of the host transmission queue (S172).

[0142] The following describes in detail the buffer control process, including the buffer control steps S120 and S159 in Figures 10 and 11. Figure 12 shows a flowchart of an example of the buffer control process. First, the buffer control 306 acquires a buffer allocation request from, for example, the command processing 305 (S201).

[0143] The buffer control 306 executes a buffer list determination step (S202). The buffer list determination step determines which buffer group to use from among buffer groups of different sizes. The buffer list determination step will be described in detail later.

[0144] Furthermore, the buffer control 306 refers to the buffer list of the selected buffer group and the buffer management table 770, and executes a maximum active number determination step (S203). The maximum active number determination step will be described in detail later. The maximum active number determination step determines whether to execute or wait for the current buffer control. If it is determined in the maximum active number determination step S203 that buffer control is to be executed, the flow proceeds to step S204.

[0145] The buffer control 306 refers to the buffer list determined in the buffer list determination step S202 (S204) and acquires the buffer entry head address 751 of the buffer list (S205). The buffer control 306 accesses the acquired buffer entry head address and acquires the buffer entry 703 (S206).

[0146] The buffer control 306 acquires the next buffer entry address 763 from the acquired buffer entry 703 (S207), and further acquires the buffer address 762 (S208).

[0147] Next, the buffer control 306 reflects the next buffer entry address 763 in the buffer entry head address 751 of the buffer list management information 750 (S209). The buffer control 306 holds the used buffer list and buffer entry as task information related to the data transfer (S210). One or more entries in each of one or more buffer lists can be held as information on a task currently being executed (before completion). The buffer control 306 increments the buffer active number 752 in the buffer list management information 750 (S211).

[0148] A data transfer using a buffer is executed (S212), and when the data transfer is completed (S213), the buffer control 306 identifies the buffer list and buffer entry corresponding to the data transfer from the task information related to the data transfer held in step 210. The buffer control 306 acquires the buffer entry head address 751 of the buffer list (S215).

[0149] The buffer control 306 reflects the acquired buffer entry head address 751 in the next buffer entry address 763 of the buffer entry specified from the task information (S216). The buffer control 306 sets the address 761 of the buffer entry specified from the task information in the buffer entry head address 751 (S217). Finally, the buffer control 306 decrements the buffer active count 752 of the buffer list.

[0150] According to the above operations, the buffer entry for which the data transfer was completed immediately before returns to the head of the buffer list. The next data transfer uses the buffer entry at the head of the buffer list. The cache area stores the data of the associated buffer area. Therefore, the same buffer is reused, and if the data of that buffer was cached, a cache hit occurs. This can reduce the number of memory accesses. Note that the released buffer may be returned to a position different from the head in the buffer list. For example, it is returned before the preset order. In the above example, the head is the preset order.

[0151] Next, the processing of the buffer list determination step S202 will be described. FIG. 13 shows a flowchart of an example of buffer list determination processing. In this example, it is assumed that three different sizes of buffers, X, Y, and Z, are managed by the corresponding buffer lists. The size relationship is as follows: X < Y < Z. For example, X = 8KB, Y = 32KB, and Z = 256KB may be used. Note that the number of types of buffer sizes is arbitrary, and each size is also arbitrary.

[0152] The buffer control 306 acquires the data size to be transferred from the host command (S251). If the data size is less than or equal to X (S252: YES), the buffer control 306 selects the buffer list of size X (S253). If the data size is greater than X and less than or equal to Y (S252: NO, S254: YES), the buffer control 306 selects the buffer list of size Y (S255).

[0153] If the data size is greater than Y and equal to or less than Z (S254: NO, S256: YES), the buffer control 306 selects a buffer list of size Z (S253). If the data size is greater than Z (S256: NO), the buffer control 306 allocates a long-size buffer greater than Z in the memory 208 (S258).

[0154] As described above, by sequentially checking the list of small-sized buffers, the buffer with the smallest buffer size among those that are equal to or larger than the transfer data size is selected to store the transfer data. This allows for efficient use of buffer space. The reason for allocating buffer space on the fly for long-sized data is that with long data sizes, the data transfer time via DMA205 becomes longer, and the processor load during data transfer is low, so allocating buffer space each time is low. Also, long-sized data easily overflows from the cache, resulting in memory access, so list management is not required.

[0155] Next, an example of performing buffer list determination using a method different from that used in buffer list determination step S202 will be described. In the example described with reference to Figures 12 and 13, transfer data is stored in a single buffer. In the example described below, large transfer data is distributed and stored in multiple buffers. This allows transfer data larger than the buffer size managed by the buffer list to be stored in the buffer managed by the buffer list.

[0156] Figure 14 is a flowchart of an example of buffer control processing. Differences from the flowchart shown in Figure 12 will be mainly described. The buffer control 306 executes buffer list determination step S271. The buffer list determination step S271 differs from the buffer list determination step S202 in Figure 12. The buffer control 306 executes step S273 after step S211, and executes step S275 after step S218.

[0157] First, the buffer list determination step S271 will be described. 10 shows a flowchart of a processing example in the determination step S271. The buffer control 306 acquires the remaining data size of the data to be transferred from the host command (S301). If the remaining data size is X or less (S302: YES), the buffer control 306 selects a buffer list of size X and further reduces the remaining data size by X (S303).

[0158] If the remaining data size is greater than X and equal to or less than Y (S302: NO, S304: YES), the buffer control 306 selects a buffer list of size Y and further reduces the remaining data size by Y (S305). If the remaining data size is greater than Y and equal to or less than Z (S304: NO, S306: YES), the buffer control 306 selects a buffer list of size Z and reduces the remaining data size by Z (S307).

[0159] If the remaining data size is greater than Z (S306: NO), or after any of steps S303, S306, or S307 has been executed, the buffer control 306 determines whether a remaining data size exists (S308). If the total size of the allocated buffers is equal to or greater than the transfer data size, the remaining data size is 0. If a remaining data size exists (S308: YES), the flow returns to step S302. If a remaining data size does not exist (S308: NO), the buffer list determination process ends.

[0160] As described with reference to Figure 15, large-sized transfer data is divided into multiple partial data, and a buffer of a corresponding size is allocated to each partial data. Returning to Figure 14, buffer control 306 executes steps S204 to S211 for each partial data. Step S273 determines whether steps S204 to S211 have been executed for all partial data. After data transfer is complete, buffer control 306 executes steps S215 to S218 for each partial data. Step S275 determines whether steps S215 to S218 have been executed for all partial data.

[0161] The maximum active number determination step S203 process is described below with reference to Figures 12 and 14. Figure 16 shows a flowchart of an example of the maximum active number determination process. In the following, it is assumed that each core 241 is assigned a separate cache threshold and a separate set of buffer lists.

[0162] The buffer control 306 acquires the buffer list determination result (S401) Next, the buffer control 306 acquires the active buffer number 752 from the buffer list management information 750 of each buffer list (S402).

[0163] The buffer control 306 calculates the total number of active buffers 752 and the number of buffers scheduled for use (scheduled active buffer number) for each scheduled buffer list indicated by the buffer list determination result, and compares this total number with the maximum active number 774 (S403). If the scheduled active buffer number for any buffer list exceeds the maximum active number 774 (S403: YES), the buffer control 306 determines to suspend processing and wait for a predetermined time (S404). After this, the flow returns to the buffer list determination steps S202 and S271.

[0164] If the planned active buffer count for each buffer list is equal to or less than the corresponding maximum active count of 774 (S403: NO), the buffer control 306 calculates the total amount of active buffers, including the amount of buffers planned for current use (S405). Specifically, the buffer control 306 calculates the sum of products of the size of the buffer list assigned to the core and the number of active buffers ((X size x X active count) + (Y size x Y active count) + (Z size x Z active count)). The buffer control 306 adds the amount of buffers planned for current use to this sum of products.

[0165] Next, the buffer control 306 compares the total active buffer capacity with the cache threshold assigned to the core (S406). The cache threshold is a value equal to or less than the cache capacity assigned to the core. If the total active buffer capacity is equal to or less than the cache threshold (S406: NO), this maximum active number determination process ends. If the total active buffer capacity is greater than the cache threshold (S406: YES), the buffer control 306 determines to suspend processing and wait (S404). After this, a predetermined time has passed, and the buffer control process is restarted.

[0166] As mentioned above, the network interface 104 controls the command processing that can be started based on the difference between the cache capacity and the total amount of active buffer usage. To prevent the cache from overflowing, the number of data transfers being processed simultaneously is limited, and the number of buffers in use is also limited. This prevents performance degradation due to memory bandwidth pressure by reducing memory accesses through cache hits.

[0167] In the above example, each core is assigned a separate set of buffer lists and a separate cache threshold. Each core may be assigned a separate cache area, or one cache area may be shared, i.e., used, by multiple cores.

[0168] In another example, the cache area and all buffer lists may be shared by all cores, and a cache threshold may be defined for the shared cache area, i.e., the cache threshold may be shared by the cores. In this example configuration, each core may perform the max active count determination process as described with reference to FIG.

[0169] In another example, a cache area may be shared by all cores, a cache threshold may be defined for the shared cache area, and buffer lists may be assigned to each core. In this configuration example, each core calculates the total active buffer size by referring to information on the buffer lists assigned to other cores in addition to the buffer lists assigned to itself.

[0170] In another example configuration, a cache threshold may be set individually for each core, and buffer lists may be shared by the cores. Each core may retain information about the buffers it uses as task information and compare the total amount of buffers it uses with its individual cache threshold.

[0171] In another example configuration, cores may be grouped into multiple groups, and each group may be assigned a cache area. For example, a cache threshold and buffer lists may be assigned for each group. Each core may use the buffer lists and cache threshold for the same group to perform the max active count determination process as described with reference to FIG. 16.

[0172] 16, whether or not to allocate a buffer is determined based on the maximum active number set for each buffer list (S403). In another example, step S403 may be omitted, and whether or not to allocate a buffer may be determined based only on the cache usage amount.

[0173] As described above, the cores may be divided into multiple groups, with different cache areas assigned to each group. In this configuration, the network interface 104 may perform performance balancing among multiple network ports, thereby improving the performance of the network interface 104.

[0174] For example, assume that there are eight cores 0 to 7, with two cores constituting each group. Here, cores 0 and 1 constitute one group, cores 2 and 3 constitute one group, cores 4 and 5 constitute one group, and cores 6 and 7 constitute one group. Assume that a cache area and a buffer list group are assigned to each group. Assume that the network interface 104 has two ports, port 0 and port 1.

[0175] The network interface 104 assigns the cores in each group to different ports. For example, core 0, core 2, core 4, and core 6 are assigned to port 0, and core 1, core 3, core 5, and core 7 are assigned to port 1. For example, in each group, a cache threshold that is half the cache capacity is assigned to each core. This allows the cache amounts used by port 0 and port 1 to be equalized, thereby balancing performance. Note that the cache thresholds of the cores in each group do not need to be the same.

[0176] Alternatively, the network interface 104 may assign cores in a group to the same port. For example, core 0, core 1, core 2, and core 3 are assigned to port 0, and core 4, core 5, core 6, and core 7 are assigned to port 1. In each group, a cache threshold may be set for each port, or a cache threshold set for a cache area may be used for both cores.

[0177] Another example of the maximum active number determination process will be described below. FIG. 17 shows a flowchart of another example of the maximum active number determination process. The following mainly explains the differences from the processing example shown in FIG. 16. This example of the maximum active number determination process is executed together with the buffer list determination process described with reference to FIG. 13. In this processing example, when a new buffer area is secured (step S258 in FIG. 13), data is transferred without determining the remaining cache capacity. If the transfer data size is large, there is a high possibility that the data will overflow from the cache. This example can improve the efficiency of the maximum active number determination process.

[0178] 17, after acquiring the buffer list determination result (S401), the buffer control 306 determines whether a new buffer area has been allocated (S411). If a new buffer area has not been allocated (S411: NO), the buffer control 306 executes step S402 and subsequent steps. These steps are as described with reference to FIG. 16. If a new buffer area has been allocated (S411: YES), the maximum active number determination process is completed.

[0179] Figure 18 shows a flowchart of another example of the maximum active number determination process. The following mainly explains the differences from the processing example shown in Figure 16. In this example, when a data transfer request exceeds the maximum buffer size, the data is transferred without determining the remaining cache capacity. This makes the process more efficient.

[0180] 18, after acquiring the buffer list determination result (S401), the buffer control 306 determines whether the transfer data size exceeds the maximum buffer size (S421). If it does not exceed the maximum buffer size (S421: YES), the buffer control 306 executes step S402 and subsequent steps. These steps are as described with reference to FIG. 16. If the transfer data size exceeds the maximum buffer size (S421: NO), the maximum active number determination process is completed.

[0181] It should be noted that the present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the described configurations. It is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment, and it is also possible to add, delete, or replace part of the configuration of each embodiment with the configuration of another embodiment.

[0182] Furthermore, the above-mentioned components, functions, processing units, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. Furthermore, the above-mentioned components, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the programs, tables, and files that realize each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card or SD card.

[0183] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected.

[0184] [Note] Several aspects of the present disclosure are described below. (1) A network interface, a processor; Memory and a cache that is accessed by the processor before the memory; The processor: a plurality of buffers for storing transfer data are allocated in the memory; managing an allocation order of free buffers among the plurality of buffers; The buffer released after the data transfer is returned to the top position of the allocation order as a free buffer. A network interface used to transfer data from the empty buffer at the head position. (2) The network interface according to (1), The processor reserves a plurality of buffer groups in the memory; each buffer group of the plurality of buffer groups is made up of a plurality of buffers of the same size; the plurality of buffer groups have different buffer sizes; A network interface, wherein when the transfer data size is equal to or smaller than the maximum buffer size of the different buffer sizes, the processor selects a group of buffers with a smallest buffer size among buffer sizes equal to or larger than the transfer data size to store the transfer data. (3) The network interface according to (1), The processor determines whether or not to allocate a buffer for new transfer data based on the amount of buffer in use. (4) (2) A network interface according to the present invention, When the transfer data size is larger than the maximum buffer size, the processor secures a new buffer area equal to or larger than the transfer data size. (5) (2) A network interface according to the present invention, If the transfer data size is greater than the maximum buffer size, the processor: A network interface divides the transfer data and allocates buffers selected from the group of buffers. (6) (2) A network interface according to the present invention, A network interface in which a maximum number of buffers that can be used simultaneously is set in each of the plurality of buffer groups. (7) (2) A network interface according to the present invention, The processor: If the transfer data size is equal to or smaller than the maximum buffer size, determining whether or not a buffer is to be allocated for new transfer data based on the amount of buffer currently in use; If the transfer data size is larger than the maximum buffer size, the network interface skips the determination and reserves a buffer for storing the transfer data. (8) The network interface according to (1), The processor determines whether or not to allocate a buffer for new transfer data based on the amount of buffer in use; A network interface in which a maximum number of buffers that can be used simultaneously is set among the plurality of buffers. (9) A method for controlling a buffer in a network interface, comprising: the network interface includes a processor, a memory, and a cache that is accessed by the processor before the memory; The buffer control method includes the steps of: a plurality of buffers for storing transfer data are allocated in the memory; managing an allocation order of free buffers among the plurality of buffers; The buffer released after the data transfer is returned to the top position of the allocation order as a free buffer. A buffer control method used to transfer data from the empty buffer at the head position. (10) (9) A buffer control method according to the present invention, The processor reserves a plurality of buffer groups in the memory; each buffer group of the plurality of buffer groups is made up of a plurality of buffers of the same size; the plurality of buffer groups have different buffer sizes; The buffer control method is a buffer control method in which, when the transfer data size is equal to or smaller than the maximum buffer size of the different buffer sizes, the processor selects a group of buffers with the smallest buffer size among the buffer sizes equal to or larger than the transfer data size to store the transfer data. (11) (9) A buffer control method according to the present invention, The buffer control method includes determining whether or not to allocate a buffer for new transfer data based on the amount of buffer in use. (12) (10) A buffer control method according to the present invention, The buffer control method further comprises the steps of: when the transfer data size is larger than the maximum buffer size, the processor allocates a new buffer area equal to or larger than the transfer data size. (13) (10) A buffer control method according to the present invention, When the transfer data size is larger than the maximum buffer size, the processor divides the transfer data and allocates a buffer selected from the group of buffers to the divided data. Fa control method. (14) (10) A buffer control method according to the present invention, A buffer control method in which a maximum number of buffers that can be used simultaneously is set in each of the plurality of buffer groups. (15) (10) A buffer control method according to the present invention, When the transfer data size is equal to or smaller than the maximum buffer size, the processor determines whether or not to allocate a buffer for new transfer data based on the amount of buffer in use; The buffer control method further comprises the steps of: when the transfer data size is greater than the maximum buffer size, the processor skips the determination and reserves a buffer for storing the transfer data. (16) (9) A buffer control method according to the present invention, the processor determines whether or not to allocate a buffer for new transfer data based on the amount of buffer in use; A buffer control method, wherein a maximum number of buffers that can be used simultaneously is set among the plurality of buffers. [Explanation of symbols]

[0185] 100 Server system, 101 Storage device, 102 Storage controller, 104 Network interface, 201 Host bus, 200 Network path, 202 Data processing unit, 208 Memory, 208 Memory controller, 203 Network controller, 204 Processor, 205 DMA controller, 206 Cache, 241 Processor core, 301 Network controller driver, 303 Protocol processing, 305 Command processing, 307 DMA control, 304 Host queue control, 306 Buffer control, 307 DMA control, 371 Internal transmit queue, 372 Internal receive queue, 401 Host transmit queue, 402 Host receive queue, 400 Host, 450 Remote site, 700-702 Buffer list, 703 Buffer entry, 750 Buffer list management information, 770 Buffer management table

Claims

1. A network interface, a processor; Memory and a cache that is accessed by the processor before the memory; The processor reserves a plurality of buffer groups in the memory; each buffer group of the plurality of buffer groups is made up of a plurality of buffers of the same size; the plurality of buffer groups have different buffer sizes; A network interface, wherein when the transfer data size of the first transfer data is equal to or smaller than the maximum buffer size of the different buffer sizes, the processor selects a group of buffers with the smallest buffer size among the buffer sizes equal to or larger than the transfer data size to store the first transfer data.

2. 2. The network interface of claim 1, When the transfer data size is larger than the maximum buffer size, the processor secures a new buffer area equal to or larger than the transfer data size.

3. 2. The network interface of claim 1, When the transfer data size is larger than the maximum buffer size, the processor divides the first transfer data and allocates buffers selected from the group of buffers.

4. 2. The network interface of claim 1, A network interface in which a maximum number of buffers that can be used simultaneously is set in each of the plurality of buffer groups.

5. 2. The network interface of claim 1, The processor: If the transfer data size is equal to or smaller than the maximum buffer size, determining whether or not a buffer is to be allocated for new transfer data based on the amount of buffer in use; If the transfer data size is larger than the maximum buffer size, the network interface skips the determination and reserves a buffer for storing the transfer data.

6. A method for controlling a buffer in a network interface, comprising: the network interface includes a processor, a memory, and a cache that is accessed by the processor before the memory; The buffer control method includes: The processor allocates a plurality of buffer groups in the memory; each buffer group of the plurality of buffer groups is made up of a plurality of buffers of the same size; the plurality of buffer groups have different buffer sizes; A buffer control method in which, when the transfer data size of first transfer data is equal to or smaller than the maximum buffer size of the different buffer sizes, the processor selects a group of buffers with the smallest buffer size among buffer sizes equal to or larger than the transfer data size to store the first transfer data.

Images