Server data transfer device, data transfer system, server data transfer method, and program

Abstract

In the present invention, an OS includes: a kernel; and a hardware interrupt processing unit (182) for which the arrival of data from an interface unit can be selected to be by a polling mode or by an interrupt mode. A server internal data transfer device (100) comprises, within the kernel: a packet arrival monitoring unit (110) that starts up a thread for monitoring packet arrival using a polling model; and a transfer processing unit (120) that, if the packet arrival monitoring unit (110) detects the arrival of a packet, notifies an application protocol processing unit (74) that the packet has arrived, without using the kernel protocol stack.

Description

Technical Field

[0001] The present invention relates to an in-server data transfer device, a data transfer system, an in-server data transfer method, and a program.

Background Art

[0002] Based on the progress of virtualization technology such as NFV (Network Functions Virtualization), systems are being constructed and operated for each service. In addition, from the form of constructing a system for each service, the service functions are divided into reusable module units and operated on an independent virtual machine (VM: Virtual Machine or container, etc.) environment, so that it can be used as needed like a component to improve operability. The form called SFC (Service Function Chaining) is becoming the mainstream.

[0003] As a technology for constructing a virtual machine, a hypervisor environment composed of Linux (registered trademark) and KVM (kernel-based virtual machine) is known. In this environment, a Host OS (the OS installed on a physical server is called a Host OS) incorporating a KVM module operates in a memory area different from the user space called the kernel space as a hypervisor. In this environment, a virtual machine operates in the user space, and a Guest OS (the OS installed on the virtual machine is called a Guest OS) operates in the virtual machine.

[0004] Unlike a physical server running a Host OS, a virtual machine running a Guest OS uses all hardware, including network devices (such as Ethernet® card devices), to control registers necessary for interrupt handling from the hardware to the Guest OS and for writing from the Guest OS to the hardware. In such register control, notifications and processes that should be performed by the physical hardware are simulated by software, so performance is generally lower than in a Host OS environment.

[0005] In addressing this performance degradation, there are technologies that reduce hardware imitation and improve communication performance and versatility through a high-speed, unified interface, particularly when communicating from the Guest OS to the Host OS and external processes outside of the virtual machine. One such technology is virtio, a device abstraction technology, or paravirtualization technology, which has already been developed and is incorporated into many general-purpose operating systems, including Linux (registered trademark) and FreeBSD (registered trademark), and is currently in use.

[0006] In virtio, data exchange for data input / output such as console, file input / output, and network communication is defined by queue operations, using queues designed with ring buffers as a transport for unidirectional data transfer. By utilizing the virtio queue specifications and preparing the appropriate number and size of queues for each device at the time of Guest OS startup, communication between the Guest OS and the outside of the virtual machine can be achieved solely through queue operations without performing hardware emulation.

[0007] [Packet forwarding using a polling model (DPDK example)] The method of connecting and coordinating multiple virtual machines is called Inter-VM Communication, and in large-scale environments such as data centers, virtual switches have been the standard for connecting VMs. However, because this method has significant communication latency, faster methods have been proposed. For example, methods using special hardware called SR-IOV (Single Root I / O Virtualization) and software-based methods using Intel DPDK (Intel Data Plane Development Kit) (hereinafter referred to as DPDK), a high-speed packet processing library, have been proposed (Non-Patent Literature 1).

[0008] DPDK is a framework for controlling NICs (Network Interface Cards) in user space, a task traditionally handled by the Linux kernel. The biggest difference from the Linux kernel's processing is its polling-based receiving mechanism called PMD (Pull Mode Driver). Normally, the Linux kernel triggers an interrupt upon data arrival at the NIC, which then initiates the receiving process. In contrast, PMD uses a dedicated thread to continuously check for data arrival and perform receiving processing. By eliminating overhead such as context switching and interrupts, it enables high-speed packet processing. DPDK significantly improves packet processing performance and throughput, freeing up more time for data plane application processing.

[0009] DPDK exclusively uses computer resources such as the CPU (Central Processing Unit) and NIC. Therefore, it is difficult to apply to applications that require flexible module-based switching, such as SFCs. SPP (Soft Patch Panel) is an application that mitigates this limitation. SPP eliminates packet copying at the virtualization layer by providing shared memory between VMs, allowing each VM to directly access the same memory space. Furthermore, it uses DPDK to accelerate packet exchange between the physical NIC and shared memory. SPP can software-change the input and output destinations of packets by controlling the memory exchange references of each VM. Through this process, SPP enables dynamic connection switching between VMs and between VMs and physical NICs.

[0010] Figure 10 illustrates packet forwarding using a polling model in the OvS-DPDK (Open vSwitch with DPDK) configuration. As shown in Figure 10, Host OS20 includes OvS-DPDK70, which is software for packet processing. OvS-DPDK70 has a function unit called vhost-user71 for connecting to a virtual machine (VM1 in this case) and a function unit called dpdk(PMD)72 for connecting to NIC(DPDK)13 (physical NIC). Furthermore, the packet processing APL1A includes dpdk(PMD)2, which is a function unit that performs polling in the Guest OS50 section. In other words, packet processing APL1A is an APL modified from packet processing APL1 shown in Figure 10 by adding dpdk(PMD)2.

[0011] The polling model for packet forwarding, as an extension of DPDK, enables GUI-based routing in SPP, which performs high-speed packet copying between Host OS20 and Guest OS50 via shared memory using zero-copy functionality.

[0012] [Rx-side packet processing using the New API (NAPI)] Figure 11 is a schematic diagram of Rx-side packet processing using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6. As shown in Figure 11, the New API (NAPI) executes a packet processing APL1 located in the user space 60 accessible to the user on a server equipped with OS 70 (e.g., Host OS), and performs packet forwarding between the NIC 13 of HW 10 connected to OS 70 and the packet processing APL1.

[0013] OS70 includes kernel71, ring buffer72, and driver73, and kernel71 includes protocol processing unit74. Kernel71 is a core function of OS70 (for example, the Host OS), and it monitors hardware and manages the execution status of programs on a process basis. Here, kernel71 responds to requests from packet processing APL1 and also transmits requests from HW10 to packet processing APL1. Kernel71 processes requests from packet processing APL1 via system calls (a user program running in unprivileged mode requests processing from the kernel running in privileged mode). Kernel71 transmits packets to packet processing APL1 via Socket75. Kernel71 receives packets from packet processing APL1 via Socket75.

[0014] The ring buffer 72 is managed by Kernel 71 and resides in the server's memory space. It is a fixed-size buffer that stores messages output by Kernel 71 as a log, and when it exceeds its upper limit, it is overwritten from the beginning.

[0015] Driver73 is a device driver used in kernel71 to monitor hardware. Note that Driver73 depends on kernel71, and if the kernel source code used to create it changes, it will become a different driver. In this case, you will need to obtain the relevant driver source code, rebuild it on the OS that will use the driver, and create the driver from there.

[0016] The protocol processing unit 74 performs protocol processing for L2 (Data Link Layer), L3 (Network Layer), and L4 (Transport Layer) as defined by the OSI (Open Systems Interconnection) reference model.

[0017] Socket75 is the interface that kernel71 uses for inter-process communication. Socket75 has a socket buffer and does not frequently perform data copying operations. The process for establishing communication via Socket75 is as follows: 1. The server creates a socket file to accept clients. 2. The server names the acceptance socket file. 3. The server creates a socket queue. 4. The server accepts the first connection from a client in the socket queue. 5. The client creates a socket file. 6. The client sends a connection request to the server. 7. The server creates a connection socket file in addition to the acceptance socket file. As a result of establishing communication, packet processing APL1 can call system calls such as read() and write() on kernel71.

[0018] In the above configuration, Kernel71 receives notification of packet arrival from NIC13 via a hardware interrupt (hardIRQ) and schedules a software interrupt (softIRQ) for packet processing. The New API (NAPI), implemented in Linux kernel 2.5 / 2.6 and above, processes packets upon arrival using a hardware interrupt (hardIRQ) followed by a software interrupt (softIRQ). As shown in Figure 11, packet forwarding using the interrupt model involves waiting for interrupt processing (see symbol c in Figure 11), which increases the delay in packet forwarding.

[0019] The following describes the overview of packet processing on the NAPI Rx side. [Rx-side packet processing configuration using New API (NAPI)] Figure 12 is a diagram illustrating the overview of Rx-side packet processing by the New API (NAPI) in the area enclosed by the dashed line in Figure 11. <Device driver> As shown in Figure 12, the Device driver includes NIC13 (physical NIC), which is a network interface card; hardIRQ81, which is a handler that is called when a processing request occurs in NIC13 and executes the requested processing (hardware interrupt); and netif_rx82, which is a software interrupt processing function unit.

[0020] <Networking layer> The Networking layer contains softIRQ83, a handler that is called when a processing request occurs in netif_rx82 and executes the requested processing (software interrupt), and do_softirq84, a control function unit that implements the software interrupt (softIRQ). It also contains net_rx_action85, a packet processing function unit that receives and executes software interrupts (softIRQ), poll_list86, which registers net device (net_device) information indicating which device the hardware interrupt from NIC13 is from, netif_receive_skb87, which creates the sk_buff structure (a structure that allows Kernel71 to perceive the status of packets), and Ring buffer72.

[0021] <Protocol layer> The protocol layer contains packet processing functions such as ip_rcv88 and arp_rcv89.

[0022] The above netif_rx82, do_softirq84, net_rx_action85, netif_receive_skb87, ip_rcv88, and arp_rcv89 are program components (function names) used for packet processing within Kernel 71.

[0023] [Rx-side packet processing operation using the New API (NAPI)] The arrows (symbols) d~o in Figure 12 indicate the flow of Rx-side packet processing. When the hardware function unit 13a of NIC13 (hereinafter referred to as NIC13) receives a packet (or frame) within a frame from the opposing device, it copies the arriving packet to the Ring buffer 72 via DMA (Direct Memory Access) transfer without using the CPU (see symbol d in Figure 12). This Ring buffer 72 is a memory space within the server and is managed by Kernel 71 (see Figure 11).

[0024] However, if NIC 13 only copies the arrived packets to Ring buffer 72, Kernel 71 cannot recognize the packets. Therefore, when a packet arrives, NIC 13 raises a hardware interrupt (hardIRQ) to hardIRQ 81 (see reference e in FIG. 12), and by executing the following processing by netif_rx82, Kernel 71 recognizes the packet. Note that hardIRQ 81 shown surrounded by an ellipse in FIG. 12 represents a handler, not a functional unit.

[0025] netif_rx82 is a function that actually performs processing. When hardIRQ 81 (handler) is raised (see reference f in FIG. 12), netif_rx82 saves information of a net_device indicating which device the hardware interrupt from NIC 13, which is one of the information of the content of the hardware interrupt (hardIRQ), belongs to, to poll_list 86, and registers queue trimming (refer to the content of the packets stored in the buffer, and delete the corresponding queue entry from the buffer in consideration of the subsequent processing to process the packet) (see reference g in FIG. 12). Specifically, upon receiving that the packets are stuffed into Ring buffer 72, netif_rx82 uses the driver of NIC 13 to register subsequent queue trimming to poll_list 86 (see reference g in FIG. 12). As a result, queue trimming information due to the packets being stuffed into Ring buffer 72 is registered in poll_list 86.

[0026] In this way, in <Device driver> of FIG. 12, when NIC 13 receives a packet, it copies the packet that has arrived at ring buffer 72 by DMA transfer. Also, NIC 13 raises hardIRQ 81 (handler), netif_rx82 registers net_device to poll_list 86, and schedules a software interrupt (softIRQ). Up to this point, the processing of the hardware interrupt in <Device driver> of FIG. 12 stops.

[0027] Subsequently, netif_rx82 uses the information (specifically pointers) in the queue stored in poll_list86 to raise a software interrupt (softIRQ) to softIRQ83 (handler) (see code h in Figure 12) to pluck the data stored in ring buffer72, and notifies do_softirq84, the software interrupt control unit (see code i in Figure 12).

[0028] do_softirq84 is the software interrupt control unit, and it defines various software interrupt functions (there are various types of packet processing, and interrupt processing is one of them; it defines interrupt processing). Based on this definition, do_softirq84 notifies net_rx_action85, which actually performs the software interrupt processing, of the request for the current (relevant) software interrupt (see symbol j in Figure 12).

[0029] When it's the turn of the softIRQ, net_rx_action85 calls a polling routine to pluck packets from the ring buffer72 based on the net_device registered in poll_list86 (see code k in Figure 12), and plucks packets (see code l in Figure 12). At this time, net_rx_action85 continues plucking until poll_list86 is empty. Subsequently, net_rx_action85 notifies netif_receive_skb87 (see symbol m in Figure 12).

[0030] netif_receive_skb87 creates an sk_buff structure, analyzes the contents of the packet, and passes the processing to the subsequent protocol processing unit 74 (see Figure 11) according to the type. In other words, netif_receive_skb87 analyzes the contents of the packet and, when processing according to the contents of the packet,<Protocol layer> The process is then passed to ip_rcv88 (symbol n in Figure 12), and if it is L2, for example, the process is passed to arp_rcv89 (symbol o in Figure 12).

[0031] Figure 13 shows an example of video (30 FPS) data transfer. The workload shown in Figure 13 has a transfer rate of 350 Mbps and performs intermittent data transfer every 30 ms.

[0032] Figure 14 shows the CPU usage rate used by the polling thread. As shown in Figure 14, the polling thread occupies a CPU core. Even with intermittent packet reception as shown in Figure 13, the CPU is always used regardless of whether packets arrive or not, which leads to the problem of high power consumption.

[0033] Next, I will explain the DPDK system. [DPDK System Configuration] Figure 15 shows the configuration of the DPDK system that controls the HW10 equipped with the accelerator 12. The DPDK system consists of HW10, OS14, DPDK15 (a high-speed data transfer middleware) and packet processing APL1, all located on user space 60. Packet processing APL1 is packet processing that takes place prior to the execution of APL. HW10 communicates with packet processing APL1 for sending and receiving data. In the following description, as shown in Figure 15, the data flow in which packet processing APL1 receives packets from HW10 is referred to as Rx-side reception, and the data flow in which packet processing APL1 sends packets to HW10 is referred to as Tx-side transmission.

[0034] HW10 includes an accelerator 12 and a NIC 13 (physical NIC) for connecting to a communication network. Accelerator 12 is a computing unit hardware that performs specific calculations at high speed based on input from the CPU. Specifically, accelerator 12 is a PLD (Programmable Logic Device) such as a GPU (Graphics Processing Unit) or FPGA (Field Programmable Gate Array). In Figure 15, accelerator 12 includes multiple Cores (Core processors) 12-1, an Rx queue (queue) 12-2 that holds data in a first-in, first-out list structure, and a Tx queue 133.

[0035] Accelerator 12 offloads part of the packet processing APL1's processing, achieving performance and power efficiency that cannot be reached with software (CPU processing) alone. In large-scale server clusters, such as data centers that utilize NFV (Network Functions Virtualization) or SDN (Software Defined Networking), it is conceivable that accelerator 12, as described above, would be applied.

[0036] NIC13 is NIC hardware that implements a network interface and includes Rx queue 131 and Tx queue 132 that hold data in a first-in, first-out list structure. NIC13 is connected to the opposite device 17, for example, via a communication network, and performs packet transmission and reception. Note that NIC13 may be, for example, a SmartNIC, which is a NIC with an accelerator. A SmartNIC is a NIC that can reduce the CPU load by offloading resource-intensive processes, such as IP packet processing, which can cause a decrease in processing power.

[0037] DPDK15 is a framework for controlling NICs in user space 60, and specifically consists of high-speed data transfer middleware. DPDK15 has a polling-based receiving mechanism called PMD (Poll Mode Driver) 16 (a driver that can select between polling mode and interrupt mode for data arrival). PMD16 continuously performs data arrival confirmation and reception processing using a dedicated thread.

[0038] DPDK15 implements packet processing functionality in user space60 where APL operates, and enables reduced packet forwarding delay by immediately harvesting packets upon arrival from user space60 using a polling model. In other words, DPDK15 harvests packets using polling (busy polling of the queue by the CPU), resulting in no waiting and low latency. [Prior art documents] [Non-patent literature]

[0039] [Non-Patent Document 1] Resource Configuration (NIC) ~DPDK Introduction, Part 6, [online], [Accessed July 6, 2022], Internet <https: / / www.ntt-tx.co.jp / column / dpdk_blog / 190610 / > [Overview of the Initiative] [Problems that the invention aims to solve]

[0040] However, both the interrupt model and the polling model for packet forwarding have the following challenges. In the interrupt model, when the kernel that has received an event (hardware interrupt) from the HW performs packet processing, packet transfer is carried out by software interrupt processing. Therefore, since the interrupt model performs packet transfer through interrupt (software interrupt) processing, there are problems such as competition with other interrupts and waiting occurring when the interrupt destination CPU is being used by a process with a higher priority, resulting in a large delay in packet transfer. In this case, when interrupt processing becomes congested, the waiting delay becomes even larger.

[0041] Supplement the mechanism by which delays occur in the interrupt model. In a general kernel, packet transfer processing is transmitted through software interrupt processing after hardware interrupt processing. When a software interrupt for packet transfer processing occurs, under the following conditions (1) to (3), the software interrupt processing cannot be executed immediately. Therefore, mediation is carried out by a scheduler such as ksoftirqd (a kernel thread for each CPU, which is executed when the load of software interrupts becomes high), and the interrupt processing is scheduled, resulting in a waiting on the order of ms. (1) When competing with other hardware interrupt processing (2) When competing with other software interrupt processing [[ID=**]] (3) When other processes or kernel threads (such as migration threads) with high priority or the interrupt destination CPU are being used Under the above conditions, the software interrupt processing cannot be executed immediately.

[0042] Also, regarding packet processing by the New API (NAPI), similarly, as shown in the dashed box p in Figure 12, due to competition in interrupt processing (softIRQ), a NW delay on the order of ms occurs.

[0043] <Problem of kernel thread monopolizing CPU core> When a kernel thread exclusively occupies a CPU core and constantly monitors packet arrivals, there is a problem of high power consumption because it always uses CPU time. Referring to FIGS. 13 and 14, the relationship between the workload and CPU utilization will be described. Even for the intermittent packet reception shown in FIG. 13, since the CPU is always used regardless of whether a packet has arrived or not, as shown in FIG. 14, the CPU utilization rate used by the polling thread becomes 100%, and it exclusively occupies the CPU core. There is a problem of increased power consumption.

[0044] DPDK also has the same problem as described above. <Problems of DPDK> In DPDK, since the kernel thread performs polling (busy polling the queue with the CPU) and exclusively occupies the CPU core, even for the intermittent packet reception shown in FIG. 13, in DPDK, regardless of whether a packet has arrived or not, the CPU is always used at 100%, so there is a problem of increased power consumption.

[0045] Thus, since DPDK realizes the polling model in user space and no softIRQ conflict occurs, and KBP realizes the polling model within the kernel and no softIRQ conflict occurs, low-latency packet transfer is possible. However, both DPDK and KBP have a problem of wasting CPU resources for constantly monitoring packet arrivals regardless of whether a packet has arrived or not, resulting in increased power consumption.

[0046] To bypass the kernel protocol stack, it is possible to define the necessary network protocol processing in user space to suit the application. For example, the connection between the RU (Radio Unit) and DU (Distributed Unit) in the RAN (Radio Access Network) of a base station (BBU: Base Band Unit) is often connected via Ethernet (L2), and vDU applications do not need L3 / L4 protocols and may want to omit them.

[0047] However, because there is a polling thread in user space, CPU frequency control is performed from user space to the CPU in accordance with the sleep control of the polling thread. As a result, a state transition occurs between user space and kernel mode, and it takes time for the frequency setting to be reflected. This presents a problem in cases where frequency reflection control on the order of a few microseconds to a few tens of microseconds is required, such as in RAN's Front Haul.

[0048] In light of this background, the present invention was made, and its objective is to avoid the overhead of context switching, enable high-speed setting reflection, and transfer data arriving at the interface unit to the application with low power consumption and low latency. [Means for solving the problem]

[0049] To solve the aforementioned problems, a server-internal data transfer device is provided that transfers data arriving at an interface unit to an application in user space via the OS, wherein the OS has a kernel and a driver that can select between polling mode and interrupt mode for data arrival from the interface unit, and the server-internal data transfer device is characterized by comprising a packet arrival monitoring unit that starts a thread to monitor packet arrival using a polling model within the kernel, and a transfer processing unit that, when the packet arrival monitoring unit detects packet arrival, notifies the protocol processing unit of the application that there is an arriving packet without using the kernel protocol stack. [Effects of the Invention]

[0050] According to the present invention, the overhead of context switching can be avoided, settings can be reflected quickly, and data arriving at the interface can be transferred to the application with low power consumption and low latency. [Brief explanation of the drawing]

[0051] [Figure 1] This is a schematic diagram of a data transfer system according to an embodiment of the present invention. [Figure 2] This is an explanatory diagram illustrating the operation of a data transfer system according to an embodiment of the present invention, which uses a method of pre-sharing a shared memory area between the application and the NIC driver. [Figure 3] This flowchart shows the operation of the NIC and hardware interrupt processing in a data transfer system according to an embodiment of the present invention, using a method in which a shared memory area is pre-distributed between the application and the NIC driver. [Figure 4] This flowchart shows the operation of a polling thread in a data transfer system according to an embodiment of the present invention, using a method that pre-distributes a shared memory area between the application and the NIC driver. [Figure 5]This is an explanatory diagram illustrating the operation of a data transfer system according to an embodiment of the present invention, which uses a method for notifying packet pointer information of the data transfer system. [Figure 6] This flowchart shows the operation of a polling thread using a method for notifying packet pointer information in a data transfer system according to an embodiment of the present invention. [Figure 7] This is a hardware configuration diagram showing an example of a computer that implements the functions of a data transfer device within a server in a data transfer system according to an embodiment of the present invention. [Figure 8] This figure shows a general-purpose Linux kernel (registered trademark) and an interrupt model in a server virtualization environment with a VM configuration for a data transfer system according to an embodiment of the present invention. [Figure 9] This diagram shows the operation of the data arrival monitoring unit of the data transfer unit in the data transfer system according to an embodiment of the present invention. [Figure 10] This diagram illustrates packet forwarding using a polling model in the OvS-DPDK configuration. [Figure 11] This is a schematic diagram of Rx-side packet processing using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6. [Figure 12] Figure 11 shows an overview of Rx-side packet processing by the New API (NAPI) in the area enclosed by the dashed line. [Figure 13] This figure shows an example of video (30 FPS) data transfer. [Figure 14] This figure shows the CPU usage of the polling thread. [Figure 15] This diagram shows the configuration of a DPDK system that controls hardware equipped with an accelerator. [Modes for carrying out the invention]

[0052] The following describes a data transfer system and the like in an embodiment of the present invention (hereinafter referred to as "this embodiment") with reference to the drawings. (Explanation of the principle) [Features of this invention] First, the features of the present invention will be described. When a polling thread exists in user space, a state transition occurs between user space and kernel mode, resulting in a delay before the frequency setting is reflected. This invention aims to shorten the time until the frequency setting is reflected, thereby achieving effective low latency and low power consumption.

[0053] Features <1> A polling thread is created within the kernel to control the CPU operating frequency and CPU idle state in kernel mode. As a result, context switch overhead is avoided, and settings can be applied quickly.

[0054] Features <2> A polling thread is provided within the kernel, but it also has a mechanism to transmit pointer information of packets arriving at user-space applications. As a result, the kernel protocol stack is bypassed, and user-space applications can select and use any protocol they wish.

[0055] [Features of polling thread] Next, I will explain the characteristics of polling threads. The polling thread (server data transfer device 100) has the following characteristics. Features <3> : Low latency The polling thread stops the softIRQ for packet processing, which is the main cause of network delays, and executes the polling thread, which is monitored by the packet arrival monitoring unit 110 (described later) of the server's data transfer device 100 when a packet arrives. Then, when a packet arrives, it processes the packet using the polling model (without softIRQ).

[0056] When a packet arrives, a polling thread is triggered by a hardware interrupt handler, avoiding softIRQ contention and enabling immediate packet forwarding. In other words, by keeping the packet arrival monitoring function on standby and triggering it with a hardware interrupt, it is possible to achieve lower latency than packet forwarding processing using software interrupts such as NAPI.

[0057] Furthermore, when packets arrive during sleep mode, a polling thread is created using a high-priority hardIRQ, thus minimizing the overhead caused by sleep mode.

[0058] Features <4> Energy saving (Part 1) The polling thread (server-internal data transfer device 100) monitors for packet arrivals and allows it to sleep when no packets are arriving. While no packets are arriving, the polling thread sleeps and the CPU frequency is set lower. This helps to suppress the power consumption increase caused by busy polling.

[0059] Features <5> Energy saving (part 2) The CPU frequency / CPU idle control unit 140 (described later) of the server's data transfer device 100 changes the CPU operating frequency and idle settings depending on whether a packet has arrived. Specifically, the CPU frequency / CPU idle control unit 140 lowers the CPU frequency when the server is asleep and increases the CPU frequency (returns the CPU operating frequency to its original value) when the server is started again. In addition, the CPU frequency / CPU idle control unit 140 changes the CPU idle setting to power saving mode when the server is asleep. Power saving is achieved by lowering the CPU operating frequency and changing the CPU idle setting to power saving mode when the server is asleep. In this way, a polling thread is provided within the kernel to control CPU frequency and CPU idle state in kernel mode. Because settings are applied quickly without context switching, it is possible to achieve setting changes on the order of a few microseconds.

[0060] (Embodiment) [Overall structure] The following describes a data transfer system and the like in an embodiment of the present invention (hereinafter referred to as "this embodiment") with reference to the drawings. [overview] Figure 1 is a schematic diagram of a data transfer system according to an embodiment of the present invention. This embodiment is an example of application to Rx-side packet processing using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6 and later. As shown in Figure 1, the data transfer system 1000 executes a packet processing APL1 located in the user space accessible to the user on a server equipped with an OS (e.g., Host OS), and performs packet transfer between the NIC 13 of the HW connected to the OS and the packet processing APL1.

[0061] The data transfer system 1000 comprises a network interface card (NIC) 13 (interface unit), a handler hardIRQ81 which is called when a processing request occurs in the NIC 13 and executes the requested processing (hardware interrupt), a hardware interrupt processing unit 182 which is a hardware interrupt processing function unit, a ring buffer 72, a polling thread (server internal data transfer device 100), and a protocol processing unit 74.

[0062] The ring buffer 72 is managed by the kernel in the server's memory space. The ring buffer 72 is a buffer of a fixed size that stores the location of packets upon arrival, and when the upper limit size is exceeded, it is overwritten from the beginning.

[0063] The protocol processing unit 74 uses Ethernet, IP, TCP / UDP, etc., located in user space. The protocol processing unit 74 performs L2 / L3 / L4 protocol processing as defined by, for example, the OSI reference model.

[0064] There are two methods for distributing pointer information to an application: (1) a method in which a shared memory area is distributed between the application and the NIC driver in advance, and (2) a method in which pointer information is notified in the packet.

[0065] (1) In the case where a shared memory area is pre-distributed between the application and the NIC driver (Figure 2), the protocol processing unit 74 obtains buffer memory address information from the driver through distribution and recognizes the location of the ring buffer 72 on the shared memory 150 (Figures 2 and 5) in advance. The protocol processing unit 74 of APL1 is only notified that a packet has arrived from the polling thread (server data transfer device 100). Therefore, the protocol processing unit 74 can refer to the ring buffer 72 on the shared memory 150 (Figures 2 and 5) (indicated by ll in Figure 2: packet reference) and obtain pointer information to confirm the location where the packet body data (payload) is stored. In this way, by obtaining pointer information, it is possible to locate the packet body.

[0066] (2) In the case of a method that notifies packet pointer information (Figure 5), when the protocol processing unit 74 receives the notification, it obtains the incoming packet based on the pointer information sent by the forwarding processing unit 120 along with the notification. That is, the protocol processing unit 74 uses the polling thread pointer information to retrieve the payload from the shared memory 150 (Figures 2 and 5).

[0067] [Data transfer device 100 within the server] <Arrangement of the in-server data transfer device 100> • Kernel space allocation for polling threads The data transfer system 1000 has a polling thread (server-internal data transfer device 100) located in kernel space. This polling thread (server-internal data transfer device 100) operates within kernel space. The data transfer system 1000 executes a packet processing APL1 located in user space on a server equipped with an OS, and performs packet transfer between the HW's NIC 13 and the packet processing APL1 via a device driver connected to the OS. The device driver includes hardIRQ81, HW interrupt processing unit 182, and ring buffer 72. A device driver is a driver used to monitor hardware.

[0068] This invention can be used when you want to define a custom protocol to use in user space, while also performing polling mode and sleep, and sending and receiving packets with low latency and low power consumption.

[0069] As described above, the server data transfer device 100 is a polling thread located in kernel space. By providing the server data transfer device 100 (polling thread) within the kernel, packet arrival monitoring and reception processing are performed using a polling model, achieving low latency.

[0070] <Configuration of the in-server data transfer device 100> The server data transfer device 100 includes a packet arrival monitoring unit 110, a transfer processing unit 120, a sleep management unit 130, and a CPU frequency / CPU idle control unit 140.

[0071] <Packet arrival monitoring unit 110> The packet arrival monitoring unit 110 is a thread used to monitor whether packets have arrived. The packet arrival monitoring unit 110 starts a thread within the kernel that monitors packet arrivals using a polling model.

[0072] The packet arrival monitoring unit 110 acquires the pointer information indicating the existence of a packet in the ring buffer 72 and the net_device information, and transmits this information (pointer information and net_device information) to the transfer processing unit 120.

[0073] <Transfer processing unit 120> When the transfer processing unit 120 detects the arrival of a packet by the packet arrival monitoring unit 110, it notifies the protocol processing unit 74 of the application that there is an arriving packet without using the kernel protocol stack. Regarding the method of circulating pointer information to the application, there are (1) a method of previously circulating a shared memory area between the application and the NIC driver, and (2) a method of notifying the pointer information of the packet.

[0074] (1) In the case of the method of previously circulating a shared memory area between the application and the NIC driver (Fig. 2), the transfer processing unit 120 only notifies APL1 that a packet has arrived without using the kernel protocol stack based on the packet arrival by the packet arrival monitoring unit 110. That is, the transfer processing unit 120 does not take out the packet from the ring buffer 72 based on the received information and transmit the packet to the protocol processing unit 74, but only notifies that a packet has arrived.

[0075] (2) In the case of the method of notifying the pointer information of the packet (Fig. 5), the transfer processing unit 120 sends the pointer information (notify + pointer information) indicating the storage destination of the arriving packet together with the notification to the protocol processing unit 74.

[0076] <Sleep management unit 130> When a packet does not arrive within a predetermined period, the sleep management unit 130 puts the polling thread to sleep, and when a packet arrives, wakes up the polling thread by a hardware interrupt (hardIRQ) of this thread.

[0077] <CPU Frequency / CPU Idle Control Unit 140> During sleep, the CPU frequency / CPU idle control unit 140 sets the CPU operating frequency of the CPU core used by the polling thread to a low level. During sleep, the CPU frequency / CPU idle control unit 140 sets the CPU idle state of the CPU core used by this polling thread to a power-saving mode.

[0078] The operation of the data transfer system 1000 will be described below. [Rx-side Packet Processing Operation according to the Present Invention] The arrows (reference signs) aa to jj in FIG. 1 indicate the flow of Rx-side packet processing. When the NIC 13 receives a packet (or frame) in the frame from the opposite device, it copies the arrived packet to the Ring buffer 72 without using the CPU by DMA transfer (see reference sign aa in FIG. 1). This Ring buffer 72 is managed by the <Device driver>.

[0079] When a packet arrives, the NIC 13 raises a hardware interrupt (hardIRQ) to the hardIRQ81 (handler) (see reference sign bb in FIG. 1), and the HW interrupt processing unit 182 executes the following processing to recognize the packet.

[0080] When the hardwire81 (handler) rises (see reference sign cc in FIG. 1), the HW interrupt processing unit 182 performs wake-up to call up the sleeping polling thread (see reference sign dd in FIG. 1). Up to this point, Figure 1<Device driver> Hardware interrupt processing in this case will be stopped.

[0081] On the other hand, the CPU frequency / CPU idle control unit 140 lowers the CPU operating frequency of the CPU core used by the thread (polling thread) during sleep. The CPU frequency / CPU idle control unit 140 sends a frequency control signal (control CPU frequency) to the CPU 11 via driver 83 such as ACPI / P-State (see symbol ee in Figure 1) (see symbol ff in Figure 1).

[0082] The packet arrival monitoring unit 110 monitors (polling) the ring buffer 72 (see the symbol gg in Figure 1) to check whether a packet has arrived. The packet arrival monitoring unit 110 stores packets in the Ring buffer 72 in a pre-allocated area, so by referring to the pre-allocated area of ​​the Ring buffer 72, it can determine if a new packet has arrived.

[0083] The packet arrival monitoring unit 110 retrieves packets from the Ring buffer 72 if packets have arrived (see code hh in Figure 1). At this time, if packet pointer information has been transmitted via a hardware interrupt, it may be used (pull packets from Ring buffer). The packet arrival monitoring unit 110 retrieves packets from the ring buffer 72 based on the received information and sends them to the forwarding processing unit 120 (see reference numeral ii in Figure 1). The forwarding processing unit 120 transmits the packets received by the packet arrival monitoring unit 110 to the protocol processing unit 74 (see the symbol jj in Figure 1).

[0084] In this case, the packet arrival monitoring unit 110 and the forwarding processing unit 120 do not use the kernel protocol stack (see dashed box kk in Figure 1), but instead notify the user space of the pointer information of the packet received from NIC 13 (using signalfd, a proprietary API, etc.). In other words, they bypass the kernel protocol stack and notify the user space of the pointer information of the packet received by the polling thread from the NIC. Furthermore, ring buffer 72 is stored and managed via DMA from NIC13 in a format that is easy for APL1 to use (for example, mbuf in the case of DPDK).

[0085] I will explain in more detail. The data transfer system 1000 installs an in-server data transfer device 100 (polling thread) within the kernel, but does not use the kernel protocol stack. Instead, it notifies the user space of the pointer information of packets received from the NIC 13 (using eventfd, signalfd, a proprietary API, etc.). In other words, the in-server data transfer device 100 bypasses the kernel protocol stack and notifies the user space of the pointer information of packets received by the polling thread from the NIC 13. The protocol processing unit 74 only receives the notification of the pointer information of packets received from the polling thread.

[0086] The protocol processing unit 74 of user space APL1 already knows the location of the ring buffer on shared memory 150. When the protocol processing unit 74 receives pointer information for a packet from NIC 13, it uses the notified pointer information to refer to the ring buffer 72 on shared memory 150 and obtain the pointer information to confirm the location where the packet data (payload) is stored. This allows user space applications to select and use the necessary protocols, similar to DPDK.

[0087] [Buffer structure and method of distributing pointer information to applications] This document describes the buffer structure of the server's data transfer device 100 and the method for distributing pointer information to the application. There are two methods for distributing pointer information to an application: (1) a method in which a shared memory area is distributed between the application and the NIC driver in advance, and (2) a method in which pointer information is notified in the packet. These will be explained in order below.

[0088] First, (1) the method of pre-sharing a shared memory area between the application and the NIC driver will be explained with reference to the operation diagram in Figure 2 and the flowcharts in Figures 3 and 4.

[0089] Figure 2 is an explanatory diagram of the operation of a data transfer system that pre-distributes a shared memory area between the application and the NIC driver. The same components as in Figure 1 are denoted by the same reference numerals. As shown in Figure 2, the shared memory 150 on the device driver is composed of hugepages, etc., and includes a packet buffer 151 and a ring buffer 72.

[0090] The device driver manages the pointer information for packet buffer 151. The protocol processing unit 74 of APL1 has already recognized the memory address information of the ring buffer 72 on the shared memory 150, and by referring to the ring buffer 72 (code ll in Figure 2: packet reference) and obtaining pointer information, it can confirm the storage location of the packet body data (payload).

[0091] By pre-allocating a shared memory area such as a hugepage between APL1 and the NIC driver, and by APL1 knowing the memory address information of ring buffer 72 in advance, it becomes possible to check the storage location of the packet body data (payload) by referring to ring buffer 72, even without being notified of packet pointer information from the polling thread.

[0092] Figure 3 is a flowchart illustrating the operation of the NIC and hardware interrupt processing using a method where a shared memory area is pre-distributed between the application and the NIC driver. The operation of this flow is described in the NIC driver. This flow starts when a packet arrives at the NIC.

[0093] In step S1, NIC13 copies the packet data that arrived via DMA to a memory area. At this time, the data format (structure) to be stored is saved in a format that is easy for APL1, which receives the packets, to use. For example, in the case of a DPDK application, it is mbuf, etc. The NIC driver stores pointer information of the memory area where the packets are stored in the ring buffer 72. The packet arrival monitoring unit 110 of the polling thread performs arrival monitoring on this ring buffer 72.

[0094] In step S2, the hardware interrupt processing unit 182 located in the NIC driver determines whether hardware interrupts are permitted. If hardware interrupts are not permitted (S2: No), the processing of this flow is terminated. If hardware interrupts are permitted (S2:Yes), in step S3, the hardware interrupt processing unit 182 invokes a hardware interrupt (hardIRQ81), and if the polling thread is sleeping, it wakes up the corresponding polling thread and terminates the processing of this flow. Because the wake-up is done by a hardware interrupt, there is low latency. At this time, the pointer information of the arrived packet may be transmitted to the polling thread.

[0095] Figure 4 is a flowchart illustrating the operation of a polling thread using a method where a shared memory area is pre-distributed between the application and the NIC driver. The polling thread is woken up by a hardware interrupt, and this flow starts.

[0096] In step S11, the sleep management unit 130 disables hardware interrupts by the relevant NIC.

[0097] In step S12, the CPU frequency / CPU idle control unit 140 sets the CPU operating frequency of the CPU core on which the polling thread is running to a higher value. The CPU frequency / CPU idle control unit 140 also returns the CPU idle state to ACTIVE. Because this process is executed in kernel mode, there is no context switch overhead associated with switching between user mode and kernel mode, allowing for fast implementation.

[0098] In step 13, the packet arrival monitoring unit 110 of the polling thread checks the ring buffer 72 to see if there are any new packets. At this time, if packet pointer information has been transmitted via a hardware interrupt, it may be used.

[0099] In step S14, the packet arrival monitoring unit 110 determines whether or not there are any new packets.

[0100] If there are new packets (S14: Yes), in step S15 the polling thread notifies the protocol processing unit 74 of APL1 in user space that there are new packets and returns to step S13. This notification involves a context switch from kernel mode to user mode.

[0101] Here, the method for notifying and transmitting user space information to applications utilizes mechanisms such as eventfd and signalfd provided by the kernel. Alternatively, a custom API (Application Programming Interface) can be defined.

[0102] Additionally, if there are multiple new packets, they may be notified and transmitted as a list (batch processing). As shown in Figure 2, even without communicating pointer information where the packet is stored to the application, the application knows the address of the pre-allocated shared memory area ring buffer 72, and can determine the location of the packet by referring to that ring buffer 72.

[0103] If no new packets are found (S14: No), in step S16, the polling thread's CPU frequency / CPU idle control unit 140 lowers the CPU operating frequency of the active CPU core. The CPU frequency / CPU idle control unit 140 also sets the CPU idle state to allow it to enter a deep sleep state. Because this process is executed in kernel mode, there is no context switch overhead associated with switching between user mode and kernel mode, allowing for fast implementation.

[0104] In step S17, the sleep management unit 130 allows hardware interrupts by the relevant NIC.

[0105] In step S18, the sleep management unit 130 puts the polling thread to sleep and terminates the processing of this flow.

[0106] Next, (2) the method for notifying packet pointer information will be explained with reference to the operation diagram in Figure 5 and the flowchart in Figure 6.

[0107] Figure 5 is an explanatory diagram of the operation of a data transfer system that uses a method for notifying packet pointer information. The same reference numerals are used for components identical to those in Figure 1. As shown in Figure 5, the shared memory 150 on the device driver is composed of hugepages, etc., and includes a packet buffer 151 and a ring buffer 72.

[0108] The device driver manages the pointer information for packet buffer 151. When the polling thread notifies APL1 of the arrival of a packet, it also notifies APL1 of the packet's pointer information (which may include the memory address information of ring buffer 72). This allows APL1 to determine the location of the packet's data (payload) without having prior knowledge of the memory address information of ring buffer 72 or packet buffer 15172.

[0109] The polling thread notifies APL1 of the packet pointer information, allowing APL1 to determine the location of the packet's data (payload). This method eliminates the need to exchange memory address information for ring buffer 72 between the application and the NIC driver beforehand, thus offering flexibility such as dynamically changing the location of ring buffer 72 and packet buffer 151.

[0110] The flowchart illustrating the operation of the NIC and hardware interrupt processing using a method for notifying packet pointer information is the same as in Figure 3, so its explanation is omitted.

[0111] Figure 6 is a flowchart illustrating the operation of a polling thread using a method that notifies packet pointer information. Steps that perform the same processing as in Figure 4 are denoted by the same reference numerals and their explanations are omitted. If there are new packets in step S14 (S14: Yes), in step S21 the polling thread notifies the protocol processing unit 74 of APL1 in user space that there are new packets, transmits the pointer information of the new packets to the protocol processing unit 74 of APL1 in user space, and returns to step S13. This notification involves a context switch from kernel mode to user mode. If there are multiple new packets, multiple packets may be transmitted as a list (batch processing).

[0112] [Hardware configuration] The server data transfer device 100 (Figures 1, 2, and 5) according to the above embodiment is implemented by a computer 900 having a configuration such as that shown in Figure 7. Figure 7 is a hardware configuration diagram showing an example of a computer 900 that implements the functions of the server data transfer device 100 (Figures 1, 2, and 5). Computer 900 has a CPU 901, ROM 902, RAM 903, HDD 904, communication interface (I / F: Interface) 906, input / output interface (I / F) 905, and media interface (I / F) 907.

[0113] The CPU 901 operates based on programs stored in the ROM 902 or HDD 904 and controls various parts of the server's data transfer device 100 (Figures 1, 2, and 5). The ROM 902 stores boot programs executed by the CPU 901 when the computer 900 starts up, as well as programs that depend on the computer 900's hardware.

[0114] The CPU 901 controls input devices 910, such as a mouse or keyboard, and output devices 911, such as a display, via the input / output interface 905. The CPU 901 acquires data from the input devices 910 via the input / output interface 905 and outputs the generated data to the output devices 911. In addition to the CPU 901, a GPU (Graphics Processing Unit) or the like may also be used as a processor.

[0115] HDD904 stores programs executed by CPU901 and data used by those programs. Communication I / F906 receives data from other devices via a communication network (e.g., NW(Network)920) and outputs it to CPU901, and also transmits data generated by CPU901 to other devices via the communication network.

[0116] The media interface 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads the program related to the desired processing from the recording medium 912 onto the RAM 903 via the media interface 907 and executes the loaded program. The recording medium 912 can be an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto Optical Disk), a magnetic recording medium, a conductive memory tape medium, or a semiconductor memory.

[0117] For example, when the computer 900 functions as the in-server data transfer device 100 (Figs. 1, 2, 5) configured as one device according to the present embodiment, the CPU 901 of the computer 900 realizes the functions of the in-server data transfer device 100 by executing the program loaded on the RAM 903. Also, the HDD 904 stores the data in the RAM 903. The CPU 901 reads and executes the program related to the target process from the recording medium 912. In addition, the CPU 901 may read the program related to the target process from another device via the communication network (NW920).

[0118] [Applicable Example] It can be applied to the configuration example in which the in-server data transfer device 100 is arranged in the OS 50. In this case, the OS is not limited. Also, it is not limited to being in the server virtualization environment. Therefore, the in-server data transfer device 100 (Figs. 1, 2, 5) can be applied to each configuration shown in Figs. 8 and 9.

[0119] <Example of Application to VM Configuration> Fig. 8 is a diagram showing an example in which the data transfer system 1000A is applied to the interrupt model in the server virtualization environment of the general-purpose Linux kernel (registered trademark) and the VM configuration. The same reference numerals are given to the same components as in Fig. 1. As shown in Fig. 8, the data transfer system 1000A includes a Host OS 80 in which a virtual machine and an external process formed outside the virtual machine can operate, and the Host OS 80 has a Kernel 81 and a Driver 82. Also, the data transfer system 1000A has a NIC 71 of the HW 70 connected to the Host OS 80 and a KVM module 91 incorporated in the hypervisor (HV) 90. Furthermore, the data transfer system 1000A includes a Guest OS 95 operating in the virtual machine, and the Guest OS 95 has a Kernel 96 and a Driver 97. And the data transfer system 1000A includes a polling thread (in-server data transfer device 100) in the kernel space.

[0120] By doing so, in a VM virtual server configuration system, data arriving at the interface can be transferred to the application with low power consumption and low latency, regardless of whether the Host OS 80 or Guest OS 95 is the operating system.

[0121] <Example of application to container configuration> Figure 9 shows an example of applying the data transfer system 1000B to an interrupt model in a containerized server virtualization environment. The same reference numerals are used for components identical to those in Figures 1 and 15. As shown in Figure 9, the data transfer system 1000B has a container configuration in which the Guest OS95 in Figure 8 is replaced with Container98. Container98 has a vNIC (virtual NIC).

[0122] In systems with virtual server configurations such as containers, data arriving at the interface can be transferred to the application with low power consumption and low latency.

[0123] <Example of application to bare-metal configuration (non-virtualized configuration)> The present invention can be applied to non-virtualized systems, such as bare-metal configurations. In non-virtualized systems, data arriving at the interface can be transferred to the application with low power consumption and low latency.

[0124] <Scale in / out> When dealing with high traffic volumes and using multiple NIC devices or NIC ports, it's possible to scale in / out polling threads while controlling hardware interrupt frequency by running multiple polling threads associated with these devices.

[0125] <Advanced Technology> When the number of traffic flows increases, the present invention can scale out against network loads by cooperating with RSS (Receive-Side Scaling) that can process inbound network traffic with multiple CPUs and increasing the number of CPUs assigned to the packet arrival monitoring thread.

[0126] <Application to PCI device I / O such as an accelerator> Although NIC (Network Interface Card) I / O has been exemplified, the present technology is also applicable to I / O of PCI devices of accelerators (FPGA / GPU, etc.). In particular, it can be used for polling, etc. when receiving a response of the offloading result to the FEC (Forward Error Correction) accelerator in vRAN.

[0127] <Application to processors other than CPUs> The present invention is also applicable to processors such as GPUs / FPGAs / ASICs (application specific integrated circuits) in addition to CPUs when they have an idle state function.

[0128] [Effect] As described above, the server data transfer device 100 (Figures 1, 2, 5) transfers data arriving at the interface unit (NIC13) (Figures 1, 2, 5) to an application (APL1) (Figures 1, 2, 5) in user space via the OS. The OS has a kernel and a driver (HW interrupt processing unit 182) that can select between polling mode and interrupt mode for data arrival from the interface unit. The server data transfer device 100 includes a packet arrival monitoring unit 110 within the kernel that starts a thread to monitor packet arrival using a polling model, and a transfer processing unit 120 that, when the packet arrival monitoring unit 110 detects packet arrival, notifies (notifies) (code jj in Figures 1, 2, 5) the application's protocol processing unit 74 of the arrival of a packet without using the kernel protocol stack (code kk in Figures 1, 2, 5).

[0129] By doing so, context switching overhead is avoided, settings can be reflected quickly, and data arriving at the interface can be transferred to the application with low power consumption and low latency. Furthermore, similar to DPDK, user-space applications will be able to select and use the protocols they need.

[0130] In the server's internal data transfer device 100 (Figures 1 and 5), there is a buffer (ring buffer 72) (Figures 1 and 5) in the memory space of the server equipped with an OS that stores pointer information indicating the storage location of the incoming packet. The transfer processing unit 120 sends pointer information (notify + pointer information) (code jj in Figure 5) along with notification to the protocol processing unit 74.

[0131] By doing this, there is no need to exchange memory address information for ring buffer 72 between the application and the NIC driver in advance, which has the effect of providing flexibility such as dynamically changing the location of ring buffer 72 and packet buffer 151.

[0132] A data transfer system 1000 (Figures 1, 2, 5) includes a server-internal data transfer device 100 (Figures 1, 2, 5) that transfers data arriving at an interface unit (NIC13) (Figures 1, 2, 5) to an application (APL1) (Figures 1, 2, 5) in user space via the OS. The server-internal data transfer device 100 has a protocol processing unit 74 in user space that performs protocol processing for data to the application, and a shared memory 150 (Figures 2, 5) accessible from the protocol processing unit 74 that has a buffer (ring buffer 72) (Figures 2, 5) indicating the storage location of arriving packets. The server-internal data transfer device 100 has an OS that includes a kernel and a driver (HW interrupt processing unit 182) that can select between polling mode and interrupt mode for data arrival from the interface unit, and a packet arrival monitoring unit 110 within the kernel that starts a thread to monitor packet arrival using a polling model, and when the packet arrival monitoring unit 110 detects packet arrival, it sets up a kernel protocol Without using a stack (code kk in Figures 1, 2, and 5), the system includes a forwarding processing unit 120 that notifies the protocol processing unit 74 (Figures 1, 2, and 5) of the arrival of a packet. The protocol processing unit 74 obtains buffer memory address information from the driver through exchange, and upon receiving a notification (notify) (code jj in Figures 1 and 2), it obtains pointer information by referring to the memory address information of the buffer (ring buffer 72) (Figures 2 and 5), and then obtains the arrival packet (packet buffer 151) (Figure 2) based on that pointer information.

[0133] By doing this, a shared memory area such as a hugepage is allocated in advance between APL1 and the NIC driver, and APL1 knows the memory address information of ring buffer 72 in advance. This makes it possible to check the storage location of the packet data (payload) by referring to ring buffer 72, even without being notified of packet pointer information from the polling thread. As a result, context switch overhead is avoided, settings can be reflected quickly, and data arriving at the interface can be transferred to the application with low power consumption and low latency.

[0134] A data transfer system 1000 (Figures 1, 2, 5) includes a server-internal data transfer device 100 (Figures 1, 2, 5) that transfers data arriving at the interface unit to an application (APL1) (Figures 1, 2, 5) in user space via the OS. The server-internal data transfer device 100 has a protocol processing unit 74 (Figures 1, 2, 5) in user space that performs protocol processing for data to the application. The server-internal data transfer device 100 has a buffer (ring buffer 72) (Figures 2, 5) on a shared memory 150 (Figures 2, 5) accessible from the protocol processing unit 74 that indicates the storage location of arriving packets. The server-internal data transfer device 100 has an OS that includes a kernel and a driver (HW interrupt processing unit 182) that can select between polling mode and interrupt mode for data arrival from the interface unit. Within the kernel, there is a packet arrival monitoring unit 110 that starts a thread to monitor packet arrival using a polling model, and when the packet arrival monitoring unit 110 detects packet arrival, it executes a kernel protocol The system includes a forwarding processing unit 120 that notifies the protocol processing unit 74 of the arrival of a packet without using a stack (code kk in Figures 1, 2, and 5). Along with the notification to the protocol processing unit 74, the forwarding processing unit 120 sends pointer information (notify + pointer information) (code jj in Figure 5) indicating the storage location of the arrival packet. Upon receiving the notification, the protocol processing unit 74 retrieves the arrival packet (packet buffer 151) (Figure 5) based on the pointer information sent from the forwarding processing unit 120.

[0135] In this way, APL1 can locate packets without knowing the locations of ring buffer 72 and packet buffer 151 in advance. Since there is no need to exchange memory address information of the ring buffer between the application and the NIC driver beforehand, it has the effect of providing flexibility, such as dynamically changing the location of ring buffer 72 and packet buffer 151.

[0136] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically may be performed manually, or all or part of the processes described as being performed manually may be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings may be changed at will unless otherwise specified. Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0137] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software that allows the processor to interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or an optical disc. [Explanation of symbols]

[0138] 1. Application (APL) 72 ring buffer 74 Protocol Processing Unit 100 Server-internal data transfer device 110 Packet Arrival Monitoring Unit 120 Transfer Processing Unit 130 sleep management department 140 CPU frequency / CPU idle control section 150 shared memory 151 packet buffer 1000, 1000A, 1000B Data Transfer System

Claims

1. A server-internal data transfer device that transfers data arriving at the interface unit to an application in user space via the OS, The OS The kernel and, The system includes a driver that can select between polling mode or interrupt mode for receiving data from the interface unit. The data transfer device within the server is Within the kernel, there is a packet arrival monitoring unit that launches a thread to monitor packet arrivals using a polling model, The packet arrival monitoring unit detects the arrival of a packet and, without using the kernel protocol stack, provides a forwarding processing unit that notifies the protocol processing unit of the application that a packet has arrived. A server-internal data transfer device characterized by the following features.

2. The server equipped with the aforementioned OS has a memory space containing a buffer that stores pointer information indicating the storage location of incoming packets. The transfer processing unit sends the pointer information along with the notification to the protocol processing unit. The server data transfer device according to feature 1.

3. A data transfer system comprising a server-internal data transfer device that transfers data arriving at the interface unit to an application in user space via the OS, The user space includes a protocol processing unit that performs protocol processing of data to the application, The shared memory accessible from the protocol processing unit has a buffer indicating the storage location of the incoming packets, The aforementioned OS, The kernel and, It has a driver that can select between polling mode or interrupt mode for receiving data from the interface unit. The data transfer device within the server is Within the kernel, there is a packet arrival monitoring unit that launches a thread to monitor packet arrivals using a polling model, The packet arrival monitoring unit detects the arrival of a packet, and the forwarding processing unit notifies the protocol processing unit of the arrival of the packet without using the kernel protocol stack. The protocol processing unit obtains the memory address information of the buffer from the driver. Upon receiving the aforementioned notification, the system obtains pointer information by referring to the memory address information and retrieves the incoming packet based on that pointer information. A data transfer system characterized by the following features.

4. A data transfer system comprising a server-internal data transfer device that transfers data arriving at the interface unit to an application in user space via the OS, The user space includes a protocol processing unit that performs protocol processing of data to the application, The shared memory accessible from the protocol processing unit has a buffer indicating the storage location of the incoming packets, The aforementioned OS, The kernel and, The system includes a driver that can select between polling mode or interrupt mode for receiving data from the interface unit. The data transfer device within the server is Within the kernel, there is a packet arrival monitoring unit that launches a thread to monitor packet arrivals using a polling model, The packet arrival monitoring unit detects the arrival of a packet, and the forwarding processing unit notifies the protocol processing unit of the arrival of the packet without using the kernel protocol stack. The forwarding processing unit, along with the notification to the protocol processing unit, sends pointer information indicating the storage location of the incoming packet. The protocol processing unit, Upon receiving the aforementioned notification, the incoming packet is acquired based on the pointer information sent from the transfer processing unit. A data transfer system characterized by the following features.

5. A server-internal data transfer method for a server-internal data transfer device that transfers data arriving at the interface unit to an application in user space via the OS, The aforementioned OS, The kernel and, The system includes a driver that can select between polling mode or interrupt mode for receiving data from the interface unit. The data transfer device within the server is The kernel includes the step of starting a thread that monitors packet arrivals using a polling model, When packet arrival is detected, the system performs a forwarding process step that notifies the application of the arrival of a packet without using the kernel protocol stack. A method for transferring data within a server, characterized by the following features.

6. The server equipped with the aforementioned OS has a memory space containing a buffer that stores pointer information indicating the storage location of incoming packets. In the transfer processing step, the pointer information is sent along with the notification to the application. The server data transfer method according to feature 5.

7. A server-internal data transfer method for a server-internal data transfer device that transfers data arriving at the interface unit to an application in user space via the OS, The user space includes a protocol processing unit that performs protocol processing of data to the application, The shared memory accessible from the protocol processing unit has a buffer indicating the storage location of the incoming packets, The aforementioned OS, The kernel and, It has a driver that can select between polling mode or interrupt mode for receiving data from the interface unit. The data transfer device within the server is The kernel includes the step of starting a thread that monitors packet arrivals using a polling model, When packet arrival is detected, a forwarding process step is executed that notifies the protocol processing unit of the application of the arrival of a packet without using the kernel protocol stack. The protocol processing unit obtains the memory address information of the buffer from the driver. Upon receiving the aforementioned notification, the system performs the following steps: obtain pointer information by referring to the memory address information, and then acquire the incoming packet based on that pointer information. A method for transferring data within a server, characterized by the following features.

8. A server-internal data transfer method for a server-internal data transfer device that transfers data arriving at the interface unit to an application in user space via the OS, The user space includes a protocol processing unit that performs protocol processing of data to the application, The shared memory accessible from the protocol processing unit has a buffer indicating the storage location of the incoming packets, The aforementioned OS, The kernel and, The system includes a driver that can select between polling mode or interrupt mode for receiving data from the interface unit. The data transfer device within the server is The kernel includes the step of starting a thread that monitors packet arrivals using a polling model, When packet arrival is detected, the protocol processing unit is notified of the arrival of a packet without using the kernel protocol stack, and pointer information indicating the storage location of the arrival packet is sent. The protocol processing unit, Upon receiving the aforementioned notification, the server executes the step of obtaining the arriving packet based on the pointer information sent from the data transfer device within the server. A method for transferring data within a server, characterized by the following features.

9. A program for causing a computer to function as an in-server data transfer device according to claim 1 or claim 2.

Images