Server data transfer device, Server data transfer method, and program

Abstract

To enable power saving by reducing CPU usage rate while maintaining low latency performance.SOLUTION: An intra-server data transfer device 200 includes: a sleep control management unit 210 that manages a data arrival schedule and performs sleep control in accordance with data arrival timing, where the sleep control management unit 210 sleeps a thread that monitors arrival of data using a polling model based on the data arrival timing in a data flow having information on specific data arrival timing, and performs operation of releasing sleep of the thread based on the data arrival timing.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an in-server data transfer device, an in-server data transfer method, and a program.

Background Art

[0002] In the background of the progress of virtualization technologies such as NFV (Network Functions Virtualization), systems are being constructed and operated for each service. Also, from the form of constructing a system for each service, service functions are divided into reusable module units and operated on an independent virtual machine (VM: Virtual Machine or container, etc.) environment, and a form called SFC (Service Function Chaining), which enhances operability by using it as needed like a component, is becoming the mainstream.

[0003] As a technology for configuring a virtual machine, a hypervisor environment composed of Linux (registered trademark) and KVM (kernel-based virtual machine) is known. In this environment, a Host OS (the OS installed on a physical server is called a Host OS) incorporating a KVM module operates in a memory area different from the user space called the kernel space as a hypervisor. In this environment, a virtual machine operates in the user space, and a Guest OS (the OS installed on the virtual machine is called a Guest OS) operates within the virtual machine.

[0004] Unlike a physical server running a Host OS, a virtual machine running a Guest OS uses all hardware, including network devices (such as Ethernet® card devices), to control registers necessary for interrupt handling from the hardware to the Guest OS and for writing from the Guest OS to the hardware. In such register control, notifications and processes that should be performed by the physical hardware are simulated by software, so performance is generally lower than in a Host OS environment.

[0005] In addressing this performance degradation, there are technologies that reduce hardware imitation and improve communication performance and versatility through a high-speed, unified interface, particularly when communicating from the Guest OS to the Host OS and external processes outside of the virtual machine. One such technology is virtio, a device abstraction technology, or paravirtualization technology, which has already been developed and is incorporated into many general-purpose operating systems, including Linux® and FreeBSD®, and is currently in use (see Patent Documents 1 and 2).

[0006] In virtio, data exchange for data input / output such as console, file input / output, and network communication is defined by queue operations, using queues designed with ring buffers as a transport for unidirectional data transfer. By utilizing the virtio queue specifications and preparing the appropriate number and size of queues for each device at the time of Guest OS startup, communication between the Guest OS and the outside of the virtual machine can be achieved solely through queue operations without performing hardware emulation.

[0007] [Packet forwarding using an interrupt model (example of a general-purpose VM configuration)] Figure 19 illustrates packet forwarding using an interrupt model in a server virtualization environment with a general-purpose Linux kernel (registered trademark) and VM configuration. HW10 has a NIC (Network Interface Card) 11 (physical NIC) (interface unit) and communicates with data processing APL (Application) 1 on user space 60 via a virtual communication path established by Host OS 20, KVM 30 (a hypervisor that builds virtual machines), virtual machines (VM1, VM2) 40, and Guest OS 50. In the following description, as shown by the thick arrow in Figure 19, the data flow in which data processing APL1 receives packets from HW10 is referred to as Rx-side reception, and the data flow in which data processing APL1 sends packets to HW10 is referred to as Tx-side transmission.

[0008] Host OS20 has kernel21, Ring Buffer22, and Driver23. Kernel21 has a kernel thread vhost-net module221A, a tap device222A, and a virtual switch(br)223A.

[0009] The tap device 222A is a kernel device for the virtual network and is supported by software. Virtual machine (VM1) 40 can communicate with Guest OS 50 and Host OS 20 via a virtual switch (br) 223A created on the virtual bridge (bridge). The tap device 222A is the device that connects to the virtual NIC (vNIC) of Guest OS 50, which is created on this virtual bridge.

[0010] Host OS20 copies configuration information (such as the size of the shared buffer queue, the number of queues, identifiers, and the starting address information for accessing the ring buffer) built within the virtual machine of Guest OS50 to the vhost-net module 221A, and builds the endpoint information on the virtual machine side within Host OS20. This vhost-net module 221A is a kernel-level backend for virtio networking, and virtualization overhead can be reduced by moving the virtio packet processing task from the user area (user space) to the vhost-net module 221A in kernel21.

[0011] Guest OS50 consists of Guest OS (Guest1) installed on virtual machine (VM1) and Guest OS (Guest2) installed on virtual machine (VM2), and Guest OS50 (Guest1,Guest2) runs within virtual machines (VM1,VM2)40. Taking Guest1 as an example of Guest OS50, Guest OS50 (Guest1) has kernel51, Ring Buffer52, and Driver53, and Driver53 includes virtio-driver531.

[0012] Specifically, within the virtual machine, there are virtio devices for the console, file input / output, and network communication, each acting as a PCI (Peripheral Component Interconnect) device (the console is called virtio-console, file input / output is called virtio-blk, and network is called virtio-net, and the corresponding OS drivers for these devices are defined in the virtio queue). When the Guest OS starts, two endpoints for data transfer (sending and receiving endpoints) are created between the Guest OS and the other side, establishing a parent-child relationship for data transmission and reception. In most cases, the parent-child relationship consists of the virtual machine (child side) and the Guest OS (parent side).

[0013] The child side exists as configuration information for devices within the virtual machine, requesting from the parent side the size of each data area, the number of endpoint combinations required, and the type of device. The parent side, in accordance with the child side's request, allocates and secures memory for a shared buffer queue to store and transfer the necessary data, and returns its address to the child side so that the child side can access it. All operations on the shared buffer queue required for data transfer are common in virtio and are executed with the agreement of both the parent and child sides. Furthermore, the size of the shared buffer queue is also agreed upon by both sides (i.e., it is determined for each device). This makes it possible for both the parent and child sides to operate the queue shared by both sides simply by providing the address to the child side.

[0014] In virtio, the shared buffer queues are designed for unidirectional communication. For example, a virtual network device called a virtio-net device consists of three Ring Buffer 52s: one for sending, one for receiving, and one for control. Communication between parent and child devices is achieved through writing to the shared buffer queue and notifying the other party of buffer updates. After writing to the Ring Buffer 52, the other party is notified. Upon receiving the notification, the other party uses virtio's common operations to check which shared buffer queue contains how much new data and retrieves the new buffer area. This completes the transfer of data from parent to child or from child to parent.

[0015] As described above, by sharing the Ring Buffer 52 for data exchange and the operation method for each ring buffer (common in virtio) between the parent and child devices, communication between Guest OS 50 and the outside world is achieved without the need for hardware emulation. This makes it possible to send and receive data between Guest OS 50 and the outside world at a much faster speed compared to conventional hardware emulation.

[0016] When Guest OS50 within a virtual machine communicates with the outside world, the child OS must connect to the outside world and act as a relay between the outside world and the parent OS to send and receive data. For example, communication between Guest OS50 and Host OS20 is one such example. In this case, assuming Host OS20 is the outside world, there are two existing communication methods.

[0017] The first method (hereinafter referred to as external communication method 1) involves constructing a child endpoint within the virtual machine and connecting the communication endpoint (usually called a tap / tun device) provided by the Host OS20 to the virtual machine for communication between the Guest OS50 and the virtual machine. This connection establishes the following connection and enables communication from the Guest OS50 to the Host OS20.

[0018] In this scenario, Guest OS50 operates in a user-space memory area with different privileges than the kernel space memory area where the tap driver and Host OS20 run. Therefore, communication from Guest OS50 to Host OS20 requires at least one memory copy.

[0019] The second method (hereinafter referred to as external communication method 2) uses a technology called vhost-net to solve this problem. With vhost-net, the configuration information of the parent side (size of the shared buffer queue, number of queues, identifier, starting address information for accessing the ring buffer, etc.) that has been built within the virtual machine is copied to the vhost-net module 221A inside the Host OS20, and the information of the child side endpoint is built inside the host. This construction makes it possible to directly perform operations on the shared buffer queue between the Guest OS50 and the Host OS20. As a result, the number of copies required is effectively zero, and compared to virtio-net, the number of copies is one less, resulting in faster data transfer compared to external communication method 1.

[0020] In this way, by reducing the number of memory copies related to virtio-net between Host OS20 and Guest OS50 connected via virtio, packet forwarding processing can be sped up.

[0021] Furthermore, with kernel v4.10 (2017.2~) and later, the tap interface specifications have been changed, and packets inserted from a tap device are now processed within the same context as the packet copy operation to the tap device. As a result, software interrupts (softIRQs) no longer occur.

[0022] [Packet forwarding using a polling model (DPDK example)] The method of connecting and coordinating multiple virtual machines is called Inter-VM Communication, and in large-scale environments such as data centers, virtual switches have been the standard for connecting VMs. However, because this method has significant communication latency, faster methods have been proposed. For example, methods using special hardware called SR-IOV (Single Root I / O Virtualization) and software-based methods using the high-speed packet processing library Intel DPDK (Intel Data Plane Development Kit) (hereinafter referred to as DPDK) have been proposed (see Non-Patent Document 1).

[0023] DPDK is a framework for controlling NICs (Network Interface Cards) in user space, a task traditionally handled by the Linux kernel. The biggest difference from the Linux kernel's processing is its polling-based receiving mechanism called PMD (Pull Mode Driver). Normally, the Linux kernel triggers an interrupt upon data arrival at the NIC, which then initiates the receiving process. In contrast, PMD uses a dedicated thread to continuously check for data arrival and perform receiving processing. By eliminating overhead such as context switching and interrupts, it enables high-speed packet processing. DPDK significantly improves packet processing performance and throughput, freeing up more time for data plane application processing.

[0024] DPDK exclusively uses computer resources such as the CPU (Central Processing Unit) and NIC. Therefore, it is difficult to apply to applications that require flexible module-based switching, such as SFCs. SPP (Soft Patch Panel) is an application that mitigates this limitation. SPP eliminates packet copying at the virtualization layer by providing shared memory between VMs, allowing each VM to directly access the same memory space. Furthermore, it uses DPDK to accelerate packet exchange between the physical NIC and shared memory. SPP can software-change the input and output destinations of packets by controlling the memory exchange references of each VM. Through this process, SPP enables dynamic connection switching between VMs and between VMs and physical NICs (see Non-Patent Literature 2).

[0025] Figure 20 illustrates packet forwarding using the polling model in the OvS-DPDK (Open vSwitch with DPDK) configuration. Components identical to those in Figure 19 are denoted by the same reference numerals, and explanations of these redundant parts are omitted. As shown in FIG. 20, the Host OS 20 includes OvS-DPDK 70 which is software for packet processing. OvS-DPDK 70 has a vhost-user 71 which is a functional unit for connecting to a virtual machine (here VM1), and a dpdk (PMD) 72 which is a functional unit for connecting to the NIC (DPDK) 11 (physical NIC). Also, the data processing APL1A includes a dpdk (PMD) 2 which is a functional unit for performing polling in the Guest OS 50 section. That is, the data processing APL1A is an APL obtained by equipping the data processing APL1 in FIG. 19 with the dpdk (PMD) 2 and modifying the data processing APL1.

[0026] Packet transfer by the polling model, as an extension of DPDK, enables path operation by GUI in the SPP that performs high-speed packet copying between the Host OS 20 and the Guest OS 50 and between the Guest OS 50s via shared memory with zero copy.

[0027] [Rx-side Packet Processing by New API (NAPI)] FIG. 21 is a schematic diagram of Rx-side packet processing by the New API (NAPI) implemented from Linux kernel 2.5 / 2.6 (see Non-Patent Document 1). The same reference numerals are assigned to the same components as in FIG. 19. As shown in FIG. 21, the New API (NAPI) executes the data processing APL1 arranged in the user space 60 that can be used by the user on a server equipped with the OS 70 (for example, the Host OS), and performs packet transfer between the NIC 11 of the HW 10 connected to the OS 70 and the data processing APL1.

[0028] The OS 70 has a kernel 71, a Ring Buffer 72, and a Driver 73, and the kernel 71 has a protocol processing unit 74. Kernel71 is a core function of the OS70 (e.g., Host OS), which monitors hardware and manages the execution status of programs on a per-process basis. Here, kernel71 responds to requests from the data processing APL1 and conveys requests from the HW10 to the data processing APL1. Kernel71 processes requests from the data processing APL1 by means of system calls (where a "user program operating in non-privileged mode" requests processing from a "kernel operating in privileged mode"). Kernel71 transmits packets to the data processing APL1 via the Socket75. Kernel71 receives packets from the data processing APL1 via the Socket75.

[0029] The Ring Buffer72 is managed by Kernel71 and is in the memory space of the server. The Ring Buffer72 is a buffer of a fixed size that stores messages output by Kernel71 as logs, and when the upper limit size is exceeded, it is overwritten from the beginning.

[0030] Driver73 is a device driver for Kernel71 to monitor hardware. Note that Driver73 depends on Kernel71, and if the created (built) kernel source changes, it becomes a different thing. In this case, the corresponding driver source needs to be obtained, rebuilt on the OS using the driver, and the driver is created.

[0031] The protocol processing unit 74 performs protocol processing for L2 (data link layer) / L3 (network layer) / L4 (transport layer) defined by the OSI (Open Systems Interconnection) reference model.

[0032] Socket75 is the interface that kernel71 uses for inter-process communication. Socket75 has a socket buffer and does not frequently perform data copying operations. The process for establishing communication via Socket75 is as follows: 1. The server creates a socket file to accept clients. 2. The server names the acceptance socket file. 3. The server creates a socket queue. 4. The server accepts the first connection from a client in the socket queue. 5. The client creates a socket file. 6. The client sends a connection request to the server. 7. The server creates a connection socket file in addition to the acceptance socket file. As a result of establishing communication, data processing APL1 can call system calls such as read() and write() on kernel71.

[0033] In the above configuration, Kernel71 receives notification of packet arrival from NIC11 via a hardware interrupt (hardIRQ) and schedules a software interrupt (softIRQ) for packet processing. The New API (NAPI), implemented in Linux kernel 2.5 / 2.6 and above, processes packets upon arrival using a hardware interrupt (hardIRQ) followed by a software interrupt (softIRQ). As shown in Figure 21, packet forwarding using the interrupt model involves waiting for interrupt processing (see symbol c in Figure 21), which increases the delay in packet forwarding.

[0034] The following describes the overview of packet processing on the NAPI Rx side. [Rx-side packet processing configuration using New API (NAPI)] Figure 22 is a diagram illustrating the overview of Rx-side packet processing by the New API (NAPI) in the area enclosed by the dashed line in Figure 21. <Device driver> As shown in Figure 22, the device driver includes NIC11 (physical NIC), which is a network interface card; hardIRQ81, which is a handler that is called when a processing request occurs on NIC11 and executes the requested processing (hardware interrupt); and netif_rx82, which is a software interrupt processing function unit.

[0035] <Networking layer> The Networking layer contains softIRQ83, a handler that is called when a processing request occurs in netif_rx82 and executes the requested processing (software interrupt), and do_softirq84, a control function unit that implements the software interrupt (softIRQ). It also contains net_rx_action85, a packet processing function unit that receives and executes software interrupts (softIRQ), poll_list86, which registers net device information indicating which device the hardware interrupt from NIC11 is from, netif_receive_skb87, which creates the sk_buff structure (a structure that allows Kernel71 to perceive the status of packets), and Ring Buffer72.

[0036] <Protocol layer> The protocol layer contains packet processing functions such as ip_rcv88 and arp_rcv89.

[0037] The above netif_rx82, do_softirq84, net_rx_action85, netif_receive_skb87, ip_rcv88, and arp_rcv89 are program components (function names) used for packet processing within Kernel 71.

[0038] [Rx-side packet processing operation using the New API (NAPI)] The arrows (symbols) d~o in Figure 22 indicate the flow of Rx-side packet processing. When the hardware function unit 11a of NIC11 (hereinafter referred to as NIC11) receives a packet (or frame) within a frame from the opposing device, it copies the arriving packet to the Ring Buffer 72 without using the CPU via DMA (Direct Memory Access) transfer (see symbol d in Figure 22). This Ring Buffer 72 is a memory space within the server and is managed by Kernel 71 (see Figure 21).

[0039] However, if NIC11 simply copies the packet that arrives in Ring Buffer72, Kernel71 will not be able to recognize that packet. Therefore, when a packet arrives, NIC11 raises a hardware interrupt (hardIRQ) to hardIRQ81 (see symbol e in Figure 22), and netif_rx82 executes the following process so that Kernel71 can recognize the packet. Note that hardIRQ81, enclosed in an ellipse in Figure 22, represents the handler, not the functional part.

[0040] netif_rx82 is the function that actually performs the processing. When hardIRQ81 (handler) is activated (see symbol f in Figure 22), it saves information about the net device (net_device), which indicates which device the hardware interrupt from NIC11 is from—one of the pieces of information inside the hardware interrupt (hardIRQ)—in poll_list86, and registers queue plucking (referencing the contents of packets accumulated in the buffer and deleting the corresponding queue entry from the buffer, taking into account the next processing to be performed) (see symbol g in Figure 22). Specifically, when netif_rx82 receives that packets have been packed into Ring Buffer72, it uses the NIC11 driver to register subsequent queue plucking in poll_list86 (see symbol g in Figure 22). As a result, poll_list86 contains queue plucking information due to packets being packed into Ring Buffer72.

[0041] Thus, Figure 22<Device driver> In this scenario, when NIC11 receives a packet, it copies the arriving packet to Ring Buffer72 via DMA transfer. NIC11 also raises hardIRQ81 (a handler), and netif_rx82 registers net_device in poll_list86, scheduling a software interrupt (softIRQ). Up to this point, Figure 22<Device driver> Hardware interrupt processing in this case will be stopped.

[0042] Subsequently, netif_rx82 uses the information (specifically pointers) in the queue stored in poll_list86 to raise a software interrupt (softIRQ) to softIRQ83 (handler) (see code h in Figure 22) to pluck the data stored in Ring Buffer72, and notifies do_softirq84, the software interrupt control unit (see code i in Figure 22).

[0043] do_softirq84 is the software interrupt control unit, and it defines various software interrupt functions (there are various types of packet processing, and interrupt processing is one of them; it defines interrupt processing). Based on this definition, do_softirq84 notifies net_rx_action85, which actually performs the software interrupt processing, of the request for the current (relevant) software interrupt (see symbol j in Figure 22).

[0044] When it's the turn of the softIRQ, net_rx_action85 calls a polling routine to pluck packets from Ring Buffer72 based on the net_device registered in poll_list86 (see code k in Figure 22), and plucks packets (see code l in Figure 22). At this time, net_rx_action85 continues plucking until poll_list86 is empty. Subsequently, net_rx_action85 notifies netif_receive_skb87 (see symbol m in Figure 22).

[0045] netif_receive_skb87 creates an sk_buff structure, analyzes the contents of the packet, and passes the processing to the subsequent protocol processing unit 74 (see Figure 21) according to the type. In other words, netif_receive_skb87 analyzes the contents of the packet and, when processing according to the contents of the packet,<Protocol layer> The process is then passed to ip_rcv88 (symbol n in Figure 22), and if it is L2, for example, the process is passed to arp_rcv89 (symbol o in Figure 22).

[0046] Non-patent document 3 describes a server-internal network delay control device (KBP: Kernel Busy Poll). KBP constantly monitors packet arrivals within the kernel using a polling model. This suppresses softIRQ and enables low-latency packet processing.

[0047] Figure 23 shows an example of video (30 FPS) data transfer. The workload shown in Figure 23 has a transfer rate of 350 Mbps and performs intermittent data transfer every 30 ms.

[0048] Figure 24 shows the CPU usage rate used by the busy poll thread in the KBP described in Non-Patent Document 3. As shown in Figure 24, in KBP, the kernel thread occupies a CPU core to perform busy polling. Even with intermittent packet reception as shown in Figure 23, KBP always uses the CPU regardless of whether packets have arrived or not, resulting in high power consumption.

[0049] Next, I will explain the DPDK system. [DPDK System Configuration] Figure 25 shows the configuration of the DPDK system that controls the HW110 equipped with accelerator 120. The DPDK system consists of HW110, OS140, DPDK150 (a high-speed data transfer middleware), and data processing APL1, all located on user space160. Data processing APL1 is packet processing that takes place prior to the execution of APL. HW110 communicates with data processing APL1 for sending and receiving data. In the following description, as shown in Figure 25, the data flow in which data processing APL1 receives packets from HW110 is referred to as Rx-side reception, and the data flow in which data processing APL1 sends packets to HW110 is referred to as Tx-side transmission.

[0050] The HW110 includes an accelerator 120 and a NIC 130 (physical NIC) for connecting to a communication network. The accelerator 120 is a computing unit hardware that performs specific calculations at high speed based on input from the CPU. Specifically, the accelerator 120 is a PLD (Programmable Logic Device) such as a GPU (Graphics Processing Unit) or FPGA (Field Programmable Gate Array). In Figure 25, the accelerator 120 includes multiple Cores (Core processors) 121, an Rx queue (queue) 122 that holds data in a first-in, first-out list structure, and a Tx queue 133.

[0051] By offloading part of the data processing APL1's processing to accelerator 120, performance and power efficiency that cannot be achieved with software (CPU processing) alone are realized. In large-scale server clusters, such as data centers that utilize NFV (Network Functions Virtualization) or SDN (Software Defined Networking), it is conceivable that accelerator 120 like the one described above would be applied.

[0052] NIC130 is NIC hardware that implements a network interface and includes Rx queue 131 and Tx queue 132 that hold data in a first-in, first-out list structure. NIC130 is connected to, for example, the opposing device 170 via a communication network to send and receive packets. Note that NIC130 may be a SmartNIC, for example, a NIC with an accelerator. A SmartNIC is a NIC that can reduce the CPU load by offloading resource-intensive processes, such as IP packet processing, which can cause a decrease in processing power.

[0053] DPDK150 is a framework for controlling NICs in user space 160, and specifically consists of high-speed data transfer middleware. DPDK150 has a polling-based receiving mechanism, PMD (Poll Mode Driver) 151 (a driver that can select between polling mode and interrupt mode for data arrival). PMD151 has a dedicated thread that continuously checks for data arrival and processes the reception.

[0054] DPDK150 implements packet processing functionality in user space 160 where APL operates, and enables reduced packet forwarding delay by immediately harvesting packets upon arrival from user space 160 using a polling model. In other words, DPDK150 harvests packets using polling (busy polling of the queue by the CPU), resulting in no waiting and low latency. [Prior art documents] [Patent Documents]

[0055] [Patent Document 1] Japanese Patent Publication No. 2015-197874 [Patent Document 2] Japanese Patent Publication No. 2018-32156 [Non-patent literature]

[0056] [Non-Patent Document 1] New API Intel, [online], [Searched July 5, 2021], Internet <http: / / lwn.net / 2002 / 0321 / a / napi-howto.php3> [Non-Patent Document 2] “Resource Configuration (NIC) ~Introduction to DPDK, Part 6~,” NTT TechnoCross, [online], [Accessed July 5, 2021], Internet <https: / / www.ntt-tx.co.jp / column / dpdk_blog / 190610 / > [Non-Patent Document 3] Kei Fujimoto, Kenichi Matsui, Masayuki Akutsu, “KBP: Kernel Enhancements for Low-Latency Networking without Application Customization in Virtual Server”, IEEE CCNC 2021. [Overview of the Initiative] [Problems that the invention aims to solve]

[0057] However, both the interrupt model and the polling model for packet forwarding have the following challenges. The interrupt model uses software interrupt processing to process packets, which is performed by the kernel upon receiving an event (hardware interrupt) from the hardware. Therefore, because the interrupt model relies on interrupt (software interrupt) processing for packet forwarding, it faces challenges such as conflicts with other interrupts and delays if the target CPU is occupied by a higher-priority process. In this case, congestion in interrupt processing further increases the waiting delay. For example, as shown in Figure 19, packet forwarding using an interrupt model involves packet forwarding via interrupt processing (see symbols a and b in Figure 19), which results in delays due to waiting for interrupt processing.

[0058] Supplement the mechanism that causes delays in the interrupt model. In a general kernel, packet transfer processing is transmitted by software interrupt processing after hardware interrupt processing. When a software interrupt for packet transfer processing occurs, under the following conditions (1) to (3), the software interrupt processing cannot be executed immediately. For this reason, mediation is performed by a scheduler such as ksoftirqd (a kernel thread for each CPU, which is executed when the load of software interrupts becomes high), and the interrupt processing is scheduled, resulting in a waiting time on the order of milliseconds. (1) When competing with other hardware interrupt processing (2) When competing with other software interrupt processing (3) When other processes or kernel threads (such as migration threads) with high priority and the interrupt destination CPU are being used Under the above conditions, the software interrupt processing cannot be executed immediately.

[0059] Also, regarding packet processing by New API (NAPI), similarly, as shown in the dashed box p in Fig. 22, due to competition in interrupt processing (softIRQ), a NW delay on the order of milliseconds occurs.

[0060] <Issues of KBP> As described above, KBP can suppress softIRQ and achieve low-latency packet processing by constantly monitoring packet arrival using the polling model within the kernel. However, since the kernel thread that constantly monitors packet arrival monopolizes the CPU core and always uses CPU time, there is a problem of high power consumption. Referring to Figs. 23 and 24, the relationship between the workload and CPU usage will be described. As shown in FIG. 24, in KBP, the kernel thread exclusively occupies the CPU core to perform busy poll. Even for the intermittent packet reception shown in FIG. 23, in KBP, regardless of whether a packet arrives or not, the CPU is always used, so there is a problem of increased power consumption.

[0061] DPDK also has the same problem as the above KBP. <Problems of DPDK> In DPDK, the kernel thread exclusively occupies the CPU core to perform polling (busy poll on the queue by the CPU). Therefore, even for the intermittent packet reception shown in FIG. 23, in DPDK, regardless of whether a packet arrives or not, the CPU is always used 100%, so there is a problem of increased power consumption.

[0062] Thus, since DPDK realizes the polling model in user space and no softIRQ conflict occurs, and KBP realizes the polling model within the kernel and no softIRQ conflict occurs, low-latency packet transfer is possible. However, both DPDK and KBP have the problem that regardless of whether a packet arrives or not, they always waste CPU resources for packet arrival monitoring, resulting in increased power consumption.

[0063] In view of such a background, the present invention has been made, and an object of the present invention is to reduce the CPU usage rate and enable power saving while maintaining low latency.

Means for Solving the Problems

[0064] To solve the aforementioned problems, the server data transfer device is provided with a sleep control management unit that manages the data arrival schedule and performs sleep control in accordance with the data arrival timing, wherein the sleep control management unit, in a data flow that has information regarding a specific data arrival timing, puts a thread that monitors data arrival using a polling model based on the data arrival timing to sleep, and then wakes the thread from sleep based on the data arrival timing. [Effects of the Invention]

[0065] According to the present invention, it is possible to reduce CPU usage and save power while maintaining low latency. [Brief explanation of the drawing]

[0066] [Figure 1] This is a schematic diagram of a server-internal data transfer system according to the first embodiment of the present invention. [Figure 2] This figure shows an example of the operation of a polling thread in a server-internal data transfer system according to the first embodiment of the present invention. [Figure 3] This is a schematic diagram of the server data transfer system in an example of acquisition 1 of the server data transfer system according to the first embodiment of the present invention. [Figure 4] This is a schematic diagram of the server data transfer system in an example 2 of the server data transfer system according to the first embodiment of the present invention. [Figure 5] This is a schematic diagram of the server data transfer system in acquisition example 3 of the server data transfer system according to the first embodiment of the present invention. [Figure 6] This flowchart shows the operation of the sleep control management unit when there is a change in the data arrival schedule information of the server data transfer system according to the first embodiment of the present invention. [Figure 7]This flowchart shows the operation of the sleep control management unit when the data transfer section of the server data transfer system according to the first embodiment of the present invention is added or removed. [Figure 8] This is a flowchart showing the operation of the sleep control unit of the data transfer section of the server data transfer system according to the first embodiment of the present invention. [Figure 9] This is a flowchart showing the operation of the data arrival monitoring unit of the data transfer unit of the server data transfer system according to the first embodiment of the present invention. [Figure 10] This is a flowchart showing the operation of the Tx data transfer unit of the data transfer unit of the server data transfer system according to the first embodiment of the present invention. [Figure 11] This flowchart shows the operation of the data transfer unit in the server data transfer system according to the first embodiment of the present invention when there is a difference in the data arrival schedule. [Figure 12] This flowchart shows the operation of the data transfer unit in the server data transfer system according to the first embodiment of the present invention when there is a difference in the data arrival schedule. [Figure 13] This is a schematic diagram of a server-internal data transfer system according to a second embodiment of the present invention. [Figure 14] This is a flowchart showing the operation of the data arrival monitoring unit of the data transfer unit of the server data transfer system according to the second embodiment of the present invention. [Figure 15] This figure shows an example of applying an in-server data transfer system to an interrupt model in a server virtualization environment using a general-purpose Linux kernel and VM configuration. [Figure 16] This figure shows an example of applying an in-server data transfer system to an interrupt model in a containerized server virtualization environment. [Figure 17] This is a schematic diagram of a server-internal data transfer system according to a third embodiment of the present invention. [Figure 18] This hardware configuration diagram shows an example of a computer that implements the functions of an in-server data transfer device in an in-server data transfer system according to an embodiment of the present invention. [Figure 19] This diagram illustrates packet forwarding using an interrupt model in a server virtualization environment with a general-purpose Linux kernel and VM configuration. [Figure 20] This diagram illustrates packet forwarding using a polling model in the OvS-DPDK configuration. [Figure 21] This is a schematic diagram of Rx-side packet processing using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6. [Figure 22] Figure 21 shows an overview of Rx-side packet processing by the New API (NAPI) in the area enclosed by the dashed line. [Figure 23] This figure shows an example of video (30 FPS) data transfer. [Figure 24] This figure shows the CPU usage rate used by the busy poll thread in the KBP described in Non-Patent Document 3. [Figure 25] This diagram shows the configuration of a DPDK system that controls hardware equipped with an accelerator. [Modes for carrying out the invention]

[0067] The following describes a server data transfer system and the like in an embodiment of the present invention (hereinafter referred to as "this embodiment") with reference to the drawings. (First Embodiment) [Overall structure] Figure 1 is a schematic diagram of a server-internal data transfer system according to a first embodiment of the present invention. Components identical to those in Figure 25 are denoted by the same reference numerals. As shown in Figure 1, the server data transfer system 1000 includes HW110, OS140, and a server data transfer device 200, which is high-speed data transfer middleware located on the user space 160. The user space 160 also contains the data processing APL1 and the data flow time slot management scheduler 2. The data processing APL1 is a program that runs in the user space 160. The data flow time slot management scheduler 2 sends schedule information to the data processing APL1 (see symbol q in Figure 1). The data flow time slot management scheduler 2 also sends data arrival schedule information to the sleep control management unit 210 (described later) (see symbol r in Figure 1).

[0068] HW110 communicates with data processing APL1 for sending and receiving data. The data flow in which data processing APL1 receives packets from HW110 is called Rx-side reception, and the data flow in which data processing APL1 sends packets to HW110 is called Tx-side transmission. The HW110 includes an accelerator 120 and a NIC 130 (physical NIC) for connecting to a communication network.

[0069] The accelerator 120 is a computing unit hardware such as a GPU or FPGA. The accelerator 120 includes multiple Cores (Core processors) 121, and Rx queues 122 and Tx queues 123 that hold data in a first-in, first-out list structure. By offloading part of the data processing APL1's processing to accelerator 120, performance and power efficiency that cannot be achieved with software (CPU processing) alone are realized.

[0070] NIC130 is NIC hardware that implements a network interface and includes Rx queue 131 and Tx queue 132 that hold data in a first-in, first-out list structure. NIC130 is connected to, for example, the opposing device 170 via a communication network to send and receive packets.

[0071] OS 140 is, for example, Linux (registered trademark). OS 140 includes a high-resolution timer 141 that performs timer management in more detail than the kernel timer. The high-resolution timer 141 uses, for example, the hrtimer of Linux (registered trademark). In hrtimer, the time when a callback occurs can be specified using the unit ktime_t. The high-resolution timer 141 notifies the sleep control unit 221 of the data transfer unit 220 described later of the data arrival timing at the specified time (see reference u in FIG. 1).

[0072] [Server internal data transfer device 200] The server internal data transfer device 200 is DPDK for performing the control of the NIC in user space 160, and specifically consists of data high-speed transfer middleware. The server internal data transfer device 200 includes a sleep control management unit 210 and a data transfer unit 220. The server internal data transfer device 200 has a PMD 151 (driver capable of selecting data arrival in polling mode or interrupt mode) (see FIG. 25) similar to the DPDK arranged on user space 160. The PMD 151 is a driver capable of selecting data arrival in polling mode or interrupt mode, and a dedicated thread continuously performs confirmation of data arrival and reception processing.

[0073] <sleep control management unit 210> The sleep control management unit 210 manages the data arrival schedule and performs sleep control of the data transfer unit 220 according to the data arrival timing. The sleep control management unit 210 collectively performs timing control of Sleep / startup of each data transfer unit 220 (see reference t in FIG. 1).

[0074] The sleep control management unit 210 manages the data arrival schedule information, distributes the data arrival schedule information to the data transfer unit 220, and performs sleep control of the data transfer unit 220. The sleep control management unit 210 includes a data transfer unit management unit 211, a data arrival schedule management unit 212, and a data arrival schedule distribution unit 213.

[0075] The data transfer unit management unit 211 maintains a list of information such as the number of data transfer units 220 and process IDs (PID: Process Identification). The data transfer unit management unit 211 transmits information such as the number of data transfer units 220 and process IDs to the data transfer units 220 in response to a request from the data arrival schedule distribution unit 213.

[0076] The data arrival schedule management unit 212 manages the data arrival schedule. The data arrival schedule management unit 212 obtains data arrival schedule information from the data flow time slot management scheduler 2 (see symbol r in Figure 1). The data arrival schedule management unit 212 detects changes to the data arrival schedule information by receiving a notification of the change from the data flow time slot management scheduler 2 when there are changes to the data arrival schedule information. Alternatively, the data arrival schedule management unit 212 detects the change by snooping the data that contains the data arrival schedule information (see Figures 4 and 5). The data arrival schedule management unit 212 transmits data arrival schedule information to the data arrival schedule distribution unit 213 (see the symbol s in Figure 1).

[0077] The data arrival schedule distribution unit 213 obtains information such as the number of data transfer units 220 and process IDs from the data transfer unit management unit 211. The data arrival schedule distribution unit 213 distributes data arrival schedule information to each data transfer unit 220 (see reference numeral t in Figure 1).

[0078] <Data transfer unit 220> The data transfer unit 220 starts a polling thread that monitors packet arrival using the polling model. Based on the data arrival schedule information distributed from the sleep control management unit 210, the data transfer unit 220 puts the thread to sleep and activates a timer immediately before data arrival to wake up the thread for sleep release. Here, in case the data transfer unit 220 receives a packet at an unintended timing by the timer, when releasing the sleep, the relevant thread is woken up by a hardware interrupt. Sleep / release will be described later under [Sleep / Release].

[0079] The data transfer unit 220 includes a sleep control unit 221, a data arrival monitoring unit 222, an Rx data transfer unit 223 (packet harvesting unit), and a Tx data transfer unit 224. The data arrival monitoring unit 222 and the Rx data transfer unit 223 are Rx-side functional units, and the Tx data transfer unit 224 is a Tx-side functional unit.

[0080] <sleep control unit 221> Based on the data arrival schedule information from the sleep control management unit 210, the sleep control unit 221 performs sleep control to stop data arrival monitoring and sleep when there is no data arrival. The sleep control unit 221 holds the data arrival schedule information received from the data arrival schedule distribution unit 213.

[0081] The sleep control unit 221 sets a timer for the data arrival timing for the data arrival monitoring unit 222 (see reference symbol v in FIG. 1). That is, the sleep control unit 221 sets a timer so that the data arrival monitoring unit 222 can start polling immediately before data arrival. Here, the sleep control unit 221 may utilize hrtimers or the like, which are high-resolution timers 141 held by the Linux kernel, and wake up the data arrival monitoring unit 222 by a hardware interrupt opportunity when the timer is activated by the hardware clock.

[0082] FIG. 2 is a diagram showing an operation example of a polling thread of the in-server data transfer device 200. The vertical axis represents the CPU usage rate [%] of the CPU core used by the polling thread, and the horizontal axis represents time. Note that FIG. 3 shows an operation example of the polling thread due to packet arrival corresponding to a data transfer example of an intermittently received video (30 FPS) shown in FIG. 13. As shown in FIG. 2, the data transfer unit 220 puts a thread (polling thread) to sleep (see reference w in FIG. 2) based on the data arrival schedule information received from the sleep control management unit 210, and wakes up from the sleep by a hardware interrupt (hardIRQ) when the sleep is released (see reference x in FIG. 2). Note that reference y in FIG. 2 is a fluctuation in the wiring voltage due to congestion use of the core CPU (Core processor), etc.

[0083] <Rx side> The data arrival monitoring unit 222 is activated immediately before data arrives according to the data arrival schedule information managed by the sleep control unit 221. The data arrival monitoring unit 222 monitors the Rx queues 122 and 131 of the accelerator 120 or the NIC 130 to check for data arrival.

[0084] The data arrival monitoring unit 222 exclusively occupies the CPU core regardless of whether data has arrived and monitors for data arrival by polling. Incidentally, if this is made an interrupt model, the delay described in the prior art of FIG. 22 (that is, when a softIRQ competes with other softIRQs, a wait occurs regarding the execution of the softIRQ, and a NW delay on the order of ms due to this wait) occurs. In this embodiment, it is a feature that the polling model sleep control is used on the Rx side.

[0085] When there is data arrival in the Rx queues 122 and 131, the data arrival monitoring unit 222 performs dequeueing of the queues stored in the Rx queues 122 and 131 (referring to the content of the packets stored in the buffer and deleting the corresponding queue entries from the buffer considering the processing of the packet and the next processing to be performed), and transfers the data to the Rx data transfer unit 223.

[0086] The Rx data transfer unit 223 transfers the received data to the data processing APL1. Similar to the Tx data transfer unit 224, since it operates only when data arrives, it does not waste the CPU.

[0087] <Tx side> The Tx data transfer unit 224 stores the received data in the Tx queues 123 and 132 of the accelerator 120 or the NIC 130. The Tx data transfer unit 224 is activated by inter-process communication when the data processing APL1 sends out data and returns to CPU idle when the data transfer is completed. Therefore, unlike the data arrival monitoring unit 222, it does not waste the CPU.

[0088] [Sleep / Release] Based on the data arrival schedule information received from the sleep control unit 221, the data transfer unit 220 puts the thread to sleep and wakes up on the occasion of a timer. <Normal time> Based on the scheduling information of the data arrival timing (data arrival schedule information), the data transfer unit 220 activates a timer immediately before the data arrival to wake up the data arrival monitoring unit thread of the data transfer unit 220. For example, using the hr_timer function standardly installed in the Linux kernel, when the timer deadline arrives, a hardware interrupt of the timer is activated, and the data arrival monitoring unit 222 wakes up the thread.

[0089] <Unexpected (when data arrives outside the schedule)> If data arrives outside of the scheduled timing, the thread of the data arrival monitoring unit 222 is in a sleep state. Also, there is no timer activation for normal operation. Therefore, a hardware interrupt is activated to notify of packet arrival when a packet arrives. As mentioned above, under normal circumstances, packets are constantly monitored in polling mode, so hardware interrupts are not necessary, and the hardware interrupt function is disabled by the driver (PMD). However, when putting the polling thread to sleep, the mode is changed to raise a hardware interrupt when a packet arrives, in case data arrives outside of the scheduled time. This way, when a packet arrives, a hardware interrupt is raised, and the data arrival monitoring unit 222 wakes up a thread using this hardware interrupt handler.

[0090] [Example of obtaining data arrival schedule information] This section describes an example of acquiring data arrival schedule information for a server-internal data transfer system according to this embodiment. An example of a data flow with a fixed data arrival schedule is signal processing in a RAN (Radio Access Network). In RAN signal processing, the MAC scheduler of MAC4 (described later) manages the timing of data arrivals using time-division multiplexing.

[0091] Signal processing for vRAN (virtual RAN) and vDU (virtual Distributed Unit) often utilizes DPDK for high-speed data transfer. By applying the inventive method, the sleep control of the data transfer unit (DPDK PMD, etc.) is performed in accordance with the data arrival timing managed by the MAC scheduler.

[0092] As methods for obtaining the data arrival timing managed by the MAC scheduler, there are <Obtaining data arrival schedule information from the MAC scheduler> (directly obtained from the MAC Scheduler) (see Fig. 3), <Snooping FAPI P7 to obtain data arrival schedule information> (obtained by snooping the FAPI P7 IF) (see Fig. 4), and <Snooping CTI to obtain data arrival schedule information> (obtained by snooping the O-RAN CTI) (see Fig. 5). These will be explained in order below.

[0093] <Obtaining data arrival schedule information from the MAC scheduler> Fig. 3 is a schematic configuration diagram of the in-server data transfer system of Acquisition Example 1. Acquisition Example 1 is an example applied to the vDU system. The same components as in Fig. 1 are labeled with the same reference numerals, and the description of overlapping parts is omitted. As shown in Fig. 3, in the in-server data transfer system 1000A of Acquisition Example 1, in user space 160, further, PHY (High) (Physical) 3, MAC (Medium Access Control) 4, and RLC (Radio Link Control) 5 are arranged. As a countermeasure device connected to NIC 130, RU (Radio Unit) 171 is connected to the receiving side of NIC 130, and vCU 172 is connected to the transmitting side of NIC 130.

[0094] The sleep control management unit 210 of the in-server data transfer system 1000A modifies the MAC scheduler of MAC 4 to obtain data arrival schedule information from MAC 4 (see reference symbol z in Fig. 3). Although an example applied to the vDU system has been described, it may be applied not only to the vDU but also to vRAN systems such as vCU.

[0095] <Snooping CTI to obtain data arrival schedule information> FIG. 4 is a schematic configuration diagram of the in-server data transfer system of Acquisition Example 2. Acquisition Example 2 is an example applied to the vCU system. The same components as those in FIG. 3 are denoted by the same reference numerals, and the description of overlapping parts is omitted. As shown in FIG. 4, in the in-server data transfer system 1000B of Acquisition Example 2, in the user space 160, an FAPI (FAPI P7) 6 is further arranged between the PHY (High) 3 and the MAC 4. Note that although the FAPI 6 is drawn inside the in-server data transfer device 200 for the sake of notation, the FAPI 6 is arranged outside the in-server data transfer device 200. The FAPI 6 is an IF (interface) that exchanges data scheduling information and the like for connecting the PHY (High) 3 and the MAC 4 defined in the SCF (Small Cell Forum) (see reference numeral aa in FIG. 4).

[0096] The sleep control management unit 210 of the in-server data transfer system 1000B snoops on the FAPI 6 and then acquires data arrival schedule information (see reference numeral bb in FIG. 4).

[0097] <Snoop on <CTI7> and acquire data arrival schedule information> FIG. 5 is a schematic configuration diagram of the in-server data transfer system of Acquisition Example 3. Acquisition Example 3 is an example applied to the vCU system. The same components as those in FIG. 3 are denoted by the same reference numerals, and the description of overlapping parts is omitted. As shown in FIG. 5, in the in-server data transfer system 1000C of Acquisition Example 3, a transmission device 173 is arranged outside the user space 160. The transmission device 173 is a transmission device defined in the O-RAN community. The MAC 4 in the user space 160 and the transmission device 173 are connected via a CTI (Collaborative Transport Interface) 7. The CTI 7 is an IF that exchanges data scheduling information and the like with the transmission device defined in the O-RAN community (see reference numeral cc in FIG. 5).

[0098] The sleep control management unit 210 of the server data transfer system 1000C snoops CTI7 and then acquires data arrival schedule information (see the symbol dd in Figure 5).

[0099] The following describes the operation of the data transfer system within the server. Since the basic operation of the server data transfer systems 1000 (see Figure 1), 1000A (see Figure 3), 1000B (see Figure 4), and 1000C (see Figure 5) is the same, we will explain the server data transfer system 1000 (see Figure 1).

[0100] [Operation of sleep control management unit 210] <If there are any changes to the data arrival schedule information> Figure 6 is a flowchart showing the operation of the sleep control management unit 210 when there is a change in the data arrival schedule information. Step S10, enclosed in a dashed line in Figure 6, represents an external factor that triggers the start of operation of the sleep control management unit 210 (hereinafter, in this specification, dashed lines in flowcharts represent external factors that trigger operation). In step S10 [External Factors], if there is a change in the data arrival schedule information, the data flow time slot management scheduler 2 (see Figure 1) notifies the data arrival schedule management unit 212 of the sleep control management unit 210 that a change has occurred (see the symbol r in Figure 1). Alternatively, as shown in Figures 4 and 5, the data arrival schedule management unit 212 of the sleep control management unit 210 (see Figure 1) detects the change by snooping the data containing the data arrival schedule information.

[0101] In step S11, the data arrival schedule management unit 212 (see Figure 1) of the sleep control management unit 210 obtains data arrival schedule information from the data flow time slot management scheduler 2 (see Figure 1).

[0102] In step S12, the data arrival schedule management unit 212 transmits data arrival schedule information to the data arrival schedule distribution unit 213 (see FIG. 1).

[0103] In step S13, the data arrival schedule distribution unit 213 of the sleep control management unit 210 acquires information such as the number and process ID of the data transfer unit 220 (see FIG. 1) from the data transfer unit management unit 211 (see FIG. 1).

[0104] In step S14, the data arrival schedule distribution unit 213 distributes the data arrival schedule information to each data transfer unit 220 (see FIG. 1) to complete the processing of this flow.

[0105] <When the addition or deletion of the data transfer unit 220 occurs> FIG. 7 is a flowchart showing the operation of the sleep control management unit 210 when the addition or deletion of the data transfer unit 220 occurs. In step S20 [external factor], when the addition or deletion of the data transfer unit 220 (see FIG. 1) occurs, the operation system of this system, maintenance operator, etc. set information such as the number and process ID of the data transfer unit 220 for the data transfer unit management unit 211 (see FIG. 1) of the sleep control management unit 210.

[0106] In step S21, the data transfer unit management unit 211 of the sleep control management unit 210 holds information such as the number and process ID of the data transfer unit 220 as a list.

[0107] In step S22, the data transfer unit management unit 211 transmits information such as the number and process ID of the data transfer unit 220 in response to a request from the data arrival schedule distribution unit 213 to complete the processing of this flow. The operation of the sleep control management unit 210 has been described above. Next, the operation of the data transfer unit 220 will be described.

[0108] [Operation of the data transfer unit 220] <Sleep control> FIG. 8 is a flowchart showing the operation of the sleep control unit 221 of the data transfer unit 220. In step S31, the sleep control unit 221 (see FIG. 1) of the data transfer unit 220 holds the data arrival schedule information received from the data arrival schedule distribution unit 213 (see FIG. 1) of the sleep control management unit 210.

[0109] Here, due to reasons such as not being synchronized with the time of the opposing device 170 (see FIG. 1), there may be a constant difference between the data arrival timing managed by the sleep control management unit 210 (see FIG. 1) and the actual data arrival timing. In this case, the data transfer unit 220 stores the difference from the data arrival timing, and if this difference data is constant, the sleep control management unit 210 may correct it by a certain difference time to handle it (details will be described later in FIGS. 11 and 12).

[0110] In step S32, the sleep control unit 221 (see FIG. 1) of the data transfer unit 220 sets a timer for the data arrival timing for the data arrival monitoring unit 222 (see FIG. 1). That is, the sleep control unit 221 sets a timer so that the data arrival monitoring unit 222 can start polling immediately before the data arrival.

[0111] At this time, a high-resolution timer 141 (see FIG. 1) such as hrtimers (registered trademark) owned by the Linux kernel (registered trademark) may be used to activate the data arrival monitoring unit 222 at the hardware interrupt opportunity when the timer is activated by the hardware clock. The operation of the sleep control unit 221 has been described above. Next, the <Rx side> and <Tx side> operations of the data transfer unit 220 will be described. The present invention has the feature that the operations on the <Rx side> and <Tx side> are different.

[0112] <Rx side> FIG. 9 is a flowchart showing the operation of the data arrival monitoring unit 222 of the data transfer unit 220. In step S41, the data arrival monitoring unit 222 (see Figure 1) of the data transfer unit 220 is activated immediately before the data arrives, according to the data arrival schedule information managed by the sleep control unit 221 (see Figure 1).

[0113] Here, if data is received from the accelerator 120 or NIC 130 (see Figure 1) while the data arrival monitoring unit 222 is sleeping, a hardware interrupt may be activated when the data is received, and the data arrival monitoring unit 222 may be activated within this hardware interrupt handler. This method is effective for handling cases where data arrives at a time that deviates from the data arrival schedule managed by the sleep control management unit 210.

[0114] In step S42, the data arrival monitoring unit 222 monitors the Rx queues 122 and 131 (see Figure 1) of the accelerator 120 or NIC 130 to check for data arrival. At this time, regardless of whether data has arrived or not, it exclusively uses a CPU core to monitor for data arrival by polling. If an interrupt model is used here, the delay described in the conventional technology in Figure 22 occurs (i.e., when a softIRQ competes with another softIRQ, a wait occurs regarding the execution of the softIRQ, and this wait results in a network delay of the order of milliseconds). A key feature of this embodiment is that the Rx side uses sleep control of the polling model.

[0115] In step S43, the data arrival monitoring unit 222 determines whether or not data has arrived in the Rx queues 122 and 131.

[0116] If data has arrived in Rx queues 122 and 131 (S43: Yes), in step S44, the data arrival monitoring unit 222 plucks the data (queues) stored in Rx queues 122 and 131 (by referring to the contents of the packets accumulated in the buffer, processing the packets, and removing the corresponding queue entries from the buffer, taking into account the next processing to be performed), and transfers them to the Rx data transfer unit 223 (see Figure 1). If there is no data arrival in the Rx queues 122 and 131 (S43: No), the process returns to step S42.

[0117] In step S45, the Rx data transfer unit 223 transfers the received data to the data processing APL1 (see FIG. 1). Similar to the Tx data transfer unit 224 (see FIG. 1) described later, the Rx data transfer unit 223 operates only when data arrives, so it does not waste the CPU.

[0118] In step S46, if no data arrives even after the elapse of a certain period specified by the operator, the sleep control management unit 210 (see FIG. 1) puts the data arrival monitoring unit 222 (see FIG. 1) to sleep and ends the processing of this flow.

[0119] <Tx side> FIG. 10 is a flowchart showing the operation of the Tx data transfer unit 224 of the data transfer unit 220. In step S50 [external factor], the data processing APL1 (see FIG. 1) transfers data to the data transfer unit 220 of the in-server data transfer device 200 (see FIG. 1).

[0120] In step S51, the Tx data transfer unit 224 of the data transfer unit 220 stores the received data in the Tx queues 123 and 132 (see FIG. 1) of the accelerator 120 or NIC 130 (see FIG. 1) and ends the processing of this flow. The Tx data transfer unit 224 is activated by inter-process communication when the data processing APL1 sends data, and returns to CPU idle when the data transfer is completed. Therefore, unlike the data arrival monitoring unit 222 on the <Rx side>, it does not waste the CPU. The operation of the data transfer unit 220 has been described above.

[0121] [Example of handling when there is a difference in the data arrival schedule] Next, we will explain how to handle situations where there is a certain time difference between the data arrival schedule known by the sleep control management unit 210 and the actual data arrival schedule. This is a supplementary explanation for step S31 in Figure 8. This embodiment assumes a use case where the data arrival schedule for RAN or similar systems is predetermined. Data arrivals with inconsistent time differences are excluded because the RAN system (APL side) does not allow them.

[0122] <Case 1: When the schedule of the data transfer unit 220 is ahead of the actual data arrival date> Figure 11 is a flowchart showing the operation of the data transfer unit 220 when there is a difference in the data arrival schedule. In step S61, the data arrival monitoring unit 222 (see Figure 1) of the data transfer unit 220 monitors the Rx queues 122 and 131 (see Figure 1) of the accelerator 120 or NIC 130, and records the time difference △ (△ is used to represent the difference) T from the data arrival schedule to the actual data arrival in a memory not shown.

[0123] In step S62, the data arrival monitoring unit 222 (see Figure 1) transmits to the sleep control unit 221 (see Figure 1) that the data arrival schedule has advanced by △T if there are multiple consecutive data arrival differences of △T. The term "multiple consecutive" here is arbitrarily set by the system operator.

[0124] In step S63, the sleep control unit 221 (see Figure 1) of the data transfer unit 220 receives a message that the data arrival schedule has advanced by △T, and delays the data arrival schedule by △T, thereby ending the processing of this flow. This makes it possible to correct the schedule when the data arrival schedule is ahead by a certain amount of time.

[0125] <Case 2: When the schedule of the data transfer unit 220 is behind the actual data arrival date> Figure 12 is a flowchart showing the operation of the data transfer unit 220 when there is a difference in the data arrival schedule. In step S71, the data arrival monitoring unit 222 (see Figure 1) of the data transfer unit 220 monitors the Rx queues 122 and 131 (see Figure 1) of the accelerator 120 or NIC 130, and if data has already arrived in the first polling after data arrival monitoring is started, it records this in memory (not shown). Further explanation: The data arrival monitoring unit 222 is started just before data arrives (see the process in step S32 of Figure 8). However, even though it is just before, there is a time interval of just before = △t, and it is expected that polling will be performed for several cycles without any actual data arrival. Therefore, if data has already arrived after polling has started, it can be determined that there is a high possibility that the schedule of the data transfer unit 220 is behind.

[0126] In step S72, if the data arrival monitoring unit 222 detects that data has already arrived at the start of polling multiple times in a row, it instructs the sleep control unit 221 (see Figure 1) to delay the data arrival schedule by a small amount of time △S. At this point, it is not possible to know exactly how much the data arrival schedule is off, so the schedule is gradually adjusted by repeatedly delaying it by a small amount of time △S arbitrarily set by the operator.

[0127] In step S73, the sleep control unit 221 receives a message that the data arrival schedule should be advanced by △S, and advances the data arrival schedule by △S, completing the processing of this flow. By repeatedly performing this time correction of △S, it becomes possible to correct the schedule when there is a delay in the data arrival schedule for a certain period of time.

[0128] As explained above, the server data transfer system 1000 has the server data transfer device 200 located on user space 160. Therefore, like DPDK, the data transfer unit 220 of the server data transfer device 200 can bypass the kernel and refer to a ring-structured buffer (a ring-structured buffer created in the memory space managed by DPDK via DMA (Direct Memory Access) when a packet arrives at the accelerator 120 or NIC 130). In other words, the server data transfer device 200 does not use the ring buffer (Ring Buffer 72) (see Figure 22) or pole list (Ring Buffer 72) (see Figure 22) within the kernel. The data transfer unit 220 can instantly detect packet arrivals by having a polling thread constantly monitor the ring-structured buffer (mbuf; a ring-structured buffer on which PMD151 copies data via DMA) created in the memory space managed by this DPDK (i.e., it is a polling model, not an interrupt model).

[0129] In addition to being located on the user space 160 mentioned above, the server-internal data transfer device 200 has the following characteristics regarding the method of waking up the polling thread. In other words, for workloads where the data arrival timing is fixed, the in-server data transfer device 200 wakes up a polling thread using a timer based on the scheduling information (data arrival schedule information) for the data arrival timing. The in-server data transfer device 200B of the third embodiment described later (see Figure 17) has a polling thread in the kernel and wakes up the polling thread in response to a hardware interrupt from the NIC 11.

[0130] Further explanation will be provided regarding the operation of the server's internal data transfer device 200. <Normal operation: Polling mode> The server's internal data transfer device 200 monitors a ring buffer deployed in memory space from accelerator 120 or NIC 130 (see Figure 1) via a polling thread in user space 160. Specifically, the PMD151 (see Figure 25) of the server's internal data transfer device 200 is a driver that can select between polling mode and interrupt mode for data arrival. When data arrives at accelerator 120 or NIC 130, the PMD151 copies the data to a ring-structured buffer called mbuf in memory space using DMA. This ring-structured buffer mbuf is monitored by the polling thread in user space 160. Therefore, the server's internal data transfer device 200 does not use the poll_list prepared by the kernel. The above describes the normal operation (polling mode). Next, we will discuss the operation of unexpected interrupt modes.

[0131] <Unexpected behavior: Interrupt mode> The server's internal data transfer device 200 changes the mode of its driver (PMD151) so that when data arrives while the polling thread is sleeping, it can raise a hardware interrupt (hardIRQ) from the accelerator 120 or NIC 130 (see Figure 1). When data arrives at the accelerator 120 or NIC 130, the driver (PMD151) triggers a hardware interrupt, allowing the polling thread to wake up. Thus, the driver (PMD151) for the server's internal data transfer device 200 has two modes: polling mode and interrupt mode.

[0132] (Second Embodiment) Figure 13 is a schematic diagram of a server-internal data transfer system according to a second embodiment of the present invention. The same reference numerals are used for components identical to those in Figure 1, and the explanation of the redundant parts is omitted. As shown in FIG. 13, the in-server data transfer system 1000D includes HW110, OS140, and an in-server data transfer device 200A which is data high-speed transfer middleware arranged on the user space 160. Similar to the in-server data transfer device 200 in FIG. 1, the in-server data transfer device 200A consists of data high-speed transfer middleware. The in-server data transfer device 200A includes a sleep control management unit 210 and a data transfer unit 220A.

[0133] The data transfer unit 220A further includes a CPU frequency / CPU idle control unit 225 (CPU frequency control unit, CPU idle control unit) in addition to the configuration of the data transfer unit 220 in FIG. 13. The CPU frequency / CPU idle control unit 225 performs control to vary the CPU operation frequency and CPU idle setting. Specifically, the CPU frequency / CPU idle control unit 225 of the polling thread (in-server data transfer device 200A) started by the hardware interrupt handler sets the CPU operation frequency of the CPU core used by the polling thread lower compared to normal use.

[0134] Here, the kernel can change the operation frequency of the CPU core by the governor setting, and the CPU frequency / CPU idle control unit 225 can use the governor setting, etc. to set the CPU operation frequency lower compared to normal use. However, the CPU idle setting depends on the CPU model. In addition, when the CPU core has enabled the CPU idle setting, it is also possible to cancel it.

[0135] The operation of the in-server data transfer system 1000D will be described below. <Rx side> FIG. 14 is a flowchart showing the operation of the data arrival monitoring unit 222 of the data transfer unit 220A. The same step numbers are assigned to the parts that perform the same processing as the flowchart shown in FIG. 9, and the description of the overlapping parts is omitted. If the data arrival monitoring unit 222 (see Figure 13) is activated in step S41 just before the data arrives, then in step S81 the CPU frequency / CPU idle control unit 225 (see Figure 13) restores the operating frequency of the CPU core used by the data transfer unit 220A at that time (increases the CPU operating frequency of the CPU core). The CPU frequency / CPU idle control unit 225 also restores the CPU idle state setting (C-State, etc., which depends on the CPU architecture) and proceeds to step S42.

[0136] If the sleep control management unit 210 (see Figure 13) puts the data arrival monitoring unit 222 (see Figure 13) to sleep in step S46, the CPU frequency / CPU idle control unit 225 lowers the operating frequency of the CPU core used by the data transfer unit 220A in step S82. The CPU frequency / CPU idle control unit 225 also sets the CPU idle state (C-State, etc., which depends on the CPU architecture) and ends the processing of this flow by setting the corresponding CPU core to the CPU idle state.

[0137] Thus, the in-server data transfer device 200A has a data transfer unit 220A equipped with a CPU frequency / CPU idle control unit 225, and by setting the CPU frequency / CPU idle state in conjunction with the sleep control of the data arrival monitoring unit 222, it is possible to achieve further power savings. Note that the process of lowering the CPU frequency setting and the process of putting the system into sleep mode may be executed simultaneously. Alternatively, the system may be put into sleep mode only after confirming that packet forwarding is complete.

[0138] [Examples of application] The server data transfer devices 200 and 200A can be any server data transfer devices that launch a thread within the kernel to monitor packet arrivals using a polling model, and the OS is not limited. Furthermore, the environment is not limited to server virtualization. Therefore, the server data transfer systems 1000 to 1000D can be applied to the configurations shown in Figures 15 and 16.

[0139] <Example of Application to VM Configuration> FIG. 15 is a diagram showing an example in which the in-server data transfer system 1000E is applied to an interrupt model in a general-purpose Linux kernel (registered trademark) and server virtualization environment of a VM configuration. The same components as those in FIGS. 1, 13, and 19 are denoted by the same reference numerals. As shown in FIG. 15, the in-server data transfer system 1000E includes HW10, Host OS 20, in-server data transfer devices 200 and 200A which are data high-speed transfer middleware arranged on user space 160, virtual switch 184, and Guest OS 70.

[0140] Specifically, the server includes a Host OS 20 in which a virtual machine and an external process formed outside the virtual machine can operate, and a Guest OS 70 operating in the virtual machine. Host OS 20 includes Kernel 91, Ring Buffer 22 (see FIG. 19) managed by Kernel 91 in the memory space of the server including Host OS 20, poll_list 86 (see FIG. 22) for registering network device information indicating which device the hardware interrupt (hardIRQ) from NIC 11 belongs to, vhost-net module 221A (see FIG. 19) which is a kernel thread, tap device 222A (see FIG. 19) which is a virtual interface created by Kernel 91, and virtual switch (br) 223A (see FIG. 19).

[0141] On the other hand, Guest OS 70 includes Kernel 181, Driver 73, Ring Buffer 52 (see FIG. 19) managed by Kernel 181 in the memory space of the server including Guest OS 70, and poll_list 86 (see FIG. 22) for registering network device information indicating which device the hardware interrupt (hardIRQ) from NIC 11 belongs to.

[0142] The server-internal data transfer system 1000E has server-internal data transfer devices 200 and 200A located on user space 160. Therefore, like DPDK, the data transfer section 220 of the server-internal data transfer devices 200 and 200A can bypass the kernel and refer to a ring-structured buffer. In other words, the server-internal data transfer devices 200 and 200A do not use the ring buffer (Ring Buffer 72) (see Figure 22) or pole list (Ring Buffer 72) (see Figure 22) within the kernel. The data transfer unit 220 can bypass the kernel and access a ring-structured buffer (Ring Buffer 72) (mbuf; a ring-structured buffer on which PMD151 copies data via DMA), allowing it to instantly detect packet arrivals (i.e., it is a polling model, not an interrupt model).

[0143] By doing this, in a virtual server configuration system using VMs, low latency is achieved by bypassing the kernel via polling mode and performing low-latency packet forwarding in both Host OS20 and Guest OS70 when data arrives. Furthermore, power saving is achieved by stopping data arrival monitoring and entering sleep mode when no data arrives. As a result, both low latency and power saving can be achieved by controlling sleep mode with timer control that takes data arrival timing into consideration. Additionally, packet forwarding can be performed with reduced latency within the server without modifying the APL.

[0144] <Example of application to container configuration> Figure 16 shows an example of applying the server data transfer system 1000B to the interrupt model in a containerized server virtualization environment. The same reference numerals are used for components identical to those in Figure 15. As shown in Figure 16, the server data transfer system 1000F includes a Guest OS 180 and a container configuration in which the OS is replaced by Container 210A. Container 210A has a vNIC (virtual NIC) 211A. The server data transfer devices 200 and 200A are located on user space 160.

[0145] In systems with virtual server configurations such as containers, sleep control can be achieved by using timer control that takes data arrival timing into consideration, thereby achieving both low latency and low power consumption. Furthermore, packet forwarding can be performed with reduced latency within the server without modifying the APL (Application Program Line).

[0146] <Example of application to pair-metal configuration (non-virtualized configuration)> This invention can be applied to non-virtualized systems, such as pair-metal configurations. In non-virtualized systems, sleep control can be achieved by using timer control that takes data arrival timing into consideration, thereby achieving both low latency and low power consumption. Furthermore, packet forwarding can be performed with reduced latency within the server without modifying the APL.

[0147] <Advanced Technology> This invention enables scaling out to handle network load by increasing the number of CPUs allocated to a packet arrival monitoring thread in conjunction with RSS (Receive-Side Scaling), which allows multiple CPUs to process inbound network traffic when the number of traffic flows increases.

[0148] <Example of application to a network system with a fixed data arrival schedule> As an example of a network system with a fixed data arrival schedule, it can also be applied to the high-speed packet forwarding processing unit in network systems where data arrival timing must be guaranteed, such as the Time Aware Shaper (TAS) in a Time Sensitive Network (TSN). In network systems with a fixed data arrival schedule, it is possible to achieve both low latency and low power consumption.

[0149] (Third embodiment) In the first and second embodiments, the server data transfer devices 200 and 200A are located on the user space 160. In the third embodiment, instead of the server data transfer devices 200 and 200A located on the user space 160, a server data transfer device 200B is provided within the kernel, which uses a polling thread to perform sleep control.

[0150] Figure 17 is a schematic diagram of a server-internal data transfer system according to a third embodiment of the present invention. Components identical to those in Figures 1, 13, and 21 are denoted by the same reference numerals, and the explanation of redundant parts is omitted. This embodiment is an example applied to packet processing using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6 and later. Note that if a polling thread is implemented within the kernel, the kernel version must be considered when using NAPI as the basis.

[0151] As shown in Figure 17, the server data transfer system 1000G comprises HW10, OS70, and a server data transfer device 200B located within the Kernel71 of OS70. More specifically, the data transfer unit 220 of the server data transfer device 200B exists only within the kernel71, and the sleep control management unit 210 of the server data transfer device 200B only needs to exist in either the user space160 or within the kernel71 (the sleep control management unit 210 may be located in either the user space160 or within the kernel71). Figure 17 shows an example in which the data transfer unit 220 and the sleep control management unit 210 (i.e., the server data transfer device 200B) are located within the kernel71.

[0152] Here, if we adopt a configuration in which the server data transfer device 200B, which performs sleep control, is placed inside kernel71, the server data transfer devices 200 and 200A placed on space160 become unnecessary (in this case, considering general-purpose operation, it is also possible to place the server data transfer devices 200 and 200A in the server data transfer system and adaptively not use the server data transfer devices 200 and 200A). The reason why the server data transfer devices 200 and 200A become unnecessary will be explained. In other words, software interrupts that cause delay problems only occur inside kernel71 if DPDK is not used, and if DPDK is not used, data is transferred to data processing APL1 without interrupts using socket75. For this reason, data can be transferred to data processing APL1 at high speed even without the server data transfer devices 200 and 200A on user space160.

[0153] OS70 includes Kernel71, a Ring Buffer22 (see Figure 19) managed by Kernel71 in the memory space of the server equipped with OS70, poll_list86 (see Figure 22) which registers network device information indicating which device the hardware interrupt (hardIRQ) from NIC11 belongs to, a kernel thread vhost-net module221A (see Figure 19), a tap device222A (see Figure 19) which is a virtual interface created by Kernel91, and a virtual switch(br)223A (see Figure 19). As described above, the in-server data transfer device 200B has at least the data transfer unit 220 (see Figure 1) located within the Kernel 71 of OS 70.

[0154] The data transfer unit 220 of the server's internal data transfer device 200B has a data arrival monitoring unit 222 (see Figure 1) for monitoring the arrival of data from the interface unit (NIC 11). When data arrives from the interface unit, the interface unit copies the arrived data into memory space without using the CPU via DMA (Direct Memory Access) and arranges this data in a ring-configured buffer. The data arrival monitoring unit 222 starts a thread that monitors packet arrivals using a polling model and detects data arrival by monitoring the ring-configured buffer.

[0155] Specifically, the data transfer unit 220 of the server's internal data transfer device 200B has an OS (OS70) which includes a kernel (Kernel71), a ring buffer (Ring Buffer72) managed by the kernel in the memory space of the server equipped with the OS, and a poll list (poll_list86) (see Figure 22) which registers network device information indicating which device the hardware interrupt (hardIRQ) from the interface unit (NIC11) belongs to. Within the kernel, a thread is launched to monitor packet arrivals using a polling model.

[0156] Thus, the data transfer unit 220 of the server's internal data transfer device 200B includes a data arrival monitoring unit 222 that monitors (polling) the poll list, an Rx data transfer unit (packet harvesting unit) 223 that, if a packet has arrived, refers to the packet held in the ring buffer and performs harvesting by deleting the corresponding queue entry from the ring buffer based on the next processing to be performed, and a sleep control unit 221 that puts the thread (polling thread) to sleep if no packet has arrived for a predetermined period of time, and wakes the thread (polling thread) from sleep by a hardware interrupt (hardIRQ) when a packet arrives.

[0157] In this way, the server's internal data transfer device 200B stops the software interrupt (softIRQ) for packet processing, which is the main cause of network delays. The data arrival monitoring unit 222 of the server's internal data transfer device 200B executes a thread to monitor packet arrivals, and the Rx data transfer unit (packet harvesting unit) 223 processes packets using a polling model (without softIRQ) when they arrive. The sleep control unit 221 then puts the thread (polling thread) to sleep if no packets arrive for a predetermined period of time, so the thread (polling thread) sleeps when no packets arrive. The sleep control unit 221 wakes the thread from sleep using a hardware interrupt (hardIRQ) when a packet arrives.

[0158] As described above, the server data transfer system 1000G includes a server data transfer device 200B that has a polling thread in the kernel, and the data transfer unit 220 of the server data transfer device 200B wakes up the polling thread in response to a hardware interrupt from the NIC 11. In particular, the data transfer unit 220 is characterized by waking up the polling thread using a timer when a polling thread is provided in the kernel. As a result, the server delay control device 200B can achieve both low latency and low power consumption by managing the sleep state of the polling thread that performs packet transfer processing.

[0159] [Hardware configuration] The in-server data transfer devices 200, 200A, and 200B according to each of the above embodiments are implemented by a computer 900 having a configuration such as that shown in Figure 18. Figure 18 is a hardware configuration diagram showing an example of a computer 900 that implements the functions of the in-server data transfer devices 200 and 200A. Computer 900 has a CPU 901, ROM 902, RAM 903, HDD 904, communication interface (I / F: Interface) 906, input / output interface (I / F) 905, and media interface (I / F) 907.

[0160] The CPU 901 operates based on programs stored in the ROM 902 or HDD 904, and controls the respective parts of the server data transfer devices 200, 200A, and 200B shown in Figures 1 and 13. The ROM 902 stores boot programs executed by the CPU 901 when the computer 900 starts up, as well as programs that depend on the computer 900's hardware.

[0161] The CPU 901 controls input devices 910, such as a mouse or keyboard, and output devices 911, such as a display, via the input / output interface 905. The CPU 901 acquires data from the input devices 910 via the input / output interface 905 and outputs the generated data to the output devices 911. In addition to the CPU 901, a GPU (Graphics Processing Unit) or the like may also be used as a processor.

[0162] HDD904 stores programs executed by CPU901 and data used by those programs. Communication I / F906 receives data from other devices via a communication network (e.g., NW(Network)920) and outputs it to CPU901, and also transmits data generated by CPU901 to other devices via the communication network.

[0163] The media interface 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads the program related to the desired processing from the recording medium 912 onto the RAM 903 via the media interface 907 and executes the loaded program. The recording medium 912 can be an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto Optical Disk), a magnetic recording medium, a conductive memory tape medium, or a semiconductor memory.

[0164] For example, when computer 900 functions as a server data transfer device 200, 200A, or 200B configured as one of the devices according to this embodiment, the CPU 901 of computer 900 realizes the function of the server data transfer device 100 by executing a program loaded onto RAM 903. The HDD 904 stores the data in RAM 903. The CPU 901 reads and executes a program related to the desired processing from the recording medium 912. Alternatively, the CPU 901 may read a program related to the desired processing from another device via a communication network (NW 920).

[0165] [effect] As described above, the server-internal data transfer device 200 controls data transfer of the interface unit (accelerator 120, NIC 130) in user space, and the OS (OS 70) has a kernel (Kernel 171), a ring buffer (mbuf; a ring-structured buffer on which PMD 151 copies data by DMA) in the memory space of the server equipped with the OS, and a driver (PMD 151) that can select between polling mode or interrupt mode for data arrival from the interface unit (accelerator 120, NIC 130), and a thread (polling) that monitors packet arrival using a polling model. The system includes a data transfer unit 220 that starts up a thread, and a sleep control management unit (sleep control management unit 210) that manages data arrival schedule information and distributes the data arrival schedule information to the data transfer unit 220 to control the sleep state of the data transfer unit 220. Based on the data arrival schedule information distributed by the sleep control management unit 210, the data transfer unit 220 puts the thread to sleep and activates a timer immediately before data arrival to wake the thread from sleep.

[0166] In this way, the sleep control management unit 210 controls the sleep / start timing of each data transfer unit 220 collectively in order to control the sleep of multiple data transfer units in accordance with the timing of data arrival. When data is arriving, the kernel is bypassed by polling mode and packet transfer is performed with low latency, thereby reducing latency. Also, when no data is arriving, data arrival monitoring is stopped and the unit goes to sleep, thereby saving power. As a result, by controlling sleep with timer control that takes data arrival timing into consideration, it is possible to achieve both low latency and power saving.

[0167] The server-internal data transfer device 200 can achieve low latency by implementing data transfer delays within the server using a polling model rather than an interrupt model. Specifically, the server-internal data transfer device 200, like DPDK, allows the data transfer unit 220 located in user space 160 to bypass the kernel and access a ring-structured buffer. By having a polling thread constantly monitor this ring-structured buffer, it is possible to instantly detect packet arrivals (this is a polling model, not an interrupt model).

[0168] Furthermore, for data flows with fixed data arrival timings, such as time-division multiplexed data flows in signal processing in vRAN, the data transfer unit 220's sleep control can be performed by considering the data arrival schedule. This reduces CPU usage while maintaining low latency, thereby achieving power savings. In other words, the problem of wasted CPU resources in the polling model can be addressed by using timer control that considers data arrival timing to perform sleep control, thereby achieving both low latency and power savings.

[0169] Furthermore, the Guest OS (Guest OS70) running within the virtual machine includes a kernel (Kernel171), a ring buffer (mbuf; a ring-structured buffer on which PMD151 copies data via DMA) in the memory space of the server equipped with the Guest OS, a driver (PMD151) that can select between polling mode or interrupt mode for data arrival from the interface unit (accelerator 120, NIC130), and a protocol processing unit 74 that performs protocol processing on packets for which harvesting has been performed. The data transfer unit 220 also includes a data transfer unit 220 that starts a polling thread to monitor packet arrival using a polling model, and a sleep control management unit (sleep control management unit 210) that manages data arrival schedule information and distributes this information to the data transfer unit 220 to control its sleep state. The data transfer unit 220 is characterized by putting the thread to sleep based on the data arrival schedule information distributed by the sleep control management unit 210, and activating a timer immediately before data arrival to wake the thread from sleep.

[0170] By doing so, in a VM virtual server configuration system, it is possible to reduce CPU usage while maintaining low latency for servers equipped with a Guest OS (Guest OS70), thereby achieving power savings.

[0171] Furthermore, the Host OS (Host OS20), on which virtual machines and external processes formed outside the virtual machine can run, includes a kernel (Kernel91), a ring buffer (mbuf; a ring-structured buffer on which PMD151 copies data via DMA) in the memory space of the server equipped with the Host OS, a driver (PMD151) that can select between polling mode and interrupt mode for data arrival from the interface unit (accelerator 120, NIC130), and a tap device 222A, which is a virtual interface created by the kernel (Kernel91), and a thread (polling) that monitors packet arrival using a polling model. The system includes a data transfer unit 220 that starts up a thread, and a sleep control management unit (sleep control management unit 210) that manages data arrival schedule information and distributes the data arrival schedule information to the data transfer unit 220 to control the sleep state of the data transfer unit 220. The data transfer unit 220 puts the thread to sleep based on the data arrival schedule information distributed by the sleep control management unit 210, and activates a timer immediately before data arrival to wake the thread from sleep.

[0172] By doing so, in a virtual server configuration system using VMs, it is possible to reduce CPU usage while maintaining low latency for servers equipped with a kernel (Kernel191) and a host OS (Host OS20), thereby achieving power savings.

[0173] Furthermore, the server data transfer device 200B is a system in which the OS (OS70) and the kernel (Kernel171) manage the ring buffer (Ring) in the memory space within the server equipped with the OS. The system comprises a buffer (72), a poll list (poll_list86) that registers information about the network device indicating which device a hardware interrupt (hardIRQ) from the interface unit (NIC11) belongs to, a data transfer unit 220 that starts a thread within the kernel to monitor packet arrivals using a polling model, a sleep control management unit (sleep control management unit 210) that manages the data arrival schedule, manages the data arrival schedule information, and distributes the data arrival schedule information to the data transfer unit 220 to control the sleep of the data transfer unit 220, and the data transfer unit 220 comprises a data arrival monitoring unit 222 that monitors (polling) the poll list, a packet harvesting unit (Rx data transfer unit 223) that, if a packet has arrived, refers to the packet held in the ring buffer and performs harvesting by deleting the corresponding queue entry from the ring buffer based on the next processing to be performed, and a thread (polling) based on the data arrival schedule information received from the sleep control management unit 210 The system includes a sleep control unit (sleep control unit 221) that puts a thread to sleep and wakes it up via a hardware interrupt (hardIRQ) when the sleep is released.

[0174] In this way, the server data transfer device 200B can achieve low latency by implementing data transfer delays within the server using a polling model rather than an interrupt model. In particular, for data flows where the data arrival timing is fixed, such as time-division multiplexed data flows in signal processing in vRAN, the sleep control of the data transfer unit 220 can be performed considering the data arrival schedule, thereby reducing CPU usage while maintaining low latency and achieving power savings. In other words, the problem of wasted CPU resources in the polling model can be addressed by sleep control using timer control that takes data arrival timing into account, thereby achieving both low latency and power savings.

[0175] The data transfer unit 220 puts the polling thread to sleep based on the data arrival schedule information received from the sleep control management unit 210, and when the sleep state is to be released, it is woken up by a hardware interrupt (hardIRQ). This will produce the effects described above, as well as the effects described in (1) and (2).

[0176] (1) Software interrupts (softIRQ) that cause delays upon packet arrival are disabled, and a polling model is implemented within the kernel (Kernel171). In other words, unlike the existing NAPI technology, the 1000G server data transfer system implements a polling model instead of the interrupt model, which is the main cause of network delays. Since packets are harvested immediately upon arrival without waiting, low-latency packet processing can be achieved.

[0177] (2) The polling thread in the server's data transfer device 200 operates as a kernel thread and monitors packet arrivals in polling mode. The kernel thread (polling thread) that monitors packet arrivals sleeps when no packets arrive. When no packets arrive, the CPU is not used by sleeping, resulting in power savings.

[0178] When a packet arrives, the polling thread, which is currently sleeping, is woken (released from sleep) by the hardIRQ handler that receives the packet. By waking the sleep in the hardIRQ handler, the polling thread can be started immediately while avoiding softIRQ contention. Here, the key feature is that the sleep release is triggered by the hardIRQ handler, not by a timer. If the traffic load is known in advance, for example, if a 30ms sleep is known as the workload transfer rate shown in Figure 23, the hardIRQ handler can be configured to wake the thread at this timing.

[0179] In this way, the server's internal data transfer device 200B can achieve both low latency and low power consumption by managing the sleep state of the polling thread that performs packet transfer processing.

[0180] The server data transfer device 200A is characterized by having a CPU frequency setting unit (CPU frequency / CPU idle control unit 225) that sets the CPU operating frequency of the CPU core used by a thread to a lower value during sleep mode.

[0181] In this way, the server's internal data transfer device 200A dynamically adjusts the CPU operating frequency to match the traffic. That is, if the CPU is not used due to sleep mode, the CPU operating frequency during sleep mode can be set lower, thereby further enhancing power saving.

[0182] The server data transfer device 200A is characterized by having a CPU idle setting unit (CPU frequency / CPU idle control unit 225) that sets the CPU idle state of the CPU core used by a thread to a power saving mode during sleep.

[0183] By doing so, the server's internal data transfer device 200A can further enhance power saving by dynamically changing the CPU idle state (power saving function depending on the CPU model, such as changing the operating voltage) in accordance with traffic.

[0184] Furthermore, among the processes described in each of the embodiments above, all or part of the processes described as being performed automatically may be performed manually, or all or part of the processes described as being performed manually may be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings may be changed at will unless otherwise specified. Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0185] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software that allows the processor to interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or an optical disc. [Explanation of symbols]

[0186] 1. Data Processing APL (Application Program) 2. Data Flow Time Slot Management Scheduler 3 PHY (High) 4 MAC 5 RLC 6 FAPI (FAPI P7) 20,70 Host OS (OS) 50 Guest OS (OS) 86 poll_list 72 Ring Buffer 91,171,181 Kernel 110 HW 120 Accelerator (Interface Unit) 121 cores (Core processor) 122,131 Rx queues 123,132 Tx queues 130 NICs (Physical NICs) (Interface Section) 140 OS 151 PMD (Driver that allows selection of data arrival mode: polling mode or interrupt mode) 160 user space 200, 200A, 200B Server In-Server Data Transfer Device 210 sleep control management section 210A Container 211 Data Transfer Department Management Section 212 Data Arrival Schedule Management Department 213 Data Arrival Schedule Distribution Department 220 Data Transfer Section 221 Sleep Control Unit 222 Data Arrival Monitoring Unit 223 Rx Data Transfer Unit (Packet Harvesting Unit) 224 Tx Data Transfer Section 225 CPU Frequency / CPU Idle Control Unit (CPU Frequency Control Unit, CPU Idle Control Unit) 1000, 1000A, 1000B, 1000C, 1000D, 1000E, 1000F, 1000G Server-internal data transfer system Mbuf is a ring-structured buffer that PMD uses to copy data via DMA.

Claims

1. A server-internal data transfer device equipped with a sleep control management unit that manages data arrival schedules and performs sleep control according to the timing of data arrival, The sleep control management unit, In a data flow that contains information about the timing of specific data arrivals, Based on the aforementioned data arrival timing, the polling model is used to put the thread that monitors data arrival to sleep. Based on the timing of the arrival of the aforementioned data, the operation to wake the aforementioned thread from sleep is performed. A server-internal data transfer device characterized by the following features.

2. The data flow having information about a specific data arrival timing in claim 1 is: This is signal processing in a RAN (Radio Access Network). A server-internal data transfer device characterized by the following features.

3. The specific data arrival timing in claim 2 is: This information is obtained from the MAC (Medium Access Control) scheduler information, which manages the transmission and reception of wireless signals in the RAN. A server-internal data transfer device characterized by the following features.

4. The MAC scheduler information in claim 3 is: This information is obtained via an interface that exchanges data schedule information between the PHY (Physical) layer and the MAC layer. A server-internal data transfer device characterized by the following features.

5. The specific data arrival timing in claim 2 is: The interface (IF) between the RAN device and the opposing transmission device is obtained from the messages being transmitted. A server-internal data transfer device characterized by the following features.

6. The data arrival timing in claim 2 is: This information is obtained from the interface that exchanges data schedule information with transmission devices in O-RAN. A server-internal data transfer device characterized by the following features.

7. A data transfer method for an in-server data transfer device that manages the data arrival schedule and performs sleep control according to the timing of data arrival, wherein The data transfer device within the server is In a data flow that contains information about the timing of specific data arrivals, The steps include: putting a thread that monitors data arrival using a polling model to sleep based on the aforementioned data arrival timing; The steps include: performing an action to wake the thread from sleep based on the timing of the arrival of the data; and executing the following steps. A method for transferring data within a server, characterized by the following features.

8. A program for causing a computer to function as a server data transfer device according to any one of claims 1 to 6.

Images