Intra-server data transfer device, intra-server data transfer method and program
The intra-server data transfer device manages polling threads based on traffic patterns to reduce CPU usage and power consumption, addressing latency and efficiency issues in virtualized server environments.
Patent Information
- Application Number
- JP2023565685
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-12-06
- Publication Date
- 2025-09-17
- Estimated Expiration
- 2041-12-06
AI Technical Summary
Existing packet transfer methods in virtualized server environments face issues of increased latency and high power consumption due to conflicts in interrupt processing and continuous CPU utilization, particularly in models like NAPI, DPDK, and KBP.
An intra-server data transfer device that employs a traffic characteristic information collection unit, a polling determination unit, and a polling instruction unit to manage polling threads based on traffic patterns, allowing them to sleep when packet arrival probability is below a threshold and wake up only when necessary, thereby reducing CPU usage and power consumption.
This approach reduces CPU usage and achieves power savings while maintaining low latency in packet transfer, optimizing resource utilization in virtualized server environments.
Smart Images

Figure 0007740368000001 
Figure 0007740368000002 
Figure 0007740368000003
Abstract
Description
[Technical Field]
[0001] The present invention relates to an intra-server data transfer device, an intra-server data transfer method, and a program. [Background technology]
[0002] With the advancement of virtualization technology such as NFV (Network Functions Virtualization), systems are being built and operated for each service.In addition, a system called Service Function Chaining (SFC) is becoming mainstream, which divides service functions into reusable modules and runs them in independent virtual machine (VM) environments (such as containers) to use them as components when needed, improving operability, rather than building systems for each service.
[0003] A known technology for configuring virtual machines is a hypervisor environment consisting of Linux (registered trademark) and KVM (kernel-based virtual machine). In this environment, a Host OS (an OS installed on a physical server is called a Host OS) with a built-in KVM module acts as a hypervisor and runs in a memory area called kernel space, which is different from user space. In this environment, a virtual machine runs in user space, and a Guest OS (an OS installed on a virtual machine is called a Guest OS) runs within that virtual machine.
[0004] Unlike a physical server running a Host OS, a virtual machine running a Guest OS controls all hardware (HW), including network devices (typically Ethernet (registered trademark) card devices), as registers required for interrupt processing from the HW to the Guest OS and for writing from the Guest OS to the hardware. In this type of register control, notifications and processing that should be performed by physical hardware are simulated by software, so performance is generally lower than in a Host OS environment.
[0005] To address this performance degradation, there is a technology that reduces HW emulation, particularly from the guest OS to the host OS or external processes that exist outside the virtual machine, and improves communication performance and versatility through a high-speed, unified interface. One such technology is a device abstraction technology called Virtio, or paravirtualization technology, which has been developed and is already incorporated into many general-purpose operating systems, including Linux (registered trademark) and FreeBSD (registered trademark) and is currently in use (see Patent Documents 1 and 2).
[0006] Virtio defines data exchange using queues designed with ring buffers as a unidirectional transport for data input / output such as console, file input / output, and network communication, through queue operations. By using the Virtio queue specifications to prepare the number and size of queues appropriate for each device when the Guest OS starts, communication between the Guest OS and outside the virtual machine can be achieved using only queue operations without performing hardware emulation.
[0007] Data transfer technologies within a server include New API (NAPI), DPDK (Data Plane Development Kit), and KBP (Kernel Busy Poll).
[0008] When a packet arrives, the New API (NAPI) processes the packet by issuing a hardware interrupt request and then a software interrupt request (see Non-Patent Document 1) (see FIG. 15 below).
[0009] DPDK implements packet processing functions in the user space where applications run, and immediately harvests packets as they arrive from the user space using a polling model (see Non-Patent Document 2). Specifically, DPDK is a framework for controlling NICs (Network Interface Cards), a function traditionally performed by the Linux kernel (registered trademark), in user space. The biggest difference from the Linux kernel's processing is the inclusion of a polling-based reception mechanism called PMD (Pull Mode Driver). Typically, in the Linux kernel, an interrupt is generated when data arrives at the NIC, triggering the execution of reception processing. In contrast, in PMD, a dedicated thread continuously checks for data arrival and performs reception processing. High-speed packet processing is possible by eliminating overhead such as context switches and interrupts. DPDK significantly improves packet processing performance and throughput, allowing more time to be devoted to data plane application processing. However, DPDK exclusively uses computer resources such as the CPU (Central Processing Unit) and NIC.
[0010] Non-Patent Document 3 describes an intra-server network delay control device (KBP: Kernel Busy Poll). KBP constantly monitors packet arrivals within the kernel using a polling model. This suppresses softIRQs and achieves low-latency packet processing.
[0011] [Rx-side packet processing using New API (NAPI)] FIG. 15 is a schematic diagram of packet processing on the Rx side using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6 and later (see Non-Patent Document 1). As shown in FIG. 15, the New API (NAPI) executes a data processing APL1 placed in a user space 60 available to a user on a server equipped with an OS 70 (e.g., a Host OS), and transfers packets between the NIC 11 of the HW 10 connected to the OS 70 and the data processing APL1.
[0012] The OS 70 includes a kernel 71 , a ring buffer 72 , and a driver 73 , and the kernel 71 includes a protocol processing unit 74 . Kernel 71 is a core function of OS 70 (for example, Host OS), and monitors hardware and manages the execution status of programs on a process-by-process basis. Here, kernel 71 responds to requests from data processing APL1 and also conveys requests from HW 10 to data processing APL1. Kernel 71 processes requests from data processing APL1 via a system call (a "user program running in non-privileged mode" requests processing from the "kernel running in privileged mode"). The kernel 71 transmits a packet to the data processing APL1 via the socket 75. The kernel 71 receives a packet from the data processing APL1 via the socket 75.
[0013] The Ring Buffer 72 is located in the memory space of the server and is managed by the Kernel 71. The Ring Buffer 72 is a buffer of a certain size that stores messages output by the Kernel 71 as logs, and when the upper limit size is exceeded, the messages are overwritten from the beginning.
[0014] Driver73 is a device driver that monitors hardware using kernel71. Note that Driver73 depends on kernel71, and if the created (built) kernel source changes, it will become a different driver. In this case, you will need to obtain the relevant driver source and rebuild it on the OS that uses the driver to create the driver.
[0015] The protocol processing unit 74 performs protocol processing of L2 (data link layer), L3 (network layer), and L4 (transport layer) defined by the OSI (Open Systems Interconnection) reference model.
[0016] Socket75 is an interface that kernel71 uses for inter-process communication. Socket75 has a socket buffer and does not require frequent data copying. The process for establishing communication via Socket75 is as follows: 1. The server creates a socket file that accepts clients. 2. A name is given to the accepting socket file. 3. A socket queue is created. 4. The first connection from the client in the socket queue is accepted. 5. A socket file is created on the client side. 6. The client issues a connection request to the server. 7. On the server side, a connection socket file is created separately from the accepting socket file. As a result of establishing communication, data processing APL1 can call system calls such as read() and write() on kernel71.
[0017] In the above configuration, the Kernel 71 receives notification of the arrival of a packet from the NIC 11 via a hardware interrupt (hardIRQ), and schedules a software interrupt (softIRQ) for packet processing. The New API (NAPI) implemented in Linux kernel 2.5 / 2.6 above processes packets using a hardware interrupt (hardIRQ) and then a software interrupt (softIRQ) when a packet arrives. As shown in Figure 15, packet transfer using the interrupt model transfers packets using interrupt processing (see symbol a in Figure 15), which causes a wait for the interrupt processing and increases the delay in packet transfer.
[0018] The following is an overview of packet processing on the NAPI Rx side. [Rx side packet processing configuration using New API (NAPI)] FIG. 16 is a diagram for explaining an outline of packet processing on the Rx side by New API (NAPI) in the area surrounded by the dashed line in FIG. <Device driver> As shown in Figure 16, the Device driver includes NIC11 (physical NIC), which is a network interface card, hardIRQ81, which is a handler that is called when a processing request is made to NIC11 and executes the requested processing (hardware interrupt), and netif_rx82, which is a processing function unit for software interrupts.
[0019] <Networking layer> The Networking layer includes softIRQ 83, a handler that is called when a processing request from netif_rx 82 occurs and executes the requested processing (software interrupt), and do_softirq 84, a control function unit that executes the software interrupt (softIRQ). Also included are net_rx_action 85, a packet processing function unit that receives and executes the software interrupt (softIRQ), poll_list 86, which registers information about the net device (net_device) that indicates which device the hardware interrupt from the NIC 11 belongs to, netif_receive_skb 87, which creates the sk_buff structure (a structure that allows the Kernel 71 to recognize the status of the packet), and Ring Buffer 72.
[0020] <Protocol layer> The protocol layer includes packet processing function units such as ip_rcv88 and arp_rcv89.
[0021] The above netif_rx 82, do_softirq 84, net_rx_action 85, netif_receive_skb 87, ip_rcv 88, and arp_rcv 89 are program components (function names) used for packet processing in the Kernel 71.
[0022] [Rx side packet processing operation using New API (NAPI)] Arrows (symbols) b to m in FIG. 16 indicate the flow of packet processing on the Rx side. When the hardware function unit 11a of the NIC 11 (hereinafter referred to as NIC 11) receives a packet (or frame) in a frame from the other device, it copies the arriving packet to the Ring Buffer 72 by DMA (Direct Memory Access) transfer without using the CPU (see symbol b in Figure 16). This Ring Buffer 72 is a memory space in the server and is managed by the Kernel 71 (see Figure 15).
[0023] However, if the NIC 11 simply copies the packet that has arrived to the Ring Buffer 72, the Kernel 71 will not be able to recognize the packet. Therefore, when a packet arrives, the NIC 11 raises a hardware interrupt (hardIRQ) to hardIRQ 81 (see symbol c in Figure 16), and netif_rx 82 executes the following process, which allows the Kernel 71 to recognize the packet. Note that hardIRQ 81, shown enclosed in an oval in Figure 16, represents a handler rather than a functional unit.
[0024] Netif_rx 82 is the function that actually performs the processing. When hardIRQ 81 (handler) is started (see symbol d in Figure 16), it saves net_device information, which is one of the pieces of information contained in the hardware interrupt (hardIRQ), indicating which device the hardware interrupt from the NIC 11 belongs to, in poll_list 86, and registers queue pruning (see symbol g in Figure 16). Here, queue pruning refers to referencing the contents of packets stored in the buffer, and deleting the corresponding queue entry from the buffer, taking into account the next processing to be performed on that packet. Specifically, when a packet is stuffed into the Ring Buffer 72, netif_rx 82 uses the driver of the NIC 11 to register future queue pruning in poll_list 86 (see symbol e in Figure 16). As a result, queue pruning information resulting from the stuffing of packets into the Ring Buffer 72 is registered in poll_list 86.
[0025] In this way, in Figure 16<Device driver> In the above example, when the NIC 11 receives a packet, it copies the packet that has arrived to the Ring Buffer 72 by DMA transfer. The NIC 11 also raises the hardIRQ 81 (handler), and the netif_rx 82 registers the net_device in the poll_list 86 and schedules a software interrupt (softIRQ). So far, we have completed the steps in Figure 16.<Device driver> Processing of hardware interrupts in the suspend state stops.
[0026] Then, netif_rx82 uses the information (specifically, the pointer) in the queue stored in poll_list86 to notify softIRQ83 (handler) via a software interrupt (softIRQ) that it will harvest the data stored in Ring Buffer72 (see symbol f in Figure 16), and notifies do_softirq84, which is the software interrupt control function unit (see symbol g in Figure 16).
[0027] do_softirq84 is a software interrupt control function unit that defines each software interrupt function (there are various types of packet processing, and interrupt processing is one of them. This defines the interrupt processing). Based on this definition, do_softirq84 notifies net_rx_action85, which actually processes the software interrupt, of the current (relevant) software interrupt request (see symbol h in Figure 16).
[0028] When the turn of the softIRQ comes, net_rx_action 85 calls a polling routine for reaping packets from Ring Buffer 72 based on net_device registered in poll_list 86 (see symbol i in FIG. 16), and reaps the packets (see symbol j in FIG. 16). At this time, net_rx_action 85 continues reaping until poll_list 86 becomes empty. Thereafter, net_rx_action 85 notifies netif_receive_skb 87 (see symbol k in FIG. 16).
[0029] The netif_receive_skb 87 creates an sk_buff structure, analyzes the contents of the packet, and sends the processing to the subsequent protocol processing unit 74 (see FIG. 15) for each type. That is, the netif_receive_skb 87 analyzes the contents of the packet, and when processing is to be performed according to the contents of the packet,<Protocol layer> If it is L2, the process is passed to arp_rcv 89 (symbol m in FIG. 16).
[0030] Figure 17 shows an example of video (30 FPS) data transfer. The workload shown in Figure 17 has a transfer rate of 350 Mbps, and data is transferred intermittently every 30 ms.
[0031] FIG. 18 is a diagram showing the CPU utilization rate used by the busy poll thread in the KBP described in Non-Patent Document 3. As shown in Fig. 18, in KBP, the kernel thread occupies a CPU core to perform busy poll. Even in the case of intermittent packet reception as shown in Fig. 17, KBP constantly uses the CPU regardless of whether packets arrive, which poses a problem of increased power consumption.
[0032] Next, we will explain the DPDK system. [DPDK system configuration] FIG. 19 is a diagram illustrating the configuration of a DPDK system that controls the HW 110 that includes the accelerator 120. The DPDK system includes HW 110, OS 140, DPDK 150, which is high-speed data transfer middleware arranged on user space 160, and data processing APL 1. The data processing APL1 is packet processing that is performed prior to the execution of the APL. The HW 110 communicates with the data processing APL 1 to send and receive data. In the following description, as shown in Fig. 19, the data flow in which the data processing APL 1 receives packets from the HW 110 is referred to as Rx-side reception, and the data flow in which the data processing APL 1 transmits packets to the HW 110 is referred to as Tx-side transmission.
[0033] The HW 110 includes an accelerator 120 and a NIC 130 (physical NIC) for connecting to a communication network. The accelerator 120 is a computation unit hardware that performs specific calculations at high speed based on input from the CPU. Specifically, the accelerator 120 is a PLD (Programmable Logic Device) such as a GPU (Graphics Processing Unit) or an FPGA (Field Programmable Gate Array). In Fig. 19, the accelerator 120 includes a plurality of Cores (Core processors) 121, Rx queues (queues) 122 that hold data in a first-in, first-out list structure, and Tx queues 123.
[0034] A part of the processing of the data processing APL1 is offloaded to the accelerator 120, thereby achieving performance and power efficiency that cannot be achieved by software (CPU processing) alone. It is conceivable that the accelerator 120 described above will be applied to a large-scale server cluster, such as a data center that implements NFV (Network Functions Virtualization) or SDN (Software Defined Network).
[0035] The NIC 130 is NIC hardware that realizes a network interface, and includes an Rx queue 131 and a Tx queue 132 that hold data in a first-in, first-out list structure. The NIC 130 is connected to an associated device 170 via, for example, a communication network, and transmits and receives packets. The NIC 130 may be, for example, a SmartNIC, which is a NIC with an accelerator. The SmartNIC is a NIC that can reduce the load on the CPU by offloading load-intensive processing, such as IP packet processing, which can cause a decrease in processing power.
[0036] DPDK 150 is a framework for controlling NICs in user space 160, and specifically consists of high-speed data transfer middleware. DPDK 150 has a polling-based reception mechanism, PMD (Poll Mode Driver) 151 (a driver that can select polling mode or interrupt mode for data arrival). PMD 151 has a dedicated thread that continuously checks for data arrival and performs reception processing.
[0037] DPDK150 realizes packet processing functions in user space 160 where APL runs, and reduces packet transfer delays by immediately reaping packets when they arrive from user space 160 using a polling model. In other words, DPDK150 reaps packets by polling (busy polling the queue in the CPU), so there is no waiting and delays are small. [Prior art documents] [Patent documents]
[0038] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-197874 [Patent Document 2] Japanese Patent Application Publication No. 2018-32156 [Non-patent literature]
[0039] [Non-Patent Document 1] New API Intel, [online], [Retrieved November 5, 2021], Internet 〈http: / / lwn.net / 2002 / 0321 / a / napi-howto.php3〉 [Non-patent document 2] “Resource Configuration (NIC) ~Introduction to DPDK Part 6~,” NTT Technocross, [online], [Retrieved November 5, 2021], Internet 〈https: / / www.ntt-tx.co.jp / column / dpdk_blog / 190610 / 〉 [Non-patent document 3] Kei Fujimoto, Kenichi Matsui, Masayuki Akutsu, “KBP: Kernel Enhancements for Low-Latency Networking without Application Customization in Virtual Server”, IEEE CCNC 2021. Summary of the Invention [Problem to be solved by the invention]
[0040] However, both packet transfer using the interrupt model and the polling model have the following problems. In the interrupt model, packets are transferred by software interrupt processing, where the kernel processes packets after receiving an event (hardware interrupt) from the hardware. Because the interrupt model transfers packets by interrupt (software interrupt) processing, there are issues such as conflicts with other interrupts and waiting times when the interrupt destination CPU is being used by a higher priority process, which can increase packet transfer delays. In this case, if the interrupt processing becomes congested, the waiting delays will increase even more.
[0041] The mechanism by which delay occurs in the interrupt model will be explained below. In a typical kernel, packet transfer processing is performed by software interrupt processing after hardware interrupt processing. When a software interrupt occurs in packet forwarding processing, the software interrupt processing cannot be executed immediately under the following conditions (1) to (3). Therefore, arbitration is performed by a scheduler such as ksoftirqd (a kernel thread for each CPU that is executed when the software interrupt load becomes high), and the interrupt processing is scheduled, causing a wait on the order of milliseconds. (1) When there is a conflict with other hardware interrupt processing (2) When there is a conflict with other software interrupt processing (3) When other high-priority processes, kernel threads (migration threads, etc.), or interrupt destination CPUs are in use Under the above conditions, the software interrupt process cannot be executed immediately.
[0042] <Network delays caused by interrupt processing contention> When a software interrupt for packet processing conflicts with a software interrupt for other processing (disk I / O, swap, etc.), the software interrupt scheduler (ksoftirqd) arbitrates the execution frequency of the software interrupt. Describe the occurrence of software interrupt conflicts. Ksoftirqd schedules software interrupt processing. When the processing is transferred to ksoftirqd, software interrupts are executed at the frequency of config_hz (about ms). Due to this waiting time, a packet transfer delay on the order of ms occurs.
[0043] <Issues with New API (NAPI)> Similarly, for packet processing by New API (NAPI), as shown in the dashed box n in Figure 16, due to the conflict of interrupt processing (softIRQ), a NW delay on the order of ms occurs.
[0044] <Issues with DPDK> In DPDK, since the kernel thread performs polling (busy poll on the queue with the CPU), it monopolizes the CPU core. So, even for the intermittent packet reception shown in Figure 17, in DPDK, regardless of whether packets arrive or not, the CPU is always used at 100%, resulting in a problem of high power consumption.
[0045] <Issues with KBP> KBP also has the same issues as the above-mentioned DPDK. As described above, KBP can suppress softIRQ and achieve low-latency packet processing by constantly monitoring packet arrival in the kernel using the polling model. However, since the kernel thread that constantly monitors packet arrival monopolizes the CPU core and always uses CPU time, there is a problem of high power consumption. Refer to Figures 17 and 18 to explain the relationship between the workload and CPU utilization. As shown in Figure 18, in KBP, the kernel thread monopolizes the CPU core to perform busy poll. Even for the intermittent packet reception shown in Figure 17, in KBP, regardless of whether packets arrive or not, the CPU is always used, resulting in a problem of high power consumption.
[0046] As such, DPDK implements a polling model in user space, so softIRQ contention does not occur, and KBP implements a polling model within the kernel, so softIRQ contention does not occur, enabling low-latency packet transfer.However, both DPDK and KBP have the problem of wasting CPU resources by constantly monitoring packet arrival, regardless of whether packets have arrived, resulting in high power consumption.
[0047] The present invention has been made in view of this background, and an object of the present invention is to enable power saving by reducing CPU utilization while maintaining low latency. [Means for solving the problem]
[0048] In order to solve the above-mentioned problems, an intra-server data transfer device that performs sleep control on a Polling thread of a data transfer unit that performs packet transfer according to a Polling model comprises: a traffic characteristic information collection unit that collects traffic characteristic information in the packet transfer; a Polling determination unit that generates Polling schedule information based on the traffic characteristic information received from the traffic characteristic information collection unit, for waking up the Polling thread at a timing when the packet arrival probability is equal to or greater than a predetermined threshold and for causing the Polling thread to sleep at other times; and a Polling instruction unit that receives time synchronization information and the Polling schedule information from the Polling determination unit, and distributes a Polling schedule that indicates the time and duration required for Polling to the Polling thread based on the time synchronization information and the Polling schedule information, and causes the Polling thread to poll for the duration instructed by the Polling schedule. The traffic characteristic information collecting unit collects channel allocation information in the time and frequency domains from an upper application and transmits the information to the polling determining unit as the traffic characteristic information. The data transfer device within the server is characterized by the above. [Effects of the Invention]
[0049] According to the present invention, it is possible to reduce CPU usage and achieve power saving while maintaining low latency. [Brief explanation of the drawings]
[0050] [Figure 1] 1 is a schematic configuration diagram of an intra-server data transfer system according to an embodiment of the present invention; [Figure 2] 10A and 10B are diagrams illustrating examples of channels, characteristics of data arrival, and sleep control countermeasures by a polling determination unit according to examples of data arrival patterns in a server data transfer system according to an embodiment of the present invention. [Figure 3] 10 is a diagram showing an example of channel allocation information in the time-frequency domain obtained from a traffic characteristic information collection unit of the intra-server data transfer system according to the embodiment of the present invention. FIG. [Figure 4] FIG. 2 is a diagram illustrating the operation of the server data transfer system according to the embodiment of the present invention. [Figure 5] 10 is a diagram illustrating the acquisition of traffic characteristics of packet arrival statistics in cooperation with high-speed data transfer middleware in the intra-server data transfer system according to an embodiment of the present invention. FIG. [Figure 6] 1 is a flowchart showing a traffic characteristic information collection process 1 of a traffic characteristic information collection unit of the intra-server data transfer system according to the embodiment of the present invention. [Figure 7] 10 is a flowchart showing traffic characteristic information collection processing 2 of the traffic characteristic information collection unit of the intra-server data transfer system according to the embodiment of the present invention. [Figure 8] 10 is a flowchart showing a sleep control schedule determination process in a polling determination unit of the server data transfer system according to the embodiment of the present invention. [Figure 9] 10A to 10C are diagrams illustrating examples of data characteristics, what can be inferred from the characteristics, and sleep control countermeasures by a polling determination unit according to examples of statistical information of an intra-server data transfer system according to an embodiment of the present invention. [Figure 10] 10 is a flowchart showing a polling schedule information distribution process of a polling determination unit of the intra-server data transfer system according to the embodiment of the present invention. [Figure 11] 10 is a subroutine showing logic for determining whether or not polling is possible in a polling determination unit of the intra-server data transfer system according to the embodiment of the present invention. [Figure 12] 10 is a flowchart showing the operation of a polling instruction unit of the intra-server data transfer system according to the embodiment of the present invention. [Figure 13] 10 is a flowchart showing the operation of high-speed data transfer middleware in the intra-server data transfer system according to the embodiment of the present invention. [Figure 14] 1 is a hardware configuration diagram showing an example of a computer that realizes the functions of an intra-server data transfer device of an intra-server data transfer system according to an embodiment of the present invention. [Figure 15] This is a schematic diagram of Rx-side packet processing using the New API (NAPI) implemented in Linux kernel 2.5 / 2.6. [Figure 16] 16 is a diagram for explaining an outline of Rx-side packet processing by New API (NAPI) in the area surrounded by the dashed line in FIG. 15. FIG. [Figure 17] FIG. 10 is a diagram showing an example of video (30 FPS) data transfer. [Figure 18] FIG. 10 is a diagram showing the CPU utilization rate used by a busy poll thread in KBP described in Non-Patent Document 3. [Figure 19] FIG. 1 is a diagram illustrating the configuration of a DPDK system that controls HW equipped with an accelerator. DETAILED DESCRIPTION OF THE INVENTION
[0051] Hereinafter, an intra-server data transfer system and the like in an embodiment for carrying out the present invention (hereinafter referred to as "the present embodiment") will be described with reference to the drawings. (Embodiment) [Overall configuration] 1 is a schematic diagram of an intra-server data transfer system according to an embodiment of the present invention, in which the same components as those in FIG. 19 are assigned the same reference numerals. As shown in Figure 1, the server internal data transfer system 1000 includes HW 110, memory 180, high-speed data transfer middleware 300 (data transfer unit) that performs packet transfer using a polling model, a server internal data transfer device 200 that is sleep control middleware that performs sleep control on the polling thread of the high-speed data transfer middleware 300, and Applications 2 (high-level applications) that are programs executed in the user space.
[0052] The HW 110 includes a NIC 130 (physical NIC) for connecting to the counterpart device 170. The NIC 130 is NIC hardware that realizes a NW interface, and is connected to the counterpart device 170 via a communication network, for example, to transmit and receive packets. The HW 110 communicates with the Applications 2 by sending and receiving data. The data flow in which the Applications 2 receives packets from the HW 110 is called Rx-side reception, and the data flow in which the Applications 2 transmits packets to the HW 110 is called Tx-side transmission.
[0053] The HW 110 may include an accelerator, which is a calculation unit hardware that performs specific calculations at high speed based on input from the CPU, such as a PLD (Programmable Logic Device) such as a GPU (Graphics Processing Unit) or an FPGA (Field Programmable Gate Array).
[0054] The memory 180 is a register that stores an Rx queue 181 and a Tx queue 182 that hold data in a first-in, first-out list structure.
[0055] [Intra-server data transfer device 200] The intra-server data transfer device 200 comprises a traffic characteristic information collection unit 210, a polling determination unit 220, and a polling instruction unit 230.
[0056] <Traffic characteristic information collection unit 210> The traffic characteristic information collection unit 210 collects traffic characteristic information (including channel allocation information (described later) and statistical information (such as packet arrival information in the past) that can be used in the Sleep control schedule), and transmits it to the Polling determination unit 220.
[0057] Specifically, the traffic characteristic information collection unit 210 collects channel allocation information from Applications2 to the time-frequency domain (see reference symbol s in FIG. 4), and transmits it to the Polling determination unit 220 as traffic characteristic information (see reference symbol o in FIG. 1). In addition, the traffic characteristic information collection unit 210 cooperates with the data high-speed transfer middleware 300, collects information that can be used in the Sleep control schedule, such as the characteristics of packet arrival frequency over time, from the data high-speed transfer middleware 300, and transmits it to the Polling determination unit 220 as traffic characteristic information (see reference symbol o in FIG. 1).
[0058] Note that an example where the traffic characteristic information collection unit 210 collects channel allocation information (traffic characteristic information) from Applications2 is shown by reference symbol s in FIG. 4 described later, and an example where it collects statistical information (traffic characteristic information) that can be used in the Sleep control schedule from the data high-speed transfer middleware 300 is shown by reference symbol t in FIG. 4 described later.
[0059] <Polling determination unit 220> Based on the traffic characteristic information received from the traffic characteristic information collection unit 210, the Polling determination unit 220 generates Polling schedule information for waking up the PollingThread310 of the data high-speed transfer middleware 300 at a timing when the packet arrival probability is above a predetermined threshold and putting it to Sleep otherwise, and transmits it to the Polling instruction unit 230 together with time synchronization information (see reference symbol p in FIG. 1).
[0060] Describe the predetermined threshold of the arrival probability of the above packet. The predetermined threshold is determined by the operator of the system according to the use case and is stored, for example, as a table value in a storage unit (not shown). The Polling determination unit 220 compares the arrival probability of the packet with the table value stored in the storage unit and generates Polling schedule information for waking up the PollingThread 310 when the arrival probability of the packet is equal to or greater than the predetermined threshold. That is, (1) In use case A, schedule information is generated based on the criterion of "waking up at the timing when a packet arrives with an arrival probability of 99% or more". In use case B, schedule information is generated based on the criterion of "waking up at the timing when a packet arrives with an arrival probability of 80% or more". In use case C, schedule information is generated based on the criterion of "waking up at the timing when a packet arrives with an arrival probability of 50% or more". And so on.
[0061] (2) The operator can determine the threshold in advance within the range of probability 0 to 1 (0% to 100%). The probability threshold can also be 0 or 1, and in fact, the probability may not be applied (all are applied). For example, for a specific UE or the like, there is an exclusion rule that the probability is not applied.
[0062] (3) The operator can dynamically change this threshold (in the above example, threshold 99%, threshold 80%, threshold 50%).
[0063] <Polling instruction unit 230> The Polling instruction unit 230 receives time synchronization information and Polling schedule information from the Polling determination unit 220, and based on the time synchronization information and the Polling schedule information, delivers a Polling schedule indicating the time and duration required for Polling to the Polling Thread 310, triggering the Polling Thread 310 and causing the Polling Thread 310 to poll for the duration specified in the Polling schedule. Specifically, the Polling instruction unit 230 instructs Polling scheduling (see symbol q in FIG. 1 ) to the Polling thread (PollingThread) 310 of the high-speed data transfer middleware 300. As will be described later, the Polling scheduling instructed by the Polling instruction unit 230 to the PollingThread 310 is a trigger that causes the Polling instruction unit 230 to actually wake up the PollingThread 310.
[0064] [High-speed data transfer middleware 300] The high-speed data transfer middleware 300 is a data transfer unit that transfers packets using a polling model. The high-speed data transfer middleware 300 has multiple polling threads 310, which are execution units of a program. The polling threads 310 are woken up from sleep by polling scheduling from the polling instruction unit 230, and transfer packets using the polling model. The polling threads 310 receive polling scheduling from the polling instruction unit 230, wake up when the packet arrival probability is equal to or greater than a predetermined value, and sleep otherwise (see symbol r in FIG. 1).
[0065] [Categorizing traffic according to characteristics] The intra-server data transfer device 200 wakes up the polling thread in the high-speed data transfer middleware 300 when the packet arrival probability is equal to or greater than a predetermined value, and puts it to sleep otherwise. To achieve this control, the characteristics of data arrival patterns are categorized (see Figure 2). For example, in a RAN (Radio Access Network) use case, traffic can be categorized according to its characteristics, and whether polling is required for each category can be determined.
[0066] We will explain examples of data arrival patterns and corresponding sleep control measures. One use case where traffic characteristic information (see symbol o in Figure 1) that can be used for sleep control scheduling can be collected is RAN. RAN radio signals have characteristic data arrival patterns for each channel. It is possible to determine a sleep control schedule that is appropriate for each of these characteristics.
[0067] FIG. 2 is a diagram illustrating an example of a channel, characteristics of data arrival, and sleep control measures taken by the polling determination unit 220 in accordance with an example of a data arrival pattern. Examples of data arrival patterns include: 1. when it is impossible to predict when data will arrive; 2. when the timing of data arrival is predictable and when it is not; and 3. when the timing of data arrival is predictable.
[0068] 1. An example of a channel when it is impossible to predict when data will arrive is the PRACH (Physical Random Access Channel) (first channel), and the characteristic of data arrival is that "it is impossible to predict when data will arrive." The sleep control countermeasure by the Polling determination unit 220 is to wait in a sleep state and wake up by a hardware interrupt when a packet arrives. Since packets arriving on the PRACH can tolerate a certain amount of delay, it is assumed that there is no need to wait by Polling. When waiting by Polling, there is no wake-up by a hardware interrupt, so there is low delay. Furthermore, when it is impossible to predict when data will arrive, this generally belongs to the category where a certain amount of delay is acceptable. For this reason, there is no need to wait by Polling, and wake up by a hardware interrupt.
[0069] 2. An example of a channel where both predictable and unpredictable conditions are possible is the PUSCH (Physical Uplink Shared Channel) (second channel), and the characteristics of data arrival are that "data will not arrive if the UE (User Equipment) is not attached, but data may arrive if the UE is attached."
[0070] This section explains UE attribution. A UE is, for example, a mobile terminal connected to a base station network. The mobile terminal network is notified in advance that the user's mobile terminal will communicate using resources divided into usable frequency and time domains, and it is known that the mobile terminal can communicate in those frequency and time domains. UE attribution means that the mobile terminal belongs to the frequency and time domains specified by the base station (regardless of whether the mobile terminal is using them or not). Here, just because the base station has assigned the frequency and time domains specified by the base station does not necessarily mean that the mobile terminal belongs to those domains and communicates there (if a UE is attributable, communication may or may not be performed using those resources). UE not attribution means that resources are assigned but no mobile terminal is using those resources (communication will not be performed using the assigned resources).
[0071] The sleep control countermeasure by the polling determination unit 220 is to sleep if no UE is associated. If no UE is associated, data will not arrive in the first place, so sleep is performed. Furthermore, if a UE is associated, control can also be performed according to information such as the UE's past packet transmission characteristics. For example, even if a UE is associated, the UE's data transmission probability is less than 1% (1% is an example and is not limiting), so sleep is performed instead of polling.
[0072] 3. An example of a channel where it is possible to predict when data will arrive is SRS (Sounding Reference Signal) (third channel), and the characteristic of data arrival is that "the timing within the subframe in which the signal is received is fixed." The sleep control countermeasure by the Polling determination unit 220 is to start polling just before the data arrives. When polling is started, there is no wake-up from a hardware interrupt, resulting in low latency. The channel example ("SRS") when it is possible to predict when data will arrive is unrelated to the UE attribution mentioned in 2 above. SRS is a synchronization signal emitted by the base station for time synchronization and a signal transmitted by the base station to obtain information, and transmits signals using a specific time and frequency domain regardless of the UE's usage status or UE attribution. Since it is possible to predict when data will arrive, polling starts immediately before the data arrives.
[0073] The determination of the sleep control schedule using the categorization of the data arrival pattern characteristics shown in FIG. 2 will be described later with reference to FIG.
[0074] The traffic characteristic information collected by the traffic characteristic information collecting unit 210 will be described. FIG. 3 is a diagram showing channel allocation information in the time-frequency domain obtained from traffic characteristic information collecting unit 210, which is an example of traffic characteristic information. The channel allocation information shown in Figure 3 is indicated by the channels allocated in the frequency ranges KL [Hz], LM [Hz], MN [Hz], NO [Hz], and OP [Hz] for each time range 1-80 [usec], 81-160 [usec], 161-240 [usec], 321-400 [usec], and 401-480 [usec] starting from the reference: hh hour, mm minute, ss second, xxxx.
[0075] The channels assigned to 321-400 [usec], 401-480 [usec], MN [Hz], NO [Hz], and OP [Hz] are the highest priority SRSs, such as synchronization signals. These SRSs are signals transmitted using specific time-frequency domains (for the sake of convenience, we will refer to SRSs as channels), and the arrival time of data is predictable. In addition, the channels allocated at 81-160 [usec], 161-240 [usec], 321-400 [usec], MN [Hz], NO [Hz], and OP [Hz] are PUSCHs. These PUSCHs are channels for cases where the data arrival timing is predictable and cases where it is not. The PRACH is a channel allocated at LM [Hz] over the entire time slots of 1-80 [usec], 81-160 [usec], 161-240 [usec], 321-400 [usec], and 401-480 [usec]. This PRACH is a channel used when it is impossible to predict when data will arrive. In FIG. 3, blank cells indicate a state where there is no channel allocation information.
[0076] For example, in the frequency domain LM [Hz], the channel PUSCH is allocated regardless of the time band (in all time periods of 1-80 [usec], 81-160 [usec], 161-240 [usec], 321-400 [usec], and 401-480 [usec]). In addition, in the frequency domains MN [Hz], NO [Hz], OP [Hz] MN [Hz], there is no channel allocation information in the range 1-80 [usec], the channel PUSCH is allocated in the ranges 81-160 [usec], 161-240 [usec], and 321-400 [usec], and the SRS is allocated in the ranges 161-240 [usec] and 321-400 [usec].
[0077] The operation of the intra-server data transfer system configured as above will now be described. [Overall operation] 4 is a diagram for explaining the operation of the server data transfer system, in which the same components as those in FIG. 1 are given the same reference numerals. Applications 2 shown in FIG. 4 are, for example, L1 / L2 Applications (data processing applications for the physical layer / data link layer). The high-speed data transfer middleware 300 has a shared memory 320 that holds information on packet arrival frequency in chronological order. The shared memory 320 stores the number of times that packets actually arrive at resource blocks assigned to each UE. The shared memory 320 transmits packet arrival statistics to the traffic characteristic information collection unit 210 as statistical information that can be used for sleep control scheduling.
[0078] In an application example to RAN, for example, the traffic characteristic information collection unit 210 collects traffic characteristic information as follows. That is, the traffic characteristic information collection unit 210 periodically acquires channel allocation information in the time-frequency domain from the L1 / L2 Application. The traffic characteristic information collection unit 210 also performs time synchronization, combines the channel allocation information with time information, and passes the information to the Polling determination unit 220.
[0079] The traffic characteristic information collecting unit 210 collects information on the packet arrival frequency via the shared memory 320 and transmits it to the Polling determining unit 220 as traffic characteristic information.
[0080] 5 is a diagram illustrating the acquisition of traffic characteristics of packet arrival statistics in cooperation with the high-speed data transfer middleware 300. In FIG. 5, "+" indicates the number of times a packet has arrived at each UE. For example, for UE1, the number of times a packet has actually arrived at the resource block to which UE1 is assigned is 4. Similarly, for UE2, the number of times a packet has arrived is 2, and for UE3, the number of times a packet has arrived is 5. The high-speed data transfer middleware 300 collects the number of times that packets actually arrive at the resource blocks allocated to each UE, and stores the collected information in the shared memory 320. The high-speed data transfer middleware 300 transmits the information stored in the shared memory 320 to the traffic characteristic information collection unit 210 (see symbol t in FIG. 4).
[0081] Based on the traffic characteristic information (see Figure 3) received from the traffic characteristic information collection unit 210, the Polling determination unit 220 generates polling schedule information that wakes up the polling thread 310 of the high-speed data transfer middleware 300 when the packet arrival probability is equal to or greater than a predetermined value, and puts it to sleep otherwise, and transmits this information to the polling instruction unit 230 (see symbol p in Figure 4).
[0082] The Polling instruction unit 230 receives time synchronization information and Polling schedule information from the Polling determination unit 220, and based on the time synchronization information and the Polling schedule information, distributes a Polling schedule indicating the time and duration at which Polling is required to the Polling Thread 310 of the high-speed data transfer middleware 300 via Polling scheduling (see symbol q in Figure 4), causing the Polling thread to poll for the duration specified in the Polling schedule.
[0083] [Operation of traffic characteristic information collection unit 210] <Traffic characteristics information collection process 1> Fig. 6 is a flowchart showing traffic characteristic information collection process 1 of the traffic characteristic information collection unit 210. Fig. 6 shows the processing flow when the traffic characteristic information collection unit 210 acquires channel allocation information to the time-frequency domain from an L1 / L2 Application (higher-level application) (see symbol s in Fig. 4). In step S11, the traffic characteristic information collection unit 210 (see FIG. 4) periodically collects channel allocation information as traffic characteristic information from the L1 / L2 Application (or at the timing when the channel allocation pattern is changed).
[0084] In step S12, the traffic characteristic information collecting unit 210 performs time synchronization and arranges the channel assignment information along the time axis. That is, as shown in Fig. 3, the traffic characteristic information collecting unit 210 arranges the channel assignment information along the time axis from the reference: hh hour, mm minute, ss second, xxxx.
[0085] In step S13, the traffic characteristic information collecting unit 210 distributes the time-synchronized channel allocation information to the Polling determining unit 220 (see FIG. 4), and the process of this flow ends.
[0086] <Traffic characteristics information collection process 2> Fig. 7 is a flowchart showing traffic characteristic information collection processing 2 of the traffic characteristic information collection unit 210. Fig. 7 shows the processing flow when the traffic characteristic information collection unit 210 acquires packet arrival statistics from the high-speed data transfer middleware 300 (see symbol t in Fig. 4). In step S21, the traffic characteristic information collection unit 210 (see FIG. 4) periodically collects information on the packet arrival frequency of each UE that is collected by the high-speed data transfer middleware 300 (see FIG. 4) and written to the shared memory 320 (see FIG. 4).
[0087] In step S22, the traffic characteristic information collecting unit 210 (see FIG. 4) forms data in the form of "UE ID: packet arrival frequency information." 1. All arrival rates so far 2. Arrival rate at the time of data collection from the immediately preceding shared memory 320 (see Figure 4) It holds two types: Using this packet arrival frequency information, the traffic characteristic information collecting unit 210 can determine whether the UE is a UE that has performed communication using the most recently allocated resources or whether this is the first time that resources have been allocated to the UE, as shown in Fig. 9. Note that whether the UE is a first-time UE is determined by acquiring all packet arrival rates and the most recent packet arrival rate based on the ID assigned to the UE, and determining that the UE is a first-time UE if no immediately previous packet has arrived (for example, if the UE ID is not recorded in the first place, it is determined that this is a first-time communication).
[0088] In step S23, when the traffic characteristic information collection unit 210 delivers the channel allocation information (see FIG. 3) to the Polling determination unit 220 (see symbol o in FIG. 4), if the channel allocation information includes a PUSCH channel, it simultaneously delivers the information "UE ID: packet arrival frequency information" and terminates the processing of this flow.
[0089] <Correction of data arrival timing information> Correction of data arrival timing information will now be described. Due to the lack of time synchronization with the opposing device 170 shown in FIG. 1 or the occurrence of a certain data processing time within the opposing device 170, etc., there may be a steady or regular difference according to the data size, etc., between the timing information passed by the traffic characteristic information collection unit 210 (see FIGS. 1 and 4) to the Polling determination unit 220 and the actual data arrival timing information. In this case, the data high-speed transfer middleware 300 (see FIGS. 1 and 4) may send the actual packet arrival time to the traffic characteristic information collection unit 210, and the traffic characteristic information collection unit 210 may correct the difference to handle it. The operation of the traffic characteristic information collection unit 210 has been described above.
[0090] [Operation of Polling Determination Unit 220] In the RAN use case, when the Polling determination unit 220 obtains the channel allocation information to the time-frequency domain shown in FIG. 3 from the traffic characteristic information collection unit 210, the Polling determination unit 220 determines the Sleep control schedule as follows and generates Polling schedule information based on the determined Sleep control schedule.
[0091] [Sleep Control Schedule Determination Process] FIG. 8 is a flowchart showing the Sleep control schedule determination process in the Polling determination unit 220. This flow determines the Sleep control schedule using the categorization of the data arrival pattern characteristics shown in FIG. 2.
[0092] In step S31, the Polling determination unit 220 scans the channel allocation information in the frequency domain for each time domain (in this specification, scanning means checking the traffic characteristic information (see FIG. 3) in time order from the reference to the time end).
[0093] In step S32, the Polling determination unit 220 determines whether a channel where "packets are known to arrive" (in the case of the traffic characteristics information in FIG. 3, "SRS") is allocated to any frequency block in the relevant time period. For example, when the Polling determination unit 220 scans the traffic characteristics information shown in FIG. 3 from the base to the time end (1-80 [usec] to 401-480 [usec]), it determines that "SRS" is allocated in the frequency domains MN [Hz], NO [Hz], OP [Hz] and in the time domains 321-400 [usec], 401-480 [usec].
[0094] If a channel ("SRS") on which "a packet is known to arrive" is not assigned to any frequency block for that time (S32: No), proceed to step S33; if it is assigned (S32: Yes), proceed to step S35.
[0095] In step S33, the Polling determination unit 220 determines whether or not a channel (in the case of the traffic characteristic information of FIG. 3, "PUSCH") that "depends on whether a packet arrives depending on whether a UE belongs" is allocated to any frequency block during the relevant time period. For example, when the Polling determination unit 220 scans the traffic characteristic information shown in FIG. 3 from the base to the time end (1-80 [usec] to 401-480 [usec]), it determines that "PUSCH" is allocated in the frequency domains MN [Hz], NO [Hz], and OP [Hz] and in the time domains 81-160 [usec], 161-240 [usec], and 241-320 [usec].
[0096] If a channel ("PUSCH") in which "whether a packet arrives or not depends on whether a UE belongs or not" is not allocated to any of the frequency blocks for that time (S33: No), proceed to step S34; if it is allocated (S33: Yes), proceed to step S36.
[0097] In step S34, the Polling determination unit 220 determines a sleep control schedule that causes the Polling thread in the high-speed data transfer middleware 300 to sleep during the relevant time and wakes it up in interrupt mode when a packet arrives, and then ends the processing of this flow. That is, if "PUSCH" is not assigned in step S33 above, the Polling determination unit 220 waits in a sleep state in step S34 and wakes up by a hardware interrupt when a packet arrives.
[0098] On the other hand, if ("SRS") is assigned in step S32 above, the Polling determination unit 220 wakes up at the corresponding time in step S35, waits for arrival of a packet by Polling, and ends the processing of this flow. By waking up at the corresponding time and starting Polling just before data arrives, the Polling determination unit 220 can avoid waking up from a hardware interrupt and achieve low latency.
[0099] If ("PUSCH") is allocated in step S33 above, the polling determination unit 220 determines whether or not a UE is associated in step S36.
[0100] If it is determined in step S36 that no UE is associated, then in step S37 the Polling determination unit 220 sleeps for the relevant time and terminates the processing of this flow. If no UE is associated, even if a ("PUSCH") is assigned, data will not arrive in the first place, so the Polling determination unit 220 determines a sleep control schedule that causes the Polling thread in the high-speed data transfer middleware 300 to sleep for the relevant time and terminates the processing of this flow.
[0101] On the other hand, if the UE is found to be associated in step S36, the Polling determination unit 220 applies logic (logic described later in FIG. 9) according to the characteristics and statistics of the UE in step S38, and ends the processing of this flow.
[0102] Through the above processing, the intra-server data transfer device 200 can wake up the polling thread in the high-speed data transfer middleware 300 when the packet arrival probability is equal to or greater than a predetermined threshold, and put it to sleep otherwise (see "Sleep control schedule determination in step S34"). When the packet arrival probability is extremely low, the polling determination unit 220 determines that putting it to sleep is appropriate.
[0103] However, if the probability of packet arrival is extremely low and sleep is put into effect, it is possible that a packet may arrive while the device is asleep. The high-speed data transfer middleware 300 is originally equipped with a function to wake up in interrupt mode when a packet arrives. If a packet arrives while the device is asleep, the high-speed data transfer middleware 300 uses this function to wake up in interrupt mode (a hardware interrupt is raised from the NIC, and the high-speed data transfer middleware 300 wakes up in response to an instruction from the NIC). However, because the device wakes up in interrupt mode, low latency cannot be achieved. A supplementary explanation follows.
[0104] Waking up from a hardware interrupt has the following disadvantages. That is, waking up from a hardware interrupt is characterized by a longer delay time compared to polling. For this reason, waking up the high-speed data transfer middleware 300 and then starting polling to acquire packets results in a shorter delay time than waking up from the context of a hardware interrupt. For this reason, the most ideal pattern is to wake up in advance (especially just before a packet arrives) and start polling, which minimizes the delay time. In the worst case scenario, even if waking up during sleep is not possible, it is still possible to wake up using a hardware interrupt, even if the delay time increases slightly.
[0105] Incidentally, when the high-speed data transfer middleware 300 is awake and performing polling, the high-speed data transfer middleware 300 prohibits the raising of hardware interrupts. In other words, even if a hardware interrupt does not occur, whether a packet has arrived can be detected by polling, so there is no need to raise a hardware interrupt and it can be prohibited (polling is performed with the raising of hardware interrupts prohibited).
[0106] As described above, the polling determination unit 220 determines the sleep control schedule and generates polling schedule information based on the determined sleep control schedule.
[0107] Fig. 9 is a diagram illustrating an example of data characteristics according to an example of statistical information, what can be inferred from the characteristics, and sleep control countermeasures by the polling determination unit 220. Fig. 9 explains the logic according to the characteristics and statistics of the relevant UE in step S38 of Fig. 8. Examples of statistical information include: 1. the burstiness of the communication of the user, 2. whether the user is a light user or not, 3. whether this is the first time that resources have been allocated to the user, and 4. downlink communication information for the user. Each of these will be explained in order below.
[0108] 1. When the statistical information example shows the burstiness of the user's communication An example of the data characteristic is "the UE in question is communicating using the resources allocated just before." What can be inferred from this data characteristic is that "if the UE was communicating last time, there is a high probability that it will also transmit data using the current resources." The sleep control countermeasure by the Polling determination unit 220 determines that the UE is highly likely to transmit data using the current resources, and waits for a packet by Polling.
[0109] 2. In the case of statistical information examples, whether the corresponding user is a light user (a user who, despite belonging to the UE, has rarely communicated using the allocated resources; a user who has only synchronized with the initial base station, etc.) An example of data characteristics is that "the corresponding UE has only used 1% of the allocated resources so far". From this data characteristic, it can be inferred that "the probability of transmitting data with this resource is low". The Sleep control countermeasure by the Polling determination unit 220 determines that the UE has a low probability of transmitting data with this resource and waits for packets by Sleep.
[0110] 3. In the case of statistical information examples, whether the resource is allocated to the corresponding user for the first time An example of data characteristics is that "there has been no communication by the corresponding UE so far, and it has newly made an allocation request and this is its first communication". From this data characteristic, it can be inferred that "it is the first time and the probability of transmitting data is high". In the case of the first time, it is determined that the UE has a high probability of transmitting data, similar to the case of the UE that had the previous communication in 1. above, and the Sleep control countermeasure by the Polling determination unit 220 waits for packets by Polling.
[0111] 4. In the case of statistical information examples regarding the downlink communication information for the corresponding user (the link when the base station sends a message to the UE) An example of data characteristics is that "some request message for the corresponding UE is being sent on the downlink". From this data characteristic, it can be inferred that "the probability of immediately transmitting data to respond to that request is high". When the base station sends a message to the UE, it is determined that the corresponding UE has a high probability of responding to the base station, and the Sleep control countermeasure by the Polling determination unit 220 waits for packets by Polling.
[0112] <Polling Schedule Information Distribution Process> FIG. 10 is a flowchart showing the polling schedule information distribution process of the polling determination unit 220. In step S41, the polling determination unit 220 starts this program when channel allocation information is distributed from the traffic characteristic information collection unit.
[0113] In step S42, the Polling determination unit 220 divides the channel allocation information (see FIG. 3) that the traffic characteristic information collection unit 210 has collected from Applications 2 (L1 / L2 Application in FIG. 4) and delivered to the Polling determination unit 220 into time slots (divided into 80 [usec] in the example of FIG. 3), and scans which channels are assigned to each slot.
[0114] In step S43, the Polling determination unit 220 determines whether or not any frequency block in the slot includes a channel where "it is known that a packet will arrive" (in the case of the traffic characteristic information in FIG. 3, "SRS"). Here, step S43 corresponds to step S32 in the flow in FIG. 8, and step S46 described below corresponds to step S35 in the flow in FIG. 8. In step S43, if "SRS," the highest priority, is assigned to a time-frequency domain channel where it is known that a packet will arrive, the process moves to step S46 without proceeding to step S44 or later.
[0115] If any frequency block of the slot does not include a channel ("SRS") where "it is known that packets will arrive" (S43: No), in step S44, the Polling determination unit 220 determines whether any frequency block of the slot includes a channel ("PUSCH" in the case of the traffic characteristic information of FIG. 3) where "packet arrival characteristics change depending on whether a UE belongs to the channel). Here, step S44 corresponds to step S33 of the flow in FIG. 8, and step S47 described below corresponds to step S36 of the flow in FIG. 8. That is, in step S44, it is determined whether "PUSCH," which has the second highest priority after "SRS," is allocated, and if any frequency block of the slot does not include a channel ("PUSCH") whose "packet arrival characteristics change depending on whether a UE is associated with it" (S44: No), in step S45 the Polling determination unit 220 decides to sleep during the relevant time, scans the next time slot, and proceeds to step S48.
[0116] On the other hand, if any of the frequency blocks in the slot in step S43 contains a channel ("SRS") where "a packet is known to arrive" (S43: Yes), in step S46 the Polling determination unit 220 determines to perform Polling during the relevant time, scans the next time slot, and proceeds to step S48.
[0117] Furthermore, if any frequency block of the slot contains a channel ("PUSCH") in which "packet arrival characteristics change depending on whether or not a UE is associated" in step S44 (S44: Yes), the polling determination unit 220 applies a predetermined logic (logic shown in the subroutine of FIG. 11) in step S47 to determine whether or not polling is possible, scans the next time slot, and proceeds to step S48.
[0118] In step S48, the Polling determination unit 220 determines whether or not the given channel allocation information is at the time end (the time end defined by the traffic characteristic information collection unit 210 (a time division that defines how far the traffic characteristic information collection unit 210 will collect next); for example, a time division of 1 ms).
[0119] If the given channel assignment information is not at the time end (S48: No), the process returns to step S42 and repeats the steps above until the given channel assignment information is at the time end.If the given channel assignment information is at the time end (S48: Yes), the process proceeds to step S49.
[0120] In step S49, the Polling determination unit 220 summarizes whether to perform Polling or Sleep for each time slot, delivers this together with time information (information for determining the reference time axis) to the Polling instruction unit, and ends the processing of this flow. For example, the Polling determination unit 220 summarizes, for each time slot (slots divided into 80 [usec] in the example of FIG. 8), information for, for example, 10 times every 80 [usec], such as performing Polling in some slots and sleeping in other slots, and passes the summarized information ("information on whether Polling is necessary") together with the time information to the Polling instruction unit 230 as Polling schedule information.
[0121] 11 shows a subroutine that shows the logic for determining whether or not polling is possible by the polling determination unit 220. The subroutine shown in FIG. 11 is called and executed by the subroutine call in step S47 of FIG. In step S51, when the Polling determination unit 220 scans the distributed channel allocation information, if any of the frequency blocks in the relevant time slot contains a "channel whose packet arrival characteristics change depending on whether or not a UE is associated," this logic is triggered (reconfigured).
[0122] In step S52, the polling determination unit 220 determines whether or not any UE belongs to any of the relevant channels. If a UE belongs to one of the relevant channels in step S52 (S52: Yes), in step S53 the Polling determination unit 220 checks whether a UE currently belonging to the key of the information group of "UE ID: packet arrival characteristics" delivered from the traffic characteristic information collection unit 210 at the same time as the channel allocation information exists.
[0123] In step S54, the polling determination unit 220 determines whether or not the ID of the UE in question exists.
[0124] If the ID of the UE in question exists in step S54 (S54: Yes), in step S55, the Polling determination unit 220 determines whether the UE in question is transmitting a packet using the most recently allocated resource.
[0125] If the UE has not transmitted a packet using the resources allocated most recently in step S55 (S55: No), then in step S56 the Polling determination unit 220 determines whether the UE has used only 1% of the resources allocated to it so far, that is, whether it has used 1%.
[0126] If it is determined in step S56 above that the UE has not used 1% of the allocated resources so far (S56: No), then in step S57 the Polling determination unit 220 determines to perform Polling for the relevant time and returns to step S47 in FIG.
[0127] On the other hand, if the Polling determination unit 220 determines in step S52 that no UE belongs to any of the relevant channels (S52: No), then in step S58 the Polling determination unit 220 determines to sleep for the relevant time and returns to step S47 in Fig. 10. In this embodiment, this flow is called and executed by the subroutine call in step S47 in Fig. 10, so the process returns to step S47 in Fig. 10, but the process in Fig. 11 may also be executed independently.
[0128] If the ID of the UE in question does not exist in step S54 (S54: No), the polling determination unit 220 determines in step S59 to sleep for the relevant time, and returns to step S47 in FIG.
[0129] If the UE is transmitting packets using the resources allocated immediately before in step S55 above (S55: Yes), in step S60, the Polling determination unit 220 determines to perform Polling during the relevant time, and returns to step S47 in FIG. 10.
[0130] If it is determined in step S56 above that the UE has used at least 1% of the resources allocated to it so far (S56: Yes), then in step S61 the Polling determination unit 220 determines to sleep for the relevant time, and the process returns to step S47 in FIG. The operation of the Polling determination unit 220 has been described above.
[0131] [Operation of Polling Instructor 230] FIG. 12 is a flowchart showing the operation of the polling instruction unit 230. In step S71, the Polling instruction unit 230 (see Figures 1 and 4) receives "information on whether Polling is necessary" (see step S49 in Figure 10) synchronized with time (for example, every 80 [usec] as in Figure 3) from the Polling determination unit 220 (see Figures 1 and 4).
[0132] In step S72, the Polling instruction unit 230 distributes the time and duration required for Polling to the high-speed data transfer middleware 300, triggers the Polling Thread 310 of the high-speed data transfer middleware 300 (wakes up the Polling Thread 310), and terminates the processing of this flow. The trigger will now be described. That is, the Polling instruction unit 230 wakes up the Polling Thread 310 of the high-speed data transfer middleware 300 just before the time for Polling. For example, the Polling instruction unit 230 instructs the Polling Thread 310 to start Polling and the Polling duration, such as to perform Polling in the next 160 [usec], or to perform Polling for the next 80 [usec]. In this way, the Polling instruction unit 230 actually waking up the Polling Thread 310 is called a trigger (in FIG. 1, the trigger is written as "Polling scheduling").
[0133] [Operation of high-speed data transfer middleware 300] FIG. 13 is a flowchart showing the operation of the high-speed data transfer middleware 300. In step S81, the high-speed data transfer middleware 300 (see FIGS. 1 and 4) is triggered by the polling instruction unit 230, and the polling thread 310 performs polling for the instructed time.
[0134] In step S82, when the polling is completed, the high-speed data transfer middleware 300 automatically goes to sleep and ends the processing of this flow.
[0135] [Hardware configuration] The intra-server data transfer device 200 according to the above embodiment is realized by a computer 900 having a configuration as shown in FIG. 14, for example. FIG. 14 is a hardware configuration diagram showing an example of a computer 900 that realizes the functions of the intra-server data transfer device 200. The computer 900 includes a CPU 901 , a ROM 902 , a RAM 903 , a HDD 904 , a communication interface (I / F) 906 , an input / output interface (I / F) 905 , and a media interface (I / F) 907 .
[0136] The CPU 901 operates based on a program stored in the ROM 902 or the HDD 904, and controls each unit of the intra-server data transfer device 200 shown in Figures 1 and 4. The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 starts up, programs that depend on the hardware of the computer 900, and the like.
[0137] The CPU 901 controls an input device 910 such as a mouse or keyboard, and an output device 911 such as a display, via an input / output I / F 905. The CPU 901 acquires data from the input device 910 via the input / output I / F 905, and outputs generated data to the output device 911. Note that a GPU (Graphics Processing Unit) or the like may be used as a processor together with the CPU 901.
[0138] The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I / F 906 receives data from other devices via a communication network (e.g., NW (Network) 920) and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.
[0139] The media I / F 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads a program related to a target process from the recording medium 912 onto the RAM 903 via the media I / F 907, and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto Optical disc), a magnetic recording medium, a conductive memory tape medium, a semiconductor memory, or the like.
[0140] For example, when the computer 900 functions as the intra-server data transfer device 200 configured as one device according to this embodiment, the CPU 901 of the computer 900 executes a program loaded onto the RAM 903 to realize the functions of the intra-server data transfer device 200. The HDD 904 stores data in the RAM 903. The CPU 901 reads and executes a program related to a target process from the recording medium 912. Alternatively, the CPU 901 may read a program related to a target process from another device via a communication network (NW 920).
[0141] [effect] As described above, the intra-server data transfer device according to this embodiment is an intra-server data transfer device 200 that performs sleep control on a polling thread (polling thread 310) of a data transfer unit (high-speed data transfer middleware 300) that transfers packets according to a polling model, and includes a traffic characteristic information collection unit 210 that collects traffic characteristic information in packet transfer, a polling determination unit 220 that generates polling schedule information based on the traffic characteristic information received from the traffic characteristic information collection unit 210, which wakes up the polling thread when the packet arrival probability is equal to or greater than a predetermined threshold and causes the polling thread to sleep at other times, and a polling instruction unit 230 that receives time synchronization information and polling schedule information from the polling determination unit 220, and based on the time synchronization information and the polling schedule information, distributes a polling schedule to the polling thread indicating the time and duration at which polling is required, and causes the polling thread to poll for the duration instructed by the polling schedule.
[0142] In this way, the Polling determination unit 220 determines whether the high-speed data transfer middleware 300 will perform reception processing in Polling mode or go to sleep, based on the traffic characteristic information received from the traffic characteristic information collection unit 210, and generates Polling schedule information. The Polling instruction unit 230 distributes the Polling schedule to the Polling thread based on the time synchronization information and Polling schedule information received from the Polling determination unit 220, and causes Polling to occur for the instructed time. This makes it possible to wake up the Polling thread in the high-speed data transfer middleware 300 when the packet arrival probability is equal to or greater than a predetermined threshold, and to put it to sleep otherwise, thereby achieving low latency and power saving in data transfer.
[0143] Furthermore, the intra-server data transfer device 200 can realize data transfer delays within a server using a polling model rather than an interrupt model, thereby achieving low latency. For example, in a use case where a sleep control schedule can be determined based on traffic characteristics, such as signal processing in a RAN, the intra-server data transfer device 200 can determine whether to wait for reception or sleep based on the polling model, thereby enabling the intra-server data transfer device 200 to perform a reception schedule according to traffic characteristics. This allows the intra-server data transfer device 200 to achieve power savings while maintaining low latency.
[0144] In the intra-server data transfer device 200, the traffic characteristic information collection unit 210 collects channel allocation information in the time and frequency domains from the upper application (Applications 2), and transmits it to the Polling determination unit 220 as traffic characteristic information.
[0145] In this way, the L1 / L2 Application (higher-level application) has time-frequency domain channel allocation information for data transfer. The traffic characteristic information collection unit 210 can collect information used for sleep control scheduling from the higher-level application. Therefore, the intra-server data transfer device 200 can acquire traffic characteristic information in a general-purpose manner without adding new components or modifying the application.
[0146] In the intra-server data transfer device 200, the data transfer unit (high-speed data transfer middleware 300) has a shared memory 320 that stores information on packet arrival frequency in chronological order, and the traffic characteristic information collection unit 210 collects information on packet arrival frequency via the shared memory 320 and transmits it to the Polling determination unit 220 as traffic characteristic information.
[0147] In this way, the traffic characteristic information collecting unit 210 can collect information to be used for the sleep control schedule, such as the characteristics of packet arrival frequency in time series, from the data transfer unit (high-speed data transfer middleware 300). Therefore, the intra-server data transfer device 200 can obtain information to be used for the sleep control schedule, such as the characteristics of packet arrival frequency in time series, without modifying the application.
[0148] In the intra-server data transfer device 200, the traffic characteristic information has channels to which packet arrival patterns (data arrival patterns) are assigned in the time domain and the frequency domain, and these channels include a first channel ("PRACH") for when packet arrival is unpredictable, a second channel ("PUSCH") for when packet arrival timing is predictable and unpredictable, and a third channel ("SRS") for when packet arrival is predictable.The polling determination unit 220 generates polling schedule information in which, for the first channel, the polling thread waits in a sleep state and is woken up by a hardware interrupt when a packet arrives, and in which, for the second channel, the polling schedule information is generated in which the polling thread sleeps, and in which, for the third channel, the polling schedule information is generated in which the polling thread starts polling immediately before a packet arrives.
[0149] In this way, the server data transfer device 200 can determine a sleep control schedule suitable for each channel and each characteristic of the data arrival pattern, like a RAN radio signal. Also, for a channel to which a UE belongs, like the second channel ("PUSCH"), it is possible to generate a sleep control schedule that reflects past traffic statistics from the corresponding user.
[0150] [Application example] The server data transfer system 1000 (see Figure 1) can be applied to any traffic with fixed characteristics in the time domain, such as packet arrival frequency, and therefore can be applied not only to uplink traffic in the RAN, but also to downlink traffic, etc. Furthermore, the intra-server data transfer system 1000 can be applied not only to network-facing data transfer but also to accelerator-facing data transfer.
[0151] It should be noted that, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.
[0152] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented by software that causes a processor to interpret and execute programs that implement the respective functions. Information on the programs, tables, files, etc. that implement the respective functions may be stored in a memory, a recording device such as a hard disk or a solid-state drive (SSD), or a recording medium such as an integrated circuit (IC) card, a secure digital (SD) card, or an optical disc. [Explanation of symbols]
[0153] 2 Applications (Top Applications) 110 HW 180 memory 200 Server data transfer device 210 Traffic characteristics information collection unit 220 Polling judgment section 230 Polling instruction section 300 High-speed data transfer middleware (data transfer section) 310 Polling Thread 320 Shared Memory 1000 Server Data Transfer System PRACH (Channel 1) PUSCH (2nd channel) SRS (Channel 3)
Claims
1. An intra-server data transfer device that performs sleep control on a polling thread of a data transfer unit that performs packet transfer using a polling model, a traffic characteristic information collecting unit for collecting traffic characteristic information in the packet forwarding; a polling determination unit that generates polling schedule information based on the traffic characteristic information received from the traffic characteristic information collection unit, the polling schedule information being used to wake up the polling thread when the packet arrival probability is equal to or greater than a predetermined threshold, and to put the polling thread to sleep otherwise; a Polling instruction unit that receives time synchronization information and the Polling schedule information from the Polling determination unit, and distributes a Polling schedule indicating the time and duration required for Polling to the Polling thread based on the time synchronization information and the Polling schedule information, and causes the Polling thread to poll for the duration indicated in the Polling schedule; The traffic characteristic information collecting unit collects channel allocation information in the time and frequency domains from an upper application and transmits the collected information as the traffic characteristic information to the polling determining unit.
1. A server data transfer device comprising:
2. An intra-server data transfer device that performs sleep control on a polling thread of a data transfer unit that performs packet transfer using a polling model, a traffic characteristic information collecting unit for collecting traffic characteristic information in the packet forwarding; a polling determination unit that generates polling schedule information based on the traffic characteristic information received from the traffic characteristic information collection unit, the polling schedule information being used to wake up the polling thread when the packet arrival probability is equal to or greater than a predetermined threshold, and to put the polling thread to sleep at other times; a Polling instruction unit that receives time synchronization information and the Polling schedule information from the Polling determination unit, and distributes a Polling schedule indicating the time and duration required for Polling to the Polling thread based on the time synchronization information and the Polling schedule information, and causes the Polling thread to poll for the duration indicated in the Polling schedule; the data transfer unit has a shared memory that stores information on packet arrival frequency in time series; The traffic characteristic information collecting unit collects information on packet arrival frequency via the shared memory and transmits the collected information as traffic characteristic information to the Polling determining unit.
1. A server data transfer device comprising:
3. An intra-server data transfer device that performs sleep control on a polling thread of a data transfer unit that performs packet transfer using a polling model, a traffic characteristic information collecting unit for collecting traffic characteristic information in the packet forwarding; a polling determination unit that generates polling schedule information based on the traffic characteristic information received from the traffic characteristic information collection unit, the polling schedule information being used to wake up the polling thread when the packet arrival probability is equal to or greater than a predetermined threshold, and to put the polling thread to sleep at other times; a Polling instruction unit that receives time synchronization information and the Polling schedule information from the Polling determination unit, and distributes a Polling schedule indicating the time and duration required for Polling to the Polling thread based on the time synchronization information and the Polling schedule information, and causes the Polling thread to poll for the duration indicated in the Polling schedule; The traffic characteristic information includes a packet arrival pattern assigned to a channel in the time domain and the frequency domain; The channel is a first channel when packet arrival is unpredictable; a second channel for both predictable and unpredictable cases; a third channel for cases where packet arrival is predictable; The Polling determination unit In the case of the first channel, the polling thread waits in a sleep state, and generates the polling schedule information for waking up the polling thread by a hardware interrupt when a packet arrives; In the case of the second channel, the polling schedule information is generated for the polling thread to sleep; In the case of the third channel, the polling thread generates the polling schedule information to start polling immediately before the arrival of a packet.
1. A server data transfer device comprising:
4. An intra-server data transfer method of an intra-server data transfer device that performs sleep control on a polling thread of a data transfer unit that performs packet transfer according to a polling model, comprising: a traffic characteristic information collecting step of collecting traffic characteristic information in the packet forwarding; a polling determination step of generating polling schedule information based on the traffic characteristic information received in the traffic characteristic information collection step, which wakes up the polling thread when the packet arrival probability is equal to or greater than a predetermined threshold, and causes the polling thread to sleep otherwise; a Polling instruction step of receiving time synchronization information and the Polling schedule information from the Polling determination step, and distributing a Polling schedule indicating the time and duration required for Polling to the Polling thread based on the time synchronization information and the Polling schedule information, and causing the Polling thread to poll for the duration instructed by the Polling schedule; In the traffic characteristic information collection step, channel allocation information in the time and frequency domains is collected from an upper application, and transmitted as the traffic characteristic information to the polling determination step.
2. A method for transferring data within a server, comprising:
5. The computer that acts as a server data transfer device that performs sleep control for the polling thread of the data transfer unit that transfers packets using the polling model. a traffic characteristic information collection procedure for collecting traffic characteristic information in the packet forwarding; a polling determination step of generating polling schedule information based on the traffic characteristic information received from the traffic characteristic information collection step, which wakes up the polling thread when the packet arrival probability is equal to or greater than a predetermined threshold, and causes the polling thread to sleep otherwise; a Polling instruction procedure for receiving time synchronization information and the Polling schedule information from the Polling determination procedure, and distributing a Polling schedule indicating the time and duration required for Polling to the Polling thread based on the time synchronization information and the Polling schedule information, and causing the Polling thread to poll for the duration instructed by the Polling schedule; Execute In the traffic characteristic information collection procedure, channel allocation information in the time and frequency domains is collected from an upper application, and is transmitted to the Polling determination procedure as the traffic characteristic information. A program to make it happen.
Citation Information
Patent Citations
Polling control system
JP1992172032A
Communication control method in radio communication network
JP2002300173A
Virtual communication path construction system, virtual communication path construction method, and virtual communication path construction program
JP2015197874A
Connection control system of virtual machine and connection control method of virtual machine
JP2018032156A