In-server delay control device, in-server delay control method, and program
By configuring polling threads in the OS kernel space, monitoring and processing packet arrivals, combining hardware interrupt wake-up and periodic wake-up mechanisms, the problem of high power consumption of interrupt model and polling model is solved, achieving low latency and low power consumption Packet transfer.
Patent Information
- Application Number
- CN202280094844.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-15
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, there is a delay problem in packet transmission based on interrupt model and polling model. The interrupt model causes an increase in waiting delay when used by conflict or process with high priority, while the polling model consumes power due to continuous monitoring of packet arrival. high.
Using an in-server delay control device, it is configured in the kernel space of the OS, and uses a polling model to monitor the thread of packet arrival, and has packet arrival monitoring, packet separating, sleep management and idle state recovery control units, and through hardware interruption Wake up threads, regularly or on time to reduce latency and power consumption.
While suppressing the occurrence of excessive HW interruption, it reduces power consumption, reduces packet transmission delays in the server, and avoids recovery delays caused by the CPU falling into a deep idle state.
Smart Images

Figure CN119013660A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a server internal delay control device, a server internal delay control method and a program. Background Art
[0002] With the development of virtualization technology based on NFV (Network Functions Virtualization), a system is built and operated for each service. In addition, according to the above-mentioned system-based approach, the service function is divided into reusable module units and operated on an independent virtual machine (VM: Virtual Machine, container, etc.) environment. Thus, the method called SFC (Service Function Chaining) is gradually becoming mainstream, which is used as needed like a component and improves operability.
[0003] As a technology for forming a virtual machine, a hypervisor environment composed of Linux (registered trademark) and KVM (kernel-based virtual machine) is known. In this environment, a host operating system (Host OS, the OS installed on a physical server is referred to as the host operating system) embedded with a KVM module operates as a hypervisor in a memory area different from a user space called a kernel space. In this environment, a virtual machine operates in a user space, and a guest operating system (Guest OS, the OS installed on a virtual machine is referred to as a guest operating system) operates in the virtual machine.
[0004] The virtual machine for the child operating system is different from the physical server for the parent operating system. It includes all the HW (hardware) of the network device (such as Ethernet card devices), and performs the register control required for interrupt processing from HW to the child operating system and writing from the child operating system to the hardware. In such register control, the notification and processing that should have been performed by the physical hardware are simulated in the form of software simulation, so the performance is generally lower than that of the parent operating system environment.
[0005] In the case of such performance degradation, there is a technology that reduces HW imitation and improves communication performance and versatility through a high-speed and unified interface, especially for the child operating system to the parent operating system and external processes existing outside the virtual machine. As such technology, virtio, a device abstraction technology or paravirtualization technology, has been developed and has been embedded in many general-purpose OSs, including Linux and FreeBSD (registered trademark), and is currently being used.
[0006] In virtio, for data input and output such as console, file input and output, and network communication, as a one-way transport for transmitting data, the data exchange performed by queues designed as a ring buffer is defined through queue operations. Then, by using the technical specifications of virtio queues, by preparing the number and size of queues suitable for each device when the guest operating system is started, it is possible to realize communication between the guest operating system and the outside of the virtual machine without performing hardware emulation but only through queue operations.
[0007] [Data packet transmission based on polling model (DPDK example)]
[0008] The method of connecting and cooperating multiple virtual machines is called Inter-VM Communication. In large-scale environments such as data centers, virtual switches have been used as a standard to connect VMs. However, since this method has a large communication delay, a new method with higher speed has been proposed. For example, a method using special hardware called SR-IOV (Single Root I / O Virtualization) and a method based on software using Intel DPDK (Intel Data Plane Development Kit) (hereinafter referred to as DPDK) as a high-speed packet processing library have been proposed.
[0009] DPDK is a framework for controlling NIC (Network Interface Card) in user space, which was previously performed by Linux kernel (registered trademark). The biggest difference from the processing in Linux kernel is that it has a polling-based receiving mechanism called PMD (Pull Mode Driver). Usually, in Linux kernel, an interrupt occurs in response to the arrival of data to NIC, and reception processing is performed based on this. On the other hand, PMD continuously confirms the arrival of data and performs reception processing by a dedicated thread. By eliminating the overhead of context switching, interrupts, etc., high-speed packet processing is possible. DPDK greatly improves the performance and throughput of packet processing, and can ensure more time for data plane / application processing.
[0010] DPDK occupies computer resources such as CPU (Central Processing Unit) and NIC. Therefore, it is difficult to apply to applications that require flexible switching in modules like SFC. SPP (Soft Patch Panel) is an application used to alleviate this situation. SPP is set up so that each VM can directly refer to the same memory space structure by preparing shared memory between VMs, thereby omitting the copying of data packets in the virtualization layer. In addition, DPDK is used to achieve high speed in the exchange of data packets between the physical NIC and the shared memory. SPP can change the input destination and output destination of the data packet by software by controlling the reference destination of the memory exchange of each VM. Through this process, SPP realizes dynamic connection switching between VMs and between VM and physical NIC.
[0011] [Rx-side packet processing based on New API (NAPI)]
[0012] Fig. 22 This is a schematic diagram of Rx-side packet processing based on the New API (NAPI) implemented by Linux kernel 2.5 / 2.6 (see Non-Patent Document 1).
[0013] like Fig. 22 As shown, on a server having OS70 (e.g., a parent operating system), a packet processing APL1 configured in a user space 60 available to a user is executed, and a New API (NAPI) performs packet transmission between the NIC11 of the HW10 connected to OS70 and the packet processing APL1.
[0014] The OS 70 includes a kernel 71 , a ring buffer 72 , and a driver 73 . The kernel 71 includes a protocol processing unit 74 .
[0015] The kernel 71 is a core function of the OS 70 (e.g., the mother operating system), and manages the monitoring of the hardware and the execution status of the program in units of processes. Here, the kernel 71 can respond to requests from the packet processing APL1, and can communicate requests from the HW 10 to the packet processing APL1. The kernel 71 processes the request from the packet processing APL1 via a system call ("a user program operating in non-privileged mode" entrusts "a kernel operating in privileged mode" to perform processing).
[0016] The kernel 71 delivers the packet to the packet processing APL1 via the socket 75. The kernel 71 receives the packet from the packet processing APL1 via the socket 75.
[0017] The ring buffer 72 is managed by the kernel 71 and is located in the memory space of the server. The ring buffer 72 is a fixed-size buffer that stores messages output by the kernel 71 as logs, and is overwritten from the beginning when the upper limit size is exceeded.
[0018] The driver 73 is a device driver for monitoring hardware by the kernel 71. It should be noted that the driver 73 depends on the kernel 71, and if the created (built) kernel source code changes, it will become another driver. In this case, the corresponding driver / source code is obtained, and the driver is rebuilt on the OS using the driver to create the driver.
[0019] The protocol processing unit 74 performs protocol processing of L2 (data link layer) / L3 (network layer) / L4 (transport layer) defined by the OSI (Open Systems Interconnection) reference model.
[0020] Socket 75 is an interface for inter-process communication of kernel 71. Socket 75 has a socket buffer, and data copy processing does not occur frequently. The process until communication is established via socket 75 is as follows: 1. The server side creates a socket file for receiving clients. 2. Name the receiving socket file. 3. Create a socket / queue. 4. Accept the first connection from the client that enters the socket / queue. 5. Create a socket file on the client side. 6. Send a connection request from the client side to the server. 7. On the server side, a connection socket file is created separately from the receiving socket file. As a result of establishing communication, the packet processing APL1 can call system calls such as read() and write() to the kernel 71.
[0021] In the above configuration, the kernel 71 receives the notification of packet arrival from the NIC 11 through a hardware interrupt (hardIRQ) and schedules a software interrupt (softIRQ) for packet processing.
[0022] The New API (NAPI) implemented by Linux kernel 2.5 / 2.6 processes data packets through software interrupts (softIRQ) after hardware interrupts (hardIRQ) when data packets arrive. Fig. 22 As shown, packet transmission based on the interrupt model is handled by interrupts (refer to Fig. 22 The data packet is transmitted by the reference numeral a), so a waiting for interrupt processing occurs, and the delay of the data packet transmission becomes large.
[0023] The following is an overview of packet processing on the NAPI Rx side.
[0024] [Rx-side packet processing structure based on New API (NAPI)]
[0025] Fig.23 Yes Fig. 22 A diagram illustrating an overview of Rx-side packet processing based on the New API (NAPI) in the area surrounded by a dotted line.
[0026] <Device Driver>
[0027] like Fig.23 As shown, the device driver is configured with: NIC11 (physical NIC), which is a network interface card; hardIRQ81, which is a handler that is called by the occurrence of a processing request of NIC11 and executes the requested processing (hardware interrupt); and netif_rx82, which is a processing function unit of a software interrupt.
[0028] <Network Layer>
[0029] The network layer is configured with: softIRQ83, which is called by the occurrence of the processing request of netif_rx82 and performs the processing of the request (software interrupt); and do_softirq84, which is the control function unit of the entity that performs the software interrupt (softIRQ). In addition, it is configured with: net_rx_action85, which is the packet processing function unit executed in response to the software interrupt (softIRQ); poll_list86, which registers the information of the network device (net_device) indicating which device the hardware interrupt from NIC11 is the interrupt of; netif_receive_skb87, which creates the sk_buff structure (a structure used to make the kernel 71 aware of what the packet is); and the ring buffer 72.
[0030] <Protocol Layer>
[0031] The protocol layer is configured with ip_rcv88, arp_rcv89, etc. as packet processing functional units.
[0032] The above-mentioned netif_rx82, do_softirq84, net_rx_action85, netif_receive_skb87, ip_rcv88, and arp_rcv89 are components (names of functions) of a program for packet processing in the kernel 71.
[0033] [Rx-side packet processing actions based on New API (NAPI)]
[0034] Fig.23 Arrows (reference numerals) b to m indicate the flow of packet processing on the Rx side.
[0035] When receiving a packet (or frame) in a frame from a counterpart device, the hardware function unit 11a of the NIC 11 (hereinafter referred to as the NIC 11) copies the received packet to the ring buffer 72 by DMA (Direct Memory Access) transfer without using the CPU (see Fig.23 The memory space of the ring buffer 72 in the server is occupied by the kernel 71 (refer to Fig. 22 ) for management.
[0036] However, the kernel 71 cannot know the data packet simply by copying the data packet arriving to the ring buffer 72 through the NIC 11. Therefore, when the data packet arrives, the NIC 11 reports the hardware interrupt (hardIRQ) to the hardIRQ 81 (refer to Fig.23 The following processing is performed by netif_rx82, so that the kernel 71 knows the data packet. It should be noted that, Fig.23 The ellipse enclosing hardIRQ 81 indicates a handler rather than a functional unit.
[0037] netif_rx82 is the function that actually does the processing. When hardIRQ81 (handler) is started (see Fig.23 The reference symbol d) stores the network device (net_device) information indicating which device the hardware interrupt from NIC 11 is interrupting as one of the information of the hardware interrupt (hardIRQ) in poll_list 86. Then, netif_rx 82 registers the extraction of the queue (referring to the content of the data packet retained in the buffer, and considering the processing to be performed next for the processing of the data packet, the corresponding queue entry is deleted from the buffer) (referring to Fig.23 Specifically, in response to the data packet being inserted into the ring buffer 72, netif_rx 82 uses the driver of NIC 11 to register the subsequent cutout of the queue in poll_list 86. Thus, the cutout information based on the queue in which the data packet is inserted into the ring buffer 72 is registered in poll_list 86.
[0038] So, in Fig.23 In the <device driver>, when a data packet is received, the NIC 11 copies the arrived data packet to the ring buffer 72 through DMA transfer. In addition, the NIC 11 starts the hardIRQ 81 (handler), and the netif_rx 82 registers the net_device in the poll_list 86, and schedules the software interrupt (softIRQ).
[0039] So far, Fig.23 The processing of a hardware interrupt in <device driver> stopped.
[0040] After that, netif_rx82 uses the queue entry information (specifically, pointer) accumulated in poll_list86 to report the data stored in the ring buffer 72 to softIRQ83 (handler) via software interrupt (softIRQ) (see Fig.23 f), notifies do_softirq84 (refer to Fig.23 denoted by reference numeral g).
[0041] do_softirq84 is a software interrupt control function unit that defines various functions of software interrupts (there are many types of data packet processing, and interrupt processing is one of them. It defines interrupt processing). Based on this definition, do_softirq84 notifies net_rx_action85, which actually performs software interrupt processing, of the (corresponding) software interrupt delegation (see Fig.23 Reference numeral h).
[0042] When it is the softIRQ's turn, net_rx_action85 is based on the net_device registered in poll_list86 (refer to Fig.23 , calling a polling routine for cutting a data packet from the ring buffer 72, cutting a data packet (refer to Fig.23 At this time, net_rx_action85 continues to harvest until poll_list86 becomes empty.
[0043] After this, net_rx_action85 notifies netif_receive_skb87 (refer to Fig.23 Reference numeral k).
[0044] netif_receive_skb87 creates a sk_buff structure, parses the contents of the data packet, and transfers the processing to the next level protocol processing unit 74 according to the type (see Fig. 22 ). That is, netif_receive_skb87 parses the contents of the data packet, and when processing is performed according to the contents of the data packet, it hands over the processing to ip_rcv88 of the <protocol layer> ( Fig.23 If it is L2, the processing is handed over to arp_rcv89 ( Fig.23 denoted by the reference numeral m).
[0045] Patent Document 1 describes a network delay control device (KBP: Kernel Busy Poll) in a server. KBP continuously monitors the arrival of data packets in the kernel through a polling model, thereby suppressing softIRQ and achieving low-delay data packet processing.
[0046] Prior art literature
[0047] Patent Literature
[0048] Patent Document 1: International Publication No. 2021 / 130828
[0049] Non-patent document 1:
[0050] Non-patent document 1: New API (NAPI), [online], [retrieved on April 4, 2004], Internet 〈URL: http: / / http: / / lwn.net / 2002 / 0321 / a / napi-howto.php3〉 Summary of the invention
[0051] Problems to be solved by the invention
[0052] However, there are the following problems with respect to either of the packet transmission based on the interrupt model and the polling model.
[0053] In the interrupt model, the kernel that receives an event (hardware interrupt) from the HW transmits the data packet through software interrupt processing for processing the data packet. Therefore, the interrupt model has the following problems: Since the data packet is transmitted through interrupt (software interrupt) processing, if there is a conflict with other interrupts or the interrupt target CPU is used by a process with a higher priority, waiting occurs, and the delay of data packet transmission becomes larger. In this case, when the interrupt processing is crowded, the waiting delay becomes larger.
[0054] For example, in packet transmission based on the interrupt model, packet transmission is performed through interrupt processing, so a wait for interrupt processing occurs, and a delay in packet transmission increases.
[0055] Supplement the mechanism for delays in the interrupt model.
[0056] In general, the kernel processes packet transmission after hardware interrupt processing and then passes it through software interrupt processing.
[0057] When a software interrupt occurs during packet transmission, the software interrupt cannot be executed immediately under the following conditions (1) to (3). Therefore, the interrupt is scheduled by a scheduler such as ksoftirqd (a kernel thread for each CPU that is executed when the software interrupt load increases), resulting in a wait of the order of ms.
[0058] (1) Conflict with other hardware interrupt processing
[0059] (2) Conflicts with other software interrupt processing
[0060] (3) When other processes or kernel threads (migration threads, etc.) with higher priority are using the interrupt target CPU
[0061] In the above conditions, the software interrupt processing cannot be executed immediately.
[0062] In addition, the same is true for data packet processing based on New API (NAPI), such as Fig.23 As shown in the dashed line n, due to the conflict of interrupt processing (softIRQ), a NW delay of the order of ms occurs.
[0063] On the other hand, if the technology described in Patent Document 1 is used, software interrupts can be suppressed by continuously monitoring the arrival of data packets, thereby achieving low-latency data packet capture. However, in order to monitor the arrival of data packets, the CPU core is occupied and CPU time is used, so power consumption increases. That is, since the kernel thread that continuously monitors the arrival of data packets occupies the CPU core and always uses CPU time, there is a problem of increased power consumption. Fig.24 and Fig.25 , explaining the relationship between workload and CPU usage.
[0064] Fig.24 This is an example of video (30FPS) data transmission. Fig.24 The workload shown is a data transmission rate of 350Mbps, with data transmission intermittently every 30ms.
[0065] Fig.25 This is a diagram showing the CPU usage rate used by the busy polling thread in the KBP described in Patent Document 1.
[0066] like Fig.25 As shown, in KBP, kernel threads perform busy polling and thus occupy the CPU core. Fig.24 The intermittent packet reception shown in the figure always uses the CPU in KBP regardless of whether a packet arrives or not, so there is a problem of increased power consumption.
[0067] The present invention has been made in view of such a background, and an object of the present invention is to transmit data packets with reduced delay in a server while suppressing excessive occurrence of HW interrupts caused by putting a polling thread to sleep and reducing power consumption.
[0068] Solutions for solving problems
[0069] In order to solve the above-mentioned problem, a delay control device in a server is adopted, which is configured in the kernel space of the OS and starts a thread that monitors the arrival of data packets using a polling model. The delay control device in the server is characterized in that it comprises: a data packet arrival monitoring unit, which monitors a polling list of information of a network device that registers information indicating which device the hardware interrupt from the interface unit is the interrupt of; a data packet cutting unit, which, when a data packet arrives, executes cutting of entries of a queue that refers to the data packet stored in a ring buffer and deletes the corresponding queue from the ring buffer; a sleep management unit, which puts the thread to sleep when a data packet does not arrive within a specified period of time, and releases the sleep of the thread through a hardware interrupt when a data packet arrives; and an idle state recovery control unit, which wakes up the thread periodically when it is sleeping or wakes up the thread just before the arrival of the data packet according to the data packet arrival timing.
[0070] Effects of the Invention
[0071] According to the present invention, it is possible to perform packet transmission with reduced delay in a server while suppressing excessive occurrence of HW interrupts due to putting a polling thread to sleep and reducing power consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 This is a schematic diagram of the structure of the delay control system in the server according to the first embodiment of the present invention.
[0073] Figure 2 It is configured in kernel space Figure 1 An example of the structure of a polling thread (delay control device within the server).
[0074] Figure 3 It is configured in user space Figure 1 An example of the structure of a polling thread (delay control device within the server).
[0075] Figure 4 It is a diagram showing an example of the polling thread operation of the intra-server delay control device of the intra-server delay control system according to the first embodiment of the present invention.
[0076] Figure 5 This is a diagram showing an example of a C-state state of the server delay control system according to the first embodiment of the present invention in a table.
[0077] Figure 6 The diagram shows a business image and a C-state transition image of a polling thread in a case where packet arrival intervals are dense in the intra-server delay control system according to the first embodiment of the present invention.
[0078] Figure 7 The diagram shows a traffic image and a C-state transition image of a polling thread when the packet arrival interval is sparse in the intra-server delay control system according to the first embodiment of the present invention.
[0079] Figure 8 The first embodiment of the present invention is a method for waking up the polling thread periodically in the server delay control system. <1> ) in the case of business image and C-state transition image of the polling thread and diagram of polling thread wakeup.
[0080] Fig. 9 This is a <pre-wakeup polling thread> (method) of the server delay control system according to the first embodiment of the present invention. <2> ) in the case of business image and C-state transition image of the polling thread and diagram of polling thread wakeup.
[0081] Fig.10 This is a flowchart showing a mode selection process of a management unit of an idle state recovery control unit of an intra-server delay control device of an intra-server delay control system according to the first embodiment of the present invention.
[0082] Fig.11 This is a flowchart showing NIC and HW interrupt processing of the intra-server delay control device of the intra-server delay control system according to the first embodiment of the present invention.
[0083] Fig.12 This is a flowchart showing the processing of the operation mode of the wake-up logic taking into account the CPU idleness of the intra-server delay control device of the intra-server delay control system according to the first embodiment of the present invention.
[0084] Fig.13 This is a schematic diagram of the configuration of a delay control system within a server according to a second embodiment of the present invention.
[0085] Fig.14 This is a diagram for explaining an example of time allocation of a downlink and an uplink in TDD of the intra-server delay control system according to the second embodiment of the present invention.
[0086] Fig.15 This is a diagram showing, in a table, the subcarrier spacing and the symbol spacing for each parameter set of the intra-server delay control system according to the second embodiment of the present invention.
[0087] Fig.16This is a flowchart showing the processing of the operation mode of the wake-up logic taking into consideration the CPU idleness of the intra-server delay control device of the intra-server delay control system according to the second embodiment of the present invention.
[0088] Fig.17 This is a hardware configuration diagram showing an example of a computer that realizes the function of an intra-server delay control device of an intra-server delay control system according to an embodiment of the present invention.
[0089] Fig.18 This diagram shows an example of applying a server internal delay control system in which a polling thread is arranged in the kernel to an interrupt model in a server virtualization environment composed of a general-purpose Linux kernel (registered trademark) and VMs.
[0090] Fig.19 This is a diagram showing an example of applying a server internal delay control system in which a polling thread is arranged in a kernel to an interrupt model in a server virtualization environment composed of containers.
[0091] Fig. 20 This diagram shows an example of applying a server internal delay control system in which a polling thread is arranged in a user space to an interrupt model in a server virtualization environment composed of a general-purpose Linux kernel (registered trademark) and VMs.
[0092] Fig.21 This is a diagram showing an example of applying a server internal delay control system in which a polling thread is configured in a user space to an interrupt model in a server virtualization environment composed of containers.
[0093] Fig. 22 This is a schematic diagram of the Rx-side packet processing based on the New API (NAPI) implemented by Linux kernel 2.5 / 2.6.
[0094] Fig.23 Yes Fig. 22 A diagram illustrating an overview of Rx-side packet processing based on the New API (NAPI) in the area surrounded by a dotted line.
[0095] Fig.24 This is a diagram showing an example of data transmission of video (30FPS).
[0096] Fig.25 This is a diagram showing the CPU usage rate used by the busy polling thread in the KBP described in Patent Document 1. DETAILED DESCRIPTION
[0097] Hereinafter, a server internal delay control system and the like in a mode for implementing the present invention (hereinafter referred to as “the present embodiment”) will be described with reference to the drawings.
[0098] [Overview] (First Embodiment)
[0099] Figure 1 This is a schematic diagram of the server delay control system of the first embodiment of the present invention. This embodiment is an example of applying to the Rx side packet processing based on the New API (NAPI) implemented by Linux kernel 2.5 / 2.6. Fig. 22 The same components are denoted by the same reference numerals.
[0100] like Figure 1 As shown, the server delay control system 1000 executes a packet processing APL1 configured in a user space available to users on a server having an OS (eg, a parent operating system), and performs packet transmission between a NIC 11 of HW connected to the OS and the packet processing APL1.
[0101] The server-internal delay control system 1000 comprises: NIC11 (physical NIC), which is a network interface card; hardIRQ81, which is a handler that is called when a processing request is generated by NIC11 and executes the requested processing (hardware interrupt); HW interrupt processing unit 182, which is a processing function unit of HW interrupt; receiving list 186; ring buffer 72; polling thread (server-internal delay control device 100); and protocol processing unit 74.
[0102] The ring buffer 72 is managed by the kernel in the memory space of the server. The ring buffer 72 is a fixed-size buffer that stores messages output by the kernel as logs, and is overwritten from the beginning when the upper limit size is exceeded.
[0103] The protocol processing unit 74 is Ethernet, IP, TCP / UDP, etc. The protocol processing unit 74 performs, for example, L2 / L3 / L4 protocol processing defined by the OSI reference model.
[0104] <Delay control device within the server>
[0105] The server internal delay control device 100 is a polling thread arranged in either the kernel space or the user space.
[0106] The server delay control device 100 includes a packet arrival monitoring unit 110 , a packet extraction unit 120 , a sleep management unit 130 , a CPU frequency / CPU idle setting unit 140 , an idle state recovery control unit 150 , and a management unit 160 .
[0107] The packet arrival monitoring unit 110 is a thread for monitoring whether a packet has arrived. The packet arrival monitoring unit 110 monitors (polls) the reception list 186 .
[0108] The packet arrival monitoring unit 110 obtains the pointer information and net_device information of the packet existing in the ring buffer 72 from the reception list 186, and passes the information (pointer information and net_device information) to the packet cutting unit 120. Here, when there are multiple packet information in the reception list 186, the information of multiple packets is passed.
[0109] When a packet arrives, the packet extraction unit 120 refers to the packet stored in the ring buffer 72 and performs extraction (hereinafter, sometimes simply referred to as extracting a packet from the ring buffer 72) to delete the entry of the corresponding queue from the ring buffer 72 based on the next processing to be performed. The packet extraction unit 120 takes out the packet from the ring buffer 72 based on the received information and passes the packet to the protocol processing unit 74.
[0110] When a plurality of data packets are accumulated in the ring buffer 72, the packet cutting unit 120 cuts the plurality of data packets collectively and transmits them to the subsequent protocol processing unit 74. It should be noted that the number of packets cut collectively is called a quota, which is also often referred to as batch processing. As for the protocol processing unit 74, since the protocol processing also processes a plurality of data packets collectively, it is high-speed.
[0111] The sleep management unit 130 puts a thread (polling thread) to sleep if a data packet does not arrive within a predetermined period of time, and releases the thread (polling thread) from sleep by a hardware interrupt (hardIRQ) when a data packet arrives (details will be described later).
[0112] The CPU frequency / CPU idle setting unit 140 sets the CPU operating frequency of the CPU core used by the thread (polling thread) to a lower level during sleep. The CPU frequency / CPU idle setting unit 140 sets the CPU idle state of the CPU core used by the thread (polling thread) to a power saving mode during sleep (details will be described later).
[0113] The idle state recovery control unit 150 wakes up the thread periodically during sleep or wakes up the thread just before the data packet arrives according to the data packet arrival timing (details will be described later). In this embodiment, an example in which the idle state recovery control unit 150 wakes up the thread periodically during sleep is described, and in the second embodiment, an example in which the thread is woken up just before the data packet arrives according to the data packet arrival timing is described.
[0114] The management unit 160 selects either a periodic wake-up control for periodically waking up the thread during sleep or a pre-wake-up control for waking up the thread just before the data packet arrives according to the data packet arrival timing (details will be described later). In addition, the management unit 160 may not select either the periodic wake-up control or the pre-wake-up control, but select the conventional method.
[0115] <Configuration of the delay control device in the server>
[0116] Figure 2 and Figure 3 Yes Figure 1 A diagram illustrating the configuration of the polling thread (delay control device 100 within the server).
[0117] Kernel space configuration of polling threads
[0118] Figure 2 It is configured in kernel space Figure 1 An example of the configuration of a polling thread (delay control device 100 within the server).
[0119] At once Figure 2 As for the server delay control system 1000 shown, a polling thread (server delay control device 100) and a protocol processing unit 74 are configured in the kernel space. The polling thread (server delay control device 100) operates in the kernel space. The server delay control system 1000 executes the packet processing APL1 configured in the user space on a server equipped with an OS, and performs packet transmission between the NIC 11 of the HW and the packet processing APL1 via a device driver connected to the OS.
[0120] It should be noted that if Figure 2 As shown, the device driver is configured with: hardIRQ 81, HW interrupt processing unit 182, reception list 186, and ring buffer 72.
[0121] A device driver is a driver for monitoring hardware.
[0122] In the method of configuring the polling thread in the kernel, the idle state recovery control unit 150 of the server delay control device 100 wakes up the thread periodically when it is in sleep mode or wakes up the thread just before the arrival of the data packet according to the data packet arrival timing. The idle state recovery control unit 150 manages HW interrupts, controls the sleep of the polling thread and the enable / disable of HW interrupts for hardIRQ81 (refer to Figure 2 xx).
[0123] The present invention can be applied to a situation where a polling thread exists inside a kernel, such as NAPI and KBP.
[0124] Userspace configuration of polling threads
[0125] Figure 3 It is configured in user space Figure 1 An example of the configuration of a polling thread (delay control device 100 within the server).
[0126] At once Figure 3 In the server delay control system 1000 shown, a polling thread (server delay control device 100) and a protocol processing unit 74 are arranged in the user space. The polling thread (server delay control device 100) operates in the user space instead of in the kernel space.
[0127] At once Figure 3 In the server delay control system 1000 shown, the polling thread (server delay control device 100) bypasses the kernel space and performs packet transmission between the device driver and the NIC 11 and the packet processing APL 1.
[0128] In the method of configuring the polling thread in the user space, the idle state recovery control unit 150 of the server delay control device 100 wakes up the thread periodically during sleep or wakes up the thread just before the arrival of the data packet according to the data packet arrival timing. The idle state recovery control unit 150 manages HW interrupts and controls the sleep of the polling thread and the enable / disable of HW interrupts for the HW interrupt processing unit 182 (refer to Figure 3 yy).
[0129] The present invention can be applied to a situation where a polling thread exists in the user space, such as DPDK.
[0130] The following describes the operation of the server delay control system 1000 configured as described above.
[0131] The present invention can be applied to any of the cases where there is a polling thread inside the kernel, such as NAPI and KBP, or where there is a polling thread in the user space, such as DPDK. The application of the case where there is a polling thread inside the kernel is used as an example for explanation.
[0132] [Rx-side data packet processing action based on the present invention]
[0133] Figure 1 to Figure 3 Arrows (reference numerals) aa to ii indicate the flow of data packet processing on the Rx side.
[0134] When receiving a packet (or frame) in a frame from a counterpart device, the NIC 11 copies the received packet to the ring buffer 72 by DMA transfer without using the CPU (see Figure 1 to Figure 3The ring buffer 72 is managed by the <device driver>.
[0135] When a packet arrives, NIC11 initiates a hardware interrupt (hardIRQ) to hardIRQ81 (handler) (see Figure 1 to Figure 3 The data packet is known by executing the following processing through the HW interrupt processing unit 182.
[0136] When hardwire81 (processor) starts (refer to Figure 1 The HW interrupt processing unit 182 stores information of a network device (net_device) indicating which device the hardware interrupt from the NIC 11 is interrupting as one of the information of the content of the hardware interrupt (hardIRQ) in the receiving list 186, and registers the cutout information of the queue. Specifically, in response to the data packet being inserted into the ring buffer 72, the HW interrupt processing unit 182 registers the cutout of the queue in the receiving list 186 (see Figure 1 to Figure 3 As a result, the cutout based on the queue in which the data packets are jammed in the ring buffer 72 is registered in the reception list 186.
[0137] The HW interrupt processing unit 182 registers net_device in the receiving list 186, but Fig.23 Unlike netif_rx82, HW interrupt processing unit 182 does not schedule software interrupts (softIRQs). Fig.23 The netif_rx82 is different.
[0138] In addition, the HW interrupt processing unit 182 performs sleep release to wake up the sleeping polling thread (see Figure 1 to Figure 3 Figure ee).
[0139] So far, Figure 1 to Figure 3 The processing of a hardware interrupt in <device driver> stopped.
[0140] In this embodiment, Fig.23 In the <Network Layer> shown, softIRQ83 and do_softirq84 are deleted, and along with this, Fig.23 The netif_rx82 shown does not notify the start of softIRQ83 (handler) (see Fig.23 Reference numeral f).
[0141] In this embodiment, the server internal delay control system 1000 deletes Fig.23Instead of the softIRQ83 and do_softirq84 shown in the figure, a polling thread (delay control device 100 in the server) is set in the <kernel space> (refer to Figure 2 Alternatively, the server delay control system 1000 sets a polling thread (server delay control device 100) in the <user space> (refer to Figure 3 ).
[0142] The packet arrival monitoring unit 110 monitors (polling) (refer to Figure 1 to Figure 3 ff) receiving list 186 to confirm whether the data packet has arrived.
[0143] The packet arrival monitoring unit 110 obtains the pointer information and net_device information of the packet existing in the ring buffer 72 from the reception list 186, and transmits the information (pointer information and net_device information) to the packet cutting unit 120 (refer to Figure 1 to Figure 3 Here, if there are a plurality of data packet information in the receiving list 186, the information of the plurality of data packets is transmitted.
[0144] The packet extraction unit 120 of the server delay control device 100 extracts the packet from the ring buffer 72 when the packet has arrived (see Figure 1 to Figure 3 Figure 1 shows the reference numeral hh).
[0145] The packet extraction unit 120 extracts the packet from the ring buffer 72 based on the received information and transfers the packet to the protocol processing unit 74 (see Figure 1 to Figure 3 ii).
[0146] [Sleep action of polling thread]
[0147] The server delay control system 1000 stops the softIRQ of the packet processing which is the main cause of the NW delay, and executes the polling thread of the packet arrival monitoring unit 110 of the server delay control device 100 to monitor the packet arrival. Then, the packet extraction unit 120 performs the packet processing by the polling model (without softIRQ) when the packet arrives.
[0148] When a packet arrives, the polling thread is called up by the hard interrupt handler, which can avoid softIRQ conflicts and enable packet transmission processing to be performed immediately. In other words, the packet arrival monitoring function is put on standby in advance and called up by a hard interrupt, thereby achieving lower latency than packet transmission processing by soft interrupts such as NAPI.
[0149] The polling thread (delay control device 100 within the server) that monitors the arrival of data packets is set to be sleepable during a period when no data packets arrive.
[0150] The polling thread (delay control device 100 in the server) sleeps according to whether a data packet arrives, and wakes up from sleep through hardIRQ81 when a data packet arrives. Specifically, the sleep management unit 130 of the delay control device 100 in the server sleeps according to whether a data packet arrives, that is, if no data packet arrives within a specified period of time. The sleep management unit 130 wakes up from sleep through hardIRQ81 when a data packet arrives. In this way, softIRQ conflicts are avoided and low latency is achieved.
[0151] The CPU frequency / CPU idle setting unit 140 of the server internal delay control device 100 changes the CPU operating frequency and idle setting according to whether the data packet arrives. Specifically, the CPU frequency / CPU idle setting unit 140 lowers the CPU frequency when sleeping, and increases the CPU frequency when restarting (restoring the CPU operating frequency to the original value). In addition, the CPU frequency / CPU idle setting unit 140 changes the CPU idle setting to power saving when sleeping. The CPU operating frequency is changed to a lower level during sleep, and the CPU idle setting is changed to power saving, thereby also achieving power saving.
[0152] Figure 4 1 is a diagram showing an example of the polling thread operation of the server delay control device 100. The vertical axis represents the CPU usage rate [%] of the CPU core used by the polling thread, and the horizontal axis represents time. Figure 4 Shown with Fig.24 The data transmission example of the video (30FPS) of intermittently receiving data packets corresponds to the polling thread operation example based on the arrival of data packets.
[0153] like Figure 4 As shown, the sleep management unit 130 of the server delay control device 100 puts the polling thread to sleep (see Figure 4 Then, the sleep management unit 130 starts the polling thread through the hardIRQ81 where the data packet arrives (see Figure 4 q).
[0154] It should be noted that during sleep, since the kernel thread does not occupy the CPU core, in addition to the use of the polling thread, the interrupt of the timer used for system stability operation enters the corresponding CPU core, or the migration thread for error handling enters the corresponding CPU core, so there is a situation where the CPU usage rate of the CPU core used by the polling thread changes (see Figure 4denoted by the reference numeral r).
[0155] [LPI (Low Power Idle) hardware control]
[0156] The CPU has a function to control the CPU idle state through hardware control, which is called LPI. LPI is often also called CPU idle or C-state. In the following, LPI is described as C-state.
[0157] When the CPU load decreases, C-state attempts to save power by shutting down part of the CPU circuit power.
[0158] Figure 5 This is a diagram showing an example of a C-state state. It should be noted that since the state definition varies depending on the CPU hardware, Figure 5 This is for reference only.
[0159] like Figure 5 As shown, the CPU idle state has levels C0 to C6, and as the time without CPU load increases, it changes to a deep sleep state. In the deep sleep state, the CPU power consumption decreases, but on the other hand, the time required for recovery is increased accordingly, which sometimes becomes a problem from the perspective of low latency.
[0160] In addition, the depth to which the CPU idle state transitions is controlled by the CPU hardware and depends on the CPU product (most of them cannot be controlled from software such as the kernel).
[0161] [Relationship between sleep control and C-state]
[0162] When the polling thread is put under sleep control, the CPU is transferred to an idle state (various states of C-state) through the hardware control of the LPI during a period when no traffic flows.
[0163] The longer the period without business, the deeper the idle state will be. In this case, the logic depends on the CPU hardware (depends on the CPU product) and cannot be controlled from the software.
[0164] Figure 6 It is a business image showing the dense arrival interval of data packets ( Figure 6 The C-state transition image of the polling thread ( Figure 6 Figure below).
[0165] exist Figure 6 In the case of dense packet arrival intervals as shown in the figure above, the transition to the idle state (each state of C-state) becomes C2 of C-state (refer to Figure 6Reference symbol s in the following figure).
[0166] Figure 7 is a service image showing the case where the packet arrival interval is sparse ( Figure 7 The C-state transition image of the polling thread ( Figure 7 Figure below).
[0167] exist Figure 7 In the case of sparse packet arrival intervals as shown in the figure above, the transition to the idle state (each state of C-state) is slow due to the long period of no traffic (refer to Figure 7 The following figure shows the reference t), and falls into a deeper idle state (here the highest level C6) (refer to Figure 7 Reference symbol u in the figure below).
[0168] If the packet arrival interval is sparse, it will fall into a deep idle state, so recovery takes time (wake up from deep sleep) (refer to Figure 7 Arrow v in the figure below).
[0169] In this way, in the case of traffic with sparse data packets arriving, the system falls into a deep idle state, and recovery from the state takes time, which sometimes causes a lag (delay) in the data packet transmission process, which is a problem.
[0170] [Basic viewpoint of the operation of the polling thread (delay control device 100 in the server)]
[0171] Reference Figure 8 and Fig. 9 , the basic viewpoints of the operation of the polling thread (delay control device 100 within the server) are explained.
[0172] The present invention periodically wakes up the polling thread in the sleep state (mode <1> )( Figure 8 ), or wake up the polling thread in advance (method <2> )( Fig. 9 ) so that the CPU idle state does not deepen when the period without data packets arrives becomes longer. It should be noted that in the method of waking up the polling thread in advance <2> In this case, the wake-up timing uses a timer, for example.
[0173] In this embodiment, the method of periodically waking up the polling thread <1> In the second embodiment, the method of waking up the polling thread in advance is described. <2> This will be described later.
[0174] <Wake up the polling thread periodically> (<mode <1> )
[0175] Figure 8 means <periodically wake up the polling thread> (<mode <1> ) in the case of business image ( Figure 8The C-state transition image of the polling thread and the polling thread wakeup ( Figure 8 Figure below).
[0176] exist Figure 8 In the case of sparse data packet arrival in the above figure, Figure 8 As shown in the figure below, the polling thread is periodically awakened ("periodic awakening"). By waking up periodically, it is prevented from falling into a deep idle state. In the case of the previous example, when the packet arrival interval is sparse, the deep idle state is prevented. Figure 7 Arrow t in the figure below), so recovery takes time (refer to Figure 7 In contrast, in the method <1> In the case where the packet arrival interval is sparse, the polling thread is periodically awakened ("periodic awakening") to prevent the CPU idle state from becoming deeper. This prevents the CPU from falling into a deep idle state, thereby speeding up the recovery time from the idle state.
[0177] <Pre-wake up polling thread> (method <2> )
[0178] Fig. 9 It means <waking up the polling thread in advance> (method <2> ) in the case of business image ( Fig. 9 The C-state transition image of the polling thread and the polling thread wakeup ( Fig. 9 Figure below).
[0179] exist Fig. 9 In the case of sparse data packet arrival in the above figure, Fig. 9 As shown in the figure below, the polling thread is woken up in advance by a timer ("pre-wake-up"). In the pre-wake-up, while falling into a deep idle state to save power as much as possible, the recovery time from the idle state is accelerated by waking up and restoring the C-state just before the arrival of a data packet.
[0180] <"Periodic wake-up" (method <1> ) and "pre-wake up" (way <2> )
[0181] "Periodic wake-up" (mode <1> ) is suitable for use in situations where the arrival timing of data packets cannot be predicted.
[0182] "Pre-wake up" <2> ) is suitable for use in cases where the arrival timing of a data packet can be predicted. For example, a case where the arrival timing of a data packet can be predicted is a RAN (Radio Access Network).
[0183] Here, during periodic wake-up / pre-wake-up, by adopting only the wake-up process, it is possible to avoid the consumption of unnecessary CPU cycles and greatly suppress the increase in power consumption caused by wake-up.
[0184] [Operation flow of polling thread (server internal delay control device 100)] <Mode selection process>
[0185] Fig.10 It is a flowchart showing the mode selection process of the management unit 160 of the idle state recovery control unit 150 of the polling thread (server internal delay control device 100).
[0186] The polling thread confirms the arrival of data packets or periodically starts this process.
[0187] In step S1, the management unit 160 of the idle state recovery control unit 150 selects an operation mode (conventional mode / mode <1> / mode <2>) that takes into account the CPU idle wake-up logic and collects the information required for switching the operation mode. The management unit 160 collects, for example, the data measured by the service measurement unit 220 ( Fig.13 ), the service mode clearly defined in advance as the technical specification of the provided service, the planned event information related to user behavior, etc., and selects (conventional mode / mode <1> / mode <2>).
[0188] In step S2, the management unit 160 determines whether the next arriving service requires low latency.
[0189] If the next arriving service does not require low latency (S2: No), in step S3, the management unit 160 determines that control considering low latency is not required, selects the conventional mode, and ends the process of this flow.
[0190] If the next arriving service requires low latency (S2: Yes), in step S4, the management unit 160 determines whether it is possible to predict future services.
[0191] If it is not possible to predict future services (S4: No), in step S5, the management unit 160 selects mode <1> and ends the process of this flow.
[0192] If it is possible to predict future services (S4: No), in step S6, the management unit 160 selects mode <2> and ends the process of this flow.
[0193] <NIC and HW interrupt processing>
[0194] Fig.11 It is a flowchart showing the NIC and HW interrupt processing of the polling thread (server internal delay control device).
[0195] While the polling thread is activated, this action flow is executed cyclically.
[0196] This process starts when a data packet arrives at the NIC 11. In step S11, the NIC 11 copies the arrived data packet to a memory area by DMA (Direct Memory Access).
[0197] In step S12, the polling thread (delay control device 100 in the server) determines whether HW interruption is allowed. If HW interruption is allowed (S12: Yes), the process proceeds to step S13, and if HW interruption is not allowed (S12: No), the process of this flow ends.
[0198] In step S13 , the NIC 11 activates the HW interrupt (hardIRQ) to the hardIRQ 81 (handler) to start the HW interrupt, and registers the packet arrival information (NIC device information, etc.) in the reception list 186 .
[0199] In step S14, when the polling thread (delay control device 100 within the server) is sleeping, the NIC 11 activates the polling thread and ends the processing of this flow.
[0200] <Poll thread action flow>
[0201] Fig.12 This is a flowchart showing the processing of the operation mode of the wake-up logic of the polling thread (delay control device in the server) taking into account the CPU idleness.
[0202] When a data packet arrives while the polling thread is sleeping, the process begins due to a HW interrupt (HW interrupt at point 1).
[0203] In step S21, the idle state recovery control unit 150 prohibits HW interruption by the NIC 11. If the HW interruption occurs during processing, the processing is interrupted, so the idle state recovery control unit 150 temporarily prohibits HW interruption by the NIC 11.
[0204] In step S22 , the CPU frequency / CPU idle setting unit 140 sets the CPU frequency of the CPU core in which the polling thread operates to a higher level, and releases the idle state if the corresponding CPU has been set to the idle state.
[0205] In step S23, the polling thread refers to the reception list 186. The polling thread learns from which device the HW interrupt has occurred, and confirms the packet arrival information in the reception list 186 in the next step S24.
[0206] It should be noted that the arrival of a data packet may be confirmed by directly referring to the ring buffer 72 instead of referring to the control plane list of the reception list 186. For example, in NAPI implemented in the Linux kernel, the control plane list of poll_list is monitored.
[0207] In step S24, the packet arrival monitoring unit 110 determines whether there is packet arrival information in the reception list 186. If there is no packet arrival information in the reception list 186 (S24: No), that is, if there is no packet to be processed, the following processing is skipped and the process proceeds to step S27.
[0208] It should be noted that, when only the polling thread is awakened and the C-state does not change to C0, etc., a small calculation can be performed here to impose a load on the CPU. In addition, the CPU load is imposed by executing the process called by the HW interrupt. Therefore, the same process as the case of the HW interrupt at point 1 can also be executed.
[0209] If there is packet arrival information in the reception list 186 (S24: Yes), in step S25, the polling thread refers to the packet data from the ring buffer 72 and transfers the corresponding data to the subsequent protocol processing unit 74. Here, when there are a plurality of data, reception processing may be performed collectively.
[0210] In step S26, the packet cutting unit 120 determines whether there are unreceived packets in the ring buffer 72. When there are unreceived packets in the ring buffer 72 (S26: Yes), the process returns to the above-mentioned step S25.
[0211] When there are no unreceived data packets in the ring buffer 72 ( S26 : No), in step S27 , the CPU frequency / CPU idle setting unit 140 sets the CPU frequency of the CPU core in which the polling thread operates to a lower level and puts the corresponding CPU into an idle state.
[0212] In step S28 , the packet arrival monitoring section 110 deletes the corresponding NIC information from the reception list 186 .
[0213] In step S29 , the idle state recovery control section 150 permits the HW interruption by the corresponding NIC.
[0214] In step S30 , the idle state recovery control unit 150 sets a timer t for periodically waking up the polling thread so that the CPU idle state does not deepen.
[0215] The above-mentioned t may be set in advance by the operator based on specifications such as the time required for transition of C-states controlled by CPU hardware and the time required for recovery from each idle state.
[0216] Furthermore, when they change dynamically, the C-state status information may be acquired from the CPU, the C-state status transition time may be learned according to business conditions, etc., and t may be dynamically determined according to the learned time.
[0217] In step S31 , the sleep management unit 130 puts the polling thread to sleep.
[0218] In step S32, the idle state recovery control unit 150 determines whether the timer t has expired. If the timer t has not expired (S32: No), the process returns to step S31. If the timer t has expired (S32: Yes), the process proceeds to step S33.
[0219] In step S33 , the idle state recovery control unit 150 wakes up the polling thread when the timer t expires, and ends the processing of this flow.
[0220] When the timer t expires, the polling thread wakes up (it only wakes up, and the C-state is restored to C0, etc.).
[0221] After that, the timer wake-up / sleep is repeated. If a data packet arrives during the repetition, the HW interrupt is triggered, and the transition to the HW interrupt at (point 1: Fig.10 the start of the process).
[0222] (Second Embodiment)
[0223] The second embodiment of the present invention is to apply the above-mentioned "pre-wake-up" (method <2> ) example.
[0224] Fig.13 FIG. 2 is a schematic diagram of a server internal delay control system according to a second embodiment of the present invention. Figure 1 The same components are denoted by the same reference numerals.
[0225] like Fig.13 As shown, the intra-server delay control system 1000A includes an intra-server delay control device 200 .
[0226] The server internal delay control device 200 is a polling thread arranged in either the kernel space or the user space.
[0227] The delay control device 200 in the server Figure 1 The server delay control device 100 further includes: a data packet arrival monitoring unit 210 and a service measurement unit 220.
[0228] The packet arrival monitoring unit 210 has Figure 1 The data packet arrives at the monitoring unit 110 with the same function.
[0229] The traffic measuring unit 220 measures traffic of inbound and outbound packets such as packet arrival.
[0230] [Specific design example of timer t]
[0231] As mentioned above, "pre-wake-up" (method <2> ) is suitable for use in situations where the arrival timing of a data packet can be predicted. For example, a situation where the arrival timing of a data packet can be predicted is RAN.
[0232] When the arrival timing of a packet can be predicted, the timer t can be set. In the following, the case of a vRAN use case is taken as an example. Specifically, the method is shown as follows <2> An example of designing the timer t in the uplink interface in the forward transport of the RAN for the target use case.
[0233] Fig.14 This is a diagram for explaining an example of time allocation of downlink and uplink in TDD (Time Division Duplex). Fig.14 This is an example of a case where uplink subframes are allocated to subframe 2 and subframe 7 (refer to non-patent literature: https: / / www.sharetechnote.com / html / 5G / 5G_FrameStructure.html).
[0234] Fig.14 In the figure, it is marked as D: downlink, S: special subframe, and U: uplink. In addition, the symbol ● indicates sleep, and the symbol ▲ indicates wake-up for C-state.
[0235] Although it depends on the multiplexing method of the RAN, in the case of time division multiplexing, the time slots used in the downlink and uplink are fixedly allocated. Therefore, except for the subframes allocated to the uplink, since no uplink data arrives, the time of timer t1 can be set and sleep can be performed in the period. The time of timer t2 can be set and sleep can be performed in the period of the subframes allocated to the uplink.
[0236] Timer t1
[0237] exist Fig.14 In the time allocation in the above figure, the time from sleep with the ● symbol to wake up for C-state with the ▲ symbol is the elapsed time of timer t1.
[0238] Timer t1 = 4 × t_sub - t_recover
[0239] For example, when t_sub = 1 ms and t_recover = 30 us, t1 is about 3.97 ms.
[0240] Among them,
[0241] Subframe interval = t_sub
[0242] Assume the recovery time from C-state = t_recover
[0243] Symbol interval = t_symbol ( Fig.15 There are specific values in the table of the parameter set)
[0244] Fig.15 It is a graph showing the subcarrier interval and symbol interval for each parameter set in a table. For example, for parameter set 0, the subcarrier interval is 15 kHz and the symbol interval is 71.4 usec. Below, as shown in parameter sets 1, 2, 3, …, there is a relationship where as the subcarrier interval [kHz] doubles, the symbol interval [usec] is halved.
[0245] Timer t2
[0246] Timer t2 = t_symbol - t_recover
[0247] When the parameter set = 1 and t_recover = 5 us, t2 is about 30 us.
[0248] [Arrival timing of vRAN service]
[0249] For vRAN, which is the target use case of mode <2>, it manages radio resources in the frequency domain and time domain, and frames arrive at the time interval of symbols, which is the smallest unit in the time direction. Therefore, it is assumed that according to the symbol interval, the polling thread is woken up in advance and recovered from the C-state.
[0250] However, since the arrival of frames varies depending on the presence or absence of the terminal's affiliation, frames do not necessarily arrive at the symbol interval. Therefore, waking up every symbol interval includes the meaning of speculative wake-up, but since the number of CPU cycles associated with wake-up is small, speculative wake-up does not contribute much to the increase in power consumption.
[0251] Below, the operation of the server internal delay control system 1000A configured as described above will be described.
[0252] The overall operation of the server internal delay control system 1000A is the same as that of Figure 1 the server internal delay control system 1000. In addition, regarding <NIC and HW interrupt handling>, it is also the same as Fig.11 the same, so the description is omitted.
[0253] <Poll thread action flow>
[0254] Fig.16 This is a flowchart showing the processing of the action mode of the wake-up logic of the polling thread (delay control device in the server) taking into account the CPU idleness. Fig.12 The same processing steps in the action flow are marked with the same figure symbols, and the description of the repeated parts is omitted.
[0255] When the polling thread is sleeping, a data packet arrives and is called by a HW interrupt (HW interrupt at point 1), and this process begins.
[0256] In step S25, the polling thread refers to the packet data from the ring buffer 72, transmits the corresponding data to the subsequent protocol processing unit 74, and proceeds to step S41.
[0257] In step S41, the data packet arrives at the traffic measurement unit 220 of the monitoring unit 210 ( Fig.13 )Measure business volume.
[0258] In step S42, the idle state recovery control unit 150 ( Fig.13 ) Based on the measured traffic volume, calculate the timer t.
[0259] In step S43, the packet cutting unit 120 determines whether or not there are any unreceived packets in the ring buffer 72. If there are any unreceived packets in the ring buffer 72 (S43: Yes), the process returns to the above-mentioned step S25.
[0260] In step S29 , the idle state recovery control unit 150 allows the HW interruption by the corresponding NIC, and the process proceeds to step S44 .
[0261] In step S44 , the idle state recovery control unit 150 sets the timer t calculated based on the traffic volume to wake up the polling thread so that the CPU idle state does not deepen.
[0262] As described above, according to the business conditions of the service to which the present invention is applied and the time required for recovery of each idle state, the above t is set so as to be able to wake up just before the timing of the next data packet arrival. In addition, in the case of irregular data packet arrival, the data packet arrival timing can also be learned and t can be determined using the learning result.
[0263] In step S33 , the idle state recovery control unit 150 wakes up the polling thread when the timer t expires, and ends the processing of this flow.
[0264] When the timer t expires, the polling thread wakes up (only waking up, C-state returns to C0, etc.). In the case where only the polling thread is woken up and the C-state does not change to C0, etc., a small calculation can be performed here to load the CPU. In addition, the CPU load is imposed by executing the process called by the HW interrupt. Therefore, the HW interrupt called by (point 1) can also be executed: Fig.16 The process is the same as the case where an HW interrupt is called at the beginning of the process).
[0265] Furthermore, when the probability of a packet arriving regularly is high, the probability of a packet arriving after the timer t expires is also high. Therefore, by transitioning to (HW interrupt at point 1) and preparing for packet reception in advance, the delay time can be suppressed.
[0266] Alternatively, the CPU idle state may be changed to a state with a smaller sequence number in stages without waking up the polling thread. For example, it may be changed from C6 to C0 or from C6 to C3 in stages. In this way, even when the accuracy of data packet arrival is low, it is possible to recover from a shallow idle state instead of a deep idle state. Even when data packets do not arrive, it is possible to ensure power saving by being in an idle state to a certain extent.
[0267] Alternatively, the wakeup by the timer t may be performed by utilizing a mechanism such as timerfd (registered trademark) in which a notification is received from the kernel after the timer time has elapsed.
[0268] [Hardware Configuration]
[0269] The server delay control devices 100 and 200 of the first and second embodiments described above are implemented by, for example, Fig.17 The illustrated configuration is implemented by a computer 900 .
[0270] Fig.17 1 is a diagram showing a hardware configuration of an example of a computer 900 that realizes the functions of the intra-server delay control devices 100 and 200 .
[0271] The computer 900 includes a CPU 901 , a ROM 902 , a RAM 903 , a HDD 904 , a communication interface (I / F) 906 , an input / output interface (I / F) 905 , and a media interface (I / F) 907 .
[0272] CPU 901 operates based on the program stored in ROM 902 or HDD 904. Figures 1 to 3 and Fig.13The control of each part of the delay control device 100 or 200 in the server shown. The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 is started, a program depending on the hardware of the computer 900, and the like.
[0273] The CPU 901 controls an input device 910 such as a mouse and a keyboard and an output device 911 such as a display via an input / output I / F 905. The CPU 901 obtains data from the input device 910 via the input / output I / F 905, and outputs the generated data to the output device 911. It should be noted that a GPU (Graphics Processing Unit: graphics processor) or the like may be used as a processor together with the CPU 901.
[0274] HDD 904 stores programs executed by CPU 901 and data used by the programs, etc. Communication I / F 906 receives data from other devices via a communication network (eg, NW (Network) 920) and outputs it to CPU 901, and transmits data generated by CPU 901 to other devices via the communication network.
[0275] The media I / F 907 reads the program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads the target processing program from the recording medium 912 to the RAM 903 via the media I / F 907, and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phasechange rewritable disk), an optical magnetic recording medium such as an MO (Magneto Optical disk), a magnetic recording medium, a conductive memory tape medium, or a semiconductor memory.
[0276] For example, when the computer 900 functions as the server delay control device 100, 200 constituting one of the devices of the present embodiment, the CPU 901 of the computer 900 implements the function of the server delay control device 100, 200 by executing the program loaded on the RAM 903. In addition, the data in the RAM 903 is stored in the HDD 904. The CPU 901 reads the program of the target process from the recording medium 912 and executes it. In addition, the CPU 901 may also read the program of the target process from other devices via the communication network (NW 920).
[0277] [Application Examples]
[0278] (How to configure polling threads in the kernel)
[0279] like Figure 2 The polling thread (server delay control device 100) shown in FIG. 1 can be applied to a server delay control device that starts a thread in the kernel that monitors the arrival of data packets using a polling model. In this case, the OS is not limited. In addition, it is not limited to a server virtualization environment. Therefore, the server delay control system can be applied to Fig.18 and Fig.19 The components shown.
[0280] <Application example to VM configuration>
[0281] Fig.18 FIG. 1 is a diagram showing an example of applying a server internal delay control system 1000B to an interrupt model in a server virtualization environment composed of a general-purpose Linux kernel (registered trademark) and VMs. Figure 1 , Fig.13 as well as Fig. 22 The same components are denoted by the same reference numerals.
[0282] like Fig.18 As shown, the intra-server delay control system 1000B is configured with intra-server delay control devices 100 and 200 in the kernel 171 of the sub-operating system 70 , and intra-server delay control devices 100 and 200 in the kernel 91 of the parent operating system 90 .
[0283] Specifically, the server includes a main operating system 90 on which a virtual machine and an external process formed outside the virtual machine can operate, and a sub-operating system 70 that operates within the virtual machine.
[0284] The parent operating system 90 includes: a kernel 91; a ring buffer 22, and a memory space in a server equipped with the parent operating system 90 is managed by the kernel 91; a receiving list 186 ( Figure 2 ), which registers information of a network device indicating which device the hardware interrupt (hardIRQ) from NIC11 is interrupting; a vhost-net module 221, which is a kernel thread; a tap device 222, which is a virtual interface created by the kernel 91; and a virtual switch (br) 223.
[0285] The core 91 includes delay control devices 100 and 200 within the server.
[0286] The kernel 91 transmits the data packet to the virtual machine 30 via the tap device 222 .
[0287] On the other hand, the sub-operating system 70 includes: a kernel 171; a ring buffer 52, and the memory space in the server having the sub-operating system 70 is managed by the kernel 171; a receiving list 186 ( Figure 2), which registers information about the network device to which the hardware interrupt (hardIRQ) from NIC11 belongs; and socket 75, which is an interface used by the kernel 171 for inter-process communication.
[0288] The core 171 includes the intra-server delay control devices 100 and 200 and a protocol processing unit 74 that performs protocol processing on the data packets that have been cut out.
[0289] The kernel 171 passes the packet to the packet processing APL1 via the protocol processing unit 74 .
[0290] As a result, in a system composed of virtual servers of VMs, data packets can be transmitted with reduced delay in the server without changing the APL in either the parent operating system 90 or the child operating system 70.
[0291] <Application example to container configuration>
[0292] Fig.19 FIG. 1 is a diagram showing an example of applying a server internal delay control system 1000C to an interrupt model in a server virtualization environment composed of containers. Figure 1 and Fig.18 The same components are denoted by the same reference numerals.
[0293] like Fig.19 As shown, the server delay control system 1000C includes a parent operating system 90 and a container structure in which the OS is replaced with a container 211. The container 211 includes a vNIC (virtual NIC) 212.
[0294] In a system composed of virtual servers such as containers, it is possible to transmit packets while reducing the delay within the server without changing the APL.
[0295] The above describes how to configure the polling thread in the kernel. Next, the following describes how to configure the polling thread in the user space.
[0296] (How to configure polling threads in user space)
[0297] like Figure 3 As shown, it can be applied to the configuration example of configuring the polling thread (delay control device 100 in the server) in the user space. In this case, the OS is not limited. In addition, it is not limited to the server virtualization environment. Therefore, the delay control system in the server can be applied Fig. 20 and Fig.21 The components shown.
[0298] <Application example to VM configuration>
[0299] Fig. 20 FIG. 1 is a diagram showing an example of applying a server internal delay control system 1000D to an interrupt model in a server virtualization environment composed of a general-purpose Linux kernel (registered trademark) and VMs. Figure 1 , Fig.13 as well as Fig.18 The same components are denoted by the same reference numerals.
[0300] like Fig. 20 As shown, the server delay control system 1000D includes a virtual machine and a parent operating system 20 on which an external process formed outside the virtual machine can operate, and the parent operating system 20 includes a kernel 21 and a driver 23. In addition, the server delay control system 1000D includes: a NIC 11 of HW connected to the parent operating system 20; a polling thread (server delay control device 100, 200) configured in the user space 60; a virtual switch 53; a child operating system 1 (50) operating in the virtual machine; and a polling thread (server delay control device 100, 200) connected to the parent operating system 20 and configured in the user space 60.
[0301] Thus, in a system composed of virtual servers of VM, in either the parent operating system 20 or the child operating system 1 (50), packet transmission can be performed with reduced delay in the server without changing the APL.
[0302] <Application example to container configuration>
[0303] Fig.21 FIG. 1 is a diagram showing an example of applying a server internal delay control system 1000E to an interrupt model in a server virtualization environment composed of containers. Figure 1 , Fig.13 as well as Fig. 20 The same components are denoted by the same reference numerals.
[0304] like Fig.21 As shown, the server delay control system 1000E has Fig. 20 The sub operating system 50 is replaced with a container configuration of a container 211. The container 211 has a vNIC (virtual NIC) 212.
[0305] In a system composed of virtual servers such as containers, it is possible to transmit packets while reducing the delay within the server without changing the APL.
[0306] <Application example to bare machine configuration (non-virtualized configuration)>
[0307] The present invention can be applied to a non-virtualized system such as a bare metal system. In a non-virtualized system, data packets can be transmitted with reduced delay in the server without changing the APL.
[0308] <Horizontal shrinking / horizontal expansion>
[0309] When the traffic volume is large and multiple NIC devices and NIC ports are used, multiple polling threads can be operated in association with them, so that the polling threads can be scaled down / scaled out while the HW interrupt frequency is controlled.
[0310] <Expansion Technology>
[0311] When the number of business flows increases, the present invention cooperates with RSS (Receive-Side Scaling) that can process inbound network traffic through multiple CPUs to increase the number of CPUs allocated to the data packet arrival monitoring thread, thereby being able to horizontally expand the network load.
[0312] <Application to PCI device I / O such as accelerators>
[0313] Although the NIC (Network Interface Card) I / O is used as an example, this technology can also be applied to the I / O of PCI devices of accelerators (FPGA / GPU, etc.). In particular, it can be effectively used in polling when receiving the response to the offload result of FEC (Forward Error Correction) to the accelerator in vRAN.
[0314] <Application to Processors Other Than CPU>
[0315] In addition to the CPU, the present invention can also be similarly applied to a processor such as a GPU / FPGA / ASIC (application specific integrated circuit) having an idle function.
[0316] [Effect]
[0317] As described above, the server delay control device 100 is configured in the kernel space of the OS and starts a thread that monitors the arrival of data packets using a polling model (see Figure 1 and Figure 2), comprising: a packet arrival monitoring unit 110, which monitors (polls) a polling list (receiving list 186) that registers information about network devices that indicate which device the hardware interrupt (hardIRQ) from the interface unit (NIC11) is interrupting; a packet cutting unit 120, which, when a packet arrives, executes cutting by referring to the packet stored in the ring buffer (ring buffer 72) and deleting the entry of the corresponding queue from the ring buffer; a sleep management unit 130, which puts the thread (polling thread) to sleep (Sleep) if the packet does not arrive within a specified period of time, and releases the sleep of the thread (polling thread) through the hardware interrupt (hardIRQ) when the packet arrives; and an idle state recovery control unit 150, which wakes up the thread periodically when sleeping or wakes up the thread just before the packet arrives according to the packet arrival timing.
[0318] By doing so, it is possible to suppress excessive occurrence of HW interrupts caused by putting the polling thread to sleep and reduce power consumption while reducing the delay in the server. In order to prevent the CPU from falling into a deep idle state due to LPI (C-state) when the packet does not arrive for a long time, the server delay control device 100 performs periodic wake-up / pre-wake-up based on the packet arrival timing, thereby suppressing the increase in delay time associated with recovery from the deep idle state (one of low delay).
[0319] In addition, since the arrival monitoring and reception processing of data packets are performed through the polling model, there will be no softIRQ conflict, which can reduce latency. In addition, when a data packet arrives during sleep, the polling thread is called up through the high-priority hardIRQ, so the overhead caused by sleep can be minimized (low latency 2).
[0320] Furthermore, during a period when no data packets arrive, the polling thread is put to sleep and the CPU frequency is controlled to be set low, thereby suppressing an increase in power consumption due to busy polling (power saving).
[0321] As described above, in the server packet transmission method / system for sending packets arriving at the NIC to the application, low latency can be achieved, and power saving can also be achieved at the same time. In particular, by performing wake-up that takes into account periodic wake-up / packet arrival timing, it is possible to avoid the problem of a longer delay time when resuming due to the polling thread sleeping CPU core falling into a deep idle state when the packet does not arrive for a long time. As a result, low latency can be guaranteed.
[0322] Furthermore, the present invention can be applied to a case where a polling thread exists inside a kernel, such as NAPI and KBP.
[0323] In addition, the delay control device in the server is configured in the user space 100, and a thread that monitors the arrival of data packets using a polling model is started (refer to Figure 1 and Figure 3 ), comprising: a packet arrival monitoring unit 110, which monitors (polls) the arrival of packets from the interface unit (NIC11); a packet cutting unit 120, which, when a packet arrives, executes cutting of the packet stored in the ring buffer (ring buffer 72) and deletes the entry of the corresponding queue from the ring buffer; a sleep management unit 130, which puts the thread (polling thread) to sleep (Sleep) if the packet does not arrive within a specified period of time, and releases the sleep of the thread (polling thread) through a hardware interrupt (hardIRQ) when a packet arrives; and an idle state recovery control unit 150, which wakes up the thread periodically when sleeping or wakes up the thread just before the packet arrives according to the packet arrival timing.
[0324] By doing so, it is possible to suppress excessive occurrence of HW interrupts caused by putting the polling thread to sleep and reduce power consumption while reducing the delay in the server to transmit packets. When there is a polling thread in the user space as in DPDK, the server delay control device 100 (see Figure 1 and Figure 3 ) can suppress the increase in delay time associated with recovery from a deep idle state and ensure low latency.
[0325] In addition, it is characterized in that the sub-operating system (sub-operating system 70) running in the virtual machine (refer to Fig.18 )(Sub-operating system 180)(Refer to Fig.19 ) has: a core (core 171); a ring buffer (ring buffer 72) (refer to Fig.18), the memory space in the server with the sub-operating system is managed by the kernel; a packet arrival monitoring unit 110, which monitors the arrival of packets from the interface unit (NIC11); a packet cutting unit 120, which, when a packet arrives, executes cutting by referring to the packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; and a protocol processing unit, which performs protocol processing on the packet after cutting, and a server delay control device 100, 200 is provided in the kernel, and the server delay control device 100, 200 starts a thread (thread) that uses a polling model to monitor the arrival of packets, and the server delay control device 100, 200 has: a sleep management unit 130, which puts the thread (polling thread) to sleep (Sleep) when the packet does not arrive within a specified period of time, and releases the sleep of the thread (polling thread) through a hardware interrupt (hardIRQ) when the packet arrives; and an idle state recovery control unit 150, which wakes up the thread periodically when sleeping or wakes up the thread just before the arrival of the packet according to the packet arrival timing.
[0326] As a result, in a system composed of virtual servers of VMs, for a server having a guest operating system (guest operating system 70), an increase in delay time associated with recovery from a deep idle state can be suppressed, and low latency can be ensured.
[0327] In addition, it is characterized in that the virtual machine and the parent operating system (parent operating system 90) on which the external process formed outside the virtual machine can operate (refer to Fig.18 )(Main operating system 20)(Refer to Fig. 20 and Fig.21 ) has: a core (core 91); a ring buffer (ring buffer 72) (refer to Fig. 22 ), the memory space in the server with the mother operating system is managed by the kernel; the packet arrival monitoring unit 110, which monitors the arrival of packets from the interface unit (NIC11); the packet cutting unit 120, when a packet arrives, performs cutting by referring to the packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; and the tap device 222, which is a virtual interface created by the kernel (kernel 21) (refer to Fig.18 ), the kernel includes a server delay control device 100, 200, the server delay control device 100, 200 starts a thread (thread) that uses a polling model to monitor the arrival of data packets, and the server delay control device 100, 200 includes: a data packet arrival monitoring unit 110 that monitors (polls) a polling list; a data packet cutting unit 120 that executes a reference to the data packet stored in the ring buffer (ring buffer 72) (reference Fig. 22) and deletes the entry of the corresponding queue from the ring buffer (ring buffer 72); a sleep management unit 130, which puts the thread (polling thread) to sleep (Sleep) when the data packet does not arrive within a specified period of time, and releases the sleep of the thread (polling thread) through a hardware interrupt (hardIRQ) when the data packet arrives; and an idle state recovery control unit 150, which wakes up the thread periodically when sleeping or wakes up the thread just before the data packet arrives according to the data packet arrival timing.
[0328] Therefore, in a system composed of virtual servers of VM, for a server having a kernel (kernel 171) and a parent operating system (parent operating system 90), it is possible to reduce power consumption while suppressing the increase in delay time associated with recovery from a deep idle state without changing the APL, thereby ensuring low latency.
[0329] The delay control device 200 in the server (refer to Fig.13 ) is characterized in that the idle state recovery control unit 150 has a management unit 160, which selects either a periodic wake-up control for periodically waking up the thread when it is asleep or a pre-wake-up control for waking up the thread just before the arrival of a data packet according to a timing of the arrival of the data packet.
[0330] Thus, the management unit 160 can select the periodic wake-up control that periodically wakes up the polling thread when the arrival timing of the data packet cannot be predicted, and can select the advance wake-up control that wakes up the polling thread in advance by a timer when the arrival timing of the data packet such as RAN can be predicted, which can expand the scope of application or implementation. In the first and second embodiments, since the previous method can also be selected, it can be universally applied to the existing system without change. Here, since the periodic wake-up can be applied to the situation where the arrival timing of the data packet cannot be predicted, the advance wake-up can maintain the deep LPI (C-state) as long as possible, so that the effectiveness of sleep control can be sought, and low latency and power saving can be further taken into account.
[0331] It should be noted that in the above embodiments, the case where a polling thread exists in the kernel as in NAPI and KBP is described, but the polling thread may also be configured in the user space as in DPDK (see Figure 3 , Fig.21 ).
[0332] It should be noted that, in each process described in the above embodiments, all or part of the process described as automatic execution may be manually executed, or all or part of the process described as manual execution may be automatically executed using a known method. In addition, the processing process, control process, specific name, and information including various data and parameters shown in the above text and drawings may be changed arbitrarily except for the cases specifically recorded.
[0333] In addition, the components of each device shown in the figure are conceptual functions and do not necessarily need to be physically configured as shown in the figure. That is, the specific method of distributing / integrating each device is not limited to the method shown in the figure, and all or part of them can be functionally or physically distributed / integrated in arbitrary units according to various loads, usage conditions, etc.
[0334] In addition, the above-mentioned structures, functions, processing parts, processing units, etc. can also be implemented by hardware, such as designing a part or all of them with integrated circuits. In addition, the above-mentioned structures, functions, etc. can also be implemented by software for a processor to interpret and execute the program of each function. Information such as programs, tables, files, etc. that implement each function can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or an IC (Integrated Circuit) card, an SD (Secure Digital) card, an optical disk, etc., or a recording medium such as an optical disk.
[0335] Description of reference numerals:
[0336] 1: Packet processing APL (application); 10: HW; 11: NIC (physical NIC) (interface); 20, 90: parent operating system (OS); 22, 72: ring buffer (Ring Buffer); 51, 70: sub-operating system (OS); 60: user space (userspace); 70: sub-operating system; 74: protocol processing unit; 86, 186: receiving list (polling list); 90: main operating system; 91, 171: kernel (Kernel); 100, 200: delay control device in the server (polling thread); 110, 210: data packet arrival monitoring unit; 120: data packet cutting unit; 130: sleep management unit; 140: CPU frequency / CPU idle setting unit; 150: idle state recovery control unit; 160: management unit; 220: business measurement unit 211: container (Container); 1000, 1000A, 1000B, 1000C, 1000D: delay control system in the server.
Claims
1. A delay control device in a server, configured in the kernel space of an operating system, and starting a thread for monitoring the arrival of data packets using a polling model, wherein the delay control device in the server is characterized in that it has: The data packet arrives at the monitoring unit, and the polling list of the network device that registers information indicating which device interrupted the hardware interrupt from the interface unit is monitored; a packet cutting unit, which, when a packet arrives, performs cutting by referring to the packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; A sleep management unit, which puts the thread to sleep if a data packet does not arrive within a specified period of time, and releases the thread from sleep through a hardware interrupt when a data packet arrives; as well as The idle state recovery control unit wakes up the thread periodically during sleep or wakes up the thread just before the data packet arrives according to the data packet arrival timing.
2. A delay control device in a server, configured in a user space, and starting a thread for monitoring the arrival of data packets using a polling model, wherein the delay control device in the server is characterized in that it has: A data packet arrival monitoring unit monitors the arrival of data packets from the interface unit; a packet cutting unit, which, when a packet arrives, performs cutting by referring to the packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; A sleep management unit, which puts the thread to sleep if a data packet does not arrive within a specified period of time, and releases the thread from sleep through a hardware interrupt when a data packet arrives; as well as The idle state recovery control unit wakes up the thread periodically during sleep or wakes up the thread just before the data packet arrives according to the data packet arrival timing.
3. A delay control device in a server, characterized in that: The sub-operating system running in the virtual machine has: Kernel; a ring buffer, a memory space in a server having the sub-operating system managed by the kernel; A data packet arrival monitoring unit monitors the arrival of data packets from the interface unit; a packet cutting unit, which, when a packet arrives, performs cutting by referring to the packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; as well as The protocol processing unit performs protocol processing on the data packets cut out. The kernel is provided with the server delay control device, and the server delay control device starts a thread that monitors the arrival of data packets using a polling model. The delay control device in the server comprises: a sleep management unit that puts the thread to sleep if a data packet does not arrive within a specified period of time, and releases the thread from sleep by a hardware interrupt when a data packet arrives; and The idle state recovery control unit wakes up the thread periodically during sleep or wakes up the thread just before the data packet arrives according to the data packet arrival timing.
4. A delay control device in a server, characterized in that: The virtual machine and the host operating system on which the external process formed outside the virtual machine can operate have: Kernel; a ring buffer, a memory space in a server having the mother operating system managed by the kernel; A data packet arrival monitoring unit monitors the arrival of data packets from the interface unit; a packet cutting unit that, when a packet arrives, performs cutting by referring to the packet stored in the ring buffer and deleting an entry of a corresponding queue from the ring buffer; and The tap device is a virtual interface created by the kernel. The kernel is provided with the server delay control device, and the server delay control device starts a thread that monitors the arrival of data packets using a polling model. The delay control device in the server comprises: The data packet arrival monitoring unit monitors the polling list for registering information of the network device indicating which device has interrupted the hardware interrupt from the interface unit; a packet cutting unit, which, when a packet arrives, performs cutting by referring to the packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; A sleep management unit, which puts the thread to sleep if a data packet does not arrive within a specified period of time, and releases the thread from sleep through a hardware interrupt when a data packet arrives; as well as The idle state recovery control unit wakes up the thread periodically during sleep or wakes up the thread just before the data packet arrives according to the data packet arrival timing.
5. The server delay control device according to any one of claims 1 to 4, characterized in that: The idle state recovery control unit includes a management unit, The management unit selects either a periodic wakeup control for periodically waking up the thread during sleep or a pre-awakening control for waking up the thread immediately before the arrival of a data packet according to the data packet arrival timing.
6. A method for controlling a delay in a server of a delay control device in a server, wherein the delay control device in the server is configured in the kernel space of an operating system and starts a thread for monitoring the arrival of data packets using a polling model, wherein the method for controlling a delay in a server is characterized by executing the following steps: monitoring a polling list for registering information of a network device indicating which device has interrupted a hardware interrupt from an interface unit; In case of arrival of a data packet, performing a cut referring to the data packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; If the data packet does not arrive within a specified period of time, the thread is put to sleep, and when the data packet arrives, the thread is released from sleep through a hardware interrupt; and The thread is periodically awakened during sleep or is awakened just before the data packet arrives according to the data packet arrival timing.
7. A method for controlling a delay in a server of a delay control device in a server, wherein the delay control device in the server is configured in a user space and starts a thread for monitoring the arrival of a data packet using a polling model, wherein the method for controlling a delay in a server is characterized by executing the following steps: Monitor the arrival of data packets from the interface department; In case of arrival of a data packet, performing a cut referring to the data packet stored in the ring buffer and deleting the entry of the corresponding queue from the ring buffer; If the data packet does not arrive within a specified period of time, the thread is put to sleep, and when the data packet arrives, the thread is released from sleep through a hardware interrupt; and The thread is periodically awakened during sleep or is awakened just before the data packet arrives according to the data packet arrival timing.
8. A program for causing a computer to function as the server internal delay control device according to any one of claims 1 to 4.
