COMMUNICATION CONTROL DEVICE, COMMUNICATION CONTROL METHOD, AND PROGRAM

Abstract

This communication control device comprises a plurality of reception control units which receive a packet from a server on which an application is operating, wherein the reception control units each include: a reception buffer; and a reception determination unit that transmits the packet to a transmission interface at a time when the packet can be received in the transmission interface connected to an external network, and that stores the packet in the reception buffer at a time when the packet cannot be received in the transmission interface.

Description

[Technical field]

[0001] The present invention relates to a communication device that performs time-division transmission such as Time Sensitive Networking. [Background technology]

[0002] Time Sensitive Networking (TSN), which guarantees delay and jitter on industrial and automotive Ethernet and can coexist with best-effort communication, has been attracting attention in recent years. In the future, it is expected that deterministic time communication services will be realized over a wide area, such as closed loop control between distant factories, as well as within closed Local Area Networks (LANs) in factories and automobiles.

[0003] To expand the scope of TSN applications, it is effective to realize TSN communication in virtual environments such as containers and virtual machines (VMs) in order to flexibly provide TSN services.

[0004] In addition, by sharing TSN (particularly IEEE 802.1Qbv, hereafter referred to as Qbv) time slots among multiple applications rather than assigning them to specific classes in a fixed manner, it becomes possible to accommodate a greater number of services.

[0005] On the other hand, when multiple containers / VMs (hereafter referred to as guests) exist on the same physical host and share a TSN transmission class, queuing delays may occur due to traffic collisions between guests, or order reversals may occur within the same class, making it impossible to achieve jitter guarantees. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] C. Xu, K. Rajamani and W. Felter, "NBWGuard: Realizing Network QoS for Kubernetes," Computer Science Proceeding of the 19th International Middleware Conference Industry, 2018. Summary of the Invention [Problem to be solved by the invention]

[0007] In an environment where multiple guests exist on the same physical host, when TSN communication is performed from a guest, the conventional technology disclosed in Non-Patent Document 1 and the like cannot prevent traffic collisions between guests and guarantee delays, other than by controlling the same traffic on the guest side as the host.

[0008] However, if the guest side performs the same traffic control as the host side, it is necessary to implement a complex time slot allocation process for each guest. Also, it is possible to disrupt the TSN communication of other guests due to misconfiguration or intentional attacks. In other words, there is an issue that there is no mechanism to guarantee jitter (delay fluctuation) in communication between the guest and the host without performing complex time division transmission control on the guest side.

[0009] The present invention has been made in consideration of the above points, and aims to provide a mechanism for guaranteeing jitter in TSN communication in an environment where multiple guests are installed on the same physical host, without the need for complex time-division transmission control on the guest side. [Means for solving the problem]

[0010] According to the disclosed technology, a plurality of reception control units are provided for receiving packets from a server on which an application runs, and each reception control unit: A receive buffer; a reception determination unit that transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface; A communication control device comprising: the transmission interface includes a classifier and a plurality of transmission queues; the communication control device further includes a class assigning unit between the plurality of reception control units and the transmission interface; The class assignment unit A time slot and a transmission class of the packet are determined based on schedule information for time division transmission control in the transmission interface, and a transmission class identifier is assigned to the packet. A communications control device is provided. Effect of the Invention

[0011] According to the disclosed technology, in TSN communication in an environment where multiple guests are installed on the same physical host, a mechanism can be realized that compensates for jitter without complex time-division transmission control on the guest side. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining a problem. [Diagram 2] FIG. 1 is a diagram for explaining a problem. [Diagram 3] FIG. 1 is a diagram for explaining a problem. [Figure 4] FIG. 1 is a diagram for explaining an overview of an embodiment. [Diagram 5] 1 is an overall configuration diagram of a system according to an embodiment of the present invention. [Figure 6] FIG. 2 is a configuration diagram of a host according to the first embodiment. [Figure 7] FIG. 4 is a configuration diagram of a reception control unit in the first embodiment. [Figure 8] FIG. 2 is a configuration diagram of a transmission control unit in the first embodiment. [Figure 9] 13 is a process flow when a packet is received by a reception determination unit in reception determination method 1. [Figure 10] 13 shows a transfer flow from a reception buffer to a class assignment unit in reception determination method 1. [Figure 11] 13 is a flow chart showing a flow of calculation of a reception permitted time in the reception determination method 2. [Figure 12] 13 is a process flow when a packet is received by a reception determination unit in reception determination method 2. [Figure 13] 13 is a flow chart showing a flow of calculation of a reception permitted time in the reception determination method 3. [Figure 14] 13 is a process flow when a packet is received by a reception determination unit in reception determination method 3. [Figure 15] 13 is a flow chart showing a flow of calculation of a reception permitted time in the reception determination method 3. [Figure 16] FIG. 1 is a diagram illustrating the relationship of variables. [Figure 17] FIG. 11 is a diagram illustrating an example of schedule information. [Figure 18] FIG. 11 is a diagram illustrating an example of schedule information. [Figure 19] FIG. 11 is a configuration diagram of a host according to a second embodiment. [Figure 20] FIG. 11 is a configuration diagram of a reception control unit in the second embodiment. [Figure 21] FIG. 11 is a configuration diagram of a class assignment unit in the second embodiment. [Figure 22] FIG. 13 is a configuration diagram of a host according to a third embodiment. [Diagram 23] FIG. 11 is a diagram illustrating an example of schedule information. [Figure 24] FIG. 13 is a configuration diagram of a reception control unit in the fourth embodiment. [Diagram 25] FIG. 13 is a diagram illustrating an example of transmission queue allocation in the fifth embodiment. [Figure 26] FIG. 13 is a configuration diagram of a reception control unit in the fifth embodiment. [Figure 27] FIG. 13 is a configuration diagram of a transmission control unit in the fifth embodiment. [Figure 28] FIG. 23 is a configuration diagram of a host in the sixth embodiment. [Figure 29] 2 is an example of a hardware configuration of the device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0014] Below, we will explain an embodiment of a technology that prevents collisions with other traffic by performing time division reception control in the receiving interface of a relay device that connects to a server that does not support time division transmission in a network that performs time division transmission such as Time Sensitive Networking.

[0015] In the present embodiment described below, the TSN method assumes the Qbv method in which multiple transmission queues are prepared for each port of a communication device, and the gate opening and closing that controls whether transmission is allowed is controlled by time based on an internal clock; however, the present invention is not limited to the Qbv method and can be applied.

[0016] Further, in this embodiment, it is assumed that a time slot in a time-division transmission schedule of Qbv is shared by a plurality of guests (TSN Talkers), but the present invention is not limited to such an assumption.

[0017] (About the assignment) First, the problems with the technology according to the embodiment described later will be described in more detail. Fig. 1 shows an example of a configuration in which three guests (containers / VMs) exist on the same physical host and share a TSN transmission class. In such a configuration, queuing delays may occur due to traffic collisions between guests, and the transmission order may be reversed within the same transmission class. Note that Fig. 1 is a diagram for explaining the problems with the conventional technology, and the configuration itself of Fig. 1 is not a publicly available existing technology. The same is true for Fig. 2.

[0018] To address the above issues, it is possible to set up time-division transmission control on the guest side that follows the same Qbv transmission schedule as the host, as shown in Figure 2. This can prevent traffic collisions. However, this requires that the guest side assign classes and perform transmission control to perform Qbv. Assigning classes requires that the L2 network layer is implemented independently for each application, or that functions such as iptables and tc provided by Linux (registered trademark) be used.

[0019] The former, custom implementation, requires the application creator to implement up to the network layer, while the latter, functions such as iptables and tc, require privileges to use the functions on the guest side, which poses security problems. If privileges are granted to the guest, it becomes possible to interfere with time-division transmission control through misconfiguration on the guest side or intentional attacks. Also, when sharing a time slot among multiple guests, it is necessary to carry out complex time slot allocation for each guest.

[0020] On the other hand, if time-division transmission control is not performed on the guest side, the application's transmission timing is not restricted, which may result in increased queuing delays due to reversal of the enqueuing order of packets to the host-side transmission interface (hereinafter referred to as I / F), or the transmission time slot may be shifted.

[0021] An example of the above situation will be described with reference to Fig. 3. The upper part of Fig. 3 shows a transmission schedule indicating the original packet transmission order of a certain transmission class, and the lower part of Fig. 3 shows the transmission order when an interruption of traffic between guests occurs. A to E represent packets transmitted during the periods of open1 and open2.

[0022] As shown in Figure 3, in the original transmission schedule, packet D is transmitted in the transmission time slot open2. However, if packet D is enqueued without waiting for the transmission schedule, packet D will arrive before packet C, which is before the original transmission time slot open2, and C will not be able to be transmitted within one time slot. Furthermore, if D is a time-sensitive packet, D will also be transmitted earlier than expected, and jitter will increase.

[0023] Next, Prior Art 1 and Prior Art 2 will be described as examples of the prior art that have been made public.

[0024] <Conventional Technology 1> Prior art 1 is a technology disclosed in Non-Patent Document 1. Prior art 1 uses an Intermediate Functional Block device (IFB), which is an extension function of Linux (registered trademark), to transfer packets received at a virtual I / F of a host connecting to a guest to the IFB. By using a Hierarchical Token Bucket (HTB) on the IFB and specifying the maximum amount of data transmission between Pods from a Kubernetes Manifest file, it is possible to limit the transmission and reception bandwidth for each Pod.

[0025] By applying conventional technique 1, the bandwidth for each guest can be managed. However, since HTB is a method for guaranteeing the bandwidth at the expense of jitter, it cannot resolve the problem of increased queuing delays and time slot misalignment caused by traffic contention between guests.

[0026] <Conventional technology 2> As disclosed in "https: / / man7.org / linux / man-pages / man8 / tc-etf.8.html", a queue control called Earliest TxTime First (hereinafter referred to as ETF) has been introduced since Linux kernel 4.19, which performs transmission processing according to the transmission request time.

[0027] In ETF, the transmission request time is added to the communication packet as in-kernel metadata, and the packet buffer (Qdisc) inside the kernel linked to the network interface uses an rbtree data structure to sort the packets in order of their transmission request time closest to the current time, and dequeues them to the ring buffer in the driver so that they arrive in time for the transmission request time.

[0028] By using ETF, packets are prevented from being sent at an unexpected timing for communication closed to the host. However, packet metadata added within the guest is usually lost when the packet is passed from the virtual interface to the host, so it cannot be used for communication between the guest and the host.

[0029] <Problems with conventional technology> As mentioned above, in an environment with multiple guests in a host, when TSN communication is performed from the guest, it is not possible to prevent traffic collisions between guests and guarantee delays. To address this issue, it is possible to implement the same traffic control on the guest side as on the host side, but this requires implementing a complex time slot allocation process for each guest, and it is also possible to disrupt the TSN communication of other guests due to configuration errors or intentional attacks.

[0030] In other words, there is a problem that there is no mechanism to compensate for jitter in communication between a guest and a host without performing complex time division transmission control on the guest side. Hereinafter, a technique according to the present embodiment for solving this problem will be described.

[0031] (Outline of the embodiment) First, an overview of this embodiment will be described with reference to Fig. 4. Fig. 4 shows an example of a host having three guests. The operation of each functional unit shown in Fig. 4 will be described in the examples described later.

[0032] In this embodiment, a buffer is provided in the host's receiving I / F that is paired with the guest's sending I / F, and receiving control is performed by opening and closing a gate in synchronization with the host's Qbv sending control.

[0033] By providing a buffer in the receiving I / F in this way, traffic flowing to the host side other than the host's sending timing can be filtered and buffered in the receiving I / F until the host's sending timing arrives. This makes it possible to prevent traffic collisions between guests and guarantee jitter.

[0034] That is, as shown in Figure 4, each guest application (TSN Talker) does not need to be aware of the transmission timing or TSN settings, and if it is not the transmission timing, the reception I / F can be controlled to buffer packets. In this way, by performing reception control in synchronization with transmission control, only traffic with the transmission timing is transferred, so packet collisions do not occur and jitter compensation can be achieved.

[0035] The system configuration and operation of this embodiment will be described in detail below.

[0036] (Example of overall system configuration) An example of the overall configuration of the system in this embodiment is shown in Fig. 5. As shown in Fig. 5, this system includes a host 100, a TSN domain 500, and a client terminal 200.

[0037] The host 100 is a physical server that accommodates multiple guests. The TSN domain 500 is a network configured with devices that support TSN (including both IEEE 802.1AS and Qbv). In FIG. 5, relay devices 300 and 400 are shown as examples of devices that configure the TSN domain 500. The client terminal 200 communicates with the guests in the host 100.

[0038] The host 100 has guests 110-1 to 110-n, and a virtual relay device 120. The multiple guests 1 to n are connected to the virtual relay device 120 in the host 100, and communicate with the client terminal 200 via the host 100's transmission / reception interface.

[0039] In this embodiment, a guest and a virtual relay device are used, but a physically wired bare metal server may be used instead of the guest, and a physical relay device may be used instead of the virtual relay device. The "guest" may be called a "server." The term "relay device" has a broad meaning including both a "virtual relay device" and a "physical relay device."

[0040] Furthermore, a "relay device" or a device including a "relay device" (eg, a host or a physical NIC in the fifth embodiment) may be called a "communication control device."

[0041] Hereinafter, the detailed configuration and operation of the host 100 will be described using embodiments 1 to 6. The embodiments 1 to 6 can be implemented in any combination.

[0042] (Example 1: Basic method) First, the basic method will be described as a first embodiment.

[0043] <Host 100 configuration> Fig. 6 shows the configuration of the host 100. In the example shown in Fig. 6, the host 100 includes guests 110-1 to 110-3, a virtual relay device 120, a schedule DB (database) 130, a transmission control unit 140, and a physical NIC 150. Each guest includes an application and a vNIC which is a virtual transmission / reception I / F.

[0044] The virtual relay device 120 includes a vNIC that connects to each guest 110, a reception control unit 121, and a class assignment unit 122. The "vNIC + reception control unit 121" may be called a reception interface.

[0045] Each guest 110 and the virtual relay device 120 are connected by their respective vNICs, and an application in each guest 110 communicates with an external network via the vNIC, the reception control unit 121, the class assignment unit 122 in the virtual relay device 120, the transmission control unit 140, and the physical NIC 150. Here, the "transmission control unit 140+physical NIC 150" may be called a transmission interface. Also, the virtual relay device 120 may include the transmission control unit 140 and the physical NIC 150 (i.e., a transmission interface).

[0046] A plurality of reception control units 121-1 to 121-3 and a transmission control unit 140 refer to the schedule DB .

[0047] In standard TSN, the terminal (Talker) that transmits a packet assigns a class identifier (VLAN (Virtual LAN) tag) for transmission control according to IEEE802.1Qbv, but in the first embodiment, the guest 110 that transmits the packet does not assign a VLAN tag, and the reception control unit 121 assigns class information as metadata of the packet based on the information in the schedule DB 130, and the class assignment unit 122 assigns a class identifier to the packet based on the class information.

[0048] <Reception control unit 121, class assignment unit 122> Fig. 7 shows an example of the configuration of the reception control unit 121 in the embodiment 1. As shown in Fig. 7, the reception control unit 121 includes a reception determination unit 11, a metadata addition unit 12, and a reception buffer 13. Note that the reception determination unit 11 may include the function of the metadata addition unit 12.

[0049] The reception determination unit 11 checks to which class the packet received from the vNIC should be assigned based on the information in the schedule DB 130, determines whether or not it is a time when the packet can be received, and then passes the packet to the metadata attachment unit 12. The detailed process of the reception determination method will be described later.

[0050] The metadata assigning unit 12 assigns class information to the packet as metadata of the packet. If the current time is a time when the packet can be received, the packet is sent to the class assigning unit 122 as is, and if the current time is a time when the packet cannot be received, the packet is stored in the receiving buffer 13, and when the time when the packet can be received arrives, the packet is sent to the class assigning unit 122.

[0051] The class assigning unit 122 acquires metadata class information from the packet and assigns a class identifier to the packet. In the case of standard TSN, a VLAN tag is assigned and a class is specified in a 3-bit Priority Code Point (PCP) area in the VLAN tag.

[0052] <Transmission control unit 140> Fig. 8 shows a configuration example of the transmission control unit 140 of the first embodiment. As shown in Fig. 8, the transmission control unit 140 includes a class classification unit 141, a plurality of transmission queues, and a transmission gate opening / closing unit 142, and performs transmission control according to IEEE802.1Qbv. Note that, although the example in Fig. 8 shows transmission queues for eight classes, this is just an example.

[0053] The classifying unit 141 distributes packets arriving at the transmission control unit 140 to transmission queues that exist for each class. The transmission gate opening / closing unit 142 transmits packets to an external network via the physical NIC 150 only during times when transmission is permitted. The time-division transmission schedule of the transmission gate opening / closing unit 142 is set by referring to the schedule DB 130.

[0054] Hereinafter, reception determination methods 1 to 3 of the reception control unit 121 in the first embodiment will be described. The relationship of each variable on the time axis used in the description is shown in FIG. 16. Note that FIG. 16 shows the relationship of t base The following shows the variables when using

[0055] <Method 1 of reception determination by the reception control unit 121 in the first embodiment> A reception determination method 1 of the reception control unit 121 in the first embodiment will be described. The reception determination unit 11 of the reception control unit 120 determines in advance from the schedule DB 130 the time T cycle , the reference time t when the transmission schedule starts 0 , offset value T from the reference time for which reception is permitted open , and the offset value T close Obtain the following.

[0056] An example of the operation of the reception determination unit 11 will be described with reference to FIG. 9 and FIG.

[0057] 9 shows a process flow of the reception determination unit 11 when receiving a packet. In S101, the reception determination unit 11 receives a packet i from the guest 110. In S102, the reception determination unit 11 receives the current timestamp value t now Get the.

[0058] In S103, the reception determination unit 11 calculates an offset value T now T now =(t now -t 0 ) mod T cycle In S104, the reception determination unit 11 calculates T now ≧T open If Yes, the process proceeds to S105, and if No, the process proceeds to S107. now ≦T close If the answer is Yes, the process proceeds to S106, and if the answer is No, the process proceeds to S108.

[0059] In S106, the reception determination unit 11 transfers the packet i to the class assignment unit 122. In S107, the reception determination unit 11 determines the next reception permission time t i open =t now +(T open -T now )+T openBased on this, the packet i is stored in the reception buffer 13 having the rbtree data structure. In S108, the reception decision unit 11 determines whether the next reception permission time t i open =t now +(Tcycle-T now )+T open Based on this, packet i is stored in the receive buffer 13 of the rbtree data structure. The receive permission time is the time at which the transmission control unit 140 can start receive processing, and may be called the receive start time or receive processing start time.

[0060] 10 shows a transfer flow from the reception buffer 13 to the class assignment unit 122. In S201, the reception determination unit 11 transfers the current timestamp value t now In S202, the reception determination unit 11 obtains the smallest t i open t now If Yes (packet present), the process proceeds to S203, and if No, the process proceeds to S204.

[0061] In S203, the reception decision unit 11 transfers the packet i to the class assignment unit 122 and deletes it from the reception buffer 13. In S204, the reception decision unit 11 waits, and in S205, the reception decision unit 11 checks whether "new data has been inserted into the rbtree and min{t i open If it is Yes, the process proceeds to S206, and if it is No, the process returns to S201. i open}-t now After the time has elapsed, the process returns to S201.

[0062] <Method 2 of reception determination by the reception control unit 121 in the first embodiment> A second reception determination method of the reception control unit 121 in the first embodiment will be described. In order to reduce the amount of processing calculations during packet reception, the reception determination unit 11 of the reception control unit 121 determines whether or not the absolute time t open , t closeis constantly calculated in a separate thread, and when a packet is received, the only thing that is done is to obtain the current time and determine whether it is within the reception permitted period.

[0063] 11 and 12, an example of the operation of the reception determination unit 11 in the reception determination method 2 will be described. The transmission process from the reception buffer is the same as in the reception determination method 1.

[0064] 11 shows a flow of calculating the reception permission time. In S301, the reception determination unit 11 calculates the cycle start time t base =t 0 The reception determination unit 11 sets the reception start time as t open =t base +T open The reception end time is calculated as t close =t base +T close It is calculated as follows.

[0065] In S303, T cycle After that, in S304, t base =t base +T cycle As base Then, the process returns to S302.

[0066] 12 shows a process flow of the reception determination unit 11 when receiving a packet. In S401, the reception determination unit 11 receives a packet i from the guest 110. In S402, the reception determination unit 11 receives the current timestamp value t now Get the.

[0067] In S403, the reception determination unit 11 now ≧t open If Yes, the process proceeds to S404, and if No, the process proceeds to S406. now ≦t close If the answer is Yes, the process proceeds to S405, and if the answer is No, the process proceeds to S407.

[0068] In S405, the reception determination unit 11 transfers the packet i to the class assignment unit 122. In S406, the reception determination unit 11 determines the reception start time t i open =t open Based on this, packet i is stored in the receive buffer 13 of the rbtree data structure.

[0069] In S407, the reception determination unit 11 determines whether the reception start time t i open =t open +T cycle Based on this, packet i is stored in the receive buffer 13 of the rbtree data structure.

[0070] <Method 3 of reception determination by the reception control unit 121 in the first embodiment> The reception determination method 3 of the reception control unit 121 in the embodiment 1 will be described. In the reception determination method 3, the amount of calculations in the flow when a packet is received is further reduced by constantly calculating canRcv, which indicates whether reception is possible, and the next transmission permission start time in the reception permission time calculation flow of the reception determination method 2. When a packet is received, the truth value canRcv is determined, and when canRcv=false, only storage in the reception buffer 13 is performed.

[0071] 13 and 14, an example of the operation of the reception determination unit 11 will be shown. The transmission process from the reception buffer 13 is the same as in the reception determination method 1.

[0072] FIG. 13 shows a flow of calculating the reception permission time. In S501, the reception determination unit 11 calculates t open =t 0 +T open , t close =t 0 +T close , canRcv=false. In S502, the reception determination unit 11 calculates the current timestamp value t now Get the.

[0073] In S503, the reception determination unit 11 determines whether t now ≧t openIf Yes, the process proceeds to S504, and if No, the process proceeds to S506. now >t close If the answer is Yes, the process proceeds to S505; if the answer is No, the process proceeds to S507.

[0074] In S505, the reception determination unit 11 determines whether canRcv=false, t open =t open +T cycle , t close =t close +T cycle In S506, canRcv=false is set, and in S507, canRcv=true is set.

[0075] 14 shows a processing flow when a packet is received by the reception determination unit 11. In S601, the reception determination unit 11 receives a packet i from the guest 110. In S602, the reception determination unit 11 determines whether or not canRcv=true, and if Yes, the process proceeds to S603, and if No, the process proceeds to S604.

[0076] In S603, the reception determination unit 11 transfers the packet i to the class assignment unit 122. In S604, the reception determination unit 11 determines the next reception permission time t i open =t open Based on this, packet i is stored in the receive buffer 13 of the rbtree data structure.

[0077] If there are multiple (n) reception permission times in one period, the first reception time t open,1 , t close,1 From the nth time t open,n , t close,n Prepare t now >t close,i If so, the next (i+1)th permitted reception time is determined.

[0078] 15 shows the flow of calculating the reception permission time in this case. In S701, i=1. In S702 to S704, the reception determination unit 11 calculates topen,i =t 0 +T open,i , t close,i =t 0 +T close,i , canRcv=false is calculated n times. In S705, i=1 is set, and in S706, the reception determination unit 11 calculates the current timestamp value t now The following process is performed n times between S707 and S712.

[0079] In S708, the reception determination unit 11 now ≧t open,i If Yes, the process proceeds to S709, and if No, the process proceeds to S713. now >t close,i If the answer is Yes, the process proceeds to S710, and if the answer is No, the process proceeds to S714.

[0080] In S710, the reception determination unit 11 determines whether canRcv=false, t open,i =t open,i +T cycle , t close,i =t close,i +T cycle In S713, canRcv=false is set, and in S714, canRcv=true is set.

[0081] <Schedule information managed in the schedule DB 130> Next, an example of schedule information managed in schedule DB 130 will be described. Schedule DB 130 manages schedule information for time division transmission control in reception control unit 121 and transmission control unit 140. An example of schedule information represented in a three-layer data structure is shown below. (a) in Fig. 17 shows the first layer, (b) shows the second layer, and (c) shows the third layer.

[0082] The first layer stores the relative time when transmission starts (gate open) and the relative time when transmission ends (gate close) for a timeslot, and the transmission class for each timeslot is stored in the second layer. The third layer manages the flow information (receiving interface on the host (relay device) side, flow identifier, transmission amount) sent within the transmission class.

[0083] The relative transmission start / end time is the reference time t 0 The flow identifier may be one or more of packet header information such as sender / receiver MAC addresses, sender / receiver IP addresses, sender / receiver port numbers, protocol type, VLAN number, VLAN PCP, DSCP value, etc., or may be a hash value calculated from them.

[0084] The transmission control unit 140 changes the transmission process of the queue linked to the transmission class in a time-division manner based on the information of the first and second layers.

[0085] FIG. 18 shows a data structure in which schedule information is organized for each reception I / F. In FIG. 8, the second to fourth hierarchical layers correspond to the first to third hierarchical layers in FIG. 17. The reception control unit 121 associated with each vNIC sets the gate open / close value of the time slot of the second hierarchical layer as T open , T close Used as.

[0086] In addition, the metadata attachment unit 12 in the reception control unit 121 refers to the fourth layer table based on flow identifier information such as the sending and receiving IP addresses of the received packet, and attaches matching sending class information to the received packet as metadata for the packet.

[0087] When the flow identifier of a packet transmitted from a guest 110 is uniquely determined (when only one type of communication is performed from the guest), the transmission class and flow identifier transmitted in one time slot are uniquely determined.

[0088] (Example 2: Method of classifying classes in the classifier) Next, a second embodiment will be described. In the second embodiment, metadata assignment of class information, which was previously performed by each reception control unit 121, is performed collectively by the class assignment unit 122. The class assignment unit 122 needs to determine the class of a packet, and therefore refers to schedule information from the schedule DB 130. In Fig. 19 showing the configuration of the host 100 of the second embodiment, a line is shown indicating that the class assignment unit 122 refers to schedule information.

[0089] Fig. 20 shows a configuration example of the reception control unit 121 in the second embodiment. As shown in Fig. 20, the reception control unit 121 in the second embodiment includes a reception determination unit 11 and a reception buffer 13. The reception control unit 121 only performs a determination of whether reception is possible by the reception determination unit 11 and a buffer process when reception is impossible. Note that "whether reception is possible" refers to whether packet reception by the transmission control unit 140 is possible or not.

[0090] Fig. 21 shows a configuration example of the class assignment unit 122 in the second embodiment. As shown in Fig. 21, the class assignment unit 122 includes a class determination unit 21 and an identifier assignment unit 22. The class determination unit 21 determines the class of a packet received from the reception control unit 121, and the identifier assignment unit 22 assigns an identifier such as a VLAN tag corresponding to the determined class. The processing is the same as that of the first embodiment, except that the class determination is performed by the class assignment unit 122 instead of the reception control unit 121.

[0091] (Example 3: A method with an interface for each communication type) Next, a description will be given of embodiment 3. In embodiments 1 and 2, when a guest 110 performs a plurality of types of communication (communications with different destination IP addresses, sending and receiving port numbers, protocol types, etc.), the amount of searches in the schedule DB 130 increases, causing delays.

[0092] Therefore, in the third embodiment, the relay device is provided with vNICs equal to the number of communications performed by the guest 110, and one vNIC is used for one communication type, thereby speeding up the class determination process.

[0093] FIG. 22 shows an example in which a certain guest 110 is connected to a virtual relay device 120 on the host 100 side by three vNICs 1 to 3.

[0094] The three vNICs 1 to 3 may belong to different networks, or may belong to the same L2 network. If they belong to the same L2 network, the guest 110 further has routing tables for each of the vNICs, and changes the routing table to be referenced depending on the source IP address. Figure 22 shows such an example.

[0095] Fig. 23 shows schedule information arranged for each receiving I / F by allocating only one communication to one vNIC. By using the schedule information shown in Fig. 23, the reception control unit 121 corresponding to each vNIC can specify the transmission class just by referring to the time slot information for determining whether reception is possible, so there is no need to refer to the DB just for determining the transmission class. This reduces the amount of searches and delays.

[0096] (Embodiment 4: Method in which the reception control unit 121 has a reception buffer for each time slot) Next, a fourth embodiment will be described. In the fourth embodiment, the reception control unit 121 prepares a reception buffer for one cycle of time slots in advance, and performs the reception determination and extraction from the reception buffer on a packet-by-packet basis, but performs the process on a time slot-by-time basis, thereby reducing the amount of calculation in the reception determination unit 11.

[0097] Fig. 24 shows an example of the configuration of the reception control unit 121 in the embodiment 4. As shown in Fig. 24, the reception control unit 121 in the embodiment 4 includes a reception determination unit 11, a transmission determination unit 15, and a reception buffer for each time slot.

[0098] The reception determination unit 11 determines in which time slot a packet received from a vNIC should be transmitted based on information in the schedule DB 130, and stores the packet in a reception buffer corresponding to the time slot in which the packet will be transmitted.

[0099] The transmission decision unit 15 reads packets from the reception buffers for each time slot according to the same schedule as that of the transmission control unit 140 based on the information in the schedule DB 130 and transmits the packets to the class assignment unit 122 .

[0100] (Example 5: Method of dividing transmission queues by time slots) Next, a fifth embodiment will be described. In the fifth embodiment, the reception control unit 121 does not control the reception timing by using a reception buffer, but the transmission control unit 140 distributes communications of the same class within one period of the transmission schedule to different transmission queues, thereby preventing deviation of the transmission schedule.

[0101] In the fifth embodiment, when the same transmission class is assigned multiple times to a schedule in one period, a separate transmission queue is prepared for each. Fig. 25 shows an example of transmission permission classes in one period. In Fig. 25, for example, 1-3 indicates the third transmission queue corresponding to transmission permission class 1. For example, 1-1, 1-2, and 1-3 indicate that different transmission queues are assigned to the same transmission class 1.

[0102] For convenience of explanation, one transmission class is assigned at the same time, but it is also possible to assign multiple transmission classes at the same time.

[0103] Fig. 26 shows a configuration example of the reception control unit 121 in the fifth embodiment. As shown in Fig. 26, the reception control unit 121 does not have a reception buffer 13, but has a class determination unit 21. Based on information in the schedule DB 130, the class determination unit 21 determines the transmission class of the received packet and the number of the time slot in one period to which the packet is assigned, assigns the transmission class and time slot information as metadata of the packet, and transmits it to the class assignment. Note that the reception control unit 121 may include a class assignment unit 122, and the reception control unit 121 may perform the class identification assignment.

[0104] As in the second embodiment, the class assignment unit 122 may directly perform the judgment and even the class assignment, instead of the reception control unit 121. In this case, the reception control unit 121 is not necessary. In this case, the class assignment unit 122 may be called the reception control unit 121. In other words, the class assignment unit 122 may be used as the reception control unit 121.

[0105] Fig. 27 shows a configuration example of the transmission control unit 140. As shown in Fig. 27, the transmission control unit 140 includes a classifying unit 141, a transmission gate opening / closing unit 142, and a plurality of transmission queues for each transmission class, and distributes packets to the transmission queues based on the transmission class and the assigned time slot information.

[0106] The transmission gate opening / closing unit 142 opens and closes the gate for each transmission queue in accordance with the schedule information. In the fifth embodiment, the transmission queue in the transmission control unit 140 plays the role of a reception buffer, thereby realizing jitter compensation.

[0107] (Example 6: Example of applying the technology according to the present invention to one physical NIC) Next, a description will be given of embodiment 6. In embodiment 6, any one or a combination of any two or more of embodiments 1 to 5 is implemented within one physical NIC 150.

[0108] Fig. 28 shows a configuration example of the host 100 in the sixth embodiment. As shown in Fig. 28, a physical NIC 150 has VFs, which are multiple virtual NICs, and a reception controller 121 (for cases other than the fifth embodiment) for each VF, and further includes a class assignment unit 122, a transmission controller 140, and a schedule DB 130. The schedule DB 130 may be provided on the host side. In the configuration shown in Fig. 28, the part consisting of the VFs, the reception controller 121, and the class assignment unit 122 may be called a "relay device."

[0109] Each guest 110 bypasses the internal processing of the host (such as network processing in the Linux kernel) and directly uses a VF on the physical NIC 150. This can be achieved by, for example, Single Root I / O Virtualization.

[0110] (Common to Examples 1 to 6) The same functions may also be responsible for other functions as the reception control unit 121, the class assignment unit 122, and the transmission control unit 140 described above. For example, the effect of the technology according to the present invention can be achieved by having the functions of the class assignment unit 122 and the transmission control unit 140 within the reception control unit 121.

[0111] Furthermore, the time at which packets are transmitted from the reception control unit 121 to the transmission control unit 141 may be set earlier by the amount of the transfer delay from the reception control unit 121 to the transmission control unit 140, after the transfer delay has been calculated in advance.

[0112] (Hardware configuration example) Any of the devices described in this embodiment (host, guest, server having the same function as the guest, relay device, virtual relay device, communication control device) can be realized by, for example, having a computer execute a program describing the processing contents described in this embodiment. The computer may be a physical machine or a virtual machine on the cloud.

[0113] The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.

[0114] Fig. 29 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 29 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are mutually connected via a bus B.

[0115] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0116] The memory device 1003 reads out and stores the program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard and mouse, buttons, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0117] (Summary of the embodiment) In the present embodiment described above, in a relay device connected to multiple servers on which communication applications are running, packet reception processing is performed at the receiving interface on the relay device side connected to each server in synchronization with a transmitting interface connected to an external network that performs time-division transmission control.

[0118] More specifically, the receiving interface determines the time slot and class in which the received packet is transmitted at the transmitting interface from the flow information and reception time of the received packet, and assigns a transmission class identifier to the packet.

[0119] As described in the first embodiment, the determined received packet is associated with the reception processing start time calculated from the transmission time of the received packet at the transmission interface, so that the packet is temporarily buffered until the reception processing start time, and the received packet is transferred from the reception interface to the physical interface immediately before transmission processing is performed at the transmission interface.

[0120] As described in the fourth embodiment, the receiving interface may be provided with an independent receiving buffer for the number of time slots included in one period of the time division transmission schedule performed by the transmitting interface, and packets may be transferred from the receiving buffer in synchronization with the time division transmission schedule performed by the transmitting interface.

[0121] As described in the third embodiment, the guest and host may be connected via multiple virtual sending and receiving interfaces, and one type of communication may be assigned to each pair of sending and receiving interfaces, thereby speeding up the class determination process in the receiving interfaces.

[0122] Furthermore, as described in the fifth embodiment, the transmitting interface may be provided with a virtual transmitting queue for each time slot of one period, and the receiving interface may determine which time slot a packet is to be transmitted in and assign an identifier to the packet, so that even packets received at a time other than the original time slot can be assigned to a transmitting queue corresponding to the original transmission time slot.

[0123] As explained in the sixth embodiment, a communication method that is one or a combination of a plurality of the above configurations may be implemented within one physical NIC.

[0124] The above-mentioned communication method does not exist in the prior art.

[0125] In other words, since the bottleneck in a normal switch is the sending side where multiple traffic streams converge, existing Linux (registered trademark) and commercially available switches do not provide a mechanism for controlling traffic on the receiving side, and the Qbv specifications also stipulate only control on the sending side.

[0126] On the other hand, there is no prior art method for performing traffic control at a plurality of receiving interfaces and for the plurality of receiving interfaces and a transmitting interface to perform traffic control in cooperation with each other, as in the technology according to the present invention.

[0127] In addition, since Qbv is controlled by the CoS value of the VLAN tag, delay guarantees within a network without VLAN tags inside a host in a virtual environment, or delay guarantees in a multi-tenant data center used by multiple different users, cannot be achieved by existing Qbv or control on the sending side alone, but can be solved by the technology of the present invention.

[0128] (Effects of the embodiment) According to the embodiment described above, it is possible to prevent an increase in queuing delay and a shift in the transmission schedule due to traffic collision with other guest machines and to guarantee jitter without performing complicated time-division transmission control on the guest machine side. In addition, it is possible to nullify the influence of setting errors on the guest machine side and DoS attacks on the TSN transmission schedule.

[0129] (Additional Note) This specification discloses at least the following communication control device, communication control method, and program. (Section 1) The system includes a plurality of reception control units for receiving packets from a server on which an application runs, and each reception control unit A receive buffer; a reception determination unit that transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface; A communication control device comprising: (Section 2) a class assignment section that determines a time slot and a transmission class of the packet based on schedule information for time division transmission control in the transmission interface, and assigns a transmission class identifier to the packet; The communication control device according to claim 1, comprising: (Section 3) The reception determination unit constantly calculates the start and end of the receivable time based on schedule information for time division transmission control in the transmission interface, and compares the current time with the start and end of the receivable time to determine whether the current time is a time when the packet can be received in the transmission interface. 3. The communication control device according to claim 1 or 2. (Section 4) The communication control device includes the reception control unit for each communication type in communication with the server. 5. A communication control device according to claim 1, wherein the communication control device is a communication control device. (Section 5) The reception control unit includes the reception buffer for each time slot included in one period of a time division transmission schedule in the transmission interface. 5. A communication control device according to claim 1. (Section 6) A plurality of reception control units for receiving packets from a server on which an application runs; a transmission control unit connected to an external network and having a plurality of transmission queues for each transmission class; the reception control unit determines a transmission class and a time slot of the packet based on schedule information for time division transmission control in the transmission control unit, adds transmission class and time slot information to the packet, and transmits the packet to the transmission control unit; The transmission control unit distributes the packets to one transmission queue based on the transmission class and time slot information assigned to the packets. Communications control device. (Section 7) A communication control method in a communication control device having a plurality of reception control units for receiving packets from a server on which an application runs, comprising: Each reception control unit includes a reception buffer, transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface. Communications control method. (Section 8) A program for causing a computer to function as each unit in a communication control device according to any one of claims 1 to 6.

[0130] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0131] 11 Reception determination unit 12 Metadata assignment section 13 Receive Buffer 15 Transmission decision unit 21 Class Judgment Department 22 Identifier Assignment Unit 100 hosts 110 Guests 120 Virtual relay device 121 Reception control section 122 Class Granting Department 130 Schedule DB 140 Transmission control section 141 Classification Section 142 Transmission gate opening / closing section 150 physical NICs 200 client terminals 300, 400 repeater 500 TSN Domains 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device

Claims

1. The system includes a plurality of reception control units for receiving packets from a server on which an application runs, and each reception control unit A receive buffer; a reception determination unit that transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface, the transmission interface includes a classifier and a plurality of transmission queues; the communication control device further includes a class assigning unit between the plurality of reception control units and the transmission interface; The class assignment unit A time slot and a transmission class of the packet are determined based on schedule information for time division transmission control in the transmission interface, and a transmission class identifier is assigned to the packet. Communications control device.

2. The reception determination unit constantly calculates the start and end of the receivable time based on schedule information for time division transmission control in the transmission interface, and compares the current time with the start and end of the receivable time to determine whether the current time is a time when the packet can be received in the transmission interface. The communication control device according to claim 1 .

3. The communication control device includes the reception control unit for each communication type in communication with the server. The communication control device according to claim 1 or 2.

4. The reception control unit includes the reception buffer for each time slot included in one period of a time division transmission schedule in the transmission interface.

4. The communication control device according to claim 1.

5. A plurality of reception control units for receiving packets from a server on which an application runs; a transmission control unit connected to an external network and having a plurality of transmission queues for each transmission class; the reception control unit determines a transmission class and a time slot of the packet based on schedule information for time division transmission control in the transmission control unit, adds transmission class and time slot information to the packet, and transmits the packet to the transmission control unit; The transmission control unit distributes the packets to one transmission queue based on the transmission class and time slot information assigned to the packets. Communications control device.

6. A communication control method in a communication control device having a plurality of reception control units for receiving packets from a server on which an application runs, comprising: a communication control method in which each reception control unit has a reception buffer, transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface, the transmission interface includes a classifier and a plurality of transmission queues; the communication control device further includes a class assigning unit between the plurality of reception control units and the transmission interface; The class assignment unit A time slot and a transmission class of the packet are determined based on schedule information for time division transmission control in the transmission interface, and a transmission class identifier is assigned to the packet. Communications control method.

7. A program for causing a computer to function as each unit in the communication control device according to any one of claims 1 to 5.

Images