A precise time synchronization method based on programmable data plane
By using programmable data plane technology and DPDK+Lua scripting language framework in the network to accelerate packet processing and using machine learning to optimize synchronization errors, the problem of low accuracy in traditional time synchronization methods in high-precision scenarios is solved, and the problem of low accuracy in different situations is achieved. High-precision time synchronization under network conditions.
Patent Information
- Application Number
- CN202111631275.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-28
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-28
AI Technical Summary
Traditional time synchronization methods have problems with high deployment costs and low synchronization accuracy in high-precision synchronization scenarios, and traditional network architectures are closed and have poor openness, making it difficult to adapt to the requirements of time synchronization by rapidly developing upper-level applications.
The network card based on programmable data plane and the programmable data plane time synchronization technology are combined with the DPDK+Lua scripting language framework to accelerate data packet processing, and predict and optimize synchronization error parameters through machine learning methods to achieve nanosecond time synchronization.
Improve synchronization accuracy under different network conditions, achieve higher time synchronization effect, and support more complex network scenarios. Compared with traditional methods, the synchronization accuracy of no-load links reaches 25 nanoseconds, and the synchronization accuracy of simulated complex links reaches 35. Nanoseconds.
Smart Images

Figure CN114337893B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of time synchronization technology, and specifically, to a precise time synchronization method and system based on a programmable data plane. The emergence of the present invention is due to the fact that in application scenarios such as industrial automation, distributed multi-point synchronization systems, and ultra-high-definition audio and video production and broadcasting networks, the time synchronization accuracy in the network is very sensitive, which requires corresponding technologies to enable the network to support higher time synchronization accuracy. Traditional synchronization methods cannot meet the requirements of high-precision synchronization scenarios due to high actual deployment costs and low synchronization accuracy. At the same time, due to the characteristics of closedness and poor openness of traditional network architectures, it is difficult to adapt to the requirements of time synchronization for the current rapidly developing upper-layer applications. The introduction of software-defined networks and programmable technologies provides new ideas for business scenarios such as ultra-high-definition audio and video production and broadcasting networks that require high-precision synchronization technologies. Background Art
[0002] DPTP is a data plane time synchronization protocol used for synchronization between switches and supports nanosecond-level synchronization accuracy. In the current development situation, time synchronization protocols are usually applied to industrial switches. For example, TSN (Time Sensitive Network) switches are based on IEEE 1588. These industrial switches process clock information at the control level.
[0003] PTP uses a master-slave clock synchronization method, and the master and slave clocks achieve time or frequency synchronization through interactive synchronization, status and delay measurement messages. PTP uses hardware timestamps, so the timestamp precision and accuracy are higher, and generally can achieve nanosecond-level or even higher synchronization accuracy. It is widely used in high-precision time synchronization solutions for systems such as communication transmission networks, mobile backhaul networks, smart grids, and high-speed railways.
[0004] SDN software-defined network technology decouples the control plane and data plane of network facilities, and uses a centralized controller to monitor and manage multiple network facilities through the OpenFlow protocol. In the software-defined network architecture, the control plane is logically controlled in a unified manner, and the entire network link is monitored at the same time. The southbound OpenFlow protocol is used to send control plane commands to the data layer of each device in the network. By separating control and forwarding, the unified management of network devices is strengthened, which helps to simplify the deployment steps on each device. Summary of the invention
[0005] Based on programmable data plane technology, the present invention proposes a technology combining network card and programmable data plane time synchronization, realizes nanosecond time synchronization, and optimizes the synchronization accuracy under different network conditions. In addition, the present invention proposes a new framework: combining the original network card receiving packet acceleration technology with the data plane time synchronization technology, using machine learning methods to optimize the synchronization error parameters obtained, thereby reducing the synchronization error, optimizing the synchronization mode, and then coordinating in different network scenarios to select the most appropriate accurate synchronization time to achieve better synchronization effect.
[0006] A method for accurate time synchronization based on a programmable data plane, characterized in that it includes processing data packets on the programmable data plane; using the DPDK+Lua scripting language framework to accelerate the sending and receiving of data packets by the network card; using a machine learning method to predict synchronization delay and link delay, and optimize the entire synchronization system;
[0007] When a request data packet is generated from the network switch, the request response timeline starts and ends when the master switch receives the corresponding response data packet. Two master-slave P4 switches are designed. The slave switch sends a synchronization request data packet to the master switch. After receiving the data packet, the master switch writes the timestamp information of its own data plane into the synchronization request data packet, and then generates a response data packet and sends it to the slave switch. After receiving the response data packet, the slave switch extracts the timestamp information carried in the data packet, which contains various delay components involved in the request-response data packet model. In the data plane, the programmable data plane provides four timestamp components: TRx, TIg, TEx, and TTx. TRx indicates the time when the data packet enters the parser; TIg indicates the time when the data packet enters the ingress port pipeline; TEx indicates the time when the data packet enters the egress port pipeline; TTx indicates the time when the data packet exits the reverse parser.
[0008] The processing of data packets in the programmable data plane comprises the following steps:
[0009] Step 201, deploy two switches, SW1 is a slave switch for clock calibration, and SW2 is a master switch for determining reference time;
[0010] Step 202, the master switch SW2 obtains the timestamp value TExt from the external clock source as the reference clock TRefsw of the current data plane slave switch SW1, and stores the timestamp TRespIg of the ingress port pipeline obtained inside the data plane SW2 into the register, and records the current timestamp as Toffset (Toffset=TRespIg);
[0011] Step 203, the x-bit counter remains not cleared within a time period of 2^x-1 seconds. In order to ensure a longer time to avoid the counter being cleared, a register needs to be added. When the counter is about to be cleared, the register will record the time before the clearing, and this time information is recorded as Tera;
[0012] Step 204, when the main switch SW2 receives a synchronization request data packet, it reads the clock source timestamp value TRefsw obtained from the outside, the timestamp Tera of the register corresponding to the time period, and the time offset variable Toffset in the ingress port pipeline;
[0013] Step 205, while the main switch SW2 is in the egress pipeline, it will read the current egress timestamp TRespEg;
[0014] Step 206, using the following formula to calculate the current timestamp TNowsw after adding the reference time TRefsw, at this time TNowsw is calculated from the timestamp T_offset of the ingress pipeline, and the current timestamp value is:
[0015]
[0016] Step 207: Since the variable TNowsw is inaccurately calculated using the time value of the egress pipeline TRespEg in order to avoid delay in the egress queue, whenever the switch receives a request packet from another switch, step 206 will be repeated to calculate the variable value of TNowsw using TRespEg;
[0017] Step 208, the slave switch SW1 sends a request data packet at time TReqTx; the request data packet is received by the master switch SW2 at time TReqRx, and the master switch SW2 reads the clock source timestamp value TRefsw2 obtained from the outside;
[0018] Step 209, in the data plane, the master switch SW2 embeds TRefsw2, TReqRx and TRespEg time variables into the request data packet by executing the action matching table and matching the flow table. The request data packet is then sent out by the master switch as a response data packet and forwarded back to the slave switch SW1;
[0019] Step 210, the slave switch SW1 receives the response data packet at the time TRespRx, repeats step 206, and calculates the current time TNowsw2 of the master switch SW2;
[0020] Step 211, the slave switch SW1 starts processing the response data packet at time TRespIg, and at this time, TRespIg is written into the register through the data plane register, which is recorded as Toffset;
[0021] Step 212, calculate the reference clock of the current slave switch. The slave switch SW1 also needs to receive the accurate time when the data packet is sent by the master switch. Therefore, the master switch SW2 will subsequently send a Follow_up packet, which contains the timestamp of the master switch accurately sending the response data packet, recorded as TRespTx.
[0022] Step 213, after obtaining the TRespTx variable value from the switch SW2, the total delay is calculated as RespD using the formula:
[0023]
[0024] Step 214, the current time TNowsw2 of the master switch SW2 and the total delay RespD can be used to accurately calculate the synchronization time of the slave switch SW1 through the following formula:
[0025]
[0026] Step 215, repeat steps 201 to 214.
[0027] Furthermore, the DPDK+Lua scripting language framework is used to accelerate the sending and receiving of data packets by the network card. DPDK is used to accelerate the sending and receiving of data packets by the network card in hardware, and Lua scripting language is used to implement the initialization and sending of synchronization request packets in software.
[0028] Using the DPDK+Lua scripting language framework to accelerate the sending and receiving of data packets by the network card includes the following steps:
[0029] Step 301, install DPDK on the host, configure Hugepages, and load the kernel module;
[0030] Step 302, using Lua scripting language to define a synchronization request packet and a packet sending rule;
[0031] Step 303, in the initialization phase, when there is no cross traffic in the link, the host will first send a DPTP detection packet to the programmable switch, and the switch will reply to the host with various timestamps TNowsw, TRespRx, TReqTx, TReqRx, TRespTx; TNowsw is the current slave switch time, TRespRx is the time when the response packet arrives at the network card, TReqTx is the time when the host sends the detection packet, TReqRx is the time when the switch receives the detection packet, and TRespTx is the time when the egress port sends the response packet;
[0032] Step 304, in the initialization phase, the link rate R=0% at this time, and the link delay NicWireDelay at this time is calculated according to the following formula:
[0033] NicWireDelay=(T RespRx -T ReqTx )-(T RespTx -T ReqRx )
[0034] Step 305, in the synchronization phase, the host reference time RespD is calculated according to the following formula, TRespTx is the time when the switch sends a response packet, and TRespEg is the time when the response data packet comes out of the switch output port pipeline:
[0035] RespD=OWD+(T RespTx -T RespEg )
[0036] Step 306, determine the uncalculated delay OWD. If the link rate R is approximately equal to 0% during the synchronization phase, then OWD = NicWireDelay / 2. Once R is not 0%, the NicWireDelay will increase linearly. In order to ensure the accuracy of synchronization and the clock on the host is synchronized with the switch, the NicWireDelay calculated in the idle link during the initialization phase of step 303, which is recorded as AvgNicWireDelay, is used to calculate OWD. OWD is calculated according to the following formula:
[0037]
[0038] Step 307, using the formulas in steps 305 and 306, calculate the reference time RespD of the host, so as to achieve synchronization between the host and the programmable switch;
[0039] Step 308, repeat steps 301 to 307.
[0040] Furthermore, machine learning methods are used to predict and compensate for synchronization delay and link delay, optimize the entire synchronization system, and count the total delay and one-way link delay between switches under different network conditions with and without load. A recurrent neural network LSTM is used to predict and compensate for the total delay and one-way link delay between switches. At the beginning of synchronization, the delay at this time is counted and used as the input layer to predict the current output delay, and the value at this time is used as the input layer for iteration;
[0041] The machine learning method is used to predict and compensate for synchronization delay and link delay, and the optimization of the entire synchronization network system includes the following steps:
[0042] Step 401, according to step 213, calculate the total delay RespD between the master and slave switches;
[0043] Step 402, according to step 306, calculating the uncalculated delay OWD between the host and the switch;
[0044] Step 403, counting the total delay and one-way link delay between switches under different network states;
[0045] Step 404, after the entire network is synchronized, due to the existence of cross traffic, various congestion, and packet loss in the network, RespD and OWD will have different changes. RespD and OWD are taken as input x and put into the recurrent neural network LSTM to predict new RespD and OWD as output h;
[0046] Step 405, predict a new RespD, and repeat step 214 to calculate the synchronization time of the slave switch SW1;
[0047] Step 406, using the predicted new OWD, repeat step 305 to calculate the reference time of the host;
[0048] Step 407, repeating steps 401 to 405, and continuously updating the synchronization time of the slave switch and the host.
[0049] The present invention embeds the timestamp variable value of each component into the synchronization data packet in the data plane, and also achieves the synchronization accuracy within 25 nanoseconds under no-load link compared with the traditional time synchronization method. At the same time, the synchronization accuracy is measured within 35 nanoseconds by simulating the situation of complex links. When building the actual physical topology, use the p4c compiler to compile it into a json file and configure it in the programmable network device, and install the DPDK operating environment on the server-side network card to synchronize the network card and the switch. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 :Technical architecture of network card (server) and switch time synchronization
[0051] Figure 2 : host.lua function call process
[0052] Figure 3 :P4 code corresponding analysis diagram
[0053] Figure 4 : Switch part table matching rules
[0054] Figure 5 :RNN network structure
[0055] Figure 6 :LSTM network structure
[0056] Figure 7 :Synchronous system logical topology diagram
[0057] Figure 8 : Synchronous information of two P4 switches
[0058] Fig. 9 : Execute the host.lua code file result
[0059] Fig.10 : Execute the host.lua code file on the network card side DETAILED DESCRIPTION
[0060] The present invention proposes a test scheme for network card (server) and switch time synchronization technology, and provides a complete technical architecture diagram.
[0061] 1. Network topology design
[0062] The present invention designs a simple data center network topology to verify and test the proposed network card (server) and time synchronization technology. The test includes 4 host hosts (2 servers, and the two ports of each server network card are regarded as 1 host) and 2 P4 switches (1 physical P4 switch is virtualized into 2). The specific network topology is shown in the figure. Figure 7 shown.
[0063] The entire synchronization technology test experiment includes a Barefoot Tofino switch, which is connected to two Huawei servers. The servers use Intel X520-DA2 network cards (supporting 10G / 25G). The P4 physical switch uses 10G optical fiber to connect the switch's ports 132 and 168 to form a loopback link, which is used to virtualize a physical switch into two switches. For the server network card, X520-DA2 itself supports obtaining the hardware timestamp of the network card, connecting the network card's port 0 to the P4 switch's port 154, and the network card's port 1 to the P4 switch's port 136, to achieve the "network card 1-P4 switch-network card 2" synchronization data packet sending and receiving link. The software environment includes the P4 switch's SDE (Software Development Environment) version bf-sde-8.9.1, the server operating system is Ubuntu-16.04-server-amd64, and the DPDK version is 20.08.
[0064] 2. Build network topology
[0065] (1) Programmable data plane
[0066] In this section, p4 code is written to implement the virtualization of the P4 physical switch and the synchronization between two P4 switches. First, the P4 compilation command is used to compile the p4 code corresponding to the business logic, and the data plane forwarding logic of the data packet is loaded into the P4 switch. The main switch sends a synchronization data packet from the switch to the main switch based on the reference clock source time information, and calculates the synchronization error between the main switch and the main switch based on the received response data packet to correct its own clock. Then, p4c is used to compile the p4 code in the P4 switch, and the compiled code is loaded into the switch. After the code is compiled, only the corresponding data packet processing logic is implemented at the data level, and then the flow table is given to the switch through the control plane, including the logic of which port to send and receive data packets from.
[0067] like Figure 8 As shown in the figure, we can see the various delay values after the two P4 switches are synchronized and the synchronization error of the two switches. As can be seen from the figure, after the synchronization data packet arrives at the master switch, the data packet parsing process takes 62 nanoseconds, the pipeline processing and queue waiting at the ingress port of the master switch takes 350 nanoseconds, the pipeline processing and data packet reverse parsing process at the egress port takes 320 nanoseconds, and the round-trip RTT from the switch sending the synchronization request data packet to receiving the response data packet is 905 nanoseconds. At the same time, it can be obtained that the absolute synchronization accuracy between the master and slave P4 switches is 6 nanoseconds.
[0068] (2) Server network card
[0069] After the P4 switch is synchronized, the network card sends a synchronization request data packet with the same header format as when the switch was synchronized. The result of executing the host.lua code is as follows Fig. 9As shown in the figure. As can be seen from the figure, the EAL (Environment Abstraction Layer) in DPDK detects and initializes the server and network card devices. The two ports of the Intel X520-DA2 network card are taken over by EAL, so that when sending and receiving synchronization request packets, the Linux kernel can be bypassed for packet processing. At the same time, the two ports of the Intel X520-DA2 network card are configured as two independent hosts, Device 0 and Device 1, respectively, and the corresponding macs are 90:E2:BA:27:A7:34 and 90:E2:BA:27:A7:35, and the network port link rate is 10000Mbit / s. In this part, the configuration of the network card, sending and receiving data packets, and accelerating the processing of data packets are mainly implemented. First, the DPDK technology is started and configured in the Linux system, including inserting the IGB_UIO module, allocating large page memory to the server, and binding the network card to a DPDK-compatible driver. Then execute the host.lua code to send the network card to the synchronized P4 switch at a rate of 2000 packets per second. The network card itself supports hardware timestamps, and the physical timestamp can be read through the network card driver API to embed the timestamp sent by the network card in the sent data packet. Finally, the server network card and switch start synchronization testing.
[0070] Execute the host.lua code on the network card side, the result is as follows Fig.10 As shown, the values of various time variables used by the network card for the time synchronization formula can be obtained. Host2 (Dev 1) sends a timesync packet, and the command is set to the response packet command (0x3). After receiving the response packet, the values of various time variables defined in the synchronization packet header can be obtained. When the command is a command to capture the timestamp of the sent packet (0x6), the values of various time variables can also be calculated. From the figure, we can get the response packet information returned by the switch. For example, the time it takes for the switch to process the network card request packet is 53 nanoseconds. In the patent, the absolute synchronization error between Host1 and the P4 switch (single hop) is 7 nanoseconds.
[0071] 3. Overall framework of the invention
[0072] The overall framework of the present invention mainly includes three parts: programmable data plane to process data packets, use DPDK+Lua scripting language framework to accelerate the sending and receiving of data packets by the network card, and use machine learning methods to predict synchronization delay and link delay to optimize the entire synchronization system.
[0073] In the part of processing packets on the programmable data plane, according to the master-slave synchronization principle of the programmable data plane, the header format of the custom synchronization packet is implemented by writing p4 code, the data plane obtains the timestamp, the logical code embedded in the synchronization packet, and the control plane is used to send the corresponding flow table to the switch, so as to achieve synchronization between two or more P4 switches. When the two P4 switches complete the synchronization, the continuous packet sending module implemented by the server-side network card based on DPDK and Lua scripting language starts to send synchronization request packets to the switch. After receiving the packet of the same format, the P4 switch will process the packet on the data plane through the action-matching pipeline, including writing the master clock of the switch after synchronization, and the timestamp component of the data plane. When all the time information of the P4 switch is written into the packet, the switch will send the response packet through the matching-action pipeline and forward it back to the server. After receiving the response packet, the server uses the DPDK packet acceleration technology to bypass the kernel and directly hand over the synchronization request packet to the upper-layer application to parse and calculate the time information. Based on the above scheme, the request response model of the network card (server) and the switch and the overall framework diagram of the experiment are proposed, as shown in the figure. Figure 1 shown.
[0074] The use of DPDK+Lua scripting language framework to accelerate the sending and receiving of network card data packets is divided into two steps: network card receiving packet acceleration and software implementation of sending and receiving packets. In the acceleration of network card receiving packets, considering that the traditional data packet processing method is through CPU interruption, memory copy, context switching and memory management, this will lead to inaccurate delay measurement. Therefore, the DPDK technology for accelerating data packet processing is adopted, which can bypass the Linux kernel protocol stack, reduce the extra time overhead caused by memory copy and management, and improve synchronization accuracy. In the software implementation of sending and receiving packets, Lua scripting language is used. MoonGen, a high-rate packet generator developed based on DPDK packet acceleration technology, uses DPDK technology at the bottom layer. On the one hand, as a "decorator" of DPDK, MoonGen can make full use of the characteristics and advantages of DPDK technology to support accelerated data packet processing; on the other hand, the entire MoonGen is completely controlled by Lua scripting language. Users can develop corresponding Lua scripts according to their own business logic to receive and send data packets. Moreover, as a lightweight embedded scripting language, Lua language has the characteristics of high efficiency, portability and simplicity. In this module, according to the requirement of sending synchronous request packets in the Host-Switch system, a packet header definition and sending and receiving module based on the Lua scripting language is deployed in the network card.
[0075] The machine learning method is used to predict the synchronization delay and link delay, optimize the entire synchronization system, and use the recurrent neural network LSTM to predict and compensate for the total delay and unidirectional link delay between switches. At the beginning of synchronization, the delay at this time is counted and used as the input layer to predict the current output delay. The current value is used as the input layer for iteration to predict the new link total delay RespD and the uncalculated delay OWD, and recalculate the synchronization time between switches and between hosts and switches. This avoids factors that affect the synchronization time, such as congestion and packet loss in the network after network synchronization, further improves the synchronization accuracy, and then cooperates with different network scenarios to select the most appropriate and accurate synchronization time to achieve better synchronization effect.
[0076] Various operating principles and implementation methods of the present invention
[0077] 1. Programmable data plane processing data packets
[0078] The programmable data plane uses the P4 language. The P4 program mainly includes Header, Parser, Match-ActionTable and Control Flow. The P4 program is designed into three parts, namely Header.p4, Parser.p4 and Main.p4. In Header.p4, the data packet header that needs to be supported is declared; in Parser.p4, the data packet header is instantiated and the data packet is parsed in sequence and according to the command in the data packet; in Main.p4, the nanosecond timestamp of the switch data layer is written into the data packet, and the corresponding forwarding logic is executed at the same time, as well as the control flow program that defines the execution order of each matching action table.
[0079] (1)Head.p4
[0080] In Head.p4, declare the packet headers that the network device may process. In this experiment, the Ethernet protocol, UDP protocol, IPV4 protocol and timesync protocol will be processed.
[0081] The data packet format declaration in the Timesync protocol is the same as that in timesync.lua to ensure that the data packet can communicate normally between the network card and the switch. For example, the UDP protocol and IPV4 protocol will not be described in detail. After declaring the timesync protocol, you need to define a Parser to parse the data packet and write the extracted relevant information into the instance of each protocol header for the subsequent action-matching table.
[0082] (2)Parser.p4
[0083] Parser usually starts with Parser start and ends with return ingress, extracting the packet header information in order. The header needs to be instantiated before parsing. Parser parsing is as follows: Figure 3 shown.
[0084] First, Ethernet is instantiated and the Ethernet field in the data packet is extracted. The corresponding information is stored in the instantiated object. The extracted Ethernet field in the Ethernet packet header determines the next Parser flow direction. In this patent, it flows to the timesync header and IPv4 protocol support respectively. Then IPv4 and timesync are instantiated, and the information of each field of timesync in the data packet is extracted.
[0085] (3)Main.p4
[0086] The action matching table defined in the Main.p4 file and the use of registers and counters to implement the switch synchronization model complete the internal synchronization of the switch in this article and the processing of synchronization request packets from the network card. In the Main.p4 file, you first need to include the other two p4 files, and then define multiple action-matching tables to implement the writing of timestamps into packets on the data plane, including the action-matching table that implements the three-layer routing forwarding function, and the use of the register data structure to temporarily store the timestamp value in the data packet for internal time synchronization of the switch. Here we only select three consecutive action-matching tables mac_forward, copy_packet and timesync_inform_cp to introduce the code flow, such as Figure 4 shown.
[0087] As can be seen from the figure, the table mac_forward is first matched, and the destination mac address in the data packet is matched with the flow table entry in the table. If the match is successful (hit), the set_egr action is executed. In the set_egr action, the output port of the metadata and the inherent metadata output port in the P4 switch ASIC are modified according to the matched entry information. The purpose of this action is to modify the output port of the data plane data packet through the downstream flow table, and control the data packet to be forwarded from the specified port in the control plane.
[0088] Next, match the table copy_packet and match the value of mdata.command with the entry in the table. If the match succeeds, execute the modify_packet action. In the modify_packet action, there is no need to modify the data packet according to the flow table. Instead, the field information in the timesync data packet is copied to the metadata mdata, and then the updated data in the metadata is written to the timesync data packet.
[0089] The third match is table timesync_inform_cp, which also matches the value of mdata.command with the entry in the table. If the match succeeds (hit), the timesync_flag_cp_learn action is executed. In this action, there is no need to modify the packet content according to the flow table, as shown in Figure 3 As shown, it is used to notify the control plane to send a Follow_up packet.
[0090] 2. Use DPDK+Lua scripting language framework to accelerate the sending and receiving of data packets by network cards
[0091] The DPDK module is mainly used to accelerate the sending and receiving of data packets. By installing the corresponding DPDK version on the server, you need to configure Hugepages (large page memory) and load the kernel module through the command line. The DPDK application runs in the user space and uses the data plane library provided by itself to send and receive data packets, bypassing the packet processing process of the Linux kernel protocol stack. The Linux kernel regards the DPDK application (including its compilation, connection and loading methods) as an ordinary user-mode process. After the DPDK program is started, there is only one main thread, and then some child threads are created and bound to the specified CPU core to run.
[0092] The Lua scripting language framework is mainly used to implement the initialization and sending of synchronization request packets. It mainly includes two files, timesync.lua and host.lua. In timesync.lua, the header format and protocol command constants of the request packet are declared; in the host.lua file, the data packet is sent, the timestamp variable value in the data packet is extracted, and the delay calculation is completed.
[0093] (1) Define the synchronization request package timesync.lua
[0094] In timesync.lua, the packet header protocol to be processed by the network device is declared. The fields defined in the packet are used to save the timestamp value. The main fields in the packet header in timesync.lua are declared as follows:
[0095] Table 1: Main field declarations in the timesync.lua header
[0096]
[0097] At the same time, it is also necessary to set a command flag for the data packet, so that when the switch or network card receives the data packet, it can know the type of the data packet and execute different code strategies. In this patent, 7 types of data packet command flags are set, namely TYPE_REQ, TYPE_RES, TYPE_DELAY_REQ, TYPE_DELAY_RES, TYPE_CAPTURE_TX, TYPE_GENDELAY_REQ and TYPE_GENREQ. Among them, TYPE_REQ represents that the current data packet is a request data packet, TYPE_RES represents that the current data packet is a response data packet, TYPE_DELAY_REQ represents that the current data packet is a request data packet carrying a delay, TYPE_DELAY_RES represents that the current data packet is a response data packet carrying a delay, TYPE_CAPTURE_TX represents that the current data packet is a data packet carrying a capture and send timestamp, TYPE_GENDELAY_REQ is a request data packet that generates a delay, and TYPE_GENREQ is a generated request data packet. By embedding different commands into the data packet in the timesync.lua file, when the P4 switch receives a synchronization request data packet, the switch adopts different data packet processing strategies.
[0098] Corresponding to the definition of the packet header format and the packet execution command, the method of obtaining the time information in the packet and the method of adding commands to the packet are also defined, as shown in Table 2. The protocol header that needs to be processed for the synchronization request packet has been declared, and the next step is to send the packet to the P4 switch through the underlying DPDK technology, and extract the time field information from the received response packet, calculate the synchronization error, and use it to correct the server network card clock synchronization.
[0099] Table 2: Methods for obtaining time information in a data packet and adding commands to a data packet
[0100] How to add data packet command Method to obtain the value of the packet timestamp variable setCommand(int) getMagic() getCommand() setMagic(int) getReference_ts_lo() getReference_ts_hi() getEraTs() getMacTs() getIgTs() getDelta() getEgTs()
[0101] Corresponding to the definition of the packet header format and the packet execution command, the method of obtaining the time information in the packet and the method of adding commands to the packet are also defined. At this point, the protocol header that needs to be processed by the synchronization request packet has been declared complete. The next step is to send the packet to the P4 switch through the underlying DPDK technology, and at the same time extract the time field information of the received response packet, calculate the synchronization error, and use it to correct the server network card clock synchronization. (2) The packet sends the file host.1ua
[0102] The time synchronization process between the server network card and the P4 switch is as follows: design and initialize the synchronization request packet on the server network card side, then send the request packet and record the timestamp value TReqTx at this time; then the synchronization request packet arrives at the P4 switch and records the current timestamp TReqTx; the synchronized P4 switch will calculate the current TNowsw according to the formula, and write the timestamp variable value into the packet on the data plane of the P4 switch: then the response packet is sent out from the outbound port pipeline of the P4 switch, and the timestamp TReqTx at this time is recorded. At the same time, in order to reduce the impact of the response packet waiting delay in the outbound queue, an additional Follow_up packet will be sent, and the timestamp of the actual sending of the TReqTx response packet saved in this packet; when the response packet arrives at the network card, the timestamp TReqRx at this time is recorded, so that the network card device can calculate the synchronization error between the same switch and use it to correct the clock of the server network card. In addition to the timestamps mentioned above, the switch also calculates the link packet rate R.
[0103] In the host.1ua file, the API of the underlying device module is called to configure the number of send queues and receive queues in the network card. At the same time, the EnableTimestamps() method is called for both queues to enable timestamps and read physical timestamps from the network card. Initialization and data packet sending are implemented in the InitiateServer() function, and the delay between the network card and the switch is calculated in the StartTimesync() function, which is used to correct the timestamp of the network card and synchronize the network card and the switch.
[0104] 3. Use machine learning methods to predict synchronization delay and link delay and optimize the entire synchronization system
[0105] 1. Time Series Model
[0106] Time series forecasting analysis is to use the time characteristics of an event in the past to predict the characteristics of the event in the future. This is a relatively complex forecasting modeling problem. Unlike the prediction of regression analysis models, time series models rely on the order of events. The same value will produce different results when the order is changed. We use the characteristics of time series models to optimize synchronization error data and select the most appropriate precise time synchronization mode in different scenarios.
[0107] (II) RNN and LSTM models
[0108] The most commonly used and powerful tool for time series models is the recurrent neural network (RNN). Compared to the independent results of ordinary neural networks, each hidden layer calculation result of RNN is related to the current input and the previous hidden layer result. In this way, the calculation result of RNN has the characteristic of remembering the previous results.
[0109] (1) RNN model
[0110] The typical RNN network structure is as follows: Figure 5 As shown in the figure, the right side is the structure generated for easy understanding and memory during calculation. Simply put, x is the input layer, o is the output layer, s is the hidden layer, and t refers to the number of calculations; V, W, U are weights, and the hidden layer state when calculating the tth time is St = f (U*Xt+W*St-1), which realizes the purpose of linking the current input result with the previous calculation.
[0111] (a) Using RNN to predict and compensate for single link delay:
[0112] At the beginning of synchronization, the synchronization error of the single link at this time is calculated and used as the input layer to predict the synchronization error of the current output single link. The current value is used as the input layer for iteration to calculate the synchronization error that adapts to the entire network structure. This synchronization error is used to synchronize the entire network again, thereby achieving improved precision synchronization effects in different scenarios.
[0113] (b) Limitations of RNN:
[0114] If the RNN model needs to achieve long-term memory, it is necessary to link the calculation of the current implicit state with the previous n calculations, that is, St = f(U*Xt+W1*St-1+W2*St-2+...+Wn*St-n). In that case, the amount of calculation will grow exponentially, resulting in a significant increase in the model training time. Therefore, the RNN model is generally used directly for long-term memory calculations.
[0115] (2) LSTM model
[0116] The LSTM (Long Short-Term Memory) model is a variant of RNN. The feature of LSTM is that valve nodes are added to each layer in addition to the RNN structure. There are three types of valves: forget valve, input valve and output valve. These valves can be opened or closed to determine whether the memory state of the model network (the previous state of the network) at the output of this layer reaches the threshold and is added to the calculation of the current layer.
[0117] The valve node uses the sigmoid function to calculate the network's memory state as input; if the output reaches the threshold, the valve output is multiplied by the calculation result of the current layer as the input of the next layer; if the threshold is not reached, the output result is forgotten. The weights of each layer including the valve node will be updated during each model back-propagation training process, such as Figure 6 shown.
[0118] (a) Memory function of LSTM model:
[0119] The memory function of the LSTM model is implemented by these valve nodes. When the valve is open, the training results of the previous model will be associated with the current model calculation, and when the valve is closed, the previous calculation results will no longer affect the current calculation. Therefore, by adjusting the opening and closing of the valve, we can achieve the influence of the early sequence on the final result.
[0120] (b) Use the LSTM model to optimize the synchronization time error:
[0121] The time delay of each part under different network conditions (with load and without load) is measured respectively, and this error is used as the input of the previous layer to calculate the time delay of the next layer. After multiple iterations, the time synchronization error under the current network condition is calculated, and this error is used to calibrate the time synchronization of the current network, so as to improve the precise synchronization effect in different scenarios. However, if the network state or network structure changes at this time, the output result of the previous layer is forgotten by utilizing the characteristics of the forget valve, that is, the output result of the previous layer has no effect on the subsequent results, thereby achieving the effect of updating the synchronization time error during the training process.
[0122] At this point, the basic design description of the present invention is completed.
Claims
1. A method for accurate time synchronization based on a programmable data plane, characterized in that: This includes processing packets in the programmable data plane; using the DPDK+Lua scripting language framework to accelerate the sending and receiving of packets by the network card; using machine learning methods to predict synchronization delays and link delays to optimize the entire synchronization system; When the slave network switch generates a request data packet, the request response timeline starts and ends when the master switch receives the corresponding response data packet. Two master-slave P4 switches are designed. The slave switch sends a synchronization request data packet to the master switch. After receiving the data packet, the master switch writes the timestamp information of its own data plane into the synchronization request data packet, and then generates a response data packet and sends it to the slave switch. After receiving the response data packet, the slave switch extracts the timestamp information carried in the data packet, which contains various delay components involved in the request-response data packet model. The processing of data in the programmable data plane comprises the following steps: Step 201, deploy two switches, SW1 is a slave switch for clock calibration, and SW2 is a master switch for determining reference time; Step 202, the master switch SW2 obtains the timestamp value TExt from the external clock source as the reference clock TRefsw of the current data plane slave switch SW1, and stores the timestamp TRespIg of the ingress port pipeline obtained inside the data plane SW2 into the register, and records the current timestamp as Toffset and Toffset=TRespIg; Step 203, the x-bit counter remains not cleared within a time period of 2^x-1 seconds. In order to ensure a longer time to avoid the counter being cleared, a register needs to be added. When the counter is about to be cleared, the register will record the time before the clearing, and this time information is recorded as Tera; Step 204, when the main switch SW2 receives a synchronization request data packet, it reads the clock source timestamp value TRefsw obtained from the outside, the timestamp Tera of the register corresponding to the time period, and the time offset variable Toffset in the ingress port pipeline; Step 205, while the main switch SW2 is in the egress pipeline, it will read the current egress timestamp TRespEg; Step 206, using the following formula to calculate the current timestamp TNowsw after adding the reference time TRefsw, at this time TNowsw is calculated from the timestamp Toffset of the ingress pipeline, and the current timestamp value is: Step 207: Since the variable TNowsw is inaccurately calculated using the time value of the egress pipeline TRespEg in order to avoid delay in the egress queue, whenever the switch receives a request packet from another switch, step 206 will be repeated to calculate the variable value of TNowsw using TRespEg; Step 208, the slave switch SW1 sends a request data packet at time TReqTx; the request data packet is received by the master switch SW2 at time TReqRx; the master switch SW2 reads the clock source timestamp value TRefsw2 obtained from the outside; Step 209, in the data plane, the master switch SW2 embeds TRefsw2, TReqRx and TRespEg time variables into the request data packet by executing the action matching table and matching the flow table. The request data packet is then sent out by the master switch as a response data packet and forwarded back to the slave switch SW1; Step 210, the slave switch SW1 receives the response data packet at the time TRespRx, repeats step 206, and calculates the current time TNowsw2 of the master switch SW2; Step 211, the slave switch SW1 starts processing the response data packet at time TRespIg, and at this time, TRespIg is written into the register through the data plane register, which is recorded as Toffset; Step 212, calculate the reference clock of the current slave switch. The slave switch SW1 also needs to receive the accurate time when the data packet is sent by the master switch. Therefore, the master switch SW2 will subsequently send a Follow_up packet, which contains the timestamp of the master switch accurately sending the response data packet, recorded as TRespTx. Step 213, after obtaining the TRespTx variable value from the switch SW2, the total delay is calculated as RespD using the formula: Step 214, the current time TNowsw2 of the master switch SW2 and the total delay RespD can be used to accurately calculate the synchronization time of the slave switch SW1 through the following formula: Step 215, repeating steps 201 to 214; The DPDK+Lua scripting language framework is used to accelerate the sending and receiving of data packets by the network card. DPDK is used to accelerate the sending and receiving of data packets by the network card on the hardware, and the Lua scripting language is used to implement the initialization and sending of synchronous request packets on the software. Using the DPDK+Lua scripting language framework to accelerate the sending and receiving of data packets by the network card includes the following steps: Step 301, install DPDK on the host, configure Hugepages, and load the kernel module; Step 302, using Lua scripting language to define a synchronization request packet and a packet sending rule; Step 303, in the initialization phase, when there is no cross traffic in the link, the host will first send a DPTP detection packet to the programmable switch, and the switch will reply to the host with various timestamps TNowsw, TRespRx, TReqTx, TReqRx, TRespTx; TNowsw is the current slave switch time, TRespRx is the time when the response packet arrives at the network card, TReqTx is the time when the host sends the detection packet, TReqRx is the time when the slave switch receives the detection packet, and TRespTx is the time when the egress port sends the response packet; Step 304, in the initialization phase, the link rate R=0% at this time, and the link delay NicWireDelay at this time is calculated according to the following formula: NicWireDelay=(T RespRx -T ReqTx )-(T RespTx -T ReqRx ) Step 305, in the synchronization phase, the host reference time RespD is calculated according to the following formula, TRespTx is the time when the switch sends a response packet, and TRespEg is the time when the response data packet comes out of the switch output port pipeline: RespD=OWD+(T RespTx -T RespEg ) Step 306, determine the uncalculated delay OWD. If the link rate R is approximately equal to 0% during the synchronization phase, then OWD = NicWireDelay / 2. Once R is not 0%, the NicWireDelay will increase linearly. In order to ensure the accuracy of synchronization and the clock on the host is synchronized with the switch, the NicWireDelay calculated in the idle link during the initialization phase of step 303, which is recorded as AvgNicWireDelay, is used to calculate OWD. OWD is calculated according to the following formula: Step 307, using the formulas in steps 305 and 306, calculate the reference time RespD of the host, so as to achieve synchronization between the host and the programmable switch; Step 308, repeat steps 301 to 307.
2. The method according to claim 1, characterized in that: The machine learning method is used to predict and compensate for synchronization delay and link delay, optimize the entire synchronization system, and count the total delay and one-way link delay between switches under different network conditions with and without load. The recurrent neural network LSTM is used to predict and compensate for the total delay and one-way link delay between switches. At the beginning of synchronization, the delay at this time is counted and used as the input layer to predict the current output delay, and the current value is used as the input layer for iteration; The machine learning method is used to predict and compensate for synchronization delay and link delay, and the optimization of the entire synchronization network system includes the following steps: Step 401, according to step 213, calculate the total delay RespD between the master and slave switches; Step 402, according to step 306, calculating the uncalculated delay OWD between the host and the switch; Step 403, counting the total delay and one-way link delay between switches under different network states; Step 404, after the entire network is synchronized, due to the existence of cross traffic, various congestion, and packet loss in the network, RespD and OWD will have different changes. RespD and OWD are taken as input x and put into the recurrent neural network LSTM to predict new RespD and OWD as output h; Step 405, predict a new RespD, and repeat step 214 to calculate the synchronization time of the slave switch SW1; Step 406, using the predicted new OWD, repeat step 305 to calculate the reference time of the host; Step 407, repeating steps 401 to 405, and continuously updating the synchronization time of the slave switch and the host.
