A Virtual Cluster Time Synchronization Measurement Method Based on PCIe Passthrough

Abstract

The present invention discloses a virtual cluster time synchronization method based on PCIe passthrough, belonging to the technical fields of virtualization and network time synchronization. First, based on the PCIe passthrough technology, the virtual cluster nodes are bound to the underlying clock card to obtain the underlying physical clock time; then, a soft interrupt signal is generated through a timer to enable the virtual nodes to read the clock information of the underlying physical clock card at the same agreed time, while measuring the timer delay and correcting the time read from the clock card in real time; finally, the clock error between the virtual cluster nodes can be obtained by calculating the physical time values of the two clock cards. The present invention utilizes the high-precision hardware of the clock card and, with the help of the PCIe passthrough technology of the virtual machine, can accurately achieve the measurement of system time synchronization between virtual clusters.

Description

Technical Field

[0001] The present invention belongs to the technical fields of virtualization and network time synchronization, and particularly relates to a virtual cluster time synchronization measurement method based on PCIe passthrough, which is applicable to the real-time monitoring and measurement of the time synchronization accuracy between virtual machines. Background Art

[0002] With the continuous development of cloud computing and virtualization technologies, virtualization platforms have been applied to various industries more and more, and virtual cluster clock synchronization has become particularly important. For virtual clusters adopting a unified time synchronization system within the same network domain, how to achieve real-time online monitoring of the time synchronization status between virtual clusters through certain technologies has also become very important.

[0003] Due to the abstraction and isolation of the underlying hardware by virtualization technologies, virtual clusters cannot achieve high-precision time synchronization by sharing supporting hardware devices such as clock cards. Therefore, virtual clusters usually use software such as NTP to complete time unification. However, software time synchronization has problems such as clock drift, low clock synchronization accuracy, and poor synchronization stability. In the application scenarios of command and control, measurement and control, the stable time synchronization between virtual system nodes is directly related to the safe and stable operation of the command and control, measurement and control systems. Therefore, it is necessary to monitor virtual system nodes to reduce the disorder of event sequence records caused by time error. There are the following problems in how to measure the time synchronization of virtual cluster nodes:

[0004] 1. Since network synchronization protocols are all based on the assumption of symmetric network link delay, and in reality, the impact of asymmetric network link delay on calculating the master-slave clock offset by the protocol cannot be eliminated, the calculated value of the protocol software cannot be used as an effective basis for evaluating the clock error between virtual clusters.

[0005] 2. The network synchronization protocol software synchronizes the operating system kernel clock, and the operating system kernel clock cannot accurately output clock signals outward like a hardware clock card, resulting in the inability to accurately measure the virtual cluster time synchronization accuracy using an oscilloscope.

[0006] Based on the above problems, it is crucial for application systems with high-precision and stable time synchronization requirements such as command and control and measurement and control to provide a time synchronization accuracy measurement method with controllable measurement errors for virtual clusters. Summary of the Invention

[0007] In view of the existing technical problems, the present invention provides a virtual cluster time synchronization measurement method based on PCIe (peripheral component interconnect express, a high-speed serial computer expansion bus standard) passthrough. With the help of PCIe passthrough technology, the virtual cluster is bound to the underlying clock card to obtain a third-party standard time, indirectly and accurately realizing the time synchronization measurement between virtual clusters.

[0008] To achieve the above object, the present invention is realized through the following technical solutions:

[0009] A virtual cluster time synchronization measurement method based on PCIe passthrough includes the following steps:

[0010] Step 1: Bind the underlying physical clock card

[0011] Bind the underlying physical clock card to two virtual nodes to be tested through PCIe passthrough technology. The clock card is linked to the time server to complete the hardware time synchronization, so that the virtual nodes can obtain a unified third clock signal source;

[0012] Step 2: Run a measurement program on virtual node 1 and virtual node 2 to realize accessing and reading the hardware clock time of the underlying clock card with the agreed same system kernel time.

[0013] First, determine the time T when the interrupt signal is generated and the time interval t by setting a timer. When the measurement program receives the nth interrupt signal, read the current system kernel time t n , and through the clock card driver interface, the nth readings of the hardware clock time observations of virtual node 1 and virtual node 2 on the underlying clock card are respectively recorded as

[0014] Step 3: Measure the time synchronization error between virtual nodes

[0015] Based on the system kernel time t recorded in step 2 n , calculate the timer interrupt offset error in real time: t n -T - nt;

[0016] Read the hardware clock time in real time, and subtract the timer interrupt offset error t n -T - nt from the hardware clock time to complete partial error correction of the hardware clock time;

[0017] Finally, complete the time synchronization error measurement between virtual nodes by taking the difference between the corrected clock card physical times read with the same system time on virtual node 1 and virtual node 2:

[0018] That is

[0019] Further, two KVM virtual machines are selected, and the PTP clock card PCIe passthrough binding is completed by setting kernel configuration items.

[0020] Further, the measurement method of the present invention can be extended to multiple virtual nodes.

[0021] Further, the clock card used in the measurement method of the present invention is not limited, as long as it meets the task requirements.

[0022] This method uses the PCIe passthrough technology to provide a unified third-party time source for the virtual cluster, and indirectly measures the kernel clock difference between virtual clusters by reading the time of the underlying hardware clock card, making it possible to accurately monitor the clock error on the virtual cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 For virtual nodes to access the underlying hardware clock card to obtain the hardware standard time;

[0024] Figure 2 For the measurement program flow chart;

[0025] Figure 3 For the main error schematic diagram of the measurement method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] To make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the present invention will be further explained and described below with reference to the accompanying drawings.

[0027] The implementation process of the present invention is as Figure 2 shown. A virtual cluster time synchronization measurement method based on PCIe passthrough includes the following steps:

[0028] Step 1, as Figure 1As shown, the virtual node under test is bound to the underlying physical clock card through the PCIe passthrough technology. The clock card is linked to the time server to complete the hardware time synchronization, so that the virtual node can obtain a unified third clock signal source. The virtual node installs the clock card driver to ensure that the physical clock time information of the underlying clock card can be accessed through the driver interface within the virtual node. The virtual node completes the system kernel clock synchronization through the time synchronization software. In this specific example, two KVM (an open-source virtualization technology built into the Linux kernel) virtual machines are selected, and the PCIe passthrough binding of the PTP clock card is completed by setting relevant kernel configuration items. It should be noted that only the measurement process under two virtual nodes is introduced here, and the proposed measurement method can be further extended to multiple virtual nodes. The clock cards used are not limited, but it should be noted that the general accuracy of software time synchronization between virtual nodes is in the millisecond or even microsecond level. To obtain a unified third-party time scale between virtual nodes and ensure the accuracy of the measurement method, the hardware clock synchronization error between clock cards should be as small as possible. In this specific example, a PTP clock card is selected, and its synchronization error can reach the nanosecond level under normal network conditions, while the synchronization accuracy of the IRIG-B code clock card is generally in the microsecond level.

[0029] Step 2: Run the measurement program on virtual node 1 and virtual node 2. First, determine the interrupt signal generation time T and the generation interval t by setting the timer. When the measurement program receives the nth interrupt signal, read the current system kernel time t n , and through the clock card driver interface, the nth readings of the hardware clock times of the underlying clock cards on virtual node 1 and virtual node 2 are respectively recorded as

[0030] Step 3: According to the system kernel time t n recorded in step 2, the timer interrupt offset error can be calculated in real time, that is, t n - T - nt. Subtract this error from the read hardware clock time in real time to complete the error correction of this part. Finally, complete the measurement of the time synchronization error between virtual nodes by taking the difference between the corrected clock card physical times read at the same system time on virtual node 1 and virtual node 2. That is

[0031] Figure 3 shows the main sources of measurement error of the present invention, which mainly consists of three parts, namely:

[0032] δ total = δ ptp_card + δ pci_cross + δ timer (1)

[0033] As shown in the above formula, δ ptp_cardRepresents the synchronization error between the hardware PTP clock cards used on different hosts. From the measurement results of the hardware clock card synchronization error, it can be seen that this part of the error is in the nanosecond order of magnitude and can be directly ignored for the overall measurement error.

[0034] δ timer Is the timer error introduced during the implementation of the measurement program. In this specific example, there is a time delay error in the system timer generating the soft interrupt signal. This error can be corrected through real-time measurement.

[0035] Such as Figure 3 As shown, δ pci_cross Is the time overhead required for the virtual node to access the underlying clock card hardware after the virtual cluster uses the PCIe passthrough technology. This part of the time overhead is the main error source of this measurement method, and this part of the error can be directly measured and estimated.

[0036] In the example of the present invention, the measurement test results of the virtual machine accessing the underlying hardware clock card latency distribution are of a Gaussian-like distribution. By calculating its probability function, the theoretical accuracy of the measurement method is obtained.

[0037] That is, Are the random variables of the virtual node 1 and the virtual node 2 accessing the underlying clock card latency respectively. Suppose there is Then there is

[0038]

[0039] And also,

[0040]

[0041] It can be seen from formula (3) that its final measurement error is Combining the assumption and the derived formula (2), it can be obtained that the error of b% of the measurement data of this measurement method does not exceed a us.

[0042] In this specific example, according to the 3σ criterion, 99.74% of the measurement data of this measurement method has an error not exceeding And the virtual machine accessing the underlying hardware clock card latency is several microseconds. This specific example ensures that the time synchronization measurement accuracy is within 10 us and can be used for the measurement of the synchronization error of about 100 us between virtual clusters.

[0043] The above content describes the basic principle and main idea of the present invention. Those skilled in the art should understand that the present invention is not limited by the above specific examples.

Claims

1. A virtual cluster time synchronization measurement method based on PCIe passthrough, characterized in that, It includes the following steps: Step 1: Bind the underlying physical clock card Bind the underlying physical clock card to the two virtual nodes under test through PCIe passthrough technology. The clock card is linked to the time server to complete hardware time synchronization, so that the virtual nodes can obtain a unified third clock signal source; Step 2: Run the measurement program on virtual node 1 and virtual node 2 to achieve accessing and reading the hardware clock time of the underlying clock card with the agreed same system kernel time First, determine the time T when the interrupt signal is generated and the time interval t by setting a timer. When the measurement program receives the nth interrupt signal, read the system kernel time t at the current moment n , and through the clock card driver interface, the nth readings of the hardware clock time observations of the underlying clock card by virtual node 1 and virtual node 2 are respectively recorded as Step 3: Measure the time synchronization error between virtual nodes According to the system kernel time t recorded in step 2 n , calculate the timer interrupt offset error in real time: t n - t - nt; Read the hardware clock time in real time, and subtract the timer interrupt offset error t n -T-nt from the hardware clock time to complete partial error correction of the hardware clock time; Finally, complete the measurement of the time synchronization error between virtual nodes by taking the difference between the corrected clock card physical times read with the same system time on virtual node 1 and virtual node 2: That is 2. The virtual cluster time synchronization measurement method based on PCIe passthrough according to claim 1, characterized in that In step 1, two KVM virtual machines are selected, and the PCIe passthrough binding of the PTP clock card is completed by setting the kernel configuration item.

3. A virtual cluster time synchronization measurement method based on PCIe passthrough according to claim 1, wherein The measurement method can be extended to multiple virtual nodes.

4. A virtual cluster time synchronization measurement method based on PCIe passthrough according to any one of claims 1-3, characterized in that, The clock card used in the measurement method is not limited, as long as it meets the task requirements.

Images