A low-latency redundant communication protocol based on an improved PRP protocol

Abstract

This invention proposes a low-latency redundant communication protocol based on an improved PRP protocol, compatible with the IEC 62439-3-2016 PRP protocol standard, operating at the link layer. Dual-redundant ports are connected to upper-layer protocols via a link redundancy entity, transparent to the upper-layer protocols. Network end nodes have dual- or triple-redundant network ports, each connected to an independent network with the same topology, operating in parallel. All network ports of end nodes supporting the PRP protocol are configured with the same IP address and MAC address. The improvements of this invention are: changing the redundancy discarding algorithm to a method where the first arriving communication frame is directly uploaded to the application layer; using {source IP address, source MAC address, frame sequence number} as the criteria for judging redundant frames; employing a pre-configured configuration file method, pre-writing the IP addresses of single-port nodes in the network into the configuration file; and adding statistics on the number of network port link connection / disconnection occurrences, over-receiver window statistics, non-contiguous frame sequence number statistics, and redundant frame asynchrony occurrence statistics.

Description

Technical Field

[0001] This invention belongs to the field of Ethernet communication technology, specifically relating to a low-latency redundant communication protocol based on an improved PRP protocol. Background Technology

[0002] Currently, ordinary broadband Ethernet switched LANs are generally built on a complex star topology based on switches. Improving communication reliability on ordinary Ethernet typically employs information redundancy methods, such as retransmission, redundancy switching and retransmission, or parallel redundancy transmission. Retransmission generally determines whether to resend based on command responses on the same physical channel. Redundancy switching and retransmission requires the communication network to have redundant physical channels, using link connectivity or timeouts to determine whether to use a backup physical channel to resend messages. Both of these methods suffer from problems such as message loss in the event of a primary failure and uncertain message transmission delays. For example, the dual-network master-slave redundancy switching mechanism commonly used in real-time measurement and control suffers from low transmission reliability, weak fault monitoring capabilities, and limited network scale. Parallel redundancy transmission has mature communication protocols, such as the PRP protocol described in the international standard IEC62439-3 (hereinafter referred to as PRP protocol). However, this protocol focuses on message transmission reliability, increasing message buffering delay compared to ordinary Ethernet transmission, reducing the responsiveness of embedded real-time systems. Furthermore, the original PRP protocol uses a MAC address-based redundancy management method, limiting its application to Layer 2 networks and failing to meet the Layer 3 reliable transmission requirements of large-scale network systems. Summary of the Invention

[0003] To address the aforementioned problems, this invention proposes a low-latency redundant communication protocol based on an improved PRP protocol, compatible with the IEC 62439-3-2016 PRP protocol standard, operating at the link layer, such as... Figure 1 As shown, the dual redundant ports are connected to the upper-layer protocol through the link redundancy entity (LRE), which is transparent to the upper-layer protocol.

[0004] This invention requires network end nodes to have dual or triple redundant network ports, each connected to an independent network with the same topology, and each network operating in parallel, such as... Figure 2 As shown.

[0005] End nodes that support the PRP protocol can have all their network ports configured with the same IP address and MAC address, which facilitates the redundancy management of multi-channel data frames.

[0006] The LRE appends a redundancy check trailer (RCT) to the end of the Ethernet frame, including the frame sequence number, network port number, LSDU length, protocol suffix, etc. This format definition is identical to the requirements of standard IEC 62439-3-2016. Figure 3 As shown:

[0007] 1) Frame sequence number: SN, 16 bits. LRE maintains a sequence number for each MAC destination address, incrementing by 1 with each transmission;

[0008] 2) LAN ID: 4 digits. For redundant LAN ports, LAN A is 1010, LAN B is 1011, and LAN C is 1100.

[0009] 3) LSDU length: 12 bits, equal to the length of the user data area plus 6;

[0010] 4) Protocol suffix: 16 bits, set to a constant value of 0x88FB.

[0011] The LRE of this invention includes three functions: redundant transmission, redundant reception, and monitoring and management.

[0012] Improved Redundancy Drop-off Algorithm

[0013] This invention is consistent with the PRP protocol, and still adopts the first-come-first-served principle for redundant data frames, discarding the redundant data frames that arrive later; however, in order to reduce the latency of message transmission and adapt to the application of real-time measurement and control systems, this invention improves the redundancy discarding algorithm: after the LRE receives a new data frame on a certain network interface, it checks whether it is in the receiving window. If it is in the window, it does not buffer it, waits for data frames from other network interfaces, and immediately removes the RCT from the data frame before uploading it.

[0014] When receiving data frames, the triplet consisting of the source MAC address, IP address, and destination MAC address is used as the condition for frame classification. The frame sequence number of the last frame of a certain triplet currently received by a certain network interface X is currentSN[x], and the frame sequence number of the last uploaded frame of a certain triplet by all redundant network interfaces is deliverySN[y]. Thus, a hash mapping between the triplet input and the currentSN[x] of each network interface and the deliverySN[y] of the node can be established. Considering that the PRP protocol is generally used for closed local area networks, only the last byte of the source MAC address, IP address, and destination MAC address can be taken as the hash calculation input, thereby reducing buffering and calculation time.

[0015] The redundancy discarding algorithm proposed in this invention consists of two steps: receiving window filtering and redundancy filtering.

[0016] 1) Receive Window Filtering: The LRE receive inspection module sets a receive window [FirstSeq, LastSeq] for frames with different triples received by network interface X. FirstSeq is currentSN[x]+1, and LastSeq is currentSN[x]+n. The constant n can be set according to the system communication cycle and the maximum number of frame drops that can be tolerated. The values ​​are consistent for all network interfaces. Frames whose sequence number is outside the receive window are discarded; those inside the receive window have currentSN[x] set to the sequence number of that frame, and further redundancy filtering is performed.

[0017] 2) Redundancy Filtering: The LRE redundancy filtering module uses triplet data and a hash lookup algorithm to obtain the last frame sequence number (deliverySN) of the uploaded data. If the frame sequence number of the data frame is greater than the deliverySN, it is directly uploaded to the protocol stack and the deliverySN is set to that frame sequence number; otherwise, it is discarded.

[0018] Redundant transmission function

[0019] The message sending process at the sending end is as follows: Figure 4 As shown.

[0020] When the sending end detects that a network interface is linked down, it disables the sending function of that network interface, and sends PRP data packets directly on the other network interface in the format with RCT.

[0021] The network driver of the node replicates the sent messages to achieve redundant transmission across multiple network interfaces. This invention proposes a method based on semaphores or spinlocks to minimize the time interval between data frame transmissions across multiple channels, thereby reducing the buffer consumption and transmission delay of the receiving node.

[0022] The first data packet sent after the sending end restarts has a sequence number of 0, and subsequent sequence numbers are flipped and start from 1, so that the receiving end can determine whether the sending end has restarted or replaced a new module, and avoid alias frames being dropped.

[0023] Redundant receiving function

[0024] Assuming network interface A receives several message frames first, it first checks whether the sequence number of each frame is within the receiving window. If it is, it directly reports to the application layer and decrements the difference between the new sequence number and the window start point by 1, adding it to the lost frame statistics. At the same time, it updates the window start point of network interface A to the new sequence number. If it is not, it discards the frame and increments the window overage statistics by 1. Simultaneously, a 1ms timer is started on network interface B. If a frame with the same sequence number is received from network interface B within 1ms, it is considered normal, and the window start point of network interface B is updated to the new sequence number. If no frame with the same sequence number is received from network interface B within 1ms, it is considered a synchronization timeout, and the synchronization error statistics are incremented by 1. At the same time, the window start point of network interface B is updated to the new sequence number. If network interface A receives a frame with a new sequence number within 1ms, it is considered that the previous sequence number frame of network interface B was lost. The window start point of network interface B is updated to the new sequence number, the synchronization error statistics are incremented by 1, and a timer for the frame with the new sequence number is started.

[0025] If network port B receives the data first, the process is similar.

[0026] Each message stream uses its own independent list, receiving window, and aging timer.

[0027] The message processing flow at the receiving end is as follows: Figure 5 .

[0028] The receive aging time is configurable based on network conditions. It applies to a specific receiving node. A node may receive messages from multiple sources with different addresses. If no message is received from a source at a certain address for more than 5 seconds, the latest received frame sequence number and receive window filter corresponding to that source address are cleared. Figure 6 That is, after 5 seconds, any message with a sequence number sent by this address will be received and uploaded, and the sequence number of the message will be set as the sequence number of the latest received frame, and a new receiving window will be set.

[0029] • Monitoring and management functions

[0030] The PRP protocol, as driver software, runs at the data link layer and needs to provide the application layer with information about the underlying network operation. Besides reporting whether read and write operations between the application software, the protocol stack, and the network driver are executing correctly, it should also possess comprehensive self-monitoring capabilities. The main monitoring content of PRP should include:

[0031] The current status of each physical network port is linked or down. The total number of times each network port has changed from on to off, and the number of times it has changed from on to off, and the number of times it has changed from off to on, all increase by 1.

[0032] Set the same receive window and super window statistics for each network port. The starting point of the receive window is the sequence number of the previous normally received frame, and the ending point is the sequence number of that frame plus 100. The initial value of the super window statistics is 0. If the sequence number of the received frame exceeds the receive window, the frame is discarded, and the super window statistics value is incremented by 1. The maximum count value is 65535, and the count is flipped when it is full.

[0033] Set up frame sequence number continuity check and lost frame statistics for each network interface. The initial value of the statistics is 0. When the PRP frame sequence number of a network interface is not continuously increasing by 1, the number of lost frames is accumulated (the number of lost frames is accumulated). The maximum count value is 65535, and the count is flipped when it is full.

[0034] For each node's redundant network interface, set a frame reception time check window and a reception time difference statistics. The window size is 1ms, and the initial value of the reception time difference statistics is 0. When the received frame exceeds the window, the reception time difference statistics value is incremented by 1, with a maximum count value of 65535. When the count is full, the count is flipped.

[0035] The receive window is set to 1ms, meaning that the time difference between receiving PRP frames from the same source through two independent networks should be less than 1ms, in order to check the synchronization status of the source device.

[0036] Of the above statistics, 1) one monitoring result corresponds to each network interface; 2) and 3) statistics are performed on the received message streams, i.e., message streams with the same source MAC address and different destination MAC addresses are grouped into one message stream for super-window statistics and lost frame accumulation; 4) one statistical result corresponds to redundant network interfaces. All self-monitoring results are provided by the underlying driver through the application layer interface.

[0037] The redundancy management and management information database of the PRP protocol are streamlined. The LRE does not send PRP protocol monitoring frames, nor does it receive or analyze PRP protocol monitoring frames sent by other nodes. Each node sends the monitoring results to the host computer through periodic network messages.

[0038] Each node's network interface A and network interface B independently perform super window statistics and lost frame statistics, and the statistics register and upload register are independent. When a network packet is transmitted to memory, it is processed by the interrupt notification software. The software first reads the system clock, then identifies which network interface the data belongs to, and then reads the messages in the data sequentially. The sequence number of the read message is subtracted from currentSN+1 and added to the lost frame statistics of that network interface. If it exceeds the receiving window, the message is lost and the super window statistics value is incremented by 1. If it is within the receiving window, the value is compared with deliverySN. If it is less than or equal to, it is discarded; if it is greater than, it is uploaded. The system clock value of the current data is compared with the system clock value of the most recent data from another network interface. If it is less than or equal to 1ms, no processing is done. If it is greater than 1ms, the message sequence numbers of the two data are compared. The number of messages with the same sequence number is added to the receiving time difference statistics.

[0039] If one network interface of a node fails and no messages are received, while the other network interface is normal, the statistical value of the faulty network interface remains unchanged during the fault process until a new message is received.

[0040] Communication methods between DANP nodes and SAN nodes

[0041] The PRP driver software has the function of reading the static configuration of SAN end nodes. The DANP end nodes in the system configure certain IP addresses in the network as SAN end nodes based on configuration files or configuration messages. When a DANP end node receives a message from a regular network node, it identifies whether it is a SAN end node by its IP address. If so, it directly submits the message to the protocol stack via LRE; otherwise, it processes it according to the aforementioned receiving process.

[0042] At the same time, PRP Ethernet driver software with the same function can be deployed on a single network port node to send and receive UDP frames in PRP frame format, thus eliminating the need to consider the problem of incorrect message handling on a single network port node.

[0043] The beneficial effects of this invention are as follows:

[0044] (1) This invention inherits the PRP frame format definition of the original international standard, but improves the redundancy discarding algorithm to reduce the transmission delay of communication frames and improve network real-time performance. The original redundancy discarding algorithm, in order to improve reliability, requires waiting for a period of time after receiving a communication frame from one channel to receive a communication frame from another channel, and then performs redundancy removal according to the PRP frame sequence number. While this method enhances the reliability of network transmission, the waiting time after reception also increases transmission delay. To meet the real-time requirements of real-time control systems such as aerospace, this invention changes the redundancy discarding algorithm to a method where the first arriving communication frame is directly uploaded to the application layer, eliminating the delay of communication frames at the link layer and improving the real-time performance of communication while maintaining compatibility with the original standard.

[0045] (2) The original international standard IEC62439-3 uses the tuple {source MAC address, frame sequence number} as the redundancy judgment condition for redundant communication frames received from two redundant channels. Because Layer 3 switching replaces the MAC address of the source node with the MAC address of the gateway, the original standard cannot be used in large-scale Layer 3 switching networks and is not suitable for the use requirements of large and complex multi-subsystems such as aerospace. This invention improves the redundancy management method by using {source IP address, source MAC address, frame sequence number} as the redundancy judgment condition, extending it to Layer 3 switching networks while maintaining compatibility with the original standard, and realizing its application in large and complex systems such as aerospace.

[0046] (3) The original international standard uses a method where all network nodes broadcast monitoring frames to distinguish between single-port (SAN) and dual-port (DANP) nodes in the network. This method has the ability to dynamically manage the network and is suitable for industrial open network systems. For closed systems such as aerospace systems where the number, status, and network connections of internal nodes are determined, this method not only increases the complexity of protocol implementation but also increases the delay in node startup and establishing system communication, affecting the system's rapid response capability. This invention addresses the characteristics of closed systems such as aerospace systems by using a pre-configured configuration file. The IP addresses of single-port nodes in the network are pre-written into the configuration file in the nodes. Each node treats nodes with IP addresses listed in the configuration file as ordinary non-redundant nodes and does not perform redundancy or related processing. This improvement simplifies the management of single and dual network redundancy, reduces network bandwidth consumption and system power-on readiness delay, and improves the rapid startup and rapid response capabilities of nodes and the system.

[0047] (4) The original international standard included the following anomaly monitoring content: frame check errors, channel number mismatch errors, and frame sequence number mismatch errors. Based on the aforementioned improvements, this invention enhances the network self-monitoring capability by adding statistics on network interface link connection / disconnection counts, over-receiver window statistics, discontinuous frame sequence number statistics, and redundant frame asynchrony counts. These statistics enable comprehensive monitoring of network link status, node transmission and reception status, and network transmission status, helping the system to promptly detect node and network anomalies.

[0048] (5) This invention proposes an implementation method based on network link layer driver software, which is particularly suitable for processor chips with dual network ports. It can achieve highly reliable parallel redundant communication without increasing hardware overhead. Compared with hardware implementation schemes based on FPGA or chips, it has the characteristics of low implementation cost and flexible implementation. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the PRP protocol;

[0050] Figure 2 This is a typical hardware configuration diagram of a network node with redundant network ports;

[0051] Figure 3 This is a diagram of the PRP protocol format;

[0052] Figure 4 This is a flowchart of the sending process;

[0053] Figure 5 This is the receiving flowchart;

[0054] Figure 6 This is a schematic diagram of the aging timer's operation. Detailed Implementation

[0055] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection claimed by the present invention.

[0056] The hardware uses the Xilinx Z7 series, model number XC7Z045-2FFG900I. The software is embedded in the FLASH chip and runs on the Z7's ARM core. There is no operating system environment; all functions, except for hardware interrupt handling, are completed in the main process. The network protocol stack uses LWIP, and the PRP protocol is extended on top of LWIP.

[0057] Network-related software can be divided into four parts:

[0058] Network card driver: Provides API functions to directly access MAC and PHY hardware registers;

[0059] Interface layer: connects the network protocol stack and the network card driver layer;

[0060] LWIP network protocol stack: provides network protocols such as ARP, ICMP, IP, UDP, TCP, DHCP, and IGMP, and supports RAWAPI and socketAPI;

[0061] Application layer: The LRM software uses RAWAPI to implement network communication and periodically sends out PRP-related statistical information via heartbeats.

[0062] The PRP protocol operates at the data link layer and has three main functions: redundant transmission, duplicated data frame reception, and redundancy monitoring and management. The module software implements these three functions by modifying the interface layer. The application layer periodically sends monitoring results externally.

[0063] The specific modifications are as follows:

[0064] (a) Network data structure

[0065] The following diagram lists the important data structures of the network protocol stack, interface layer, and network card driver. The software modified the structure layer data structure xemacpsif_s, which originally managed one network port, but now manages two network ports. In addition, the receiving and transmitting space has also been changed from one network port to two network ports.

[0066] (b) Network initialization

[0067] The interface layer code was modified so that, instead of initializing only one network, two networks are now initialized.

[0068] (c) Redundant transmission of PRP data

[0069] Application-layer UDP transmission uses only PBUF_RAM, not PBUF_REF. Modify the interface layer code `low_level_output` to allocate two PBUF_RAM spaces, `q` and `r`, with a size of `p->len+12`. Copy the content of `p` from the protocol layer to `q`, and append 12 bytes of RCT (interface A) to the end of `q`. Then copy the content of `q` to `r`, and change the interface A in `RCT` to interface B. Finally, send `q` and `r` out from interfaces A and B respectively.

[0070] (d) PRP data reception and discarding

[0071] Modify the interface layer data reception interrupt code to determine if the received data is a PRP message. If not, enqueue it as a normal message. If the PRP message is within the reception range and is the first packet received, enqueue it and upload it; otherwise, discard it.

[0072] (e) Redundant monitoring management

[0073] The system utilizes a timer interrupt service routine to maintain the data structures required by PRP, check metrics such as the 1ms receive window and 5s receive aging time, and report the monitoring status to the host computer via heartbeat messages.

[0074] The patent protection points of the present invention, such as the redundancy discarding algorithm, the three-layer redundancy communication method, and the network monitoring method, can be implemented in the form of software drivers or in the form of FPGA, ASIC chips, etc.

[0075] Furthermore, the patent protection points described in this invention can be used not only in dual-redundant ordinary Ethernet networks, but also in other bus networks with independent redundancy. For example, they can be used in Time-Triggered Ethernet (TTE) to enhance transmission reliability without reducing BE stream delay.

[0076] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A low-latency redundant communication architecture based on an improved PRP protocol, characterized in that, It is compatible with the IEC 62439-3-2016 PRP protocol standard and operates at the link layer; dual redundant ports are connected to upper-layer protocols through link redundancy entities, which are transparent to the upper-layer protocols; network end nodes have dual or triple redundant network ports, which are connected to independent networks with the same topology, and each network works in parallel; all network ports of end nodes that support the PRP protocol are configured with the same IP address and MAC address. The link redundancy entity appends a redundancy identifier to the end of the Ethernet data frame, including the frame sequence number, network port number, LSDU length, and protocol suffix. This format definition is the same as that required by standard IEC62439-3-2016. When a link redundancy entity receives a new data frame on a certain network interface, it checks whether the frame is within the receiving window. If it is, it does not buffer the frame but waits for data frames from other network interfaces. It then immediately removes the redundancy identifier from the data frame and uploads it. When receiving data frames, the triplet consisting of the source MAC address, IP address, and destination MAC address is used as the condition for frame classification. The frame sequence number of the last frame of a certain triplet currently received by a certain network interface X is recorded as currentSN[x], and the frame sequence number of the last uploaded frame of a certain triplet by all redundant network interfaces is recorded as deliverySN[y]. This allows the establishment of a hash mapping between the triplet input and the currentSN[x] of each network interface and the deliverySN[y] of the node. The link redundancy entity receiving and checking module sets a receiving window [FirstSeq, LastSeq] for frames with different triples received by network port X. FirstSeq is currentSN[x]+1, and LastSeq is currentSN[x]+n. The constant n can be set according to the system communication cycle and the maximum number of frame drops that can be tolerated. The values ​​are consistent for each network port. If the frame sequence number of the received frame is outside the receiving window, it is discarded; if it is inside the receiving window, currentSN[x] is set to the frame sequence number, and redundancy filtering is further performed. The link redundancy entity redundancy filtering module uses triplet data and a hash lookup algorithm to obtain the last frame sequence number (deliverySN) of the uploaded data. If the frame sequence number of the data frame is greater than the deliverySN, it is directly uploaded to the protocol stack and the deliverySN is set to that frame sequence number; otherwise, it is discarded.

2. The low-latency redundant communication architecture based on the improved PRP protocol according to claim 1, characterized in that, The link redundancy entity includes redundant sending, redundant receiving, and monitoring and management functions.

Images