ST2110 transport protocol processing method, device and system realized based on FPGA (Field Programmable Gate Array)

Abstract

The invention provides an ST2110 transmission protocol processing method, device and system based on an FPGA (Field Programmable Gate Array), and solves the technical contradictions of high delay of an existing CPU (Central Processing Unit) scheme and poor flexibility of an ASIC (Application Specific Integrated Circuit) scheme. The method comprises the following steps: constructing an information classification module to distinguish a video / audio stream from an SDP / PTP message; media parameters specified by RFC 4566 are extracted through an SDP analysis module; an RTP module is adopted to execute RFC 3550 protocol packaging; iEEE 1588v2 clock synchronization is realized on the basis of double clock sources; a token bucket traffic shaping module is utilized to meet the ST2110-21 standard and compensate for clock drift. The device is realized on a Xilinx XC7K325T FPGA platform, and comprises a reconfigurable assembly line dynamic configuration RTP packaging module, a PTP synchronizing module and a shaping module; the hardware state machine drives SDP parameter analysis; and the 10G Ethernet module is integrated with ST 2110-22 VLAN (Virtual Local Area Network) priority mapping. According to the design, the FPGA is adopted for acceleration, the problem that performance and cost performance cannot be well compatible at present is solved to a certain extent, and the strict requirement for 4K ultra-high-definition video transmission is met.

Description

Technical Field

[0001] The present invention belongs to the field of FPGA application and professional audio and video network transmission technology, and specifically relates to a processing method, device and system for the ST2110 transmission protocol based on FPGA. Background Art

[0002] The SMPTE ST 2110 family of standards defines specifications for the transmission of professional-quality uncompressed video, audio, and ancillary data over IP-based networks. It represents the core technical standard for the transition from SDI to IP-based production and distribution in broadcast and professional audio and video production. Efficient and stable implementation of the ST 2110 protocol stack is key to building modern IP-based audio and video systems, thereby bringing true flexibility to media workflows.

[0003] Typically, the ST2110 transport protocol is implemented using software on a general-purpose CPU on a PC. This involves using languages ​​such as C / C++, Java, and Python to call the network socket interface provided by the operating system. This is combined with open-source libraries to implement the ST2110 protocol stack processing logic on a general-purpose server or high-performance workstation. Some chip manufacturers also offer dedicated ASIC / SOC chips that inherit the ST2110 protocol processing hardware.

[0004] However, these mainstream implementations all have their limitations. Software solutions based on general-purpose CPUs, while flexible and convenient to develop, struggle to overcome performance bottlenecks, high latency, and jitter caused by operating system scheduling, and face significant challenges in meeting the stringent ST2110 requirements for ultra-low, deterministic latency and high throughput. Solutions based on dedicated ASIC / SOC chips, while offering excellent performance, power consumption, and latency determinism, lack flexibility and are costly. Summary of the Invention

[0005] Based on this, the embodiments of the present invention provide a processing method, device, and system for an FPGA-based ST2110 transmission protocol system to address the trade-offs faced by developers in the prior art between high performance, low latency, determinism, flexibility, and cost-effectiveness due to the drawbacks of general-purpose CPUs and dedicated ASICs / SOCs.

[0006] A first aspect of an embodiment of the present invention provides a design method for an ST2110 transmission protocol based on an FPGA, the method comprising:

[0007] Different modules are established based on the requirements of the ST2110 transport protocol and its sub-protocols, and each module jointly implements the ST2110 transport protocol; according to the protocol requirements, the modules are divided into information classification module, session description protocol SDP module, real-time transport protocol RTP module, PTP time synchronization module, and Ethernet module.

[0008] Build an information classification module to distinguish the input end code stream signal type, at least distinguishing the video information, audio information and specific format information SDP message.

[0009] Furthermore, a Session Description Protocol (SDP) module is constructed to monitor and parse the information carried in the SDP message described in IETF RFC 4566, including media stream attributes, transmission control information, and quality of service parameters, and pass the parsed structured parameters to the subsequent RTP and traffic shaping modules.

[0010] Furthermore, a real-time transport protocol (RTP) module is constructed to encapsulate the input stream information in accordance with the RFC 3550 protocol, providing services including payload type identification, sequence numbering, and timestamps for real-time data such as interactive audio and video.

[0011] Furthermore, the timestamp part constructs a PTP time synchronization module that meets the IEEE 1588v2 protocol. Through the combined action of an external precise clock source and an internal crystal oscillator, the internal clock error is corrected to obtain the correct timestamp and provide it to the RTP module.

[0012] Furthermore, a traffic shaping module is built to extract the input processed by the RTP module and output the video data to the video processing pipeline. This module is responsible for maintaining the traffic shaping and delivery time requirements specified by ST2110-21 and compensating for clock drift between the FPGA internal clock and the PTP synchronization clock.

[0013] Finally, after being processed by the traffic shaping module, it is encapsulated into a UDP message through an Ethernet module that is compatible with the 10Gbps Ethernet physical layer specification, and output as a network packet compliant with the ST2110 protocol through the physical layer Ethernet port.

[0014] A second aspect of the embodiments of the present invention provides an apparatus and system for implementing the ST2110 transmission protocol based on a Xilinx XC7K325T FPGA hardware platform, the apparatus and system comprising:

[0015] Construct an information classification module to analyze whether the input data has continuity. If it has continuity, it is judged as the input code stream signal. If there is an end mark, the information type is determined. If the information format meets the corresponding requirements of the SDP message, it is judged as an SDP message. If the information format meets the corresponding requirements of the PTP message, it is judged as a PTP message. Otherwise, it is directly discarded.

[0016] Furthermore, a reconfigurable hardware pipeline is constructed, wherein the reconfigurable hardware pipeline at least includes an RTP encapsulation module, a PTP clock synchronization module and a traffic shaping module that are dynamically configured through FPGA programmable logic.

[0017] The RTP encapsulation module uses a parallel pipeline architecture to perform 4K video row-level fragmentation operations; it integrates a serial number cycle counter and a timestamp register, and the serial number generation content is controlled by dynamic parsing parameters, with a processing delay of ≤0.5μs / packet;

[0018] The timestamp register is controlled by a PTP clock synchronization module. The PTP clock synchronization module uses a dual-clock source architecture, achieving IEEE 1588v2 synchronization based on the FPGA's own crystal oscillator and an external RTC high-precision clock module. The RTC high-precision clock module provides an absolute time reference for second-level counting, while the FPGA's own crystal oscillator achieves nanosecond-level precision timing. Through dual-clock source collaboration and dynamic drift compensation, the module achieves ±50ns synchronization accuracy.

[0019] Furthermore, the traffic shaping module dynamically controls the RTP packet header sequence number generation strategy according to the content obtained by dynamic analysis of the protocol parameters, and calculates the target bandwidth according to the video bit rate and frame rate parameters to configure the token injection rate of the traffic shaper.

[0020] Parse the SDP session description protocol in real time to drive hardware reconstruction.

[0021] The dynamic parsing parameters are scanned in real time by a dedicated state machine during the parsing process. The information specified in IETF RFC 4566 is correctly parsed to extract the video resolution, number of audio channels, and FEC enable flag to generate protocol configuration parameters. The extracted video resolution parameter directly controls the RTP fragment packet length, the FEC flag activates the corresponding error correction engine, and the number of audio channels parameter configures the multiplexing logic.

[0022] Furthermore, the traffic shaper uses a hardware token bucket algorithm to achieve constant bit rate output to the Ethernet port, the token generation rate matches the target bandwidth requirement, and the bucket depth is configured as 1.5 times the standard Ethernet maximum transmission unit (MTU) to absorb burst traffic.

[0023] Furthermore, the Ethernet module is compatible with the 10Gbps Ethernet physical layer specification and meets the error correction requirements of the ST 2110-22 standard. Its data link layer implements hardware-level CRC checking and low-latency frame encapsulation, and maps the quality of service parameters specified in the SDP to the PCP bit of the IEEE 802.1Q VLAN tag, ensuring high-priority transmission of critical audio and video streams.

[0024] Through the FPGA-based ST2110 transmission protocol processing method, device and system, CPU software is replaced by a hardware pipeline frame, scheduling jitter of the operating system is eliminated, and end-to-end transmission delay is greatly reduced; PTP synchronization using dual clock sources is used to lock the clock accuracy within ±50ns, meeting the requirements of SMPTE ST 2059; the RTP encapsulation engine adopts a parallel pipeline architecture, supports 4K video line-level slicing, and fully utilizes the characteristics of FPGA parallel computing to significantly shorten processing time; the traffic shaper is dynamically adjusted through the SDN interface to effectively handle network congestion. In actual use, it can simultaneously take into account high performance, low latency, determinism, flexibility and cost-effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Implementation flow chart of an FPGA-based ST2110 transmission protocol method, device, and system provided in an embodiment of the present invention;

[0026] Figure 2 A system implementation diagram of an FPGA-based ST2110 transmission protocol method, device, and system provided in an embodiment of the present invention; DETAILED DESCRIPTION

[0027] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0028] like Figure 1 As shown, Figure 1 A method for implementing the ST2110 transmission protocol based on the Xilinx XC7K325T FPGA hardware platform provided by an embodiment of the present invention is shown, specifically including steps S01 to S06.

[0029] Step S01, construct an information classification module, analyze whether the input data has continuity, if it has continuity, determine it as the input code stream signal, if there is an end mark, determine the information type, if the information format meets the corresponding requirements of the SDP message, determine it as an SDP message, if the information format meets the corresponding requirements of the PTP message, determine it as a PTP message, otherwise it will be discarded directly.

[0030] It should be noted that the input data is transmitted through the Ethernet port. Before being processed by the information classification module, the IP / UDP header needs to be stripped and a 32-bit CRC check is performed to ensure the authenticity and accuracy of the data.

[0031] Step S02: construct an SDP module to parse the SDP session description protocol in real time to drive hardware reconstruction.

[0032] The parsing process uses a dedicated state machine to scan SDP signaling in real time, correctly parse the information specified in IETF RFC 4566, extract the video resolution, number of audio channels, and FEC enable flag to generate protocol configuration parameters. The extracted video resolution parameter directly controls the RTP fragment packet length, the FEC flag activates the corresponding error correction engine, and the number of audio channels parameter configures the multiplexing logic.

[0033] Specifically, dynamic parameter parsing includes four steps: parsing the SDP message and completing the signal setting to freeze the RTP pipeline input; batch writing the configuration register group through the AXI-Stream bus; hardware synchronization circuits ensure that all registers are updated synchronously; releasing the pipeline freeze state and activating the new parameters for operation.

[0034] The SDP message parsing is implemented through a designed state machine. After entering the media description block, the parsing state machine monitors the attribute line starting identifier "a=" in real time using a hardware keyword comparator. Upon detecting a valid identifier, the state machine jumps to a dedicated processing state based on the attribute type: If "a=imageattr" is recognized, the state machine enters the video parameter extraction state, activating the sliding window comparator to capture the resolution value and calculate the fragment packet length; if a match is "a=rtpmap", the state machine enters the audio parameter processing state, generating the channel enable mask using a bitfield extraction circuit; and if "a=fec-enabled" is detected, the state machine jumps to the FEC control state, directly outputting an enable signal to the error correction engine.

[0035] Furthermore, the parameter writing phase employs a hardware-level synchronization strategy: upon the rising edge of the register update enable signal, the clock gating circuit opens the global clock to drive all registers for writing, ensuring that the 128-bit configuration parameters take effect synchronously within the same clock cycle. After the register update is complete, the freeze control unit releases the pipeline freeze state. Finally, the status monitoring circuit verifies the continuity of the RTP engine's first packet timestamp and resumes media streaming.

[0036] Step S03: Construct a PTP clock synchronization module. The PTP clock synchronization unit adopts a dual-clock source architecture, implements IEEE 1588v2 synchronization based on the FPGA's own crystal oscillator and an external RTC high-precision clock module, uses the RTC high-precision clock module to provide an absolute time reference for second-level counting, and cooperates with the FPGA's own crystal oscillator to achieve nanosecond-level precision timing. Through dual-clock source collaboration and dynamic drift compensation, the synchronization accuracy of ±50ns is achieved.

[0037] Specifically, PTP clock synchronization requires the cooperation of a master clock and a slave clock. In this example, the design carries both master and slave clock functions to ensure that all devices in the same clock domain use the same clock.

[0038] The master clock functions mainly include accurate counting, clock format conversion and PTP message sending.

[0039] Specifically, implementing the master clock function involves four steps: reading the external precise clock module RTC (model DS3231 in this example) through I2C to obtain the current Unix timestamp, and using the internal counter to obtain the nanosecond clock; combining the two clocks, using the Unix timestamp (1970-01-01 00:00:00 UTC) as the first year of the calendar, calculating the difference between the combined time and the first year, converting it into binary clock form, and writing it into the register; combining the PTP message header with the current clock to form a PTP packet and send it out; setting the cycle time to send a PTP message once at a fixed time.

[0040] It is worth noting that sending PTP messages is not completed in one go. The master clock usually needs to send Sync, Follow_up, and Delay_Resp messages separately to ensure the accuracy of the slave clock synchronization. The sending scenario will be mentioned later.

[0041] The slave clock function mainly includes parsing PTP messages, clock conversion mode and writing into the RTC module.

[0042] Specifically, the implementation of the slave clock function includes four operations: parsing the received PTP message type, writing the clocks t1, t2, and t4 respectively and writing them into the register; when parsing the Follow_up message, writing the current clock t3 into the register and sending a Delay_Req message to the master clock; after t1, t2, t3, and t4 are written into the register, it is considered that a complete PTP information transmission has been completed, and a clock synchronization mechanism is used to calculate the current time t, which is the time after synchronization; converting the time t after synchronization from binary to the current date, and using I 2C updates the Unix timestamp of the RTC module, and the nanosecond count dynamically adjusts the DPLL phase compensation offset, waiting for the arrival of the next Sync message.

[0043] The clock synchronization mechanism follows the master-slave synchronization protocol defined in the IEEE 1588v2 standard.

[0044] The interaction process includes: the master initiates a Sync message and records the sending timestamp t1; the slave receives the Sync message and records the reception time t2; the master sends a Follow_Up message carrying the exact t1 value; the slave sends a Delay_Req message and records the sending time t3; the master records the Delay_Req reception time t4 and returns it through a Delay_Resp message; finally, the slave calculates the transmission delay and clock offset based on the {t1, t2, t3, t4} timestamp set.

[0045] T delay =((t2-t1)+(t4-t3)) / 2

[0046] offset=((t2-t4)-(t1-t3)) / 2

[0047] From this we can get the current time t

[0048] t=t1+T delay +T cmp +T delta =((t1+t4)-(t2+t3)) / 2+T cmp +T delta

[0049] Where T_delay is the transmission delay; T_delta is the time from when the slave receives the sync packet to when the slave receives the delay_resp packet; and T_cmp is the compensation time used to compensate for the delay in network data packet processing.

[0050] It is worth noting that in this example, T_cmp is set to 0, using FPGA hardware timestamps to eliminate processing delays.

[0051] Step S04: construct an RTP encapsulation module. The RTP encapsulation module adopts a parallel pipeline architecture to perform 4K video row-level fragmentation operations. It integrates a serial number cycle counter and a timestamp register, and the serial number generation content is controlled by dynamic parsing parameters.

[0052] The RTP encapsulation module adopts a parallel pipeline architecture to perform 4K video row-level fragmentation operations; it integrates a serial number cycle counter and a timestamp register, and the serial number generation content is controlled by dynamic parsing parameters.

[0053] Specifically, in the transmit pipeline, the header fields are initially calculated and written to the RTP header register, which has the maximum length of the RTP header. The header is then passed down to the traffic shaping module by transmitting the first 64 bits and shifting the register by 64 bits after each successful transmission. Four 16-bit FIFOs are used to buffer video data arriving from the information classification module. Data is read from the 16-bit FIFO set in 16-bit, 32-bit, 48-bit, or 64-bit increments, based on control signals, in the order in which it was written to the FIFOs.

[0054] The calculated header fields shall meet the requirements of ST2110-20 and RFC 3550:

[0055] The timestamp content it carries should meet the following requirements:

[0056] The timestamp takes a random value at the beginning of the transmission. During the transmission process, the timestamp satisfies the following formula:

[0057] RTP Timestamp(n)=t×Sequence+RTP Timestamp(n-1)

[0058] Where t is the number of seconds corresponding to the sampling time point, provided by the system clock generated by the PTP synchronization module; Sequence is the sampling rate of the media.

[0059] In addition, the rest of the information sent in the header field is provided by the SDP session description protocol.

[0060] Step S05, construct a traffic shaping module, which uses a hardware token bucket algorithm to achieve constant bit rate output to the Ethernet port, the token generation rate matches the target bandwidth requirement, and the bucket depth is configured as 1.5 times the standard Ethernet maximum transmission unit (MTU) to absorb burst traffic.

[0061] Specifically, the traffic shaping module achieves a constant bitrate output through a hardware token bucket architecture. During the initialization phase, the target bandwidth register is configured, and the token generator periodically calculates the number of new tokens: number of new tokens = target bandwidth × time interval / 8. The token bucket memory is preset to a depth of 1.5 times the standard Ethernet maximum transmission unit, or 2277 bytes, to absorb burst traffic.

[0062] When a packet arrives at the shaping queue, the arbiter first checks whether the current token bucket balance is greater than or equal to the packet length. If sufficient tokens are available, the packet is immediately sent and the corresponding token count is deducted. If insufficient tokens are available, a delay mechanism is activated: the packet is temporarily stored in the buffer FIFO, while the token generator continuously injects new tokens until sufficient tokens are available for sending. This process strictly adheres to the first-come, first-served principle, but preemptive sending is enabled for high-priority audio and video streams.

[0063] A clock drift compensation unit continuously monitors the deviation between the PTP synchronized clock and the local clock, dynamically fine-tuning the token generation time base. A bucket depth monitoring circuit triggers an early warning interrupt when the margin falls below 10%, preventing persistent congestion and packet loss. The shaped data stream is then sent through the 10G Ethernet MAC layer.

[0064] Step S06: construct an Ethernet module to output network packets. The Ethernet module is compatible with the 10Gbps Ethernet physical layer specification and meets the error correction requirements of the ST 2110-22 standard.

[0065] When the ST 2110-22 enable flag is activated, the Ethernet module adds only a 2-byte CRC checksum in pass-through mode. The encapsulation phase performs three layers of processing: the payload layer inserts the RTP / UDP / IP protocol header; the link layer adds the Ethernet frame header and IEEE 802.1Q VLAN tag; and the physical layer converts the data format and injects the clock alignment code group.

[0066] The above disclosure is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A design method for the ST2110 transmission protocol based on FPGA, characterized in that: Different modules are established based on the requirements of the ST2110 transmission protocol and its sub-protocols, and each module jointly implements the ST2110 transmission protocol. The method includes: According to the protocol requirements, the modules are divided into information classification module, session description protocol SDP module, real-time transport protocol RTP module, PTP time synchronization module, and Ethernet module. Build an information classification module to distinguish the input end code stream signal type, at least distinguishing the video information, audio information and specific format information SDP message. Furthermore, a Session Description Protocol (SDP) module is constructed to monitor and parse the information carried in the SDP message described in IETF RFC 4566, including media stream attributes, transmission control information, and quality of service parameters, and pass the parsed structured parameters to the subsequent RTP and traffic shaping modules. Furthermore, a real-time transport protocol (RTP) module is constructed to encapsulate the input stream information in accordance with the RFC 3550 protocol, providing services including payload type identification, sequence numbering, and timestamps for real-time data such as interactive audio and video. Furthermore, the timestamp part constructs a PTP time synchronization module that meets the IEEE 1588v2 protocol. Through the combined action of an external precise clock source and an internal crystal oscillator, the internal clock error is corrected to obtain the correct timestamp and provide it to the RTP module. Furthermore, a traffic shaping module is built to extract the input processed by the RTP module and output the video data to the video processing pipeline. This module is responsible for maintaining the traffic shaping and delivery time requirements specified by ST2110-21 and compensating for clock drift between the FPGA internal clock and the PTP synchronization clock. Finally, after being processed by the traffic shaping module, it is encapsulated into a UDP message through an Ethernet module that is compatible with the 10Gbps Ethernet physical layer specification, and output as a network packet compliant with the ST2110 protocol through the physical layer Ethernet port.

2. The information classification module according to claim 1 is based on the Xilinx XC7K325T FPGA hardware platform to establish a device and system, characterized in that: Construct an information classification module to analyze whether the input data has continuity. If it has continuity, it is judged as the input code stream signal. If there is an end mark, the information type is determined. If the information format meets the corresponding requirements of the SDP message, it is judged as an SDP message. If the information format meets the corresponding requirements of the PTP message, it is judged as a PTP message. Otherwise, it is directly discarded.

3. The Session Description Protocol (SDP) module according to claim 1 is constructed based on a Xilinx XC7K325T FPGA hardware platform, and is characterized in that: Dynamic parsing parameters: The parsing process scans SDP signaling in real time through a dedicated state machine, correctly parses the information specified in IETF RFC4566, extracts video resolution, number of audio channels, and FEC enable flag to generate protocol configuration parameters. The extracted video resolution parameter directly controls the RTP fragment packet length, the FEC flag activates the corresponding error correction engine, and the number of audio channels parameter configures the multiplexing logic.

4. The real-time transport protocol (RTP) module according to claim 1 is based on a Xilinx XC7K325T FPGA hardware platform to establish a device and system, characterized in that: The RTP encapsulation module adopts a parallel pipeline architecture to perform 4K video row-level slicing operations; It integrates a serial number cycle counter and a timestamp register. The serial number generation content is controlled by dynamic parsing parameters, and the processing delay is ≤0.5μs / packet.

5. The PTP time synchronization module according to claim 1 is based on the Xilinx XC7K325TFPGA hardware platform to establish the device and system, characterized in that: A dual-clock source architecture is adopted to achieve IEEE 1588v2 synchronization based on the FPGA's own crystal oscillator and an external RTC high-precision clock module. The RTC high-precision clock module provides an absolute time reference for second-level counting, and cooperates with the FPGA's own crystal oscillator to achieve nanosecond-level precision timing. Through dual clock source collaboration and dynamic drift compensation, the synchronization accuracy of ±50ns is achieved.

6. The traffic shaping module according to claim 1 is a device and system built on the Xilinx XC7K325T FPGA hardware platform, characterized in that: The RTP packet header sequence number generation strategy is dynamically controlled according to the content obtained from the dynamic analysis of the protocol parameters. At the same time, the target bandwidth is calculated based on the video bit rate and frame rate parameters to configure the token injection rate of the traffic shaper.

7. The Ethernet module according to claim 1 is based on a Xilinx XC7K325T FPGA hardware platform to establish a device and system, characterized in that: Meeting the error correction requirements of the ST 2110-22 standard, its data link layer implements hardware-level CRC checking and low-latency frame encapsulation, and maps the quality of service parameters specified in the SDP to the PCP bits of the IEEE 802.1Q VLAN tag, ensuring high-priority transmission of critical audio and video streams.

Images