Airborne photoelectric pod image and communication network conversion device and method based on heterogeneous calculation

Abstract

The invention provides an airborne photoelectric pod image and communication network conversion device based on heterogeneous calculation, and belongs to the technical field of networked transmission of video and communication data. The FPGA preprocessing unit is connected with the 3G-SDI video input unit; the RK3588 encoding unit is connected with the FPGA preprocessing unit, and an H.265 hardware encoder and a dynamic code rate control module are arranged in the RK3588 encoding unit; the RS422 / UDP protocol conversion unit is connected with the RK3588 coding unit and an external RS422 interface; and a gigabit Ethernet physical interface. According to the device, the FPGA preprocessing unit and the RK3588 coding unit are integrated, so that efficient video coding and communication protocol lossless conversion are realized, and dynamic code rate control and low-delay transmission are supported.

Description

Technical Field

[0001] This invention belongs to the field of networked transmission technology of video and communication data, and specifically relates to an airborne optoelectronic pod image and communication network conversion device and method based on heterogeneous computing. Background Technology

[0002] Traditional airborne optoelectronic pods typically use 3G-SDI video interfaces and RS422 serial communication interfaces. These suffer from limitations such as point-to-point transmission architecture, difficulty in supporting multi-sensor data fusion, fixed bandwidth allocation, inability to adapt to dynamic mission scenarios, and lack of support for IP-based Ethernet architectures. As airborne systems evolve towards integrated modular avionics and open architectures, traditional communication methods have become insufficient in terms of protocol compatibility, scalability, and information sharing efficiency, necessitating upgrades through network switching technology. Summary of the Invention

[0003] The purpose of this invention is to solve the above-mentioned technical problems by integrating an FPGA preprocessing unit and an RK3588 encoding unit to achieve efficient video encoding and lossless conversion of communication protocols, supporting dynamic bitrate control and low-latency transmission.

[0004] To achieve the above objectives, the present invention provides the following technical solution: an airborne optoelectronic pod image and communication network conversion device based on heterogeneous computing, comprising: The 3G-SDI video input unit is configured to receive the raw video stream from the optoelectronic pod and perform clock recovery. The FPGA preprocessing unit, connected to the 3G-SDI video input unit, is used to integrate a pipelined architecture frame buffer controller and ROI dynamic cropping module. The RK3588 encoding unit, connected to the FPGA preprocessing unit, has a built-in H.265 hardware encoder and dynamic bit rate control module, used to achieve dynamic bit rate control in the range of 0.1-32Mbps; The RS422 / UDP protocol conversion unit, connected to the RK3588 encoding unit and the external RS422 interface, is used to achieve lossless bidirectional conversion between aviation bus and Ethernet protocols. A Gigabit Ethernet physical interface, connected to the RK3588 encoding unit and RS422 / UDP protocol conversion unit, is used to support the SRTP secure transmission protocol.

[0005] The airborne optoelectronic pod image and communication network conversion device based on heterogeneous computing provided by this invention also has the following technical feature: the FPGA preprocessing unit includes: A DDR3-based ring frame cache architecture is used to configure a 128-bit wide memory controller. The pixel-level background modeling module is configured to use the improved ViBe algorithm to detect moving regions. The programmable ROI partitioning module is configured to support adaptive cropping of four dynamic regions of interest.

[0006] The airborne optoelectronic pod image and communication network conversion device based on heterogeneous computing provided by this invention also has the following technical feature: the implementation method of the H.265 hardware encoder includes: Build a quad-core video processing pipeline, with each core configured with an independent QP value quantizer; Deploy a frame prediction acceleration module based on convolutional neural networks; It integrates aviation-specific metadata embedding channels and supports overlay encoding of optoelectronic pod status data.

[0007] The airborne optoelectronic pod image and communication network conversion device based on heterogeneous computing provided by this invention also has the following technical feature: the RS422 / UDP protocol conversion unit includes: RS422 physical layer driver circuit, supporting adaptive baud rates from 0.1Mbps to 10Mbps; The UDP packet encapsulation / parsing module implements the mapping between application layer data and the aviation bus protocol; A dual-buffered queue architecture, configured with a priority scheduling algorithm, ensures low-latency transmission of critical data.

[0008] Another objective of this invention is to provide a method for converting images from an airborne optoelectronic pod to a communication network based on heterogeneous computing, the method comprising the following steps: The system receives the original video stream from the optoelectronic pod via the 3G-SDI video input unit and performs clock recovery. The video stream is frame buffered and dynamically cropped using the FPGA preprocessing unit; The preprocessed video is H.265 encoded and dynamically bitrate controlled using the RK3588 encoding unit. The RS422 serial port data is converted into UDP packets, or vice versa, through the RS422 / UDP protocol conversion unit. It outputs encoded video and protocol data through a gigabit Ethernet interface and supports SRTP encrypted transmission.

[0009] Beneficial effects: The conversion device provided by this invention integrates an FPGA preprocessing unit and an RK3588 encoding unit to achieve efficient video encoding and lossless conversion of communication protocols, and supports dynamic bitrate control and low-latency transmission. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a hardware architecture diagram of the network conversion device provided in the embodiments of the present invention; Figure 2 This is a timing diagram of the 3G-SDI video processing pipeline provided in an embodiment of the present invention; Figure 3 This is the RS422 / UDP protocol switching state machine provided in the embodiments of the present invention; Figure 4 This is a schematic diagram showing the composition of the network conversion device and the connection relationship of related equipment provided in the embodiments of the present invention. Detailed Implementation

[0012] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be noted that these embodiments are not intended to limit the present invention. Equivalent changes or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the protection scope of the present invention.

[0013] In the description of the embodiments of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the invention.

[0014] Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0015] The terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art will understand the specific meaning of these terms in this invention based on the specific circumstances.

[0016] like Figure 1-4 As shown, this embodiment of the invention provides an airborne optoelectronic pod image and communication network conversion device based on heterogeneous computing, comprising: The 3G-SDI video input unit is configured to receive the raw video stream from the optoelectronic pod and perform clock recovery. The FPGA preprocessing unit, connected to the 3G-SDI video input unit, is used to integrate a pipelined architecture frame buffer controller and ROI dynamic cropping module. The RK3588 encoding unit, connected to the FPGA preprocessing unit, has a built-in H.265 hardware encoder and dynamic bit rate control module, used to achieve dynamic bit rate control in the range of 0.1-32Mbps; An RS422 / UDP protocol conversion unit, connected to the RK3588 encoding unit and an external RS422 interface, is used to achieve lossless bidirectional conversion between avionics bus and Ethernet protocols; and A Gigabit Ethernet physical interface, connected to the RK3588 encoding unit and RS422 / UDP protocol conversion unit, is used to support the SRTP secure transmission protocol.

[0017] In some embodiments, the 3G-SDI video processing pipeline timing includes stages such as video input, frame buffering, ROI cropping, and encoding output to ensure continuous processing of video data.

[0018] The device provided in the above embodiments uses FPGA to realize clock recovery and format conversion of 3G-SDI video signals and adopts hardware acceleration for dynamic ROI cropping; the RK3588 equipped with an NPU realizes real-time H.265 encoding with a dynamic bitrate control range of 0.1-32Mbps; the bidirectional protocol conversion engine realizes lossless bidirectional conversion between RS422 serial port data and UDP packets with a latency of less than 2ms. It overcomes the problem of traditional airborne optoelectronic pods being unable to quickly access network-based aircraft architectures. The transmission bandwidth can be adjusted in real time according to the airborne link bandwidth, and the encoding efficiency is four times that of traditional pure software encoding schemes, providing key technical support for the network upgrade of traditional airborne optoelectronic pods.

[0019] In some embodiments, the FPGA preprocessing unit includes: A DDR3-based ring frame cache architecture is used to configure a 128-bit wide memory controller. The pixel-level background modeling module is configured to use the improved ViBe algorithm to detect moving regions. The programmable ROI partitioning module is configured to support adaptive cropping of four dynamic regions of interest.

[0020] The FPGA used is from the Xilinx Artix-7 series, and the video pipeline configuration includes: 3G-SDI clock recovery module: Uses CDR technology to achieve ±1000ppm frequency offset correction; Dynamic downsampling module: performs spatial domain hierarchical processing based on motion vector detection results; Metadata embedding channel: Encapsulates the status data of the optoelectronic pod according to the SMPTE ST291 standard.

[0021] In some embodiments, the implementation method of the H.265 hardware encoder includes: Build a quad-core video processing pipeline, with each core configured with an independent QP value quantizer; Deploy a frame prediction acceleration module based on convolutional neural networks; It integrates aviation-specific metadata embedding channels and supports overlay encoding of optoelectronic pod status data.

[0022] In some embodiments, the workflow of the RK3588 encoding unit is as follows: The 3G-SDI video input unit receives YUV420 frame data; The motion estimation coprocessor executes CNNs to accelerate the search; The RK3588 quad-core encoding engine processes different ROI regions in parallel; The rate control module dynamically adjusts the QP value based on network conditions; The stream encapsulation unit generates a standard H.265 bitstream and adds a dedicated extended SEI.

[0023] In some embodiments, the RS422 / UDP protocol conversion unit includes: RS422 physical layer driver circuit, supporting adaptive baud rates from 0.1Mbps to 10Mbps; The UDP packet encapsulation / parsing module implements the mapping between application layer data and the aviation bus protocol; A dual-buffered queue architecture, configured with a priority scheduling algorithm, ensures low-latency transmission of critical data.

[0024] In some embodiments, the RS422 / UDP protocol conversion unit includes multiple working states such as idle, parsing, encapsulation, transmission, and error handling to ensure the reliability and low latency of protocol conversion.

[0025] The protocol conversion engine is implemented as follows: Design an aviation bus command parser that supports RS422 protocol conversion; Deploy a UDP retransmission module with a sliding window mechanism to ensure data reliability; Configure QoS policies to prioritize the transmission of video streams, control commands, and status information. The network transmission module integrates a PHY chip to implement the 1000BASE-T standard and uses the SRTP protocol to encapsulate and encrypt the video stream; Supports dynamic session key negotiation. Tests show that the bit error rate is less than 10⁻⁻⁴ under typical aviation electromagnetic interference conditions. 9 .

[0026] In some embodiments, a method for converting images from an airborne optoelectronic pod to a communication network based on heterogeneous computing is provided, the method comprising the following steps: The system receives the original video stream from the optoelectronic pod via the 3G-SDI video input unit and performs clock recovery. The video stream is frame buffered and dynamically cropped using the FPGA preprocessing unit; The preprocessed video is H.265 encoded and dynamically bitrate controlled using the RK3588 encoding unit. The RS422 serial port data is converted into UDP packets, or vice versa, through the RS422 / UDP protocol conversion unit. It outputs encoded video and protocol data through a gigabit Ethernet interface and supports SRTP encrypted transmission.

[0027] The measured performance of the device provided in any of the foregoing embodiments is as follows: 1080P 30Hz video is compressed to a minimum of 100Kbps, RS422 / UDP conversion rate is up to 10Mbps, end-to-end processing latency is 65ms, and encoding efficiency is 4 times higher than traditional pure software encoding schemes, meeting the GJB669A-2019 aviation network transmission requirements.

[0028] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention. The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the protection scope of the present invention.

Claims

1. A heterogeneous computing based airborne opto-electronic pod image and communication network conversion device, characterized in that, The application relates to a video processing method and device based on 3G-SDI and RS422 / UDP protocol conversion. 3G-SDI video input unit configured to receive optoelectronic pod raw video stream and perform clock recovery; FPGA preprocessing unit connected with the 3G-SDI video input unit, used for integrating a frame buffer controller and a ROI dynamic cropping module in a pipeline architecture; RK3588 encoding unit connected with the FPGA preprocessing unit, internally provided with an H.265 hardware encoder and a dynamic code rate control module, used for realizing dynamic code rate control in a range of 0.1-32 Mbps; RS422 / UDP protocol conversion unit connected with the RK3588 encoding unit and an external RS422 interface, used for realizing lossless bidirectional conversion between an aviation bus and an Ethernet protocol; Gigabit Ethernet physical interface connected with the RK3588 encoding unit and the RS422 / UDP protocol conversion unit, used for supporting an SRTP secure transmission protocol.

2. The heterogeneous computing based airborne opto-electronic pod image and communication network conversion device of claim 1, wherein, The FPGA preprocessing unit comprises: a DDR3-based ring-shaped frame buffer architecture used for configuring a 128-bit wide storage controller; a pixel-level background modeling module configured to realize motion area detection by using an improved ViBe algorithm; a programmable ROI partition module configured to support adaptive cropping of four dynamic focus areas.

3. The heterogeneous computing based airborne opto-electronic pod image and communication network conversion device of claim 1, wherein, The implementation method of the H.265 hardware encoder comprises: constructing a four-core video processing pipeline, and configuring an independent QP value quantizer for each core; deploying an interframe prediction acceleration module based on a convolutional neural network; integrating an aviation special metadata embedding channel to support superimposed coding of optoelectronic pod state data.

4. The heterogeneous computing based airborne opto-electronic pod image and communication network conversion device of claim 1, wherein, The RS422 / UDP protocol conversion unit comprises: an RS422 physical layer driving circuit supporting adaptive baud rate of 0.1-10 Mbps; a UDP message encapsulation / analysis module realizing mapping of application layer data and aviation bus protocol; a double-buffer queue architecture configured with a priority scheduling algorithm to guarantee low-latency transmission of key data.

5. A method for converting airborne optoelectronic pod images and communication networks based on heterogeneous computing, characterized in that, The method comprises the following steps: receiving an optoelectronic pod raw video stream through a 3G-SDI video input unit and performing clock recovery; performing frame buffering and ROI dynamic cropping on the video stream through an FPGA preprocessing unit; performing H.265 encoding and dynamic code rate control on the preprocessed video through an RK3588 encoding unit; converting RS422 serial port data into UDP messages or vice versa through an RS422 / UDP protocol conversion unit; outputting encoded video and protocol data through a gigabit Ethernet interface, and supporting SRTP encrypted transmission.

Images