Video processing system and method of airborne intelligent display

Abstract

The invention provides a video processing system and method for an airborne intelligent display, belongs to the technical field of airborne display videos, and is realized based on an FPGA (Field Programmable Gate Array) processing framework. The FPGA comprises a DP decoding module, an ARINC818 decoding module, a TPG module, a video detection and selection module, a video interaction module, a video cache control module, a video superposition module, a DP coding module, a memory control module, a Flash controller, a central processing module, a bus interaction processing module, a UART module and a key module. Through the processing scheme provided by the invention, multiple paths of DP video signals can be decoded at the same time, a multi-link high-bandwidth ARINC818 decoding processing capability, a high-bandwidth and low-delay external view superposition capability and an eDP coding capability are realized, and the performance requirements of a modern intelligent display can be met.

Description

Technical Field

[0001] The present application relates to the field of airborne display video technology, and in particular to a video processing system and method for an airborne intelligent display. Background Art

[0002] With the development of aircraft cockpit technology, airborne displays are moving towards high resolution and intelligence, which requires the displays to have both high-resolution drawing and display as well as intelligent computing and processing. This requires airborne displays to have both high-performance GPUs and high-performance AI processors. The video output interface of high-performance GPUs and high-performance AI processors generally uses a DP video interface, and the output high-resolution video signal needs to be selected by the video processing module before being transmitted to the display panel for display. In addition to processing high-resolution video signals, the video processing module also needs to overlay the input high-resolution ARINC818 video interface signal of the external input.

[0003] Most of the traditional airborne video processing modules process low-resolution videos such as 1920x1080@60hz and 1600x1200@60hz, and do not specifically process high-resolution video signals (4K@60hz) for the DP ultra-clear video interface. The link rate of the ARINC818 video link of the external input of the traditional airborne video processing module generally adopts a single link of 4.25Gbps, and the load video resolution that can be transmitted is relatively limited. In addition, the overlay processing method of the traditional video processing module cannot be applied to the occasions of high-resolution multi-channel DP video and high-resolution ARINC818 video overlay due to the relatively low resolution and low video clock of the processing logic, the consumed FPGA logic resources, and the timing is easier to converge. This patent discloses a high-resolution video processing method for airborne intelligent displays, which can decode three high-resolution DP video signals at the same time, select one of them to be overlaid with the ARINC818 external video, and then transmit it to the eDP panel through the eDP interface for display. This video processing method has the characteristics of multi-channel DP decoding, high-bandwidth video processing, and low-latency video superposition. Summary of the invention

[0004] In view of this, the conventional video processing module cannot meet the occasions of video processing of existing intelligent displays, the existing video processing module cannot decode multiple channels of high-resolution DP videos, it is difficult to process multiple channels of 4K high-resolution videos, there is a lack of multi-pixel video overlay processing methods for multi-link ARINC818 decoded videos, and it is impossible to encode and display high-bandwidth eDP video signals. The embodiment of the present application proposes a video processing system and method for an airborne intelligent display, which can simultaneously decode three channels of 4K resolution DP video signals. The DP video signals come from the latest SOC, GPU, etc., and have multi-link high-bandwidth ARINC818 decoding processing capabilities, high-bandwidth, low-latency external scene overlay capabilities, and eDP encoding capabilities, which can meet the performance requirements of modern intelligent displays.

[0005] In a first aspect, an embodiment of the present application provides a video processing system for an airborne intelligent display, the system being implemented based on an FPGA processing framework, the FPGA comprising a DP decoding module, an ARINC818 decoding module, a TPG module, a video detection and selection module, a video interaction module, a video cache control module, a video overlay module, a DP encoding module, a memory control module, a Flash controller, a central processing module, a bus interaction processing module, a UART module and a key module;

[0006] The DP decoding module, the video interaction module and the video cache control module are connected in sequence, the ARINC818 decoding module and the TPG module are connected to the video detection and selection module, the video detection and selection module and the video cache control module are both connected to the video overlay module, the video overlay module is connected to the DP decoding module, the bus interaction processing module is respectively connected to the UART module, the key module, the video detection and selection module, the video interaction module, the central processing module, the Flash controller and the video overlay module, and the video cache control module is also connected to the memory control module;

[0007] The DP decoding module is provided with multiple modules for converting multiple serial DP data signals into video data signals in 4-pixel mode for output; the video interaction module is used to select one of the 4-pixel mode video data signals output by the DP decoding module and transmit it to the video buffer control module, and the video buffer control module is used for video frame storage processing control; the ARINC818 decoding module is used to decode the ARINC818 video signal and convert it into a video signal in Native format, the TPG module is used to generate a video-free signal, and the video detection and selection module is used to detect whether the video signal output by the ARINC818 decoding module exists. If not, the video detection and selection module selects the video-free signal for output display. If it exists, the video detection and selection module selects the Na tive format video signal for output and display; the video overlay module is used to overlay the DP video signal after the frame storage of the video cache control module with the video signal output by the video detection and selection module; the DP encoding module is used to convert the 4-pixel mode video data signal after the overlay processing into a serial DP data signal output; the Flash controller is used to control the reading and writing of the external Flash chip, the memory control module is used to control the access to the external memory, the central processing module is used for FPGA internal system scheduling and configuration of each module, the bus interaction processing module is used for bus interconnection of each module, the UART module is used for communication between the FPGA processing framework and other processors, and the key module is used to select one of the decoded DP video signals to enter the frame storage.

[0008] According to a specific implementation method of an embodiment of the present application, three DP decoding modules are provided, and the three DP decoding modules are respectively connected to a graphics board, a backup graphics board and an intelligent computing board through a DP receiving port. The graphics board is used to draw and generate an onboard display screen and output video data through a DP video interface. The backup graphics board is used to back up the graphics board. The intelligent computing board is used for AI calculations and displays image processing results through a DP video interface.

[0009] According to a specific implementation method of an embodiment of the present application, each of the DP decoding modules includes a first vid_phy module and a dp_rx_subsystem module. The first vid_phy module operates in 5.4 Gbps mode and is used to convert the serial 4laneDP signal into 4 groups of parallel DP data signals; the dp_rx_subsystem module has a DP protocol parsing function and is used to convert the 4 groups of parallel DP data signals into 4-pixel mode video data signals for output.

[0010] According to a specific implementation method of an embodiment of the present application, the DP encoding module includes a second vid_phy module and a dp_tx_subsystem module, the dp_tx_subsystem module has a DP protocol encoding function, and is used to convert the video data signal of the 4-pixel mode into 4 groups of parallel DP data signals. The second vid_phy module operates in 5.4Gbps mode and is used to convert the 4 groups of parallel DP data signals into serial 4laneDP signal conversion.

[0011] According to a specific implementation method of an embodiment of the present application, the video overlay module includes an AXISData FIFO module, a read enable controller module and an overlay calculation module, the read enable controller module is used to generate a read enable control signal rd_en of the AXISData FIFO module, the AXIS Data FIFO module is used to receive the 4-pixel AXIS data output by the video cache control module, and output 4-pixel RGB data according to whether the rd_en signal is valid, and the overlay calculation module is used to perform overlay calculation on the video signal in Native format and the 4-pixel RGB data and generate a video signal in Native format.

[0012] According to a specific implementation method of an embodiment of the present application, the video signal in the Native format includes a VS signal, a HS signal, a DE signal and a Data signal, the VS signal is a video frame synchronization signal, the HS signal is a video line synchronization signal, the DE signal is a video valid signal, and the Data signal is a 4-pixel video data signal.

[0013] According to a specific implementation method of an embodiment of the present application, the link rates supported by the ARINC818 decoding module include 1.0625 Gbps, 2.125 Gbps, 4.25 Gbps, 8.5 Gbps and 10 Gbps, and the link quantity transmission supported by the ARINC818 decoding module includes single-link transmission, dual-link transmission and quad-link transmission.

[0014] According to a specific implementation of the embodiment of the present application, the memory is configured as a DDR memory for implementing a frame buffer of the video.

[0015] In a second aspect, an embodiment of the present application provides a video processing method for an airborne intelligent display, which is implemented based on the video processing system as described in any embodiment of the first aspect, and the method includes:

[0016] The DP decoding module decodes multiple DP video signals from the onboard display;

[0017] The external key information is transmitted to the MicroBlaze module, and then the MicroBlaze module controls the video interaction module to select one of the DP video signal outputs;

[0018] The output DP video signal is cached into the memory through the video cache control module;

[0019] Another external video signal is decoded by the ARINC818 decoding module and decoded into a video signal in Native format;

[0020] The video overlay module controls the synchronization of the cached video signal and the video signal in the Native format. After the synchronization, the video signal is sent to the DP encoding module for encoding and output after the video overlay calculation.

[0021] According to a specific implementation of the embodiment of the present application, the video overlay module controls the synchronization of the cached video signal and the video signal in the Native format, including:

[0022] Step 1: After the system is powered on, the MicroBlaze module determines whether the arinc818_valid signal and the vid_valid signal output by the ARINC818 decoding module are valid. The arinc818_valid signal represents that the ARINC818 data is valid, and the vid_valid signal represents that the decoded video signal is valid. If the arinc818_valid signal and the vid_valid signal are valid, jump to step 2; if not, jump to step 1.

[0023] Step 2: The MicroBlaze module configures the video cache control module to start working, and generates a vdma_config_done signal to transmit to the read enable controller module. The video cache control module starts to read video data from the memory and sends it to the AXIS Data FIFO module, and then jumps to step 3;

[0024] Step 3: The read enable controller module determines whether the vdma_config_done signal is valid, if it is valid, jump to step 4, if it is invalid, jump to step 3;

[0025] Step 4: Determine whether the fifo_data_valid and fifo_data_tuser signals output by the AXIS Data FIFO module are valid, if valid, jump to step 5, otherwise, keep at step 4;

[0026] Step 5: Wait for the decoded video frame count signal v_cnt to be equal to 5. If it is equal to 5, jump to step 6, otherwise, keep at step 5;

[0027] Step 6: Turn on enable rd_en, whose value is the value of de, and jump to step 7:

[0028] Step 7: Determine whether the arinc818_valid signal and the vid_valid signal output by the ARINC818 decoding module are valid. If they are valid, keep in step 7; otherwise, proceed to step 8;

[0029] Step 8: Enable no video input signal and go to step 1.

[0030] Beneficial effects:

[0031] The video processing system and method of the airborne intelligent display in the embodiment of the present application adopts the DP codec interface as the interface of the video processing module, which has a higher bandwidth transmission capability than the DVI, RGB and other video interfaces adopted by the previous airborne display video processing module. The DP video interface can achieve up to 4K@60Hz ultra-clear video codec;

[0032] This application adopts a 4-pixel video processing architecture. The DP decoding module, DP encoding module, ARINC818 decoding module, video interaction module, video detection and selection module, video cache control module, and video overlay module all adopt a 4-pixel processing architecture. Its video processing capability is far greater than the single-pixel processing mechanism of the traditional architecture. Without increasing the pixel clock, the video processing bandwidth is increased by four times. Solve the timing problems such as the difficulty of FPGA internal wiring and the shortage of clock resources caused by increasing the video clock.

[0033] This application has great advantages in the field of smart displays. The traditional video processing module uses a DVI interface. With the improvement of GPU performance, its interface has become a higher-speed DP interface. This traditional architecture can no longer be used. However, this application has a 3-way DP decoding module that can decode 3 high-performance SOCs or GPUs at the same time. That is, it can take into account the high-performance GPU drawing display requirements and solve the ultra-clear video display requirements of the high-performance computing SOC in the smart display.

[0034] The processing architecture designed by the present invention uses the ARINC818 decoding module to directly decode and output the video data in Native format. Compared with the traditional architecture that uses frame buffer to realize the decoding cache and conversion of ARINC818 external video, this method has the characteristics of low-latency video processing, and the delay is much lower than the refresh time of a frame of video. In addition, the video synchronization and superposition method designed by the present invention has the ability of four-pixel video processing synchronization and superposition, and realizes high-bandwidth video synchronization and superposition. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 is a logic block diagram of an FPGA according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of video synchronization and superposition according to an embodiment of the present invention;

[0038] Figure 3 is a flow chart of video synchronization control according to an embodiment of the present invention;

[0039] Figure 4 is a timing diagram of synchronization control according to an embodiment of the present invention;

[0040] Figure 5 The present invention is a block diagram of a video overlay computing module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0043] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present application, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0044] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. The illustrations only show the components related to the present application rather than being drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0045] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0046] The present application provides a video processing system and method for an airborne intelligent display. Figures 1 to 5 Describe in detail.

[0047] In a first aspect, an embodiment of the present application provides a video processing system for an airborne intelligent display, the system being implemented based on an FPGA processing framework, the FPGA comprising a DP decoding module, an ARINC818 decoding module, a TPG module, a video detection and selection module, a video interaction module, a video cache control module, a video overlay module, a DP encoding module, a memory control module, a Flash controller, a central processing module, a bus interaction processing module, a UART module and a key module;

[0048] The DP decoding module, the video interaction module and the video cache control module are connected in sequence, the ARINC818 decoding module and the TPG module are connected to the video detection and selection module, the video detection and selection module and the video cache control module are both connected to the video overlay module, the video overlay module is connected to the DP decoding module, the bus interaction processing module is respectively connected to the UART module, the key module, the video detection and selection module, the video interaction module, the central processing module, the Flash controller and the video overlay module, and the video cache control module is also connected to the memory control module;

[0049] The DP decoding module is provided with multiple modules for converting multiple serial DP data signals into video data signals in 4-pixel mode for output; the video interaction module is used to select one of the 4-pixel mode video data signals output by the DP decoding module and transmit it to the video buffer control module, and the video buffer control module is used for video frame storage processing control; the ARINC818 decoding module is used to decode the ARINC818 video signal and convert it into a video signal in Native format, the TPG module is used to generate a video-free signal, and the video detection and selection module is used to detect whether the video signal output by the ARINC818 decoding module exists. If not, the video detection and selection module selects the video-free signal for output display. If it exists, the video detection and selection module selects the Na tive format video signal for output and display; the video overlay module is used to overlay the DP video signal after the frame storage of the video cache control module with the video signal output by the video detection and selection module; the DP encoding module is used to convert the 4-pixel mode video data signal after the overlay processing into a serial DP data signal output; the Flash controller is used to control the reading and writing of the external Flash chip, the memory control module is used to control the access to the external memory, the central processing module is used for FPGA internal system scheduling and configuration of each module, the bus interaction processing module is used for bus interconnection of each module, the UART module is used for communication between the FPGA processing framework and other processors, and the key module is used to select one of the decoded DP video signals to enter the frame storage.

[0050] The video processing system of the airborne intelligent display in this embodiment is designed to address situations where traditional video processing modules cannot meet the requirements of existing intelligent display video processing, existing video processing modules cannot decode multiple channels of high-resolution DP videos, are difficult to process multiple channels of 4K high-resolution videos, lack a multi-pixel video overlay processing method for multi-link ARINC818 decoded videos, and cannot encode and display high-bandwidth eDP video signals. This embodiment proposes a video processing system that can simultaneously decode three channels of 4K resolution DP video signals. The DP video signals come from the latest SOC, GPU, etc., and have multi-link high-bandwidth ARINC818 decoding and processing capabilities, high-bandwidth, low-latency external scene overlay capabilities, and eDP encoding capabilities, which can meet the performance requirements of modern intelligent displays.

[0051] In this embodiment, the FLASH chip is used to store the FPGA configuration solidification file, and the Flash controller is an IP core based on the AXI4Lite bus, which is used to control the reading and writing of the Flash chip. The memory is a DDR memory, which is used for the frame buffer function of the video; the memory controller is used to control the access to the DDR memory memory, and its function is to convert the internal bus of the FPGA into the control bus of the DDR, which is implemented by the MIG IP core of Xilinx. The central processing module is used for the functions of FPGA internal system scheduling, FPGA internal IP core configuration, etc., and it has an AXI4Lite bus interface. The bus interaction processing module is used for the bus interconnection function of each IP core in the FPGA system. The video interaction module is used to select one of the three DP videos to the VDMA module, and the selection judgment logic is selected by an external key signal. The video cache control module is used for the frame storage processing control of the video, and the VDMA of Xilinx is used to implement this function. The three-frame storage design is used to avoid problems such as screen tearing. The TPG module is used to generate a video-free screen in a video-free situation. The UART module described in the present invention is used for communication between the FPGA system and other processors. The ARINC818 decoding module is used to decode the ARINC818 video signal from the display and control processor. The video detection and selection module is used to detect the presence or absence of the video signal output by the ARINC818 decoding module. If the video signal decoded and output by the ARINC818 module does not exist, the module selects the video signal generated by the TPG for output display. If the video signal decoded by the ARINC818 decoding module exists, the module outputs the video signal decoded by the ARINC818 decoding module. The video overlay module is used to overlay the DP video signal after frame storage with the video signal decoded by ARINC818.

[0052] Specifically, the ARINC818 decoding module is used to decode the ARINC818 video signal from the display and control processor. The module can support different link rates, including 1.0625Gbps, 2.125Gbps, 4.25Gbps, 8.5Gbps and 10Gbps, and also supports different link quantity transmissions, including single-link transmission, dual-link transmission, quad-link transmission, etc. The module supports ARINC818 decoding of multiple resolutions and supports a maximum video bandwidth of 40Gbps. However, in the present invention, a 4lane10Gbps link is used to receive the ARINC818 video signal to achieve higher bandwidth video transmission. The ARINC818 decoding module decodes the ARINC818 decoded video signal into a video signal in Native format. The video signal in Native format includes VS, HS, DE and Data video signals. The VS signal is a video frame synchronization signal, the HS signal is a video line synchronization signal, the DE signal is a video valid signal, and the Data video signal is a 4-pixel video data signal.

[0053] In one embodiment, three DP decoding modules are provided, and the three DP decoding modules are respectively connected to a graphics board, a backup graphics board, and an intelligent computing board through a DP receiving port. The graphics board is used to draw pictures to generate an onboard display and output video data through a DP video interface. The backup graphics board is used to back up the graphics board. The intelligent computing board is used for AI calculations and displays image processing results through a DP video interface.

[0054] Reference Figure 1 The video processing system of the airborne intelligent display is controlled by FPGA, including 3 DP receiving ports, 2 eDP sending ports, and 1 ARINC818 port. The 3 DP interface ports are connected to the graphics board, backup graphics board and intelligent computing board respectively, and the ARINC818 port is connected to the external visual source.

[0055] Specifically, the graphics board described in this embodiment is used to draw and generate the onboard display screen, which is generally composed of a high-performance CPU and a high-performance GPU. Its output video interface is generally a DP video interface, and the drawing resolution can reach 4K@60Hz. The backup graphics board function described in this embodiment is the same as the function of the graphics board, which serves as a backup function of the graphics board. The intelligent computing board described in this embodiment is used for AI computing functions, and the image processing results will be displayed through the DP video interface.

[0056] In one embodiment, each of the DP decoding modules includes a first vid_phy module and a dp_rx_subsystem module. The first vid_phy module operates in 5.4 Gbps mode and is used to convert the serial 4laneDP signal into 4 groups of parallel DP data signals; the dp_rx_subsystem module has a DP protocol parsing function and is used to convert the 4 groups of parallel DP data signals into 4-pixel mode video data signals for output.

[0057] In specific implementation, the DP decoding module includes the vid_phy module and the dp_rx_subsystem module, both of which are Xilinx IP cores. The vid_phy module works in 5.4Gbps mode and is used to convert the serial 4lane DP signal into 4 sets of parallel DP data signals. The dp_rx_subsystem module has the DP protocol parsing function and is used to convert the 4 sets of parallel DP data signals into 4-pixel mode video data signals for output.

[0058] In one embodiment, the DP encoding module includes a second vid_phy module and a dp_tx_subsystem module, the dp_tx_subsystem module has a DP protocol encoding function, and is used to convert a 4-pixel mode video data signal into 4 groups of parallel DP data signals. The second vid_phy module operates in a 5.4 Gbps mode and is used to convert 4 groups of parallel DP data signals into serial 4lane DP signal conversion.

[0059] In specific implementation, the DP encoding module includes the vid_phy module and the dp_tx_subsystem module. Both modules are Xilinx IP cores. The vid_phy module works in 5.4Gbps mode and is used to convert 4 sets of parallel DP data signals into serial 4lane DP signals. The dp_tx_subsystem module has the DP protocol encoding function and is used to convert the 4-pixel mode video data signal into 4 sets of parallel DP data signals.

[0060] In specific implementation, the FPGA uses at least 6 groups of high-speed GTXbanks for parallel-to-serial and serial-to-parallel conversion of high-speed data.

[0061] In specific implementation, the DP decoding module and the eDP encoding module use GTX as the high-speed transceiver of Video PHY.

[0062] In one embodiment, the video overlay module includes an AXIS Data FIFO module, a read enable controller module and an overlay calculation module, the read enable controller module is used to generate a read enable control signal rd_en of the AXIS Data FIFO module, the AXIS Data FIFO module is used to receive the 4-pixel AXIS data output by the video cache control module, and output 4-pixel RGB data according to whether the rd_en signal is valid, and the overlay calculation module is used to perform overlay calculation on the video signal in Native format and the 4-pixel RGB data and generate a video signal in Native format.

[0063] In one embodiment, the video signal in the Native format includes a VS signal, a HS signal, a DE signal and a Data signal, the VS signal is a video frame synchronization signal, the HS signal is a video line synchronization signal, the DE signal is a video valid signal, and the Data signal is a 4-pixel video data signal.

[0064] In specific implementation, the video overlay module is used to overlay the DP video signal after frame storage with the video signal after ARINC818 decoding. The implementation principle block diagram is as follows: Figure 2 As shown in FIG. 1 , since the video signal decoded by ARINC818 is not synchronized with the video signal decoded by DP, the video buffer control module is required to synchronize the two video signals before the video overlay operation can be performed. The video buffer control module is Figure 2 The VDMA module shown is implemented using Xilinx's VDMAIP.

[0065] In this embodiment, the video signal after ARINC818 decoding is a 4-pixel Native format video data, and the DP video signal after VDMA buffering is a 4-pixel Axis format video data. Figure 2 As shown in the figure, the video data of the AXIS interface output by the VDMA is connected to a Data FIFO of the AXIS interface, namely the AXIS Data FIFO. The AXISData FIFO outputs two flag signals, signal 4 and signal 5, which are respectively the fifo_data_valid and fifo_data_tuser signals of the AXIS interface of the AXIS Data FIFO. The fifo_data_valid is the data valid signal of the AXIS Data FIFO, and the fifo_data_tuser signal is the data start signal of the AXIS Data FIFO, and also represents the valid signal of the first pixel of the video frame. These two signals are respectively given to the read enable controller module.

[0066] The read enable controller module is used to generate the read enable control signal rd_en of the AXIS Data FIFO, which is also called the fifo_data_tready signal. The 4-pixel RGB data output by the AXIS Data FIFO is output according to whether rd_en is valid. The read enable controller module controls the rd_en signal according to signal 3, i.e., the vdma_config_done signal. The signals 1 and 2 described in this embodiment come from the ARINC818 decoding module. These two signals are input to MicroBlaze, which makes judgments and controls them.

[0067] In specific implementation, the flowchart of video synchronization is as follows Figure 3 As shown, the steps are as follows:

[0068] Step 1: After the system is powered on, the MicroBlaze module determines whether the arinc818_valid and vid_valid signals are valid. These two signals represent the validity of the ARINC818 data and the decoded video signal, respectively. If valid, jump to step 2; if invalid, jump to step 1.

[0069] Step 2: The MicroBlaze module configures VDMA to start working and generates a vdma_config_done signal to send to the read enable controller module. VDMA starts to read video data from DDR and sends it to the AXIS Data FIFO module, and then jumps to step 3;

[0070] Step 3: The read enable controller module determines whether the vdma_config_done signal is valid. If it is valid, it jumps to step 4; if it is invalid, it jumps to step 3;

[0071] Step 4: Determine whether the fifo_data_valid and fifo_data_tuser signals of the AXIS Data FIFO are valid. If they are valid, jump to step 5. If they are invalid, stay in step 4.

[0072] Step 5: Wait for the decoded video frame count signal v_cnt to be equal to 5. If it is equal to 5, jump to step 6, otherwise, keep at step 5;

[0073] Step 6: Enable rd_en, whose value is the value of de. In fact, de is played twice before being given to rd_en. Jump to step 7:

[0074] Step 7: The detection function is the same as step 1. If it is valid, keep in step 7, otherwise go to step 8;

[0075] Step 8: Enable no video input signal and go to step 1;

[0076] This application can achieve the synchronization of ARINC818 decoded video signal and DP video signal through the above steps. The synchronization control timing diagram is as follows: Figure 4 As shown in the figure, after the arinc818_valid and vid_valid signals are valid, VDMA is configured. After the configuration is completed, the vdma_config_done signal is valid. After this signal is valid, video data begins to appear in the AXIS Data FIFO, and the fifo_data_tuser and fifo_data_valid signals become valid. The module then starts waiting for the arrival of the de signal to achieve video synchronization. After vs_cnt is equal to 5, the de video signal is sent to the fifo_data_tready signal after tapping, and then video data can be continuously obtained from DDR. The data rate that VDMA obtains from DDR must be faster than the output rate of AXIS Data FIFO, so that the synchronized video signal will not be lost.

[0077] The video data flow process of the video processing system of the airborne intelligent display of the present application is as follows. The DP decoding module decodes the DP video signal from the graphics board, the backup graphics board, and the intelligent computing board. The external button information is transmitted to the MicroBlaze module, and then the MicroBlaze module controls the video interaction module to select one of the DP video signal outputs, and the output DP video signal is cached into the DDR through the video cache control module. Another external video signal is decoded by the ARINC818 decoding module and decoded into a video signal in Native format. After that, the video overlay module controls the synchronization of the cached video signal and the ARINC818 decoded video signal. After the synchronization, the video signal is sent to the DP encoding module for encoding and output after the video overlay calculation. In the present invention, in order to increase the processing bandwidth of the FPGA system and reduce the problems such as the difficulty of timing convergence caused by the layout and wiring difficulties caused by increasing the pixel clock frequency, all 4-pixel format video signals (4-pixel Native format video signals, 4-pixel AXIS format video signals) are used for processing, which can meet the requirements of high-resolution and high-frame rate video processing.

[0078] Furthermore, the video overlay principle is as follows Figure 5 As shown in the figure, the video overlay of a single pixel is shown. To achieve 4-pixel parallel overlay, four Figure 5The module calculates the video superposition value for 4 pixels at the same time. As shown in the figure, the contrast is 8 bits, and it is multiplied with the RGB components of Video1 by XOR calculation with the hexadecimal value 8'hFF. The RGB components of another video Video2 are directly multiplied with the 8-bit contrast. The addition result of the above two multiplication results is divided by the hexadecimal value 8'hFF to get the superposition value of the RGB components.

[0079] In one embodiment, the link rates supported by the ARINC818 decoding module include 1.0625 Gbps, 2.125 Gbps, 4.25 Gbps, 8.5 Gbps and 10 Gbps, and the link quantity transmission supported by the ARINC818 decoding module includes single-link transmission, dual-link transmission and quad-link transmission.

[0080] In one embodiment, the memory is configured as a DDR memory for implementing a frame buffer of a video.

[0081] In a second aspect, an embodiment of the present application provides a video processing method for an airborne intelligent display, which is implemented based on the video processing system as described in any embodiment of the first aspect, and the method includes:

[0082] The DP decoding module decodes multiple DP video signals from the onboard display;

[0083] The external key information is transmitted to the MicroBlaze module, and then the MicroBlaze module controls the video interaction module to select one of the DP video signal outputs;

[0084] The output DP video signal is cached into the memory through the video cache control module;

[0085] Another external video signal is decoded by the ARINC818 decoding module and decoded into a video signal in Native format;

[0086] The video overlay module controls the synchronization of the cached video signal and the video signal in the Native format. After the synchronization, the video signal is sent to the DP encoding module for encoding and output after the video overlay calculation.

[0087] Furthermore, the video overlay module controls the synchronization of the cached video signal and the video signal in the Native format, including:

[0088] Step 1: After the system is powered on, the MicroBlaze module determines whether the arinc818_valid signal and the vid_valid signal output by the ARINC818 decoding module are valid. The arinc818_valid signal represents that the ARINC818 data is valid, and the vid_valid signal represents that the decoded video signal is valid. If the arinc818_valid signal and the vid_valid signal are valid, jump to step 2; if not, jump to step 1.

[0089] Step 2: The MicroBlaze module configures the video cache control module to start working, and generates a vdma_config_done signal to transmit to the read enable controller module. The video cache control module starts to read video data from the memory and sends it to the AXIS Data FIFO module, and then jumps to step 3;

[0090] Step 3: The read enable controller module determines whether the vdma_config_done signal is valid, if it is valid, jump to step 4, if it is invalid, jump to step 3;

[0091] Step 4: Determine whether the fifo_data_valid and fifo_data_tuser signals output by the AXIS Data FIFO module are valid, if valid, jump to step 5, otherwise, keep at step 4;

[0092] Step 5: Wait for the decoded video frame count signal v_cnt to be equal to 5. If it is equal to 5, jump to step 6, otherwise, keep at step 5;

[0093] Step 6: Turn on enable rd_en, whose value is the value of de, and jump to step 7:

[0094] Step 7: Determine whether the arinc818_valid signal and the vid_valid signal output by the ARINC818 decoding module are valid. If they are valid, keep in step 7; otherwise, proceed to step 8;

[0095] Step 8: Enable no video input signal and go to step 1.

[0096] The above steps can realize the synchronization between the ARINC818 decoded video signal and the DP video signal. For the specific contents, please refer to the first aspect and will not be repeated here.

[0097] The video processing system and method of the airborne intelligent display of the present application are described below with a specific example.

[0098] Three DP video decoding modules are required to decode the 4K@60Hz DP video signals output by two X100s and one RK3588 respectively. Two X100s are used for onboard video mapping functions and back up each other. One RK3588 is used for AI intelligent computing (face recognition, etc.). The decoded DP video signal is AXIS video data in 4-pixel format.

[0099] The video interaction module is selected by an external button to select one of the DP video signals for output. This DP video signal is input into DDR through the video cache control module VDMA for video frame storage, waiting for video synchronization processing. Another 4K@60Hz video is input through the 4Lane 10Gbps ARINC818 interface and decoded into 4-pixel Native format video by the ARINC818 decoding module.

[0100] The 4-pixel AXIS interface video data output by the VDMA is connected to a Data FIFO of the AXIS interface, namely the AXIS Data FIFO. The AXIS Data FIFO outputs two flag signals, fifo_data_valid and fifo_data_tuser. The read enable controller module controls the rd_en signal according to signal 3, namely the vdma_config_done signal.

[0101] The steps of the synchronization process diagram of the entire video are as follows:

[0102] Step 1: The MicroBlaze module determines whether the arinc818_valid and vid_valid signals are valid. If valid, it jumps to step 2; if not, it jumps to step 1.

[0103] Step 2: The MicroBlaze module configures VDMA to start working and generates a vdma_config_done signal to send to the read enable controller module. VDMA starts to read video data from DDR and sends it to the AXIS Data FIFO module, and then jumps to step 3;

[0104] Step 3: The read enable controller module determines whether the vdma_config_done signal is valid. If it is valid, it jumps to step 4; if it is invalid, it jumps to step 3;

[0105] Step 4: Determine whether the fifo_data_valid and fifo_data_tuser signals of the AXIS Data FIFO are valid. If they are valid, jump to step 5. If they are invalid, stay in step 4.

[0106] Step 5: Wait for the decoded video frame count signal v_cnt to be equal to 5. If it is equal to 5, jump to step 6, otherwise, keep at step 5;

[0107] Step 6: Enable rd_en, whose value is the value of de. In fact, de is played twice before being given to rd_en. Jump to step 7:

[0108] Step 7: The detection function is the same as step 1. If it is valid, keep in step 7, otherwise go to step 8;

[0109] Step 8: Enable no video input signal and go to step 1;

[0110] The above steps can realize the synchronization of ARINC818 decoded video signal and DP video signal. The data rate obtained by VDMA from DDR is faster than the output rate of AXIS Data FIFO, so the synchronized video signal will not be lost.

[0111] After the input DP video signal is synchronized with the ARINC818 video signal, the superposition calculation can be started. The contrast is 8 bits, and the RGB components of Video1 are multiplied by the XOR calculation with the hexadecimal value 8'hFF. The RGB components of another video Video2 are directly multiplied by the 8-bit contrast. The addition result of the above two multiplication results is divided by the hexadecimal value 8'hFF to get the superposition value of the RGB components.

[0112] The superimposed video data are respectively transmitted to two DP encoding modules, one for screen display and the other for video recording unit for video recording.

[0113] Through the above process, 4K video processing can be realized, and the video processing requirements of two X100s and one RK3588 intelligent processor can be met, completing the design and development of the smart display.

[0114] The embodiment provided by the present invention adopts the DP codec interface as the interface of the video processing module. Compared with the DVI, RGB and other video interfaces adopted by the previous airborne display video processing module, it has higher bandwidth transmission capability. The DP video interface can achieve up to 4K@60Hz ultra-high-definition video encoding and decoding.

[0115] The present invention adopts a 4-pixel video processing architecture. The DP decoding module, DP encoding module, ARINC818 decoding module, video interaction module, video detection and selection module, video cache control module, and video overlay module all adopt a 4-pixel processing architecture. Its video processing capability is much greater than the single-pixel processing mechanism of the traditional architecture. Without increasing the pixel clock, the video processing bandwidth is increased by four times. It solves the timing problems such as the difficulty of FPGA internal wiring and the shortage of clock resources caused by increasing the video clock.

[0116] The present invention has great advantages in the field of intelligent display. The traditional video processing module adopts DVI interface. With the improvement of GPU performance, its interface has become a higher-speed DP interface. The traditional architecture can no longer be used. The present invention has a 3-way DP decoding module, which can decode 3 high-performance SOCs or GPUs at the same time. It can not only meet the demand for high-performance GPU graphics display, but also solve the ultra-clear video display demand of high-performance computing SOC in intelligent display.

[0117] The processing architecture designed by the present invention uses the ARINC818 decoding module to directly decode and output the video data in Native format. Compared with the traditional architecture that uses frame buffer to realize the decoding cache and conversion of ARINC818 external video, this method has the characteristics of low-latency video processing, and the delay is much lower than the refresh time of a frame of video. In addition, the video synchronization and superposition method designed by the present invention has the ability of four-pixel video processing synchronization and superposition, and realizes high-bandwidth video synchronization and superposition.

[0118] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A video processing system for an airborne intelligent display, characterized in that: The system is implemented based on an FPGA processing framework, and the FPGA includes a DP decoding module, an ARINC818 decoding module, a TPG module, a video detection and selection module, a video interaction module, a video cache control module, a video overlay module, a DP encoding module, a memory control module, a Flash controller, a central processing module, a bus interaction processing module, a UART module and a key module; The DP decoding module, the video interaction module and the video cache control module are connected in sequence, the ARINC818 decoding module and the TPG module are connected to the video detection and selection module, the video detection and selection module and the video cache control module are both connected to the video overlay module, the video overlay module is connected to the DP decoding module, the bus interaction processing module is respectively connected to the UART module, the key module, the video detection and selection module, the video interaction module, the central processing module, the Flash controller and the video overlay module, and the video cache control module is also connected to the memory control module; The DP decoding module is provided with multiple modules for converting multiple serial DP data signals into video data signals in 4-pixel mode for output; the video interaction module is used to select one of the 4-pixel mode video data signals output by the DP decoding module and transmit it to the video buffer control module, and the video buffer control module is used for video frame storage processing control; the ARINC818 decoding module is used to decode the ARINC818 video signal and convert it into a video signal in Native format, the TPG module is used to generate a video-free signal, and the video detection and selection module is used to detect whether the video signal output by the ARINC818 decoding module exists. If not, the video detection and selection module selects the video-free signal for output display. If it exists, the video detection and selection module selects the Na tive format video signal for output and display; the video overlay module is used to overlay the DP video signal after the frame storage of the video cache control module with the video signal output by the video detection and selection module; the DP encoding module is used to convert the 4-pixel mode video data signal after the overlay processing into a serial DP data signal output; the Flash controller is used to control the reading and writing of the external Flash chip, the memory control module is used to control the access to the external memory, the central processing module is used for FPGA internal system scheduling and configuration of each module, the bus interaction processing module is used for bus interconnection of each module, the UART module is used for communication between the FPGA processing framework and other processors, and the key module is used to select one of the decoded DP video signals to enter the frame storage.

2. The video processing system of the airborne intelligent display according to claim 1, characterized in that: There are three DP decoding modules, which are respectively connected to the graphics board, the backup graphics board and the intelligent computing board through the DP receiving port. The graphics board is used to draw pictures to generate the onboard display screen and output video data through the DP video interface. The backup graphics board is used to back up the graphics board. The intelligent computing board is used for AI calculation and displays the image processing results through the DP video interface.

3. The video processing system of the airborne intelligent display according to claim 1, characterized in that: Each of the DP decoding modules includes a first vid_phy module and a dp_rx_subsystem module. The first vid_phy module operates in 5.4 Gbps mode and is used to convert the serial 4-lane DP signal into 4 groups of parallel DP data signals; the dp_rx_subsystem module has a DP protocol parsing function and is used to convert the 4 groups of parallel DP data signals into 4-pixel mode video data signals for output.

4. The video processing system of the airborne intelligent display according to claim 1, characterized in that: The DP encoding module includes a second vid_phy module and a dp_tx_subsystem module. The dp_tx_subsystem module has a DP protocol encoding function and is used to convert a 4-pixel mode video data signal into 4 groups of parallel DP data signals. The second vid_phy module operates in a 5.4 Gbps mode and is used to convert 4 groups of parallel DP data signals into serial 4lane DP signal conversion.

5. The video processing system of the airborne intelligent display according to claim 4, characterized in that: The video overlay module includes an AXISData FIFO module, a read enable controller module and an overlay calculation module. The read enable controller module is used to generate a read enable control signal rd_en of the AXISData FIFO module. The AXISData FIFO module is used to receive the 4-pixel AXIS data output by the video cache control module, and output 4-pixel RGB data according to whether the rd_en signal is valid. The overlay calculation module is used to perform overlay calculation on the video signal in Native format and the 4-pixel RGB data and generate a video signal in Native format.

6. The video processing system of the airborne intelligent display according to claim 1, characterized in that: The video signal in the Native format includes a VS signal, a HS signal, a DE signal and a Data signal, wherein the VS signal is a video frame synchronization signal, the HS signal is a video line synchronization signal, the DE signal is a video valid signal, and the Data signal is a 4-pixel video data signal.

7. The video processing system of the airborne intelligent display according to claim 1, characterized in that: The link rates supported by the ARINC818 decoding module include 1.0625 Gbps, 2.125 Gbps, 4.25 Gbps, 8.5 Gbps and 10 Gbps, and the link quantity transmission supported by the ARINC818 decoding module includes single-link transmission, dual-link transmission and quad-link transmission.

8. The video processing system of the airborne intelligent display according to claim 1, characterized in that: The memory is configured as a DDR memory for implementing a frame buffer of a video.

9. A video processing method for an airborne intelligent display, characterized in that: Based on the video processing system according to any one of claims 1 to 8, the method comprises: The DP decoding module decodes multiple DP video signals from the onboard display; The external key information is transmitted to the MicroBlaze module, and then the MicroBlaze module controls the video interaction module to select one of the DP video signal outputs; The output DP video signal is cached into the memory through the video cache control module; Another external video signal is decoded by the ARINC818 decoding module and decoded into a video signal in Native format; The video overlay module controls the synchronization of the cached video signal and the video signal in the Native format. After the synchronization, the video signal is sent to the DP encoding module for encoding and output after the video overlay calculation.

10. The video processing method of the airborne intelligent display according to claim 9, characterized in that: The video overlay module controls the synchronization of the cached video signal and the video signal in the Native format, including: Step 1: After the system is powered on, the MicroBlaze module determines whether the arinc818_valid signal and the vid_valid signal output by the ARINC818 decoding module are valid. The arinc818_valid signal represents that the ARINC818 data is valid, and the vid_valid signal represents that the decoded video signal is valid. If the arinc818_valid signal and the vid_valid signal are valid, jump to step 2; if not, jump to step 1. Step 2: The MicroBlaze module configures the video cache control module to start working, and generates a vdma_config_done signal to transmit to the read enable controller module. The video cache control module starts to read video data from the memory and sends it to the AXISData FIFO module, and then jumps to step 3; Step 3: The read enable controller module determines whether the vdma_config_done signal is valid, if it is valid, jump to step 4, if it is invalid, jump to step 3; Step 4: Determine whether the fifo_data_val id and fifo_data_tuser signals output by the AXISData FIFO module are valid, if valid, jump to step 5, otherwise, keep at step 4; Step 5: Wait for the decoded video frame count signal v_cnt to be equal to 5. If it is equal to 5, jump to step 6, otherwise, keep at step 5; Step 6: Turn on enable rd_en, whose value is the value of de, and jump to step 7: Step 7: Determine whether the arinc818_valid signal and the vid_valid signal output by the ARINC818 decoding module are valid. If they are valid, keep in step 7; otherwise, proceed to step 8; Step 8: Enable no video input signal and go to step 1.

Images