A multi-channel video processing mainboard and a multi-channel video processing method

Abstract

The application discloses a multi-channel video processing mainboard and a multi-channel video processing method. The mainboard comprises a central processor chip, a graphics processor chip and a field programmable gate array chip. The central processor chip is used for executing a top-level application program and operating system management; the graphics processor chip is connected with the central processor chip and is used for executing a graphics rendering task; the field programmable gate array chip is connected with the central processor chip and the graphics processor chip and is used for converging, processing and superimposing multi-channel heterogeneous video input signals and outputting processed video signals. The application solves the technical problem that the prior art is strongly dependent on foreign components and is difficult to realize independent controllability in realizing high-performance and low-delay multi-channel video processing by constructing a heterogeneous computing architecture in which a domestic central processor, a graphics processor and a field programmable gate array chip work cooperatively.

Description

Technical Field

[0001] This application relates to the field of video signal processing technology, and in particular to a multi-channel video processing motherboard and a multi-channel video processing method. Background Technology

[0002] In high-performance display and control systems such as airborne and vehicle-mounted systems, real-time acquisition, processing, and fusion display of multiple heterogeneous video signals are crucial for ensuring system functionality. Current technology faces a fundamental contradiction: on the one hand, the ever-increasing demand for high-definition video streams and complex graphics overlay requires processing platforms with extremely high data throughput and extremely low signal processing latency; on the other hand, in critical fields such as aviation and defense, the requirement for independent control of core processing platforms has reached unprecedented levels. Traditional architectures based on general-purpose processors (CPUs) and graphics processing units (GPUs), while capable of handling complex applications, easily become performance bottlenecks when dealing with multi-channel parallel, low-latency video streams. While heterogeneous architectures incorporating field-programmable gate arrays (FPGAs) can solve performance issues, their core components, especially high-performance CPU, GPU, and FPGA chips, have long been dominated by foreign manufacturers, posing potential risks to information security in critical areas. Therefore, the industry urgently needs a solution that can simultaneously meet the dual goals of high-performance processing and complete independent control. Summary of the Invention

[0003] The purpose of this application is to provide a multi-channel video processing motherboard and method, which aims to solve the technical problem that existing technologies rely heavily on foreign components and are difficult to achieve independent control when realizing high-performance, low-latency multi-channel video processing.

[0004] In a first aspect, this application provides a multi-channel video processing motherboard, comprising: a central processing unit (CPU) chip for executing top-level applications and operating system management; a graphics processing unit (GPU) chip communicatively connected to the CPU chip for executing graphics rendering tasks; and a field-programmable gate array (FPGA) chip communicatively connected to both the CPU chip and the GPU chip for converging, processing, and superimposing multiple heterogeneous video input signals, and outputting the processed and superimposed video signal; the FPGA chip is further configured to evaluate the regional complexity of the input video, extract region of interest (ROI) parameters, and feed them back to the CPU chip; the CPU chip is further configured to dynamically generate rendering adjustment instructions and fusion resource allocation instructions based on the ROI parameters, so as to control the rendering detail level of the GPU chip and the local superimposition strategy of the FPGA chip, respectively.

[0005] Optionally, the process by which the field-programmable gate array chip evaluates the regional complexity of the input video, extracts parameters of the region of interest, and feeds them back to the central processing unit chip includes: The field-programmable gate array chip performs inter-frame pixel gradient calculation on the input video according to macroblocks of a preset size, and obtains the motion activity index of each macroblock; The region of interest is extracted based on the motion activity index and spatial high-frequency components, and parameters of the region of interest containing target coordinates and complexity weights are generated. The parameters of the region of interest are sent as a hardware interrupt signal to the central processing unit chip via the peripheral component interconnection expansion channel.

[0006] Optionally, the process by which the central processing unit chip dynamically generates rendering adjustment instructions and fusion resource allocation instructions based on the region of interest parameters includes: The real-time operating system embedded in the central processing unit chip captures the parameters of the region of interest through interrupt service routines; The complexity weights are input into a built-in global resource arbitration algorithm to calculate the geometric detail level allocated to the graphics processor chip. The fusion resource allocation instruction is generated based on the target coordinates to instruct the field-programmable gate array chip to enable multi-level buffering and floating-point mixed logic in the region corresponding to the target coordinates, and to enable transparent keying logic in other regions.

[0007] Optionally, the field-programmable gate array chip is also used to monitor the validity status of each signal in the multiple heterogeneous video input signals; and, in response to the power management instructions issued by the central processing unit chip, to control the power supply of the video receiving circuit or video driving circuit corresponding to the invalid signal, so as to realize dynamic power management.

[0008] Optionally, the field-programmable gate array (FPGA) chip includes a static logic region and at least one reconfigurable region; the central processing unit (CPU) chip is further configured to load a partial bitstream file corresponding to a target functional mode from a memory according to a mode switching instruction of a top-level application, and write the partial bitstream file into the reconfigurable region through a bus interface, so as to dynamically change the video processing logic function of the FPGA chip without interrupting the operation of the static logic region.

[0009] Optionally, the motherboard is also configured with a cross-chip closed-loop feedback mechanism based on video content. Under this mechanism, the computing resource allocation between the field-programmable gate array chip and the graphics processor chip is reconstructed in the time domain according to the region of interest parameters at a preset update frequency, and the frame rate of the video signal output by the graphics processor chip is controlled by the rendering adjustment instructions issued by the central processing unit chip.

[0010] Optionally, the central processing unit chip is a Phytium processor module, which includes a Phytium FT2000-4 processor, memory, electrically erasable programmable read-only memory, and a real-time clock; wherein the memory is DDR4 memory with a dual-channel design.

[0011] Optionally, the central processing unit chip is connected to the graphics processing unit chip via a first PCIe bus; the graphics processing unit chip outputs a DP signal, which is converted into a DVI signal by a video conversion chip and then input to the field programmable gate array chip as one of the multiple heterogeneous video input signals.

[0012] Optionally, the multiple heterogeneous video input signals include at least two XGA signals, at least two LVDS signals, and at least one DP signal; the field-programmable gate array chip processes and superimposes the multiple heterogeneous video input signals to output at least two XGA signals and at least one LVDS signal.

[0013] Secondly, this application provides a multi-channel video processing method, comprising: a central processing unit (CPU) chip executing a top-level application and operating system management; a graphics processing unit (GPU) chip executing a graphics rendering task and sending the rendered graphics signal to a field-programmable gate array (FPGA); the FPGA chip aggregating the graphics signal and other multiple heterogeneous video input signals; the FPGA chip processing and superimposing the aggregated multiple signals to generate a target video signal; outputting the target video signal; the FPGA chip evaluating the regional complexity of the input video, extracting region-of-interest (ROI) parameters and feeding them back to the CPU chip; and the CPU chip dynamically generating rendering adjustment instructions and fusion resource allocation instructions based on the RIO parameters to control the rendering detail level of the GPU chip and the local overlay strategy of the FPGA chip, respectively.

[0014] Optionally, it further includes: monitoring the validity status of each of the multiple heterogeneous video input signals by the field-programmable gate array chip; and controlling the power supply of the video receiving circuit or video driving circuit corresponding to the invalid signal in response to the power management instructions issued by the central processing unit chip.

[0015] Optionally, it further includes: loading a partial bitstream file corresponding to a target functional mode from a memory by the central processing unit chip according to a mode switching instruction of a top-level application; and writing the partial bitstream file into a reconfigurable area of ​​the field-programmable gate array chip by the central processing unit chip through a bus interface to dynamically change the video processing logic function of the field-programmable gate array chip.

[0016] The beneficial effects of this application are as follows: By constructing a heterogeneous computing architecture in which three domestically produced CPU chips (Phytium FT2000-4), GPU chips (Phytium X100), and FPGA chips (Fudan Micro JFM7K325T) work collaboratively, this architecture organically combines the powerful general-purpose computing and system management capabilities of the CPU, the efficient graphics rendering capabilities of the GPU, and the parallel, low-latency, and high-bandwidth data processing capabilities of the FPGA. This architecture not only solves the performance bottleneck of real-time processing of multiple heterogeneous video streams, but more importantly, its core processing units, storage devices, operating system, and firmware all use domestic brands, building a complete, fully domestically produced technology system from the hardware layer to the software layer. Furthermore, this application introduces a cross-chip closed-loop feedback mechanism based on video content awareness, a dynamic power consumption management mechanism based on signal validity detection, and a hardware function dynamic loading mechanism based on partial reconstruction technology, realizing intelligent resource scheduling, refined energy consumption control, and the flexibility of "software-defined hardware." This design fundamentally eliminates dependence on foreign technology, achieving a balance between high performance and independent control, and providing a reliable, efficient, and flexible video processing platform for key fields such as airborne and vehicle-mounted applications. Attached Figure Description

[0017] Figure 1 This is a block diagram of a multi-channel video processing motherboard based on Phytium FT2000-4 provided in the embodiments of this application; Figure 2 This is a schematic block diagram of a multi-channel video processing motherboard based on Phytium FT2000-4 provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0020] In a specific implementation, this method constructs a heterogeneous computing platform consisting of a central processing unit (CPU) chip, a graphics processing unit (GPU) chip, and a field-programmable gate array (FPGA) chip. It also collaboratively designs the data paths and task allocation among these three components. This enables efficient and parallel aggregation, processing, overlay, and distribution of multiple heterogeneous video and data signals. This method addresses the technical problem of existing high-performance video processing platforms relying on foreign core chips, which poses risks to the supply chain and information security. It achieves the beneficial effect of ensuring the performance of complex video processing tasks while maintaining full-chain autonomy and control from hardware to software.

[0021] Reference Figure 1 , Figure 2 The schematic diagram of the motherboard in this embodiment of the application shown can be decomposed into the following core processing steps.

[0022] S100: The central processing unit chip executes top-level applications and manages the operating system.

[0023] In one embodiment, the central processing unit (CPU) chip and its associated circuitry together constitute a CPU module. The core of this module is a domestically produced Phytium FT2000-4 processor chip. This processor integrates four 64-bit high-performance FTC663 processor cores, employing an ARMv8 instruction set architecture. Its operating frequency is configurable to 2.2GHz, providing powerful general-purpose computing capabilities for upper-layer applications. To support the efficient operation of this processor, a memory system is configured on the motherboard. This memory system uses a dual-channel DDR4 design, supporting a maximum memory capacity of 32GB, and a memory clock frequency of up to 3200MHz. In this embodiment, DDR4 memory chips manufactured by Changxin Memory Technologies Co., Ltd. (CXMT) are selected, ensuring the complete domestic production of both the core computing unit and the core storage unit. The CPU module also integrates an electrically erasable programmable read-only memory (EEPROM) for storing non-volatile data such as key motherboard configuration information, serial numbers, and firmware boot parameters. Simultaneously, a real-time clock (RTC) circuit is integrated into this module to provide a precise time base for the system, maintaining timing even when the motherboard is powered off via a backup battery. This central processing unit (CPU) chip is the core of the entire motherboard's management and task scheduling. It is responsible for running the domestically produced SylixOS real-time operating system, which provides a stable and reliable operating environment for the execution of complex upper-layer applications (such as human-computer interaction interfaces, task decision logic, data logging and communication management, etc.).

[0024] For example, in a vehicle-mounted panoramic imaging application scenario, the top-level application running on the central processing unit (CPU) chip needs to process control commands from the vehicle's CAN bus, such as commands from the user to switch display modes via the central control screen. Assuming the user issues a command to "switch to a 360-degree overhead view," this command is transmitted to the motherboard via the CAN bus and received and parsed by the CPU chip's communication interface. The operating system then dispatches this command to the top-level application. Upon receiving the command, the application executes corresponding logical processing, which may involve multiple computational tasks. First, the application needs to send rendering commands to the graphics processing unit (GPU), requesting it to generate graphical elements such as icons and text for the interface display. Second, it needs to send control commands through the communication interface with the field-programmable gate array (FPGA) chip, instructing the FPGA to change its internal video stitching and fusion algorithm parameters to generate an overhead view image. For example, the CPU chip might write a control word (e.g., 0x00000001) to a specified register address (e.g., 0x40001000) on the FPGA via the PCIe bus. This control word "0x00000001" is predefined by the FPGA's internal logic as "top-down mode." Simultaneously, the CPU chip also needs to manage the onboard storage system. For instance, it might read pre-stored vehicle model data from a 128GB onboard solid-state storage chip and send this data to the GPU to overlay a virtual vehicle model onto the top-down image, thereby enhancing the display effect. Throughout this process, the CPU chip continuously queries the status of various hardware units, schedules system resources, and ensures that all tasks are executed correctly according to the predetermined logic and timing, demonstrating its core role as the system's "brain." The execution time of this series of operations is strictly controlled at the millisecond level to ensure smooth user interaction.

[0025] S200: The graphics processor chip performs the graphics rendering task and sends the rendered graphics signals to the field programmable gate array chip.

[0026] In one embodiment, the graphics processing unit (GPU) chip is the X100 chip from Phytium Technology, a domestic Chinese company. This chip integrates a low-power GPU core with a clock speed of 400MHz, designed specifically for graphics acceleration and display processing. To achieve high-speed data exchange between the CPU and GPU, the motherboard features a dedicated PCIe (Peripheral Component Interconnect Express) bus channel. Specifically, a PCIe x8 signal from the Phytium FT2000-4 processor is directly connected to the X100 chip. The PCIe x8 configuration means that this channel has 8 data pairs, providing a theoretical bandwidth of tens of GB / s, sufficient to meet the real-time transmission requirements of the large amounts of vertex, texture, and instruction data needed for high-resolution graphics rendering. When upper-layer applications need to display complex graphical user interfaces (GUIs), dynamic charts, or 3D models, the CPU sends the raw data and rendering instructions of these graphics to the X100 chip via the PCIe x8 bus. Upon receiving this data and instructions, the X100 chip utilizes its internal dedicated graphics processing unit for hardware-accelerated rendering, generating digitized image frame data. The rendered image is output as a digital video stream through one of the DisplayPort (DP) interfaces of the X100 chip. To ensure interface compatibility with subsequent FPGA chips, this DP signal is not output directly but is first fed into a video conversion chip (such as the CS5262). This video conversion chip converts the DP signal format to a Digital Video Interface (DVI) signal format. Finally, this DVI signal, carrying the graphical interface meticulously rendered by the GPU, is input into the Field Programmable Gate Array (FPGA) chip as an important video source, awaiting further processing and overlay with other video signals.

[0027] For example, in an airborne display and control system application, the pilot needs to see a navigation map on the main display, overlaid with the aircraft's real-time flight parameters, such as heading, speed, and altitude. This task is performed by the graphics processing unit chip X100. The central processing unit chip FT2000-4 first packages and sends vector map data obtained from the navigation computer, and real-time flight parameters (e.g., heading H=270 degrees, speed V=500 knots, altitude A=30000 ft) obtained from the flight data computer to the X100 via the PCIe x8 bus. After receiving this data, the X100's internal GPU core begins executing the rendering pipeline. First, the vertex shader processes the vector map data, converting it into geometric primitives in screen coordinates. Then, the rasterization unit converts these primitives into pixel fragments. Simultaneously, another set of rendering instructions drives the GPU to generate characters and symbols used to display the flight parameters. For example, the number "270" is rendered as pixelated text. The texture unit may load pre-stored aircraft icon textures. Finally, the pixel shader blends and colors all layers, including the map background, parameter text, and airplane icons, to generate a complete 32-bit color depth image with a resolution of 1920x1080. This frame of image data is sent as a pixel stream to the X100's DP output interface. Assuming the DP interface is configured to operate at a refresh rate of 60Hz, 60 such frames are generated and output per second. This DP signal stream enters a video conversion chip (such as Analog Devices' ADV7625), which parses the DP packets, extracts the video timing signals (pixel clock, horizontal sync, vertical sync) and parallel RGB pixel data, and then re-encodes them into a TMDS (Transition Minimized Differential Signaling) differential signal conforming to the DVI standard. Ultimately, this high-quality DVI signal, as a dynamic and information-rich graphical interface layer, is sent to the FPGA, providing the core overlay content for subsequent video fusion.

[0028] S300: The graphics signal and other multiple heterogeneous video input signals are aggregated by a field-programmable gate array chip.

[0029] In one embodiment, the field-programmable gate array (FPGA) chip is the JFM7K325T chip from Fudan Microelectronics, a domestic manufacturer. This chip belongs to the Kintex-7 series and is renowned for its abundant logic resources, high-speed transceivers (SERDES), and internal memory resources, making it an ideal choice for processing high-bandwidth, multi-channel parallel signals. This FPGA chip serves as the data aggregation and processing center for the entire motherboard, responsible for receiving and uniformly processing video and data signals from different sources and formats. The signal aggregation process is as follows: First, as described in S200, the DVI signal rendered and converted by the GPU is connected to a high-speed input interface of the FPGA. Second, the motherboard is designed with two XGA (Extended Graphics Array) signal input interfaces and two LVDS (Low-Voltage Differential Signaling) signal input interfaces to receive video signals from external devices (such as radar, infrared cameras, external computers, etc.). These analog XGA signals and differential LVDS signals first pass through a dedicated video receiving and decoding chip. The video receiver chip is responsible for conditioning and driving the signal, while the video decoder chip is responsible for converting analog signals or specific digital formats (such as LVDS) into a unified parallel TTL (Transistor-Transistor Logic) digital video format that is easy for the FPGA to process. This format typically includes parallel R / G / B data buses as well as pixel clock, line synchronization (HSYNC), and field synchronization (VSYNC) signals. These converted TTL video streams are directly fed into the FPGA's general-purpose I / O pins. In addition to video signals, the FPGA is also responsible for aggregating other types of data signals. For example, the motherboard has six ARINC429 bus signal input interfaces. ARINC429 is a serial data bus widely used in avionics equipment. These six signals first pass through a dedicated ARINC429 receiver chip, which converts the bipolar RZ code signals on the bus into standard logic levels before sending them to the FPGA. The FPGA's internal logic resources are configured to implement six parallel ARINC429 protocol decoding cores for parsing the bus data.

[0030] For example, assuming a specific application scenario, the FPGA needs to process the following five video and data inputs simultaneously: DVI input (from GPU): One 1920x1080@60Hz DVI signal, carrying the GUI interface. The FPGA's internal DVI receiving module (usually implemented using SERDES resources) decodes and converts the TMDS signal from serial to parallel, recovering 24-bit RGB data and timing signals. XGA input 1 (from radar): One 1024x768@60Hz XGA signal, displaying the radar scan image. This analog signal is first received by a video receiving chip (such as Chengdu Zhenxin's GM7145) and converted into an analog RGB signal, then sent to a video decoding chip (such as Chengdu Zhenxin's GM7002), which converts the analog RGB signal into a parallel TTL digital video signal (including data and horizontal and vertical synchronization timing), and finally sent to the FPGA. XGA input 2 (from backup computer): Another 1024x768@60Hz XGA signal, serving as a backup display source. The processing method is the same as above, using the cascaded solution of the video receiver chip and video decoder chip. LVDS Input 1 (from the infrared camera): One 800x600@60Hz LVDS signal, transmitting infrared thermal imaging images. This signal is processed by an LVDS receiver chip (such as TI's DS90CF384A), converting the LVDS differential pairs into parallel TTL signals, which are then sent to the FPGA. LVDS Input 2 (from the visible light camera): Another 1280x720@60Hz LVDS signal. The processing method is the same as above. ARINC429 Inputs: Six ARINC429 buses, one of which transmits GPS location data at a rate of 100kbps. The data word format is: tag=101, SDI=01, data=..., SSM=00, parity bit=1. The corresponding decoding core inside the FPGA continuously monitors the bus. When a valid 32-bit data word is detected, it captures, verifies, and extracts the data field from it, storing it in the internal FIFO (First-In, First-Out) buffer, waiting for the CPU to read it through the PCIe bus.

[0031] In this way, the FPGA digitizes all these signals from different sources and in different formats and incorporates them into its internal unified clock domain and data processing pipeline, completing the key step of data aggregation and preparing for subsequent processing and overlay.

[0032] S400: A field-programmable gate array (FPGA) chip processes and superimposes the aggregated multiple signals to generate a target video signal.

[0033] After all input signals are converged and unified into the data format within the FPGA, the FPGA utilizes its powerful parallel processing capabilities to process and overlay these data streams in real time. This process is implemented entirely in hardware logic, ensuring extremely low processing latency. The logic resources within the FPGA are configured into a complex, multi-layered video processing pipeline. This pipeline may include the following core functional modules: a video format and timing unification module. Since the converged video signals may have different resolutions and refresh rates, this module first unifies all input video streams to the resolution and timing of the target output video through algorithms such as scaling and frame rate conversion. For example, it may stretch an 800x600 video to 1024x768, or synchronize asynchronous video sources through frame buffers. The video overlay and fusion module is the core of video processing. Based on instructions from the central processing unit (CPU) chip, this module overlays different video layers according to a preset layout and transparency. For example, it can use the GPU's GUI interface (DVI input) as the top layer, setting specific colors (such as pure black) to transparent (Color Keying), and then overlaying it onto video from an infrared camera (LVDS input). Alternatively, it can implement a picture-in-picture function, scaling down one XGA video stream and placing it in a corner of another video stream. More complex processing can include alpha blending, which uses an independent alpha channel value to determine the blending ratio of pixels in two video layers, achieving a semi-transparent overlay effect. The data processing and OSD generation module, in addition to video overlay, is also responsible for visualizing non-video data (such as parsed ARINC429 data) and overlaying it onto the video frame. The FPGA can be configured with a character generator or graphics rendering engine. Based on flight parameters received from the ARINC429 decoding core, it generates pixel arrays of corresponding text or symbols in real time; this is known as On-Screen Display (OSD). For example, GPS data can be displayed as the string "LAT: N 39°54' LONG: E 116°23'" and overlaid on a specified position on the screen. Regarding the interface and communication logic module, the FPGA also implements a PCIe x4 interface IP core for high-speed communication with the FT2000-4 central processing unit chip. The CPU can use this interface to configure the operating parameters of all processing modules within the FPGA (such as video layout, overlay method, OSD content, etc.) and read the status information or data processed by the FPGA (such as parsed ARINC429 data packets).In addition, the FPGA utilizes its general-purpose GPIO (General-Purpose Input / Output) pins to implement four discrete signal inputs / outputs via optocouplers, used for isolation and communication with external switching devices. Simultaneously, the FPGA internally instantiates nine UART (Universal Asynchronous Receiver / Transmitter) IP cores. The serial output signals of these UARTs are converted into nine RS422 serial interfaces after passing through an RS422 level converter chip, used to connect to external devices requiring long-distance, high-reliability serial communication.

[0034] For example, continuing the above scenario, the FPGA's processing flow is as follows: The CPU sends a command through the PCIe interface, requesting the FPGA to execute a task of "fusion of infrared and GUI, and display of GPS location". After receiving the command, the FPGA's internal control logic configures the video processing pipeline: Input selection: LVDS input 1 (infrared camera) is selected as the main background video, and DVI input (GPU's GUI) is selected as the overlay layer; Format unification: The infrared video (800x600) is scaled up in real time to the target output resolution of 1024x768 by a hardware scaler; Data processing: The OSD generation module reads the latest GPS data from the FIFO of the ARINC429 decoding core and converts it into a pixel string; Overlay processing: The video overlay module processes each pixel. For each pixel on the screen, it first reads the scaled infrared video pixel value. Then, it checks the GUI video pixel value at the same location. If the GUI pixel value is a preset transparent color (e.g., RGB=0x000000), the final output pixel is the infrared pixel. If the GUI pixel value is opaque, the GUI pixel value is output. Meanwhile, for the area reserved at the top of the screen for OSD, the OSD generation module forcibly overwrites the original video pixels with the generated GPS string pixels. Output generation: After the above processing and overlay, a final target video signal with a resolution of 1024x768, integrating infrared images, a graphical interface, and flight parameters, is generated.

[0035] S500: Output the target video signal.

[0036] After complex and ordered processing and superposition within the FPGA, the final target video signal needs to be output to a display device or other external system. The motherboard in this embodiment is designed with multiple video output interfaces to adapt to different application requirements. Specifically, the parallel TTL digital video stream output from the FPGA's video processing pipeline is distributed to different output channels. Two channels are sent to the video encoding chip and the video driver chip, which work together to convert the TTL signal into a standard analog XGA signal format and provide sufficient driving capability, ultimately outputting two independent XGA signals through the motherboard's connector. The other TTL signal is sent to another video encoding chip, which encodes it into an LVDS signal format and outputs an LVDS signal through the connector. This multi-format, multi-channel output capability allows the motherboard to simultaneously drive multiple displays with different interfaces or distribute video signals to different downstream devices. In addition to the video signal, other data signals processed by the FPGA are also output synchronously. For example, the 6 channels of ARINC429 data processed internally by the FPGA can be re-encoded and enhanced by a dedicated ARINC429 driver chip to output 6 channels of ARINC429 bus signals to achieve data forwarding or transmission.

[0037] For example, the 1024x768@60Hz fused video signal generated in the FPGA has its parallel 24-bit RGB data and timing signals copied into three copies. The first copy is sent to a video encoding chip (such as Chengdu Zhenxin's GM7123C), which is responsible for converting the TTL digital signal output by the FPGA into an analog RGB signal. Subsequently, this analog RGB signal is sent to a video driver chip (such as Chengdu Zhenxin's GM7148), which buffers and drives the signal, outputting a standard XGA signal to drive the driver's side display. The second copy is sent to another set of functionally identical circuits (composed of the video encoding chip GM7123C and the video driver chip GM7148) to generate a second XGA signal to drive the passenger side display, achieving synchronous display for both the driver and passenger. The third data is sent to an LVDS transmitter chip (such as the MS90C385B from Ruimeng Technology). This chip serializes the parallel TTL data and encodes it into multiple pairs of low-voltage differential signals, forming an LVDS video output for connecting to a long-distance, high-interference-resistance airborne task display. Simultaneously, the central processing unit (CPU) chip can send instructions to the FPGA via the PCIe bus, requesting that received external sensor data (e.g., input via an RS422 interface) be processed and output via an ARINC429 bus. Upon receiving the instructions, the FPGA configures its internal data routing, sending the corresponding data stream to the ARINC429 transmitter IP core. This IP core packages the data according to the ARINC429 protocol standard and drives an external ARINC429 driver chip (such as Holt's HI-8597) to send the data onto the bus. Through this flexible configuration, the motherboard implements a full-function "input, processing, output" link for video and data signals.

[0038] In addition, to ensure the stable and reliable operation of the entire system, this motherboard is also designed with comprehensive auxiliary circuitry. Optionally, the motherboard includes a programmable logic chip, specifically a CPLD (Complex Programmable Logic Device) chip. This CPLD acts as the "manager" of the entire motherboard. It is responsible for power-on sequence control, ensuring that core chips such as the CPU, GPU, and FPGA are powered on and reset in the correct order. It is also responsible for signal level conversion, such as converting the CPU's 1.8V logic level UART signal to a 3.3V level that the CPLD can process, and then having other chips convert it to RS232 levels. Simultaneously, it monitors some critical status signals, such as the GPU's "card in place" signal, and polls the voltage and temperature sensors on the board to achieve low-level monitoring of the motherboard's health status.

[0039] Optionally, the motherboard includes a clock circuit, and the synchronous operation of the entire motherboard relies on a precise and stable clock system. This embodiment uses the GMM392 clock generator chip from the domestic company Zhenxin Technology. This chip uses a 25MHz crystal oscillator as a reference input and generates various frequency clock signals required on the motherboard through its internal phase-locked loop (PLL) and frequency divider, such as providing a 100MHz reference clock for the PCIe bus and a 125MHz clock for the network physical layer chips. This chip can ensure extremely low jitter and offset between the output clock signals, providing a fundamental guarantee for the stable transmission of high-speed digital signals.

[0040] Optionally, the motherboard includes a power supply circuit. A complex power supply circuit is designed to power the numerous chips on the motherboard with different voltage requirements. An external 5V main power supply is input via a CPCI connector. This 5V power supply is converted into various core operating voltages, such as 3.3V, 1.8V, 1.2V, and 1.0V, through multiple DC-DC switching power supply modules and low-dropout linear regulators (LDOs), to power different units such as the FPGA, CPU, GPU, and memory. All selected power chips, such as BL8033 and AST4644CB, are domestic brands, ensuring the independent controllability of the power supply system.

[0041] S600: The field-programmable gate array (FPGA) chip performs regional complexity assessment on the input video, extracts the region of interest (ROI) parameters, and feeds them back to the central processing unit (CPU) chip; based on the ROI parameters, the CPU chip dynamically generates rendering adjustment instructions and fusion resource allocation instructions to control the rendering detail level of the graphics processing unit (GPU) chip and the local overlay strategy of the FPGA chip, respectively.

[0042] S610: The field-programmable gate array chip performs inter-frame pixel gradient calculation on the input video according to macroblocks of a preset size, and obtains the motion activity index of each macroblock.

[0043] The method includes employing a complexity assessment module for video content perception. The complexity assessment module is instantiated internally within the field-programmable gate array (FPGA) chip. Upon receiving a low-voltage differential signal video stream from an external infrared camera, the complexity assessment module divides it into multiple macroblocks of size 16×16 pixels. For each macroblock... The complexity evaluation module calculates the sum of the absolute values ​​of the brightness differences between corresponding pixels in the current frame and the previous frame using a hardware pixel pipeline to generate the motion activity index. .

[0044] For example, suppose the macroblock currently being processed In the Frame and the The luminance matrices of the frames are respectively and The absolute error and calculation performed by the complexity evaluation module are defined as follows: If the calculated activity level index is... If the value is 1205 (dimensionless), then this value is directly latched into an internal register and used as the data reference for subsequent region of interest determination.

[0045] S620: Extract the region of interest based on the motion activity index and the high-frequency spatial components, and generate the region of interest parameters including target coordinates and complexity weights.

[0046] The complexity evaluation module includes a spatial high-frequency component extraction unit and a built-in dual-threshold decision unit. The spatial high-frequency component extraction unit employs a two-dimensional spatial convolution kernel architecture to perform feature extraction based on the Sobel edge extraction operator for each macroblock. This spatial high-frequency component extraction unit utilizes a preset... Horizontal and vertical convolution kernels, for the current frame The pixel brightness values ​​within the macroblock are subjected to a two-dimensional spatial convolution operation to generate corresponding horizontal and vertical gradient matrices. Based on the root-sum of the squares of the horizontal and vertical gradient matrices, the spatial high-frequency component extraction unit obtains the absolute edge intensity of the pixels within the macroblock and outputs the mean of the absolute edge intensity as the spatial high-frequency component index of the macroblock. For example, a size of After the macroblock of pixels is processed by the Sobel edge extraction operator, its output spatial high-frequency component index is... It is calculated to be 125 (dimensionless). Subsequently, the complexity assessment module performs a weighted fusion step to combine the motion activity index. With the aforementioned spatial high-frequency component index Integrating into a single comprehensive complexity index For example, the preset motion weight coefficient Set to 0.7 (dimensionless), the preset spatial weighting coefficient. It is set to 0.3 (dimensionless).

[0047] The dual-threshold decision maker will consider the overall complexity index. With a preset static background threshold and a high dynamic threshold A comparison is performed. When the comprehensive complexity index... Greater than the high dynamic threshold At that time, the corresponding macroblock is marked as the core block of the region of interest.

[0048] The complexity assessment module aggregates the spatial locations of all core blocks, calculates the minimum bounding rectangle that can enclose all core blocks, and defines the coordinates of the top-left and bottom-right corners of this minimum bounding rectangle as the target coordinates. Simultaneously, the complexity assessment module generates the complexity weights based on the normalized average value of the motion activity index within this region. .

[0049] For example, static background threshold Set to 500 (dimensionless), high dynamic threshold It was set to 1500 (dimensionless). During one computation cycle, multiple [items] were detected in the center area of ​​the screen. Macroblocks exceeding 1500. The target coordinates of the minimum bounding rectangle calculated by the complexity evaluation module are ( px, px, px, (px). Within this area The average value is 2100, obtained through a linear mapping function. After normalization, the generated complexity weights The value is 0.6 (dimensionless). The target coordinates and the complexity weight are physically packaged and merged into a 64-bit region of interest parameter data packet.

[0050] S630: The parameters of the region of interest are sent as a hardware interrupt signal to the central processing unit chip via the peripheral component interconnection extension channel.

[0051] Upon acquiring the region of interest parameter data packet, the Direct Memory Access Controller (DMemory Controller) of the Peripheral Component Interconnect (PCI) within the Field Programmable Gate Array (FPGA) chip triggers a specific hardware interrupt request to the CPU chip. This interrupt request pulls a specific pin of the CPU chip low via the physical bus, forcing the CPU chip to suspend its current non-critical tasks. During the interrupt response, the DMemory Controller writes the region of interest parameter data packet into the pre-allocated main memory address space of the CPU chip.

[0052] For example, the peripheral component interconnection expansion interface triggers a message signal interrupt, with the interrupt vector number set to 0x1A. Upon receiving the interrupt vector, the central processing unit chip jumps to the corresponding kernel-mode interrupt service routine and reads the generated 64-bit region of interest parameter data packet from physical address 0x0000000080100000.

[0053] S640: The real-time operating system embedded in the central processing unit chip captures the parameters of the region of interest through an interrupt service routine, and inputs the complexity weight into a built-in global resource arbitration algorithm to calculate the geometric detail level allocated to the graphics processing unit chip.

[0054] After extracting the complexity weights, the central processing unit (CPU) chip inputs them into a global resource arbitration model maintained by the top-level application. This global resource arbitration model employs a negative correlation mapping function to adjust the resource quotas for non-core rendering tasks based on the complexity weights. This processing architecture can allocate more computing and bus bandwidth resources to the video overlay link based on the presence of highly dynamic targets in the external video input.

[0055] For example, the negative correlation mapping formula defined by the global resource arbitration model is as follows: Among them, the upper limit of the geometric detail level. Set to 5 (representing the highest polygon rendering precision), lower limit of the level. Set it to 1 (representing minimum precision). Substitute the complexity weight from the previous example. The calculation process is as follows The central processing unit chip generates a rendering adjustment instruction accordingly, forcing the graphics processing unit chip to reduce the rendering precision of the current 3D dashboard from level 5 to level 3, thereby freeing up approximately 40% of the rendering cycle time.

[0056] S650: Generate the fusion resource allocation instruction based on the target coordinates to instruct the field programmable gate array chip to enable multi-level buffering and floating-point mixed logic in the region corresponding to the target coordinates, and enable transparent keying logic in other regions.

[0057] The central processing unit (CPU) chip uses the acquired target coordinates to generate a fusion resource allocation instruction containing a defined coordinate range and strategy code, and sends it to the configuration register of the field-programmable gate array (FPGA) chip via the peripheral component interconnect channel. After parsing the instruction, the video overlay and fusion module of the FPGA chip implements spatially isolated heterogeneous processing logic during the line and field scan cycles of screen redrawing. Within the boundary defined by the target coordinates, the system allocates an independent on-chip block random access memory as a triple-buffered mechanism and calls the internal digital signal processing slice to perform alpha pixel blending based on floating-point operations. Outside the boundary, the system disconnects the data path of the digital signal processing slice and uses a pixel-by-pixel color value comparator that consumes only basic lookup table resources for transparent color keying.

[0058] For example, the strategy code of the fusion resource allocation instruction is set to 0x03 (representing region enhancement overlay). When scanning pixel coordinates In ( )and( When the range is within ), the digital signal processing slice executes the formula. , where the alpha value It can be finely set to 0.75. When a scanned pixel leaves this coordinate range (e.g., coordinates (100, 100)), the video overlay module only performs one logical judgment: if the foreground pixel... (Pure black transparent color) will output the background pixels. Otherwise, output the foreground pixels. This spatial isolation process reduces the power consumption of the field-programmable gate array chip by 1.5W, while ensuring absolutely no tearing and high smoothness in high dynamic range images.

[0059] In an optional embodiment, the motherboard also provides a dynamic power management mechanism based on signal validity detection.

[0060] The field-programmable gate array (FPGA) chip is also used to monitor the validity status of each signal in the multiple heterogeneous video input signals; and, in response to power management instructions issued by the central processing unit (CPU) chip, to control the power supply of the video receiving circuit or video driving circuit corresponding to the invalid signal, so as to achieve dynamic power management. Specifically, the FPGA internally configures dedicated synchronization signal monitoring logic for each video input channel (e.g., XGA input 1, LVDS input 2). This logic continuously monitors the periodicity of the horizontal synchronization (HSYNC) and vertical synchronization (VSYNC) signals of the corresponding channel. When the synchronization signal of a channel is continuously missing within a preset time window (e.g., 100 milliseconds, dimensionless), the corresponding bit in a status register inside the FPGA is set, indicating that the channel signal is invalid. The CPU chip periodically polls the status register through the PCIe bus, or the FPGA actively initiates an interrupt when the status changes. When the CPU chip detects that a video input is invalid and, based on the upper-level application logic, confirms that the video input does not need to be displayed in the current task mode, it sends a power management command to the CPLD chip, which acts as the system "manager," via a low-speed communication interface (such as the I2C bus). This command explicitly specifies the video channel that needs to be powered down. Upon receiving the command, the CPLD pulls down the power enable pin of the corresponding video receiver or decoder chip (such as the GM7145 and GM7002 chips used for XGA input), thereby cutting off its core operating power. Similarly, for video output channels, if the upper-level application indicates that a display does not need to output, the CPU can also instruct the CPLD to turn off the power to the corresponding video encoding and driver chip (such as the GM7123C).

[0061] For example, suppose the system is running a task requiring only a single infrared video display, and the external radar XGA signal source is physically disconnected. The FPGA's monitoring logic detects the disappearance of the HSYNC signal from XGA input channel 1 after 100 milliseconds and sets bit 0 of the power status register (address 0x40002000) to 1. The CPU reads this status change during the next poll (every 200 milliseconds) and checks the current application status, confirming that XGA input 1 is not required in this mode. Therefore, the CPU writes instruction 0x01 (representing power off channel 1) to the CPLD's (address 0x5A) command register via the I2C bus. After parsing the instruction, the CPLD changes the output level of one of its GPIO pins (connected to the EN pin of the GM7145 chip) from high to low, thus putting the video receiver chip into a low-power shutdown mode, saving approximately 300mW of system power. When the radar signal is reconnected, the FPGA detects the restoration of the synchronization signal, resets the status register, and the CPU reverses the above process, instructing the CPLD to re-enable the chip power supply, thus realizing fine-grained and automated control of the board's functions and power consumption.

[0062] In another alternative embodiment, the motherboard also provides a dynamic hardware function adjustment mechanism based on FPGA partial reconfiguration.

[0063] The field-programmable gate array (FPGA) chip includes a static logic region and at least one reconfigurable region. The central processing unit (CPU) chip is further configured to load a partial bitstream file corresponding to a target functional mode from a memory according to a mode switching instruction from a top-level application, and write the partial bitstream file into the reconfigurable region via a bus interface. This allows for dynamic modification of the FPGA's video processing logic function without interrupting the operation of the static logic region. In this architecture, the FPGA's logic resources are divided into two parts: the static region stores core logic closely related to stable system operation, such as the PCIe interface controller, DDR4 memory controller, and interface logic with the CPLD and clock circuits. This region is configured after power-on and remains unchanged during system operation. The reconfigurable region acts as a "logic slot" for loading different video processing pipelines according to real-time task requirements. Various hardware accelerator modules for specific functions are pre-stored in the form of encrypted "partial bitstream files" in onboard EEPROM or solid-state memory.

[0064] When a top-level application needs to switch operating modes, such as from "Wide Area Navigation Mode" to "Key Target Warning Mode," the application sends a mode switch request to the operating system. The operating system kernel responds to this request by performing the following sequence of operations: First, it locates and reads the partial bitstream file corresponding to the "Warning Mode" (e.g., target_tracker.pbit, 5MB in size) from memory. Then, the operating system loads this file into a reserved buffer in the CPU's DDR4 main memory. Next, the CPU sends an instruction via the PCIe bus to a dedicated "Partial Reconfiguration Controller" (PR Controller) IP core in the FPGA's static area, providing the physical address and size of the bitstream file in main memory. The PR controller then takes over the subsequent operations, retrieving the encrypted data from the partial bitstream file from the system's main memory using DMA (Direct Memory Access). To ensure hardware security during the configuration process, the field-programmable gate array chip integrates a dedicated hardware decryption engine between the partial reconfiguration controller and the internal configuration port (such as ICAP). Before the data stream reaches the configuration port, the hardware decryption engine automatically intercepts the extracted ciphertext state data and reads a preset configuration key from the physically isolated electronic fuse (eFUSE) array inside the field-programmable gate array chip. For example, the configuration key is fixed to a length of 256 bits. Using the configuration key, the hardware decryption engine performs Advanced Encryption Standard (AES) decryption on the intercepted ciphertext state data at line speed, and simultaneously performs Cyclic Redundancy Check (CRC) on the data blocks. After verifying successful decryption and verification, the hardware decryption engine writes the generated plaintext bitstream data to the designated reconfigurable area through the internal configuration port. If a decryption failure or CRC error is detected, the partial reconstruction controller immediately stops data transmission to block illegal logic loading. The entire reconstruction process can be completed within tens of milliseconds, during which the static area functions of the FPGA (such as communication with the CPU) are completely unaffected, achieving "hot-swappable" hardware functions.

[0065] For example, in "Wide Area Navigation Mode," the reconfigurable area loads a "Multi-channel Video Fusion and OSD Overlay Module" (fusionosd.pbit). This module's hardware pipeline is optimized for simultaneously processing two XGA and two LVDS video inputs, performing efficient alpha blending and scaling operations, and rendering the multi-channel ARINC429 data from the CPU into OSD information in real time, overlaying it onto the fused background video to provide the operator with comprehensive situational awareness. When the system switches to "Key Target Warning Mode," the CPU loads and configures a "High-Precision Target Tracking Module" (targettracker.pbit). This new module has a completely different hardware structure; it may contain a processing pipeline specifically designed for single-channel high-definition infrared video, internally implementing a Kalman filter-based position prediction algorithm and correlation-matching-based target locking logic. At this point, the FPGA's main task shifts from "multi-channel fusion" to "single-channel precision processing," concentrating computational resources on the precise tracking of a specific target. After calculating the tracking results (such as target coordinates and velocity vectors), the results are sent back to the CPU via the PCIe bus for decision-making. This "software-defined hardware" capability greatly expands the functional boundaries and task adaptability of the motherboard, maximizing the efficiency of hardware resource utilization.

[0066] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware.

[0067] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0068] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware.

[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-channel video processing motherboard, characterized in that, include: A central processing unit chip used to execute top-level applications and manage the operating system; A graphics processing unit chip, which is communicatively connected to the central processing unit chip, is used to perform graphics rendering tasks; as well as A field-programmable gate array (FPGA) chip is communicatively connected to the central processing unit (CPU) chip and the graphics processing unit (GPU) chip, respectively, for converging, processing and superimposing multiple heterogeneous video input signals, and outputting the processed and superimposed video signal; The field-programmable gate array chip is also used to evaluate the regional complexity of the input video, extract the parameters of the region of interest, and feed them back to the central processing unit chip; The central processing unit chip is also used to dynamically generate rendering adjustment instructions and fusion resource allocation instructions based on the region of interest parameters, so as to control the rendering detail level of the graphics processing unit chip and the local overlay strategy of the field programmable gate array chip, respectively.

2. The motherboard according to claim 1, characterized in that, The process by which the field-programmable gate array (FPGA) chip evaluates the regional complexity of the input video, extracts parameters of the region of interest, and feeds them back to the central processing unit (CPU) chip includes: The field-programmable gate array chip performs inter-frame pixel gradient calculation on the input video according to macroblocks of a preset size, and obtains the motion activity index of each macroblock; The region of interest is extracted based on the motion activity index and spatial high-frequency components, and parameters of the region of interest containing target coordinates and complexity weights are generated. The parameters of the region of interest are sent as a hardware interrupt signal to the central processing unit chip via the peripheral component interconnection expansion channel.

3. The motherboard according to claim 2, characterized in that, The process by which the central processing unit chip dynamically generates rendering adjustment instructions and fusion resource allocation instructions based on the region of interest parameters includes: The real-time operating system embedded in the central processing unit chip captures the parameters of the region of interest through interrupt service routines; The complexity weights are input into a built-in global resource arbitration algorithm to calculate the geometric detail level allocated to the graphics processor chip. The fusion resource allocation instruction is generated based on the target coordinates to instruct the field-programmable gate array chip to enable multi-level buffering and floating-point mixed logic in the region corresponding to the target coordinates, and to enable transparent keying logic in other regions.

4. The motherboard according to claim 1, characterized in that, The field-programmable gate array chip is also used to monitor the validity status of each signal in the multiple heterogeneous video input signals; and, in response to the power management instructions issued by the central processing unit chip, to control the power supply of the video receiving circuit or video driving circuit corresponding to the invalid signal, so as to realize dynamic power management.

5. The motherboard according to claim 1, characterized in that, The field-programmable gate array (FPGA) chip includes a static logic region and at least one reconfigurable region. The central processing unit (CPU) chip is also configured to load a partial bitstream file corresponding to a target function mode from a memory according to a mode switching instruction of a top-level application, and write the partial bitstream file into the reconfigurable region through a bus interface, so as to dynamically change the video processing logic function of the FPGA chip without interrupting the operation of the static logic region.

6. The motherboard according to claim 1, characterized in that, The motherboard is also equipped with a cross-chip closed-loop feedback mechanism based on video content. Under this mechanism, the computing resource allocation between the field-programmable gate array chip and the graphics processor chip is reconstructed in the time domain according to the region of interest parameters at a preset update frequency, and the frame rate of the video signal output by the graphics processor chip is controlled by the rendering adjustment instructions issued by the central processing unit chip.

7. The motherboard according to claim 1, characterized in that, The central processing unit chip is a Phytium processor module, which includes a Phytium FT2000-4 processor, memory, electrically erasable programmable read-only memory, and a real-time clock; wherein, the memory is DDR4 memory with a dual-channel design.

8. The motherboard according to claim 2, characterized in that, The central processing unit chip is connected to the graphics processing unit chip via a first PCIe bus; the graphics processing unit chip outputs a DP signal, which is converted into a DVI signal by a video conversion chip and then input to the field programmable gate array chip as one of the multiple heterogeneous video input signals.

9. The motherboard according to claim 2 or 3, characterized in that, The multi-channel heterogeneous video input signals include at least two XGA signals, at least two LVDS signals, and at least one DP signal; the field-programmable gate array chip processes and superimposes the multi-channel heterogeneous video input signals to output at least two XGA signals and at least one LVDS signal.

10. A multi-channel video processing method, characterized in that, include: A central processing unit chip executes top-level applications and manages the operating system. A graphics processing unit chip performs the graphics rendering task and sends the rendered graphics signals to a field-programmable gate array chip. The field-programmable gate array chip aggregates the graphics signal and other heterogeneous video input signals. The field-programmable gate array chip processes and superimposes the converged multiple signals to generate a target video signal; Output the target video signal; The field-programmable gate array chip performs regional complexity evaluation on the input video, extracts the parameters of the region of interest, and feeds them back to the central processing unit chip. as well as The central processing unit chip dynamically generates rendering adjustment instructions and fusion resource allocation instructions based on the parameters of the region of interest, so as to control the rendering detail level of the graphics processing unit chip and the local overlay strategy of the field programmable gate array chip, respectively.

Images