FPGA-based asynchronous quad monitor for broadcast-quality 4K or 8K

Abstract

The present invention provides an asynchronous four-split monitor for broadcasting based on FPGA, including: a four-way SDI / HDMI2.0 codec module for decoding FPGA signals and rearranging the decoded data; an input channel data cross selection module for input channel selection when a single-channel input is displayed in full screen; a four-way video scaling / deinterlacing module for scaling and adaptive deinterlacing of videos of independent channels; a four-way image enhancement / color management module for adjusting the image brightness, contrast, sharpness, and color temperature of independent channels and realizing 3D-LUT based on a hexahedral interpolation algorithm; a video overlay / splicing synchronization module; a memory module; a screen interface encoding output display module; and a bus control module for forming a video flow path for the above modules. The present invention can automatically adjust the phase difference of different input signals, receive high resolution, reduce equipment costs, and has great economic value and equipment benefits.

Description

Technical Field

[0001] The present invention relates to the technical field of monitors, and in particular to an asynchronous four-split monitor based on FPGA for broadcast-level 4K or 8K. Background Art

[0002] Monitors are important hardware equipment in the field of radio and television, and play an important role in color grading, production, and broadcasting. In addition to having various input and output interfaces, a good monitor also has powerful color management functions and various auxiliary tools, which can improve productivity in color grading, production, and broadcasting. However, traditional monitors cannot receive 4-channel 12GSDI signals at the same time, and due to bandwidth constraints, they cannot receive 4-channel signals at any resolution at the same time. In addition, due to the influence of the use environment, the 4-channel input signals cause phase differences due to routing through intermediate devices. Summary of the invention

[0003] In order to overcome the problems existing in the above-mentioned prior art, the present invention provides an asynchronous four-split monitor for broadcast-level 4K or 8K based on FPGA.

[0004] The technical solution of the present invention is:

[0005] FPGA-based asynchronous quad-split monitor for broadcast-quality 4K or 8K, including:

[0006] Four-channel SDI / HDMI2.0 codec module, used to decode FPGA signals and rearrange the decoded data;

[0007] Input channel data cross selection module, used for input channel selection when single channel input is displayed in full screen;

[0008] Four-channel video scaling / deinterlacing module, used for scaling and adaptive deinterlacing of independent channel videos;

[0009] Four-channel image enhancement / color management module, used to adjust the image brightness, contrast, sharpness, color temperature of independent channels and implement 3D-LUT based on hexahedral interpolation algorithm;

[0010] The video overlay / splicing synchronization module selects the synchronization clock as the synchronization reference clock of the memory module according to different synchronization output functions;

[0011] Memory module, used to synchronize the decoded signals of four-channel SDI / HDMI2.0 codec modules;

[0012] Screen interface (Lvds / V-by-one) encoding output display module; used to achieve four-way four-split synchronous display with different resolutions;

[0013] A bus control module, used to form a video flow path for the above modules;

[0014] The output end of the SDI / HDMI2.0 codec module is connected to the input channel data cross selection module, the output end of the input channel data cross selection module is connected to the four-way video scaling / de-interlacing module, the output end of the four-way video scaling / de-interlacing module is connected to the four-way image enhancement / color management module, the output end of the four-way image enhancement / color management module is connected to the video overlay / splicing synchronization module, and the output end of the video overlay / splicing synchronization module is connected to the screen interface encoding output display module;

[0015] The bus control module is connected with the four-way SDI / HDMI2.0 codec module, the input channel data cross selection module, the four-way video scaling / de-interlacing module, the four-way image enhancement / color management module, the video overlay / splicing synchronization module, the memory module, and the screen interface encoding output display module.

[0016] As a further technical solution of the present invention, the SDI / HDMI2.0 codec module decodes the high-speed SerDes signal of the FPGA into 12G SDI and HDMI2.0, and rearranges the decoded data according to the SMPTE protocol to obtain a four-channel YUV444 format signal for use as back-end video processing data.

[0017] As a further technical solution of the present invention, the video scaling / deinterlacing module adopts a temporal interpolation algorithm and a spatial interpolation algorithm according to the stillness and motion of the image. When the images in two adjacent fields are still, the temporal interpolation algorithm is adopted; when there are moving scenes in adjacent fields, the spatial interpolation algorithm is adopted.

[0018] As a further technical solution of the present invention, the video scaling / deinterlacing module simultaneously adopts a temporal interpolation algorithm and a spatial interpolation algorithm, and uses a motion compensation algorithm to calculate the movement of each pixel, obtain the motion vector of the object, and then interpolate along the direction of the motion trajectory.

[0019] As a further technical solution of the present invention, the video scaling / deinterlacing module divides the scaling technology into two levels, the front level, which makes a nonlinear correction to the spatial distance of the interpolation points according to the intrinsic correlation and geometric similarity of the image; the back level adjusts the grayscale weights of the surrounding pixels according to the gradient values ​​obtained by calculating the known pixels around the pixel to be interpolated, so that the grayscale value of the pixel after interpolation is close to the grayscale value corresponding to the pixel in the flat area.

[0020] As a further technical solution of the present invention, it also includes a synchronous phase-locking module, and the synchronous phase-locking module includes:

[0021] The time management module adjusts the pixel clock frequency of the output image to synchronize the input field signal and the output image field signal, namely, PIVS and POVS;

[0022] The frame rate conversion module controls the frequency of the pixel clock POCLK of the output image by comparing the phase difference between POVS and PIVS and adjusting the duty cycle of the PWM pulse output from the pulse signal Pofld pin of the input image to change the input voltage of the VCXO (voltage controlled oscillator).

[0023] Video timing generation module, when the duty cycle of the PWM pulse becomes larger or smaller, the output voltage of the low-pass filter becomes higher or lower, and the output frequency of the VCXO becomes higher or lower to control the phase and frequency of the field signal POVS of the output image.

[0024] As a further technical solution of the present invention, it also includes: a phase difference automatic adjustment module, which calculates the window sliding position according to the delay information of SDI decoded by FPGA, dynamically adjusts the window position and fills the cache.

[0025] The beneficial effects of the present invention are:

[0026] The present invention has the functions of 8K resolution front and rear two-stage multi-phase nonlinear scaling, 4-way asynchronous splicing display with arbitrary resolution and size, and automatic adjustment of the phase difference of 4-way input signals; it also supports 4-way asynchronous four-split display with arbitrary resolution and size, and can receive 8K signals and 4-way SDI up to 12G, and the monitor can automatically adjust the phase difference of different input signals. It can not only solve the phase difference problem caused by equipment routing in complex environments, but also solve the problem of multiple signal inputs at the same workstation, high-resolution reception, and high-quality presentation, thereby improving productivity, reducing equipment costs and energy consumption, and has great economic value and equipment benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a structural diagram of an asynchronous four-split monitor based on FPGA for broadcast-grade 4K or 8K proposed by the present invention;

[0028] Figure 2 It is a 4-channel asynchronous splicing display image of any resolution size according to an embodiment of the present invention;

[0029] Figure 3 is a block diagram of a synchronous phase-locking module according to an embodiment of the present invention;

[0030] Figure 4 This is a block diagram of a 4-channel 12G SDI input synchronous splicing 8K processing module according to an embodiment of the present invention;

[0031] Figure 5 It is a data arrangement format diagram of the SQD and 2SI modes in the SDI mode of an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with an embodiment and the accompanying drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, the SDI interface involved in the patent is only an embodiment and can also be replaced by other interfaces that can transmit video.

[0033] See also Figures 1 to 5 ,in Figure 1 This is a structural diagram of an asynchronous four-split monitor based on FPGA for broadcast-grade 4K or 8K proposed by the present invention; Figure 2 It is a 4-channel asynchronous splicing display image of any resolution size according to an embodiment of the present invention; Figure 3 is a block diagram of a GENLOCK synchronization module according to an embodiment of the present invention; Figure 4 This is a block diagram of a 4-channel 12G SDI input synchronous splicing 8K processing module according to an embodiment of the present invention; Figure 5 It is a data arrangement format diagram of the SQD and 2SI modes in the SDI mode of an embodiment of the present invention.

[0034] like Figure 1 As shown, the FPGA-based asynchronous quad split monitor for broadcast-grade 4K or 8K includes:

[0035] Four-channel SDI / HDMI2.0 codec module, used to decode FPGA signals and rearrange the decoded data;

[0036] Input channel data cross selection module, used for input channel selection when single channel input is displayed in full screen;

[0037] Four-channel video scaling / deinterlacing module, used for scaling and adaptive deinterlacing of independent channel videos;

[0038] Four-channel image enhancement / color management module, used to adjust the image brightness, contrast, sharpness, color temperature of independent channels and implement 3D-LUT based on hexahedral interpolation algorithm;

[0039] The video overlay / splicing synchronization module selects the synchronization clock as the synchronization reference clock of the memory module according to different synchronization output functions;

[0040] Memory module, used to synchronize the decoded signals of four-channel SDI / HDMI2.0 codec modules;

[0041] Screen interface (Lvds / V-by-one) encoding output display module; used to achieve four-way four-split synchronous display with different resolutions;

[0042] A bus control module, used to form a video flow path for the above modules;

[0043] The output end of the SDI / HDMI2.0 codec module is connected to the input channel data cross selection module, the output end of the input channel data cross selection module is connected to the four-way video scaling / de-interlacing module, the output end of the four-way video scaling / de-interlacing module is connected to the four-way image enhancement / color management module, the output end of the four-way image enhancement / color management module is connected to the video overlay / splicing synchronization module, and the output end of the video overlay / splicing synchronization module is connected to the screen interface encoding output display module;

[0044] The bus control module is connected with the four-way SDI / HDMI2.0 codec module, the input channel data cross selection module, the video scaling / de-interlacing module, the image enhancement / color management module, the video overlay / splicing synchronization module, the memory module, and the screen interface encoding output display module.

[0045] The present invention provides a broadcast-level 4K four-split monitor based on FPGA that can simultaneously display four different resolutions. Different modules form a video flow path through a bus control module. The present invention mainly uses FPGA as a main video processing chip, and externally connects four DDR3 chips for core functions to synchronize four signals with a maximum resolution of 4K. The signal feature code decoded by FPGA is used to automatically calculate the phase deviation of the four-channel signal to achieve signal synchronization, and the motion adaptive de-interlacing function of the four-channel signal.

[0046] The SDI / HDMI decoding module is a high-speed serial receiving module. The core function of this module is to decode 12G SDI and HDMI2.0 for the high-speed SerDes signal of FPGA, and rearrange the decoded data according to the SMPTE protocol to obtain a four-channel YUV444 format signal as the back-end video processing data. Compared with traditional external SDI and HDMI decoding chips, the SDI / HDMI decoding module has the advantages of saving hardware costs and smaller PCB board area.

[0047] The SDI encoding and decoding module includes 4 12G SDI decoding modules, and can combine and decode 4 12G SDIs to obtain a complete 8K resolution image; each 12G SDI decoding module is compatible with the 3G SDI decoding mode, where each 12G SDI decoding mode can decode 4 3G signals, and the 3G SDI decoding mode can decode 1 3G signal, and can also de-embed audio information, auxiliary information, etc.

[0048] The input channel data cross selection module is used for input channel selection when single-channel input is displayed in full screen, and can be used for data channel selection when the monitor displays single screen, two screens and four screens.

[0049] The image enhancement and color management module is used to adjust the image brightness, contrast, sharpness, color temperature of independent channels and the 3D-LUT function based on the hexahedral interpolation algorithm. Specifically, it converts the input nonlinear signal into a linear one, and then processes it through the 3DLUT color correction matrix to achieve color conversion management of various standards. The memory module synchronizes the signal with 4K resolution; the SDI module decodes a total of 64 channels of audio data, and can output any two channels.

[0050] The video overlay and splicing synchronization module selects PO1CLK or PO2CLK or PO3CLK or PO4CLK as the synchronization reference clock of IP1CLK, IP2CLK, IP3CLK and IP4CLK of DDR3 video cache according to different synchronization output functions to realize four-way four-split synchronization display with four different resolutions. The clock management module in the present invention has flexible input and output selection functions to meet the synchronization mechanism of complex situations.

[0051] The present invention realizes 8K signal input on one monitor, with a maximum of 4K60Hz input, and can display at any resolution at the same time, and all four channels have broadcast-level motion adaptive de-interlacing technology.

[0052] Broadcasting-level motion adaptive deinterlacing technology uses both temporal and spatial interpolation algorithms, that is, when the images in two adjacent fields are static, temporal interpolation is used to obtain accurate images, and when there are moving scenes in adjacent fields, spatial interpolation is used; or spatial and temporal interpolation is performed simultaneously, and motion compensation algorithms are used to calculate the movement of each pixel, obtain the motion vector of the object, and then interpolate along the direction of the motion trajectory. This module has the characteristics of low latency and good effect of converting interlaced signals to progressive signals.

[0053] In the embodiment of the present invention, the video scaling / de-interlacing module divides the scaling technology into two levels, the hierarchical multi-phase scaling module. Compared with the traditional scaling technology, this module divides the scaling technology into two levels, the front level performs nonlinear correction on the spatial distance of the interpolation point according to the intrinsic correlation and geometric similarity of the image, and the back level adjusts the grayscale weights of the surrounding pixels according to the gradient values ​​obtained by calculating the known pixels around the pixel to be interpolated, so that the grayscale value of the pixel after interpolation can be closer to the grayscale value corresponding to the pixel in the flatter area. Compared with the traditional method, this algorithm can save 2 DDR3s and has the characteristics of low hardware cost and good interpolation effect.

[0054] In an embodiment of the present invention, a synchronous phase locking module is further included, and the synchronous phase locking module includes:

[0055] The time management module adjusts the pixel clock frequency of the output image to synchronize the input field signal and the output image field signal, namely, PIVS and POVS;

[0056] The frame rate conversion module controls the frequency of the pixel clock POCLK of the output image by comparing the phase difference between POVS and PIVS and adjusting the duty cycle of the PWM pulse output from the pulse signal Pofld pin of the input image to change the input voltage of the VCXO (voltage controlled oscillator).

[0057] Video timing generation module, when the duty cycle of the PWM pulse becomes larger or smaller, the output voltage of the low-pass filter becomes higher or lower, and the output frequency of the VCXO becomes higher or lower to control the phase and frequency of the field signal POVS of the output image.

[0058] The specific steps are: 1) measure the phase difference between the reference VS and the feedback VS, 2) adjust the output PWM pulse of Pofld according to the phase difference. Then, the phase and frequency of the feedback VS are changed. Repeating steps 1) and 2) can synchronize PIVSB and POVS.

[0059] The present invention has the function of automatically adjusting the phase difference of the input signal, calculates the window sliding position according to the delay information of SDI decoded by FPGA, dynamically adjusts the window position and fills the cache.

[0060] The present invention provides automatic synchronization when 4 SDI lines have different lengths and cause phase differences, improves the ability of the monitor to adapt to other devices, can simultaneously monitor 4 signals, supports a maximum resolution of 4K60Hz, and can simultaneously receive different resolutions, effectively reducing the equipment cost in the application scenario and improving the user's operability. On the basis of ensuring performance and video display effects, the present invention proposes a phase difference solution and a front and back two-stage scaling technology to reduce hardware costs.

[0061] The present invention is described in detail above, but the present invention is not limited to the above-mentioned embodiments. Various changes can be made within the knowledge of ordinary technicians in the field without departing from the purpose of the present invention. Many other changes and modifications can be made without departing from the concept and scope of the present invention. It should be understood that the present invention is not limited to specific embodiments, and the scope of the present invention is defined by the appended claims.

Claims

1. FPGA-based asynchronous quad split monitor for broadcast-grade 4K or 8K, featuring: include: Four-channel SDI / HDMI2.0 codec module, used to decode FPGA signals and rearrange the decoded data; Input channel data cross selection module, used for input channel selection when single channel input is displayed in full screen; Four-channel video scaling / deinterlacing module, used for scaling and adaptive deinterlacing of independent channel videos; Four-channel image enhancement / color management module, used to adjust the image brightness, contrast, sharpness, color temperature of independent channels and implement 3D-LUT based on hexahedral interpolation algorithm; The video overlay / splicing synchronization module selects the synchronization clock as the synchronization reference clock of the memory module according to different synchronization output functions; Memory module, used to synchronize the decoded signals of four-channel SDI / HDMI2.0 codec modules; Screen interface encoding output display module; used to realize four-way four-split synchronous display with different resolutions; A bus control module, used to form a video flow path for the above modules; The output end of the SDI / HDMI2.0 codec module is connected to the input channel data cross selection module, the output end of the input channel data cross selection module is connected to the four-way video scaling / de-interlacing module, the output end of the four-way video scaling / de-interlacing module is connected to the four-way image enhancement / color management module, the output end of the four-way image enhancement / color management module is connected to the video overlay / splicing synchronization module, and the output end of the video overlay / splicing synchronization module is connected to the screen interface encoding output display module; The bus control module is connected with the four-channel SDI / HDMI2.0 codec module, the input channel data cross selection module, the four-channel video scaling / de-interlacing module, the four-channel image enhancement / color management module, the video overlay / splicing synchronization module, the memory module, and the screen interface encoding output display module; The video scaling / de-interlacing module uses a temporal interpolation algorithm and a spatial interpolation algorithm according to the stillness and motion of the image. When the images in two adjacent fields are still, the temporal interpolation algorithm is used; when there are moving scenes in adjacent fields, the spatial interpolation algorithm is used.

2. The asynchronous four-split monitor according to claim 1, characterized in that: The SDI / HDMI2.0 codec module decodes the high-speed SerDes signal of the FPGA into 12G SDI and HDMI2.0, and rearranges the decoded data according to the SMPTE protocol to obtain a four-channel YUV444 format signal for back-end video processing data.

3. The asynchronous four-split monitor according to claim 1, characterized in that: The video scaling / de-interlacing module adopts both a temporal interpolation algorithm and a spatial interpolation algorithm, and uses a motion compensation algorithm to calculate the motion of each pixel, obtain the motion vector of the object, and then interpolate along the direction of the motion trajectory.

4. The asynchronous four-split monitor according to claim 1, characterized in that: The video scaling / de-interlacing module divides the scaling technology into two levels, the front and back levels. The front level performs nonlinear correction on the spatial distance of the interpolation points according to the intrinsic correlation and geometric similarity of the image; the back level adjusts the grayscale weights of the surrounding pixels according to the gradient values ​​obtained by calculating the known pixels around the pixel to be interpolated, so that the grayscale value of the pixel after interpolation is close to the grayscale value corresponding to the pixel in the flat area.

5. The asynchronous four-split monitor according to claim 1, characterized in that: It also includes a synchronous phase-locking module, which includes: The time management module adjusts the pixel clock frequency of the output image to synchronize the input field signal and the output image field signal, namely, PIVS and POVS; The frame rate conversion module controls the frequency of the pixel clock POCLK of the output image by comparing the phase difference between POVS and PIVS and adjusting the duty cycle of the PWM pulse output from the pulse signal Pofld pin of the input image to change the input voltage of the VCXO; Video timing generation module, when the duty cycle of the PWM pulse becomes larger or smaller, the output voltage of the low-pass filter becomes higher or lower, and the output frequency of the VCXO becomes higher or lower to control the phase and frequency of the field signal POVS of the output image.

6. The asynchronous four-split monitor according to claim 1, characterized in that: Also includes: The phase difference automatic adjustment module calculates the window sliding position according to the delay information of SDI decoded by FPGA, dynamically adjusts the window position and fills the cache.

Images