A real-time monitoring and comparison method and system for ultra-high-definition video signals

Abstract

This invention provides a method and system for real-time monitoring and comparison of ultra-high-definition video signals. The method includes the following steps: Step 1: Signal acquisition: Acquiring ultra-high-definition video signals; Step 2: Signal analysis: Extracting fingerprints from the acquired ultra-high-definition video signals; Step 3: Content comparison: Retrieving the fingerprint information of two ultra-high-definition video signals to be compared from the task queue, aligning them, and then performing real-time comparison using a content comparison algorithm; Step 4: Arbitration switching and alarm display: After detecting inconsistencies in the comparison results, performing arbitration switching and alarm, and displaying the comparison results. This invention fills the gap in anomaly monitoring of ultra-high-definition video signals, ensures the safe broadcast of ultra-high-definition video signals, improves the security and compatibility of the entire system, and enables real-time monitoring and alarm analysis of all signals in an ultra-high-definition IP broadcasting system.

Description

Technical Field

[0001] The present invention belongs to the technical field of broadcasting and television, and in particular relates to a real-time monitoring and comparison method and system for ultra-high-definition video signals. Background Art

[0002] In order to meet the requirements of broadcast safety, signal monitoring uses signal monitoring tools to check the signal. When the signal is abnormal, it can alarm in real time to help the duty officer find the signal abnormality and take emergency measures in time. The signal monitoring tool can detect the audio and video content of the signal and detect it one by one.

[0003] In recent years, with the rapid development of television and video media, users' demand for ultra-high-definition video has increased. In the field of radio and television, there are relatively strict safety broadcasting requirements for ultra-high-definition video and high-definition video. However, the existing signal monitoring tools only support abnormal monitoring of standard-definition and high-definition video signals. Since the broadcast of ultra-high-definition video is a new technology, there are currently no tools and methods for monitoring ultra-high-definition video signals. Therefore, how to monitor and compare ultra-high-definition video signals is an urgent technical problem to be solved.

[0004] The existing technology generally uses high-definition consistency comparison detection software to detect the audio and video content of the signal. The broadcast system uses the card I / O to collect baseband or IP high-definition signals into the server's memory, decodes the collected media files, and obtains the fingerprint information of the decoded media files, including media fingerprints and time information. The fingerprint information is sent to the comparison server, and the comparison server compares the fingerprint information of the reference source and the comparison source based on the time information, thereby realizing audio and video consistency detection.

[0005] However, for ultra-high-definition video signals, the existing technology has the following problems:

[0006] 1. The existing technology uses card I / O to collect baseband or IP HD signals into the server's memory, but does not support ultra-high-definition video signal collection;

[0007] 2. The existing technology only supports comparison and detection of standard definition and high definition signal types, and cannot perform full-link detection in ultra-high-definition IP broadcasting systems. Summary of the invention

[0008] The purpose of the present invention is to provide a real-time monitoring and comparison method and system for ultra-high-definition video signals to solve the above technical problems.

[0009] In order to achieve the above object, the technical solution of the present invention is:

[0010] The present invention discloses a real-time monitoring and comparison method for ultra-high-definition video signals, the method comprising the following steps:

[0011] Step 1: Signal acquisition: Acquire ultra-high-definition video signals, including baseband SDI signals, uncompressed IP signal streams based on the 2110 standard, and compressed TS signal streams;

[0012] Step 2: Signal analysis: First, the collected UHD video signal is preprocessed, including down-conversion and tone mapping. Then, the fingerprint is extracted from the preprocessed signal, and the current acquisition time is used as the frame number time, and the timestamp is entered to form the fingerprint information. Finally, the extracted UHD video signal fingerprint information is added to the comparison task queue.

[0013] Step 3: Content comparison: The fingerprint information of the two UHD video signals to be compared is taken out from the task queue for alignment processing, and then the fingerprint information of the UHD video signals is compared in real time through the content comparison algorithm;

[0014] Step 4: Arbitration switching and alarm display: After detecting inconsistent comparison results, arbitration switching and alarm are performed, and the comparison results are displayed.

[0015] Furthermore, the specific process of the down-conversion processing is to use regional difference and Gaussian filtering algorithm to down-sample the collected YUV video data, based on the perceptual hash value of YUV, to avoid the influence of inter-line flicker and jagged effect of the down-converted motion image, and scale the original multiple pixels to represent them with one pixel.

[0016] Furthermore, the specific process of the tone mapping process is to use a tone mapping algorithm to first map the image brightness and then perform color mapping. The color mapping is to obtain new color information through the original image information. The color mapping adopts the formula:

[0017]

[0018] Among them, C and c represent the colors before and after tone mapping, respectively, L represents the HDR image brightness, T is the brightness value after tone mapping, and s is the parameter for adjusting saturation.

[0019] Furthermore, the specific process of fingerprint extraction is: calculating the block brightness mean and brightness histogram of the preprocessed ultra-high-definition video signal frame data, and splicing the calculation results into a vector as the fingerprint of the ultra-high-definition video signal.

[0020] Furthermore, the specific steps of the content comparison algorithm are:

[0021] 1) First, frame alignment is performed, using segmented overall matching to find the position with the maximum average similarity. If the maximum similarity is greater than the set first threshold, the alignment is considered successful, otherwise it enters the next alignment process;

[0022] 2) After alignment, the similarity of the fingerprints of the two signals is calculated frame by frame. If the fingerprint similarity is lower than the set second threshold, the two signals are considered to be mismatched. If the number of consecutive mismatches exceeds the tolerance, the two signals are considered inconsistent, an alarm is issued and the alignment process is entered. After alignment, frame by frame comparison is performed again.

[0023] Furthermore, the first threshold is 90, the second threshold is 80, and the tolerance is 5 frames.

[0024] The present invention also discloses a real-time monitoring and comparison system for ultra-high-definition video signals, the system comprising a signal acquisition device, a signal analysis device, a content comparison device and a client; the signal acquisition device is used to acquire ultra-high-definition video signals, the signal analysis device is used to extract fingerprints from the acquired ultra-high-definition video signals, the content comparison device is used to compare the extracted fingerprint information, and the client is used to arbitrate switching, display comparison results and alarm;

[0025] The signal acquisition device is provided with a professional-grade SDI board, an IP I / O board and an IP network card. The professional-grade SDI board is used to acquire baseband SDI signals, and the IP I / O board and the IP network card are used to acquire uncompressed IP signal streams and compressed TS signal streams based on the 2110 standard.

[0026] The signal analysis device comprises a down-conversion module and a tone mapping module for pre-processing the collected ultra-high-definition signal, and a fingerprint extraction module for extracting fingerprints from the pre-processed signal; the down-conversion module is used to perform down-conversion processing on the signal, and the tone mapping module is used to perform tone mapping processing on the signal;

[0027] The content comparison device comprises a sequence alignment module and a comparison module, wherein the sequence alignment module is used to align the fingerprint information of the two ultra-high-definition signals to be compared, and the comparison module is used to perform real-time comparison on the fingerprint information of the two ultra-high-definition signals after alignment;

[0028] The client is provided with an arbitration switching module and an alarm display module. The arbitration switching module is used to perform arbitration according to the comparison result and perform signal switching according to the arbitration result. The alarm display module is used to display the comparison result and the sound and light alarm.

[0029] The beneficial effects of the present invention are as follows: the present invention fills the gap in the prior art for abnormal monitoring of ultra-high-definition video signals, ensures the safe broadcasting of ultra-high-definition video signals, develops advanced core technologies for signal acquisition, encoding and decoding, and fingerprint analysis, realizes real-time detection of ultra-high-definition video signals, improves the security and compatibility of the entire system, and can perform real-time monitoring and alarm analysis of all signals of the ultra-high-definition IP broadcasting system.

[0030] The present invention is further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of a real-time monitoring and comparison method for ultra-high-definition video signals;

[0032] Figure 2 This is the principle block diagram of the real-time monitoring and comparison system for ultra-high-definition video signals. DETAILED DESCRIPTION

[0033] The present invention discloses a real-time monitoring and comparison method and system for ultra-high-definition video signals, such as Figure 1 As shown, the method comprises the following steps:

[0034] Step 1: Signal acquisition: Acquire ultra-high-definition video signals. For baseband SDI signal transmission, use professional-grade SDI boards for acquisition; for uncompressed IP signal streams and compressed TS signal streams based on the 2110 standard, use IP I / O cards or IP network cards for acquisition. For uncompressed video signal streams, there is no need to re-encode during signal processing, and real-time signal processing can be achieved; for compressed video signal streams, decode first and then acquire the signal.

[0035] Step 2: Signal analysis: First, pre-process the collected UHD video signal, including down-conversion and tone mapping. Then, perform fingerprint extraction on the pre-processed signal, use the current acquisition time as the frame number time, enter the timestamp to form the fingerprint information, and finally add the extracted UHD video signal fingerprint information to the comparison task queue.

[0036] Among them, the specific process of down-conversion processing is: down-sampling the collected YUV video data. Since the image resolution of ultra-high-definition video signals is increased from high-definition 1920x1080 to 3840×2160 and 7680*4320 compared with high-definition video signals, each frame has more than 8 million pixels, and the quantization bit is increased from 8 bits of high-definition to 10 bits. The sampling is more intensive. In order to ensure subsequent calculations and comparisons as well as calculation efficiency, the regional difference and Gaussian filtering algorithms are used to down-sample the image. Based on the perceptual hash value of YUV, the proximity of slight changes in the source is guaranteed to avoid the influence of inter-line flickering, aliasing and other effects of the down-converted moving image, and the original multiple pixels are scaled and represented by one pixel.

[0037] The specific process of tone mapping is as follows: the color gamut of ultra-high-definition video signals is upgraded from Rec.709 to BT.2020, and the color space is also changed from SDR to HDR. The tone mapping algorithm is used to map the image brightness first, and then perform color mapping. Color mapping is to obtain new color information through the original image information. The color mapping adopts the formula:

[0038]

[0039] Among them, C and c represent the colors before and after tone mapping, respectively, L represents the HDR image brightness, T is the brightness value after tone mapping, and s is the parameter for adjusting saturation.

[0040] The specific process of fingerprint extraction is: calculate the block brightness mean and brightness histogram of the preprocessed ultra-high-definition video signal frame data, and splice the calculation results into a vector as the fingerprint of the ultra-high-definition video signal.

[0041] Step 3: Content comparison: The fingerprint information of the two UHD video signals to be compared is taken out from the task queue for alignment processing, and then the UHD video signal fingerprint information is compared in real time through the content comparison algorithm, and abnormal situations are detected. Generally, the comparison and detection is performed on the main and backup signals, but other signals can also be used.

[0042] The specific steps of the content comparison algorithm are as follows:

[0043] 1) First, frame alignment is performed, using segmented overall matching to find the position with the maximum average similarity nearby. If the maximum similarity is greater than the set first threshold (reference value is 90), the alignment is considered successful, otherwise it enters the next alignment process;

[0044] 2) After alignment, the similarity of the fingerprints of the two signals is calculated frame by frame. If the similarity is lower than the set second threshold (reference value is 80), the two signals are considered to be mismatched. If the number of consecutive mismatches exceeds the tolerance (reference value is 5 frames), the two signals are considered inconsistent, an alarm is issued and the alignment process is entered. After alignment, frame by frame comparison is performed again.

[0045] During the signal comparison process, abnormal conditions will be detected. The abnormal detection items mainly include: black field, still frame, video loss, color field, color bars, station logo loss, test pattern and other specific scenes.

[0046] Step 4: Arbitration switching and alarm display: When the comparison results of the two signals are detected to be inconsistent, arbitration switching is performed. According to the preset comparison strategy, the comparison software performs arbitration, and according to the arbitration results, through the RS422 serial port or network transmission, the switch panel is given instructions to control the four-in-one switch to perform signal switching and sound and light alarms, and the comparison results and switching records are intuitively presented on the display page. At the same time, the link diagram configured on the display page can directly display the real-time screen of each monitoring node. When a signal fails, it can be displayed on the link node to help customers quickly locate the problem point and assist customers in solving the problem.

[0047] The real-time monitoring and comparison system of the ultra-high-definition video signal includes a signal acquisition device, a signal analysis device, a content comparison device and a client, such as Figure 2 As shown; the signal acquisition device is used to collect ultra-high-definition video signals, the signal analysis device is used to extract fingerprints from the collected ultra-high-definition video signals, the content comparison device is used to compare the extracted fingerprint information, and the client is used to arbitrate switching, display comparison results and alarm;

[0048] The signal acquisition device is provided with a professional-grade SDI board, an IP I / O board and an IP network card. The professional-grade SDI board is used to acquire baseband SDI signals, and the IP I / O board and the IP network card are used to acquire uncompressed IP signal streams and compressed TS signal streams based on the 2110 standard.

[0049] The signal analysis device comprises a down-conversion module and a tone mapping module for pre-processing the collected ultra-high-definition signal, and a fingerprint extraction module for extracting fingerprints from the pre-processed signal; the down-conversion module is used to perform down-conversion processing on the signal, and the tone mapping module is used to perform tone mapping processing on the signal;

[0050] The content comparison device comprises a sequence alignment module and a comparison module, wherein the sequence alignment module is used to align the fingerprint information of the two ultra-high-definition signals to be compared, and the comparison module is used to perform real-time comparison on the fingerprint information of the two ultra-high-definition signals after alignment;

[0051] The client is provided with an arbitration switching module and an alarm display module. The arbitration switching module is used to perform arbitration according to the comparison result and perform signal switching according to the arbitration result. The alarm display module is used to display the comparison result and the sound and light alarm.

[0052] The hardware architecture of the above system includes a consistency comparison server, a control panel, and a four-choice server. The consistency comparison software is deployed on the consistency comparison server. The consistency comparison software is the core of the system and can compare multiple ultra-high-definition signals. The consistency comparison platform is a CS+BS architecture, in which CS is mainly used for background technical review and comparison processing and centralized control, and BS is used as the front-end aggregation and display software to display the signal comparison results output by CS on the front-end.

[0053] Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred arrangement scheme, a person skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A real-time monitoring and comparison method for ultra-high-definition video signals, characterized in that: The method comprises the following steps: Step 1: Signal acquisition: Acquire ultra-high-definition video signals, including baseband SDI signals, uncompressed IP signal streams based on the 2110 standard, and compressed TS signal streams; Step 2: Signal analysis: First, the collected UHD video signal is preprocessed, including down-conversion and tone mapping. Then, the fingerprint is extracted from the preprocessed signal, and the current acquisition time is used as the frame number time, and the timestamp is entered to form the fingerprint information. Finally, the extracted UHD video signal fingerprint information is added to the comparison task queue. The specific process of the down-conversion processing is to use the regional difference and Gaussian filtering algorithm to down-sample the collected YUV video data, avoid the influence of inter-line flickering and sawtooth effect of the down-converted moving image based on the perceptual hash value of YUV, and scale the original multiple pixels to be represented by one pixel; The specific process of the tone mapping process is to use a tone mapping algorithm to first map the image brightness and then perform color mapping. The color mapping is to obtain new color information through the original image information. The color mapping adopts the formula: Where C and c represent the colors before and after tone mapping, respectively, L represents the brightness of the HDR image, T is the brightness value after tone mapping, and s is the parameter for adjusting saturation; The specific process of fingerprint extraction is: calculating the block brightness mean and brightness histogram of the pre-processed ultra-high-definition video signal frame data, and splicing the calculation results into a vector as the fingerprint of the ultra-high-definition video signal; Step 3: Content comparison: The fingerprint information of the two UHD video signals to be compared is taken out from the task queue for alignment processing, and then the fingerprint information of the UHD video signals is compared in real time through the content comparison algorithm; Step 4: Arbitration switching and alarm display: After detecting inconsistent comparison results, arbitration switching and alarm are performed, and the comparison results are displayed.

2. The real-time monitoring and comparison method of ultra-high-definition video signals according to claim 1 is characterized in that: The specific steps of the content comparison algorithm are: 1) First, frame alignment is performed, using segmented overall matching to find the position with the maximum average similarity. If the maximum similarity is greater than the set first threshold, the alignment is considered successful, otherwise it enters the next alignment process; 2) After alignment, the similarity of the fingerprints of the two signals is calculated frame by frame. If the fingerprint similarity is lower than the set second threshold, the two signals are considered to be mismatched. If the number of consecutive mismatches exceeds the tolerance, the two signals are considered inconsistent, an alarm is issued and the alignment process is entered. After alignment, frame by frame comparison is performed again.

3. The real-time monitoring and comparison method of ultra-high-definition video signals according to claim 2 is characterized in that: The first threshold is 90, the second threshold is 80, and the tolerance is 5 frames.

4. A real-time monitoring and comparison system for ultra-high-definition video signals implementing the method described in any one of claims 1 to 3, characterized in that: The system includes a signal acquisition device, a signal analysis device, a content comparison device and a client; the signal acquisition device is used to acquire ultra-high-definition video signals, the signal analysis device is used to extract fingerprints from the acquired ultra-high-definition video signals, the content comparison device is used to compare the extracted fingerprint information, and the client is used to arbitrate switching, display comparison results and alarm; The signal acquisition device is provided with a professional-grade SDI board, an IPI / O board and an IP network card. The professional-grade SDI board is used to acquire baseband SDI signals, and the IP I / O board and IP network card are used to acquire uncompressed IP signal streams and compressed TS signal streams based on the 2110 standard. The signal analysis device comprises a down-conversion module and a tone mapping module for pre-processing the collected ultra-high-definition signal, and a fingerprint extraction module for extracting fingerprints from the pre-processed signal; the down-conversion module is used to perform down-conversion processing on the signal, and the tone mapping module is used to perform tone mapping processing on the signal; The content comparison device comprises a sequence alignment module and a comparison module, wherein the sequence alignment module is used to align the fingerprint information of the two ultra-high-definition signals to be compared, and the comparison module is used to perform real-time comparison on the fingerprint information of the two ultra-high-definition signals after alignment; The client is provided with an arbitration switching module and an alarm display module. The arbitration switching module is used to perform arbitration according to the comparison result and perform signal switching according to the arbitration result. The alarm display module is used to display the comparison result and the sound and light alarm.

Images