A method and system for comparing signal consistency before and after broadcast link delay

Abstract

The present invention provides a method and system for comparing the consistency of signals before and after the delay of a broadcast link. First, the alignment time period and the delay time are determined, and the following steps are sequentially performed within the alignment time period: synchronously acquiring multi-frame pre-delay signals and multi-frame post-delay signals respectively; extracting the characteristic values ​​of the multi-frame pre-delay signal segments as the pre-delay comparison features; rolling and splitting the multi-frame post-delay signals into multiple groups of post-delay signals; extracting the characteristic values ​​of each group of post-delay signal segments as the post-delay comparison features, comparing them with the pre-delay comparison features, assigning scores and numbers; determining the highest scoring group, and comparing the comparison results with the set threshold. The present invention quickly realizes the alignment of two-way delayed videos by adopting a segmented overall matching method, memorizes the maximum similarity of the two ways after alignment, ensures real-time frame-by-frame comparison of the two ways, ensures the alignment accuracy, and can effectively respond to the delay time jump for automatic realignment, greatly reducing the error.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a method and system for comparing signal consistency before and after a broadcast link delay. Background Art

[0002] In a TV station, the broadcast system is responsible for broadcasting the program material files or external real-time signals to be broadcast in sequence according to the time arrangement plan of the program list. In this process, the broadcast signal needs to be processed by a series of nodes on the broadcast link, such as switching and broadcasting of video file materials / external broadcast signals through the video server, loudness control through the loudness controller, station logo clock through the keyer, subtitles through the graphic packaging system, emergency switching of the main and backup channels through the emergency switch, bit rate compression through the encoder, etc. Each processing node in this process may affect the technical indicators or audio and video content of the audio and video broadcast signal, and will cause different degrees of delay to the signal.

[0003] In order to ensure the security of the broadcast signal and the effectiveness of the content, it is generally necessary to conduct consistency comparison and automatic review of technical indicators (hereinafter referred to as automatic technical review) on the signals processed by each important processing node on the signal link in the broadcast system. The consistency comparison is to ensure the consistency of the content at each location on the signal link to prevent tampering or a significant decline in indicators. The automatic technical review is to prevent the signal processed by the processing node from introducing new technical indicator failures. However, since each processing node in the signal link will cause delays in signal transmission, the content sampled from various locations on the link is likely to be inconsistent due to time delays. Direct automated consistency content comparison will cause a large number of false alarms. Therefore, the common practice adopted by current technology is to manually align the signals before and after the delay before performing consistency comparison and automatic technical review.

[0004] However, current technologies have the following drawbacks:

[0005] First, manual alignment is time-consuming and labor-intensive, and has large errors. At present, the frame rate of domestic high-definition television is 25 frames per second, and the frame rate of ultra-high-definition television is 50 frames per second. Manual alignment needs to ensure that the error of the picture before and after the delay is within 40 milliseconds or 20 milliseconds. Alignment is difficult, inefficient, and prone to errors.

[0006] Second, the manual alignment method is not sensitive to delay jumps. After the manual alignment delay, the delay of each sampling point for consistency comparison is fixed. At this time, when the delay of a processing node in video processing changes, it is difficult to detect in time, which will cause false alarms of inconsistent content comparison caused by delay. Even if it is discovered in time, re-alignment is inefficient. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for comparing signal consistency before and after broadcast link delay to solve the above technical problems.

[0008] In order to achieve the above object, the technical solution of the present invention is:

[0009] The present invention discloses a method for comparing the consistency of signals before and after the delay of a broadcast link, which is an alignment processing method for the signals before and after the delay before the automatic review of the broadcast link. First, the alignment time period and the delay time are determined, and the following steps are sequentially performed within the alignment time period:

[0010] Step 1: synchronously obtain multiple frames of pre-delay signals and multi-frame post-delay signals respectively. The pre-delay signals and post-delay signals are obtained from the broadcast link. The number of frames of the post-delay signal is greater than the number of frames of the pre-delay signal.

[0011] Step 2: extract the feature values ​​of the signal segments before the multi-frame delay as the comparison features before the delay;

[0012] Step 3: rollingly split the multi-frame delayed signals into multiple groups of delayed signals, numbering each group of delayed signals, the number of frames of each group of delayed signals is equal to the number of frames of the pre-delayed signals, and the last frame of the last group of delayed signals is the last frame of the multi-frame delayed signals;

[0013] Step 4: Extract each group of delayed signal segment feature values ​​as delayed comparison features, compare each group of delayed comparison features with the pre-delay comparison features, assign scores to the consistency of the comparison, high consistency means high scores, low consistency means low scores, and assign corresponding numbers to each group of consistency scores;

[0014] Step 5: Determine the highest scoring group, extract a group of delayed signal fragment feature values ​​at the beginning of the frame sorted by the highest scoring group number from the multi-frame delayed signal as the delayed comparison feature, compare it with the pre-delay comparison feature, and compare the comparison result with the set threshold. When the comparison result is higher than the set threshold, the highest scoring group is used as the alignment point for comparison. After the end of this alignment time period, return to the first step to enter the next alignment time period. When the comparison result is lower than the set threshold, an alarm is issued.

[0015] Furthermore, the number of frames of the pre-delay signal is determined by manual input and is at least 4 frames.

[0016] Furthermore, the delay time does not exceed 1 second.

[0017] Furthermore, the alignment time period is 1 minute.

[0018] Furthermore, the characteristic values ​​of the signal segment before delay and the characteristic values ​​of the signal segment after delay include brightness sampling and spectrum energy distribution of specific pixels based on the frame sequence.

[0019] The present invention also discloses a signal consistency comparison system before and after the broadcast link delay, the system comprising a frame data buffer pool 1, a segment feature value extractor 1, a frame data buffer pool 2, a frame data rolling split marker, a segment feature value extractor 2, a consistency comparison module, a score register, a flag indicator, and a result presentation and alarm module;

[0020] The frame data buffer pool 1 is used to buffer the multi-frame pre-delay signal obtained from the broadcast link;

[0021] The segment feature value extractor 1 is used to extract segment feature values ​​from the multi-frame pre-delay signal in the frame data buffer pool 1;

[0022] The frame data buffer pool 2 is used to buffer the multi-frame delayed signals obtained from the broadcast link;

[0023] The frame data rolling splitting marker is used to rollingly split the multi-frame delayed signals in the frame data buffer pool 2 into multiple groups of delayed signals, and number each group of delayed signals, the number of frames of each group of delayed signals is equal to the number of frames of the pre-delayed signals, and the last frame of the last group of delayed signals is the last frame of the multi-frame delayed signals;

[0024] The segment feature value extractor 2 is used to extract segment feature values ​​from multiple groups of delayed signals in the frame data rolling split marker; it is also used to read a group of delayed signals starting with a frame sorted by the number from the frame data buffer pool 2 and extract segment feature values ​​according to the number sent by the flag indicator;

[0025] The consistency comparison module is used to compare each group of delayed signal segment feature values ​​extracted by the segment feature value extractor 2 with the multi-frame pre-delay signal segment feature values ​​extracted by the segment feature value extractor 1, and send the compared consistency score and the corresponding number to the score register; the consistency comparison module is also used to send the comparison result of the pre-delay signal segment feature value and a group of delayed signal segment feature values ​​at the beginning of the frame sorted by the number sent by the mark indicator to the result presentation and alarm module;

[0026] The score register is used to receive and store the consistency score and corresponding number sent by the consistency comparison module;

[0027] The flag indicator is used to retrieve the highest score group from the score register and send its number to the segment feature value extractor 2;

[0028] The result presentation and alarm module is used to display the consistency comparison result, and compare the consistency comparison result with a set threshold value, and issue an alarm when the consistency comparison result is lower than the set threshold value.

[0029] The beneficial effects of the present invention are as follows: the present invention proposes a method and system for comparing the consistency of signals before and after the delay of a broadcast link, which can automatically align the time difference delay generated by each processing node on the broadcast link, and quickly align the two-way delayed video by comparing the signals before and after the delay in real time, and by adopting a segmented overall matching method. After alignment, the maximum similarity of the two ways is memorized to ensure real-time frame-by-frame comparison of the two ways, thereby improving the alignment efficiency, ensuring the alignment accuracy, and being able to effectively respond to the delay time jump by automatic realignment. In addition, since the continuous front and back frames in the broadcast screen may have a high degree of content similarity, if the automatic alignment is performed by single frame comparison, there may be a large error. Therefore, the automatic comparison object in the present invention is a frame group composed of multiple frames of video and audio data, which greatly reduces the error.

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the signal consistency comparison method before and after the broadcast link delay;

[0032] Figure 2 A schematic diagram of a segment feature value extractor 1 extracting segment feature values ​​from a pre-delay signal;

[0033] Figure 3 Schematic diagram of the signal after the frame data buffer pool 2 caches the delay;

[0034] Figure 4 A frame data rolling splitting marker rollingly splits a plurality of delayed frames in a frame data buffer pool 2 into a plurality of groups of delayed signals;

[0035] Figure 5 A schematic diagram of extracting segment feature values ​​from the first group of delayed signals by the segment feature value extractor 2;

[0036] Figure 6 A schematic diagram of a consistency comparison module sending the comparison result of the first group of delayed signals to a score register;

[0037] Figure 7 It is a schematic diagram of the consistency scores and corresponding numbers of each group of delayed signals;

[0038] Figure 8 A schematic diagram of a flag indicator sending the number of the highest scoring group to a segment feature value extractor 2;

[0039] Fig. 9 A schematic diagram of a segment feature value extractor 2 reading a group of delayed signals starting with a frame sorted by the highest score group number from a frame data buffer pool 2 and performing comparison. DETAILED DESCRIPTION

[0040] The present invention discloses a signal consistency comparison method and system before and after the broadcast link delay. The method is an alignment processing method for the signals before and after the delay before the broadcast link is automatically reviewed. First, the alignment time period and the delay time are determined. The delay time can be roughly determined according to the delay range observed by human eyes, which is generally not more than 1 second. The following steps are sequentially performed within the alignment time period, such as: Figure 1 As shown:

[0041] Step 1: synchronously obtain multiple frames of pre-delay signals and post-delay signals respectively. The pre-delay signals and post-delay signals are obtained from the broadcast link (main broadcast link or backup broadcast link). The number of frames of the post-delay signal is greater than the number of frames of the pre-delay signal. The pre-delay signal is a sampling node of the audio and video signals at the front of the broadcast link, and the post-delay signal is a sampling node of the audio and video signals at the back of the broadcast link. Since the signal has passed through a series of processing nodes, it has a backward offset. The pre-delay signal needs to be compared with the post-delay signal for consistency to determine the health of the link.

[0042] Step 2: Extract the characteristic values ​​of the signal segments before the multi-frame delay as the comparison features before the delay; the key to extracting the characteristic values ​​of the segments is to extract the characteristic values ​​of the signals before the multi-frame delay as a whole, and the extracted characteristic values ​​are used for consistency comparison. The characteristic values ​​extracted in the present invention include brightness sampling of specific pixels based on the frame sequence, spectrum energy distribution, etc.

[0043] Step 3: Rollingly split the multi-frame delayed signals into multiple groups of delayed signals with relatively small frame numbers, and number each group of delayed signals. The number of frames of each group of delayed signals after splitting is equal to the number of frames of the pre-delay signal, and the last frame of the last group of delayed signals is the last frame of the multi-frame delayed signals. This is to facilitate the comparison of the split delayed signals with the pre-delay signals.

[0044] Step 4: Extract the feature values ​​of each group of delayed signal segments after splitting as the delayed comparison features, compare each group of delayed comparison features with the pre-delay comparison features, and assign scores to the consistency of the comparison. High consistency means high scores, low consistency means low scores, and assign corresponding numbers to each group of consistency scores; the consistency score is the similarity score, and the higher the score, the more similar the signal data (image and audio content, etc.). Each group of scores corresponds to a number, and the highest score group can be clearly judged, that is, the group of delayed signals has the highest similarity with the pre-delay signals.

[0045] Step 5: Determine the highest scoring group, extract the segment feature values ​​of a group of delayed signals at the beginning of the frame sorted by the highest scoring group number from the multi-frame delayed signals as the delayed comparison features, compare them with the pre-delay comparison features, and compare the comparison results with the set threshold. The set threshold is generally determined after multiple rounds of debugging based on the actual situation, so as to be sensitive to abnormalities while minimizing false alarms. When the comparison result is higher than the set threshold, the highest scoring group is used as the alignment point for comparison. After the end of this alignment time period, return to the first step to enter the next alignment time period. When the comparison result is lower than the set threshold, an alarm is issued.

[0046] Specifically, the number of frames of the signal before delay is determined by manual input, and in the present invention, it is at least 4 frames. The alignment time period is set in advance, for example, it can be 1 minute. The feature value of the signal segment before delay and the feature value of the signal segment after delay are the feature values ​​of the video and audio array.

[0047] The system includes a frame data buffer pool 1, a fragment feature value extractor 1, a frame data buffer pool 2, a frame data rolling split marker, a fragment feature value extractor 2, a consistency comparison module, a score register, a flag indicator, and a result presentation and alarm module;

[0048] The frame data buffer pool 1 is used to cache the multi-frame pre-delay signals obtained from the broadcast link, and the number of frames of the cached pre-delay signals is determined by manual input; the fragment feature value extractor 1 is used to extract fragment feature values ​​from the multi-frame pre-delay signals in the frame data buffer pool 1; the frame data buffer pool 2 is used to cache the multi-frame post-delay signals obtained from the broadcast link, and the number of frames of the cached post-delay signals is determined by manual input, and the number of frames of the delayed signals is at least one times greater than the number of frames of the pre-delay signals; the frame data rolling split marker is used to rollingly split the multi-frame post-delay signals in the frame data buffer pool 2 into multiple groups of post-delay signals with relatively small frame numbers, and number each group of post-delay signals, the number of frames of each group of post-delay signals is equal to the number of frames of the pre-delay signals, and the last frame of the last group of post-delay signals is the last frame of the multi-frame post-delay signals; the fragment feature value extractor 2 is used to extract fragment feature values ​​from the multiple groups of post-delay signals in the frame data rolling split marker, and is also used to select a group from the frame data buffer pool 2 according to the number of the highest scoring group sent by the flag indicator. A group of delayed signals starting with the frame sorted by the number is read and the fragment feature value is extracted; the consistency comparison module is used to compare each group of delayed signal fragment feature values ​​extracted by the fragment feature value extractor 2 with the multi-frame pre-delay signal fragment feature values ​​extracted by the fragment feature value extractor 1, and send the compared consistency score and the corresponding number to the score register, and the consistency comparison module is also used to send the comparison result of the pre-delay signal fragment feature value with a group of delayed signal fragment feature values ​​starting with the frame sorted by the highest score group number sent by the flag indicator to the result presentation and alarm module; the score register is used to receive and store the consistency score and the corresponding number sent by the consistency comparison module; the flag indicator is used to retrieve the highest score group from the score register and send its number to the fragment feature value extractor 2; the result presentation and alarm module is used to display the comparison result sent by the consistency comparison module, and compare the comparison result with the set threshold value, and alarm when the comparison result is lower than the set threshold value.

[0049] In order to more clearly illustrate the method and system of the present invention, a specific application example is provided as follows. The method comprises the following steps:

[0050] Step 1: Sample the pre-delay signal at the front end of the broadcast link and input it into the frame data buffer pool 1 for buffering. The number of frames of the buffered pre-delay signal is manually set in advance. In this embodiment, the pre-delay signal buffered by the frame data buffer pool 1 is 4 frames. The segment feature value extractor 1 extracts the segment feature value of the pre-delay signal in the frame data buffer pool 1 as the pre-delay comparison feature, such as Figure 2 shown.

[0051] Step 2: Sample the delayed signal at the back end of the broadcast link and input it into the frame data buffer pool 2 for caching. The number of frames of the cached delayed signal is manually set in advance, and the number of frames of the cached delayed signal must be greater than the number of frames of the cached pre-delay signal. In this embodiment, the delayed signal cached by the frame data buffer pool 2 is 9 frames. Figure 3 shown.

[0052] Step 3: The frame data rolling splitting marker rolls the multi-frame delayed signals (9 frames in this embodiment) in the frame data buffer pool 2 into multiple groups of delayed signals (6 groups in this embodiment) according to the number of frames of the pre-delayed signals in the frame data buffer pool 1 (4 frames in this embodiment), and adds numbers to each group of delayed signals after the split (such as splitting 9 frames of data into the first group of 1, 2, 3, 4 frames, the second group of 2, 3, 4, 5 frames ... the sixth group of 6, 7, 8, 9 frames), such as Figure 4 shown.

[0053] Step 4: The segment feature value extractor 2 reads the first group of delayed signals from the frame data rolling split marker, extracts the segment feature values ​​and marks them as the first group, such as Figure 5 shown.

[0054] Step 5: The consistency comparison module extracts the feature values ​​to be compared from the segment feature value extractor 1 and the segment feature value extractor 2 respectively for comparison, and after the comparison, sends the consistency score and number to the score register for storage, such as Figure 6 shown.

[0055] Step 6: Repeat steps 4 and 5 to compare multiple groups of delayed signals in the frame data rolling split marker in turn, and send the consistency score and corresponding number of each group of delayed signals to the score register for storage, such as Figure 7 shown.

[0056] Step 7: The flag indicator retrieves the highest score group from the score register and sends the group number to the segment feature value extractor 2, such as Figure 8 shown.

[0057] Step 8: The segment feature value extractor 2 uses the number of the highest scoring group as the basis for determining the delay, reads a group of delayed signals starting with the frame sorted by the number from the frame data buffer pool 2, extracts the segment feature value and sends it to the consistency comparison module, and sends the consistency comparison result to the result presentation and alarm module. The result presentation and alarm module displays the comparison result and compares the comparison result with the set threshold. When the comparison result is lower than the set threshold, an alarm is issued, such as Fig. 9 shown.

[0058] Step 9: Frame data buffer pool 1 and frame data buffer pool 2 continue to scroll and obtain the buffer queue from the pre-delay signal and the post-delay signal respectively and update the buffer, and repeat step 8.

[0059] Step 10: Repeat steps 1 to 9 according to the pre-set alignment period. In this embodiment, the alignment period is 1 minute.

[0060] During the broadcast process, since the continuous previous and next frame data in the broadcast screen may have a high degree of content similarity, if the single frame data is automatically aligned, there may be a large error. The above method uses the frame data group as the automatic comparison object, and adopts the segmented overall matching method to quickly realize the alignment of the two-way delayed video. After alignment, the maximum similarity of the two channels is memorized to ensure real-time frame-by-frame comparison of the two channels, which improves the alignment accuracy and efficiency. Setting the alignment period is a periodic alignment measure to prevent the aligned signal from jumping in time series and then changing again.

[0061] Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred arrangement scheme, a person skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for comparing the consistency of signals before and after the delay of a broadcast link is a method for aligning the signals before and after the delay before the automatic review of the broadcast link, characterized in that: First, determine the alignment time period and delay time, and perform the following steps in sequence within the alignment time period: Step 1: synchronously obtain multiple frames of pre-delay signals and multi-frame post-delay signals respectively. The pre-delay signals and post-delay signals are obtained from the broadcast link. The number of frames of the post-delay signal is greater than the number of frames of the pre-delay signal. Step 2: extract the feature values ​​of the signal segments before the multi-frame delay as the comparison features before the delay; Step 3: rollingly split the multi-frame delayed signals into multiple groups of delayed signals, numbering each group of delayed signals, the number of frames of each group of delayed signals is equal to the number of frames of the pre-delayed signals, and the last frame of the last group of delayed signals is the last frame of the multi-frame delayed signals; Step 4: Extract each group of delayed signal segment feature values ​​as delayed comparison features, compare each group of delayed comparison features with the pre-delay comparison features, assign scores to the consistency of the comparison, high consistency means high scores, low consistency means low scores, and assign corresponding numbers to each group of consistency scores; Step 5: Determine the highest scoring group, extract a group of delayed signal fragment feature values ​​at the beginning of the frame sorted by the highest scoring group number from the multi-frame delayed signal as the delayed comparison feature, compare it with the pre-delay comparison feature, and compare the comparison result with the set threshold. When the comparison result is higher than the set threshold, the highest scoring group is used as the alignment point for comparison. After the end of this alignment time period, return to the first step to enter the next alignment time period. When the comparison result is lower than the set threshold, an alarm is issued.

2. According to claim 1, the signal consistency comparison method before and after the broadcast link delay is characterized in that: The number of frames of the pre-delay signal is determined by manual input and is at least 4 frames.

3. According to claim 1, the signal consistency comparison method before and after the broadcast link delay is characterized in that: The delay time does not exceed 1 second.

4. According to claim 1, the signal consistency comparison method before and after the broadcast link delay is characterized in that: The alignment time period is 1 minute.

5. According to claim 1, the signal consistency comparison method before and after the broadcast link delay is characterized in that: The pre-delay signal segment characteristic value and the post-delay signal segment characteristic value include brightness sampling and spectrum energy distribution of specific pixels based on the frame sequence.

6. A broadcast link delay signal consistency comparison system that implements a broadcast link delay signal consistency comparison method according to any one of claims 1 to 5, characterized in that: The system includes a first frame data buffer pool, a first segment feature value extractor, a second frame data buffer pool, a frame data rolling split marker, a second segment feature value extractor, a consistency comparison module, a score register, a flag indicator, and a result presentation and alarm module; The first frame data buffer pool is used to buffer the multi-frame pre-delay signal obtained from the broadcast link; The first segment feature value extractor is used to extract segment feature values ​​from the multi-frame pre-delay signal in the first frame data buffer pool; The second frame data buffer pool is used to buffer the multi-frame delayed signal obtained from the broadcast link; The frame data rolling splitting marker is used to rollingly split the multi-frame delayed signals in the second frame data buffer pool into multiple groups of delayed signals, and number each group of delayed signals, the number of frames of each group of delayed signals is equal to the number of frames of the pre-delayed signals, and the last frame of the last group of delayed signals is the last frame of the multi-frame delayed signals; The second segment feature value extractor is used to extract segment feature values ​​from multiple groups of delayed signals in the frame data rolling split marker; and is also used to read a group of delayed signals starting with a frame sorted by the number from the second frame data buffer pool according to the number sent by the flag indicator and extract segment feature values; The consistency comparison module is used to compare each group of delayed signal segment feature values ​​extracted by the second segment feature value extractor with the multi-frame pre-delay signal segment feature values ​​extracted by the first segment feature value extractor, and send the compared consistency score and the corresponding number to the score register; the consistency comparison module is also used to send the comparison result of the pre-delay signal segment feature value and a group of delayed signal segment feature values ​​at the beginning of the frame sorted by the number sent by the flag indicator to the result presentation and alarm module; The score register is used to receive and store the consistency score and corresponding number sent by the consistency comparison module; The flag indicator is used to retrieve the highest score group from the score register and send its number to the second segment feature value extractor; The result presentation and alarm module is used to display the consistency comparison result, and compare the consistency comparison result with a set threshold value, and issue an alarm when the consistency comparison result is lower than the set threshold value.

Images