A method and system for identifying black edges in broadcast and television broadcast monitoring

Abstract

The present invention discloses a method and system for identifying black edges in broadcast and television relay monitoring, and mainly relates to the technical field of IPTV relay monitoring. The method comprises the following steps: obtaining a live stream and a relay stream, and decoding two video streams; obtaining key frames and key frame lists of the live stream and the relay stream; matching the obtained key frames, calculating the image similarity between the live stream and the relay stream key frame lists, and performing black edge identification if the image similarity is greater than a threshold, and obtaining the size of the black edges on the top, bottom, left, and right sides and the black edge adding mode; after obtaining the black edge adding mode, the decoded image is first processed according to the black edge size and the black edge adding mode, and a new image is obtained for comparison. The beneficial effect of the present invention is that it can make up for the lack of processing of black edges in relay monitoring, avoid a large number of false alarms caused by black edges, and improve the accuracy of relay monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of IPTV relay monitoring, in particular to a method and system for identifying black edges in broadcast television relay monitoring. Background Art

[0002] Major programs, such as sports events, need to be broadcast by various TV stations. However, some TV stations will add a few pixels of black edges to the original picture during the broadcast, and there are three modes of adding: translation, overlay and zoom. For the audience, a small amount of black edges does not affect the viewing, but when using algorithms for monitoring, it will cause many false alarms. Therefore, in broadcast monitoring, it is necessary to identify the black edges and their patterns, and remove them in a targeted manner.

[0003] At present, the methods for broadcast monitoring do not process black edges, that is, they assume that there are no black edges by default, which is not in line with the actual situation. CN114979736B, CN116012749A and CN112995535A are mainly about black edge detection, but the black edge processing method is not suitable for broadcast monitoring.

[0004] Therefore, there is an urgent need for a black edge recognition method and system in radio and television broadcast monitoring to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for identifying black edges in radio and television broadcast monitoring, which can make up for the lack of black edge processing in broadcast monitoring, avoid a large number of false alarms caused by black edges, and improve the accuracy of broadcast monitoring.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] On the one hand, a method for identifying black edges in broadcast and television broadcast monitoring is provided, comprising the following steps:

[0008] S1: Obtain the live stream and the relay stream, and decode the two video streams;

[0009] S2: Obtain key frames and key frame lists of live stream and relay stream;

[0010] S3: Match the acquired key frames, calculate the image similarity between the key frame lists of the live stream and the broadcast stream, and perform black border recognition if the image similarity is greater than a threshold to obtain the size of the black borders on the top, bottom, left, and right sides and the black border adding mode;

[0011] S4: After obtaining the black border adding mode, the decoded image is first processed according to the black border size and the black border adding mode, and a new image is obtained and then compared.

[0012] Preferably, the decoding of two video streams in step S1 is specifically as follows:

[0013] Create a UDP network connection, receive data packets in real time, identify video stream data packets based on ffpmeg, obtain parameter information, decode video frames, and store them in the message queue.

[0014] Preferably, the step S2 of obtaining the key frames of the live stream and the relay stream is specifically as follows:

[0015] Convert the image to a grayscale image, count the number of pixels of each grayscale, and obtain a grayscale histogram. Calculate the average of the difference between the grayscale histograms of the previous and next frames. If this value is greater than the threshold, the next frame is determined to be a key frame and added to the key frame list.

[0016] Preferably, the step S3 matches the acquired key frames, specifically:

[0017] After determining that a frame is a key frame, calculate its image similarity with all key frames of another video stream:

[0018] If the similarity is greater than the threshold, the two images are considered to have the same content, and the size of the black border and the adding mode are identified based on the two images and the similarity;

[0019] If the key frame cannot be matched within the set time, the black edges of the broadcast image are calculated using the latest image, and the threshold is adjusted according to the number of black edges. The larger the black edges, the smaller the threshold, and the match is performed again.

[0020] Preferably, the picture similarity is calculated in step S3, specifically:

[0021]

[0022] in, and Respectively, images and The average pixel value of and Respectively, images and The pixel variance of For images and The covariance between and are two constants;

[0023] The following steps are involved:

[0024] S31: Scale the two images proportionally, specifically:

[0025] If the image is a high-definition image, it will be reduced to 192 pixels × 108 pixels;

[0026] If the image is a standard definition image, it will be reduced to 180 pixels × 144 pixels;

[0027] S32: Convert the image into YUV format and extract the Y component to obtain a two-dimensional matrix;

[0028] S33: Use an 11×11 sliding window to slide on the Y component, calculate the SSIM value, and then take the average as the similarity measure of the two images.

[0029] Preferably, the sizes of the black borders on the top, bottom, left and right sides are obtained in step S3, specifically:

[0030] Get the size of the upper black border:

[0031] For the key frame of the broadcast channel, calculate the mean and maximum value of all pixels in the first row. If the mean value is less than the threshold and the maximum value is also less than the threshold, it is considered to be a black border row. Otherwise, it is not. If it is judged to be a black border row, continue to judge whether the next row is a black border row, until it is not a black border row, and calculate the size of the top black border;

[0032] Get the size of the bottom black border: start from the last row, the calculation process is the same as getting the size of the top black border;

[0033] Get the size of the left black border: start from the first column, the calculation process is the same as getting the size of the upper black border;

[0034] Get the size of the right black border: start calculating from the last column. The calculation process is the same as that of getting the size of the upper black border.

[0035] Preferably, the step S4 is specifically:

[0036] S41: If the sizes of the upper, lower, left, and right black edges are all 0, determine whether the original similarity of the image is greater than the key frame matching threshold. If so, it is considered that there is no black edge. Otherwise, re-match the key frame and perform black edge recognition.

[0037] S42: In video scaling mode: cut off the black edge of the broadcast image, enlarge it to the original size, and calculate the similarity with the live image, which is recorded as sim_scale;

[0038] In video overlay mode: cut off the black border of the broadcast image, and correspondingly cut off the live image, calculate the similarity, and record it as sim_cover;

[0039] In video pan mode:

[0040] Calculate the left and right black edges:

[0041] If there is a black border on the left, the black border on the left of the broadcast image is cut off, and the border of the same width on the right of the live image is cut off, and the similarity is calculated, which is recorded as sim_left; if there is a black border on the right, the black border on the right of the broadcast image is cut off, and the border of the same width on the left of the live image is cut off, and the similarity is calculated, which is recorded as sim_right; if sim_left is greater than sim_right and sim_left is greater than sim, the black border on the left is considered to be in translation mode, and the black border on the right is set to zero; if sim_right is greater than sim_left and sim_right is greater than sim, the black border on the right is considered to be in translation mode, and the black border on the left is set to zero; otherwise, the black borders on both sides are considered not in translation mode, and both sides are set to zero;

[0042] Calculate the upper and lower black edges:

[0043] If there is a black border at the top, the top black border of the broadcast image is cut off, and the border of the same width at the bottom of the live image is cut off to calculate the similarity, which is recorded as sim_top; if there is a black border at the bottom, the bottom black border of the broadcast image is cut off, and the border of the same width at the top of the live image is cut off to calculate the similarity, which is recorded as sim_bottom. If sim_top is greater than sim_bottom and sim_top is greater than sim, the top black border is considered to be in translation mode, and the bottom black border is set to zero; if sim_bottom is greater than sim_top and sim_bottom is greater than sim, the bottom black border is considered to be in translation mode, and the top black border is set to zero; otherwise, the upper and lower black borders are considered not in translation mode, and both are set to zero;

[0044] If there is a black border in the translation mode, the upper and lower, left and right black border modes are processed uniformly to obtain the final translation mode similarity, which is recorded as ssim_move;

[0045] S43: Compare the sizes of sim_scale, sim_cover and ssim_move. If the maximum value is greater than the key frame matching threshold, the mode corresponding to the maximum value is taken as the black edge mode. Otherwise, re-match the key frame and perform black edge recognition.

[0046] On the other hand, a system based on the black edge recognition method in broadcast and television broadcast monitoring as described above is provided, comprising:

[0047] The data acquisition module is used to: acquire the live stream and the relay stream, and decode the two video streams; acquire the key frames and key frame lists of the live stream and the relay stream;

[0048] The data analysis module is used to match the acquired key frames, calculate the image similarity between the key frame lists of the live stream and the broadcast stream, and perform black border recognition if the image similarity is greater than a threshold to obtain the size of the black borders on the top, bottom, left, and right sides and the black border adding mode;

[0049] The data comparison module is used to: after obtaining the black border adding mode, the decoded image is first processed according to the black border size and black border adding mode, and the new image is then compared

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] The present invention provides a method and system for identifying black edges in broadcast and television broadcast monitoring. First, a live video stream and a broadcast video stream are decoded, and then key frames are extracted based on the difference in grayscale histograms. Then, the similarity between the live stream key frames and the broadcast stream key frames is calculated based on the SSIM structural similarity. If it is greater than a threshold, it is considered that the match is successful. Then, the size of the black edge and the adding mode are identified based on the two pictures and the similarity. The subsequent broadcast monitoring task can first process the picture according to the black edge mode and then perform a comparison.

[0052] The present invention solves the problem of inaccurate monitoring caused by adding black edges to the broadcast video stream, and significantly reduces false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the method of the present invention;

[0054] Figure 2 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the application equally.

[0056] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0057] In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. Relevant scientific research or technical personnel in this field can determine the specific meanings of the above terms in the present invention according to specific circumstances, and they should not be understood as limiting the present invention.

[0058] Example:

[0059] like Figure 1 As shown, this embodiment provides a method for identifying black edges in broadcast and television broadcast monitoring, comprising the following steps:

[0060] S1: Obtain the live stream and the relay stream, and decode the two video streams;

[0061] S2: Obtain key frames and key frame lists of live stream and relay stream;

[0062] S3: Match the acquired key frames, calculate the image similarity between the key frame lists of the live stream and the broadcast stream, and perform black border recognition if the image similarity is greater than a threshold to obtain the size of the black borders on the top, bottom, left, and right sides and the black border adding mode;

[0063] S4: After obtaining the black border adding mode, the decoded image is first processed according to the black border size and the black border adding mode, and a new image is obtained and then compared.

[0064] In this embodiment, take the news program broadcast of TV station A to TV station B as an example:

[0065] 1. The news program starts at 19:00. Pull the UDP streams of TV station A and TV station B respectively, obtain video information, decode the video streams to get pictures, scale them to 960×540 resolution, and save them to the message queue;

[0066] 2. Read data from two message queues respectively, calculate the grayscale histogram of the picture, subtract it from the grayscale histogram of the previous frame and calculate the average, save the data and the value to a list, if the value is greater than 400, add the frame and its index to the key frame sequence, match the frame with the key frame sequence of another video stream, that is, calculate the SSIM similarity value of the picture and the picture in another sequence in a reverse loop, if the value is greater than 0.85, the picture is considered to be matched successfully;

[0067] 3. If the key frame cannot be matched within 15 seconds, the black edges are calculated through the latest image, and the threshold is adjusted according to the number of black edges. If the number of black edges is greater than 0 and less than 10, the threshold is reduced to 0.8; if it is greater than 10 and less than 20, the threshold is reduced to 0.7; if it is greater than 20, the threshold is reduced to 0.6, and the match is repeated;

[0068] 4. If the match is successful, identify the size and mode of the black edge based on this pair of key frames:

[0069] First, for the key frames of the broadcast channel, start from the first row and determine whether it is a black-bordered row row by row. Calculate the mean and maximum value of all pixels in this row. If the mean is less than 30 and the maximum value is also less than 50, it is considered to be a black-bordered row, otherwise it is not. If not, exit the loop and the size of the top black border is the current row number minus one. Then for the key frames of the live channel, calculate the size of the top black border. If the difference between the size of the black border of the live picture and the size of the black border of the broadcast picture is very small, it is considered that there is no top black border. If the size of the black border of the live picture is zero, it is considered that the size of the top black border is the size of the black border of the broadcast picture. In other cases, the group of key frames is considered inappropriate and the key frames are re-matched;

[0070] The size of the bottom black border starts from the last row, and it is determined row by row whether it is a black border row. Calculate the mean and maximum value of all pixels in this row. If the mean value is less than 30 and the maximum value is also less than 50, it is considered a black border row, otherwise it is not. If not, exit the loop, and the size of the bottom black border is the image height minus the current number of rows. If the difference between the black border size of the live image and the black border size of the broadcast image is very small, it is considered that there is no bottom black border. If the black border size of the live image is zero, it is considered that the bottom black border size is the black border size of the broadcast image. In other cases, it is considered that the group of key frames is inappropriate and the key frames are re-matched.

[0071] The size of the black border on the left starts from the first column, and it is determined whether it is a black border column column by column. Calculate the mean and maximum value of all pixels in this column. If the mean is less than 30 and the maximum value is also less than 50, it is considered a black border column, otherwise it is not. If not, exit the loop, and the size of the black border on the left is the current column number minus one. Then calculate the size of the black border on the left for the key frame of the live channel. If the difference between the black border size of the live picture and the black border size of the broadcast picture is very small, it is considered that there is no black border on the left. If the black border size of the live picture is zero, it is considered that the black border size on the left is the black border size of the broadcast picture. In other cases, the group of key frames is considered inappropriate and the key frames are re-matched.

[0072] The size of the black border on the right starts from the last column, and it is determined whether it is a black border column column by column. Calculate the mean and maximum value of all pixels in this column. If the mean is less than 30 and the maximum value is also less than 50, it is considered a black border column, otherwise it is not. If not, exit the loop, and the size of the black border on the left is the image width minus the current number of columns. Then calculate the size of the black border on the left for the key frame of the live channel. If the difference between the black border size of the live picture and the black border size of the broadcast picture is very small, it is considered that there is no black border on the left. If the black border size of the live picture is zero, it is considered that the black border size on the left is the black border size of the broadcast picture. In other cases, the group of key frames is considered inappropriate and the key frames are re-matched.

[0073] If the sizes of the upper, lower, left, and right black edges are all 0, determine whether the similarity is greater than 0.9. If so, it is considered that there is no black edge. Otherwise, re-match the key frames and perform black edge recognition.

[0074] Assuming it is in zoom mode, cut off the black edge of the broadcast image, enlarge it to the original size, and calculate the similarity with the live broadcast image, which is recorded as sim_scale.

[0075] Assuming it is in overlay mode, cut off the black border of the broadcast image, and correspondingly cut off the live image, calculate the similarity, and record it as sim_cover;

[0076] Assuming it is in translation mode, first calculate the left and right black edges:

[0077] If there is a black border on the left, the black border on the left of the broadcast image is cut off, and the border of the same width on the right of the live image is cut off, and the similarity is calculated, which is recorded as sim_left; if there is a black border on the right, the black border on the right of the broadcast image is cut off, and the border of the same width on the left of the live image is cut off, and the similarity is calculated, which is recorded as sim_right. If sim_left is greater than sim_right and sim_left is greater than sim, it is temporarily considered that the black border on the left is in translation mode, and the black border on the right is set to zero; if sim_right is greater than sim_left and sim_right is greater than sim, it is temporarily considered that the black border on the right is in translation mode, and the black border on the left is set to zero; otherwise, it is considered that the black borders on both sides are not in translation mode, and both sides are set to zero.

[0078] Then calculate the upper and lower black edges:

[0079] If there is a black border at the top, the top black border of the broadcast image is cut off, and the border of the same width at the bottom of the live image is cut off to calculate the similarity, which is recorded as sim_top; if there is a black border at the bottom, the bottom black border of the broadcast image is cut off, and the border of the same width at the top of the live image is cut off to calculate the similarity, which is recorded as sim_bottom. If sim_top is greater than sim_bottom and sim_top is greater than sim, the top black border is temporarily considered to be in translation mode, and the bottom black border is set to zero; if sim_bottom is greater than sim_top and sim_bottom is greater than sim, the bottom black border is temporarily considered to be in translation mode, and the top black border is set to zero; otherwise, the upper and lower black borders are considered not in translation mode, and both are set to zero.

[0080] If there is a black border in the translation mode, the upper and lower, left and right black border modes are processed uniformly to obtain the final translation mode similarity, which is recorded as ssim_move.

[0081] Then compare the sizes of sim_scale, sim_cover and ssim_move. If the maximum value is greater than sim and greater than 0.9, the mode corresponding to the maximum value is taken as the black edge mode. Otherwise, re-match the key frames and perform black edge recognition.

[0082] 5. After the key frames are matched and the black edge pattern is recognized, the two video stream images are processed first, and then compared frame by frame.

[0083] The processing method is:

[0084] If there are black edges and the image is in zoom mode, the black edges of the broadcast stream image will be cut off and the image will be zoomed to the original size. The live stream image will not be processed.

[0085] If there are black edges and the mode is overlay, the black edges of the broadcast stream image will be cut off, and the live stream image will be cut off accordingly;

[0086] If there is a black border and it is in translation mode, first determine the left and right sides. If there is a black border on the left, cut off the black border on the left side of the broadcast stream image, and cut off the edge of the same size on the right side of the live stream image. If there is a black border on the right, cut off the black border on the right side of the broadcast stream image, and cut off the edge of the same size on the left side of the live stream image; then determine the upper and lower sides. If there is a black border on the upper side, cut off the black border on the upper side of the broadcast stream image, and cut off the edge of the same size on the lower side of the live stream image. If there is a black border on the lower side, cut off the black border on the lower side of the broadcast stream image, and cut off the edge of the same size on the upper side of the live stream image.

[0087] like Figure 2 As shown, this embodiment also provides a system based on the black edge recognition method in broadcast and television broadcast monitoring, including:

[0088] The data acquisition module is used to: acquire the live stream and the relay stream, and decode the two video streams; acquire the key frames and key frame lists of the live stream and the relay stream;

[0089] The data analysis module is used to match the acquired key frames, calculate the image similarity between the key frame lists of the live stream and the broadcast stream, and perform black border recognition if the image similarity is greater than a threshold to obtain the size of the black borders on the top, bottom, left, and right sides and the black border adding mode;

[0090] The data comparison module is used to: after obtaining the black border adding mode, the decoded picture is first processed according to the black border size and the black border adding mode, and the new picture is obtained and then compared.

[0091] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for identifying black edges in broadcast and television broadcast monitoring, characterized in that: The following steps are involved: Get the live stream and the relay stream, decode the two video streams to get the pictures, and save them to the message queue, read the data from the two message queues respectively, calculate the grayscale histogram of the picture, subtract it from the grayscale histogram of the previous frame and calculate the average, save the data and the value to the list, if the value is greater than the threshold, add the frame picture and its index to the key frame sequence, match the frame picture with the key frame sequence of another video stream, including: after determining that a frame is a key frame, calculate its similarity with all the key frames of the other video stream: if the similarity is greater than the threshold, it is considered that the two pictures have the same content, and identify the size of the black borders on the top, bottom, left and right and the adding mode based on the two pictures and the similarity, including: For the key frames of the broadcast channel, calculate the mean and maximum value of all pixels in this row starting from the first row. If the mean value is less than the threshold and the maximum value is also less than the threshold, it is considered to be a black-bordered row. Otherwise, if it is judged to be a black-bordered row, continue to judge whether the next row is a black-bordered row, until it is not a black-bordered row, calculate the size of the top black border, and then for the key frames of the live channel, calculate the size of the top black border. If the difference between the black border size of the live picture and the black border size of the broadcast picture is very small, it is considered that there is no top black border. If the black border size of the live picture is zero, it is considered that the top black border size is the same as the black border size of the broadcast picture. In other cases, it is considered that the group of key frames is inappropriate and the key frames are re-matched; Get the size of the bottom black border: start from the last row, the calculation process is the same as getting the size of the top black border; Get the size of the left black border: start from the first column, the calculation process is the same as getting the size of the upper black border; Get the size of the right black border: start from the last column, the calculation process is the same as getting the size of the upper black border; After obtaining the black border adding mode, the decoded image is first processed according to the black border size and the black border adding mode, and the new image is obtained and then compared, including: If the sizes of the upper, lower, left, and right black edges are all 0, determine whether the original similarity of the image is greater than the key frame matching threshold. If so, it is considered that there is no black edge. Otherwise, re-match the key frame and perform black edge recognition. In video scaling mode: the black edges of the broadcast image are cut off, the image is enlarged to its original size, and the similarity is calculated with the live image, which is recorded as sim_scale; In video overlay mode: cut off the black border of the broadcast image, and correspondingly cut off the live image, calculate the similarity, and record it as sim_cover; In video pan mode: Calculate the left and right black edges: If there is a black border on the left, the black border on the left of the broadcast image is cut off, and the border of the same width on the right of the live image is cut off, and the similarity is calculated, which is recorded as sim_left; if there is a black border on the right, the black border on the right of the broadcast image is cut off, and the border of the same width on the left of the live image is cut off, and the similarity is calculated, which is recorded as sim_right; if sim_left is greater than sim_right and sim_left is greater than sim, the black border on the left is considered to be in translation mode, and the black border on the right is set to zero; if sim_right is greater than sim_left and sim_right is greater than sim, the black border on the right is considered to be in translation mode, and the black border on the left is set to zero; otherwise, the black borders on both sides are considered not in translation mode, and both sides are set to zero; Calculate the upper and lower black edges: If there is a black border at the top, the top black border of the broadcast image is cut off, and the border of the same width at the bottom of the live image is cut off to calculate the similarity, which is recorded as sim_top; if there is a black border at the bottom, the bottom black border of the broadcast image is cut off, and the border of the same width at the top of the live image is cut off to calculate the similarity, which is recorded as sim_bottom. If sim_top is greater than sim_bottom and sim_top is greater than sim, the top black border is considered to be in translation mode, and the bottom black border is set to zero; if sim_bottom is greater than sim_top and sim_bottom is greater than sim, the bottom black border is considered to be in translation mode, and the top black border is set to zero; otherwise, the upper and lower black borders are considered not in translation mode, and both are set to zero; If there is a black border in the translation mode, the black border modes of the top and bottom, left and right are processed uniformly to obtain the final translation mode similarity, which is recorded as ssim_move; Compare the sizes of sim_scale, sim_cover and ssim_move. If the maximum value is greater than the key frame matching threshold, the mode corresponding to the maximum value is taken as the black edge mode. Otherwise, re-match the key frame and perform black edge recognition. If the key frame cannot be matched within the set time, the black edges of the broadcast image are calculated using the latest image, and the threshold is adjusted according to the number of black edges. The larger the black edges, the smaller the threshold, and the match is performed again.

2. The method for identifying black edges in broadcast and television broadcast monitoring according to claim 1, characterized in that: The calculation of the picture similarity is specifically as follows: in, and Respectively, images and The average pixel value of and Respectively, images and The pixel variance of For images and The covariance between and are two constants; The following steps are involved: S31: Scale the two images proportionally, specifically: If the image is a high-definition image, it will be reduced to 192 pixels × 108 pixels; If the image is a standard definition image, it will be reduced to 180 pixels × 144 pixels; S32: Convert the image into YUV format and extract the Y component to obtain a two-dimensional matrix; S33: Use an 11×11 sliding window to slide on the Y component, calculate the SSIM value, and then take the average as the similarity measure of the two images.

3. A system based on the black edge recognition method in broadcast and television broadcast monitoring as claimed in claim 1, characterized in that: include: The data acquisition module is used to obtain the live stream and the relay stream, and decode the two video streams; Get the key frames and key frame lists of live and relay streams; The data analysis module is used to match the acquired key frames, calculate the image similarity between the key frame lists of the live stream and the broadcast stream, and perform black border recognition if the image similarity is greater than a threshold to obtain the size of the black borders on the top, bottom, left, and right sides and the black border adding mode; The data comparison module is used to: after obtaining the black border adding mode, the decoded picture is first processed according to the black border size and the black border adding mode, and the new picture is obtained and then compared.

Images