A method and device for automatically selecting primary and standby video channels

Abstract

The present invention provides a method and device for automatic optimization of main and standby video channels. The method comprises: inputting the main channel image and the standby channel image into a trained first recognition model respectively, obtaining the target category, and performing image still frame recognition or cutout processing based on the category; inputting the original image or the image before the cutout processing into a trained second recognition model to obtain the image category; if the image category is a black screen or a blue screen, the corresponding image is recognized as a still frame; if the image category is a normal video, the inter-frame difference method is used to perform a differential operation on the normal video image or the image after the cutout processing, if the number of pixels after the differential operation is greater than the first threshold, the image is not still frame, otherwise, the image is still frame; the main channel image and the standby channel image are optimized based on the still frame recognition result. The present invention can assist the director to complete the live broadcast task, avoid the broadcast of still frame pictures, and ensure the smoothness and stability of the live broadcast program.

Description

Technical Field

[0001] The present invention belongs to the technical field of video playback, and in particular relates to a method and a device for automatically optimizing primary and standby video channels. Background Art

[0002] In live TV broadcasting tasks, video image freezing and frame loss often occur. Image freezing refers to the phenomenon that the video playback process is stuck, the picture is frozen at a certain frame, and the frame is repeatedly broadcast. It also includes the situation where there is no moving target or light and shadow changes in the shooting scene. In the process of video image transmission and processing, image freezing often occurs on network transmission equipment. After a communication abnormality, the equipment usually continues to broadcast the last frame received until the communication is restored to display the real-time image, or displays "Video Lost", "No Video Screen", "RTSP Interaction Failed" and other prompt screens when the communication is disconnected for a long time. It usually occurs when the network communication is not smooth or disconnected. After the network communication is unblocked and the connection is restored, the image freezing phenomenon disappears.

[0003] In order to avoid the loss of key information due to video stillness and frame loss, dual channels or multiple channels are generally used to transmit one video signal at the same time, and the main and backup video channels are formed on the director terminal to back up each other. When one channel is not smooth, the backup channel image can be switched to ensure the stability and continuity of the live video. At present, most director systems use manual selection to complete the switching of the main and backup video channels. The disadvantages are: when the main and backup channel images are unstable and the signals are intermittent, the main and backup channels need to be replaced repeatedly, and the operator's reaction speed cannot keep up with the high-frequency switching requirements; for video images with little change in the shooting scene, the image still frame phenomenon is relatively hidden, which can easily mislead the director to miss the best reaction time, resulting in incomplete display of key frame images and causing director accidents. There are also automatic recognition switching modes, which use steps and methods such as image denoising, feature extraction, feature selection, and classification model construction, or use methods such as comparison of the brightness average value, average grayscale value, and grayscale variance value of the previous and next frames of the image to construct an automatic recognition method for the image still frame phenomenon, automatically discover the image still frame and link the device to switch the main and backup video channels. The problem with the automatic mode is that when there are pseudo-moving targets such as noise, interference and crosstalk signals, and superposition of active graphics such as clocks, scrolling subtitles, dynamic station logos, and communication status in the image, the processing equipment in the transmission link superimposes the above active graphics on the still frame image. The actual broadcast image is not a true still frame image, resulting in the inability of the automatic recognition technology for still frame phenomena to recognize and trigger the selection instruction. The actual use effect is not as good as the manual selection working mode. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method and device for automatically optimizing primary and standby video channels.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions.

[0006] In a first aspect, the present invention provides a method for automatically optimizing a primary and standby video channel, comprising the following steps:

[0007] Input the main channel image and the backup channel image into the trained first recognition model respectively to obtain the target category and position size, the target category includes: prompt information "video lost", dynamic characters or graphics, and space;

[0008] If the target category is the prompt information "video lost", the image is identified as a still frame;

[0009] If the target category is a dynamic character or graphic, the image is cut out to remove the dynamic character or graphic, and the cut out area is filled with a set pixel value, and the image after the cut out process is saved;

[0010] If the target category is empty, the image with an empty target category or the image before the cutout process is input into the trained second recognition model to obtain the image category, including: black screen, blue screen, normal video;

[0011] If the image category is black screen or blue screen, the corresponding image is identified as a frozen frame;

[0012] If the image category is a normal video, a frame difference method is used to perform a difference operation on the normal video image or the image after the cutout processing. If the number of pixel points after the difference operation is greater than the first threshold, the image is not frozen, otherwise, the image is frozen;

[0013] The main channel image and the backup channel image are scored based on the still frame recognition result, and the main channel image or the backup channel image with a high score is output by sending a channel switching instruction to the image switching device.

[0014] Furthermore, the first recognition model adopts a YOLO series model; by collecting images containing the prompt information "video lost" and images superimposed with dynamic characters or graphics, and labeling the collected images by categories, a training data set is established, and the first recognition model is trained using the training data set to obtain a trained first recognition model.

[0015] Furthermore, the second recognition model is an image classification model using a K-nearest neighbor algorithm; by collecting black screen, blue screen and normal video images, and labeling the collected images by categories, calculating the average grayscale value and grayscale variance value of each image, generating a training data set, and using the training data set to train the second recognition model to obtain a trained second recognition model;

[0016] The calculation formulas for the average grayscale value and grayscale variance value of an image are:

[0017]

[0018]

[0019] X i =0.299R i +0.587G i +0.114B i (3)

[0020] Where, X i is the gray value of the i-th pixel, R i , G i , B i are the R, G, and B components of the i-th pixel, respectively. is the average gray value, Δ is the gray variance value, i=1,2,......,N, N is the number of pixels.

[0021] Furthermore, before the inter-frame difference method is used to perform the difference operation, the method further includes: performing low-pass filtering on the image to filter out noise and interference signals in the image.

[0022] Furthermore, the low-pass filtering of the image includes: first performing FFT transformation on the image, and then performing low-pass filtering on the image in the frequency domain.

[0023] Furthermore, after performing the difference operation using the inter-frame difference method, the method further includes: filtering out noise and interference signals of the image by eliminating active pixel points whose pixel values ​​are less than a second threshold.

[0024] Furthermore, the method of performing differential operation using the inter-frame differential method includes:

[0025] Get the saved image after the cutout process. The cutout area of ​​the k-th frame image is [a1, a2, ..., a n ], the cutout area of ​​the k-1th frame image is [b1,b2,...,b m ], n, m are the number of cutout regions, k = 1, 2, ...;

[0026] Calculation: a1∪a2∪...∪a n , b1∪b2∪...∪b m , in the k-th frame image b1∪b2∪...∪b m The pixel value set for area filling is a1∪a2∪...∪a in the k-1th frame image. n The pixel value set for area filling;

[0027] Perform a differential operation on the k-th frame image and the k-1-th frame image. The differential operation formula is:

[0028] D k (i,j)=A k (i,j)-A k-1 (i,j)(4)

[0029] In the formula, A k (i,j),A k-1 (i, j) are the grayscale values ​​of the pixels in the i-th row and j-th column of the k-th frame image and the k-1-th frame image respectively. k (i,j) is the difference, k=1,2,......

[0030] Furthermore, the method for scoring the main channel image and the backup channel image includes:

[0031] The image that is identified as a frozen frame by a black screen or a blue screen is scored as 0;

[0032] The image that is identified as a frozen frame by the prompt message "Video Lost" is scored as 0;

[0033] The image identified as a still frame by a normal video is scored as 1;

[0034] The image identified as a non-still frame from a normal video is scored as 2.

[0035] Furthermore, the method also includes setting a minimum channel switching time interval.

[0036] In a second aspect, the present invention provides an automatic optimization device for primary and standby video channels, comprising:

[0037] The first recognition module is used to input the main channel image and the backup channel image into the trained first recognition model respectively to obtain the target category and position size, and the target category includes: prompt information "video lost", dynamic characters or graphics, and space;

[0038] A first still frame recognition module, configured to recognize the image as a still frame if the target category is the prompt information "video lost";

[0039] A cutout processing module is used to cut out the image if the target category is a dynamic character or graphic, remove the dynamic character or graphic, fill the cutout area with a set pixel value, and save the image after the cutout processing;

[0040] The second recognition module is used to input the image with the empty target category or the image before the cutout process into the trained second recognition model if the target category is empty, and obtain the image category, including: black screen, blue screen, normal video;

[0041] A second still frame recognition module, configured to recognize the corresponding image as a still frame if the image category is a black screen or a blue screen;

[0042] A third still frame recognition module is used for performing a difference operation on the normal video image or the image after the cutout processing by using an inter-frame difference method if the image category is a normal video. If the number of pixel points after the difference operation is greater than a first threshold, the image is not still frame, otherwise, the image is still frame;

[0043] The image switching module is used to score the main channel image and the backup channel image based on the still frame recognition result, and output the main channel image or the backup channel image with a high score by sending a channel switching instruction to the image switching device.

[0044] Compared with the prior art, the present invention has the following beneficial effects.

[0045] The present invention solves the problem that static frames of signal sources cannot be automatically identified due to active graphics such as clocks, scrolling subtitles, dynamic station logos, communication status, etc. superimposed on the image by processing equipment in the transmission link by adding the steps of dynamic character or graphic recognition, positioning and image cutting processing. It provides a fast and accurate image static frame detection and automatic processing means, which can be applied to various audio and video director systems to assist directors in completing live broadcast tasks, avoid the broadcast of static frames, and ensure the smoothness and stability of live broadcast programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention is a flowchart of a method for automatically optimizing primary and standby video channels according to an embodiment of the present invention.

[0047] Figure 2 It is a schematic diagram of the hardware architecture involved in the embodiment of the present invention.

[0048] Figure 3 The present invention is a block diagram of a device for automatically optimizing primary and standby video channels according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] Figure 1 The present invention is a flowchart of a method for automatically selecting a primary and standby video channel according to an embodiment of the present invention, comprising the following steps:

[0051] Step 101, input the main channel image and the backup channel image into the trained first recognition model respectively, and obtain the target category and position size, the target category includes: prompt information "video lost", dynamic characters or graphics, and space;

[0052] Step 102, if the target category is the prompt information "video lost", the image is identified as a still frame;

[0053] Step 103, if the target category is a dynamic character or graphic, the image is cut out to remove the dynamic character or graphic, and the cut out area is filled with a set pixel value, and the image after the cut out processing is saved;

[0054] Step 104, if the target category is empty, input the image with the target category empty or the image before the cutout process into the trained second recognition model to obtain the image category, including: black screen, blue screen, normal video;

[0055] Step 105, if the image category is a black screen or a blue screen, the corresponding image is identified as a still frame;

[0056] Step 106, if the image category is a normal video, a frame difference method is used to perform a difference operation on the normal video image or the image after the cutout processing. If the number of pixel points after the difference operation is greater than a first threshold, the image is not frozen, otherwise, the image is frozen;

[0057] Step 107 , scoring the main channel image and the backup channel image based on the still frame recognition result, and outputting the main channel image or the backup channel image with a higher score by sending a channel switching instruction to the image switching device.

[0058] The hardware architecture diagram of this embodiment is as follows: Figure 2 As shown, it includes an image switching device, an image acquisition device, a software operation server and a network device. Steps 101 to 107 of the method described in this embodiment are executed in the software operation server. The software operation server identifies whether the main channel image and the spare channel image captured by the image acquisition device have been still frames, and scores the main channel image and the spare channel image based on the recognition result, and uses the main channel image or the spare channel image with a high score as the preferred output image, and sends a channel switching instruction to the image switching device through the network device. The channel switching device outputs the preferred image to the main program of the director system according to the instruction.

[0059] In this embodiment, step 101 is mainly used for target detection of the main channel image and the backup channel image. In this embodiment, the target category, the center position, width and height of the target detection frame are obtained by inputting the main channel image and the backup channel image into the trained first recognition model respectively, and the recognition result is given in a list form, such as [(category, coordinate x, coordinate y, width, height, confidence), ....]. The target category includes the prompt information "video loss", dynamic characters or graphics and space. In addition to "video loss", the prompt information may also be "no video screen", "RTSP interaction failure", etc. Dynamic characters or graphics refer to images superimposed with clocks, scrolling subtitles, dynamic logos, communication status icons, etc. Empty means that the above two types of targets are not detected.

[0060] In this embodiment, steps 102 to 104 are mainly used to perform still frame recognition or further image category recognition according to the recognition result of step 101 .

[0061] Step 102 directly identifies the image with the target category of prompt information "video lost" (or "no video screen", "RTSP interaction failed", etc.) as a still frame.

[0062] Step 103 performs a cutout process on the image whose target category is dynamic characters or graphics, removes the superimposed information such as clocks, scrolling subtitles, dynamic logos, communication status icons, etc., and fills the cutout area with the set pixel value, and then saves the image after the cutout process. Filling the cutout area with the set pixel value is to avoid affecting the inter-frame difference processing result of step 107. The inter-frame difference processing of the image will perform a subtraction process on the pixel values ​​of the pixels at the same position of the two frames of images. Filling equal pixel values ​​can make the difference result 0, which means that the pixel value here has no change, that is, there is no active pixel, and excludes the influence of pseudo-active pixels of graphics such as clocks, scrolling subtitles, dynamic logos, and communication status icons. Saving the image after the cutout process is to avoid the coordinates and width and height values ​​or quantity changes and shape changes of the dynamic logo and other graphics in the recognition results of the previous and next frame images affecting the subsequent inter-frame difference results.

[0063] Step 104 inputs the image with empty target category or the image before the cutout process into the trained second recognition model to perform supplementary recognition on the image to obtain the image category. The second recognition model mainly recognizes images with black screen and blue screen, and recognizes three types of images, namely black screen, blue screen and normal video.

[0064] In this embodiment, steps 105 and 106 are mainly used to perform still frame recognition according to the recognition result of the second recognition model in step 104 .

[0065] Step 105 directly identifies the image whose image category is black screen or blue screen as a still frame.

[0066] Step 106 uses the inter-frame difference method to perform a difference operation on the image whose image category is a normal video or the image after the cutout processing, and then performs still frame recognition based on the number of pixels after the difference operation. Since the pixel values ​​of the pixels of the fixed objects in the image remain unchanged in several consecutive frames, they are offset (the pixel values ​​become 0) after the difference operation. The pixels after the difference operation are the pixels of the moving objects (and noise, interference), that is, active pixels. Therefore, moving target detection can be performed based on the number of pixels. If the number of pixels is greater than the set threshold, that is, the first threshold, the detection considers that the moving target is detected, that is, the image is not still; otherwise, the image is considered to be still.

[0067] In this embodiment, step 107 is mainly used to evaluate the main channel image and the backup channel image according to the still frame recognition result, so as to output the preferred image. In this embodiment, the main channel image and the backup channel image are scored based on the still frame recognition result to achieve quantitative evaluation of the two channel images. The main channel image or the backup channel image with a high score is used as the preferred output image, and a channel switching instruction is sent to the image switching device through the network device. The channel switching device outputs the preferred image to the main program of the director system according to the instruction.

[0068] As an optional embodiment, the first recognition model adopts a YOLO series model; by collecting images containing the prompt information "video lost" and images with superimposed dynamic characters or graphics, and annotating the collected images, a training data set is established, and the first recognition model is trained using the training data set to obtain a trained first recognition model.

[0069] The present embodiment provides a technical solution for the first recognition model. The first recognition model of the present embodiment adopts the YOLO series model. The YOLO series model has the advantages of good real-time performance (fast speed) and high detection accuracy. Since the image categories recognized by the first recognition model include images containing prompt information "video lost" and images superimposed with dynamic characters or graphics such as clocks, it is necessary to establish a training data set including images containing prompt information "video lost" and images of dynamic characters or graphics. Of course, the collected images must be annotated, that is, the category, center coordinates, width and height of the target are annotated on the RGB image. Then the first recognition model is trained using the training data set. In the prediction stage, the main channel image and the backup channel image are respectively input into the trained first recognition model, and the target category can be output.

[0070] As an optional embodiment, the second recognition model is an image classification model using a K-nearest neighbor algorithm; by collecting black screen, blue screen and normal video images, and labeling the collected images by category, calculating the average grayscale value and grayscale variance value of each image, generating a training data set, and using the training data set to train the second recognition model to obtain a trained second recognition model;

[0071] The calculation formulas for the average grayscale value and grayscale variance value of an image are:

[0072]

[0073]

[0074] X i =0.299R i +0.587G i +0.114B i (3)

[0075] Where, X i is the gray value of the i-th pixel, R i , G i , B i are the R, G, and B components of the i-th pixel, respectively. is the average gray value, Δ is the gray variance value, i=1,2,......,N, N is the number of pixels.

[0076] This embodiment provides a technical solution for the second recognition model. The second recognition model of this embodiment is an image classification model using the K-nearest neighbor algorithm KNN. Like the first recognition model, a training data set must be established first. The difference is that the image materials constituting the training data set have become black screens, blue screens and normal video images (including crosstalk signals, clocks, subtitles, station logos, device status icons, etc.), and the image materials must be subjected to grayscale feature extraction, and the average grayscale value and grayscale variance value of each image must be calculated. The calculation formula is as shown in formulas (1) to (3), and the training data set is generated by the average grayscale value and grayscale variance value of the image. Since the number of different black screen and blue screen images that actually appear is small, generally only slightly different in color or different in corner marks, different crosstalk signals, it does not meet the general deep neural network's requirements for the number of data sets; and extracting the grayscale feature value of the image, using the K-nearest neighbor algorithm to complete the black screen, blue screen or normal image classification task, from the establishment of the training data set to training and classification, the cost-effectiveness is high, and the accuracy is good. Therefore, the second recognition model of this embodiment uses the K-nearest neighbor algorithm and uses the grayscale feature value to establish a training data set. The first recognition model needs to identify whether the image contains prompt information such as "video lost" and whether there are dynamic characters or graphics such as clocks superimposed. This cannot be recognized by grayscale features, and it is necessary to extract the edge features of the graphics. The traditional sliding frame method is too inefficient and cannot meet the requirements in terms of accuracy and timeliness for the variants of the above graphics. The convolutional neural network YOLO series model is characterized by fast speed, high accuracy and good practicality. Its training data set consists of RGB images with labeled object categories and positions and sizes, rather than grayscale feature data.

[0077] As an optional embodiment, the method further includes: performing low-pass filtering on the image to filter out noise and interference signals in the image before using the inter-frame difference method to perform the difference operation.

[0078] In order to reduce the adverse effects of noise and crosstalk signals on differential operations, this embodiment adds a step of low-pass filtering the image before differential operations. Low-frequency information in the image corresponds to the grayscale components that change slowly in the image, and high-frequency information corresponds to the grayscale components that change quickly in the image, and also corresponds to the contours, edges, noise and crosstalk signals in the image. Therefore, by performing low-pass filtering on the image, noise and crosstalk signals in the image can be removed.

[0079] As an optional embodiment, the performing low-pass filtering on the image includes: first performing FFT transformation on the image, and then performing low-pass filtering on the image in the frequency domain.

[0080] This embodiment provides a technical solution for low-pass filtering an image. The filtering process of an image can be performed in the time domain or in the frequency domain. This embodiment adopts the latter solution. First, the image is subjected to a fast Fourier transform FFT to convert the time domain signal into a frequency domain signal, and then the image is subjected to a low-pass filtering process in the frequency domain. In order to improve the transformation speed of FFT, a two-dimensional FFT transformation can be performed on the image. The separability of the two-dimensional discrete Fourier transform is used to decompose the two-dimensional FFT transform into two one-dimensional FFT transforms, and the two-dimensional FFT transform is obtained by calculating two one-dimensional FFTs. The number of rows and columns of image pixels is 2 to the nth power. If it is not satisfied, it is achieved by zero padding. Then, according to the translation property of the Fourier transform, the zero-frequency component in the image spectrum is moved to the center position of the frequency domain image; a mask is set and multiplied with the frequency domain to weaken the high-frequency signal of the image, wherein the mask needs to be iteratively modified according to the actual test results; finally, an inverse Fourier transform is performed, first a one-dimensional inverse Fourier transform is performed on the column, and then a one-dimensional inverse Fourier transform is performed on the row.

[0081] As an optional embodiment, after performing the difference operation using the inter-frame difference method, the method further includes: filtering out noise and interference signals of the image by eliminating active pixel points whose pixel values ​​are less than a second threshold.

[0082] In order to improve the accuracy of moving target detection (or still frame recognition), this embodiment further filters out the noise and interference signals in the image after the differential operation. Unlike the aforementioned low-pass filtering method, this embodiment adopts a threshold processing method, that is, removes pixels in the image whose pixel values ​​are less than the second threshold. The key to this method is the setting of the threshold. Insufficient or excessive suppression will lead to misjudgment. Since the elimination of noise and interference signals cannot be directly judged based on the image histogram, it is necessary to obtain a reasonable threshold through iterative testing and data accumulation based on the denoising effect of the actual noise and interference signals. For example, considering the interference of subtitles such as station logos in the video signal, the second threshold of the normal video picture can be set at about 20, and the second threshold of the blue screen video picture can be set at about 70 or higher.

[0083] As an optional embodiment, the method of performing differential operation using the inter-frame differential method includes:

[0084] Get the saved image after the cutout process. The cutout area of ​​the k-th frame image is [a1, a2, ..., a n ], the cutout area of ​​the k-1th frame image is [b1,b2,...,b m ], n, m are the number of cutout regions, k = 1, 2, ...;

[0085] Calculation: a1∪a2∪...∪a n , b1∪b2∪...∪b m , in the k-th frame image b1∪b2∪...∪b m The pixel value set for area filling is a1∪a2∪...∪a in the k-1th frame image. n The pixel value set for area filling;

[0086] Perform a differential operation on the k-th frame image and the k-1-th frame image. The differential operation formula is:

[0087] D k (i,j)=A k (i,j)-A k-1 (i,j)(4)

[0088] In the formula, A k (i,j),A k-1 (i, j) are the grayscale values ​​of the pixels in the i-th row and j-th column of the k-th frame image and the k-1-th frame image respectively. k (i,j) is the difference, k=1,2,......

[0089] This embodiment provides a technical solution for differential operation using the inter-frame difference method. This embodiment first performs cross-cutting processing on two adjacent frames (or three frames) of images, and then performs differential operation according to formula (4). The purpose of cross-cutting processing is to make the total cutout area of ​​two adjacent frames of images the same, which is a1∪a2∪...∪a n ∪b1∪b2∪...∪b m , in order to eliminate the influence of different cutout areas on the differential operation - the output after the differential operation of different cutout areas is not 0, forming active pixels, thereby generating pseudo-moving targets.

[0090] As an optional embodiment, the method for scoring the main channel image and the backup channel image includes:

[0091] The image that is identified as a frozen frame by a black screen or a blue screen is scored as 0;

[0092] The image that is identified as a frozen frame by the prompt message "Video Lost" is scored as 0;

[0093] The image identified as a still frame by a normal video is scored as 1;

[0094] The image identified as a non-still frame from a normal video is scored as 2.

[0095] This embodiment provides a technical solution for scoring the main channel image and the backup channel image. This embodiment scores the images of the main and backup channels based on the still frame recognition results, and assigns certain scores for different still frame situations: the normal video image without still frame has the highest score, with a score of 2; the image identified as a still frame by a black screen, a blue screen and the prompt message "video lost" has the lowest score, with a score of 0; the normal video image identified as a still frame has a medium score, with a score of 1. The image with the highest score is selected as the preferred output. According to the above scoring rules, it is easier for the two channel images to have equal scores. At this time, the preferred output is set. Generally, the preferred output is set as the main channel, that is, when the main and backup channel images have equal scores, the main channel image is output. It is worth noting that there are two situations where the scores are the same: one is that the scores of the main and backup channel images are equal for the first time after the software is run, and the output is the set preferred output; the other is that the scores are equal for non-first time, and the output is not changed at this time.

[0096] As an optional embodiment, the method further includes setting a minimum channel switching time interval.

[0097] In order to avoid viewing discomfort caused by repeated switching of the main and backup channels, this embodiment limits the channel switching time interval, that is, sets the minimum time interval for issuing the switching instruction. When a change in the score size relationship of the main and backup channel images is detected, the timing is triggered. When the minimum time interval is reached and the size relationship has not changed during this period, the switching instruction is issued. The minimum time interval is determined based on actual experience, and can be set to 0.5 seconds, for example.

[0098] Figure 3 The present invention is a schematic diagram of a device for automatically optimizing the primary and standby video channels according to an embodiment of the present invention, wherein the device comprises:

[0099] The first recognition module 11 is used to input the main channel image and the backup channel image into the trained first recognition model respectively to obtain the target category and position size, and the target category includes: prompt information "video lost", dynamic characters or graphics, and space;

[0100] A first still frame recognition module 12, configured to recognize the image as a still frame if the target category is the prompt information "video lost";

[0101] The cutout processing module 13 is used to perform cutout processing on the image if the target category is a dynamic character or graphic, remove the dynamic character or graphic, fill the cutout area with a set pixel value, and save the image after the cutout processing;

[0102] The second recognition module 14 is used to input the image with the empty target category or the image before the cutout process into the trained second recognition model if the target category is empty, and obtain the image category, including: black screen, blue screen, normal video;

[0103] A second still frame recognition module 15, configured to recognize the corresponding image as a still frame if the image category is a black screen or a blue screen;

[0104] The third still frame recognition module 16 is used for performing a difference operation on the normal video image or the image after the cutout processing by using an inter-frame difference method if the image category is a normal video. If the number of pixel points after the difference operation is greater than a first threshold, the image is not still frame, otherwise, the image is still frame;

[0105] The image switching module 17 is used to score the main channel image and the backup channel image based on the still frame recognition result, and output the main channel image or the backup channel image with a high score by sending a channel switching instruction to the image switching device.

[0106] The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0107] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for automatically selecting primary and standby video channels, characterized in that: The following steps are involved: Input the main channel image and the backup channel image into the trained first recognition model respectively to obtain the target category and position size, the target category includes: prompt information "video lost", dynamic characters or graphics, and space; If the target category is the prompt information "video lost", the image is identified as a still frame; If the target category is a dynamic character or graphic, the image is cut out to remove the dynamic character or graphic, and the cut out area is filled with a set pixel value, and the image after the cut out process is saved; If the target category is empty, proceed to the next step; Input an image with an empty target category or an image before the cutout process into the trained second recognition model to obtain image categories, including: black screen, blue screen, and normal video; If the image category is black screen or blue screen, the corresponding image is identified as a frozen frame; If the image category is a normal video, a frame difference method is used to perform a difference operation on the normal video image or the image after the cutout processing. If the number of pixel points after the difference operation is greater than the first threshold, the image is not frozen, otherwise, the image is frozen; The main channel image and the backup channel image are scored based on the still frame recognition result, and the main channel image or the backup channel image with a high score is output by sending a channel switching instruction to the image switching device.

2. The method for automatically selecting the primary and standby video channels according to claim 1, characterized in that: The first recognition model adopts the YOLO series model; by collecting images containing the prompt information "video lost" and images superimposed with dynamic characters or graphics, and labeling the collected images by category, a training data set is established, and the first recognition model is trained using the training data set to obtain a trained first recognition model.

3. The method for automatically selecting the primary and standby video channels according to claim 1, characterized in that: The second recognition model is an image classification model using a K-nearest neighbor algorithm; by collecting black screen, blue screen and normal video images, and labeling the collected images by category, calculating the average grayscale value and grayscale variance value of each image, generating a training data set, and using the training data set to train the second recognition model to obtain a trained second recognition model; The calculation formula for the average gray value and gray variance value of the image is: X i =0.299R i +0.587G i +0.114B i (3) Where, X i is the gray value of the i-th pixel, R i , G i , B i are the R, G, and B components of the i-th pixel, respectively. is the average gray value, Δ is the gray variance value, i=1,2,......,N, N is the number of pixels.

4. The method for automatically selecting the primary and standby video channels according to claim 1, characterized in that: The method also includes: performing low-pass filtering on the image to filter out noise and interference signals in the image before using the inter-frame difference method to perform the difference operation.

5. The method for automatically selecting the primary and standby video channels according to claim 4, characterized in that: The low-pass filtering process for the image includes: firstly performing FFT transformation on the image, and then performing low-pass filtering on the image in the frequency domain.

6. The method for automatically optimizing the primary and standby video channels according to claim 5, characterized in that: After performing the difference operation by using the inter-frame difference method, the method further comprises: filtering out the noise and interference signal of the image by eliminating the active pixel points whose pixel values ​​are less than the second threshold value.

7. The method for automatically optimizing the primary and standby video channels according to claim 1, characterized in that: Methods for performing differential operations using the inter-frame differential method include: Get the saved image after the cutout process. The cutout area of ​​the k-th frame image is [a1, a2, ..., a n ], the cutout area of ​​the k-1th frame image is [b1,b2,...,b m ], n, m are the number of cutout regions, k = 1, 2, ...; Calculation: a1∪a2∪...∪a n , b1∪b2∪...∪b m , in the k-th frame image b1∪b2∪...∪b m The pixel value set for area filling is a1∪a2∪...∪a in the k-1th frame image. n The pixel value set for area filling; Perform a differential operation on the k-th frame image and the k-1-th frame image. The differential operation formula is: D k (i,j)=A k (i,j)-A k-1 (i,j) (4) In the formula, A k (i,j),A k-1 (i, j) are the grayscale values ​​of the pixels in the i-th row and j-th column of the k-th frame image and the k-1-th frame image respectively. k (i,j) is the difference, k=1,2,......

8. The method for automatically optimizing the primary and standby video channels according to claim 1, characterized in that: The method for scoring the main channel image and the backup channel image includes: The image that is identified as a frozen frame by a black screen or a blue screen is scored as 0; The image that is identified as a still frame by the prompt message "Video Lost" is scored as 0; The image identified as a still frame by a normal video is scored as 1; The image identified as a non-still frame from a normal video is scored as 2.

9. The method for automatically selecting the primary and standby video channels according to claim 1, characterized in that: The method also includes setting a minimum channel switching time interval.

10. An automatic optimization device for primary and standby video channels, characterized in that: include: The first recognition module is used to input the main channel image and the backup channel image into the trained first recognition model respectively to obtain the target category and position size, and the target category includes: prompt information "video lost", dynamic characters or graphics, and space; A first still frame recognition module, configured to recognize the image as a still frame if the target category is the prompt information "video lost"; A cutout processing module is used to cut out the image if the target category is a dynamic character or graphic, remove the dynamic character or graphic, fill the cutout area with a set pixel value, and save the image after the cutout processing; The second recognition module is used to proceed to the next step if the target category is empty; Input an image with an empty target category or an image before the cutout process into the trained second recognition model to obtain image categories, including: black screen, blue screen, and normal video; A second still frame recognition module, configured to recognize the corresponding image as a still frame if the image category is a black screen or a blue screen; A third still frame recognition module is used for performing a difference operation on the normal video image or the image after the cutout processing by using an inter-frame difference method if the image category is a normal video. If the number of pixel points after the difference operation is greater than a first threshold, the image is not still frame, otherwise, the image is still frame; The image switching module is used to score the main channel image and the backup channel image based on the still frame recognition result, and output the main channel image or the backup channel image with a high score by sending a channel switching instruction to the image switching device.

Images