A method for online technical review, internal review and comparison based on real-time signals

By adopting multi-dimensional detection and arbitration methods in the broadcasting service system, combining pixel analysis and AI intelligent analysis, the shortcomings of real-time signal quality and security detection of radio and television level are solved, and multi-dimensional detection and content of radio and television level programs are realized Audits reduce costs and improve work efficiency.

CN115942059BActive Publication Date: 2024-11-12HUNAN RADIO FILM & TELEVISION GRP CO LTD +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202211646369.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-21
Publication Date
2024-11-12
Estimated Expiration
2042-12-21

AI Technical Summary

Technical Problem

The existing broadcasting service system has shortcomings in ensuring the security and quality of real-time signals at the broadcast level, especially the lack of multi-dimensional detection and real-time monitoring capabilities for content, making it difficult to meet the high standards for picture quality, content and copyright. , and cost control is difficult to achieve.

Method used

Using an online technical internal audit and comparison method based on real-time signals, through multi-dimensional detection and arbitration, combined with pixel analysis, AI intelligent analysis and video fingerprint comparison, select the optimal path for signal output to ensure broadcast of radio and television programs. quality and safety.

Benefits of technology

Multi-dimensional detection and content audit of real-time broadcasting and television signals is realized, which reduces detection costs, improves work efficiency, and ensures the quality and safety of broadcast signals.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115942059B_ABST
    Figure CN115942059B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for online technical review, internal review and comparison based on real-time signals, comprising: receiving audio and video data; online technical review; internal review; comparison; arbitration; output. The present invention uses pixel analysis to conduct online technical review of real-time signals, uses AI technology for internal review, and compares video fingerprints, and the entire business system conducts arbitration based on the detection results of the three. According to the arbitration results, the final signal is comprehensively judged, and the optimal path is selected to ensure that the final PGM output is the optimal choice. The present invention integrates multiple online monitoring methods, greatly reducing detection costs and improving work efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an online technical review, internal review and comparison method based on real-time signals, a video digital signal processing method, and a broadcasting-level real-time video digital signal detection, monitoring and processing method. Background Art

[0002] In the current broadcasting business system, in order to ensure the safety of the broadcasting system, it is necessary to perform real-time detection of the signals of each node in the broadcasting link to ensure the safety of the entire broadcasting link and ensure the final safe output of the PGM signal. The existing on-site fault detection and auditing methods usually use multi-picture division to conduct online technical review and detection of faults at the picture and audio level of real-time signals. For the signals in the main and backup links, the main and backup signals are used for picture comparison. This method covers the technical review and comparison of signals and can meet the use of some scenarios. With the strict requirements for signal quality and content, the existing detection and monitoring methods can no longer meet the growing needs of customers. The most obvious demand is the lack of content review for broadcasting-level real-time signals. The broadcasting requirements of broadcasting-level programs are relatively similar to those of video signals on the Internet, including: picture quality, content, copyright, etc. must meet higher standards before they can be broadcast, and real-time monitoring is required to replace video content that does not meet the requirements at any time. On the other hand, with the development of the Internet, it has also caused a great impact on the broadcasting and television field, and cost reduction and efficiency improvement have also been recognized by everyone. How to implement online technical review, comparison, content review and multi-dimensional detection of real-time signals at a lower cost is a problem that needs to be solved. Summary of the invention

[0003] In order to overcome the problems of the prior art, the present invention proposes a method for online technical review, internal review and comparison based on real-time signals. The method performs multi-dimensional detection on real-time signals, arbitrates the detection results, makes a comprehensive judgment on the final signal according to the arbitration results, and selects the optimal path therein to ensure that the final PGM output is the optimal choice.

[0004] The object of the present invention is achieved as follows: a method for online technical review, internal review and comparison based on real-time signals, the system used in the method comprises: an input module capable of receiving multiple sources, the input module is connected to a video buffer module, the video buffer module is connected to a fingerprint extraction module, an internal review module with an AI intelligent analysis library, and an online technical review module, the fingerprint extraction module is connected to a fingerprint comparison module, the fingerprint comparison module, the internal review module, and the online technical review module are connected to an arbitration module, and the arbitration module is connected to an output module, the method comprises the following steps:

[0005] Step 1, receiving audio and video data: the input module receives the audited audio and video data in various formats, including the primary and backup, triple backup and two-way video signals of the same audio and video data;

[0006] The following steps 2-4 are performed simultaneously;

[0007] Step 2, online technical review: The online technical review module performs real-time frame-by-frame detection on real-time signals, extracts features from the collected audio and video buffers according to the categories of the detection items, calculates the indicators of the corresponding detection items, and judges whether the signal meets the quality inspection standards. It collects data for each pixel in real time, implements signal fault detection, and realizes real-time online technical review;

[0008] Based on the results of the online technical review, the current road signal is scored and the results are divided into three levels, as follows:

[0009] 1) Common faults such as still frame, silence, and front cycle sound faults are scored as 1 point;

[0010] 2) For more serious faults such as black field, color field, black field and silence, color field and silence, the faults are scored as 2 points;

[0011] 3) The most serious fault: video loss and color bar fault are 3 points;

[0012] Step 3, internal audit: The internal audit module calls the AI ​​intelligent analysis library to perform frame-by-frame verification of the real-time signal at the broadcasting level, and verifies it against the sensitive figures in the AI ​​library. After the verification is successful, an alarm is issued, and real-time verification and analysis of pornographic, bloody and violent content in the real-time signal is performed. When pornographic, bloody and violent content is detected in the signal, an alarm is issued in real time; the current signal is scored based on the AI ​​internal audit results, and the results are divided into three levels: 1 point for bloody and violent content, 2 points for sensitive figures, and 3 points for pornographic content;

[0013] Step 4, comparison: perform real-time comparison of the two sets of terminal signals of the primary and backup and the third backup. The specific process is as follows:

[0014] 1) After acquiring the signal data, the fingerprint extraction module is used to extract the fingerprint, and after extraction, the fingerprints of the two signals are submitted to the bottom layer;

[0015] 2) After obtaining the fingerprint information, the fingerprints of the two signals are matched in an asynchronous working mode. The steps are as follows: first, the frames are aligned, and then compared. If the number of subsequent mismatched frames exceeds the tolerance, realignment is performed and comparison is performed again after alignment.

[0016] 3) Detect and calculate the brightness of the signal:

[0017] a. Calculate the similarity of Y, U, and V of the two signals respectively;

[0018] b. Calculate the standard deviation of the ratio of Y, U, and V of the two paths;

[0019] c. Sort the histogram fingerprints from small to large, and then calculate the similarity;

[0020] d. Obtain the Y similarity threshold according to the “brightness overall difference threshold”;

[0021] e. If the Y similarity is lower than the threshold, the standard deviation is less than the threshold, and the U, V, and histogram similarities are higher than the threshold, it is considered that only the overall brightness is different;

[0022] 4) Calculate the similarity threshold of the two signals based on the brightness and similarity. If it exceeds the set threshold, an inconsistency prompt will be issued;

[0023] Step 5, arbitration: arbitrate the technical review, internal review, and comparison results, and conduct real-time analysis on the quality of each signal, score each signal, and obtain the final result based on the scoring results. The specific calculation results are as follows:

[0024] 1) The score of online technical review is A; the score of internal review is B; the score of comparison is C;

[0025] 2) The weight of technical review is 50%; the weight of internal review is 30%; the weight of comparison is 20%;

[0026] 3) The final total score of a certain route is M = 50%*A+30%*B+20%C;

[0027] Step 6, output: the picture with the highest score is used as the output signal, outputted through the output module, and the review is ended.

[0028] The advantages and beneficial effects of the present invention are: the present invention uses pixel analysis to conduct online technical review of real-time signals, uses AI technology for internal review, and compares video fingerprints, and the entire business system conducts arbitration based on the detection results of the three, and makes a comprehensive judgment on the final signal according to the arbitration result, selects the optimal path, and ensures that the final PGM output is the optimal choice. The present invention integrates multiple online monitoring methods, greatly reducing detection costs and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0030] Figure 1 is a system principle block diagram used by the method described in the embodiment of the present invention;

[0031] Figure 2 is a flow chart of the method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] Embodiment 1:

[0033] This embodiment is a method for online technical review, internal review and comparison based on real-time signals. The system used in the method is as follows Figure 1 The system used by the method described in this embodiment includes: an input module capable of receiving multiple information sources, the input module is connected to a video buffer module, the video buffer module is connected to a fingerprint extraction module, an internal audit module with an AI intelligent analysis library, and an online technical audit module, the fingerprint extraction module is connected to a fingerprint comparison module, the fingerprint comparison module, the internal audit module, and the online technical audit module are connected to an arbitration module, and the arbitration module is connected to an output module.

[0034] This embodiment mainly conducts online technical review, internal review and comparison of the final signal in real time. Scores are given based on the real-time online technical review results, internal review results and comparison results, and the scores are comprehensively scored using the corresponding weights, and then the scores are arbitrated to ensure that the broadcast signal is correct and of good quality.

[0035] The input module should at least have an SDI IO card, an IP IO card, a compressed or uncompressed IP network card, etc., which can receive video data signals sent by a broadcast server or network.

[0036] The output module can be connected to a broadcast network or a broadcast transmission system (such as a terrestrial television transmitter).

[0037] This embodiment mainly provides an online technical review module, an internal review module with an AI intelligent analysis library, and a comparison module, and applies the three modules to meet the business needs for image quality, content, and other detection.

[0038] The online technical review module is mainly used to review the image quality of video images, including whether each pixel correctly expresses the image, while the internal review is mainly used to review the content of the picture, such as sensitive figures in the image, pornography, etc., and the comparison is to see whether the source of the image is legal.

[0039] The method described in this embodiment includes the following steps: Figure 2 As shown:

[0040] Step 1, receiving audio and video data: the input module receives audited audio and video data in various formats, including primary, backup, and triple backup two-way video signals of the same audio and video data.

[0041] Broadcast signals are usually sent in multiple formats or resolutions to facilitate customer selection. Since the present embodiment is at the end of the broadcast, multiple formats or resolution signals of the same program can usually be received.

[0042] The following 2-4 steps are carried out simultaneously, i.e. technical review, internal review and comparison are carried out simultaneously. Multi-threaded processing can improve efficiency.

[0043] Step 2, online technical review: The online technical review module performs real-time frame-by-frame detection on the real-time signal, extracts features from the collected audio and video buffer according to the category of the detection item, calculates the indicators of the corresponding detection item, and judges whether the signal meets the quality inspection standards based on this. It collects data for each pixel in real time, implements signal fault detection, and realizes real-time online technical review.

[0044] After fault detection, the current road signal is scored according to the results of the online technical review, and the results are divided into three levels to quantify the fault so that it can be brought into the formula for calculation in the overall evaluation, as follows:

[0045] 1) Common faults: Still frame, silence, and front-end sound faults are scored 1 point. Still frame, silence, and front-end sound are usually momentary image quality deterioration, and can be quickly restored to normal broadcasting, with less visual impact, so the fault score is lower.

[0046] 2) The more serious faults such as black field, color field, black field and silence, and color field and silence are scored 2 points. Black field, color field, black field and silence, and color field and silence are more serious faults, which can be considered as static frames, silence, and prolonged front-end sound, usually lasting for a few seconds, but can be restored, with a greater visual impact, so higher scores are given.

[0047] 3) The most serious fault: video loss and color bar fault are 3 points. Video loss and color bar fault can be considered as a long period of black field, color field, black field and silence, color field and silence, which cannot be restored, causing the playback process to pause. Of course, it has the greatest visual impact, so it is given the highest score.

[0048] Step 3, internal audit: The internal audit module calls the AI ​​intelligent analysis library to perform frame-by-frame verification of the real-time signal at the broadcasting level, and verifies it against the sensitive figures in the AI ​​library. After the verification is successful, an alarm is issued, and real-time verification and analysis of pornographic, bloody and violent content in the real-time signal are performed. When pornographic, bloody and violent content is detected in the signal, a real-time alarm is issued; the current signal is scored based on the AI ​​internal audit results, and the results are divided into three levels: 1 point for bloody and violent content, 2 points for sensitive figures, and 3 points for pornographic content.

[0049] The intelligent internal audit adopts the technical route of AI analysis library, that is, to build a fault database, and continuously learn and supplement the fault database in actual applications, and continuously learn how to use fault data, and verify the video signal frame by frame, intelligently compare and analyze the image data stored in the database, and score the image content.

[0050] Step 4, comparison: perform real-time comparison of the two sets of terminal signals of the primary and backup and the third backup. The specific process is as follows:

[0051] 1) In order to achieve efficient and stable comparison results, after obtaining the signal data, the fingerprint extraction module is first used to extract the audio and video fingerprints, and then the fingerprints of the two signals are submitted to the bottom layer.

[0052] 2) After obtaining the fingerprint information, the fingerprints of the two signals are matched in an asynchronous manner. The process is divided into two steps. The first is frame alignment. After alignment, the similarity of the fingerprints of the two signals is calculated frame by frame for comparison. If the similarity is lower than the set threshold, the two signals are considered to be mismatched. If the number of consecutive mismatched frames exceeds the tolerance, the two signals are considered to be inconsistent, an alarm is triggered, and realignment is performed. After alignment, the two signals are compared frame by frame again.

[0053] 3) Further detect whether the two signals differ only in brightness, and perform detection and calculation on the brightness of the signals:

[0054] a. Calculate the similarity of Y, U, and V of the two signals respectively.

[0055] b. Calculate the standard deviation of the ratio of the two paths Y, U, and V.

[0056] c. Sort the histogram fingerprints from small to large, and then calculate the similarity.

[0057] d. Obtain the Y similarity threshold based on the “brightness overall difference threshold”.

[0058] e. If the Y similarity is lower than the threshold, the standard deviation is less than the threshold (the empirical value that can be used is 0.03), and the U, V, and histogram similarities are higher than the threshold, it is considered that only the overall brightness is different.

[0059] 4) Calculate the similarity threshold of the two signals based on the brightness and similarity. If it exceeds the set threshold, an inconsistency prompt will be given.

[0060] Step 5, arbitration: arbitrate the technical review, internal review, and comparison results, and conduct real-time analysis on the quality of each signal, score each signal, and obtain the final result based on the scoring results. The specific calculation results are as follows:

[0061] 1) The score of online technical review is A; the score of internal review is B; the score of comparison is C;

[0062] 2) The weight of technical review is 50%; the weight of internal review is 30%; the weight of comparison is 20%;

[0063] 3) The final total score of a certain route is M = 50% A + 30% B + 20% C;

[0064] Each channel can get a final evaluation score according to the above formula as the score of this channel. In the final stage, the best picture will be selected as the output signal according to the scores of each channel.

[0065] Step 6, output: the picture with the highest score is used as the output signal, outputted through the output module, and the review is ended.

[0066] Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred arrangement scheme, a person skilled in the art should understand that the technical solution of the present invention (such as the system used, the type of input or output signal, the sequence of steps, etc.) can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for online technical review, internal review and comparison based on real-time signals, the system used in the method includes: The method comprises the following steps: Step 1, receiving audio and video data: the input module receives the audited audio and video data in various formats, including the primary and backup, triple backup and two-way video signals of the same audio and video data; The following steps 2-4 are performed simultaneously; Step 2, online technical review: The online technical review module performs real-time frame-by-frame detection on real-time signals, extracts features from the collected audio and video buffers according to the categories of the detection items, calculates the indicators of the corresponding detection items, and judges whether the signal meets the quality inspection standards. It collects data for each pixel in real time, implements signal fault detection, and realizes real-time online technical review; Based on the results of the online technical review, the current road signal is scored and the results are divided into three levels, as follows: 1) Common faults such as still frame, silence, and front cycle sound faults are scored as 1 point; 2) For more serious faults such as black field, color field, black field and silence, color field and silence, the faults are scored as 2 points; 3) The most serious fault: video loss and color bar fault are 3 points; Step 3, internal audit: The internal audit module calls the AI ​​intelligent analysis library to perform frame-by-frame verification of the real-time signal at the broadcasting level, and verifies it against the sensitive figures in the AI ​​library. After the verification is successful, an alarm is issued, and real-time verification and analysis of pornographic, bloody and violent content in the real-time signal is performed. When pornographic, bloody and violent content is detected in the signal, an alarm is issued in real time; the current signal is scored based on the AI ​​internal audit results, and the results are divided into three levels: 1 point for bloody and violent content, 2 points for sensitive figures, and 3 points for pornographic content; Step 4, comparison: perform real-time comparison of the two sets of terminal signals of the primary and backup and the third backup. The specific process is as follows: 1) After acquiring the signal data, the fingerprint extraction module is used to extract the fingerprint, and after extraction, the fingerprints of the two signals are submitted to the bottom layer; 2) After obtaining the fingerprint information, the fingerprints of the two signals are matched in an asynchronous manner. This is divided into two steps. The first is frame alignment. After the frames are aligned, they are compared. If the number of subsequent mismatched frames exceeds the tolerance, realignment is performed and comparison is repeated after alignment. 3) Detect and calculate the brightness of the signal: a. Calculate the similarity of Y, U, and V of the two signals respectively; b. Calculate the standard deviation of the ratio of Y, U, and V of the two paths; c. Sort the histogram fingerprints from small to large, and then calculate the similarity; d. Get the Y similarity threshold according to the "brightness overall difference threshold"; e. If the Y similarity is lower than the threshold, the standard deviation is less than the threshold, and the U and V histogram similarities are higher than the threshold, it is considered that only the overall brightness is different; 4) Calculate the similarity threshold of the two signals based on the brightness and similarity. If it exceeds the set threshold, an inconsistency prompt will be given; Step 5, arbitration: arbitrate the technical review, internal review, and comparison results, and conduct real-time analysis on the quality of each signal, score each signal, and obtain the final result based on the scoring results. The specific calculation results are as follows: 1) The score of online technical review is A; the score of internal review is B; the score of comparison is C; 2) The weight of technical review is 50%; the weight of internal review is 30%; the weight of comparison is 20%; 3) The final total score of a certain route is M = 50% A + 30% B + 20% C; Step 6, output: the picture with the highest score is used as the output signal, outputted through the output module, and the review is ended.

Citation Information

Patent Citations

  • Safe broadcast monitoring method based on video and audio comparison

    CN102006498A

  • Method for detecting video and audio quality of digital television program file

    CN102006499A

  • Streaming media signal quality detection method and system

    CN113378633A

  • Auditing video analytics through essence generation

    US20110280547A1