Real-time broadcast program emergency processing system and method

Abstract

The invention discloses a real-time broadcast program emergency processing system and method, and the system comprises a hardware layer which completes signal physical access, computing power support and data storage; the data acquisition and preprocessing layer is used for uniformly acquiring and processing multi-source heterogeneous monitoring signals, mainly realizing demodulation, descrambling and demultiplexing of the signals, code stream trimming and IP multicast, and realizing radio frequency monitoring, TS stream monitoring, environment monitoring and EFD intrusion alarm; the core algorithm layer is used for completing feature modeling, multi-channel signal comparison, abnormal classification positioning and continuous model iteration; the application service layer is used for realizing business logic processing, fault warning and automatic handling, and data query and analysis; and the user interaction layer realizes man-machine interaction, state display, model management and compliance auditing. In the aspect of broadcast program degradation identification, the accuracy and the response speed can be improved, so that the aims of reducing the error rate and the accident level of safe broadcast and improving the operation and management level of the safe broadcast are fulfilled.

Description

Technical Field

[0001] This invention relates to program broadcasting technology, and more specifically, to an emergency response system and method for real-time broadcast programs. Background Technology

[0002] Currently, emergency response for live broadcast programs heavily relies on manpower. Although most platforms have deployed real-time monitoring systems to collect, process, analyze, display, and alarm signals, they primarily focus on independent anomaly monitoring and single-point threshold alarms for each monitoring point, lacking comprehensive analysis and judgment of related broadcast links. Furthermore, anomaly judgment relies on existing anomaly libraries, resulting in slow iteration and upgrades. Therefore, the common situation is independent monitoring and handling with low levels of intelligence, leading to numerous false alarms and increasing the workload of broadcast security personnel, as well as the risk of misjudgment and misoperation. With the increasing standards for defining broadcast accidents in current broadcast management regulations, existing platforms are no longer sufficient to meet stringent broadcast security requirements. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the purpose of this invention is to provide a real-time broadcast program emergency handling system and method. This system fully utilizes broadcast transmission characteristics and integrates industry-wide models to improve the accuracy and response speed in identifying broadcast program degradation. This aims to reduce the error rate and accident level of safe broadcasting and improve the level of safe broadcasting operation and management.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] The first aspect of this invention provides a real-time broadcast program emergency handling system, comprising:

[0006] At the hardware layer, physical signal access, computing power support, and data storage are completed.

[0007] The data acquisition and preprocessing layer performs unified acquisition and processing of multi-source heterogeneous monitoring signals. It mainly realizes signal demodulation, descrambling and demultiplexing, code stream trimming, IP multicast, and realizes radio frequency monitoring, TS stream monitoring, environmental monitoring, and EFD intrusion alarm.

[0008] The core algorithm layer completes feature modeling, multi-channel signal comparison, anomaly classification and localization, and continuous model iteration.

[0009] The application service layer enables business logic processing, fault alarms and automatic handling, and data query and analysis.

[0010] The user interaction layer enables human-computer interaction, status display, model management, and compliance auditing.

[0011] Preferably, the hardware layer includes:

[0012] The signal acquisition equipment completes the dedicated unified acquisition and processing of heterogeneous signals such as ASI / SDI / IP. Each signal is equipped with dual acquisition cards, which automatically switch in case of failure.

[0013] The computing equipment uses GPU clusters and supports load balancing to maintain high availability in the event of node failures.

[0014] The storage device uses a RAID disk array to store abnormal fragments and monitoring logs.

[0015] Preferably, the core algorithm layer includes:

[0016] The multimodal feature extraction module converts the original audio and video signals into computable and comparable multimodal feature vectors;

[0017] The consistency comparison module monitors single-channel anomalies, multi-channel inconsistencies, and timing misalignments in multiple signals from the same source in broadcasting, and implements audio-visual feature similarity calculation, timing consistency verification, and consistency determination logic.

[0018] The anomaly identification module constructs a dedicated anomaly library for broadcasting based on industry anomalies and historical anomaly fragments, identifies core anomalies in broadcasting, and outputs the fault level.

[0019] The incremental learning module enables continuous iteration of the anomaly recognition module.

[0020] The second aspect of this invention provides an emergency handling method for live broadcast programs, which involves performing the following steps using the live broadcast program emergency handling system provided in the first aspect of this invention:

[0021] S1, the data acquisition and preprocessing layer performs signal acquisition, processing and conversion;

[0022] S2, multi-source synchronous calibration, uses audio and video synchronization algorithms to perform intelligent frame synchronization for signals from different nodes to compensate for the delay caused by each signal passing through intermediate devices;

[0023] S3, the core algorithm layer performs multimodal feature extraction, extracts video spatiotemporal features and audio spectrum features, and obtains near-subjective quality evaluation results through quality analysis algorithms;

[0024] S4, after intelligent frame synchronization, performs consistency comparison of associated monitoring points based on feature vectors and quality evaluation results, analyzes the structural similarity of audio and video, and provides a quantitative assessment of similarity.

[0025] S5, through the anomaly recognition module of the core algorithm layer, using multimodal content recognition technology, potential illegal content is automatically identified and marked;

[0026] S6, the application service layer performs fault assessment and determines emergency strategies based on the comparison and identification results;

[0027] S7, Emergency Response: Take appropriate action according to the emergency strategy.

[0028] Preferably, in step S1, the signal from the monitoring point is SDI, ASI, or IP, and is bypassed to the signal acquisition device of the hardware layer.

[0029] The signal acquisition equipment performs corresponding decoding, demultiplexing, bitstream trimming, and IP multicasting, and analyzes and alarms according to the TR101290 protocol requirements.

[0030] Preferably, the consistency comparison in step S4 specifically includes:

[0031] Based on the multimodal fusion feature vectors of the multi-channel signals, output the pairwise similarity matrix between the multi-channel signals;

[0032] Calculate the timing offset between the two signals and output the consistency result based on the timing offset threshold;

[0033] Based on the calculated comparison data, a judgment is made according to the rules of the logical analysis engine and the similarity of features in the abnormal feature library.

[0034] Preferably, step S5 specifically includes the following steps:

[0035] S51, Multimodal Feature Extraction, extracts effective information from raw data;

[0036] S52, Feature Alignment and Fusion, enables features from different modalities to communicate;

[0037] S53, Anomaly Recognition and Decision Making, makes a final judgment based on feature fusion.

[0038] Preferably, step S51 specifically includes:

[0039] Video feature extraction: extract spatial features of single frames, use temporal models to analyze inter-frame relationships, and identify still frames, video stuttering, and time-series anomalies such as clipping.

[0040] Audio feature extraction converts audio signals into time-frequency graphs such as Mel spectrograms to analyze audio interruptions, abnormal noise, and audio-visual synchronization issues.

[0041] Text feature extraction involves using model encoding to extract keywords and semantic content of sensitive information from text for subsequent consistency verification; and / or

[0042] Step S52 specifically includes:

[0043] Feature alignment aligns features from different modalities that exist in different vector spaces using modal connectors.

[0044] Feature fusion integrates aligned features and, through splicing, weighted averaging, or cross-attention mechanisms, enables the consistency comparison module to notice relevant audio and textual cues when analyzing videos; and / or

[0045] Step S53 specifically includes:

[0046] Anomaly detection involves enabling the consistency comparison module to learn a large amount of normal program data, thus generating alarms for abnormalities such as black screens, color bars, and silence that deviate from the normal pattern.

[0047] Quality analysis involves frame-level comparison when a lossless reference source is available, to accurately quantify the degree of mosaic and bitrate degradation.

[0048] Classification and regression are used to identify specific types of anomalies, including content violations, signal interruptions, or to predict severity.

[0049] Preferably, in step S5, the incremental learning module further enables continuous iteration of the anomaly recognition module, specifically including:

[0050] A sample set is selected and constructed, which includes regular normal samples and audio / video clips;

[0051] The sample set is cleaned and labeled;

[0052] The sample set is then used for training.

[0053] Preferably, training the sample set specifically includes:

[0054] Data preprocessing and partitioning: video is decoded into frames, audio is converted into spectrum, text is segmented, and data format and size are standardized for preprocessing. Training set, validation set and test set are partitioned proportionally.

[0055] Pre-trained model fine-tuning: Based on a model pre-trained on a large-scale general dataset, domain-adaptive fine-tuning is performed using broadcast samples;

[0056] Joint learning task to simultaneously predict the type and severity of anomalies;

[0057] Incremental learning involves periodically updating the incremental learning module with new samples.

[0058] Evaluation and optimization.

[0059] The present invention provides a real-time broadcast program emergency handling system and method that can meet the three core requirements of millisecond-level real-time emergency switching, stability, and industry-specific anomaly identification for broadcast program monitoring. It can realize a closed loop of the entire chain of signal acquisition, real-time analysis, anomaly identification, automatic emergency response, and incremental learning, and solve core problems such as multi-source signal consistency comparison, black field still frame audio distortion historical anomaly detection, and automatic fault switching in broadcast scenarios. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the structural framework of the real-time broadcast program emergency handling system of the present invention;

[0061] Figure 2 This is a flowchart illustrating the emergency handling method for real-time broadcast programs according to the present invention.

[0062] Figure 3 This is a schematic diagram of a typical broadcast link in an embodiment of the emergency handling method for real-time broadcast programs of the present invention. Detailed Implementation

[0063] To better understand the above-mentioned technical solutions of the present invention, the technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0064] Combination Figure 1 As shown, the present invention provides an emergency handling system for real-time broadcast programs, comprising:

[0065] The hardware layer handles physical signal access, computing power support, and data storage, including:

[0066] The signal acquisition equipment completes the dedicated unified acquisition and processing of heterogeneous signals such as ASI / SDI / IP. Each signal is equipped with dual acquisition cards, which automatically switch in case of failure.

[0067] The computing equipment uses GPU clusters and supports load balancing to maintain high availability in the event of node failures.

[0068] The storage device uses a RAID disk array to store abnormal fragments and monitoring logs, and the retention period complies with the broadcasting operation and maintenance specifications.

[0069] The data acquisition and preprocessing layer performs unified acquisition and processing of multi-source heterogeneous monitoring signals, resolving compatibility issues in heterogeneous environments where signals coexist. The acquired signals do not affect existing broadcast links and are applicable to all broadcast, transmission, and emergency switching stages involving multiple sources. It primarily performs signal demodulation, descrambling, and demultiplexing; bitstream trimming; IP multicasting; and implements RF monitoring, TS stream monitoring, environmental monitoring, and EFD intrusion alarms, thereby achieving signal processing and conversion, data cleaning, synchronization calibration, and compliance processing.

[0070] The core algorithm layer completes feature modeling, multi-channel signal comparison, anomaly classification and localization, and continuous model iteration, including:

[0071] The multimodal feature extraction module, along with the multimodal feature extraction engine and comparison module, forms the core analysis hub of the real-time monitoring platform for broadcast audio signals. It transforms raw audio and video signals into computable and comparable multimodal feature vectors. Through cross-modal fusion and multi-channel signal consistency verification, it provides core data support for anomaly identification. Linked with the large-scale model training closed loop, the extracted feature vectors are directly used as input to the large model, reducing training data preprocessing costs. Through the anomaly samples accumulated in the feature database, the feature extraction capability of the large model is continuously optimized. Simultaneously, the feature database enables the storage, querying, and management of feature vectors, establishing a closed-loop foundation for anomaly feature accumulation, comparison, and model iteration, providing data support for large-scale model iteration.

[0072] The consistency comparison module, a multi-channel signal consistency comparison engine, can monitor single-channel anomalies, multi-channel inconsistencies, and timing misalignments in multiple signals originating from the same source in broadcasting. It primarily implements audio / video feature similarity calculation, timing consistency verification, and consistency determination logic. Feature similarity calculation outputs a pairwise similarity matrix between multiple signals based on the multimodal fusion feature vectors. It can calculate the timing offset between two signals and output a consistency result based on the timing offset threshold. Based on the calculated comparison data, it makes a determination according to the rules of the logical analysis engine and the feature similarity level in the anomaly feature library.

[0073] The anomaly identification module constructs a broadcasting-specific anomaly library based on industry-specific anomalies and historical anomaly segments, identifies core broadcasting anomalies, and outputs the fault level. Multimodal fusion integrates audio and video features, corrects single-modal errors, and outputs a fused multimodal feature vector. Anomaly classification identifies core industry-specific anomalies and outputs anomaly category, confidence level, and fault level. Anomaly localization locates video anomaly regions and marks the start and end times of audio anomalies. Anomaly segment acquisition automatically saves anomaly segments on the platform and allows maintenance personnel to manually mark new anomaly types.

[0074] The incremental learning module enables continuous iteration of the anomaly detection module. It filters high-value old and new samples to avoid data redundancy, de-identifies program content segments, and constructs an incremental dataset. Incremental training fine-tunes the broadcast-specific detection, retaining old knowledge while learning new anomalies. A dual mechanism of EWC and knowledge distillation is employed: EWC protects the old anomaly detection capabilities, while knowledge distillation transfers knowledge from the old model to the new model, preventing forgetting. Annotated anomaly samples (such as silence, black screen, audio-visual asynchrony, etc.) are used for multi-task learning, outputting anomaly type probabilities and confidence scores for supervised fine-tuning. Online incremental learning is also performed to adapt to newly emerging anomaly patterns. Offline training is conducted on backup GPU nodes, without consuming online monitoring resources. Online updates are performed using a hot deployment mode, ensuring that the monitoring business is unaware when a new model replaces an old one.

[0075] The application service layer handles business logic processing, fault alarms and automatic handling, and data query and analysis. It can monitor the operational status of hardware, algorithms, and services in real time, trigger alarms and switch emergency responses based on logical analysis, and record end-to-end operation and fault logs to meet broadcast compliance requirements. Based on the type and level of anomalies, it determines emergency strategies through fault assessment. It can achieve millisecond-level switching between primary and backup signals to avoid broadcast interruptions and can push alarms to corresponding maintenance personnel according to fault level via SMS gateways, maintenance platform pop-ups, etc. It can also manage abnormal segments and logs, achieving separation of hot and cold data.

[0076] The user interaction layer enables human-computer interaction, status display, model management, and compliance auditing. The visual monitoring dashboard provides real-time updates on audio and video playback and fault alarms; the operations and maintenance console allows for on-demand fault handling and data query and analysis; model management enables manual labeling of new anomalies, incremental dataset construction, and fine-tuning of model parameters; and log auditing, through centralized log storage and standardized auditing, effectively identifies potential risks and improves operational security and efficiency.

[0077] Combination Figure 2 As shown, the present invention also provides an emergency handling method for real-time broadcast programs, which involves performing the following steps using the real-time broadcast program emergency handling system of the present invention:

[0078] S1, the data acquisition and preprocessing layer, performs signal acquisition, processing and conversion; the signals from the monitoring points are SDI, ASI or IP, and are bypassed to the signal acquisition equipment in the hardware layer.

[0079] The signal acquisition equipment performs corresponding decoding, demultiplexing, and bitstream trimming, as well as IP multicasting, and analyzes and alarms according to the TR101290 protocol requirements.

[0080] S2, multi-source synchronous calibration, uses audio and video synchronization algorithms to intelligently synchronize frames for signals from different nodes to compensate for the delay caused by each signal passing through intermediate devices.

[0081] S3, the core algorithm layer, performs multimodal feature extraction, extracting video spatiotemporal features and audio spectral features, and obtains near-subjective quality evaluation results through quality analysis algorithms.

[0082] S4, after intelligent frame synchronization, performs consistency comparison of associated monitoring points based on feature vectors and quality evaluation results, analyzes the structural similarity of audio and video, and provides a quantitative assessment of similarity.

[0083] S5, through the anomaly recognition module of the core algorithm layer, uses multimodal content recognition technologies such as image recognition, face recognition, speech recognition, and text recognition to automatically identify and mark potentially illegal content. It can not only detect sensitive information in images and audio, but also recognize hidden meanings in the context.

[0084] S6, the application service layer, performs fault analysis and determines emergency strategies based on the comparison and identification results. It can detect anomalies that are difficult to detect under normal circumstances, such as signal degradation, equipment aging, and abnormal signal quality.

[0085] S7, Emergency Response, handles situations according to emergency strategies. Emergency events automatically switch between primary and backup circuits, while routine events are managed through a user interface layer with alarms and prompts for appropriate maintenance operations.

[0086] In step S2 above, the multi-source audio-visual synchronization algorithm first unifies the time base through a global clock, then accurately measures the delay differences of each source, and finally dynamically compensates for the offset through various methods to achieve multi-source frame alignment.

[0087] 1) First, the time base of all information sources is unified to the absolute time axis through a global time synchronization protocol.

[0088] 2) Next, measure the delay difference. Signals from the same source accumulate delays, such as fixed delay and dynamic delay, as they pass through intermediate devices like encoding, transmission, and decoding. Accurately measure and model these delays, and then perform targeted compensation. The following scenario illustrates the steps for compensating for dynamic delay:

[0089] The latency of the primary and backup links changes dynamically with environmental conditions and equipment load, resulting in a lag in the traditional "real-time measurement - passive compensation" mode. This can be optimized through the following steps:

[0090] Modeling: Record the mapping relationship between link latency and temperature and device CPU utilization over a period of time, such as temperature latency coefficient d1 and CPU latency coefficient d2;

[0091] Real-time prediction: Based on the current temperature and CPU value, predict the delay of the next moment using linear regression: ΔT = d1×ΔT_temp + d2×ΔT_cpu + baseline delay;

[0092] Advance compensation: Adjust the link buffer in advance based on the predicted value. For example, if the expected delay increases by 5μs, expand the buffer by 5μs in advance to avoid synchronization offset caused by sudden delay changes.

[0093] The actual effect is that the dynamic latency compensation response time is reduced from 10ms to 4ms, and the stability of the primary and backup links is improved by 40%.

[0094] 3) After unifying the time base and quantizing the delay offset, the audio and video frames from different sources are aligned to the same playback time through buffering adjustment, rate adaptation, and other means, while ensuring the synchronization of audio and video content within a single source. Cross-source synchronization brings the signals from all sources to the same target playback time. Taking the source with the largest delay as the benchmark, buffering compensation is applied to sources with smaller delays, or acceleration processing is applied to sources with larger delays.

[0095] The consistency comparison in step S4 above specifically includes:

[0096] Based on the multimodal fusion feature vectors of the multi-channel signals, output the pairwise similarity matrix between the multi-channel signals;

[0097] Calculate the timing offset between the two signals and output the consistency result based on the timing offset threshold;

[0098] Based on the calculated comparison data, a judgment is made according to the rules of the logical analysis engine and the similarity of features in the abnormal feature library.

[0099] Step S5 above specifically includes the following steps:

[0100] S51, multimodal feature extraction, extracts effective information from raw data, including video feature extraction, that is, extracting spatial features of single frame (such as objects, scenes, text, image quality degradation), using temporal models to analyze inter-frame relationships, and identifying still frames, stuttering, and time-series anomalies.

[0101] Audio feature extraction converts audio signals into time-frequency graphs such as Mel spectrograms to analyze issues such as audio interruptions, abnormal noise, and audio-visual synchronization.

[0102] Text feature extraction involves using model encoding to extract keywords and sensitive semantic content from the text for subsequent consistency verification.

[0103] S52, Feature Alignment and Fusion, enables communication between features from different modalities. Feature alignment involves aligning features from different modalities that reside in different vector spaces using modality connectors.

[0104] Feature fusion integrates aligned features and uses splicing, weighted averaging, or cross-attention mechanisms to enable the consistency comparison module to notice relevant audio and textual cues when analyzing videos.

[0105] S53, anomaly recognition and decision-making, makes a final judgment based on feature fusion. Anomaly detection, by having the consistency comparison module learn a large amount of normal program data, enables it to generate alarms for abnormalities such as black screens, color bars, and silence that deviate from the normal pattern;

[0106] Quality analysis involves frame-level comparison when a lossless reference source is available, to accurately quantify the degree of mosaic and bitrate degradation.

[0107] Classification and regression are used to identify specific types of anomalies, including content violations, signal interruptions, or to predict their severity (such as the duration of buffering).

[0108] In step S5 above, the anomaly recognition module is continuously iterated through an incremental learning module, specifically including:

[0109] A sample set is selected and constructed, including regular, normal samples and audio / video clips without any quality issues or content risks; these are collected from daily broadcast streams and constitute the main body of the sample library. Anomaly samples in audio / video quality, such as black screens, still frames, mosaics, silence, and audio pops, are obtained from historical fault databases or by artificially simulating specific faults, such as intentionally interrupting the signal. Anomaly samples in content security, such as illegal images, sensitive text, and illegal QR codes, are obtained from professional and authoritative sensitive content databases or generated through technical synthesis.

[0110] The sample set is cleaned and labeled, and the samples are accurately labeled, such as the type of abnormality, the time frame of occurrence, and the severity level.

[0111] Train on the sample set.

[0112] Training on the sample set specifically includes:

[0113] Data preprocessing and partitioning: video is decoded into frames, audio is converted into spectrum, text is segmented, and data format and size are standardized for preprocessing. Training set, validation set and test set are divided proportionally to ensure that the proportion of each type of anomaly is balanced in each set.

[0114] Pre-trained model fine-tuning: Based on a model pre-trained on a large-scale general dataset, domain-adaptive fine-tuning is performed using broadcast samples to quickly achieve high performance.

[0115] Joint learning tasks simultaneously predict the type and severity of anomalies, improving the high availability of the model in learning feature representations;

[0116] To cope with the continuous emergence of new anomalies, incremental learning is adopted, and the model is updated regularly with new samples, while avoiding catastrophic forgetting of old knowledge.

[0117] Evaluation metrics, in addition to accuracy, also focus on recall to avoid false negatives and precision to reduce false positives. For quality assessment tasks, metrics that are highly correlated with subjective evaluations can be used. Alarm thresholds can be flexibly adjusted according to business needs. After the model goes live, it needs to be continuously monitored to collect false positive and false negative cases and add them to the training set for iterative optimization.

[0118] Example

[0119] Combination Figure 3 As shown, this embodiment uses a typical broadcast link as an example to illustrate the process of the emergency handling method for real-time broadcast programs adopted in this embodiment.

[0120] Figure 3 In this configuration, Link 1 and Link 2 are primary and backup multi-source links with the same signal. After passing through Device 1 (primary) and Device 1 (backup), the signal enters the switch for a two-to-one switching. Five monitoring points are set up on this link. Under normal conditions, the signal from monitoring point 5 originates from the main monitoring point 3, and the signals from all five monitoring points should be consistent.

[0121] The procedure for handling emergency situations of live broadcast programs used in this embodiment is as follows:

[0122] Signal acquisition: The signals from the five monitoring points can be SDI, ASI, or IP, and can be bypassed and connected to the centralized signal acquisition device. The signals are then decoded, demultiplexed, and trimmed by the centralized acquisition device, and multicast via IP. Analysis and alarm functions can be generated according to the TR101290 protocol requirements.

[0123] Multi-source synchronous calibration utilizes audio and video synchronization algorithms to intelligently synchronize frames for signals from different nodes, compensating for delays caused by intermediate devices passing through various signals.

[0124] Multimodal feature extraction is the core of audio and video quality analysis. It extracts spatiotemporal features of video and spectral features of audio, and obtains near-subjective quality evaluation results through reference-based quality analysis algorithms.

[0125] After intelligent frame synchronization, consistency analysis is performed on associated monitoring points based on feature vectors and quality evaluation results, and structural similarity of audio and video is analyzed to provide a quantitative assessment of similarity.

[0126] Meanwhile, through the heteromorphic recognition big model, by organically combining multimodal content recognition technologies such as image recognition, face recognition, speech recognition, and text recognition, it can automatically identify and mark potential illegal content. It can not only detect sensitive information in images and audio, but also identify hidden meanings in the context.

[0127] Based on the above comparison and identification results, the logic analysis engine performs fault assessment and determines emergency strategies, and can discover anomalies that are difficult to detect under normal circumstances, such as signal degradation, equipment aging, and abnormal signal quality.

[0128] Emergency response involves taking appropriate actions based on the emergency strategy. In case of emergency, automatic switching between primary and backup circuits is implemented. For routine events, alarm management is conducted through the user interface layer, prompting users to perform corresponding maintenance operations.

[0129] This embodiment uses three application scenarios as examples to illustrate:

[0130] Scenario 1: If the signals of monitoring points 1, 2 and 4 are consistent after comparison, and the signal of monitoring point 3 is inconsistent with the other three collection points but not abnormal, although the signal of monitoring point 3 is not abnormal, the system can automatically switch the signal of monitoring point 4 to output monitoring point 5 according to the predetermined emergency strategy.

[0131] Scenario 2: If the signals from monitoring points 1, 2, and 3 are consistent after comparison, but the signal from monitoring point 4 is inconsistent with the other three collection points, the system can remind the user that device 1 may have an anomaly and needs to be inspected according to the predetermined processing strategy.

[0132] Scenario 3: Real-time speech recognition and sensitive person identification are performed on monitoring point 1 and monitoring point 2, which can detect in real time images and sounds that are broadcast normally but contain illegal or irregular content.

[0133] Those skilled in the art should recognize that the above embodiments are merely illustrative of the present invention and are not intended to limit the present invention. Any variations or modifications to the above embodiments that are within the spirit and essence of the present invention will fall within the scope of the claims of the present invention.

Claims

1. A real-time broadcast program emergency handling system, characterized in that, include: At the hardware layer, physical signal access, computing power support, and data storage are completed. The data acquisition and preprocessing layer performs unified acquisition and processing of multi-source heterogeneous monitoring signals. It mainly realizes signal demodulation, descrambling and demultiplexing, code stream trimming, IP multicast, and realizes radio frequency monitoring, TS stream monitoring, environmental monitoring, and EFD intrusion alarm. The core algorithm layer completes feature modeling, multi-channel signal comparison, anomaly classification and localization, and continuous model iteration. The application service layer enables business logic processing, fault alarms and automatic handling, and data query and analysis. The user interaction layer enables human-computer interaction, status display, model management, and compliance auditing.

2. The real-time broadcast program emergency handling system according to claim 1, characterized in that, The hardware layer includes: The signal acquisition equipment completes the dedicated unified acquisition and processing of heterogeneous signals such as ASI / SDI / IP. Each signal is equipped with dual acquisition cards, which automatically switch in case of failure. The computing equipment uses GPU clusters and supports load balancing to maintain high availability in the event of node failures. The storage device uses a RAID disk array to store abnormal fragments and monitoring logs.

3. The real-time broadcast program emergency handling system according to claim 1, characterized in that, The core algorithm layer includes: The multimodal feature extraction module converts the original audio and video signals into computable and comparable multimodal feature vectors; The consistency comparison module monitors single-channel anomalies, multi-channel inconsistencies, and timing misalignments in multiple signals from the same source in broadcasting, and implements audio-visual feature similarity calculation, timing consistency verification, and consistency determination logic. The anomaly identification module constructs a dedicated anomaly library for broadcasting based on industry anomalies and historical anomaly fragments, identifies core anomalies in broadcasting, and outputs the fault level. The incremental learning module enables continuous iteration of the anomaly recognition module.

4. An emergency handling method for live broadcast programs, characterized in that, The following steps are performed using the live broadcast program emergency handling system as described in any one of claims 1-3: S1, the data acquisition and preprocessing layer performs signal acquisition, processing and conversion; S2, multi-source synchronous calibration, uses audio and video synchronization algorithms to perform intelligent frame synchronization for signals from different nodes to compensate for the delay caused by each signal passing through intermediate devices; S3, the core algorithm layer performs multimodal feature extraction, extracts video spatiotemporal features and audio spectrum features, and obtains near-subjective quality evaluation results through quality analysis algorithms; S4, after intelligent frame synchronization, performs consistency comparison of associated monitoring points based on feature vectors and quality evaluation results, analyzes the structural similarity of audio and video, and provides a quantitative assessment of similarity. S5, through the anomaly recognition module of the core algorithm layer, using multimodal content recognition technology, potential illegal content is automatically identified and marked; S6, the application service layer performs fault assessment and determines emergency strategies based on the comparison and identification results; S7, Emergency Response: Take appropriate action according to the emergency strategy.

5. The emergency handling method for real-time broadcast programs according to claim 4, characterized in that: In step S1, the signal of the monitoring point is SDI, ASI or IP, and is bypassed to the signal acquisition device of the hardware layer. The signal acquisition equipment performs corresponding decoding, demultiplexing, bitstream trimming, and IP multicasting, and analyzes and alarms according to the TR101290 protocol requirements.

6. The emergency handling method for real-time broadcast programs according to claim 4, characterized in that, The consistency comparison in step S4 specifically includes: Based on the multimodal fusion feature vectors of the multi-channel signals, output the pairwise similarity matrix between the multi-channel signals; Calculate the timing offset between the two signals and output the consistency result based on the timing offset threshold; Based on the calculated comparison data, a judgment is made according to the rules of the logical analysis engine and the similarity of features in the abnormal feature library.

7. The emergency handling method for real-time broadcast programs according to claim 4, characterized in that, Step S5 specifically includes the following steps: S51, Multimodal Feature Extraction, extracts effective information from raw data; S52, Feature Alignment and Fusion, enables features from different modalities to communicate; S53, Anomaly Recognition and Decision Making, makes a final judgment based on feature fusion.

8. The emergency handling method for real-time broadcast programs according to claim 7, characterized in that, Step S51 specifically includes: Video feature extraction: extract spatial features of single frames, use temporal models to analyze inter-frame relationships, and identify still frames, video stuttering, and time-series anomalies such as clipping. Audio feature extraction converts audio signals into Mel-frequency spectrograms to analyze audio interruptions, abnormal noise, and audio-visual synchronization issues. Text feature extraction involves using model encoding to extract keywords and semantic content of sensitive information from text for subsequent consistency verification; and / or Step S52 specifically includes: Feature alignment aligns features from different modalities that exist in different vector spaces using modal connectors. Feature fusion integrates aligned features and, through splicing, weighted averaging, or cross-attention mechanisms, enables the consistency comparison module to notice relevant audio and textual cues when analyzing videos; and / or Step S53 specifically includes: Anomaly detection involves enabling the consistency comparison module to learn a large amount of normal program data, thus generating alarms for abnormalities such as black screens, color bars, and silence that deviate from the normal pattern. Quality analysis involves frame-level comparison when a lossless reference source is available, to accurately quantify the degree of mosaic and bitrate degradation. Classification and regression are used to identify specific types of anomalies, including content violations, signal interruptions, or to predict severity.

9. The emergency handling method for real-time broadcast programs according to claim 8, characterized in that, In step S5, the incremental learning module is used to achieve continuous iteration of the anomaly recognition module, specifically including: A sample set is selected and constructed, which includes regular normal samples and audio / video clips; The sample set is cleaned and labeled; The sample set is then used for training.

10. The emergency handling method for real-time broadcast programs according to claim 9, characterized in that, Training the sample set specifically includes: Data preprocessing and partitioning: video is decoded into frames, audio is converted into spectrum, text is segmented, and data format and size are standardized for preprocessing. Training set, validation set and test set are partitioned proportionally. Pre-trained model fine-tuning: Based on a model pre-trained on a large-scale general dataset, domain-adaptive fine-tuning is performed using broadcast samples; Joint learning task to simultaneously predict the type and severity of anomalies; Incremental learning involves periodically updating the incremental learning module with new samples. Evaluation and optimization.

Images