Broadcast audio transmission equipment fault early warning method and system based on multi-modal data fusion

Abstract

The invention discloses a broadcast audio transmission equipment fault early warning method and system based on multi-modal data fusion. The method specifically comprises the following steps: constructing a six-dimensional index system adaptive to broadcast equipment, wherein the six-dimensional index system covers the operation stability, the maintenance frequency, the alarm improvement rate, the audio signal-to-noise ratio, the total harmonic distortion and the transmission time consumption; collecting data and carrying out dynamic threshold normalization preprocessing; second-level data fusion is executed, the first-level data fusion calculates a health feature value based on the weight and the time decay factor, and the second-level data fusion outputs a health state probability by using a lightweight LSTM model; and finally, generating an operation and maintenance instruction containing a specific operation suggestion according to the state. According to the method, the problems of single evaluation dimension and poor model adaptability in the prior art are solved, and accurate quantitative early warning and closed-loop active operation and maintenance of the health state of the broadcast equipment are realized.

Description

Technical Field

[0001] This invention relates to the field of broadcast communication equipment operation and maintenance technology, and in particular to a fault early warning method and system for broadcast audio transmission equipment based on multimodal data fusion. It is applicable to health status monitoring, fault early warning and operation and maintenance decision support for audio signal transmission node equipment (such as audio encoders, transmission switches, signal amplifiers, etc.) of radio stations and television stations at all levels. Background Technology

[0002] Broadcast audio transmission node equipment is the core infrastructure for ensuring uninterrupted, high-quality transmission of broadcast signals. Its operational health directly determines key indicators such as the signal-to-noise ratio (SNR), transmission delay, and distortion of the audio signal. With the digitalization and networking of broadcast technology, transmission links are becoming increasingly complex, placing higher demands on equipment operation and maintenance.

[0003] The existing technologies for health assessment and fault early warning of this type of equipment have the following main shortcomings: A single evaluation dimension can easily lead to misjudgments or omissions: Traditional methods often rely on a single indicator for judgment. For example, judging a fault solely by the number of alarms, or using only discrete testing with specialized audio quality testing equipment. This approach cannot cover the entire lifecycle characteristics of a device, from operational stability to fault history to repair effectiveness. For instance, a device may currently have no alarms, but has undergone multiple frequent restarts, posing a significant hidden danger; or a device may meet audio quality standards, but its transmission latency is gradually deteriorating. A single-dimensional evaluation is highly susceptible to misjudgments or omissions.

[0004] Lack of data correlation makes it difficult to pinpoint the root cause: Current technologies have not yet established a sound mechanism for comparing data before and after maintenance, making it impossible to quantify the maintenance effect. Furthermore, cross-indicator correlation analysis is not implemented; for example, high maintenance frequency cannot be coupled with high-frequency alarms after maintenance, making it difficult to locate the root cause of the fault through data correlation.

[0005] The models suffer from poor adaptability and fail to meet the characteristics of the broadcasting industry: Currently used multimodal evaluation models for industrial equipment (such as general evaluation models based on vibration and temperature data) are not optimized for the characteristics of broadcasting equipment. Broadcasting equipment is extremely sensitive to audio quality and has stringent requirements for real-time transmission. General models lack specific design for indicators such as audio harmonic distortion and signal transmission time, and do not consider the drift of indicator benchmarks caused by equipment aging, resulting in evaluation results that are out of touch with actual operation and maintenance needs.

[0006] Insufficient disclosure of technical details makes implementation difficult: Although some related patents propose the concept of fault warning, they often fail to specify the equipment parameters for data collection, the specific quantitative evaluation formulas, and the details of model training (such as network structure and parameter settings), making it difficult to reproduce and implement the technical solutions. Summary of the Invention

[0007] This invention primarily addresses the aforementioned technical problems in existing technologies by providing a method and system for fault early warning of broadcast audio transmission equipment based on multimodal data fusion. By constructing a six-dimensional indicator system encompassing audio quality, operational stability, and maintenance effectiveness, and combining dynamic threshold normalization with a two-level data fusion algorithm, it achieves accurate quantitative assessment and fault early warning of equipment health status, and can automatically generate executable operation and maintenance instructions, thereby improving operation and maintenance efficiency and reducing equipment failure rate.

[0008] The present invention addresses the aforementioned technical problems primarily through the following technical solution: a fault early warning method for broadcast audio transmission equipment based on multimodal data fusion, comprising the following steps: S1: Construct a six-dimensional multimodal evaluation index system adapted to broadcast audio transmission equipment. The index system includes: equipment continuous operation stability index, equipment maintenance frequency index, alarm improvement rate before and after maintenance index, audio signal-to-noise ratio before and after maintenance index, audio total harmonic distortion before and after maintenance index, and signal transmission time before and after maintenance index. S2: Collect the operating data and audio signal data of the broadcast audio transmission equipment, and calculate the values ​​of each indicator in the six-dimensional multimodal evaluation index system; S3: Preprocess the calculated values ​​of each indicator. The preprocessing includes outlier removal based on the 3σ criterion and normalization based on dynamic threshold to obtain the normalized values ​​of each indicator. S4: Perform secondary data fusion processing: First-level feature fusion: Based on preset weight coefficients and time decay factors, the normalized values ​​of the indicators are weighted and summed and smoothed to obtain health feature values ​​that reflect the long-term health trend of the equipment. Second-level decision-level fusion: The health feature values ​​and the normalized values ​​of each indicator are used as input vectors and input into a pre-trained lightweight long short-term memory network (LSTM) model to output a probability vector representing the health status of the device. S5: Determine the current health level of the device based on the probability vector, and generate and output the corresponding operation and maintenance instructions according to the current health level; The operation and maintenance instructions include at least key monitoring instructions for sub-healthy states, preventive maintenance instructions for fault risk states, and shutdown maintenance instructions for fault states.

[0009] Different from traditional single physical monitoring, this indicator system integrates physical operation data (stability, maintenance frequency), service quality data (signal-to-noise ratio, distortion, time consumption), and operation and maintenance effect data (alarm improvement rate). In addition, as a pre-check for data validity, this invention can also introduce a signal level indicator. If the collected signal level is not within the preset range (such as [-18dBV, -12dBV]), it can be directly determined that the signal is lost or abnormal, and subsequent complex fusion calculations will not be performed, thus saving computing power.

[0010] Preferably, in step S2, the equipment continuous operation stability indicator S act The calculation formula is: S run =[(T act -T down -T plan ) / T std ×100%; Where, T act is the total actual operation duration of the equipment within the statistical period, which is collected in real time by the PLC module at a frequency of 1 time / minute; T down is the unplanned downtime duration of the equipment within the statistical period, and it is required to satisfy T down <T act , otherwise, data anomaly verification is triggered, and after verification, the equipment log is automatically retrieved to check the downtime reason; T plan is the planned maintenance downtime duration of the equipment within the statistical period, which needs to be entered into the system in advance as a fixed parameter; T std is the designed standard continuous operation duration of the equipment, such as 5000h / year, which can be adjusted according to the equipment model; When the calculated S run <0 or S run >100%, it is determined that the data is abnormal, and the average value of the first 3 valid data is used for substitution.

[0011] High-frequency collection at a frequency of 1 time / minute can accurately capture the instant start and stop of the equipment. Setting the abnormal value substitution logic is to prevent data deviation caused by sensor failures of the PLC module itself and ensure the robustness of the characteristic data input into the model. When S run ≥90%, it is determined as excellent; 70%≤S run <90%, it is determined as qualified; S run <70% is determined as unqualified.

[0012] Preferably, in step S2, the equipment maintenance frequency indicator F main The calculation formula is: F main =(N main ×w main +N minor ×wminor ) / T cycle ; Where, N main This refers to the number of major maintenance operations within the statistical period, where major maintenance is defined as the downtime T. repair Maintenance time ≥4 hours; w main This is the weighting coefficient for major maintenance, with a value of 1.0. N minor To determine the number of minor maintenance operations within a statistical period, the downtime T is used as an example. repair For maintenance operations lasting less than 1 hour, each occurrence is recorded as a minor maintenance operation; for downtime of 1 hour ≤ T... repair Maintenance that takes less than 4 hours is counted as 1.6 minor maintenance incidents per incident; w minor The weighting factor for minor maintenance is 0.3. T cycle The statistical period is in months and must be consistent with the data collection period.

[0013] In broadcast maintenance practice, maintenance exceeding 1 hour but less than 4 hours usually indicates a more complex hidden problem with the equipment, and its negative impact on health is far greater than that of a regular quick restart (<1 hour). Therefore, this solution counts maintenance durations of 1-4 hours as 1.6 times. This weighted calculation more accurately reflects the equipment's susceptibility to failure. When F main ≤1.2 times / month is considered low risk, 1.2 <F main ≤2.0 times / month is considered medium risk, F main A score greater than 2.0 indicates a high risk.

[0014] Preferably, in step S2, the alarm improvement rate index R before and after maintenance is... alarm The calculation formula is: R alarm =max(((F alarm-before -F alarm-after ) / F alarm-before ()×100%,-50%); Among them, F alarm-before and F alarm-after These represent the weighted alarm frequency before maintenance and the weighted alarm frequency after maintenance, respectively. Both are calculated using the following weighted alarm frequency formula: F=(∑ n i=1 N alarm-i ×S i ) / T run ; For F alarm-before Calculation: The statistical period is 30 days before the maintenance, where N i T represents the number of alarms of type i (such as power alarm, signal loss alarm) within this period.run The effective operating time of the equipment within this period is generally required to be no less than 500 hours; For F alarm-after Calculation: The statistical period is 30 days after the maintenance, where N i T represents the number of alarms of type i within this period. run This refers to the effective operating time of the equipment within this period; S i The severity coefficient for the i-th type of alarm is set as follows: the severity coefficient for power supply alarm is 1.0, the severity coefficient for signal fluctuation alarm is 0.5, and the severity coefficient for temperature over-limit alarm is 0.8.

[0015] This scheme uses the max function to set a lower limit of -50% to handle extreme deterioration situations and prevent excessively small values ​​from causing abnormal model gradients. Simultaneously, the weighting coefficients are set (power supply > temperature > signal fluctuation) strictly adhere to the safety priority of broadcast equipment: power failure will lead to direct broadcast interruption, with the most severe consequences; excessively high temperatures pose a long-term hazard; signal fluctuations may be caused by external interference and therefore have relatively lower weights. When R... alarm ≥40% indicates excellent maintenance results, 20% ≤R alarm <40% indicates the repair effect is acceptable. alarm <20% indicates poor maintenance results.

[0016] For complex signals such as music and human voice, this invention achieves distortion quantization through adaptive fundamental frequency extraction and reference signal comparison. The core logic is: the harmonic distortion generated by the device has a fixed frequency correlation (integer multiple of the fundamental frequency), while the overtones of the program itself do not have strict integer multiple characteristics. Based on this, the two are distinguished and THD is calculated.

[0017] Preferably, in step S2, the total harmonic distortion (THD) in the audio total harmonic distortion index before and after maintenance is calculated using an adaptive fundamental frequency extraction and reference signal comparison method, specifically including the following sub-steps: S21: The instantaneous fundamental frequency f0(t) of the audio signal is extracted using the Teager-Kaiser energy operator combined with a sliding window. The formula for calculating the energy operator is as follows: f0(t)=(1 / 2π)×sqrt(((x(t+1)-x(t)) 2 -(x(t+1)-2x(t)+x(t-1))(x(t)-x(t-1))) / x(t) 2 ); The window size is set to 100ms to balance fundamental frequency stability and real-time performance. x(t) represents the audio sample value at time t. This operator highlights the instantaneous energy peak of the signal, thereby locking the dominant fundamental frequency within each window (e.g., human voice fundamental frequency 85-300Hz, music melody fundamental frequency 100-500Hz). The extracted f0(t) is smoothed to remove abrupt changes (e.g., fundamental frequency jumps between syllables), resulting in a stable fundamental frequency sequence f'0(t). S22: Obtain the calibrated program source backup link signal as the reference signal x ref (t), and the output signal x of the device under test. test (t) Perform timestamp synchronization (synchronization accuracy ≤ 1ms) to eliminate spectral deviation caused by transmission delay; x ref (t) is a standard transmission sample of the same program signal obtained through the program source backup link of the broadcast system. This sample is transmitted through a calibrated distortion-free link to ensure that it contains only the program's own overtones. S23: For x ref (t) and x test (t) Perform Fourier transforms to obtain the spectrum X ref (t) and X test (t), calculate the amplitude difference A at integer multiples of frequency k×f'0 of f'0(t). dist (k×f'0)=|X test (k×f'0)|-|X ref (k×f'0)|, where k=2,3,…,10, A dist (k×f'0) is the distortion amplitude generated by the device at the kth harmonic frequency. If it is negative, it means that there is no device distortion at that frequency point, so we take 0. S24: Calculate the total harmonic distortion (THD) based on the amplitude difference. THD=(sqrt(∑(A dist (k×f'0)) 2 ) / |X ref (f'0)| 2 The formula is calculated as 100%, where the summation applies to k = 2 to 10. This formula only calculates harmonic distortion introduced by the equipment, completely excluding interference from the program's own overtones. The measurement accuracy has been verified to deviate from the standard signal method by ≤0.1%.

[0018] Traditional THD measurements rely on a single-frequency standard signal, making them unsuitable for online detection during music or vocal playback. This invention leverages the superior instantaneous energy tracking capability of the Teager-Kaiser operator to pinpoint the fundamental frequency from complex program signals. By comparing it with a distortion-free reference signal, only the additional harmonic components generated by the device are calculated, thus enabling online THD monitoring during normal program broadcast. This comparison logic overcomes the problem of interference from overtones in program content that plagues traditional methods.

[0019] In addition, to ensure measurement accuracy, the system can calibrate the measurement results once a month using a 1kHz pure sine wave standard signal. If the deviation is >0.1%, the smoothing coefficient of the energy operator is automatically corrected to ensure long-term measurement stability.

[0020] Specific calculations of audio signal-to-noise ratio before and after maintenance: The raw values ​​of the indicators are expressed in decibels (dB). Specifically, the system collects the audio signal-to-noise ratio (SNR) before the maintenance operation. before Audio signal-to-noise ratio (SNR) after maintenance operations after The calculation formula is standardized as follows: SNR = 10lg(Ps) ignal / P noise ), where P signal P represents the effective power of the audio signal. noise The background noise power is measured, and external electromagnetic interference must be shielded during data acquisition (e.g., away from frequency converters and high-voltage lines).

[0021] When performing the secondary data fusion processing described in step S4, the present invention selects SNR. after This serves as a representative value for the indicator. This is because the core of fault early warning lies in assessing whether the "current" (i.e., after maintenance) health status of the equipment is sufficient to support stable operation in the next stage.

[0022] It should be noted that SNR before It is mainly used for pre-validation of data validity and as an auxiliary input (i.e., the rate of change of indicators) for LSTM models. Specifically, this invention requires that the validation action must produce a substantial effect, that is, meet the SNR (Score-to-Noise Ratio). after -SNR before ≥3dB and SNR after ≥48dB. If this condition is not met, it indicates that the repair was ineffective or that the equipment has a deep hardware fault. The system will directly output a fault alarm instead of proceeding with the regular health rating process. After meeting the above conditions, the SNR will be... after Substituting the values ​​into the positive index normalization formula, the resulting values ​​are used in subsequent weighted summation and neural network operations.

[0023] Specific explanation of signal transmission time indicators before and after maintenance: This metric reflects the real-time performance of broadcast audio signals during internal equipment processing and link transmission, and is crucial for ensuring "zero latency" in live broadcast signals. The core of this metric's calculation lies in accurately measuring the signal transmission time T. trans .

[0024] The system collects signal transmission time before and after maintenance. For any given state, the transmission time T is... trans The following formula is used for calculation: T trans =(1 / M)∑ M k=1 (T out-k -T in-k )-T delay-fixed ; Wherein: T in-k and T out-k These are the input timestamp when the k-th test data packet enters the device and the output timestamp when it leaves the device (unit: ms). To eliminate random errors in a single measurement, this invention sets the sampling quantity M=100, that is, to continuously count the average time taken for 100 data packets. T delay-fixed (Fixed transmission delay) is a correction parameter specific to the broadcasting station environment in this invention. Because broadcasting equipment rooms typically have long cabling (e.g., over 50 meters), the cables themselves introduce physical delay. delay-fixed The delay needs to be measured in advance using calibration equipment (e.g., the delay of a standard 50m cable is approximately 0.25ms) and deducted when calculating the total time to ensure T. trans It only reflects the processing performance of the device itself, excluding interference from physical link length.

[0025] In addition to focusing on absolute time consumption, this invention also introduces a time consumption fluctuation coefficient C. trans To assess the stability of the transmission: C trans =σ T_trans / μ T_trans ; Where σ T_trans Let μ be the standard deviation of the transmission time of the above M data packets. T_trans The mean is given. If C... trans A value >0.1 indicates that the device's processing performance is unstable (there is jitter), and the transmission stability is directly judged as unqualified.

[0026] When performing secondary data fusion processing, this invention adopts the following logic: Main input value: Select the transmission time T after maintenance. trans-after As core data, since shorter transmission time is better and is considered a negative indicator, a negative normalization formula is used in step S3; Validity verification and auxiliary input: Calculate the transmission time improvement rate Rtrans : R trans =((T trans-before -T trans-after ) / T trans-before )×100%; The present invention sets the criterion for determining effective maintenance as: T trans-after ≤45ms (meets industry real-time standards), C trans Less than or equal to 0.1 and R trans ≥15%. Only data that meets the above conditions are considered valid health features and are included in the LSTM model for training or inference.

[0027] For the calculated index values, outlier removal is performed first. This scheme uses the 3σ criterion (based on the normal distribution characteristics, removing data exceeding the mean ± 3 standard deviations to identify outliers) combined with sliding window verification for each type of index X: a. Calculate the mean μ and standard deviation σ of the data within the sliding window (window size = 30); b. If data x i Satisfy |x i -μ|>3σ, marked as an outlier; c. If three consecutive data points are outliers, trigger a device fault alarm (there may be a hardware fault); otherwise, replace the outliers with the mean value within the window.

[0028] Then, normalization is performed to eliminate the influence of dimensions.

[0029] Preferably, in step S3, the normalization process based on the dynamic threshold specifically includes: For positive indicators, the normalized value X norm =(XX min (t)) / (X max (t)-X min (t)); For negative indicators, the normalized value X norm =(X max (t)-X) / (X max (t)-X min (t)); Among them, the dynamic maximum threshold X max (t) and dynamic minimum threshold X min (t) varies with the equipment usage time t (in months), and the calculation formula is: X max (t)=X max-static ×(1+0.05×floor(t / 12)); X min (t)=Xmin-static ×(1-0.03×floor(t / 12)); X max-static and X min-static These are the static maximum and static minimum values ​​of the initial state indicators of the equipment, respectively. `floor` is the floor function.

[0030] This dynamic adjustment mechanism addresses the issue of poor model adaptability. As equipment ages, the baseline values ​​of its various indicators naturally drift (for example, older equipment will naturally have a higher noise floor than newer equipment). If a fixed threshold is used, older equipment may be falsely judged as suboptimal for an extended period. The dynamic threshold mechanism allows for a reasonable performance decline within the aging range of the equipment, thereby reducing the false alarm rate.

[0031] This invention performs two-level data fusion processing, including feature-level fusion and decision-level fusion.

[0032] Preferably, in step S4, the health feature value H is calculated in the first-level feature-level fusion. feat The formula for (t) is: H feat (t)=∑(w i ×X i-norm (t)×α t ); Among them, w i X is the weighting coefficient for the i-th type of indicator, t is the equipment usage time (in years), and X is the weighting coefficient for the i-th type of indicator. i-norm (t) is the normalized value of the i-th index at time t, and α is the time decay factor, with a value of α=0.95; Furthermore, to avoid short-term data fluctuations causing H feat (t) sudden change, using the exponential moving average algorithm to adjust H feat (t) is smoothed: H feat-smoothed (t)=β×H feat (t)+(1-β)×H feat-smoothed (t-1); Among them, H feat-smoothed (t) represents the smoothed health feature value at time t, β is the smoothing coefficient, β=0.7, balancing real-time performance and stability, H feat-smoothed (t-1) represents the smoothed health feature value at time t-1, with an initial value H. feat-smoothed (0)=H feat (0).

[0033] Introducing a time decay factor α tThis is to reflect the principle that the longer the equipment has been used, the less its early historical state has an impact on the current assessment. Smoothing eliminates the interference of short-term data fluctuations (such as occasional interference) on the judgment of the overall health trend.

[0034] When performing the first-level feature-level fusion, the weight coefficient w of each indicator... i This directly affects the accuracy of health characteristic values. This invention, combining the operational requirements of prioritizing audio quality and ensuring operational stability in broadcast equipment, uses the Analytic Hierarchy Process (AHP) to determine the weights of each indicator.

[0035] As a preferred option, the weighting coefficients for the above six-dimensional indicator system are set as follows: Equipment continuous operation stability index (S run ): w1=0.20; Equipment maintenance frequency index (F) main ): w2=0.18; Alarm improvement rate before and after maintenance (R) alarm ): w3=0.22; Audio signal-to-noise ratio (SNR) before and after maintenance: w4=0.15; Total harmonic distortion (THD) index before and after maintenance: w5=0.10; Signal transmission time indicators before and after maintenance (T) trans ): w6=0.15.

[0036] The above weight allocation ensures that the model focuses on both the long-term stability of the device (total weight 0.38) and the improvement of alarm conditions (weight 0.22) and the core audio transmission quality (total weight 0.40), which is in line with the actual industry standards.

[0037] Preferably, in step S4, the lightweight Long Short-Term Memory (LSTM) network model in the second-level decision-level fusion has the following structural features: The input layer contains 12 neurons, and the input vector received consists of 6 normalized values ​​of the six-dimensional multimodal evaluation index system and the rate of change of the 6 indexes; the input is a 12-dimensional vector, and the output is a 12-dimensional feature vector to the first hidden layer. The first hidden layer is an LSTM layer containing 24 neurons, using the ReLU activation function, with the forget gate threshold set to 0.5, and the input gate weights initialized using a He normal distribution. The input is a 12-dimensional vector, and the output is a 24-dimensional vector to the second hidden layer. The second hidden layer is an LSTM layer containing 16 neurons, using the ReLU activation function, with the cell state update coefficient set to 0.8 to avoid gradient vanishing; the input is a 24-dimensional vector, and the output is a 16-dimensional feature vector to the Dropout layer or the output layer. The Dropout layer, with a dropout rate set to 0.2, takes a 16-dimensional vector as input and outputs a 16-dimensional vector to the output layer. This layer is only enabled during the model's training phase. During training, this layer randomly sets the output of some neurons to zero with a preset probability, thereby breaking the joint adaptation between neurons and effectively preventing overfitting on small sample data. During the online inference phase (i.e., when actually providing fault warnings to the device), the Dropout layer is disabled (or set to a fully connected state). In this case, all neurons in the network participate in the computation, and their weights remain unchanged. This setting ensures that the model can output constant and deterministic prediction results for the same input data, guaranteeing the stability and reliability of the fault warning system.

[0038] The output layer contains 4 neurons, using the Softmax activation function. The input is a 16-dimensional vector, and the output 4-dimensional probability vector corresponds to four states: healthy, sub-healthy, fault risk, and fault.

[0039] This network architecture is specifically designed for the real-time requirements of broadcast equipment. The specific narrow-depth structure, combined with knowledge distillation training, allows the model to maintain high accuracy while having a much smaller number of parameters than general-purpose models, thus achieving an inference speed of less than 80ms.

[0040] Preferably, in step S5, generating the corresponding operation and maintenance instructions specifically includes: When the probability vector indicates that the device is in a sub-healthy state, an operation and maintenance instruction containing the content of "focus on monitoring alarm frequency and transmission time, and re-check within 15 days" is generated; When the probability vector indicates that the device is in a fault risk state, an operation and maintenance instruction containing the content "carry out preventive maintenance within 72 hours, and prioritize checking the audio module" is generated; When the probability vector indicates that the device is in a fault state, an operation and maintenance instruction containing the content "immediately stop the machine for maintenance and check the power supply and signal transmission links" is generated.

[0041] By outputting instructions with clear time limits (15 days, 72 hours, immediate) and operation targets (alarm frequency, audio module, power link), the decision-making cost of operation and maintenance personnel is greatly reduced, realizing proactive operation and maintenance.

[0042] The specific process of training and optimizing the LSTM model is as follows: a. Dataset construction: Collect one year of operational data from 100 similar broadcast audio transmission devices (a total of 365,000 samples), and divide them into training set, validation set, and test set in a 7:2:1 ratio; b. Training parameters: The optimizer uses AdamW (weight decay coefficient = 1e-4, to prevent overfitting), and the learning rate α = 0.001 (dynamically adjusted using cosine annealing strategy). c. Model compression: The knowledge distillation technique (using a large model to guide the training of a small model, compressing the model size while ensuring accuracy) is adopted to distill the trained large model (94.2% accuracy) into a lightweight model, reducing the parameter size by 60% and the inference time from 500ms to 80ms, meeting the requirements of online real-time evaluation of the device (≤100ms).

[0043] The training and inference of LSTM models rely on explicit input features and learning objectives (labels). Both must be strongly tied to the core requirements of device health assessment to ensure that the model output has engineering applicability. Specific details are as follows: The core input features of the LSTM model: Input features must take into account the real-time status of the device, historical trends, and the correlation between indicators. The final input vector is 12-dimensional, and its composition and source strictly match the multimodal indicator system to ensure that the model focuses on key operational dimensions. Input features include the normalized values ​​of basic indicators and the rate of change of indicators.

[0044] For the normalized values ​​of basic indicators, the specific content includes the continuous operational stability of the equipment (S... run ), maintenance frequency (F) main Alarm improvement rate (R) alarm ), Audio SNR (SNR) after Audio THD (THD) after ), transmission time improvement rate (R) trans The source is the 3σ outlier removal + dynamic threshold normalization process, which maps to the [0,1] interval to eliminate dimensional differences.

[0045] The rate of change of the indicators is calculated as "(current value - historical average) / historical average" of the above six types of indicators. The historical average is calculated based on a sliding window (window size = 30) to reflect the short-term fluctuation trend of the indicators and help identify potential faults (such as a sudden 30% drop in SNR). The feature selection criteria are as follows: the input features are highly matched with the weights of the six multimodal indicators (such as alarm improvement rate weight 0.22 and equipment continuous operation stability weight 0.20), ensuring that the model training process prioritizes the dimensions that have the greatest impact on the health status of the equipment, thereby improving the evaluation accuracy.

[0046] The definition and annotation logic of LSTM model labels are explained below: The labels are the learning targets of the model and directly correspond to the actual health status of the equipment. In this solution, the labels are four-category discrete labels, which are strictly aligned with the four operation and maintenance scenarios of health, sub-health, fault risk and fault, to ensure that the model output can directly guide actual work.

[0047] (1) Label definition and encoding To meet the probability calculation requirements of the Softmax output layer, the tags adopt a dual format of integer encoding + one-hot encoding. The encoding rules are clear and interpretable, and the specific correspondence is shown in the table below: (2) Labeling logic Labeling does not rely on a single indicator, but rather combines three core dimensions for comprehensive judgment: quantitative indicator thresholds, the number of indicators meeting the standards, and fault / maintenance records. This ensures the authenticity and accuracy of the labels and avoids mislabeling caused by fluctuations in a single indicator. Specific labeling rules must simultaneously meet the following conditions: (4) Criteria for determining health status Combine H feat-smoothed (t), model output probability (p) j Based on the dual verification logic of indicator consistency verification and historical trend verification, the following classification judgment rules are formulated: A fault early warning system for broadcast audio transmission equipment based on multimodal data fusion, operating the early warning method described above, includes: The data acquisition and index calculation module is configured to collect operating data and audio signal data of broadcast audio transmission equipment, and calculate six-dimensional index values ​​including equipment continuous operation stability, maintenance frequency, alarm improvement rate, audio signal-to-noise ratio, audio total harmonic distortion and signal transmission time. The data preprocessing module is configured to remove outliers based on the 3σ criterion and normalize the index values ​​using a dynamic threshold that changes with the duration of device use. The secondary fusion judgment module is configured to first calculate health feature values ​​based on weight coefficients and time decay factors, and then input the health feature values ​​and index normalized values ​​into a preset lightweight long short-term memory network (LSTM) model to output the device health status probability. The early warning interaction module is configured to determine the health level based on the probability of the device's health status and generate and output corresponding operation and maintenance instructions.

[0048] The substantial effects of this invention are: 1. Comprehensive evaluation dimensions, eliminating blind spots: A 6-dimensional multimodal indicator system covering operational stability, maintenance frequency, alarm improvement, audio quality (SNR, THD) and transmission time has been constructed, which solves the problem of single dimensions in traditional methods.

[0049] 2. Adapting to equipment aging characteristics: By introducing a time decay factor (α=0.95) and a dynamic threshold adjustment mechanism, the evaluation model can adapt to the aging process of the equipment, maintain high evaluation accuracy throughout the entire life cycle of the equipment, and effectively reduce the misjudgment rate.

[0050] 3. Balancing Real-Time Performance and Accuracy: Employing a lightweight LSTM model (trained with knowledge distillation), the inference time is controlled within 80ms, meeting the real-time monitoring requirements of broadcast live scenarios. Experimental data shows that this invention improves the fault warning accuracy from 68% of traditional manual assessment to 92.5%.

[0051] 4. Closed-loop operation and maintenance, improving efficiency: It not only outputs health status but also directly generates specific operation and maintenance instructions. Verification data shows that after applying this method, the operation and maintenance response time was shortened from 48 hours to 24 hours, the annual equipment failure rate decreased by 35%, and the safety and reliability of broadcasting were significantly improved. Attached Figure Description

[0052] Figure 1 This is a flowchart of a fault early warning method for broadcast audio transmission equipment based on multimodal data fusion according to the present invention. Detailed Implementation

[0053] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0054] Example 1: System Architecture and Core Algorithm Flow This embodiment provides a fault early warning system for broadcast audio transmission equipment based on multimodal data fusion. The system is deployed in the operation and maintenance server of a broadcasting station and is connected to the front-end audio transmission node equipment (such as audio encoders, digital audio switchers, etc.) through a PLC (Programmable Logic Controller) module and a high-precision audio analysis interface.

[0055] like Figure 1 As shown in the figure, a fault early warning method for broadcast audio transmission equipment based on multimodal data fusion in this embodiment includes the following steps: S1: Construct a six-dimensional multimodal evaluation index system adapted to broadcast audio transmission equipment. The index system includes: equipment continuous operation stability index, equipment maintenance frequency index, alarm improvement rate before and after maintenance index, audio signal-to-noise ratio before and after maintenance index, audio total harmonic distortion before and after maintenance index, and signal transmission time before and after maintenance index. S2: Collect the operating data and audio signal data of the broadcast audio transmission equipment, and calculate the values ​​of each indicator in the six-dimensional multimodal evaluation index system; S3: Preprocess the calculated values ​​of each indicator. The preprocessing includes outlier removal based on the 3σ criterion and normalization based on dynamic threshold to obtain the normalized values ​​of each indicator. S4: Perform secondary data fusion processing: First-level feature fusion: Based on preset weight coefficients and time decay factors, the normalized values ​​of the indicators are weighted and summed and smoothed to obtain health feature values ​​that reflect the long-term health trend of the equipment. Second-level decision-level fusion: The health feature values ​​and the normalized values ​​of each indicator are used as input vectors and input into a pre-trained lightweight long short-term memory network (LSTM) model to output a probability vector representing the health status of the device. S5: Determine the current health level of the device based on the probability vector, and generate and output the corresponding operation and maintenance instructions according to the current health level; The operation and maintenance instructions include at least key monitoring instructions for sub-healthy states, preventive maintenance instructions for fault risk states, and shutdown maintenance instructions for fault states.

[0056] The steps are explained in detail below: 1. Data Acquisition and Pre-verification (Signal Level Filtering) The system first collects real-time operating data from the device. To avoid performing invalid and complex calculations in the absence of signal or under conditions of severe signal abnormality, this embodiment introduces signal level as a pre-verification indicator.

[0057] Calculation formula: Level = 20lg(V) rms / V ref ), where V ref =0.775V.

[0058] Verification logic: If the acquired signal level is not within the preset valid range [-18dBV, -12dBV], the system directly determines it as signal loss or input abnormality, triggering an immediate shutdown and maintenance alarm, and will not execute subsequent SNR, THD calculations, and multimodal fusion processes. The system will only proceed to the next step after the signal level verification passes.

[0059] 2. Refined calculation of core indicators For the data that passes verification, the system calculates the six-dimensional indicators as described in the claims. The specific processing logic for some of these indicators is as follows: Equipment maintenance frequency index (F) main The conversion logic of ) Considering the characteristics of broadcast equipment operation and maintenance, this embodiment further subdivides minor repairs. Within the statistical period, if an outage lasts for a duration T... repairRepairs lasting between 1 and 4 hours (typically involving board resets or software rollbacks), while not meeting the criteria for major repairs, carry a higher risk than regular reboots. Therefore, each such repair is counted as 1.6 minor repairs; repairs lasting less than 1 hour are counted as 1. Substituting these values ​​into formula F... main =(1.0×N main +0.3×N minor ) / T cycle .

[0060] Adaptive extraction of total harmonic distortion (THD) before and after maintenance: In order to detect distortion even when playing complex program content such as music or vocals, the system performs the following operations: (1) Fundamental frequency locking: The sliding window was set to 100ms, and the instantaneous fundamental frequency f0(t) was calculated in real time using the Teager-Kaiser energy operator. This operator is extremely sensitive to instantaneous energy changes and can effectively capture rapid changes between syllables. The extracted f0(t) was smoothed to obtain a stable fundamental frequency sequence f'0(t).

[0061] (2) Reference comparison: Obtain the reference signal from the program source backup link, and control the timestamp synchronization accuracy within 1ms through the NTP protocol and cross-correlation algorithm.

[0062] (3) Differential calculation: Only the increment of the spectral amplitude of the signal under test at k×f'0(t) (k=2,…,10) relative to the reference signal is calculated. If the increment is negative, it indicates that the equipment has not introduced distortion, and it is set to 0. This method completely eliminates the interference of program sources with overtones (such as instrument overtones) on THD measurement.

[0063] Signal transmission time before and after maintenance (T) trans Physical corrections: Considering the long cables in the broadcast equipment room, the system has a preset fixed cable delay parameter T. delay-fixed For example, for a standard transmission cable 50 meters long, a preset T... delay-fixed =0.25ms. When calculating the final transmission time, the system automatically deducts this value from the total latency to ensure that the indicator only reflects the processing performance of the device itself.

[0064] 3. Determination of weighting coefficients (AHP method implementation process) The indicator weights in this embodiment are not arbitrarily set, but are rigorously calculated using the Analytic Hierarchy Process (AHP). The specific process is as follows: (1) Constructing the judgment matrix: Five senior operations engineers with over 10 years of experience were invited to score the pairwise importance of the six indicators (using a 1-9 scale). For example, "alarm improvement rate" was considered slightly more important than "operational stability," and assigned a value of 1.1. The final judgment matrix is ​​as follows: Where: a ij Let be the importance ratio of the i-th type of indicator to the j-th type of indicator, satisfying a ij =1 / a ji ; (2) Calculate the weight vector: The weights are calculated using the square root method. (2-1) Calculate the product M of the elements in each row of the matrix. i =Π 6 j=1 a ij ; (2-2) Calculate M i The sixth root of w' i ; (2-3) Normalize the vector w': w i =w' i / ∑ 6 k=1 w' k .

[0065] The final characteristic vector obtained is W=[0.20,0.18,0.22,0.15,0.10,0.15] T .

[0066] (3) Consistency check: To ensure that the expert scoring logic is not conflicting, the maximum eigenvalue λ of the judgment matrix is ​​calculated. max and the consistency index CI=(λ) max -6) / (6-1). Looking up the table, the average random consistency index RI for a 6th-order matrix is ​​1.24. The consistency ratio CR = CI / RI = 0.08 < 0.1, indicating that the judgment matrix has satisfactory consistency. The above weighting coefficients are effective and can be used for subsequent feature-level fusion calculations.

[0067] (4) Final weights: The normalized feature vectors are the final weights: S run (0.20), F main (0.18), R alarm (0.22), SNR (0.15), THD (0.10), T trans (0.15).

[0068] 4. Dynamic threshold preprocessing To adapt to equipment aging, the system automatically calculates the current usage time t (months) based on the equipment's manufacturing date.

[0069] For example, for a device that has been used for 24 months (2 years), the dynamic maximum threshold X of its positive metric is... max (24)=X max-static ×(1+0.05×floor(24 / 12))=1.1×X max-static This means that the system allows for a reasonable 10% decrease in the performance benchmark of older equipment, avoiding false alarms caused by being overly strict with older equipment.

[0070] 5. Second-level fusion and model reasoning Feature-level fusion: A time decay factor of α=0.95 is used to reduce the weight of historical data from one year ago on the current assessment to 0.95 of the initial value. 12 ≈54%, which reflects the principle that the recent state is more important.

[0071] Decision-level fusion (LSTM model): Model structure: Input layer (12 nodes) - LSTM hidden layer 1 (24 nodes) - LSTM hidden layer 2 (16 nodes) - Dropout layer - Output layer (4 nodes).

[0072] Lightweight Training (Knowledge Distillation): To meet the 80ms inference speed requirement of embedded systems, this model employs knowledge distillation during the training phase. First, a teacher network with a large parameter set is trained (achieving 94.2% accuracy). Then, its output soft labels guide the training of the current lightweight student network. Ultimately, this lightweight model maintains extremely high accuracy while reducing the number of parameters by 60%.

[0073] Special handling for the Dropout layer: The Dropout rate is set to 0.2. This layer is only enabled during offline training to prevent overfitting; it is completely disabled during online inference (i.e., treated as fully connected) to ensure the determinism and stability of the fault prediction results.

[0074] Example 2: Actual warning process of AE-2023 audio encoder An AE-2023 audio encoder that has been in service for 2 years was selected as the test object, and the statistical period was set to 3 months (i.e., T). cycle =3).

[0075] 1. Data Calculation Stability (S) run ): The actual runtime T collected. act =1720h, unplanned downtime T down =35h, planned downtime T plan =48h, standard duration T std =1500h.

[0076] Initial calculation of S run =(1720-35-48) / 1500×100%≈109.1.

[0077] Anomaly Handling: Since the result is greater than 100%, data anomaly verification is triggered. The system automatically retrieves the previous 3 valid data points (102%, 98%, and 105%), and takes their average of 101.7% as the corrected S. run .

[0078] Maintenance frequency (F) main ): The records show that there was one major overhaul (weight 1.0) and one minor overhaul that lasted 2 hours (calculated as 1.6 times, weight 0.3).

[0079] F main =(1×1.0+1.6×0.3) / 3≈0.49 times / month.

[0080] Audio metrics: Actual SNR measured after maintenance after =51dB (meets standards), THD after =0.6% (meets the standard), T trans-after =41ms.

[0081] 2. Integration Assessment The system normalizes the above data using a dynamic threshold, combines it with historical health feature values ​​(which are smoothed to 0.76), and inputs them into the LSTM model.

[0082] Model output probability vector: Phealthy = 0.12, Psub-healthy = 0.88, Pfailure risk = 0.00, Pfailure = 0.00.

[0083] 3. Result Determination and Command Output Judgment: Based on the probability vector, the highest probability corresponds to the "sub-healthy" state.

[0084] Consistency check: The system checked various indicators and found that although the audio quality met the standards, the maintenance frequency was slightly high and the historical trend showed a slow decline, which is consistent with the characteristics of sub-health.

[0085] Command Generation: The system automatically generates the following maintenance work orders and pushes them to the maintenance personnel's mobile app: [Operations and Maintenance Commands - Sub-health] Device ID: AE-2023-04 Recommended action: Focus on monitoring alarm frequency and transmission time.

[0086] Execution timeframe: Review within 15 days.

[0087] 4. Implementation Results The maintenance personnel conducted a targeted inspection of the equipment according to the instructions and found slight oxidation on the transmission link interface. After cleaning the interface, a re-evaluation was performed 30 days later, and the model output P. 健康 =0.93, the equipment returned to a healthy state. This process verifies that the present invention can detect potential problems in advance and avoid sudden broadcast interruptions.

[0088] As can be seen from the above embodiments, the present invention achieves accurate early warning and closed-loop management of the health status of broadcasting equipment through refined indicator calculation (such as 1.6 times conversion and THD adaptive extraction), scientific weight allocation (AHP), and targeted model optimization (knowledge distillation and Dropout strategy).

[0089] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0090] Although this document uses various terms extensively, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of the invention; interpreting them as any additional limitation would contradict the spirit of the invention.

Claims

1. A method for early warning of faults in broadcast audio transmission equipment based on multimodal data fusion, characterized in that, Includes the following steps: S1: Construct a six-dimensional multimodal evaluation index system adapted to broadcast audio transmission equipment. The index system includes: equipment continuous operation stability index, equipment maintenance frequency index, alarm improvement rate before and after maintenance index, audio signal-to-noise ratio before and after maintenance index, audio total harmonic distortion before and after maintenance index, and signal transmission time before and after maintenance index. S2: Collect the operating data and audio signal data of the broadcast audio transmission equipment, and calculate the values ​​of each indicator in the six-dimensional multimodal evaluation index system; S3: Preprocess the calculated values ​​of each indicator. The preprocessing includes outlier removal based on the 3σ criterion and normalization based on dynamic threshold to obtain the normalized values ​​of each indicator. S4: Perform secondary data fusion processing: First-level feature fusion: Based on preset weight coefficients and time decay factors, the normalized values ​​of the indicators are weighted and summed and smoothed to obtain health feature values ​​that reflect the long-term health trend of the equipment. Second-level decision-level fusion: The health feature values ​​and the normalized values ​​of each indicator are used as input vectors and input into a pre-trained lightweight long short-term memory network model to output a probability vector representing the health status of the device. S5: Determine the current health level of the device based on the probability vector, and generate and output the corresponding operation and maintenance instructions according to the current health level; The operation and maintenance instructions include at least key monitoring instructions for sub-healthy states, preventive maintenance instructions for fault risk states, and shutdown maintenance instructions for fault states.

2. The method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S2, the continuous operation stability index S of the device run The calculation formula is: S run =[(T act -T down -T plan ) / T std ]×100%; Among them, T act To measure the total actual operating time of the equipment within the statistical period, data is collected in real time via a PLC module at a frequency of once per minute; T down The unplanned downtime of equipment within the statistical period must meet the T standard. down <T act ;T plan T represents the planned maintenance downtime of equipment within the statistical period. std The standard continuous operating time for equipment design; When the calculated S run <0 or S run If the percentage is greater than 100%, the data is considered abnormal, and the mean of the previous three valid data points is used as the replacement.

3. The method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S2, the equipment maintenance frequency index F main The calculation formula is: F main =(N main ×w main +N minor ×w minor ) / T cycle ; Where, N main This refers to the number of major maintenance operations within the statistical period, where major maintenance is defined as the downtime T. repair Maintenance time ≥4 hours; w main This is the weighting coefficient for major maintenance. N minor To determine the number of minor maintenance operations within a statistical period, the downtime T is used as an example. repair For maintenance operations lasting less than 1 hour, each occurrence is recorded as a minor maintenance operation; for downtime of 1 hour ≤ T... repair Maintenance that takes less than 4 hours is counted as 1.6 minor maintenance incidents per incident; w minor This is a weighting factor for minor repairs; T cycle The statistical period is in months.

4. The method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S2, the alarm improvement rate index R before and after maintenance is... alarm The calculation formula is: R alarm =max(((F alarm-before -F alarm-after ) / F alarm-before )×100%,-50%); Among them, F alarm-before and F alarm-after These represent the weighted alarm frequency before maintenance and the weighted alarm frequency after maintenance, respectively. Both are calculated using the following weighted alarm frequency formula: F=(∑ n i=1 N alarm-i ×S i ) / T run ; For F alarm-before Calculation: The statistical period is 30 days before the maintenance, where N i T represents the number of alarms of type i within this period. run This refers to the effective operating time of the equipment within this period; For F alarm-after Calculation: The statistical period is 30 days after the maintenance, where N i T represents the number of alarms of type i within this period. run This refers to the effective operating time of the equipment within this period; S i is the severity coefficient for the i-th type of alarm.

5. The method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S2, the total harmonic distortion (THD) in the audio total harmonic distortion index before and after maintenance is calculated using an adaptive fundamental frequency extraction and reference signal comparison method, specifically including the following sub-steps: S21: The instantaneous fundamental frequency f0(t) of the audio signal is extracted using the Teager-Kaiser energy operator combined with a sliding window. The formula for calculating the energy operator is as follows: f0(t)=(1 / 2π)×sqrt(((x(t+1)-x(t)) 2 -(x(t+1)-2x(t)+x(t-1))(x(t)-x(t-1))) / x(t) 2 ); Where x(t) is the audio sample value at time t; the extracted f0(t) is smoothed to obtain the stable fundamental wave sequence f'0(t); S22: Obtain the calibrated program source backup link signal as the reference signal x ref (t), and the output signal x of the device under test. test (t) Perform timestamp synchronization; S23: For x ref (t) and x test (t) Perform Fourier transforms to obtain the spectrum X ref (t) and X test (t), calculate the amplitude difference A at integer multiples of frequency k×f'0 of f'0(t). dist (k×f'0)=|X test (k×f'0)|-|X ref (k×f'0)|, where k=2,3,…,10, and 0 is taken if the result is negative; S24: Calculate the total harmonic distortion (THD) based on the amplitude difference. THD=(sqrt(∑(A dist (k×f'0)) 2 ) / |X ref (f'0)| 2 )×100%, where the summation applies to k=2 to 10.

6. The method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S3, the normalization process based on the dynamic threshold specifically includes: For positive indicators, the normalized value X norm =(XX min (t)) / (X max (t)-X min (t)); For negative indicators, the normalized value X norm =(X max (t)-X) / (X max (t)-X min (t)); Among them, the dynamic maximum threshold X max (t) and dynamic minimum threshold X min (t) varies with the equipment usage time t, and the calculation formula is: X max (t)=X max-static ×(1+0.05×floor(t / 12)); X min (t)=X min-static ×(1-0.03×floor(t / 12)); X max-static and X min-static These are the static maximum and static minimum values ​​of the device in its initial state, respectively.

7. A method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S4, the health feature value H is calculated in the first-level feature-level fusion. feat The formula for (t) is: H feat (t)=∑(w i ×X i-norm (t)×α t ); Among them, w i Let X be the weighting coefficient of the i-th type of indicator. i-norm (t) is the normalized value of the i-th index at time t, and α is the time decay factor; Furthermore, the exponential moving average algorithm was used to analyze H. feat (t) is smoothed: H feat-smoothed (t)=β×H feat (t)+(1-β)×H feat-smoothed (t-1); Among them, H feat-smoothed (t) represents the smoothed health feature value at time t, and β is the smoothing coefficient.

8. The method for fault early warning of broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S4, the lightweight long short-term memory network model in the second-level decision-level fusion has the following structural features: The input layer contains 12 neurons, and the input vector it receives consists of 6 normalized values ​​of the six-dimensional multimodal evaluation index system and the rate of change of these 6 indexes. The first hidden layer is an LSTM layer containing 24 neurons, with the forget gate threshold set to 0.5; The second hidden layer is an LSTM layer containing 16 neurons, with a cell state update coefficient set to 0.

8. Dropout layer, with a dropout rate set to 0.2, is only enabled during the training phase; The output layer contains 4 neurons, using the Softmax activation function, and the output probability vector corresponds to four states: healthy, sub-healthy, fault risk, and fault.

9. A method for early warning of faults in broadcast audio transmission equipment based on multimodal data fusion according to claim 1, characterized in that, In step S5, generating the corresponding operation and maintenance instructions specifically includes: When the probability vector indicates that the device is in a sub-healthy state, an operation and maintenance instruction containing the content of "focus on monitoring alarm frequency and transmission time, and re-check within 15 days" is generated; When the probability vector indicates that the device is in a fault risk state, an operation and maintenance instruction containing the content "carry out preventive maintenance within 72 hours, and prioritize checking the audio module" is generated; When the probability vector indicates that the device is in a fault state, an operation and maintenance instruction containing the content "immediately stop the machine for maintenance and check the power supply and signal transmission links" is generated.

10. A fault early warning system for broadcast audio transmission equipment based on multimodal data fusion, characterized in that, The system, which operates according to the method of claim 1, comprises: The data acquisition and index calculation module is configured to collect operating data and audio signal data of broadcast audio transmission equipment, and calculate six-dimensional index values ​​including equipment continuous operation stability, maintenance frequency, alarm improvement rate, audio signal-to-noise ratio, audio total harmonic distortion and signal transmission time. The data preprocessing module is configured to remove outliers based on the 3σ criterion and normalize the index values ​​using a dynamic threshold that changes with the duration of device use. The secondary fusion judgment module is configured to first calculate health feature values ​​based on weight coefficients and time decay factors, and then input the health feature values ​​and index normalized values ​​into a preset lightweight long short-term memory network model to output the device health status probability. The early warning interaction module is configured to determine the health level based on the probability of the device's health status and generate and output corresponding operation and maintenance instructions.

Images