Multimedia equipment operation and maintenance management system based on AI

Abstract

The invention relates to the technical field of audio and video analysis and data processing, in particular to an AI-based multimedia equipment operation and maintenance management system, which comprises a data acquisition module, an analysis module, an equipment state modeling module, a fault prediction module, a maintenance strategy optimization module, a resource allocation module and a visual platform. The data acquisition module acquires temperature, vibration, audio and video feature data of the equipment through the sensor and the video equipment, and the data are preprocessed and then transmitted to the cloud; the equipment state modeling module constructs a dynamic baseline model based on historical data, abnormal signals are generated through real-time comparison, and the visual platform realizes multi-dimensional state monitoring through 3D topology rendering, AR auxiliary diagnosis and early warning linkage. The system solves the problems of response lag and low resource utilization rate caused by dependence on manual operation and maintenance in the prior art through a closed-loop management mechanism driven by a data stream, and improves the fault diagnosis precision and the operation and maintenance efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an AI-based multimedia equipment operation and maintenance management system. Background Art

[0002] With the popularization of multimedia equipment in education, business, and conference scenarios, its operation and maintenance management faces the challenge of insufficient efficiency and intelligence. The existing operation and maintenance model mainly relies on manual inspections and post-fault handling, which makes it difficult to monitor the operating status of equipment in real time, resulting in the inability to identify potential hidden dangers in a timely manner. Manual inspections are time-consuming and labor-intensive, and delayed responses can easily cause business interruptions. In addition, the lack of means to collect and analyze equipment operation data makes it difficult to predict performance trends, prevent faults, and optimize resources, resulting in a lack of basis for maintenance costs. In existing technologies, passive maintenance and decentralized management further limit the efficiency of equipment utilization and cannot meet the needs of high stability and efficient operation and maintenance. There is an urgent need to achieve full life cycle monitoring and proactive maintenance of equipment through intelligent means. Summary of the Invention

[0003] In response to the shortcomings of existing technologies, the present invention provides an AI-based multimedia equipment operation and maintenance management system to solve the problems of insufficient real-time monitoring, delayed fault response, high maintenance costs and lack of data support for resource optimization caused by the existing reliance on manual inspections for multimedia equipment operation and maintenance.

[0004] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: The present invention provides an AI-based multimedia equipment operation and maintenance management system, which includes: a data acquisition module, an analysis module, an equipment status modeling module, a fault prediction module, a maintenance strategy optimization module, a resource allocation module, an operation and maintenance center decision module, and a visualization platform; The data acquisition module is used to collect the operating data of multimedia devices through sensors and video equipment, and pre-process the data before transmitting it to the cloud. The data acquisition module includes a non-contact video monitoring unit and an audio analysis unit. The non-contact video monitoring unit is used to capture changes in the device's appearance and extract key frame features through frame differencing. The audio analysis unit is used to extract voiceprint features from the device's operating noise through Mel-frequency cepstral coefficients. The analysis module analyzes the performance quality of multimedia devices in real time based on the audio and video data obtained by audio and video sensors, obtains quality evaluation results, and feeds the quality evaluation results back to the decision module of the operation and maintenance center; The equipment status modeling module is used to build a dynamic baseline model based on historical data, compare the collected data with the baseline model in real time to generate abnormal signals, and send the abnormal signals to the fault prediction module; The fault prediction module is used to generate fault diagnosis results based on the fault modes in the knowledge graph based on abnormal signals, fuse multimodal data, and trigger the maintenance strategy optimization module; The maintenance strategy optimization module is used to calculate the equipment health score based on the fault diagnosis results, generate the maintenance priority queue, and output the work order after verifying the maintenance plan through digital twin simulation; The resource allocation module is used to generate resource scheduling plans based on device performance tags and business demand forecasts, and update the device deployment topology map; The visualization platform is used to display equipment health status, maintenance work order progress, and resource heat maps in real time, and supports AR-assisted diagnosis and cross-module data linkage.

[0005] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention further includes the following: Non-contact video monitoring unit, used to capture changes in the device's appearance and extract key frame features through frame differencing; An audio analysis unit, used to extract voiceprint features from device operating noise using Mel-frequency cepstral coefficients; The data encoding unit is used to uniformly encode key frame features, voiceprint features, and temperature and power consumption data collected by sensors into a structured data stream, and transmit it to the device state modeling module for baseline model comparison.

[0006] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention includes a device status modeling module: Dynamic baseline update unit, used to retrain the ARIMA and LSTM fusion model on historical data based on the sliding window mechanism to generate dynamic parameter fluctuation thresholds; The anomaly correlation analysis unit is used to retrieve the operating data of equipment in the same area associated with the abnormal signal from the cloud data lake, calculate the probability of anomaly propagation through the Pearson correlation coefficient, and distinguish between single equipment failures and systemic risks.

[0007] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention includes a fault prediction module: A knowledge graph query unit is used to retrieve related failure modes and historical maintenance cases from a pre-built three-level failure knowledge graph based on the equipment type and failure type included in the abnormal signal; The multimodal reasoning unit is used to fuse audio data and video data, video key frame features, and equipment deployment environment parameters through a graph attention network to generate fault root cause analysis results and trigger the maintenance strategy optimization module.

[0008] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention includes a maintenance strategy optimization module: A health scoring unit is used to calculate the equipment health score based on the failure probability in the fault root cause analysis results, the output value of the equipment remaining life prediction model and the preset service priority weight; The digital twin simulation unit is used to generate maintenance priority queues based on equipment health scores, load equipment deployment topology maps in a virtual environment, and simulate the impact of maintenance operations on business continuity to optimize scheduling plans.

[0009] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention includes a resource allocation module: The performance clustering unit is used to generate device performance labels based on device health scores and historical failure frequencies using the K-means clustering algorithm; The load balancing unit is used to calculate the optimal device deployment plan through genetic algorithms based on device performance tags and business demand predictions, and update the resource topology map to the visualization platform for dynamic rendering.

[0010] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention includes a visualization platform comprising: 3D topology rendering unit, used to dynamically colorize device models based on device health scores and overlay real-time parameter curves of audio and video data; AR-assisted diagnosis unit, used to scan the device through a mobile terminal and overlay a virtual structure diagram corresponding to the fault root cause analysis results, marking the faulty components and maintenance instructions; The early warning linkage unit is used to display early warning information according to the fault probability level, and link the jump to the root cause result page and the resource scheduling solution recommendation page.

[0011] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention further includes: The data encoding unit integrates a Transformer-based multimodal feature adaptive fusion optimization algorithm; The multimodal feature adaptive fusion optimization algorithm aligns video key frame features, voiceprint features, and the temporal dimensions of audio and video data through a self-attention mechanism, dynamically calculates the fusion weights of each modal feature, suppresses redundant feature interference, and enhances the key feature representation capability, generating a high-dimensional, low-noise structured data stream to improve the accuracy of baseline model comparison in the device state modeling module.

[0012] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention also includes a microservice simulation verification system; The microservice simulation and verification system consists of data simulation services, policy verification services, and risk deduction services. It is deployed in Docker containers and orchestrated through Kubernetes. The data simulation service generates simulated equipment operation scenarios including anomaly injection based on historical operation data and outputs multimodal simulation data sets; The strategy verification service calls the virtual topology map of the digital twin simulation unit, simulates the maintenance operations of each device in the maintenance priority queue, calculates the service interruption duration, spare parts consumption forecast value and the impact of related equipment; the risk deduction service calls the efficiency label data of the resource allocation module through the microservice interface, simulates the robustness of the equipment deployment plan under sudden failures, and generates risk response recommendations.

[0013] Furthermore, the AI-based multimedia device operation and maintenance management system of the present invention further includes a model dynamic fine-tuning unit; The model dynamic fine-tuning unit uses incremental learning technology to dynamically adjust the graph attention network parameters and ARIMA-LSTM fusion model weights based on the simulation error data output by the microservice simulation verification system. The simulation error data includes the fault root cause prediction deviation and baseline threshold matching error. When the mean square error between the simulation prediction value and the actual operating data exceeds the preset threshold, the model fine-tuning process is triggered. The simulation annotation data is introduced through transfer learning to update the model parameters, and the fault mode association rules in the three-level fault knowledge graph are simultaneously updated to improve the accuracy of the fault root cause analysis results and the scenario adaptability of the dynamic baseline model.

[0014] Beneficial effects of the present invention: The present invention uses a data acquisition module to collect temperature, vibration, audio, and video feature data of multimedia devices in real time, generates abnormal signals through dynamic baseline model comparison, and accurately locates the root cause by combining a three-level fault knowledge graph with multimodal reasoning technology, thus solving the problem of delayed fault response caused by existing operation and maintenance relying on manual inspections; the maintenance strategy optimization module generates the optimal maintenance plan based on health scores and digital twin simulation, and the resource allocation module dynamically adjusts equipment deployment through efficiency clustering and genetic algorithms, achieving collaborative optimization of resource utilization efficiency and business continuity; the visualization platform uses 3D topology rendering and AR-assisted diagnosis technology to display equipment status, maintenance work orders, and resource heat maps in a multi-dimensional linkage manner, thereby improving the intuitiveness and execution efficiency of operation and maintenance decisions. Each module forms a complete technical chain from anomaly detection, diagnostic reasoning to resource scheduling through a closed-loop management mechanism driven by data flow, significantly reducing the cost of manual intervention and improving equipment reliability and operation and maintenance response accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings without paying any creative labor.

[0016] Figure 1 This is a system architecture diagram of an AI-based multimedia device operation and maintenance management system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The technical solutions provided by each embodiment of the present invention are described in detail below in conjunction with the drawings. In order to better understand the purpose of the present invention, the present invention is further described in detail below.

[0018] See also Figure 1 , the present invention provides an AI-based multimedia equipment operation and maintenance management system, including: a data acquisition module, an equipment status modeling module, a fault prediction module, a maintenance strategy optimization module, a resource allocation module and a visualization platform; The data acquisition module is used to collect the operating data of multimedia devices through sensors and video equipment, and pre-process the data before transmitting it to the cloud; The analysis module analyzes the performance quality of multimedia devices in real time based on the audio and video data obtained by audio and video sensors, obtains quality evaluation results, and feeds the quality evaluation results back to the decision module of the operation and maintenance center; The equipment status modeling module is used to build a dynamic baseline model based on historical data, compare the collected data with the baseline model in real time to generate abnormal signals, and send the abnormal signals to the fault prediction module; The fault prediction module is used to generate fault diagnosis results based on the fault modes in the knowledge graph based on abnormal signals, fuse multimodal data, and trigger the maintenance strategy optimization module; The maintenance strategy optimization module is used to calculate the equipment health score based on the fault diagnosis results, generate the maintenance priority queue, and output the work order after verifying the maintenance plan through digital twin simulation; The resource allocation module is used to generate resource scheduling plans based on device performance tags and business demand forecasts, and update the device deployment topology map; The visualization platform is used to display equipment health status, maintenance work order progress, and resource heat maps in real time, and supports AR-assisted diagnosis and cross-module data linkage.

[0019] The data acquisition module captures changes in the device's appearance through non-contact video monitoring units deployed at key locations on multimedia devices. It uses a frame difference algorithm to analyze continuous video streams, identifying physical anomalies such as deformation of the device's housing, dust accumulation on the heat dissipation port, or screen color spots. It extracts key frame features and generates a time series of appearance changes. The audio analysis unit uses a high-sensitivity microphone array to collect device operating noise, applies Mel-frequency cepstral coefficients to frame and window the audio signal, maps it to a Mel-scale filter bank via Fourier transform, and extracts voiceprint feature vectors that characterize mechanical wear or abnormal fan noise. The data encoding unit receives video key frame features, voiceprint features, and real-time parameters collected by temperature sensors and power consumption sensors, encodes them into a structured data stream according to device identification and timestamps, and transmits them to cloud storage via a message queue. This process establishes a multimodal data foundation for device status monitoring, providing a unified format input source for subsequent status modeling.

[0020] The analysis module uses the raw audio and video data collected by the audio and video sensors to analyze audio quality parameters using a voiceprint feature comparison algorithm. It also employs an image recognition algorithm to detect performance indicators such as color distortion and signal delay in the video output. A weighted fusion of the audio quality score and video performance indicators is then generated to generate a comprehensive quality evaluation result, which is then sent to the decision-making module in the operations and maintenance center. This step enables real-time quantitative evaluation of device performance, providing an objective basis for operations and maintenance decisions.

[0021] The device status modeling module, based on a cloud-based historical data warehouse, uses a sliding window mechanism to periodically call the ARIMA time series model and LSTM neural network to construct a dynamic baseline model. The ARIMA model captures the linear trend characteristics of device operating parameters, while the LSTM network learns the nonlinear fluctuation patterns of the parameters. The outputs of these two models are fused to generate dynamic parameter fluctuation thresholds. After inputting real-time data streams, a time series comparison algorithm calculates the deviation between the collected data and the baseline model. When core parameters such as temperature and vibration frequency exceed dynamic thresholds, an anomaly signal is generated. The anomaly signal is tagged with the device location and the deviation dimension and transmitted to the fault prediction module to trigger the diagnostic process. This step establishes dynamic evaluation criteria for device health status and enables automated identification of anomalies.

[0022] After the fault prediction module receives the anomaly signal, the knowledge graph query unit parses the device type code and anomaly type label and retrieves matching fault mode nodes from the pre-built three-level fault knowledge graph. The first-level nodes associate device type classifications, the second-level nodes map fault modes, and the third-level nodes link root cause features and historical maintenance solutions. The multimodal reasoning unit inputs real-time audio and video data, video keyframe features, and equipment deployment environment parameters into the graph attention network. Using the attention weight matrix, it calculates the feature correlation between different data sources and fuses them to output a probability distribution of the root cause of the fault. The root cause analysis results are linked to the solution data in the knowledge graph, triggering the maintenance strategy optimization module. This process enables accurate mapping of fault modes and root cause reasoning, providing a diagnostic basis for maintenance decisions.

[0023] The health scoring unit of the maintenance strategy optimization module uses a weighted algorithm to generate a device health score based on the failure probability value in the root cause analysis results, the output value of the equipment remaining life prediction model, and the preset business impact factor. A maintenance priority queue is constructed according to the descending order of the score value, and high-risk equipment requiring emergency intervention is marked. The digital twin simulation unit loads a virtual model of the equipment deployment topology diagram, imports the maintenance queue data, and simulates operation plans such as component replacement and firmware upgrade. It calculates the impact time and resource consumption of each plan on business continuity and outputs the optimized maintenance work order. This step converts the diagnostic results into an executable maintenance plan and improves the feasibility of the plan through simulation verification.

[0024] The resource allocation module's performance clustering unit uses health scores and historical fault records to classify device performance using the K-means clustering algorithm, generating three performance labels: high, medium, and low. The load balancing unit constructs a multi-objective function for optimizing device deployment based on the distribution of performance labels and business demand forecasts. It uses a genetic algorithm to iteratively calculate the optimal device migration path and spare parts allocation plan. The output drives the structural update of the device deployment topology and pushes the scheduling plan to the visualization platform. This process dynamically optimizes resource utilization and ensures business continuity.

[0025] The 3D topology rendering unit of the visualization platform calls the equipment health score data and colors the equipment model into three states: healthy, warning, and faulty according to the preset threshold. Real-time audio data and video data curves are superimposed on the surface of the three-dimensional model, and the degree of parameter deviation from the baseline is reflected through gradient color levels. The AR-assisted diagnosis unit scans the physical equipment through the mobile terminal and superimposes the three-dimensional structure diagram of the faulty component and the maintenance operation guidance animation on the camera screen. The early warning linkage unit displays the alarm information in a graded manner according to the fault probability value. Click the alarm icon to jump to the root cause analysis details page and the resource scheduling recommendation page. Each display unit realizes data linkage through an event-driven mechanism. For example, the update of the work order status triggers the real-time re-rendering of the topology map. This step establishes visual monitoring of the entire operation and maintenance process to improve the efficiency of decision-making execution.

[0026] Data flows between modules through distributed messaging middleware: the structured data flow of the data acquisition module triggers baseline comparison in the state modeling module, abnormal signals drive knowledge graph retrieval in the fault prediction module, root cause analysis results activate scoring calculation in the maintenance strategy optimization module, and maintenance work order updates trigger topology reconstruction in the resource allocation module, ultimately achieving closed-loop monitoring through a visualization platform. This data flow-driven collaborative mechanism forms a complete technical chain, enabling automated management of the entire process from condition monitoring and fault diagnosis to resource scheduling.

[0027] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention, the data acquisition module also includes: Non-contact video monitoring unit, used to capture changes in the device's appearance and extract key frame features through frame differencing; An audio analysis unit, used to extract voiceprint features from device operating noise using Mel-frequency cepstral coefficients; The data encoding unit is used to uniformly encode key frame features, voiceprint features, and temperature and power consumption data collected by sensors into a structured data stream, and transmit it to the device state modeling module for baseline model comparison.

[0028] In the AI-based multimedia device operation and maintenance management system of the present invention, the data acquisition module continuously monitors the appearance status of multimedia devices through a non-contact video monitoring unit. It uses a camera to capture a continuous video stream of the device during operation, employs frame differencing to calculate pixel differences between adjacent frames, identifies physical state changes such as device housing deformation, dust accumulation on the heat dissipation vents, or screen color spots, extracts keyframe features, and generates a time series of device appearance changes. The audio analysis unit deploys a high-sensitivity microphone array to collect device operating noise. It performs frame-by-frame windowing on the audio signal using Mel-frequency cepstral coefficients, maps it to a Mel-scale filter bank after Fourier transformation, and extracts soundprint feature vectors that characterize mechanical wear or abnormal fan noise, forming a device acoustic state label. The data encoding unit receives video keyframe features, soundprint features, and real-time parameters collected by temperature and power sensors. It encodes the multimodal data into a structured data stream in JSON format according to device ID and timestamp. This data is then pushed to a cloud message queue via the MQTT protocol, divided by topic, for invocation by the device state modeling module and real-time comparison with the baseline model, thereby establishing a multidimensional monitoring data foundation for the device's operating status.

[0029] The key frame features extracted by the video monitoring unit are compressed by the edge node and transmitted synchronously with the voiceprint features to avoid excessive data stream bandwidth usage; the audio analysis unit uses a noise suppression algorithm to filter environmental background noise before extracting the voiceprint features, thereby improving the feature differentiation of equipment operation noise. The data encoding unit embeds the device type code and sensor location identifier in the structured data stream, enabling the device status modeling module to match the corresponding dynamic baseline model according to the device type, and at the same time associate the physical source of the abnormal signal according to the sensor location. After the structured data stream is transmitted to the cloud, the device status modeling module calls the ARIMA-LSTM fusion model generated by historical data training, and compares the real-time data with the parameter fluctuation threshold of the baseline model item by item. When the temperature or vibration data exceeds the threshold, an abnormal signal is triggered, and a correlation analysis is performed with the operating data of the equipment in the same area to distinguish between local faults and systemic risks, forming a closed-loop logical link from data collection to anomaly detection.

[0030] The data collection frequency of the non-contact video monitoring unit and the audio analysis unit is dynamically adjusted according to the type of equipment. For example, the video sampling rate is increased for high-heat-generating equipment to capture changes in the heat dissipation state, and the audio sampling accuracy is improved for precision audio equipment to identify subtle abnormal sounds. The data encoding unit aligns the timestamps of multimodal data before transmission to synchronize the video, audio, and audio data with the video data in the time dimension, avoiding misjudgments due to timing deviations during model comparison. After receiving the structured data stream, the device status modeling module prioritizes marking high-confidence abnormal signals, and combines the predefined fault mode association rules in the knowledge graph to provide multi-dimensional data input for the subsequent fault prediction module, realizing an end-to-end processing flow from raw data collection to intelligent diagnosis.

[0031] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention includes a device status modeling module: Dynamic baseline update unit, used to retrain the ARIMA and LSTM fusion model on historical data based on the sliding window mechanism to generate dynamic parameter fluctuation thresholds; The anomaly correlation analysis unit is used to retrieve the operating data of equipment in the same area associated with the abnormal signal from the cloud data lake, calculate the probability of anomaly propagation through the Pearson correlation coefficient, and distinguish between single equipment failures and systemic risks.

[0032] In the AI-based multimedia device operation and maintenance management system of the present invention, the dynamic baseline update unit of the device status modeling module extracts historical operating data within a specified time window from the cloud data lake and retrains the baseline model that integrates the ARIMA model and the LSTM neural network at a preset period (for example, every 24 hours) based on a sliding window mechanism. The ARIMA model is used to capture the linear trends and periodic characteristics of device operating parameters, while the LSTM network learns the nonlinear dependencies in the data. The prediction results output by the two are weighted and fused to generate dynamic parameter fluctuation thresholds, reflecting in real time the impact of device aging and environmental changes on the parameter baseline. The dynamic baseline update unit pushes the updated thresholds to the anomaly detection process as a standard for real-time data comparison.

[0033] After receiving the anomaly signal output by the dynamic baseline model, the anomaly correlation analysis unit retrieves operational data from other devices of the same type and location as the anomaly device from the cloud data lake, extracting sensor parameters and video features within the same time window. The unit uses the Pearson correlation coefficient to calculate the correlation between the anomaly device's parameter fluctuations and the data from devices in the same location. If the correlation coefficient exceeds a set threshold (e.g., >0.7), the probability of anomaly propagation is determined to be high, triggering a systemic risk warning. If the correlation coefficient falls below the threshold, the unit flags the anomaly as an isolated single device failure. The anomaly correlation analysis unit appends the results, including the fault type label and risk level, to the anomaly signal for the fault prediction module to access.

[0034] During model training, the dynamic baseline update unit automatically adjusts the sliding window length based on the device type. For example, it uses a shorter window for high-load devices to quickly respond to parameter changes, and a longer window for stable devices to reduce noise interference. The trained fusion model outputs multi-dimensional parameter thresholds, including upper and lower limits for temperature, vibration frequency, and power consumption, and embeds the device model identifier to avoid misuse across device types. When retrieving data from devices in the same area, the anomaly correlation analysis unit prioritizes device datasets in the same deployment environment (e.g., consistent temperature and humidity conditions in the computer room). Data standardization eliminates the impact of environmental differences on correlation calculations, improving the accuracy of anomaly propagation analysis.

[0035] The parameter thresholds and anomaly correlation analysis results generated by the dynamic baseline model are synchronized to the visualization platform via a message queue. Abnormal devices and associated risk areas are color-coded in the device deployment topology. When an anomaly signal is flagged as a systemic risk, the resource allocation module automatically triggers preventive inspection tasks for devices in the same area and updates the maintenance priority queue. The dynamic baseline update unit records parameter threshold adjustment logs during model iterations and transmits them back to the knowledge graph as training data for fault mode association rules, forming a closed-loop feedback mechanism for model optimization and fault diagnosis.

[0036] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention includes a fault prediction module: A knowledge graph query unit is used to retrieve related failure modes and historical maintenance cases from a pre-built three-level failure knowledge graph based on the equipment type and failure type included in the abnormal signal; The multimodal reasoning unit is used to fuse audio data and video data, video key frame features, and equipment deployment environment parameters through a graph attention network to generate fault root cause analysis results and trigger the maintenance strategy optimization module.

[0037] In the AI-based multimedia equipment operation and maintenance management system of the present invention, after the knowledge graph query unit of the fault prediction module receives the abnormal signal sent by the device status modeling module, it parses the device type code and abnormal type label carried in the signal and retrieves the matching fault mode from the pre-built three-level fault knowledge graph. The first-level nodes of the knowledge graph are device type classifications (such as projectors and speakers), the second-level nodes are associated with fault modes (such as hardware damage and poor heat dissipation), and the third-level nodes are mapped to root cause features (such as capacitor aging and firmware version conflicts), while also associating solutions and component replacement records in historical maintenance logs. During the retrieval process, a semantic similarity algorithm is used to match the abnormal type description with the text features of the fault mode in the knowledge graph, and a list of candidate fault modes and the confidence scores of the associated maintenance cases are output.

[0038] After receiving the knowledge graph query results, the multimodal inference unit inputs the device's real-time audio and video data (temperature, vibration frequency), video keyframe features (dust accumulation on heat sinks, screen flicker frequency), and device deployment environment parameters (room temperature and humidity, power supply stability) into the graph attention network. Nodes in the network represent feature vectors from different data sources. Edge weights are dynamically calculated using an attention mechanism to capture the correlation between sudden changes in audio and video data and anomalies in video features, while also assessing the impact of environmental parameters on fault propagation. The network's output layer aggregates node features to generate a probability distribution for the fault root cause. Root cause labels with probabilities exceeding a threshold are screened and assigned solution recommendations, triggering the maintenance strategy optimization module to generate maintenance work orders.

[0039] The knowledge graph query unit prioritizes recent repair case data during retrieval and dynamically adjusts the weight of historical cases based on the length of time the equipment has been in use. For example, it prioritizes capacitor aging faults for equipment that has exceeded its warranty period. When integrating environmental parameters, the multimodal inference unit uses data standardization to eliminate interference from differences in temperature and humidity in the computer room on model inference. For example, temperature data is converted into an offset relative to the rated operating temperature of the equipment. The training data for the graph attention network includes labeled root cause labels and multimodal feature combinations. During training, a negative sampling strategy is used to balance the sample distribution of different fault types, improving the recognition accuracy of low-probability faults.

[0040] After the root cause analysis results are generated, they are pushed to the maintenance strategy optimization module via the messaging middleware, triggering the calculation of health scores and maintenance queue sorting. The knowledge graph query unit regularly receives feedback from maintenance work orders and updates the solution effectiveness scores in the third-level nodes. For example, it adjusts the recommended priority of maintenance strategies based on the recurrence rate of similar faults. When outputting root cause labels, the multimodal reasoning unit simultaneously generates an explainability report, recording the feature contribution of each data source and the influencing factors of environmental parameters. This report is then called up by the visualization platform and displayed overlaid on the AR-assisted diagnosis interface, forming a closed-loop decision-making chain from fault retrieval to root cause reasoning.

[0041] Specifically, the AI-based multimedia equipment operation and maintenance management system of the present invention includes a maintenance strategy optimization module: A health scoring unit is used to calculate the equipment health score based on the failure probability in the fault root cause analysis results, the output value of the equipment remaining life prediction model and the preset service priority weight; The digital twin simulation unit is used to generate maintenance priority queues based on equipment health scores, load equipment deployment topology maps in a virtual environment, and simulate the impact of maintenance operations on business continuity to optimize scheduling plans.

[0042] In the AI-based multimedia equipment operation and maintenance management system of the present invention, the health scoring unit of the maintenance strategy optimization module receives the root cause analysis results of the fault prediction module, extracts the probability of fault occurrence, the remaining life cycle value output by the equipment remaining life prediction model, and the preset business priority weight coefficient. The equipment health score is calculated by a weighted fusion algorithm, where the failure probability reflects the urgency of the current equipment status, the remaining life prediction value represents the long-term reliability of the equipment, and the business priority weight is dynamically adjusted according to the criticality of the equipment in the business scenario. After the health score is generated, a maintenance priority queue is generated by arranging the scores in descending order, giving priority to equipment with scores below the preset threshold, and marking high-risk equipment that requires urgent maintenance.

[0043] The digital twin simulation unit obtains the device deployment topology from the resource allocation module, loads the 3D digital twin model corresponding to the physical device, and imports the device information in the maintenance priority queue. Maintenance operations, including component replacement, firmware upgrades, and device migration plans, are simulated in a virtual environment, with real-time calculation of equipment downtime, spare parts consumption, and the impact on related businesses during maintenance. During the simulation, the maintenance sequence and resource allocation are dynamically adjusted. For example, maintenance tasks for equipment in the same area can be centrally scheduled to reduce personnel movement time, or replacement strategies can be optimized based on spare parts inventory status. The simulation results output cost-benefit analysis reports for multiple maintenance plans, selecting the optimal solution with the least impact on business continuity.

[0044] When calculating the weight coefficient, the health scoring unit associates the business demand forecast data in the visualization platform, for example, increasing the priority weight of the multimedia podium during peak teaching hours. When loading the topology diagram, the digital twin simulation unit synchronously accesses real-time audio and video data to update the virtual device status parameters, ensuring that the operating status of the simulation environment is consistent with that of the physical device. The maintenance plan generated by the simulation is pushed to the operation and maintenance personnel terminal through the work order system, and triggers the resource allocation module to update the spare parts inventory data and personnel scheduling plan. After the maintenance operation is performed, the latest status data of the equipment is transmitted back to the dynamic baseline model for incremental training to optimize the calculation accuracy of subsequent health scores.

[0045] The health scoring unit annotates maintenance dependencies between devices when generating a queue. For example, devices under the same power module require simultaneous maintenance to avoid duplicate downtime. The digital twin simulation unit incorporates a random event model into the simulation process to simulate the impact of sudden failures or delayed spare parts delivery on maintenance plans, thereby enhancing the robustness of the scheduling solution. After comparing maintenance work order execution results with simulation prediction data, the difference information is fed back into the knowledge graph to correct the weight parameters in the failure probability calculation model, forming a closed-loop optimization mechanism from score calculation to solution verification.

[0046] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention includes a resource allocation module: The performance clustering unit is used to generate device performance labels based on device health scores and historical failure frequencies using the K-means clustering algorithm; The load balancing unit is used to calculate the optimal device deployment plan through genetic algorithms based on device performance tags and business demand predictions, and update the resource topology map to the visualization platform for dynamic rendering.

[0047] In the AI-based multimedia device operation and maintenance management system of the present invention, the efficiency clustering unit of the resource allocation module receives device health scores and historical failure frequency data. After eliminating dimensional differences through data normalization, it constructs a two-dimensional feature vector including the health score and the number of failures. The feature vector is unsupervisedly classified using the K-means clustering algorithm. The number of cluster centers is set to reflect the device efficiency level (e.g., high efficiency, medium efficiency, low efficiency), and the device efficiency label is output and attached with a cluster confidence score. The efficiency label is dynamically updated based on the characteristic distribution of the device's cluster center. For example, devices with a continuously declining health score are automatically migrated to the low-efficiency cluster, triggering a resource scheduling warning.

[0048] The load balancing unit utilizes the performance tags and business demand forecast data generated by the performance clustering unit to model the device deployment optimization problem as a multi-objective optimization task, including minimizing resource idleness, balancing regional loads, and reducing failure risk. When initializing the population, the genetic algorithm prioritizes deployment plans with high-performance tags as high-quality genes. A crossover operation swaps device combinations across different regions, and a mutation operation introduces random device migration strategies to explore the solution space. The fitness function comprehensively evaluates the cost-effectiveness of deployment plans. After selecting the optimal solution that meets the business demand forecast, it generates device migration paths, spare parts allocation plans, and personnel scheduling instructions.

[0049] When updating performance labels, the performance clustering unit links real-time resource heat map data from the visualization platform and dynamically adjusts the cluster center threshold. For example, during peak business hours, health score requirements for devices in high-load areas can be relaxed to avoid insufficient available devices due to excessive clustering. During the genetic algorithm iteration process, the load balancing unit updates population parameters in real time based on changes in performance labels. For example, the migration priority of devices with low-efficiency labels increases as failure frequency increases, optimizing the algorithm's convergence speed.

[0050] The optimal deployment plan output by the load balancing unit is pushed to the visualization platform via an API, driving the dynamic rendering of the resource topology map. The topology map uses color gradients to distinguish device performance labels, overlaid with a heat map representing business demand forecasts to demonstrate the degree of resource shortage in the region. After the deployment plan is executed, actual device operating data is transmitted back to the performance clustering unit, triggering incremental updates to the clustering model. For example, cluster center locations are recalculated when the health of a migrated device improves, forming a closed-loop optimization chain from performance evaluation to resource scheduling.

[0051] Data exchange between the performance clustering unit and the load balancing unit is achieved through a distributed message queue, with performance tag update events triggering recalculation of the genetic algorithm. The dynamic rendering module of the resource topology map uses the device location coordinates and performance tags in the deployment plan, synchronizing them in real time to the AR-assisted diagnosis interface via the WebSocket protocol, assisting operations personnel in adjusting the equipment layout on-site. When generating migration paths, the load balancing unit incorporates the historical maintenance records of the digital twin simulation unit to avoid areas where recent maintenance plans conflict, improving the feasibility of the scheduling plan.

[0052] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention includes a visualization platform: 3D topology rendering unit, used to dynamically colorize device models based on device health scores and overlay real-time parameter curves of audio and video data; AR-assisted diagnosis unit, used to scan the device through a mobile terminal and overlay a virtual structure diagram corresponding to the fault root cause analysis results, marking the faulty components and maintenance instructions; The early warning linkage unit is used to display early warning information according to the fault probability level, and link the jump to the root cause result page and the resource scheduling solution recommendation page.

[0053] In the AI-based multimedia device operation and maintenance management system of the present invention, the 3D topology rendering unit of the visualization platform obtains the device health score from the device status modeling module, divides the device model into healthy (green), warning (yellow) and fault (red) states according to the score threshold, and superimposes the real-time audio data and video data curves transmitted by the data acquisition module on the model surface. The audio data and video data include temperature, vibration frequency and power consumption parameters. The curves are dynamically updated on the time axis, and the degree to which the parameters deviate from the baseline is reflected by color gradients. The rendered 3D topology map is projected to the visualization interface in real time through the WebGL engine, supports zooming and perspective switching, and synchronously displays the load heat map of the device deployment area.

[0054] The AR-assisted diagnosis unit uses a mobile device's camera to scan the physical device, invokes the root-cause analysis results from the fault prediction module, and overlays a 3D virtual diagram on the physical device, highlighting the location of faulty components (such as cooling fans and capacitor modules). The virtual diagram is linked to maintenance tutorials from the knowledge graph, presenting step-by-step disassembly procedures, spare part models, and installation precautions. Voice commands are also supported to switch between operation views. As maintenance personnel perform maintenance operations, the AR interface compares the steps with the standard process in real time, triggering deviation warnings and prompting corrective actions.

[0055] The early warning linkage unit receives probability-based data from the fault prediction module and categorizes warnings into red (immediate action), orange (action within 48 hours), and yellow (observation) levels. These warnings are displayed in a floating pop-up window in the visualization interface, synchronized with a flashing icon on the topology map. Clicking the warning icon links to the root cause analysis page, which displays historical similar fault cases, maintenance plan comparisons, and spare parts inventory and personnel schedule data from the resource scheduling recommendations page. The root cause analysis page includes a chart comparing the health status of equipment in the same area. The resource scheduling recommendations page recommends equipment migration paths and a list of replacement devices based on deployment plans generated by a genetic algorithm.

[0056] When coloring the equipment model, the 3D topology rendering unit uses the efficiency tag data from the resource allocation module to add texture markings to the model surface (e.g., stripes indicate inefficient equipment, grids indicate equipment to be phased out). When annotating faulty components, the AR-assisted diagnosis unit simultaneously accesses the historical maintenance records of the digital twin simulation unit, displaying the average repair time and spare parts consumption for similar faults. The early warning linkage unit's hierarchical logic links the health score threshold of the maintenance strategy optimization module. When the score falls below the critical value, the warning level is automatically upgraded, triggering the emergency dispatch task of the resource allocation module.

[0057] The visualization platform synchronizes with the data sources of each module in real time through the message middleware, and the update frequency of the 3D topology rendering is consistent with the anomaly detection cycle of the device status modeling module. When scanning the equipment, the AR-assisted diagnosis unit matches the multi-dimensional feature data in the cloud data lake through the device ID and dynamically loads the corresponding fault analysis model. After the early warning linkage unit jumps to the resource scheduling suggestion page, it automatically pre-fills the equipment information and maintenance suggestions required for the work order to reduce manual input errors. The data linkage between the various units is achieved through an event-driven architecture. For example, the warning level change event triggers the re-rendering of the topology map and the update of the AR interface, forming a seamless connection from status visualization to decision execution.

[0058] Specifically, in the AI-based multimedia device operation and maintenance management system of the present invention, the data encoding unit integrates a Transformer-based multimodal feature adaptive fusion optimization algorithm; The multimodal feature adaptive fusion optimization algorithm aligns video key frame features, voiceprint features, and the temporal dimensions of audio and video data through a self-attention mechanism, dynamically calculates the fusion weights of each modal feature, suppresses redundant feature interference, and enhances the key feature representation capability, generating a high-dimensional, low-noise structured data stream to improve the accuracy of baseline model comparison in the device state modeling module.

[0059] The Transformer-based multimodal feature adaptive fusion optimization algorithm integrated in the data encoding unit of the present invention has the following technical steps and logical associations: First, we perform temporal alignment on multimodal data. Due to differences in sampling frequencies between acquisition devices (e.g., video is typically acquired at 25 frames per second, audio at 44.1kHz, and sensors at 10Hz), video keyframe features, voiceprint features, and audio and video data require temporal alignment via a timestamp synchronization module. This module uses the acquisition time of the audio and video data as a benchmark to resample or interpolate video keyframe features (each frame corresponds to 0.04 seconds) and voiceprint features (each frame corresponds to 0.023 seconds), generating a multimodal feature sequence with unified timestamps. This ensures a consistent temporal relationship between the various modal data, providing a consistent input foundation for subsequent feature interactions within the self-attention mechanism.

[0060] Secondly, feature interaction and weight calculation are performed based on the self-attention mechanism. The aligned multimodal feature sequence (video keyframe feature matrix, voiceprint feature vector, audio data, and video data vector) is input into the Transformer model's query, key, and value linear transformation layers, generating semantically associated Q, K, and V vectors. Dot-product attention is used to calculate correlation scores between each modal feature. This is specifically demonstrated by the correlation between dust accumulation on the heat sink in the video keyframe and abnormal sensor temperature, and the matching degree between mechanical noise frequency in the voiceprint feature and sudden changes in vibration audio and video data. These scores are then normalized using softmax to form dynamic weight coefficients for each modal feature. Higher weights indicate a greater contribution of the feature to the device state representation.

[0061] Furthermore, redundant features are suppressed and key features are enhanced. During the feature fusion stage, the features of each modality are weighted and summed based on the attention weight. Low-weight features (such as the voiceprint component corresponding to environmental background noise and temperature fluctuations in non-core areas of the device) are naturally suppressed due to their low contribution, while high-weight features (such as sudden temperature changes in core equipment components and abnormal voiceprint frequency bands corresponding to mechanical wear) are strengthened and retained. At the same time, the fused features are nonlinearly transformed through the Transformer's feedforward neural network layer to further amplify the discriminability of key features. For example, the texture contrast of the screen color spot area in the video feature is enhanced, and the energy amplitude of the abnormal frequency band in the voiceprint feature is increased, forming a high-dimensional, high-discrimination fusion feature.

[0062] Finally, a structured data stream is generated and output. The fused high-dimensional features are mapped via a linear projection layer to a feature space (e.g., a 256-dimensional vector) that matches the input dimensions of the device state modeling module. Metadata such as the device's unique identifier (e.g., MAC address), timestamp, and sensor location code (e.g., the projector's heat dissipation vent sensor is labeled "P-01-TS") are appended to form a structured data stream containing multimodal feature information. This data stream is encapsulated in JSON format and pushed to a cloud message queue via the MQTT protocol, topic by topic. This data stream is then called by the device state modeling module and used for time series comparison with the dynamic baseline model. Because the fused features filter out redundant information and enhance key representations, they effectively improve the baseline model's accuracy in identifying device anomalies.

[0063] Each of these steps is closely linked: time alignment ensures temporal correspondence in multimodal data, providing a foundation for feature interaction; the self-attention mechanism distinguishes feature importance through dynamic weight calculation; redundancy suppression and key enhancement optimize feature quality; and structured encapsulation meets the input requirements of subsequent modules. Through this complete feature fusion process, the final output data stream more accurately reflects the device's operating status, thereby improving the accuracy of baseline model comparisons.

[0064] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention also includes a microservice simulation verification system; The microservice simulation and verification system consists of data simulation services, policy verification services, and risk deduction services. It is deployed in Docker containers and orchestrated through Kubernetes. The data simulation service generates simulated equipment operation scenarios including anomaly injection based on historical operation data and outputs multimodal simulation data sets; The strategy verification service calls the virtual topology map of the digital twin simulation unit, simulates the maintenance operations of each device in the maintenance priority queue, calculates the service interruption duration, spare parts consumption forecast value and the impact of related equipment; the risk deduction service calls the efficiency label data of the resource allocation module through the microservice interface, simulates the robustness of the equipment deployment plan under sudden failures, and generates risk response recommendations.

[0065] The specific implementation of the microservice simulation verification system of the present invention includes the following technical steps and logical associations: The microservice-based simulation verification system uses Docker containerization technology to independently package and deploy each service. First, for the data simulation service, it builds basic simulation scenario templates based on historical operational data stored in a cloud-based data lake (including multimodal historical datasets such as device temperature, vibration, voiceprint, and video keyframes). The data cleaning module filters out outliers and missing values, then constructs a basic simulation scenario template. The anomaly injection module perturbs sensor parameters (e.g., a 5°C temperature increase), voiceprint features (e.g., superimposing mechanical noise frequency bands), and video keyframes (e.g., simulating the expansion of screen color spots) in the basic scenario based on a predefined fault mode library (e.g., dust accumulation on heat sinks, capacitor aging, and firmware conflicts). This generates simulated device operation scenarios with various anomaly types, ranging from single device failures to systemic risks. The generated multimodal simulation dataset (including simulated video streams, simulated audio files, and simulated sensor time series data) is output to the policy verification service via a message queue, providing test input for subsequent policy verification.

[0066] The policy verification service uses a gRPC interface to access the virtual topology of the digital twin simulation units in the maintenance strategy optimization module (including 3D device models, deployment locations, and connectivity). This service loads the maintenance priority queue (a list of devices awaiting maintenance sorted by device health score). The simulation execution module sequentially simulates maintenance operations for each device in the queue, including component replacements (e.g., replacing a capacitor), firmware upgrades (e.g., refreshing a projector's firmware), and device migrations (e.g., moving a faulty speaker to a spare area). The maintenance start and end times are recorded to calculate service interruption duration. The spare parts inventory management interface deducts the corresponding component inventory (e.g., reducing capacitor inventory by one unit) to calculate spare parts consumption forecasts. Furthermore, the operating status of associated devices (e.g., other devices under the same power module) is monitored (e.g., voltage fluctuations) to assess the impact of the maintenance operation on these devices. The simulation output is a multi-dimensional assessment report that includes outage duration, spare parts consumption, and associated impacts. This report is fed back to the maintenance strategy optimization module to optimize the maintenance plan.

[0067] The risk simulation service uses a RESTful API to access the resource allocation module's performance tag data (including clustering results for high-efficiency, medium-efficiency, and low-efficiency devices) to build a simulation environment for device deployment plans (including current device locations, performance tags, and business demand forecast data). The sudden fault injection module simulates pre-defined failure scenarios, such as power outages (e.g., a power failure in a computer room) and spare parts delays (e.g., a 48-hour delay in capacitor delivery). It monitors the load transfer of devices in the deployment plan (e.g., the time it takes to migrate business load to high-efficiency devices after an inefficient device fails), the activation of backup devices (e.g., the startup time of a backup projector), and business continuity indicators (e.g., the duration of meeting recovery after interruption). Based on robustness metrics collected during the simulation (e.g., load transfer success rate and backup device availability), the risk analysis module generates risk response recommendations, including increasing the number of backup devices in key areas, adjusting spare parts procurement cycles, or optimizing the load distribution ratio of high-efficiency devices. These recommendations are then pushed to the decision-making module in the operations and maintenance center via WebSocket.

[0068] Each service in the microservice-based simulation and verification system is orchestrated through Kubernetes: the data simulation service, policy verification service, and risk inference service are each packaged as a separate Docker image. These images include the required libraries (such as the Python Pandas data processing library and the TensorFlow simulation framework) and configuration files (such as exception injection rules and interface addresses). The Kubernetes cluster manages the number of service replicas through the Deployment controller (for example, deploying three replicas of the data simulation service to increase concurrent processing capabilities). The Service component provides load balancing (for example, gRPC requests for the policy verification service are distributed to different replicas through the Service). The Horizontal Pod Autoscaler automatically scales the system based on CPU utilization (for example, automatically increasing the number of risk inference service replicas during peak simulation workloads). Communication between services is achieved through Kubernetes' DNS service discovery mechanism (for example, the data simulation service accesses the policy verification service through the service name "data-simulation"), ensuring high availability and scalability.

[0069] The above technical steps form a closed-loop verification logic: the data simulation service generates test inputs, the policy verification service verifies the effectiveness of maintenance strategies, the risk simulation service assesses the robustness of deployment plans, and Docker containerization and Kubernetes orchestration ensure stable system operation. This complete microservices-based verification process provides multi-dimensional simulation verification support for operations and maintenance decisions, improving the reliability of maintenance strategies and resource scheduling solutions.

[0070] Specifically, the AI-based multimedia device operation and maintenance management system of the present invention further includes a model dynamic fine-tuning unit; The model dynamic fine-tuning unit uses incremental learning technology to dynamically adjust the graph attention network parameters and ARIMA-LSTM fusion model weights based on the simulation error data output by the microservice simulation verification system. The simulation error data includes the fault root cause prediction deviation and baseline threshold matching error. When the mean square error between the simulation prediction value and the actual operating data exceeds the preset threshold, the model fine-tuning process is triggered. The simulation annotation data is introduced through transfer learning to update the model parameters, and the fault mode association rules in the three-level fault knowledge graph are simultaneously updated to improve the accuracy of the fault root cause analysis results and the scenario adaptability of the dynamic baseline model.

[0071] The specific implementation of the model dynamic fine-tuning unit of the present invention includes the following technical steps and logical associations: Error data from the model's dynamic fine-tuning unit is obtained based on the output of the microservice-based simulation verification system. The simulation error data output by the policy verification service includes the root cause prediction deviation (i.e., the inconsistency between the root cause label predicted by the graph attention network and the root cause label confirmed by actual maintenance) and the baseline threshold matching error (i.e., the matching error log between the dynamic parameter threshold generated by the ARIMA-LSTM fusion model and the actual operating parameters of the device) output by the risk deduction service. This error data is categorized by device type and anomaly type and stored in a cloud-based error database. This creates a structured error dataset consisting of device ID, predicted root cause, actual root cause, threshold deviation, and timestamp, providing a data foundation for subsequent model adjustments.

[0072] During the incremental learning adjustment phase, the model's dynamic fine-tuning unit first loads the current parameters of the graph attention network in the fault prediction module (such as the attention weight matrix and fully connected layer weights) and the current weights of the ARIMA-LSTM fusion model in the device state modeling module (such as the ARIMA autoregressive coefficients and the LSTM gating unit parameters). Based on the root cause prediction deviations in the error dataset, the model extracts feature dimensions related to the deviations (such as abnormal frequency bands in voiceprint features and dust accumulation areas on heat sinks in video features). Using the incremental learning algorithm, the model updates only the attention weight layers of the corresponding features in the graph attention network, retaining the parameters of other unaffected layers to maintain the model's original generalization capability. Based on the baseline threshold matching error, the model extracts time series features related to the deviations (such as periodic fluctuations in temperature parameters and nonlinear growth trends in power consumption). The weights of the ARIMA autoregressive terms or LSTM memory units in the ARIMA-LSTM fusion model responsible for predicting these time series features are adjusted to prevent the model's ability to fit historical stable data from being compromised due to global parameter updates.

[0073] The model fine-tuning process is triggered based on a quantitative evaluation of error data. The model's dynamic fine-tuning unit periodically (e.g., at midnight daily) extracts the last 24 hours of simulated predictions and actual operating data from the cloud-based error database and calculates the mean squared error (MSE) between the two (reflecting the overall degree of deviation between the predicted and actual results). When the MSE exceeds a preset threshold (e.g., 5%), the fine-tuning process is triggered; if it does not, error data continues to accumulate. For example, if the MSE of the graph attention network's prediction of the root cause of "capacitor aging" reaches 7% (exceeding the 5% threshold), or if the MSE of the ARIMA-LSTM model's matching of the temperature threshold reaches 6% (exceeding the 5% threshold), the system automatically initiates the transfer learning update process.

[0074] During the transfer learning update phase, the model dynamic fine-tuning unit uses a simulated annotated dataset generated by the microservice-based simulation verification system (including manually annotated root cause labels and baseline threshold correction suggestions) as the source domain data for transfer learning, and performs feature alignment with the actual operating data (target domain data). For the graph attention network, parameter transfer is used to transfer the attention weight pattern corresponding to the "capacitor aging" root cause in the source domain data to the target domain model, adjusting the classification threshold of the fully connected layer. For the ARIMA-LSTM fusion model, feature transfer is used to transfer the timing pattern of "temperature threshold correction" in the source domain data to the target domain model, optimizing the activation function parameters of the LSTM gating unit. After transfer learning is complete, the model dynamic fine-tuning unit outputs the updated graph attention network parameters and ARIMA-LSTM fusion model weights, replacing the original model parameters to complete the model iteration.

[0075] Synchronous updates to the knowledge graph and model fine-tuning form a closed-loop feedback loop. After completing the model parameter update, the model dynamic fine-tuning unit extracts the newly added fault root cause association patterns in the graph attention network (for example, the association probability of "temperature anomaly + abnormal voiceprint" and "capacitor aging" increases from 60% to 85%), as well as the revised baseline threshold rules in the ARIMA-LSTM fusion model (for example, the projector temperature threshold is adjusted from 45°C±3°C to 45°C±2.5°C). The association rules in the three-level fault knowledge graph are updated through the knowledge graph management interface. Specifically, the association features of "temperature anomaly + abnormal voiceprint" are added to the third-level node "capacitor aging" and the confidence weight of this node is adjusted; the fluctuation range of the projector temperature threshold is updated in the dynamic baseline model association rules, ensuring that the fault patterns in the knowledge graph are synchronized with the model parameters, providing more accurate rule support for knowledge retrieval in the subsequent fault prediction module.

[0076] The above technical steps are closely linked: error data provides the basis for model adjustment, incremental learning achieves local parameter optimization, error thresholds trigger global transfer learning, transfer learning updates model parameters, and knowledge graph synchronization ensures consistency between rules and models. This complete dynamic fine-tuning process continuously improves the accuracy of root cause analysis results and the adaptability of the dynamic baseline model to new scenarios (such as equipment aging and environmental changes).

[0077] The following is an explanation of the algorithms, models and strategies in the present invention: The data acquisition module uses a frame differencing algorithm to process video monitoring data. This algorithm calculates pixel differences frame by frame within a continuous video stream to identify areas of sudden change in the device's appearance. A Mel-frequency cepstral coefficient algorithm processes audio signals, extracting voiceprint features that characterize the device's mechanical state through frame windowing, Fourier transforms, and Mel filter bank processing. The data encoding process utilizes a structured encapsulation strategy to integrate heterogeneous data from multiple sources into a unified data stream based on device identification, enabling standardized transmission of multimodal data.

[0078] The analysis module implements a voiceprint feature comparison algorithm and an image recognition algorithm. The voiceprint feature comparison algorithm uses pattern matching technology to compare real-time audio features with a reference template library and output an audio quality score. The image recognition algorithm, based on a convolutional neural network structure, detects areas of color distortion and signal delay frames in the video output and generates video performance metrics. The quality evaluation strategy uses a weighted fusion method to combine audio scores and video metrics to form a comprehensive evaluation of device performance.

[0079] The device state modeling module builds an ARIMA-LSTM fusion model. The ARIMA time series model analyzes linear trends and cyclical characteristics in device parameters. The LSTM neural network captures nonlinear dependencies between parameters. The model fusion strategy uses a weighted output mechanism to combine the prediction results of the two models to generate dynamic parameter fluctuation thresholds. The anomaly detection strategy generates anomaly signals with confidence scores by comparing real-time data against thresholds dimension by dimension.

[0080] The fault prediction module applies a three-level fault knowledge graph retrieval strategy. The knowledge graph uses a hierarchical structure to organize the relationships between device types, fault modes, and root cause features. A graph attention network performs multimodal reasoning. Nodes in the network represent features from different data sources, and edge weights dynamically calculate inter-feature correlations using an attention mechanism. A root cause analysis strategy aggregates node features to output a fault probability distribution, enabling collaborative diagnosis of multi-source data.

[0081] The maintenance strategy optimization module implements a health scoring strategy. The scoring algorithm integrates three parameters: failure probability, predicted remaining equipment life, and business impact factors, quantifying the equipment's health status through a weighted calculation. The maintenance priority queue generation strategy prioritizes equipment maintenance in descending order based on the score. The digital twin simulation strategy loads the equipment topology model into a virtual environment, simulates the impact of maintenance operations on business continuity, and outputs optimized work order solutions.

[0082] The resource allocation module uses a K-means clustering algorithm to generate device performance labels. The algorithm constructs a two-dimensional feature space based on health scores and historical failure frequencies, and uses unsupervised learning to categorize device performance. The load balancing strategy models the device deployment problem as a multi-objective optimization task, using a genetic algorithm to iteratively solve the optimal resource scheduling solution through selection, crossover, and mutation operations.

[0083] The visualization platform implements a dynamic 3D topology rendering strategy. The rendering engine color-codes device models based on health score thresholds and overlays sensor parameter curves in real time. The AR-assisted diagnosis strategy uses mobile devices to identify physical devices and overlays a 3D model of the faulty component and maintenance operation guide animations on the video screen. The early warning linkage strategy establishes a hierarchical alarm mechanism, intelligently linking early warning information with root cause analysis pages and resource scheduling plans.

[0084] The model's dynamic fine-tuning unit implements an incremental learning strategy. This strategy updates local parameters in the feature layers of the graph attention network that are strongly correlated with prediction bias, preserving the network's original generalization capabilities. A transfer learning strategy incorporates annotated data generated by the simulation verification system and transfers optimized feature association patterns to the target model. The knowledge graph collaborative update strategy synchronously modifies fault mode association rules to maintain the consistency of the system's knowledge architecture.

[0085] The microservice-based simulation and verification system utilizes a containerized deployment strategy. The system encapsulates data simulation, policy verification, and risk assessment functions as independent microservices, orchestrating them through Kubernetes. An exception injection strategy embeds pre-set failure modes into historical data to create multi-dimensional test scenarios. A robustness verification strategy simulates sudden failure events to assess the fault tolerance of resource scheduling solutions.

[0086] These algorithms, models, and strategies form a collaborative mechanism through distributed messaging middleware. The output of the data acquisition module triggers baseline updates in the state modeling module. Abnormal signals drive knowledge retrieval in the fault prediction module. Root cause analysis results activate the maintenance strategy optimization process. Resource scheduling plan updates are linked to visualization platform rendering. This closed-loop management mechanism enables autonomous decision-making throughout the entire process, from condition monitoring and fault diagnosis to resource optimization.

[0087] When the AI-based multimedia equipment operation and maintenance management system provided by the present invention is specifically implemented in education, business and conference scenarios, temperature sensors, vibration sensors and high-definition cameras are deployed at key parts of multimedia equipment through the data acquisition module. The temperature sensor monitors the operating temperature of the projector bulb and the cooling fan in real time, the vibration sensor captures the mechanical vibration frequency of the audio equipment, and the camera analyzes changes in the appearance of the equipment through the frame difference method, such as screen color spots or the degree of dust accumulation in the heat dissipation port. The audio analysis unit uses a high-sensitivity microphone array to collect equipment operating noise, extracts voiceprint features through Mel-frequency cepstral coefficients, and identifies abnormal fan noise or mechanical wear. The collected multimodal data is uniformly encapsulated into a structured data stream in JSON format by the data encoding unit, and transmitted to the cloud data lake storage by topic through the MQTT protocol for subsequent module calls.

[0088] The device status modeling module uses historical data stored in the cloud to retrain a dynamic baseline model combining ARIMA and LSTM every 24 hours through a sliding window mechanism. The ARIMA model captures linear trends in device temperature and power consumption, while the LSTM network learns nonlinear fluctuations in vibration data. The two are weighted to output dynamic parameter fluctuation thresholds. For example, the baseline for a projector within the rated temperature range is 45°C ± 3°C; an anomaly signal is triggered when the real-time temperature exceeds 48°C. The anomaly correlation analysis unit retrieves temperature data from other projectors in the same room and calculates the anomaly propagation probability using the Pearson correlation coefficient. If the correlation coefficient exceeds 0.7, it is determined to be a systemic heat dissipation risk; otherwise, it is marked as a single device failure. The anomaly signal is labeled with the device ID, anomaly type, and confidence level before being pushed to the fault prediction module.

[0089] The fault prediction module's knowledge graph query unit retrieves associated fault patterns from a pre-built three-level fault knowledge graph based on device type (e.g., audio) and anomaly type (e.g., audio distortion). For example, the second-level node "audio distortion" is associated with the third-level nodes "capacitor aging" and "firmware version conflict," matching solutions from historical repair cases. The multimodal reasoning unit inputs real-time audio features, temperature data, and equipment room humidity parameters into a graph attention network. The network nodes represent feature vectors from different data sources, and the correlation between audio distortion and capacitor temperature is calculated using attention weights. The resulting probability distribution of the root cause of the fault is then output. When the probability of capacitor aging exceeds 85%, a fault diagnosis result is generated, triggering the maintenance strategy optimization module.

[0090] The health scoring unit in the maintenance strategy optimization module calculates a health score (e.g., 62) based on the failure probability (85%), the predicted remaining lifespan (6 months), and the business priority weight (e.g., conference room equipment has a weight of 0.8). Devices with a score below 50 are placed in a high-priority maintenance queue. The digital twin simulation unit loads the device deployment topology, simulates the impact of replacing a capacitor on the meeting schedule, calculates downtime (e.g., 2 hours) and spare parts consumption, and selects the solution with the least impact on business continuity. The simulation results generate a work order and push it to the operations and maintenance personnel's terminal, simultaneously updating the spare parts inventory data in the resource allocation module.

[0091] The resource allocation module's efficiency clustering unit uses the K-means algorithm to categorize devices into three categories: high-efficiency, medium-efficiency, and low-efficiency, based on their health scores and historical failure frequencies. For example, devices with a score above 70 and fewer than three failures are marked as high-efficiency and prioritized for deployment in core areas. The load balancing unit, combined with business demand forecast data (such as the peak meeting period next week), generates a device migration plan using a genetic algorithm, replacing inefficient devices with backup devices. The deployment topology is updated and dynamically rendered through the visualization platform. The visualization platform's 3D topology displays device efficiency labels using a color gradient, with red indicating inefficient devices that need replacement and green indicating efficient devices that can maintain operation.

[0092] The AR-assisted diagnosis unit of the visualization platform scans the faulty equipment through a mobile terminal, superimposes a virtual structural diagram, highlights the location of the faulty capacitor, and displays the selection of disassembly tools and welding operation specifications in steps. When the operation and maintenance personnel perform the operation, the AR interface compares the action with the standard process in real time, and pops up a correction prompt when a deviation is found. The early warning linkage unit displays a red warning (immediate processing) or a yellow warning (observation) based on the probability of failure. Clicking the warning icon jumps to the root cause analysis page, which displays the maintenance records and resource scheduling suggestions for similar faults, such as the recommendation to use a replacement capacitor with model C-2035. Each module synchronizes data through a distributed message middleware. For example, the update of the work order status triggers the re-rendering of the topology map, forming a closed-loop management from data collection to maintenance execution, significantly improving operation and maintenance efficiency and equipment reliability.

[0093] The technical solution of the present invention solves the problem of insufficient real-time monitoring of manual inspections through real-time multimodal data acquisition and intelligent analysis mechanisms. The data acquisition module deploys a non-contact video monitoring unit and a high-sensitivity audio analysis unit to continuously capture changes in the appearance of the equipment and the characteristics of its operating voiceprint. Combined with temperature and vibration sensor data, a multi-dimensional equipment status data stream is formed through structured coding. The equipment status modeling module trains a dynamic baseline model based on historical data, and generates abnormal signals by comparing current data with dynamic thresholds in real time. The visualization platform dynamically superimposes the equipment health status, abnormal signals and maintenance instructions on the physical equipment through 3D topological rendering and AR-assisted diagnosis technology, achieving second-level perception and visual monitoring of the equipment status, eliminating the monitoring blind spots and time delays of manual inspections.

[0094] The fault prediction and maintenance optimization modules work together to solve the problems of delayed fault response and high maintenance costs. The fault prediction module inputs abnormal signals into a pre-built three-level fault knowledge graph, fuses multimodal data features through a graph attention network, and outputs the probability distribution of the root cause of the fault and the associated maintenance plan. The maintenance strategy optimization module calculates the health score based on the failure probability, the predicted value of the remaining life of the equipment, and the business weight, and generates a maintenance priority queue. The digital twin simulation unit simulates the impact of maintenance operations on business continuity in a virtual environment, optimizing the execution order of work orders and resource allocation. This technology chain achieves minute-level location of the root cause of the fault and automatic generation of maintenance plans, significantly shortening the diagnostic response time and reducing business losses caused by equipment downtime through preventive maintenance.

[0095] The resource allocation module builds a data-driven closed-loop optimization mechanism to solve the problem of low resource utilization. The performance clustering unit generates equipment performance labels using the K-means algorithm based on health scores and historical failure frequencies. The load balancing unit combines business demand forecast data with a genetic algorithm to calculate the optimal equipment deployment plan and dynamically adjust the resource topology. The visualization platform displays resource heat maps and equipment performance distribution in real time, allowing operation and maintenance personnel to predict resource gaps based on business peaks. Each module forms a closed loop through data flow linkage: the execution results of maintenance work orders are fed back to the equipment status modeling module to update the baseline model, and the resource scheduling plan verifies the data-driven knowledge graph rule iteration. The data-driven resource optimization mechanism improves equipment utilization efficiency and achieves a balance between operation and maintenance costs and business continuity.

Claims

1. An AI-based multimedia equipment operation and maintenance management system, characterized in that: include: Data acquisition module, analysis module, equipment status modeling module, fault prediction module, maintenance strategy optimization module, resource allocation module, operation and maintenance center decision module and visualization platform; The data acquisition module is used to collect the operating data of multimedia devices through sensors and video equipment, and pre-process the data before transmitting it to the cloud. The data acquisition module includes a non-contact video monitoring unit and an audio analysis unit. The non-contact video monitoring unit is used to capture changes in the device's appearance and extract key frame features through frame differencing. The audio analysis unit is used to extract voiceprint features from the device's operating noise through Mel-frequency cepstral coefficients. The analysis module analyzes the performance quality of multimedia devices in real time based on the audio and video data obtained by audio and video sensors, obtains quality evaluation results, and feeds the quality evaluation results back to the decision module of the operation and maintenance center; The equipment status modeling module is used to build a dynamic baseline model based on historical data, compare the collected data with the baseline model in real time to generate abnormal signals, and send the abnormal signals to the fault prediction module; The fault prediction module is used to generate fault diagnosis results based on the fault modes in the knowledge graph based on abnormal signals, fuse multimodal data, and trigger the maintenance strategy optimization module; The maintenance strategy optimization module is used to calculate the equipment health score based on the fault diagnosis results, generate the maintenance priority queue, and output the work order after verifying the maintenance plan through digital twin simulation; The resource allocation module is used to generate resource scheduling plans based on device performance tags and business demand forecasts, and update the device deployment topology map; The visualization platform is used to display equipment health status, maintenance work order progress, and resource heat maps in real time, and supports AR-assisted diagnosis and cross-module data linkage.

2. The AI-based multimedia equipment operation and maintenance management system according to claim 1, characterized in that: The data acquisition module also includes: The data encoding unit is used to uniformly encode key frame features, voiceprint features, and temperature and power consumption data collected by sensors into a structured data stream, and transmit it to the device state modeling module for baseline model comparison.

3. The AI-based multimedia equipment operation and maintenance management system according to claim 2, characterized in that: The equipment status modeling module includes: Dynamic baseline update unit, used to retrain the ARIMA and LSTM fusion model on historical data based on the sliding window mechanism to generate dynamic parameter fluctuation thresholds; The anomaly correlation analysis unit is used to retrieve the operating data of equipment in the same area associated with the abnormal signal from the cloud data lake, calculate the probability of anomaly propagation through the Pearson correlation coefficient, and distinguish between single equipment failures and systemic risks.

4. The AI-based multimedia equipment operation and maintenance management system according to claim 3, characterized in that: The fault prediction module includes: A knowledge graph query unit is used to retrieve related failure modes and historical maintenance cases from a pre-built three-level failure knowledge graph based on the equipment type and failure type included in the abnormal signal; The multimodal reasoning unit is used to fuse audio data and video data, video key frame features, and equipment deployment environment parameters through a graph attention network to generate fault root cause analysis results and trigger the maintenance strategy optimization module.

5. The AI-based multimedia equipment operation and maintenance management system according to claim 4, characterized in that: The maintenance strategy optimization module includes: A health scoring unit is used to calculate the equipment health score based on the failure probability in the fault root cause analysis results, the output value of the equipment remaining life prediction model and the preset service priority weight; The digital twin simulation unit is used to generate maintenance priority queues based on equipment health scores, load equipment deployment topology maps in a virtual environment, and simulate the impact of maintenance operations on business continuity to optimize scheduling plans.

6. The AI-based multimedia equipment operation and maintenance management system according to claim 5, characterized in that: The resource allocation module includes: The performance clustering unit is used to generate device performance labels based on device health scores and historical failure frequencies using the K-means clustering algorithm; The load balancing unit is used to calculate the optimal device deployment plan through genetic algorithms based on device performance tags and business demand predictions, and update the resource topology map to the visualization platform for dynamic rendering.

7. The AI-based multimedia device operation and maintenance management system according to claim 6, characterized in that: The visualization platform includes: 3D topology rendering unit, used to dynamically colorize device models based on device health scores and overlay real-time parameter curves of audio and video data; AR-assisted diagnosis unit, used to scan the device through a mobile terminal and overlay a virtual structure diagram corresponding to the fault root cause analysis results, marking the faulty components and maintenance instructions; The early warning linkage unit is used to display early warning information according to the fault probability level, and link the jump to the root cause result page and the resource scheduling solution recommendation page.

8. The AI-based multimedia equipment operation and maintenance management system according to claim 7, characterized in that: Also includes: The data encoding unit integrates a Transformer-based multimodal feature adaptive fusion optimization algorithm; The multimodal feature adaptive fusion optimization algorithm aligns video key frame features, voiceprint features, and the temporal dimensions of audio and video data through a self-attention mechanism, dynamically calculates the fusion weights of each modal feature, suppresses redundant feature interference, and enhances the key feature representation capability, generating a high-dimensional, low-noise structured data stream to improve the accuracy of baseline model comparison in the device state modeling module.

9. The AI-based multimedia equipment operation and maintenance management system according to claim 8, characterized in that: It also includes a microservice simulation and verification system; The microservice simulation and verification system consists of data simulation services, policy verification services, and risk deduction services. It is deployed in Docker containers and orchestrated through Kubernetes. The data simulation service generates simulated equipment operation scenarios including anomaly injection based on historical operation data and outputs multimodal simulation data sets; The policy verification service uses the virtual topology of the digital twin simulation unit to simulate maintenance operations for each device in the maintenance priority queue, and calculates the duration of service interruption, predicted spare parts consumption, and the impact on related devices. The risk simulation service calls the efficiency label data of the resource allocation module through the microservice interface, simulates the robustness of the equipment deployment plan under sudden failures, and generates risk response recommendations.

10. The AI-based multimedia equipment operation and maintenance management system according to claim 9, characterized in that: Also included is a model dynamics fine-tuning unit; The model dynamic fine-tuning unit uses incremental learning technology to dynamically adjust the graph attention network parameters and ARIMA-LSTM fusion model weights based on the simulation error data output by the microservice simulation verification system. The simulation error data includes the fault root cause prediction deviation and baseline threshold matching error. When the mean square error between the simulation prediction value and the actual operating data exceeds the preset threshold, the model fine-tuning process is triggered. The simulation annotation data is introduced through transfer learning to update the model parameters, and the fault mode association rules in the three-level fault knowledge graph are simultaneously updated to improve the accuracy of the fault root cause analysis results and the scenario adaptability of the dynamic baseline model.

Images