A full-link-aware conference device failure prediction method and system

Abstract

This invention discloses a method and system for predicting faults in conferencing equipment with end-to-end awareness. The method includes: constructing a device dependency graph, comprising a main graph and at least two subgraphs; real-time acquisition of multi-dimensional operational status time-series data of each device node, and extraction of degradation feature sets; fusion embedding encoding of each device node using a link fault coding model to obtain a fusion embedding vector, the model being constructed based on a graph attention network and a causal temporal convolutional network; and, based on the fusion embedding vector, prediction of the fault probability and remaining lifespan of each device node using a multi-head task predictor, and determination of the warning level. This invention solves the technical problem of existing technologies being unable to accurately predict cross-link cascading faults based on quantitatively modeling the coupling dependencies between devices by using hierarchical spatial reasoning of the device dependency graph and adaptive fusion of multi-modal temporal degradation features.

Description

Technical Field

[0001] This invention relates to the field of intelligent conference equipment operation and maintenance management technology, specifically to a conference equipment fault prediction method and system with end-to-end perception. Background Technology

[0002] Intelligent conference room systems are widely used in modern enterprises. Typical deployments include microphone arrays, speaker systems, projection display equipment, network switching equipment, audio and video codecs, and environmental control systems. These devices are cascaded together according to signal processing links to support the normal operation of the meeting. The operational reliability of the conference equipment directly affects the enterprise's collaboration efficiency and meeting experience.

[0003] Currently, the operation and maintenance management of conference equipment mainly adopts the following methods: First, independent monitoring of a single equipment type, such as monitoring only the port traffic of network equipment or focusing only on the basic parameters of audio equipment; second, triggering an alarm when a certain operating indicator exceeds a preset threshold, such as issuing a notification when the packet loss rate exceeds 5%; and third, manual inspection and equipment checks are carried out by operation and maintenance personnel according to fixed cycles.

[0004] Existing methods for the operation and maintenance of conference equipment have the following shortcomings: monitoring of individual devices is isolated, failing to analyze upstream and downstream devices belonging to the same signal link as a whole, making it difficult to detect the downstream cascading effects caused by the performance degradation of upstream devices; alarm mechanisms typically only trigger after indicators exceed fixed thresholds, failing to capture the gradual degradation trend of equipment performance; manual inspection methods are inefficient, have limited coverage, and cannot identify sudden failures within inspection intervals. Therefore, how to achieve accurate prediction of cross-link cascading failures based on quantitative modeling of coupling dependencies between devices is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] This invention provides a method for predicting faults in conferencing equipment with end-to-end awareness, addressing the technical problem in existing technologies where equipment monitoring is isolated and cannot achieve accurate prediction of cross-link cascading faults based on quantitative modeling of inter-device coupling dependencies. In view of the above problems, this invention also provides a system for predicting faults in conferencing equipment with end-to-end awareness.

[0006] In a first aspect, the present invention provides a method for predicting faults in conferencing equipment with end-to-end awareness, the method comprising: Obtain the physical connection relationships and data flow dependencies between device nodes in the conference system, and construct a device dependency graph accordingly, wherein the device dependency graph includes a main graph and at least two subgraphs; Real-time acquisition of multi-dimensional operational status time-series data of each device node, and extraction of degradation feature set; Based on the device dependency graph and the degradation feature set, and combined with the pre-built link fault coding model, each device node is fused and embedded to obtain the fused embedding vector of each device node. The link fault coding model is constructed based on graph attention network and causal temporal convolutional network. Based on the fused embedding vector, the failure probability and remaining service life of each device node are predicted by the corresponding multi-head task predictor, and the warning level of each device node is determined accordingly.

[0007] Secondly, the present invention also provides a full-link awareness system for predicting faults in conference equipment, the system comprising: The graph construction module is used to obtain the physical connection relationships and data flow dependencies between device nodes in the conference system, and construct a device dependency graph accordingly. The device dependency graph includes a main graph and at least two subgraphs. The perception and feature extraction module is used to collect multi-dimensional operating status time-series data of each device node in real time and extract degradation feature sets. The fusion embedding coding module is used to perform fusion embedding coding on each device node based on the device dependency graph and the degradation feature set, combined with a pre-built link fault coding model, to obtain the fusion embedding vector of each device node. The link fault coding model is constructed based on a graph attention network and a causal temporal convolutional network. The prediction and early warning module is used to predict the failure probability and remaining service life of each device node based on the fused embedding vector and through the corresponding multi-head task predictor, and to determine the early warning level of each device node accordingly.

[0008] One or more technical solutions provided in this invention have at least the following technical effects or advantages: First, this invention formally models the physical connections and data flow dependencies between conferencing devices by constructing a device dependency graph, dividing it into a main graph and subgraphs connected by cascading interface edges. Based on a graph attention network, hierarchical spatial reasoning is performed. After aggregating upstream and downstream device state information within a subgraph, it is then fed up to the main graph via cascading edges to complete cross-modal global aggregation, and then distributed down to each subgraph node. This ensures that the spatial embedding vector of each device node simultaneously contains its own state, upstream and downstream influences within the link, and cross-modal global dependency information. Combining temporal degradation features extracted by a causal temporal convolutional network, a cross-attention fusion mechanism adaptively integrates spatial and temporal information, solving the technical problem of existing technologies where device monitoring is isolated and accurate prediction of cross-link cascading faults cannot be achieved based on quantified modeling of coupling dependencies between devices.

[0009] Second, this invention uses a multi-task predictor to simultaneously output the failure probability and remaining service life of device nodes, and combines the centrality index of the device in the dependency graph and the current usage status to comprehensively determine the warning level, so that operation and maintenance decisions can comprehensively consider the urgency and scope of the failure, thereby improving the operability and accuracy of the warning results.

[0010] In summary, this invention solves the technical problem in the prior art where device monitoring is isolated from each other and cannot achieve accurate prediction of cross-link cascading faults based on quantitative modeling of coupling dependencies between devices by using hierarchical spatial reasoning of device dependency graphs and adaptive fusion of multimodal temporal degradation features. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a method for predicting faults in conferencing equipment with end-to-end awareness, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a conference equipment fault prediction system with end-to-end sensing provided in an embodiment of the present invention; The diagram is labeled as follows: Graph construction module 11, perception and feature extraction module 12, fusion embedding and encoding module 13, prediction and early warning module 14. Detailed Implementation

[0012] This invention provides a method and system for predicting faults in conference equipment with end-to-end awareness, which solves the technical problem in the prior art where equipment monitoring is isolated from each other and it is impossible to achieve accurate prediction of cross-link cascading faults based on quantitative modeling of the coupling dependency relationship between equipment.

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0014] It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0015] Example 1 like Figure 1 As shown, this invention provides a method for predicting faults in conferencing equipment with end-to-end awareness, the method comprising: S100: Obtain the physical connection relationship and data flow dependency relationship between each device node in the conference system, and construct a device dependency graph accordingly, wherein the device dependency graph includes a main graph and at least two subgraphs; In intelligent conferencing systems, the dependencies between devices exhibit a natural hierarchical structure. Common equipment such as network switches and UPS power supplies simultaneously carry multiple audio and video data streams, shared by all modalities; while devices like microphones, audio encoders, cameras, video decoders, and projectors belong to dedicated modalities such as audio links and video links, with close signal transmission relationships between devices within each link. Existing conferencing equipment monitoring methods typically treat each device as an independent entity, failing to structurally represent this hierarchical device relationship. When upstream equipment experiences performance degradation, downstream devices, although operating normally themselves, may exhibit fault symptoms due to a cascading decline in signal quality. Isolated monitoring methods struggle to identify the source of such cascading failures.

[0016] Step S100 in the method provided in this embodiment of the invention includes: obtaining the physical connection relationship and data flow dependency relationship between each device node in the conference system, and constructing a device dependency graph accordingly, including: Based on the functional affiliation of each device node in the conference system, the device nodes are divided into common device nodes shared by multiple modes and modal device nodes dedicated to each modal link. Using multiple common device nodes as graph nodes and the dependencies between the multiple common device nodes as relation edges, the main graph in the device dependency graph is constructed. For each modal link, a subgraph is constructed using the corresponding multiple modal device nodes as graph nodes and the signal flow dependencies between the multiple modal device nodes as relation edges. This process is repeated to obtain at least two subgraphs. Identify the boundary nodes of the data flow entering and exiting the main graph in multiple subgraphs, and establish cascaded interface edges between the multiple boundary nodes and the corresponding common device nodes in the main graph to obtain the device dependency graph.

[0017] The specific implementation method is as follows: First, based on the functional affiliation of each device node in the conference system, the device nodes are divided into two categories.

[0018] The first category consists of common equipment nodes shared by multiple modalities. These devices form the infrastructure upon which all modal links in the conference room depend, and do not belong to any specific modality. Common equipment nodes include: network switching equipment, through which all audio and video streams are aggregated and distributed; power supply equipment, which provides power to all devices; environmental control equipment, whose output environmental parameters such as temperature and humidity affect all devices; and central control equipment, which coordinates the operating status of all devices.

[0019] The second category consists of modal device nodes dedicated to each modal link. These devices serve only a specific modality, forming a chain-like dependency relationship according to the signal flow direction. For example, the modal device nodes in an audio modal link include a microphone, audio processor, audio encoder, audio decoder, power amplifier, and speaker. The signal flow starts with microphone acquisition, sequentially undergoes noise reduction and echo cancellation by the audio processor, compression encoding by the audio encoder before entering the network for transmission, decoding by the audio decoder, amplification by the power amplifier, and output by the speaker. The modal device nodes in a video modal link include a camera, video processor, video encoder, video decoder, and projection display device. The signal flow starts with camera capture, sequentially undergoes noise reduction and enhancement by the video processor, compression encoding by the video encoder before entering the network for transmission, decoding by the video decoder, and presentation by the projection display device. The modal device nodes in a control modal link include a touch panel, central control processor, and wireless projection receiver. The device nodes within each of these modal links form upstream and downstream dependencies according to the signal flow transmission order.

[0020] Furthermore, using multiple common device nodes as graph nodes and the dependencies between multiple common device nodes as relation edges, a main graph is constructed in the device dependency graph.

[0021] The main map nodes include network switching equipment nodes, power supply equipment nodes, environmental control equipment nodes, and central control equipment nodes. Each main map node carries its own attribute information, including equipment type code and rated parameters.

[0022] The main graph relationship edges describe the dependencies between common equipment nodes, including: power supply dependency edges from power supply equipment nodes to network switching equipment nodes, indicating that the network switching equipment depends on the power supply equipment for power; power supply dependency edges from power supply equipment nodes to central control equipment nodes; environmental impact edges from environmental control equipment nodes to network switching equipment nodes, indicating that the heat dissipation of the network switching equipment is affected by ambient temperature and dust; and environmental impact edges from environmental control equipment nodes to power supply equipment nodes, indicating that the operating temperature of the power supply equipment is affected by the environment.

[0023] Furthermore, for each modal link, a subgraph is constructed using the corresponding multiple modal device nodes as graph nodes and the signal flow dependencies between the multiple modal device nodes as relation edges. This process is repeated to obtain at least two subgraphs.

[0024] Taking an audio modal link as an example, the audio link consists of a microphone array node, an audio encoder node, a network switch node, an audio decoder node, and a speaker node. The signal flow relationships within the audio subgraph are as follows: the microphone array node points to the audio encoder node, the audio encoder node points to the network switch node, the network switch node points to the audio decoder node, and the audio decoder node points to the speaker node. Specifically, the audio encoder node represents the boundary where the data flow leaves the upper part of the subgraph, and the audio decoder node represents the boundary where the data flow enters the lower part of the subgraph from the outside.

[0025] Taking a video modal link as an example, a video link consists of camera nodes, video encoder nodes, network switch nodes, video decoder nodes, and projection display device nodes. The signal flow relationships within the video subgraph are as follows: camera node points to video encoder node, video encoder node points to network switch node, network switch node points to video decoder node, and video decoder node points to projection display device node. Here, the video encoder node represents the upstream boundary of the data flow, and the video decoder node represents the downstream boundary of the data flow.

[0026] It is worth noting that both the audio and video links contain network switch nodes, and these nodes are the same network switch. This network switch carries both audio and video streams and is a key shared node connecting the audio and video subgraphs. When establishing cascade interface edges, this network switch will be used as a common device node in the main graph. The audio encoder node and video encoder node will establish cascade interface edges with it as the uplink boundary nodes of each subgraph, and the audio decoder node and video decoder node will establish cascade interface edges with it as the downlink boundary nodes of each subgraph.

[0027] The control subgraph includes a touch panel node, a central control processor node, and a wireless projection receiver node. The signal flow relationships within the control subgraph are as follows: the touch panel node points to the central control processor node.

[0028] The signal flow relationship edges within each subgraph have a clear directionality, reflecting the upstream and downstream relationship of data transmission between devices. This directionality is used to determine the direction of fault propagation in subsequent spatial reasoning.

[0029] Finally, the boundary nodes where data flows into and out of the main graph in multiple subgraphs are identified, and cascaded interface edges between multiple boundary nodes and the corresponding common device nodes in the main graph are established to obtain a complete device dependency graph.

[0030] For the audio subgraph, its uplink boundary node is the audio encoder node. The data stream leaves the audio subgraph through this node and enters the network for transmission. An uplink cascaded interface edge is established between the audio encoder node and the network switch node in the main graph. Its downlink boundary node is the audio decoder node. The data stream enters the audio subgraph from the network for transmission. A downlink cascaded interface edge is established between the network switch node in the main graph and the audio decoder node.

[0031] For the video subgraph, its uplink boundary node is the video encoder node, and an uplink cascade interface edge is established from the video encoder node to the network switch node in the main graph; its downlink boundary node is the video decoder node, and a downlink cascade interface edge is established from the network switch node in the main graph to the video decoder node.

[0032] For the control subgraph, there is bidirectional control signaling interaction between the central control processor node and the network switch node in the main graph, and a bidirectional cascading interface edge is established between the central control processor node and the network switch node; the wireless projection receiver node transmits projection data stream to the network switch node, and an uplink cascading interface edge is established between the wireless projection receiver node and the network switch node.

[0033] In addition, the environmental control device node in the main graph has a cross-level influence on the device nodes in each subgraph that are significantly affected by environmental factors. An environmental influence edge is established from the environmental control device node to the projection display device node, and an environmental influence edge is established from the environmental control device node to the microphone array node.

[0034] Through the establishment of the aforementioned cascading interface edges, the network switch node acts as the core hub node in the main graph, simultaneously connecting the audio encoder and decoder nodes of the audio subgraph, the video encoder and decoder nodes of the video subgraph, and the central control processor node and wireless projection receiver node of the control subgraph. This structure expresses the shared hub status of the network switch between audio and video dual modes. The uplink cascading interface edge represents the transmission path where the data streams of each subgraph mode converge into the network switch node, and the downlink cascading interface edge represents the transmission path where the network switch node distributes the data streams to the downlink nodes of each subgraph. When network switch ports degrade, the transmission quality of both audio and video streams will be affected simultaneously. This structure provides a foundation for subsequent identification of cross-modal cascading faults.

[0035] The following technical effects were achieved through this step: First, the dispersed devices in the conference system are divided into common device nodes and modal device nodes according to their functional affiliation, and a main graph and subgraph are constructed for each. Cross-level connections are established through cascading interface edges. Compared with the existing technology that treats each device as an isolated individual, the device dependency graph formally expresses the hierarchical dependency relationships and data flow paths between devices, providing a structural foundation for the subsequent identification of cross-link cascading faults.

[0036] Second, the division of the main diagram and sub-diagrams supports modular expansion. When a new modal device is added to the conference room, only a new sub-diagram needs to be added and connected to the main diagram via a cascading interface, without needing to reconstruct the existing diagram structure; when devices within a modal link are replaced or added or removed, only the corresponding sub-diagram is affected, while the main diagram and other sub-diagrams remain unaffected.

[0037] S200: Real-time acquisition of multi-dimensional operating status time-series data of each device node, and extraction of degradation feature set; In predicting conference equipment failures, operational data from a single source is insufficient to comprehensively reflect the equipment's health status. Conference equipment is diverse, with microphones, speakers, projectors, network switches, and other devices exhibiting varying degrees of degradation. Some degradation is reflected in electrical parameters, others in physical condition, and still others are significantly affected by environmental factors. Furthermore, equipment failures are usually not sudden but leave gradual traces of degradation during operation, manifesting as trend shifts, increased fluctuations, or changes in the spectral structure of sensor readings. Relying solely on built-in diagnostic data limits the coverage and data acquisition dimensions; using only raw sensor readings makes it difficult to capture the degradation patterns hidden within time-series data.

[0038] Step S200 in the method provided in this embodiment of the invention includes: Real-time acquisition of multi-dimensional operational status time-series data for each of the aforementioned device nodes, including: The existing diagnostic data interfaces of each device node are reused to collect the built-in operating parameters of each device node; The physical state parameters of the device nodes and the environmental parameters of their surroundings are collected by multimodal sensors added at preset locations on each of the device nodes. Data cleaning, missing value interpolation, and time alignment are performed on the built-in operating parameters, physical state parameters, and environmental parameters to generate aligned data under a unified time reference. Based on the equipment's rated parameters, the aligned data is normalized, and the normalization result is encoded into a unified equipment status observation vector, which is output as the multidimensional operating status time series data. The equipment status observation vector includes original sensor reading components, derived feature components, environmental parameter components, and equipment static metadata components.

[0039] The specific implementation method is as follows: First, existing diagnostic data interfaces for each device node are reused to collect their built-in operating parameters. For network switch nodes, network layer metrics such as port traffic, error frame count, packet loss rate, and CPU utilization are collected via Simple Network Management Protocol (SMAP). For audio / video codec nodes, encoding / decoding metrics such as encoded frame rate, actual bit rate, encoding latency, and number of dropped frames are collected via device management interfaces. For projection display device nodes, cumulative light source usage time and color temperature data are read via control protocol interfaces. For microphone array nodes, audio metrics such as sensitivity, signal-to-noise ratio, and automatic gain control gain are read via the audio processing chip's debug interface. For speaker nodes, output power and impedance data are read via the power amplifier module's digital interface, and total harmonic distortion plus noise is calculated via the audio processing module. Existing data sources are reused first to minimize hardware modification costs.

[0040] Multimodal sensors are added to preset locations at each device node to collect physical state parameters of the device nodes and environmental parameters of the surrounding environment. For critical physical quantities that cannot be covered by existing data sources, lightweight sensors are added to key locations on the device. For example, an infrared temperature sensor is attached to the heat sink near the speaker voice coil to collect the voice coil temperature; a thermocouple is installed in the optical engine heat dissipation area of ​​the projection display device, and a photosensitive sensor is installed near the light output port to collect the relative values ​​of optical engine temperature and luminous flux; a digital temperature sensor is attached to the surface of the network switch to collect the device surface temperature; a temperature and humidity sensor is attached to the microphone array housing to collect the ambient temperature and humidity near the microphone; and an environmental monitoring module is installed on the wall of the conference room, integrating temperature, humidity, dust concentration, and ambient noise sensors to collect the overall environmental parameters of the conference room.

[0041] Furthermore, data cleaning, missing value interpolation, and time-series alignment are performed on the collected built-in operating parameters, physical state parameters, and environmental parameters.

[0042] Data cleaning employs a sliding window-based median filtering method to replace outlier data points that deviate from the local median by more than three times the local standard deviation, thereby removing outliers caused by momentary sensor malfunctions or communication interruptions.

[0043] Missing value interpolation employs an adaptive interpolation strategy based on device type. For slowly changing physical quantities such as temperature and light source usage time, linear interpolation is used; for rapidly fluctuating physical quantities such as network packet loss rate and audio signal-to-noise ratio, periodic interpolation based on the previous cycle pattern is used.

[0044] Time alignment unifies sensor data from different sampling frequencies to a common time base. Different sensors have different sampling frequencies; for example, network and audio metrics may be sampled once per second, while environmental parameters may be sampled once every ten seconds. Resampling is performed with once per second as the basic time granularity. Data above 1 Hz is downsampled by taking the mean, and data below 1 Hz is padded with the most recent value to generate aligned data under a unified time base.

[0045] Finally, the aligned data is normalized based on the equipment's rated parameters. The normalization method involves subtracting the equipment's nominal parameter value from the current value and then dividing by the nominal parameter value, making the data of similar equipment from different brands and models comparable.

[0046] The normalization results are encoded into a unified equipment status observation vector, outputting multi-dimensional operational status time-series data. The equipment status observation vector consists of four components: the original sensor reading component, containing the normalized current values ​​of all collected indicators for the equipment; the derived feature component, containing degradation features calculated from the original readings through feature engineering; the environmental parameter component, containing the temperature, humidity, dust concentration, and noise level of the conference room where the equipment is located; and the equipment static metadata component, containing the equipment type code, brand and model code, service life, and rated parameters. The original sensor reading components of different equipment types have different dimensions; after standardization and padding, they are uniformly mapped to a vector space of the same dimension, achieving a unified data representation for multiple equipment types.

[0047] Step S200 in the method provided in this embodiment of the invention further includes: Extract the degradation feature set, including: Based on the device status observation vector of each device node, with a preset sliding time window as the observation range, the linear regression slope of each component is calculated and the combined output is a trend feature. Using the sliding time window as the observation range, calculate the rate of change of the standard deviation of each component in the current prediction round and the standard deviation of each component in the previous prediction round, and combine the output as fluctuation characteristics. Perform time-frequency transformation on the time-series data of each component, calculate the spectral entropy of the normalized power spectral density, and combine the outputs to form frequency domain features; The lag autocorrelation coefficients of each component are calculated within the sliding time window, and the combined output is the autocorrelation characteristic. Based on the device type of each device node, extract the device-specific degradation index corresponding to the device type to obtain the device-specific features; The degradation feature set is obtained by combining the trend feature, the fluctuation feature, the frequency domain feature, the autocorrelation feature, and the device-specific feature.

[0048] The specific implementation method is as follows: Degradation features are high-order features extracted from raw sensor readings that characterize the gradual decline in device performance. The observation scope is a preset sliding time window, the size of which can be set according to the device type, for example, 24 hours or 168 hours.

[0049] The slope of a sliding window linear regression is used to characterize the long-term trend of each component index. Linear regression is performed on the time-series data within the sliding window, and the slope of the regression line is calculated. A slope greater than zero indicates an upward trend in the index, such as increased temperature or increased distortion rate; a slope less than zero indicates a downward trend in the index, such as decreased sensitivity or decreased luminous flux. The trend slopes of all components are combined and output as the trend feature.

[0050] Equipment performance degradation is often accompanied by increased fluctuations in operating parameters. The standard deviation of each component within the current prediction cycle's sliding window is calculated and compared to the standard deviation within the previous prediction cycle's sliding window. The rate of change is calculated, and the combined output is the fluctuation characteristic. A larger fluctuation characteristic indicates poorer equipment operational stability.

[0051] The spectral structure changes of the signal are extracted using short-time Fourier transform. Short-time Fourier transforms are performed on the time-series data of each component within the sliding window to obtain the normalized power spectral density, and then the spectral entropy is calculated. An increase in spectral entropy indicates a more complex signal spectral structure and increased noise components, usually related to equipment degradation. The spectral entropies of all components are combined and output as frequency domain features.

[0052] The one-step autocorrelation coefficient of each component's time series data within the sliding window is calculated to characterize the regularity of signal changes. Time series data during normal equipment operation typically exhibits a certain periodicity and regularity; this regularity weakens during degradation, leading to a decrease in the autocorrelation coefficient. The autocorrelation coefficients of all components are combined and output as the autocorrelation feature.

[0053] Based on different device types, specific degradation metrics are extracted. For example, the sensitivity decay rate and self-noise level rise rate of microphone devices; the total harmonic distortion plus noise degradation rate and voice coil temperature anomaly offset of speaker devices; the luminous flux decay rate and thermal cycling count of projection display devices; and the latency jitter trend and burst packet loss frequency of network switching devices. These device-specific metrics are then combined into device-specific characteristics.

[0054] Finally, the trend features, fluctuation features, frequency domain features, autocorrelation features, and device-specific features are combined to obtain a complete set of degradation features.

[0055] The following technical effects were achieved through this step: First, a dual-track acquisition strategy of reusing existing data sources and supplementing with lightweight sensors is adopted. Under the premise of minimizing hardware modification costs, it covers multiple dimensions of equipment electrical parameters, physical status and environmental factors, and realizes full-link perception of the operating status of conference equipment.

[0056] Second, by extracting five types of degradation features—trend features, fluctuation features, frequency domain features, autocorrelation features, and equipment-specific features—the original sensor readings are transformed into high-order feature representations that can characterize the gradual decline trend of equipment performance. This allows the implicit laws of equipment degradation to be explicitly expressed, providing high-quality feature inputs for subsequent time series modeling.

[0057] S300: Based on the device dependency graph and the degradation feature set, and combined with the pre-built link fault coding model, each device node is fused and embedded to obtain the fused embedding vector of each device node. The link fault coding model is constructed based on a graph attention network and a causal temporal convolutional network. In predicting conference equipment failures, single-dimensional information is insufficient to accurately capture the complex propagation patterns of cascading failures. Relying solely on the equipment's historical operating data can characterize its degradation trend, but it cannot perceive the potential impact of upstream equipment anomalies on the equipment itself. Analyzing only the dependencies between equipment can identify failure propagation paths, but it lacks a quantitative understanding of the degree of degradation of the equipment itself. Furthermore, different types of equipment have varying degrees of dependence on these two types of information. For example, the aging of a projector's light source is mainly determined by the cumulative effect of its own usage time, while the operating quality of an audio decoder is highly dependent on the transmission status of the network switch.

[0058] Step S300 in the method provided in this embodiment of the invention includes: Based on the device dependency graph and the degradation feature set, and combined with the pre-built link fault coding model, each device node undergoes fusion embedding coding to obtain a fusion embedding vector for each device node, including: Based on the device dependency graph, the graph attention network in the link fault coding model is used to perform multi-level spatial reasoning on each device node to obtain the spatial embedding vector of each device node. By using the causal temporal convolutional network in the link fault coding model, causal convolutional coding is performed on the temporal data corresponding to the degradation features of each device node to obtain the temporal embedding vector of each device node. The spatial embedding vector and the temporal embedding vector are fused using a cross-attention fusion mechanism to obtain the fused embedding vector for each device node.

[0059] The specific implementation method is as follows: First, based on the device dependency graph constructed in step S100, multi-level spatial reasoning is performed on each device node through the graph attention network in the link fault coding model to obtain the spatial embedding vector of each device node. The graph attention network assigns different weights to each neighbor node in the graph through an attention mechanism, and performs message passing and feature aggregation along the edges of the device dependency graph. Spatial reasoning is divided into three levels: intra-modal message passing within each subgraph to aggregate the state information of upstream and downstream device nodes within the same modal link; uplinking the aggregation information of the subgraph boundary nodes to the main graph node for cross-modal aggregation through cascaded interface edges; and downlinking the global context information of the main graph node back to each subgraph boundary node. After three levels of message passing, the spatial embedding vector obtained by each device node simultaneously contains three types of information: its own state, the influence of upstream and downstream devices within the modal link, and cross-modal global dependency relationships.

[0060] Furthermore, the causal temporal convolutional network in the link fault coding model performs causal convolutional encoding on the temporal data corresponding to the degradation features of each device node, obtaining the temporal embedding vector of each device node. The causal temporal convolutional network adopts a dilated causal convolution structure. Causal convolution ensures that the output at the current time depends only on the input at past time steps, and dilated convolution expands the receptive field by inserting gaps between the convolution kernel elements. The degradation feature temporal data of each device node are input into the causal temporal convolutional network for independent encoding, and the output is a temporal embedding vector with the same dimension.

[0061] Finally, spatial and temporal embedding vectors are fused using a cross-attention fusion mechanism. A bidirectional cross-attention mechanism is constructed, using the spatial embedding vector as the query and the temporal embedding vector as the key to calculate the spatially guided temporal attention output, and using the temporal embedding vector as the query and the spatial embedding vector as the key to calculate the temporally guided spatial attention output. Learnable gating vectors are configured to adaptively determine the fusion weights of the attention outputs in both directions, and these weights are dynamically adjusted based on the functional affiliation of the device node in the device dependency graph. The temporal and spatial attention outputs are then weighted and summed according to the adjusted fusion weights to obtain the fused embedding vector for each device node.

[0062] Step S300 in the method provided in this embodiment of the invention further includes: Based on the device dependency graph, multi-level spatial reasoning is performed on each device node through the graph attention network in the link fault coding model to obtain the spatial embedding vector of each device node, including: Using device type as the analysis boundary, within each subgraph, the graph attention network performs intramodal message passing on each device node in the subgraph, aggregates the state information of upstream and downstream device nodes in the same modal link, and obtains local aggregation features for each subgraph. Through the cascaded interface edge, multiple local aggregated features are transmitted uplink to the corresponding public device nodes in the main graph. Within the main graph, each public device node performs cross-modal aggregation on the local aggregated features corresponding to multiple modal chains to obtain cross-modal global aggregated features. The cross-modal global aggregation features of each common device node in the main graph are distributed downlink to the associated subgraph boundary nodes; Based on the downlink distribution results, the spatial embedding vectors of each device node are obtained by traversal.

[0063] The specific implementation method is as follows: Multi-level spatial reasoning performs three levels of message passing along the device dependency graph via a graph attention network.

[0064] The first layer involves message passing within each subgraph. Analysis is performed independently within each subgraph, using device type as the boundary. The graph attention network calculates attention weights between each node in the subgraph and its neighbors. The weights are calculated as follows: the feature vector of the current node and the feature vectors of its neighbors are linearly transformed and concatenated, multiplied by a learnable attention vector, and then processed through an activation function and a normalization function to obtain the attention weights of the neighbors for the current node. These weights reflect the importance of the upstream and downstream device nodes' influence on the current device node's state within the current modality link. The feature values ​​of the neighbors are weighted and summed according to their attention weights, then combined with the current node's own features and linearly transformed to obtain the output feature of this layer. After passing through multiple layers of the graph attention network, each device node within the subgraph obtains a locally aggregated feature that incorporates the states of upstream and downstream devices within its modality link.

[0065] Taking the audio subgraph as an example, the sensitivity changes and signal-to-noise ratio (SNR) decreases of the microphone array nodes are passed to the audio encoder nodes after attention weighting. The audio encoder nodes then fuse their own encoding latency and frame drop rate status information with information from the upstream microphone array nodes to form a new feature representation. As the uplink boundary node of the audio subgraph, the local aggregated features of the audio encoder node encapsulate information about the overall health of the audio link. Similarly, the local aggregated features of the video encoder nodes in the video subgraph encapsulate information about the overall health of the video link.

[0066] The second layer involves uplink transmission and cross-modal aggregation. Through the cascading interface edges established in step S100, the local aggregation features of the uplink boundary nodes of each subgraph are transmitted uplink to the corresponding common device nodes in the main graph. The local aggregation features of the audio encoder node are transmitted to the network switch node via the uplink cascading interface edges, and the local aggregation features of the video encoder node are also transmitted to the same network switch node.

[0067] After receiving uplink information from multiple subgraphs, network switch nodes perform cross-modal aggregation of local aggregated features corresponding to multiple modal chains within the main graph, handled by each common device node. The graph attention network within the main graph facilitates message passing between network switch nodes, power supply equipment nodes, environmental control equipment nodes, and central control equipment nodes. Network switch nodes aggregate ambient temperature information from environmental control equipment nodes and power supply status information from power supply equipment nodes, combining uplink information from multiple modalities to form a cross-modal global aggregated feature.

[0068] The key role of this layer is as follows: when the audio encoder shows a degradation trend of increasing encoding latency, this information is transmitted to the network switch node via the uplink; if the video encoder also detects a decline in transmission quality, the uplink information from the two subgraphs forms cross-validation at the network switch node. Simultaneously, the high temperature information provided by the environmental control device node and the normal power supply information provided by the power supply device node together help the cross-modal global aggregation feature of the network switch node point to poor heat dissipation rather than power supply issues as the root cause of the degradation.

[0069] The third layer is downlink distribution. The cross-modal global aggregated features of each common device node in the main graph are distributed downlink to the associated subgraph boundary nodes. Network switch nodes transmit their cross-modal global aggregated features downlink to the audio decoder and video decoder nodes via downlink cascaded interface edges. Upon receiving the downlink information from the network switch nodes, the audio decoder nodes learn the global context information that "even if their own hardware is normal, the input signal quality may continue to degrade due to network port degradation and rack heat dissipation issues." The video decoder nodes receive similar downlink information.

[0070] Through the three-level message passing described above, all device nodes in the main graph and all subgraphs are traversed, and a spatial embedding vector is output for each device node. The spatial embedding vector of each device node contains information at three levels: the device's own state information, the impact information of upstream and downstream devices within the modal link, and the cross-modal global dependency information.

[0071] Step S300 in the method provided in this embodiment of the invention further includes: The spatial embedding vector and the temporal embedding vector are fused using a cross-attention fusion mechanism to obtain the fused embedding vector for each device node, including: Using the spatial embedding vector of each device node as the query and the temporal embedding vector as the key, calculate the spatially guided temporal attention output; Using the temporal embedding vector of each device node as the query and the spatial embedding vector as the key, calculate the temporally guided spatial attention output; Configure device type-oriented learnable gating vectors to adaptively determine the fusion weights of the temporal attention output and the spatial attention output; The fusion weights are dynamically adjusted based on the functional affiliation of each device node in the device dependency graph. Specifically, the fusion weights of the temporal attention output are increased for device nodes in the subgraph, and the fusion weights of the spatial attention output are increased for device nodes in the main graph. Based on the adjusted fusion weights, the temporal attention output and the spatial attention output are weighted and summed to obtain the fusion embedding vector of each device node.

[0072] The specific implementation method is as follows: First, a bidirectional cross-attention mechanism is constructed to interactively fuse spatial embedding vectors and temporal embedding vectors from two directions.

[0073] The first direction is spatially guided temporal attention. Using the spatial embedding vectors of each device node as queries and the temporal embedding vectors as keys and values, spatially guided temporal attention output is obtained through attention calculation. This output reflects which temporal degradation information is more critical from the perspective of spatial dependencies. For example, for an audio decoder node, its spatial embedding vector already contains downlink information from the network switch node, indicating a decline in network transmission quality. In this direction, degradation features in the temporal data related to decoding buffer usage and packet loss recovery counts will receive higher attention weights.

[0074] The second direction is time-guided spatial attention. Using the temporal embedding vector of each device node as the query and the spatial embedding vector as the key and value, time-guided spatial attention output is obtained through attention calculation. This output reflects which upstream and downstream dependencies are more worthy of attention from the perspective of the device's own degradation state. For example, for a projection display device node, its temporal embedding vector encodes the continuous decay trend of luminous flux. In this direction, dependencies related to environmental regulation equipment in the spatial embedding vector, such as the impact of ambient temperature on the heat dissipation of the optomechanical system, will receive higher attention weights.

[0075] The principle of attention calculation is as follows: perform a dot product operation between the query vector and the key vector, and obtain the attention weights after processing by scaling and normalization functions. Then, sum the value vectors according to the attention weights to obtain the output.

[0076] Furthermore, a device-type-oriented learnable gating vector is configured to adaptively determine the fusion weights of temporal attention output and spatial attention output. The gating vector is generated as follows: the spatial embedding vector and temporal embedding vector of the current device node are concatenated, mapped through a fully connected layer, and then activated by a Sigmoid activation function to output a scalar value between 0 and 1 as the gating value.

[0077] The fusion weights are dynamically adjusted based on the functional affiliation of each device node in the device dependency graph. For device nodes located in subgraphs, since their primary failure mode is performance degradation, the fusion weight of temporal attention output is increased based on the gating value, making the fusion embedding vector more focused on reflecting the temporal information of the device's own degradation trend. For common device nodes located in the main graph, since their failures may trigger cross-modal cascading effects, the fusion weight of spatial attention output is increased based on the gating value, making the fusion embedding vector more focused on reflecting the spatial information of global dependencies.

[0078] For example, in the subgraph, the light flux attenuation of the projector node is mainly determined by the aging of its own light source, and the gating mechanism will automatically increase the weight of the temporal attention output; while in the main graph, the port degradation of the network switch node will affect both audio and video modalities, and the gating mechanism will automatically increase the weight of the spatial attention output.

[0079] Finally, based on the adjusted fusion weights, the temporal attention output and spatial attention output are weighted and summed to obtain the fusion embedding vector for each device node. This vector integrates two types of information: spatial reasoning and the device's own temporal degradation, and can adaptively adjust the contribution ratio of the two types of information according to the device type and function affiliation.

[0080] The following technical effects were achieved through this step: First, a graph attention network is used to perform three levels of spatial reasoning along the device dependency graph, enabling each device node's spatial embedding vector to simultaneously contain three types of information: its own state, the influence of upstream and downstream devices within the modal link, and cross-modal global dependencies. When upstream devices experience performance degradation, downstream devices, even if their own sensor indicators are normal, can still detect potential risks through spatial reasoning, providing an informational basis for identifying cascading failures.

[0081] Second, a cross-attention fusion mechanism is used to bidirectionally fuse spatial embedding vectors and temporal embedding vectors, and a learnable gating vector is used to adaptively adjust the fusion weights according to the functional affiliation of device nodes. Subgraph devices focus more on their own temporal degradation information, while maingraph devices focus more on cross-modal spatial dependency information, so that the fused embedding vectors of different types of devices can highlight the information dimensions most valuable for their fault prediction.

[0082] S400: Based on the fused embedding vector, the failure probability and remaining service life of each device node are predicted by the corresponding multi-head task predictor, and the warning level of each device node is determined accordingly.

[0083] The fused embedding vector output in step S300 contains comprehensive information on the degradation trends of each device node and cross-device cascading dependencies. However, obtaining the fused embedding vector alone is insufficient to support operational decisions. Operations personnel need to know when a device might fail, the probability of failure, its remaining lifespan, and its importance in the current meeting scenario. Different alert levels require different response actions, ranging from automatically switching to backup devices to scheduling maintenance plans.

[0084] Step S400 in the method provided in this embodiment of the invention includes: inputting the fused embedding vector of each device node into the corresponding fault probability prediction head in the multi-head task predictor to obtain the fault probability of each device node at multiple future time steps; The fused embedding vector of each device node is input into the corresponding remaining lifetime regression head in the multi-head task predictor to obtain the remaining lifetime and uncertainty standard deviation of each device node, and to calculate the associated confidence interval. Based on the centrality index and current usage status of each device node in the device dependency graph, calculate the importance score of each device node. Based on the failure probability, remaining service life, and importance score, the early warning level of each device node is divided into four levels: emergency, warning, alert, and healthy. The emergency level triggers automatic switching to backup equipment and generation of emergency maintenance work orders; the warning level triggers pre-locking of backup equipment and inclusion in the maintenance plan; the alert level triggers increased monitoring frequency; and the healthy level maintains regular monitoring.

[0085] The specific implementation method is as follows: First, the failure probability is predicted. The fused embedding vectors of each device node are input into the failure probability prediction head of the multi-head task predictor to obtain the failure probability of each device node at multiple future time steps. The failure probability prediction head consists of two fully connected layers and a sigmoid activation function. The fused embedding vector undergoes a nonlinear transformation through the first fully connected layer and a linear rectified activation function, and is then mapped by the second fully connected layer to a vector of the same dimension as the number of prediction time steps. The sigmoid function compresses each element to between zero and one, outputting the failure probability at each of the multiple future time steps. The number of prediction time steps can be set according to actual needs; for example, outputting the failure probability for the next 6 time steps, each step being 1 hour.

[0086] Furthermore, the remaining useful life is predicted. The fused embedding vectors of each device node are input into the remaining useful life regression head of the multi-head task predictor to obtain the remaining useful life and uncertainty standard deviation of each device node. The remaining useful life regression head consists of two fully connected layers, outputting a two-dimensional vector representing the original mean and standard deviation of the remaining useful life, respectively. The original mean value is transformed by the Softplus function to obtain the predicted remaining useful life in hours. The original standard deviation value is transformed by the Softplus function and a small constant is added to obtain the uncertainty standard deviation. Based on the predicted remaining useful life and the uncertainty standard deviation, a 95% confidence interval for the remaining useful life is calculated. The lower limit of the confidence interval is the predicted value minus 1.96 times the standard deviation, and the upper limit is the predicted value plus 1.96 times the standard deviation.

[0087] The failure probability prediction head and the remaining useful life regression head share the same fused embedding vector, but each uses independent network parameters and is optimized for classification and regression tasks respectively.

[0088] Further, importance scores are calculated. Based on the centrality index and current usage status of each device node in the device dependency graph, an importance score is calculated for each device node. The centrality index can be a normalized value of degree centrality or betweenness centrality, reflecting the hub status of the device node in the dependency graph—for example, a network switch node connecting the audio and video subgraphs has a much higher centrality index than a speaker node located at the end of the link. The current usage status indicates whether the device is currently in active use. The importance score is a weighted sum of the centrality index, usage status, and the current meeting urgency code. The meeting urgency is set according to different meeting types; for example, a higher value is set for important meetings, a lower value for regular meetings, and the lowest value when there are no meetings.

[0089] Finally, the warning levels are classified. Based on the failure probability, remaining service life, and importance score, the warning levels of each device node are divided into four levels: emergency, warning, alert, and healthy.

[0090] It should be noted that the failure probability, service life, and response time of the trigger conditions for the following emergency levels are all exemplary preferred values. In actual deployment, they can be flexibly adjusted according to the reliability level requirements of the conference system, equipment type, historical operation and maintenance experience, and performance indicators on the model validation set, and do not constitute a limitation of the present invention.

[0091] The emergency level is triggered when: the maximum probability of failure over multiple future time steps is greater than or equal to 0.85, or the remaining service life is less than or equal to 2 hours. Upon triggering the emergency level, the following actions are automatically performed: automatically switching to backup equipment to ensure the meeting is not affected, generating an emergency maintenance work order and notifying maintenance personnel to handle it immediately, and informing the meeting organizer of the current equipment status. The emergency level response time requirement is immediate.

[0092] The warning level is triggered when: the emergency level has not been reached, but the maximum probability of failure is greater than or equal to 0.60, or the remaining service life is less than or equal to 24 hours. Upon triggering the warning level, the following actions will be automatically performed: backup equipment resources will be pre-locked, the equipment will be scheduled for maintenance, and replacement is recommended during non-meeting periods. The warning level response time requirement is within 4 hours.

[0093] The trigger conditions for the alert level are: the warning level has not been reached, but the maximum probability of failure is greater than or equal to 0.35, or the remaining service life is less than or equal to 168 hours (7 days). Upon triggering the alert level, the following actions will be automatically performed: increase the monitoring frequency of the device and mark it as a device of high concern. The alert level response time requirement is within 7 days.

[0094] The health level is triggered when none of the above conditions are met, the equipment is operating normally, and the regular monitoring frequency is maintained.

[0095] The following technical effects were achieved through this step: First, by using a multi-task predictor to simultaneously output the failure probability and remaining service life, it provides both a probability judgment of whether a failure will occur and an estimate of the time window and confidence interval for the failure to occur, enabling maintenance personnel to assess equipment risks from multiple dimensions.

[0096] Second, by combining the centrality index of the equipment in the dependency graph with its current usage status to calculate the importance score, the urgency of the failure and the scope of the equipment's impact are comprehensively incorporated into the early warning level classification. This avoids the shortcomings of treating core hub equipment and peripheral equipment equally when making decisions based solely on the probability of failure, and makes the early warning results closer to actual operation and maintenance needs.

[0097] Steps S100 to S400 above describe the complete online inference process, including equipment dependency graph construction, multi-dimensional operational state time-series data acquisition and degradation feature extraction, fusion embedding encoding based on the link fault coding model, and fault probability prediction and early warning level determination based on a multi-task predictor. In this process, the use of the link fault coding model and the multi-task predictor assumes that the model has already been trained. The pre-construction of the link fault coding model involves a three-level training strategy and a multi-task joint loss function, which are described in detail below.

[0098] The pre-construction of the link failure coding model includes: Obtain normal operation data of the conference system, and based on the normal operation data, establish a contrastive learning task and a mask prediction task, and perform self-supervised pre-training; Using modality as the matching target, an industrial equipment fault dataset is obtained, and cross-domain transfer learning is performed based on the industrial equipment fault dataset through a gradient inversion coding layer. The conference equipment failure samples of the conference system are obtained, and combined with the conference equipment failure samples and the synthetic failures based on the physical degradation model, a few-sample fine-tuning is performed to obtain the link failure coding model based on graph attention network and causal temporal convolutional network. The training of the link fault coding model employs a multi-task joint loss function, which includes at least: a weighted binary cross-entropy loss for fault probability prediction, a negative log-likelihood loss for remaining useful life regression, a logical consistency constraint loss between fault probability and remaining useful life, and a hierarchical consistency constraint loss between the fault prediction results of the main graph node and the associated subgraph node.

[0099] The specific implementation method is as follows: First, self-supervised pre-training is performed. Normal operation data of the conference system is acquired. Since the conference system operates normally most of the time, a large amount of fault-free labeled operational data can be accumulated. Two parallel self-supervised pre-training tasks are established based on this normal operation data; no fault labels are required at this stage.

[0100] The first task is a contrastive learning task. For data from the same device node within the same time window, two different data augmentation operations—temporal warp and amplitude scaling—are applied, generating two augmented views as positive sample pairs. Temporal warp slightly stretches or compresses the time axis, while amplitude scaling slightly adjusts the numerical range. Data from different device nodes or different time windows within the same batch are used as negative samples. The model is trained to make the representation vectors of positive sample pairs as close as possible to each other in the feature space, and as far apart as possible from the representation vectors of negative sample pairs. The contrastive learning loss uses normalized temperature cross-entropy loss, with the temperature parameter controlling the concentration of sample distribution in the feature space.

[0101] The second task is mask prediction. This involves randomly masking some feature dimensions of device nodes in the device dependency graph, or randomly masking data from some time steps within a time series window. The model is then trained to reconstruct the masked portions from the remaining visible information. This task enables the model to learn the association patterns between devices and the inherent structure of the time series data, allowing it to infer the global state even when faced with partially missing information.

[0102] By jointly training with comparative learning tasks and mask prediction tasks, the link fault coding model learns the general representation ability of conference equipment operation data and the correlation between devices without any fault labels.

[0103] Furthermore, cross-domain transfer learning is performed. Using modality as the matching target, an industrial equipment failure dataset is obtained as the source domain. The industrial equipment failure dataset can be a publicly available predictive maintenance dataset for industrial equipment, containing complete lifecycle data of the equipment from normal operation to gradual degradation and eventual failure, with sufficient failure degradation labels.

[0104] A domain discriminator is added to the encoder output of the link fault coding model to distinguish whether features originate from the source or target domain. A gradient inversion coding layer is inserted between the encoder and the domain discriminator. During backpropagation, the gradient inversion layer multiplies the gradient of the domain discriminator by a negative coefficient before backpropagating it to the encoder. The encoder is trained to deceive the domain discriminator, making it unable to distinguish whether features originate from the industrial equipment domain or the conference equipment domain, thus enabling the degradation patterns learned by the encoder to generalize across domains. Domain adversarial training is jointly executed with a fault prediction task in the source domain, allowing the link fault coding to retain fault discrimination capabilities during domain adaptation.

[0105] Furthermore, fine-tuning is performed using a small number of samples. Fault samples from the conference system's equipment are obtained. Since conference equipment failures are low-probability events, the number of real fault samples accumulated through actual operation is scarce. Based on these real fault samples, synthetic fault samples based on a physical degradation model are used to expand the training data.

[0106] The methods for generating synthetic fault samples include: gradual decay type, which superimposes a slowly increasing offset on a certain indicator of normal operating data to simulate gradual degradation; sudden deterioration type, which randomly inserts sudden performance jumps into normal operating data to simulate sudden deterioration of components; and periodic deterioration type, which superimposes gradually intensifying periodic fluctuations into normal operating data to simulate the decrease in stability of equipment during aging.

[0107] A meta-learning framework is employed to fine-tune the link fault coding model with a small number of samples. The core idea of ​​meta-learning is to find an initialization parameter that allows the link fault coding model to achieve good performance on new tasks with only a small number of samples and gradient steps. Meta-learning consists of two phases: an inner loop and an outer loop. In the inner loop, a task is sampled from the device type distribution, and several gradient updates are performed on a small number of labeled samples to obtain task-specific parameters. In the outer loop, the original model parameters are optimized based on the loss of the task-specific parameters on validation samples. By iterating through tasks of different device types and repeatedly executing the inner and outer loops, the link fault coding model gains the ability to quickly adapt to the prediction of faults of new device types.

[0108] Finally, a link fault coding model based on graph attention network and causal temporal convolutional network is obtained.

[0109] The link failure coding model is trained using a multi-task joint loss function, which consists of a weighted sum of four loss terms.

[0110] The failure probability prediction loss uses a weighted binary cross-entropy loss with class weights. Since there are far more samples in the normal operating state than in the failure state, failure samples are assigned a larger positive class weight, while normal samples are assigned a smaller negative class weight, in order to emphasize the prediction accuracy of failure samples.

[0111] The remaining useful life regression loss uses negative log-likelihood loss. It is assumed that the prediction error of the remaining useful life follows a Gaussian distribution centered at the prediction mean and scaled by the variance of the uncertainty, and the likelihood probability of the true remaining useful life value under this distribution is maximized.

[0112] The logical consistency constraint loss ensures logical consistency between the failure probability prediction and the remaining useful life prediction. If the failure probability of a device node is high, its remaining useful life should be short; conversely, if the remaining useful life is long, its failure probability should not be high. This loss term constrains the monotonically decreasing relationship between the two through a linear rectification function.

[0113] The hierarchical consistency constraint loss ensures hierarchical consistency of fault prediction results between main graph nodes and associated subgraph nodes. Upward consistency ensures that when the fault probability of an upward boundary node in a subgraph is high, the fault probability of an associated node in the main graph should not be too low; downward consistency ensures that when the fault probability of a main graph node is high, the fault probability of its associated downward boundary node in the subgraph should be appropriately increased. This loss term constrains the degree of inconsistency in the upward and downward directions through a linear rectification function, allowing for differences within a reasonable deviation margin.

[0114] The final loss function is the weighted sum of the four loss terms mentioned above according to preset hyperparameters, which are determined by grid search on the validation set.

[0115] The pre-construction of the link fault coding model in this invention has the following technical effects or advantages: The three-level training strategy sequentially solves the problem of scarce fault samples in conference equipment by learning general equipment state representations through self-supervised pre-training, introducing prior knowledge of degradation patterns in the industrial equipment field through cross-domain transfer learning, and fine-tuning with few samples to adapt to specific fault scenarios of conference equipment. This enables the model to achieve high prediction accuracy and generalization ability under the condition of very few actual fault samples.

[0116] Example 2 like Figure 2As shown, based on the same inventive concept as the end-to-end sensing conference equipment fault prediction method provided in Embodiment 1, this embodiment of the invention also provides an end-to-end sensing conference equipment fault prediction system. The system includes a graph construction module, a sensing and feature extraction module, a fusion embedding coding module, and a prediction and early warning module. The system includes: Graph construction module 11 is used to obtain the physical connection relationship and data flow dependency relationship between each device node in the conference system, and construct a device dependency graph accordingly, wherein the device dependency graph includes a main graph and at least two subgraphs; The perception and feature extraction module 12 is used to collect multi-dimensional operating status time-series data of each device node in real time and extract degradation feature sets; The fusion embedding coding module 13 is used to perform fusion embedding coding on each of the device nodes based on the device dependency graph and the degradation feature set, combined with a pre-built link fault coding model, to obtain the fusion embedding vector of each of the device nodes. The link fault coding model is constructed based on a graph attention network and a causal temporal convolutional network. The prediction and early warning module 14 is used to predict the failure probability and remaining service life of each device node based on the fused embedding vector and through the corresponding multi-head task predictor, and to determine the early warning level of each device node accordingly.

[0117] In one embodiment, the graph construction module 11 is further configured to obtain the physical connection relationships and data flow dependencies between device nodes in the conference system, and construct a corresponding device dependency graph, including: Based on the functional affiliation of each device node in the conference system, the device nodes are divided into common device nodes shared by multiple modes and modal device nodes dedicated to each modal link. Using multiple common device nodes as graph nodes and the dependencies between the multiple common device nodes as relation edges, the main graph in the device dependency graph is constructed. For each modal link, a subgraph is constructed using the corresponding multiple modal device nodes as graph nodes and the signal flow dependencies between the multiple modal device nodes as relation edges. This process is repeated to obtain at least two subgraphs. Identify the boundary nodes of the data flow entering and exiting the main graph in multiple subgraphs, and establish cascaded interface edges between the multiple boundary nodes and the corresponding common device nodes in the main graph to obtain the device dependency graph.

[0118] In one embodiment, the perception and feature extraction module 12 is further configured to collect multi-dimensional operational status time-series data of each device node in real time, including: The existing diagnostic data interfaces of each device node are reused to collect the built-in operating parameters of each device node; The physical state parameters of the device nodes and the environmental parameters of their surroundings are collected by multimodal sensors added at preset locations on each of the device nodes. Data cleaning, missing value interpolation, and time alignment are performed on the built-in operating parameters, physical state parameters, and environmental parameters to generate aligned data under a unified time reference. Based on the equipment's rated parameters, the aligned data is normalized, and the normalization result is encoded into a unified equipment status observation vector, which is output as the multidimensional operating status time series data. The equipment status observation vector includes original sensor reading components, derived feature components, environmental parameter components, and equipment static metadata components.

[0119] Extract the degradation feature set, including: Based on the device status observation vector of each device node, with a preset sliding time window as the observation range, the linear regression slope of each component is calculated and the combined output is a trend feature. Using the sliding time window as the observation range, calculate the rate of change of the standard deviation of each component in the current prediction round and the standard deviation of each component in the previous prediction round, and combine the output as fluctuation characteristics. Perform time-frequency transformation on the time-series data of each component, calculate the spectral entropy of the normalized power spectral density, and combine the outputs to form frequency domain features; The lag autocorrelation coefficients of each component are calculated within the sliding time window, and the combined output is the autocorrelation characteristic. Based on the device type of each device node, extract the device-specific degradation index corresponding to the device type to obtain the device-specific features; The degradation feature set is obtained by combining the trend feature, the fluctuation feature, the frequency domain feature, the autocorrelation feature, and the device-specific feature.

[0120] In one embodiment, the fusion embedding coding module 13 is further configured to perform fusion embedding coding on each of the device nodes based on the device dependency graph and the degradation feature set, combined with a pre-built link fault coding model, to obtain a fusion embedding vector for each of the device nodes, including: Based on the device dependency graph, the graph attention network in the link fault coding model is used to perform multi-level spatial reasoning on each device node to obtain the spatial embedding vector of each device node. By using the causal temporal convolutional network in the link fault coding model, causal convolutional coding is performed on the temporal data corresponding to the degradation features of each device node to obtain the temporal embedding vector of each device node. The spatial embedding vector and the temporal embedding vector are fused using a cross-attention fusion mechanism to obtain the fused embedding vector for each device node.

[0121] Based on the device dependency graph, multi-level spatial reasoning is performed on each device node through the graph attention network in the link fault coding model to obtain the spatial embedding vector of each device node, including: Using device type as the analysis boundary, within each subgraph, the graph attention network performs intramodal message passing on each device node in the subgraph, aggregates the state information of upstream and downstream device nodes in the same modal link, and obtains local aggregation features for each subgraph. Through the cascaded interface edge, multiple local aggregated features are transmitted uplink to the corresponding public device nodes in the main graph. Within the main graph, each public device node performs cross-modal aggregation on the local aggregated features corresponding to multiple modal chains to obtain cross-modal global aggregated features. The cross-modal global aggregation features of each common device node in the main graph are distributed downlink to the associated subgraph boundary nodes; Based on the downlink distribution results, the spatial embedding vectors of each device node are obtained by traversal.

[0122] The spatial embedding vector and the temporal embedding vector are fused using a cross-attention fusion mechanism to obtain the fused embedding vector for each device node, including: Using the spatial embedding vector of each device node as the query and the temporal embedding vector as the key, calculate the spatially guided temporal attention output; Using the temporal embedding vector of each device node as the query and the spatial embedding vector as the key, calculate the temporally guided spatial attention output; Configure device type-oriented learnable gating vectors to adaptively determine the fusion weights of the temporal attention output and the spatial attention output; The fusion weights are dynamically adjusted based on the functional affiliation of each device node in the device dependency graph. Specifically, the fusion weights of the temporal attention output are increased for device nodes in the subgraph, and the fusion weights of the spatial attention output are increased for device nodes in the main graph. Based on the adjusted fusion weights, the temporal attention output and the spatial attention output are weighted and summed to obtain the fusion embedding vector of each device node.

[0123] In one embodiment, the prediction and early warning module 14 is further configured to predict the failure probability and remaining service life of each device node based on the fused embedding vector and through the corresponding multi-head task predictor, and determine the early warning level of each device node accordingly, including: The fused embedding vector of each device node is input into the corresponding fault probability prediction head in the multi-head task predictor to obtain the fault probability of each device node at multiple future time steps. The fused embedding vector of each device node is input into the corresponding remaining lifetime regression head in the multi-head task predictor to obtain the remaining lifetime and uncertainty standard deviation of each device node, and to calculate the associated confidence interval. Based on the centrality index and current usage status of each device node in the device dependency graph, calculate the importance score of each device node. Based on the failure probability, remaining service life, and importance score, the early warning level of each device node is divided into four levels: emergency, warning, alert, and healthy. The emergency level triggers automatic switching to backup equipment and generation of emergency maintenance work orders; the warning level triggers pre-locking of backup equipment and inclusion in the maintenance plan; the alert level triggers increased monitoring frequency; and the healthy level maintains regular monitoring.

[0124] It should be noted that the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting faults in conferencing equipment with end-to-end awareness, characterized in that, include: Obtain the physical connection relationships and data flow dependencies between device nodes in the conference system, and construct a device dependency graph accordingly, wherein the device dependency graph includes a main graph and at least two subgraphs; Real-time acquisition of multi-dimensional operational status time-series data of each device node, and extraction of degradation feature set; Based on the device dependency graph and the degradation feature set, and combined with the pre-built link fault coding model, each device node is fused and embedded to obtain the fused embedding vector of each device node. The link fault coding model is constructed based on graph attention network and causal temporal convolutional network. Based on the fused embedding vector, the failure probability and remaining service life of each device node are predicted by the corresponding multi-head task predictor, and the warning level of each device node is determined accordingly. This includes obtaining the physical connection relationships and data flow dependencies between device nodes in the conference system, and constructing a corresponding device dependency graph, including: Based on the functional affiliation of each device node in the conference system, the device nodes are divided into common device nodes shared by multiple modes and modal device nodes dedicated to each modal link. Using multiple common device nodes as graph nodes and the dependencies between the multiple common device nodes as relation edges, the main graph in the device dependency graph is constructed. For each modal link, a subgraph is constructed using the corresponding multiple modal device nodes as graph nodes and the signal flow dependencies between the multiple modal device nodes as relation edges. This process is repeated to obtain at least two subgraphs. Identify the boundary nodes where data flows into and out of the main graph in multiple subgraphs, and establish cascaded interface edges between multiple boundary nodes and the corresponding common device nodes in the main graph to obtain the device dependency graph; Specifically, based on the device dependency graph and the degradation feature set, and combined with a pre-constructed link fault coding model, each device node undergoes fusion embedding coding to obtain a fusion embedding vector for each device node, including: Based on the device dependency graph, the graph attention network in the link fault coding model is used to perform multi-level spatial reasoning on each device node to obtain the spatial embedding vector of each device node. By using the causal temporal convolutional network in the link fault coding model, causal convolutional coding is performed on the temporal data corresponding to the degradation features of each device node to obtain the temporal embedding vector of each device node. The spatial embedding vector and the temporal embedding vector are fused using a cross-attention fusion mechanism to obtain the fused embedding vector for each device node.

2. The end-to-end sensing method for predicting conference equipment faults as described in claim 1, characterized in that, Real-time acquisition of multi-dimensional operational status time-series data for each of the aforementioned device nodes, including: The existing diagnostic data interfaces of each device node are reused to collect the built-in operating parameters of each device node; The physical state parameters of the device nodes and the environmental parameters of their surroundings are collected by multimodal sensors added at preset locations on each of the device nodes. Data cleaning, missing value interpolation, and time alignment are performed on the built-in operating parameters, physical state parameters, and environmental parameters to generate aligned data under a unified time reference. Based on the equipment's rated parameters, the aligned data is normalized, and the normalization result is encoded into a unified equipment status observation vector, which is output as the multidimensional operating status time series data. The equipment status observation vector includes original sensor reading components, derived feature components, environmental parameter components, and equipment static metadata components.

3. The end-to-end sensing method for predicting conference equipment faults as described in claim 1, characterized in that, Extract the degradation feature set, including: Based on the device status observation vector of each device node, with a preset sliding time window as the observation range, the linear regression slope of each component is calculated and the combined output is a trend feature. Using the sliding time window as the observation range, calculate the rate of change of the standard deviation of each component in the current prediction round and the standard deviation of each component in the previous prediction round, and combine the output as fluctuation characteristics. Perform time-frequency transformation on the time-series data of each component, calculate the spectral entropy of the normalized power spectral density, and combine the outputs to form frequency domain features; The lag autocorrelation coefficients of each component are calculated within the sliding time window, and the combined output is the autocorrelation characteristic. Based on the device type of each device node, extract the device-specific degradation index corresponding to the device type to obtain the device-specific features; The degradation feature set is obtained by combining the trend feature, the fluctuation feature, the frequency domain feature, the autocorrelation feature, and the device-specific feature.

4. The end-to-end sensing method for predicting conference equipment faults as described in claim 1, characterized in that, Based on the device dependency graph, multi-level spatial reasoning is performed on each device node through the graph attention network in the link fault coding model to obtain the spatial embedding vector of each device node, including: Using device type as the analysis boundary, within each subgraph, the graph attention network performs intramodal message passing on each device node in the subgraph, aggregates the state information of upstream and downstream device nodes in the same modal link, and obtains local aggregation features for each subgraph. Through the cascaded interface edge, multiple local aggregated features are transmitted uplink to the corresponding common device nodes in the main graph. Within the main graph, each common device node performs cross-modal aggregation on the local aggregated features corresponding to multiple modal links to obtain cross-modal global aggregated features. The cross-modal global aggregation features of each common device node in the main graph are distributed downlink to the associated subgraph boundary nodes; Based on the downlink distribution results, the spatial embedding vectors of each device node are obtained by traversal.

5. The end-to-end sensing method for predicting conference equipment faults as described in claim 1, characterized in that, The spatial embedding vector and the temporal embedding vector are fused using a cross-attention fusion mechanism to obtain the fused embedding vector for each device node, including: Using the spatial embedding vector of each device node as the query and the temporal embedding vector as the key, calculate the spatially guided temporal attention output; Using the temporal embedding vector of each device node as the query and the spatial embedding vector as the key, calculate the temporally guided spatial attention output; Configure device type-oriented learnable gating vectors to adaptively determine the fusion weights of the temporal attention output and the spatial attention output; The fusion weights are dynamically adjusted based on the functional affiliation of each device node in the device dependency graph. Specifically, the fusion weights of the temporal attention output are increased for device nodes in the subgraph, and the fusion weights of the spatial attention output are increased for device nodes in the main graph. Based on the adjusted fusion weights, the temporal attention output and the spatial attention output are weighted and summed to obtain the fusion embedding vector of each device node.

6. The end-to-end sensing method for predicting conference equipment faults as described in claim 1, characterized in that, The pre-construction of the link failure coding model includes: Obtain normal operation data of the conference system, and based on the normal operation data, establish a contrastive learning task and a mask prediction task, and perform self-supervised pre-training; Using modality as the matching target, an industrial equipment fault dataset is obtained, and cross-domain transfer learning is performed based on the industrial equipment fault dataset through a gradient inversion coding layer. The conference equipment failure samples of the conference system are obtained, and combined with the conference equipment failure samples and the synthetic failures based on the physical degradation model, a few-sample fine-tuning is performed to obtain the link failure coding model based on graph attention network and causal temporal convolutional network. The training of the link fault coding model employs a multi-task joint loss function, which includes at least: a weighted binary cross-entropy loss for fault probability prediction, a negative log-likelihood loss for remaining useful life regression, a logical consistency constraint loss between fault probability and remaining useful life, and a hierarchical consistency constraint loss between the fault prediction results of the main graph node and the associated subgraph node.

7. The end-to-end sensing method for predicting conference equipment faults as described in claim 1, characterized in that, Based on the fused embedding vector, the failure probability and remaining lifespan of each device node are predicted using the corresponding multi-head task predictor, and the warning level of each device node is determined accordingly, including: The fused embedding vector of each device node is input into the corresponding fault probability prediction head in the multi-head task predictor to obtain the fault probability of each device node at multiple future time steps. The fused embedding vector of each device node is input into the corresponding remaining lifetime regression head in the multi-head task predictor to obtain the remaining lifetime and uncertainty standard deviation of each device node, and to calculate the associated confidence interval. Based on the centrality index and current usage status of each device node in the device dependency graph, calculate the importance score of each device node. Based on the failure probability, remaining service life, and importance score, the early warning level of each device node is divided into four levels: emergency, warning, alert, and healthy. The emergency level triggers automatic switching to backup equipment and generation of emergency maintenance work orders; the warning level triggers pre-locking of backup equipment and inclusion in the maintenance plan; the alert level triggers increased monitoring frequency; and the healthy level maintains regular monitoring.

8. A fault prediction system for conference equipment with end-to-end awareness, characterized in that, For implementing the end-to-end sensing method for predicting conference equipment faults according to any one of claims 1 to 7, the system comprises: The graph construction module is used to obtain the physical connection relationships and data flow dependencies between device nodes in the conference system, and construct a device dependency graph accordingly. The device dependency graph includes a main graph and at least two subgraphs. The perception and feature extraction module is used to collect multi-dimensional operating status time-series data of each device node in real time and extract degradation feature sets. The fusion embedding coding module is used to perform fusion embedding coding on each device node based on the device dependency graph and the degradation feature set, combined with a pre-built link fault coding model, to obtain the fusion embedding vector of each device node. The link fault coding model is constructed based on a graph attention network and a causal temporal convolutional network. The prediction and early warning module is used to predict the failure probability and remaining service life of each device node based on the fused embedding vector and through the corresponding multi-head task predictor, and to determine the early warning level of each device node accordingly.

Images