Container state intelligent prediction method and system based on time sequence multi-modal feature fusion

Abstract

The invention discloses a container state intelligent prediction method and system based on time sequence multi-modal feature fusion, and relates to the field of artificial intelligence. According to the scheme, a visual image sequence and sensor data of a container are collected, and time sequence alignment is carried out; then extracting the structure, texture and time-frequency features of the data, and obtaining modal features with spatio-temporal contexts through a time sequence convolutional network and a double-flow Transform; bidirectional correlation enhancement is performed on the vision and sensing features through a feature cross attention fusion module, and fusion joint feature representation is output; on the basis, a causal graph model is constructed, causal effect estimation and anti-fact reasoning are carried out, and an abnormal cause interpretation report is generated; and finally, inputting the fusion features and the causal information into a multi-task prediction network of a shared trunk, and outputting multi-level alarm and priority operation and maintenance suggestions. According to the method, unified modeling, abnormal cause interpretability and prediction decision linkage of multi-modal data are realized, and the safety and the operation and maintenance intelligence level of the container transportation process are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method and system for intelligent prediction of container status based on temporal multimodal feature fusion. Background Technology

[0002] As a core carrier of global logistics, the condition of containers during transportation directly affects cargo safety and transportation efficiency. Currently, monitoring the condition of containers faces the following technical bottlenecks: Passive and reactive: Traditional methods mainly rely on visual inspection by station personnel or opening and inspecting the goods upon arrival at the destination, which cannot provide early warning and intervention for damage, abnormal temperature, or deviation from the route during transportation.

[0003] Limited monitoring dimensions: Existing technologies mostly use independent sensors (such as temperature recorders) or video monitoring, resulting in isolated data that cannot form a comprehensive judgment. For example, temperature data alone cannot distinguish whether it is a sensor malfunction or a genuine abnormality in the refrigeration system; images alone cannot detect changes in the temperature inside the chamber.

[0004] Lack of predictive capability: Most existing systems can only report current or past anomalies, lacking the ability to predict potential risks based on historical and real-time data, which prevents managers from taking proactive measures to avoid losses.

[0005] Therefore, there is an urgent need for a container status monitoring solution that can integrate multi-dimensional information and has proactive predictive capabilities. Summary of the Invention

[0006] The purpose of this invention is to address the lack of proactive measures for container safety by proposing an intelligent prediction method and system for container status based on temporal multimodal feature fusion.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a container state intelligent prediction method based on temporal multimodal feature fusion, comprising the following steps: Collect container status data and perform unified time-series calibration on the data; Input collected data and preprocess it, then output a multimodal data stream; Extract feature data from multimodal data streams and capture short-term and long-term dependencies of feature data to obtain modal features with spatiotemporal context; The time-encoded visual features and sensor features are subjected to feature cross-attention fusion processing. The association between visual query and sensor query is enhanced through a bidirectional attention mechanism, and the fused joint feature representation is output. Based on the fused joint feature representation, a candidate causal edge set is constructed. A causal graph model is generated by using a time-series causal discovery algorithm and structural constraint optimization learning. Marginal causal effects and counterfactual inferences are calculated through dual machine learning methods to form an anomaly cause explanation report. The multi-task prediction network integrates joint features and causal information into a shared backbone, while simultaneously outputting prediction results. Based on the prediction results and the set threshold, alarm judgments are executed, and priority operation and maintenance suggestions and intervention measures are generated through the expected benefit model.

[0008] As a further description of the above technical solution: the container status data includes the container's image sequence, three-axis acceleration, temperature, humidity and location information, and the timing calibration is a unified timing calibration of visual frames and sensor data by the Network Time Synchronization (NTP) mechanism.

[0009] As a further description of the above technical solution: the preprocessing steps for the collected data include: Perform box target detection, denoising, and enhancement on the image sequence; Perform outlier detection, interpolation, and smoothing on the sensor data; The Dynamic Time Warping (DTW) algorithm is used to achieve cross-modal timing alignment and output a multimodal data stream with consistent timing.

[0010] As a further description of the above technical solution: the feature data extraction of the multimodal data stream includes the following methods: Deep convolutional neural network models are used to extract structural and texture features from image data. Short-time Fourier Transform (FFT) and Long Short-term Memory (LSTM) networks are used to extract the time-frequency features and hidden state features of vibration and temperature-humidity sequences, respectively, and output feature set data. Position encoding is applied to the feature set data and input into a temporal convolutional network (TCN) and a two-stream Transformer structure to capture short-term and long-term dependencies and obtain modal features with spatiotemporal context.

[0011] As a further description of the above technical solution: In the temporal encoding step, a sliding time window is set and a learnable positional encoding is introduced to maintain temporal information. Short-term dependencies are modeled by a temporal convolutional network, and long-term dependencies are modeled by a Transformer self-attention layer.

[0012] As a further description of the above technical solution: the feature cross-attention fusion module includes a multi-head cross-attention layer, a residual connection layer and a layer normalization layer. By enhancing the attention of visual features to vibration events and giving attention weight to temperature data to visual anomalies, cross-modal temporal correlation learning is achieved.

[0013] As a further description of the above technical solution: the construction of the candidate causal edge set adopts the structural causal model learning method based on time lag constraint (NOTEARS temporal extension) to optimize the causal edges, and generates post-intervention result predictions through counterfactual simulation to form an interpretable report of abnormal causes and intervention recommendations.

[0014] As a further description of the above technical solution: the multi-task prediction network with shared backbone is a combination of Transformer and convolutional network. Each task output head is jointly trained using an independent loss function, and the confidence interval is output through the MCDropout method. The prediction results include the probability of container damage risk, temperature control trend and path deviation.

[0015] As a further description of the above technical solution: the set threshold of the prediction result is used to perform alarm judgment. The damage risk score threshold T1, the temperature control trend threshold T2 and the path deviation index T3 are used to perform multi-level alarm judgment. The intervention measures are sorted and priority operation and maintenance suggestions and intervention measures are output in combination with the expected return model.

[0016] As a further description of the above technical solution: The container state intelligent prediction system based on temporal multimodal feature fusion includes a data acquisition and preprocessing module. The data acquisition and preprocessing module outputs a multimodal data stream to a feature cross-attention fusion module. The feature cross-attention fusion module extracts feature data from the multimodal data stream and captures the short-term and long-term dependencies of the feature data. It then performs feature cross-attention fusion processing and outputs a fused joint feature representation. It also includes a multimodal anomaly interpretation module for causal inference, which constructs a candidate causal edge set from the fused joint features and performs counterfactual inference. Then, the fused joint features and causal information are input into the multivariate state prediction module to output priority operation and maintenance suggestions and intervention measures.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: This scheme captures the dynamic correspondence between vibration and damage, and temperature and structural stress by setting up an attention path with vision as the query and sensing as the key. Simultaneously, a reverse attention path is established to correct the visual focus area based on sensor changes, thereby achieving intermodal information exchange and adaptive weight adjustment. This mechanism enables the model to automatically focus on anomaly-related multimodal feature regions in complex transportation environments, effectively reducing noise interference and modal bias. It significantly improves damage prediction accuracy and achieves early warning of temperature control anomalies and structural stress anomalies, achieving cross-modal anomaly identification and prediction effects that traditional single-modal methods cannot achieve. Attached Figure Description

[0018] Figure 1This is the overall flowchart of the present invention; Figure 2 This is a schematic diagram of the temporal multimodal feature fusion module (FCAF) of the present invention; Figure 3 This is a schematic diagram of the architecture of the multi-task prediction network of the present invention; Figure 4 This is a schematic diagram of the hardware deployment of the system of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 - Figure 4 As shown, the intelligent prediction method for container state based on temporal multimodal feature fusion provided by this invention includes the following steps: This invention introduces a temporal multimodal feature fusion mechanism into the container status monitoring process, realizing unified modeling and dynamic correlation of visual, vibration, temperature, humidity and location information. This enables the system to perform real-time and predictive monitoring of container structural status and environmental changes throughout the entire transportation cycle, thereby significantly improving the accuracy and response speed of anomaly identification.

[0021] Data acquisition and preprocessing module S1: Multimodal time series data acquisition and preprocessing The container's triaxial acceleration (vibration), temperature, humidity, and location information are collected by IoT sensing units installed on the container. Image sequences of the container are collected by camera devices deployed on gates, cranes, or transport vehicles, forming raw state data containing visual and sensor information.

[0022] A network time synchronization (NTP) mechanism is used to perform unified temporal calibration of visual frames and sensor data. The visual data is preprocessed by denoising, enhancement and box target detection; the sensor data is processed by outlier detection, interpolation and smoothing; finally, the dynamic time warping (DTW) algorithm is used to achieve cross-modal temporal alignment.

[0023] This step solves the temporal mismatch problem caused by the inconsistency between visual frame rate and sensor sampling rate in existing technologies by unifying the temporal calibration and dynamic time warping of multimodal data. This enables joint modeling of different modal information under the same time reference, laying the foundation for subsequent feature fusion.

[0024] S2: Spatiotemporal Feature Extraction Feature extraction is performed on the multimodal data stream obtained from S1; Image sequences are input into a deep convolutional neural network (CNN) to extract structural and texture features, which are used to characterize signs of damage such as scratches, rust, and deformation. Fast Fourier Transform (FFT) is performed on the vibration sequence in the sensor data to extract the vibration energy spectrum features and obtain its time-frequency features in order to distinguish between normal driving vibration and abnormal impact. Statistical and temporal features, including mean, variance, and slope, are extracted from temperature and humidity data using a Long Short-Term Memory (LSTM) network to capture temperature change trends. Subsequently, positional encoding is applied to the multimodal features and input into a temporal convolutional network (TCN) and a two-stream Transformer structure to model short-term and long-term dependencies respectively, and output modal features with spatiotemporal context semantics.

[0025] This step combines temporal convolutional networks with a two-stream Transformer structure to achieve collaborative modeling of short-term fluctuations and long-term trends. It can simultaneously capture damage signs caused by instantaneous vibration and shock, as well as potential abnormal trends caused by gradual changes in environmental parameters, thus accurately expressing the dynamic evolution characteristics of the container's state.

[0026] Feature Cross-Attention Fusion Module The visual features and sensor features output by S2 are input into the feature cross-attention fusion module (FCAF). This module is based on a bidirectional cross-attention mechanism to achieve interactive enhancement of querying sensor features with vision and querying visual features with sensor. Specifically, it includes: ① When the sensor detects abnormal vibration, FCAF enhances the visual feature weights for the corresponding time period to capture potential damage areas; ② When visual detection detects anomalies in the door or sealing structure, FCAF focuses on the temperature feature changes in the corresponding time period. Through this bidirectional attention mechanism, the model can learn cross-modal correlations that cannot be revealed by a single modality, significantly improving the accuracy and stability of anomaly recognition and outputting a joint feature representation that integrates spatiotemporal context information.

[0027] This step utilizes a feature cross-attention fusion mechanism to achieve correlation focusing of visual features on sensor signals and inverse correction of sensor signals on visual regions. This strengthens the implicit correlations between modalities, effectively reduces misjudgments of single modalities and feature noise interference, thereby maintaining stable recognition performance in complex transportation environments. The Feature Cross-Attention Fusion (FCAF) module model is shown below: Multi-head cross-attention single-head (visual query sensor) in: Multi-head parallel connection and output projection Two-way fusion and residual normalization Time-series event weights (used in bias matrix M) Let T be the temporal feature matrix of the visual modality, with shape T×d. Let be the temporal feature matrix of the sensing mode, with shape T×d.

[0028] Let be the linear projection matrix of the i-th head. The vector dimension of the Key is a scalar.

[0029] This is the temporal bias matrix, used to emphasize time points near the event. h represents the number of attention heads.

[0030] This is the output projection matrix after multi-head splicing. To be The representation after linear projection onto the visual dimension is used to sum the residuals.

[0031] This is a vibrational energy index at time t, such as short-time energy or spectral energy. Here are the sample mean and sample standard deviation of the vibration energy.

[0032] This refers to the Sigmoid function. These are hyperparameters that control the weight slope, bias magnitude, and time window half-width, respectively.

[0033] This is an indicator function; it returns 1 if true. For layer normalization operators.

[0034] softmax(·) is a row-wise softmax operation.

[0035] Multimodal anomaly interpretation module for causal inference This module's function is to learn temporal causal structure, estimate causal effects, and generate interpretability based on fused features, thereby upgrading from "predicting anomalies" to "explaining anomalies." The module's components and workflow are as follows: The module consists of: a causal graph learner, a structural causal model (SCM) builder, a causal effect estimator, an explanation generator, and an online feedback updater.

[0036] S3.1: Using the fusion joint feature representation of S3 as input, construct a time-series causal graph model and perform causal effect estimation and counterfactual inference; The specific process includes: A candidate causal edge set is constructed based on time lag features, and the causal structure is optimized using the temporal structure causal model learning algorithm (NOTEARS temporal extension). (The inputs are the temporal fusion feature sequence output by FCAF, historical maintenance logs, operation and event annotation data, and the outputs are a causal graph, key causal chains, causal contribution scores for each candidate causal path, effect prediction based on hypothetical intervention, and readable explanatory text with credibility scores for operation and maintenance personnel.) The learned causal graph is transformed into a structural causal model, and the marginal causal effects of univariate and multivariate models are estimated. (First, a set of candidate causal edges is constructed based on time lag features. A temporal causal discovery algorithm is used for preliminary structural learning, supplemented by a neural network-based causal discovery method for nonlinear relationship mining. Supervised correction is applied to labeled intervention samples and maintenance records to improve structural accuracy. Temporal causal information is preserved during structural learning to support the modeling of lagged causal edges.) Perform counterfactual reasoning to simulate changes in outcomes under hypothetical interventions, used to assess the preventative effects of potential measures; (transforming the learned causal graph into a structural causal model, employing causal effect estimation algorithms to estimate the marginal causal effects of univariate and multivariate interventions, supporting do-based counterfactual reasoning to assess the effectiveness of hypothetical measures. For example, assessing how much a preventative intervention "immediately after a collision is detected" would reduce the probability of damage.) Based on the causal contribution score, key causal chains are extracted, and a natural language interpretation report is generated to explain the cause of the anomaly, the key evidence time points, and recommended actions, and output a confidence score; (based on the causal contribution score, high-contribution causal chains are extracted, and a natural language interpretation template is generated to explain the possible causes of the anomaly, the key evidence time points, recommended actions and estimated effects, and provide a confidence level. At the same time, the causal chain and key evidence frames are presented in a timeline format on the management interface for auditing purposes.) The cause-effect structure and parameters are updated regularly based on operation and maintenance feedback and repair records to achieve online closed-loop learning.

[0037] At the level of anomaly interpretation, this step combines causal graph modeling with counterfactual reasoning. This solution no longer only outputs anomaly prediction results, but can identify the causal chain of anomalies. For example, it can identify causal relationships such as "excessive vibration leads to structural cracks, which in turn causes abnormal temperature control." This achieves a leap from "result prediction" to "cause interpretation," significantly improving the interpretability and decision support capabilities of the system.

[0038] The causal inference-driven multimodal anomaly interpretation module model is as follows: Causal model with time delay structure (vector form) Causal structure learning objective (with sparse regularization and acyclic constraint) in Acyclic constraint functions can be taken as follows: .

[0039] 3. Dual Machine Learning (DML) Marginal Causal Effect Estimation (Single Treatment Variable Example) The first step is to estimate the conditional expectation: The second step is to construct the residuals and perform regression: 4. Counterfactual prediction (do operation approximation) 5. Path contribution calculation (on path p) If the path consists of a coefficient sequence The composition, then the approximate path influence is: : Multimodal fusion joint feature vector at time t, dimension d. p: Maximum lag order.

[0040] : The coefficient matrix of lag τ, where the element B^{(τ)}_{ij} represents the estimated linear effect of variable i on variable j at lag τ.

[0041] Independent noise vector. Frobenius norm. : The sum of the absolute values ​​of matrix elements, used for sparse regularization. : By absolute value of elements.

[0042] Trace operation. : Matrix index. d: Dimension of the variable. : Regularization weight hyperparameter. Y: Outcome variable sample. D: Treatment variable or candidate cause variable sample.

[0043] Z: Control covariate vector or high-dimensional feature. The conditional expectation function estimated using a machine learning model can be achieved in the first stage using random forest, GBDT, neural network, or other regressors.

[0044] : The marginal causal effect size estimated by DML. Approximate causal contribution value of path p, used for ranking and interpretation.

[0045] Multivariate state prediction module S4: The fusion and joint features of S3 and the causal information of S3.1 are input into a multi-task prediction network with a shared backbone. This network combines Transformer and convolutional structure, and sets up multiple task output heads on the basis of the shared feature extraction layer to output damage risk score, temperature control trend prediction and path deviation index respectively.

[0046] The damage risk score represents the probability of structural damage occurring within a future time window. When the score exceeds the threshold T1, an inspection warning is triggered. The temperature control trend prediction outputs the temperature change trend, and triggers a refrigeration system maintenance suggestion when the predicted over-temperature risk exceeds the threshold T2. The path deviation index determines whether the transportation trajectory deviates from the planned route. When the deviation index exceeds the threshold T3, a path anomaly alarm is issued. Simultaneously, the network outputs the confidence scores of each prediction result and a summarized causal explanation to aid decision-making. At the output level, this step achieves joint prediction of multiple indicators such as damage risk, temperature control trend, and path deviation through a multi-task prediction network structure with a shared backbone. This allows different tasks to share underlying feature information without interfering with each other, improving overall prediction efficiency and generalization ability.

[0047] In the alarm and intervention phase, by introducing multi-level threshold judgment and expected benefit model, the system can make a comprehensive judgment based on damage risk score, temperature control deviation degree and path deviation index, and automatically generate priority-ranked operation and maintenance suggestions and intervention measures, so that the prediction results directly serve decision execution and form a closed loop from detection to disposal.

[0048] In summary, this invention represents a significant improvement over existing technologies in terms of multimodal data fusion accuracy, anomaly prediction interpretability, and prediction-intervention linkage. It enables intelligent prediction and proactive intervention of the container's full lifecycle status, greatly enhancing the safety, controllability, and operational decision-making efficiency of the transportation process.

[0049] The container state intelligent prediction system based on temporal multimodal feature fusion includes a data acquisition and preprocessing module. The data acquisition and preprocessing module outputs a multimodal data stream to a feature cross-attention fusion module. The feature cross-attention fusion module extracts feature data from the multimodal data stream and captures the short-term and long-term dependencies of the feature data. It then performs feature cross-attention fusion processing and outputs a fused joint feature representation. It also includes a multimodal anomaly interpretation module for causal inference, which constructs a candidate causal edge set from the fused joint features and performs counterfactual inference. Then, the fused joint features and causal information are input into the multivariate state prediction module to output priority operation and maintenance suggestions and intervention measures.

[0050] The system acquires visual image sequences and sensor data (vibration, temperature, humidity, and location information) of containers, and achieves temporal alignment through time synchronization calibration and dynamic time warping. It extracts structural, texture, and time-frequency features using deep convolutional networks, fast Fourier transforms, and long short-term memory networks, and obtains modal features with spatiotemporal context through temporal convolutional networks and dual-stream Transformers. A feature cross-attention fusion module enhances the bidirectional association between visual and sensor features, outputting a fused joint feature representation. Based on this, a causal graph model is constructed, and causal effect estimation and counterfactual reasoning are performed to generate an anomaly cause explanation report. Finally, the fused features and causal information are input into a multi-task prediction network sharing a common backbone, outputting a damage risk score, temperature control trend, and path deviation index, and executing multi-level alarms and priority maintenance suggestions based on set thresholds. This method achieves unified modeling of multimodal data, interpretable anomaly causes, and linkage between prediction and decision-making, significantly improving the safety and intelligent operation and maintenance level of container transportation.

[0051] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A container state intelligent prediction method based on temporal multimodal feature fusion, characterized in that, Includes the following steps: Collect container status data and perform unified time-series calibration on the data; Input collected data and preprocess it, then output a multimodal data stream; Extract feature data from multimodal data streams and capture short-term and long-term dependencies of feature data to obtain modal features with spatiotemporal context; The time-encoded visual features and sensor features are subjected to feature cross-attention fusion processing. The association between visual query and sensor query is enhanced through a bidirectional attention mechanism, and the fused joint feature representation is output. Based on the fused joint feature representation, a candidate causal edge set is constructed. A causal graph model is generated by using a time-series causal discovery algorithm and structural constraint optimization learning. Marginal causal effects and counterfactual inferences are calculated through dual machine learning methods to form an anomaly cause explanation report. The multi-task prediction network integrates joint features and causal information into a shared backbone, while simultaneously outputting prediction results. Based on the prediction results and the set threshold, alarm judgments are executed, and priority operation and maintenance suggestions and intervention measures are generated through the expected benefit model.

2. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, The container status data includes the container's image sequence, three-axis acceleration, temperature, humidity, and location information. The timing calibration is performed by using the Network Time Synchronization (NTP) mechanism to uniformly calibrate the visual frames and sensor data.

3. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, The preprocessing steps for the collected data include: Perform box target detection, denoising, and enhancement on the image sequence; Perform outlier detection, interpolation, and smoothing on the sensor data; The Dynamic Time Warping (DTW) algorithm is used to achieve cross-modal timing alignment and output a multimodal data stream with consistent timing.

4. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, The feature data extraction of the multimodal data stream includes the following methods: Deep convolutional neural network models are used to extract structural and texture features from image data. Short-time Fourier Transform (FFT) and Long Short-term Memory (LSTM) networks are used to extract the time-frequency features and hidden state features of vibration and temperature-humidity sequences, respectively, and output feature set data. Position encoding is applied to the feature set data and input into a temporal convolutional network (TCN) and a two-stream Transformer structure to capture short-term and long-term dependencies and obtain modal features with spatiotemporal context.

5. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, In the temporal encoding step, a sliding time window is set and a learnable positional encoding is introduced to preserve temporal information. Short-term dependencies are modeled by a temporal convolutional network, and long-term dependencies are modeled by a Transformer self-attention layer.

6. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, The feature cross-attention fusion module includes a multi-head cross-attention layer, a residual connection layer, and a layer normalization layer. It achieves cross-modal temporal correlation learning by enhancing the attention of visual features to vibration events and the attention weight of temperature data to visual anomalies.

7. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, The construction of the candidate causal edge set adopts a structural causal model learning method based on time lag constraints to optimize the causal edges, and generates post-intervention outcome predictions through counterfactual simulation to form an interpretable report of abnormal causes and intervention recommendations.

8. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 1, characterized in that, The shared backbone multi-task prediction network is a combination of Transformer and convolutional network. Each task output head is jointly trained using an independent loss function, and the confidence interval is output through the MC Dropout method. The prediction results include the probability of container damage risk, temperature control trend, and path deviation.

9. The intelligent prediction method for container state based on temporal multimodal feature fusion according to claim 8, characterized in that, The predicted results are used to set thresholds to trigger alarm judgments. The damage risk score threshold T1, temperature control trend threshold T2, and path deviation index T3 are used to perform multi-level alarm judgments. The intervention measures are then sorted and priority maintenance suggestions and intervention measures are output in combination with the expected return model.

10. A container state intelligent prediction system based on temporal multimodal feature fusion, characterized in that, It includes a data acquisition and preprocessing module, which outputs a multimodal data stream to a feature cross-attention fusion module. The feature cross-attention fusion module extracts feature data from the multimodal data stream, captures short-term and long-term dependencies of the feature data, performs feature cross-attention fusion processing, and outputs a fused joint feature representation. It also includes a multimodal anomaly interpretation module for causal inference, which constructs a candidate causal edge set from the fused joint features and performs counterfactual inference. Then, the fused joint features and causal information are input into the multivariate state prediction module to output priority operation and maintenance suggestions and intervention measures.

Images