A multimodal monitoring system and method for edge computing equipment in the field of intelligence

Abstract

The present invention discloses a multimodal monitoring system and method for edge computing devices in the intelligent field, relating to the field of intelligent monitoring. The system comprises: a multimodal sensor array for real-time acquisition of multidimensional parameter data representing the operating status of the device; an LSTM prediction model module comprising an LSTM neural network model trained based on historical operating data, which outputs predicted operating state vectors for multiple future time nodes; a state matching and anomaly determination module, when a future time node arrives, obtains the actual operating state vector at that moment, calculates the overall matching degree between the actual operating state vector and the predicted operating state vector, and determines the operating state; and an anomaly tracing and correction module, which screens abnormal operating parameters by calculating the independent matching degrees of various operating parameters and obtains the cause of abnormal operation traceability. The present invention has the advantages of accurately warning of potential faults and performing rapid anomaly tracing and correction by combining the LSTM prediction model with the anomaly detection mechanism, thereby improving system stability and operation and maintenance efficiency.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring, and in particular to a multimodal monitoring system and method for edge computing equipment in the intelligent field. Background Art

[0002] With the development of the Internet of Things, artificial intelligence, and big data technologies, edge computing has become a crucial component of modern intelligent systems. Against this backdrop, multimodal monitoring systems for edge computing devices have emerged. Combining edge computing with multimodal data acquisition technologies, these systems can monitor and analyze data from various sensors in real time. This multimodal fusion monitoring approach enables more accurate early warning and intervention, elevating the system's intelligence level. This provides reliable data support and decision-making for various intelligent applications, driving the further development of smart cities, the Industrial Internet, and other fields.

[0003] Current multimodal monitoring systems for intelligent edge computing devices in the market have limitations in data collection and processing. Many existing systems rely on single-modality sensors, failing to fully cover the multi-dimensional operational status of devices, resulting in inaccurate and incomplete monitoring data. Furthermore, these systems rely heavily on cloud computing for data preprocessing, lacking the real-time processing capabilities of edge computing. This can lead to data transmission delays and inefficient processing. Even some systems employ machine learning models, most rely solely on traditional regression or simple neural network predictions, lacking advanced models like LSTM that can capture time series dependencies. This results in low accuracy in predicting future states. Anomaly detection modules often use static thresholding methods and lack adaptive adjustment mechanisms, which can easily lead to misjudgments or missed detections. Summary of the Invention

[0004] In order to improve the existing systems and methods, a multimodal monitoring system and method for edge computing devices in the intelligent field is provided. This method collects and processes multi-dimensional data in real time, combines the LSTM prediction model and anomaly detection mechanism, can accurately warn of potential faults and quickly trace and correct anomalies, thereby improving system stability and operation and maintenance efficiency.

[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0006] A multimodal monitoring system for edge computing devices in the intelligent field, comprising:

[0007] Multimodal sensor array: This is deployed inside edge computing devices and on key components to collect multi-dimensional parameter data representing their operating status in real time. Key components include the CPU, GPU, battery, communication, and storage components.

[0008] Edge data preprocessing module: The edge data preprocessing module is connected to the multimodal sensor array and deployed locally on the edge computing device. It is used to perform real-time cleaning, denoising, and normalization on the collected raw multimodal data, and aggregate it according to a preset time window to generate a structured time series data stream;

[0009] LSTM prediction model module: The LSTM prediction model module is deployed locally on the edge computing device and includes an LSTM neural network model trained based on historical operating data. The model receives the structured time series data stream output by the edge data preprocessing module as input and outputs predicted operating state vectors for multiple future time nodes. The predicted operating state vectors include predicted values ​​for various operating parameters at future time nodes.

[0010] State matching and anomaly determination module: This module is connected to the LSTM prediction model module. When a future time node arrives, it obtains the actual operating state vector at that moment, calculates the overall matching degree between it and the predicted operating state vector, and determines the operating state of the edge computing device.

[0011] Abnormal tracing and correction module: The abnormal tracing and correction module responds to the abnormal analysis triggered by the state matching and abnormal judgment module, screens abnormal operation parameters by calculating the independent matching degree of each operation parameter, and obtains the cause of abnormal operation.

[0012] Preferably, the multimodal sensor array specifically includes:

[0013] Thermal environment unit: The thermal environment unit is deployed at the air inlet and outlet of the cabinet, attached to the chip heat sink, and monitors and collects the ambient temperature and temperature parameters of key chips and modules in real time;

[0014] Energy supply unit: The energy supply unit is deployed in the main power supply line and each module to monitor and collect the overall power consumption and the power consumption of each module in real time;

[0015] Computational load unit: The computational load unit is integrated into the edge computing motherboard to monitor and collect CPU, GPU, NPU utilization, memory pressure, and process status in real time.

[0016] Network unit: The network unit is connected to the storage controller to monitor and collect disk read and write rates in real time.

[0017] Preferably, the LSTM prediction model module specifically includes:

[0018] Input coding unit: The input coding unit performs data fusion on the structured time series data stream from the preprocessing module;

[0019] Multi-branch LSTM unit: The multi-branch LSTM unit independently learns intra-modal temporal dependencies through the hidden layers of parallel LSTM branches to perform dynamic modeling of heterogeneous features;

[0020] Feature fusion unit: The feature fusion unit splices each branch and performs weighted fusion of key features through a fully connected layer to generate a feature vector;

[0021] Prediction output unit: The prediction output unit outputs the predicted operating state vectors of multiple future time nodes, and the predicted operating state vectors include the predicted values ​​of various operating parameters of the future time nodes.

[0022] Preferably, the state matching and abnormality determination module specifically includes:

[0023] Spatiotemporal alignment unit: The spatiotemporal alignment unit aligns the actual running state vector and the predicted running state vector based on their timestamps, and compensates for missing data through spline interpolation.

[0024] Matching unit: The matching unit calculates the matching rate of the actual running state vector and the predicted running state vector at the same timestamp using distance-based metrics, including Euclidean distance, Manhattan distance, and cosine similarity. The final matching rate is obtained by weighted fusion of the three matching rates.

[0025] Adaptive threshold unit: The adaptive threshold unit dynamically adjusts the threshold based on historical matching degree distribution and real-time operating status data of computing devices;

[0026] Abnormal decision unit: The abnormal decision unit performs multi-level abnormality judgment and classification based on the relationship between the matching rate and the threshold.

[0027] Preferably, the anomaly tracing and correction module specifically includes:

[0028] Abnormal parameter screening unit: The abnormal parameter screening unit performs independent matching calculation on each parameter, screens out abnormal parameters with independent matching degrees lower than a threshold, and generates an abnormal parameter list;

[0029] Correlation analysis unit: The correlation analysis unit constructs an abnormal propagation graph and obtains the abnormal impact path by analyzing the correlation between the abnormal parameter and other parameters;

[0030] Causal reasoning unit: The causal reasoning unit analyzes real-time data and historical abnormal data to obtain possible failure modes and determines the abnormal cause by calculating the posterior probability of each candidate abnormal cause;

[0031] Correction unit: The correction unit generates a correction strategy based on the cause of the abnormality, including automatic and manual intervention measures.

[0032] Based on the same inventive concept, a multimodal monitoring method for edge computing devices in the intelligent field is also proposed, which adopts the multimodal monitoring system for edge smart stations in the intelligent field, including:

[0033] The multimodal sensor array collects various operating parameters of edge computing devices in real time, and pre-processes the collected data in real time on the edge side to generate a structured time series data stream;

[0034] Input the preprocessed time series data stream into the trained LSTM prediction model to obtain the predicted operating state vector at the preset time node in the future;

[0035] When a future time node arrives, the actual operating state vector at that moment is obtained, the matching ratio between the predicted state vector and the actual state vector is calculated, and a threshold is set to determine whether the operating state of the computing device is abnormal.

[0036] Based on the abnormal situation, the independent matching rate of each parameter is calculated and compared with the matching rate threshold of each parameter to obtain the abnormal parameter;

[0037] Trace the cause based on abnormal parameters, obtain the cause of abnormal operation, and make corrections.

[0038] Compared with the prior art, the advantages of the present invention are:

[0039] Real-time data collection via a multimodal sensor array accurately characterizes all equipment operating parameters, ensuring the comprehensiveness and accuracy of monitoring data. The edge data preprocessing module performs local, real-time data cleaning, denoising, and normalization to ensure input data quality and improve prediction accuracy. The introduction of an LSTM prediction model, trained on historical operating data, accurately predicts future operating states, providing early warning of potential anomalies and reducing the risk of system failure. The state matching and anomaly determination module compares predicted and actual states, combining dynamically adjusted adaptive thresholds to ensure high accuracy in anomaly detection. Furthermore, the anomaly tracing and correction module uses causal reasoning analysis to quickly track the root cause of anomalies and promptly restore normal system operation through automatic or manual corrective measures. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the system proposed by the present invention.

[0041] Figure 2 Schematic diagram of the method proposed in the present invention.

[0042] Figure 3 Schematic diagram of the multimodal sensor array proposed in this invention.

[0043] Figure 4Schematic diagram of the LSTM prediction model module proposed in this invention.

[0044] Figure 5 This is a schematic diagram of the state matching and abnormality determination module proposed in the present invention.

[0045] Figure 6 This is a schematic diagram of the anomaly tracing and correction module proposed in the present invention. DETAILED DESCRIPTION

[0046] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0047] See Figure 1 As shown, a multimodal monitoring system for edge computing devices in the intelligent field includes:

[0048] Multimodal sensor array: This is deployed inside edge computing devices and on key components to collect multi-dimensional parameter data representing their operating status in real time. Key components include the CPU, GPU, battery, communication, and storage components.

[0049] Edge data preprocessing module: The edge data preprocessing module is connected to the multimodal sensor array and deployed locally on the edge computing device. It is used to perform real-time cleaning, denoising, and normalization on the collected raw multimodal data, and aggregate it according to a preset time window to generate a structured time series data stream;

[0050] LSTM prediction model module: The LSTM prediction model module is deployed locally on the edge computing device and includes an LSTM neural network model trained based on historical operating data. The model receives the structured time series data stream output by the edge data preprocessing module as input and outputs predicted operating state vectors for multiple future time nodes. The predicted operating state vectors include predicted values ​​for various operating parameters at future time nodes.

[0051] State matching and anomaly determination module: This module is connected to the LSTM prediction model module. When a future time node arrives, it obtains the actual operating state vector at that moment, calculates the overall matching degree between it and the predicted operating state vector, and determines the operating state of the edge computing device.

[0052] Abnormal tracing and correction module: The abnormal tracing and correction module responds to the abnormal analysis triggered by the state matching and abnormal judgment module, screens abnormal operation parameters by calculating the independent matching degree of each operation parameter, and obtains the cause of abnormal operation.

[0053] Specifically, edge computing devices refer to intelligent computing platforms deployed at the edge of the network, that is, close to the source of data generation. They have built-in hardware resources such as processors, memory, and storage, and can process, analyze, and execute computing tasks locally without uploading all data to the cloud.

[0054] See Figure 3 As shown, the multimodal sensor array specifically includes:

[0055] Thermal environment unit: The thermal environment unit is deployed at the air inlet and outlet of the cabinet, attached to the chip heat sink, and monitors and collects the ambient temperature and temperature parameters of key chips and modules in real time;

[0056] Energy supply unit: The energy supply unit is deployed in the main power supply line and each module to monitor and collect the overall power consumption and the power consumption of each module in real time;

[0057] Computational load unit: The computational load unit is integrated into the edge computing motherboard to monitor and collect CPU, GPU, NPU utilization, memory pressure, and process status in real time.

[0058] Network unit: The network unit is connected to the storage controller to monitor and collect disk read and write rates in real time.

[0059] Specifically, thermal environment units are deployed at the cabinet's air inlet and outlet, as well as on key chip heat sinks, to monitor ambient temperature, chip temperature, and module temperature parameters in real time, including average temperature, thermal gradient, and overtemperature risk index. These parameters are used to assess the system's heat distribution and heat dissipation efficiency, preventing performance degradation or hardware damage caused by overheating.

[0060] The energy supply unit is deployed at the cabinet's air inlet and outlet, as well as on key chip heat sinks, to monitor ambient temperature, chip temperature, and module temperature parameters in real time. These parameters are used to assess the system's heat distribution and heat dissipation efficiency, preventing performance degradation or hardware damage caused by overheating.

[0061] The computing load unit is integrated into the edge computing motherboard and monitors the CPU, GPU, and NPU utilization, memory pressure, and process status in real time. These parameters are used to assess the system computing load and optimize task scheduling and resource allocation.

[0062] The NU connects to the storage controller (such as SATA or NVMe controller) and monitors the disk read and write rates in real time. These parameters are used to evaluate the performance of the storage subsystem and identify I / O bottlenecks.

[0063] See Figure 4 As shown in the figure, the LSTM prediction model module specifically includes:

[0064] Input coding unit: The input coding unit performs data fusion on the structured time series data stream from the preprocessing module;

[0065] Multi-branch LSTM unit: The multi-branch LSTM unit independently learns intra-modal temporal dependencies through the hidden layers of parallel LSTM branches to perform dynamic modeling of heterogeneous features;

[0066] Feature fusion unit: The feature fusion unit splices each branch and performs weighted fusion of key features through a fully connected layer to generate a feature vector;

[0067] Prediction output unit: The prediction output unit outputs the predicted operating state vectors of multiple future time nodes, and the predicted operating state vectors include the predicted values ​​of various operating parameters of the future time nodes.

[0068] Specifically, the input encoding unit is the entry point for multimodal data processing, responsible for receiving structured time-series data streams from the thermal environment, energy supply, computing load, and network units, and realizing heterogeneous data fusion through three-layer processing; the time alignment layer uses an adaptive sliding window mechanism to unify data of different frequencies to the reference time axis; the feature projection layer uses a fully connected network with shared weights for dimensionality unification; and the modal embedding layer generates feature vectors with modal identification;

[0069] The multi-branch LSTM unit adopts a heterogeneous LSTM branch structure, with each branch containing three special mechanisms. Modal adaptive gating adds a modal feature gate to the standard LSTM to adjust the intensity of cell state updates and enhance modality-specific learning. The hierarchical memory mechanism divides cell states into two components, fast and slow, to capture long-term trends and short-term fluctuations, respectively. Random state sampling introduces state space perturbations during training to enhance robustness.

[0070] Finally, the predicted operating state vector is output through the prediction output unit.

[0071] See Figure 5 As shown, the state matching and abnormality determination module specifically includes:

[0072] Spatiotemporal alignment unit: The spatiotemporal alignment unit aligns the actual running state vector and the predicted running state vector based on their timestamps, and compensates for missing data through spline interpolation.

[0073] Matching unit: The matching unit calculates the matching rate of the actual running state vector and the predicted running state vector at the same timestamp using distance-based metrics, including Euclidean distance, Manhattan distance, and cosine similarity. The final matching rate is obtained by weighted fusion of the three matching rates.

[0074] Adaptive threshold unit: The adaptive threshold unit dynamically adjusts the threshold based on historical matching degree distribution and real-time operating status data of computing devices;

[0075] Abnormal decision unit: The abnormal decision unit performs multi-level abnormality judgment and classification based on the relationship between the matching rate and the threshold.

[0076] Specifically, the spatiotemporal alignment unit is responsible for synchronizing the actual operating state vector (derived from real-time sensor data) with the predicted operating state vector (derived from the LSTM prediction module), ensuring that both are compared on the same time basis. Because sensor data may be delayed, lost, or have different sampling rates, a dynamic time warping algorithm is used to align unevenly sampled time series. Missing data is filled using cubic spline interpolation, ensuring that the interpolated curve is smooth and conforms to physical laws.

[0077] The matching unit calculates the matching degree between the actual running state vector and the predicted running state vector, using a multi-dimensional similarity metric to avoid single indicator deviation, including:

[0078] Euclidean distance measures the absolute difference between vectors;

[0079] Manhattan distance is more robust to outliers;

[0080] Cosine similarity measures the consistency of vector directions;

[0081] The final matching rate is calculated by combining the three metrics, and the weights are dynamically adjusted based on demand;

[0082] The adaptive threshold unit fits the historical matching rate through Gaussian distribution and adjusts the threshold based on the 3σ principle and real-time load conditions;

[0083] The abnormality decision unit performs multi-level abnormality judgment based on the relationship between the matching rate and the dynamic threshold, including: minor abnormalities only triggering log records without alarms; recoverable abnormalities only triggering automatic recovery strategies (such as degraded operation); serious faults triggering emergency shutdown alarms.

[0084] See Figure 6 As shown in the figure, the abnormality tracing and correction module specifically includes:

[0085] Abnormal parameter screening unit: The abnormal parameter screening unit performs independent matching calculation on each parameter, screens out abnormal parameters with independent matching degrees lower than a threshold, and generates an abnormal parameter list;

[0086] Correlation analysis unit: The correlation analysis unit constructs an abnormal propagation graph and obtains the abnormal impact path by analyzing the correlation between the abnormal parameter and other parameters;

[0087] Causal reasoning unit: The causal reasoning unit analyzes real-time data and historical abnormal data to obtain possible failure modes and determines the abnormal cause by calculating the posterior probability of each candidate abnormal cause;

[0088] Correction unit: The correction unit generates a correction strategy based on the cause of the abnormality, including automatic and manual intervention measures.

[0089] Specifically, the abnormal parameter screening unit accurately locates abnormal parameters from multimodal sensor data to avoid global misjudgment. It adopts a hierarchical screening strategy, calculates the matching degree for each operating parameter separately, and sets adaptive thresholds based on the historical volatility of the parameters to generate an abnormal parameter list.

[0090] The correlation analysis unit constructs an anomaly propagation network and locates the root cause by analyzing the physical and logical correlations between parameters. The physical correlation constructs a static correlation matrix based on the system architecture diagram and calculates the Granger causality index of historical data. A larger value indicates stronger causality.

[0091] By generating a directed weighted graph with abnormal parameter sets as nodes and physical associations as edges, and attaching weights to edges based on the causality of the associations, an improved Dijkstra algorithm is used to find the highest weight propagation path;

[0092] The causal reasoning unit combines real-time data with a historical fault database to determine the most likely fault mode through probabilistic reasoning. Historical fault cases are represented as multi-tuples. The unit calculates the posterior probabilities of the current anomaly and each fault mode, and uses multi-dimensional Gaussian distribution modeling to output a list of candidate root causes. These lists are sorted in descending order of posterior probability to identify the anomaly cause with the highest probability.

[0093] The correction unit generates a hierarchical correction strategy based on the abnormality type and system status, and adopts PID control for controllable parameters.

[0094] See Figure 2 As shown, a multimodal monitoring method for edge computing devices in the intelligent field adopts the above-mentioned multimodal monitoring system for edge smart stations in the intelligent field, including:

[0095] The multimodal sensor array collects various operating parameters of edge computing devices in real time, and pre-processes the collected data in real time on the edge side to generate a structured time series data stream;

[0096] Input the preprocessed time series data stream into the trained LSTM prediction model to obtain the predicted operating state vector at the preset time node in the future;

[0097] When a future time node arrives, the actual operating state vector at that moment is obtained, the matching ratio between the predicted state vector and the actual state vector is calculated, and a threshold is set to determine whether the operating state of the computing device is abnormal.

[0098] Based on the abnormal situation, the independent matching rate of each parameter is calculated and compared with the matching rate threshold of each parameter to obtain the abnormal parameter;

[0099] Trace the cause based on abnormal parameters, obtain the cause of abnormal operation, and make corrections.

[0100] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimodal monitoring system for edge computing devices in the field of intelligence, characterized by: include: Multimodal sensor array: This is deployed inside edge computing devices and on key components to collect multi-dimensional parameter data representing their operating status in real time. Key components include the CPU, GPU, battery, communication, and storage components. Edge data preprocessing module: The edge data preprocessing module is connected to the multimodal sensor array and deployed locally on the edge computing device. It is used to perform real-time cleaning, denoising, and normalization on the collected raw multimodal data, and aggregate it according to a preset time window to generate a structured time series data stream; LSTM prediction model module: The LSTM prediction model module is deployed locally on the edge computing device and includes an LSTM neural network model trained based on historical operating data. The model receives the structured time series data stream output by the edge data preprocessing module as input and outputs predicted operating state vectors for multiple future time nodes. The predicted operating state vectors include predicted values ​​for various operating parameters at future time nodes. State matching and anomaly determination module: This module is connected to the LSTM prediction model module. When a future time node arrives, it obtains the actual operating state vector at that moment, calculates the overall matching degree between it and the predicted operating state vector, and determines the operating state of the edge computing device. Abnormal tracing and correction module: In response to the abnormal analysis triggered by the state matching and abnormality determination module, the abnormal tracing and correction module screens abnormal operation parameters by calculating the independent matching degree of each operation parameter to obtain the cause of the abnormal operation; The LSTM prediction model module specifically includes: Input coding unit: The input coding unit performs data fusion on the structured time series data stream from the preprocessing module; Multi-branch LSTM unit: The multi-branch LSTM unit independently learns intra-modal temporal dependencies through the hidden layers of parallel LSTM branches to perform dynamic modeling of heterogeneous features; Feature fusion unit: The feature fusion unit splices each branch and performs weighted fusion of key features through a fully connected layer to generate a feature vector; Prediction output unit: The prediction output unit outputs the predicted operating state vectors of multiple future time nodes, and the predicted operating state vectors include the predicted values ​​of various operating parameters of the future time nodes.

2. The multimodal monitoring system for edge computing devices in the field of intelligence according to claim 1, characterized in that: The multimodal sensor array specifically includes: Thermal environment unit: The thermal environment unit is deployed at the air inlet and outlet of the cabinet, attached to the chip heat sink, and monitors and collects the ambient temperature and temperature parameters of key chips and modules in real time; Energy supply unit: The energy supply unit is deployed in the main power supply line and each module to monitor and collect the overall power consumption and the power consumption of each module in real time; Computational load unit: The computational load unit is integrated into the edge computing motherboard to monitor and collect CPU, GPU, NPU utilization, memory pressure, and process status in real time. Network unit: The network unit is connected to the storage controller to monitor and collect disk read and write rates in real time.

3. The multimodal monitoring system for edge computing devices in the intelligent field according to claim 1, characterized in that: The state matching and abnormality determination module specifically includes: Spatiotemporal alignment unit: The spatiotemporal alignment unit aligns the actual running state vector and the predicted running state vector based on their timestamps, and compensates for missing data through spline interpolation. Matching unit: The matching unit calculates the matching rate of the actual running state vector and the predicted running state vector at the same timestamp using distance-based metrics, including Euclidean distance, Manhattan distance, and cosine similarity. The final matching rate is obtained by weighted fusion of the three matching rates. Adaptive threshold unit: The adaptive threshold unit dynamically adjusts the threshold based on historical matching degree distribution and real-time operating status data of computing devices; Abnormal decision unit: The abnormal decision unit performs multi-level abnormality judgment and classification based on the relationship between the matching rate and the threshold.

4. The multimodal monitoring system for edge computing devices in the intelligent field according to claim 1, characterized in that: The abnormality tracing and correction module specifically includes: Abnormal parameter screening unit: The abnormal parameter screening unit performs independent matching calculation on each parameter, screens out abnormal parameters with independent matching degrees lower than a threshold, and generates an abnormal parameter list; Correlation analysis unit: The correlation analysis unit constructs an abnormal propagation graph and obtains the abnormal impact path by analyzing the correlation between the abnormal parameter and other parameters; Causal reasoning unit: The causal reasoning unit analyzes real-time data and historical abnormal data to obtain possible failure modes and determines the abnormal cause by calculating the posterior probability of each candidate abnormal cause; Correction unit: The correction unit generates a correction strategy based on the cause of the abnormality, including automatic and manual intervention measures.

5. A multimodal monitoring method for edge computing devices in the intelligent field, using the multimodal monitoring system for edge computing devices in the intelligent field as described in any one of claims 1 to 4, characterized in that: include: The multimodal sensor array collects various operating parameters of edge computing devices in real time, and pre-processes the collected data in real time on the edge side to generate a structured time series data stream; Input the preprocessed time series data stream into the trained LSTM prediction model to obtain the predicted operating state vector at the preset time node in the future; When the future time node arrives, the actual operating state vector at this moment is obtained, the matching rate between the predicted state vector and the actual state vector is calculated, and the operating state of the computing device is judged to be abnormal by setting a threshold; Based on the abnormal situation, the independent matching rate of each parameter is calculated and compared with the matching rate threshold of each parameter to obtain the abnormal parameter; Trace the cause based on abnormal parameters, obtain the cause of abnormal operation, and make corrections.

Images