Internet of Things equipment real-time early warning method and system based on digital twinning

Abstract

The invention discloses an Internet of Things equipment real-time early warning method and system based on digital twinning, and relates to the technical field of equipment operation management, and the method comprises the steps: collecting multi-modal original data of Internet of Things equipment, constructing a graph structure, and generating a semantic graph vector; obtaining a digital twinborn model state corresponding to the target equipment, forming an extended state vector by the digital twinborn state, the physical equipment state and the semantic map vector, and inputting the extended state vector into an extended Kalman filter for state fusion to obtain a state estimation result; and performing Monte Carlo simulation according to the state estimation result, generating a plurality of abnormal state samples, calculating a residual mean value between the abnormal samples and the current estimation result, correcting a covariance parameter of the extended Kalman filter, and generating an early warning tag. Fine estimation and risk level early warning of the state of the Internet of Things equipment are realized, and the state fusion precision and the abnormal response timeliness are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of equipment operation management, in particular to a real-time warning method and system for Internet of Things equipment based on digital twinning. BACKGROUND

[0002] With the rapid development of Internet of Things technology, the operation state monitoring and remote management capability of industrial equipment is continuously improved, and the digital level of equipment is significantly improved. In practical applications, Internet of Things equipment faces complex and changeable operating environments and highly heterogeneous data sources. How to realize efficient fusion of multi-source data and accurate identification of abnormal states has become an important research direction of intelligent operation and maintenance systems. Existing equipment warning methods mostly rely on static rule judgment or simple time series prediction algorithms, and lack the modeling capability of complex relationships between equipment structure state, environmental factors and behavior semantics, making it difficult to cope with complex scenarios such as nonlinear disturbance, sudden failure or multi-modal data anomaly.

[0003] In recent years, the introduction of digital twinning technology has provided a new way for equipment warning. By constructing a simulation model of equipment operation in a virtual space, the physical equipment state can be deduced and predicted under different working conditions. However, traditional digital twinning methods are mainly used for offline modeling and system-level simulation, and it is still difficult to realize high-frequency interaction and fusion processing with real-time collected data. On the other hand, state estimation methods such as extended Kalman filter have certain dynamic modeling capability, but when faced with multi-modal and unstructured data, their state estimation accuracy and covariance adjustment capability are limited, especially in the process of equipment state mutation or abnormal evolution, the warning sensitivity and robustness are still insufficient. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a real-time warning method for Internet of Things equipment based on digital twinning, which solves the problems of structural separation, large estimation deviation and lack of closed-loop feedback optimization in the multi-modal data fusion capability, state estimation accuracy and abnormal evolution response mechanism of existing Internet of Things equipment warning methods.

[0006] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides a real-time warning method for Internet of Things equipment based on digital twinning, which includes: Collecting multi-modal raw data of Internet of Things equipment, constructing a graph structure, and generating a semantic graph vector; Obtaining a digital twinning model state corresponding to the target equipment, and combining the digital twinning state, the physical equipment state and the semantic graph vector to form an extended state vector, inputting the extended state vector into an extended Kalman filter for state fusion to obtain a state estimation result; According to the state estimation result, a Monte Carlo simulation is performed to generate a plurality of abnormal state samples, a residual mean between the abnormal samples and the current estimation result is calculated, a covariance parameter of an extended Kalman filter is corrected, and a warning label is generated.

[0007] As a preferred scheme of the real-time warning method of the digital twin Internet of Things device, the multi-modal original data includes time series signal data, spatial environment parameters and digital twin model output states. The construction of the graph structure includes: performing feature extraction on the multi-modal original data respectively to obtain time series modal features, spatial modal features and twin modal features; the above modal features are taken as nodes of the graph structure to form a modal feature node set, wherein the initial feature vector of each node is an expression vector generated by frequency domain transformation of the corresponding modal original data; when the cosine similarity between any two modal features exceeds a set threshold, a connection relationship between the nodes is established to form an edge set, and the weight of each edge is set as the mutual information value between the corresponding modal features.

[0008] As a preferred scheme of the real-time warning method of the digital twin Internet of Things device, the generation of the semantic graph vector includes: taking the initial feature vectors of the modal feature node set as input node features, taking the edge set and its edge weights as graph structure inputs, and constructing an input graph of a graph neural network; performing multi-layer graph convolution operations on the input graph in turn, updating the embedding vector representation of the nodes layer by layer, and introducing a cross-modal attention mechanism in the node updating process to adjust the information aggregation weight according to the semantic correlation between different modalities; taking the embedding vector obtained by each node in the last layer of the graph convolution network as the final feature of the node, and aggregating the final features of all nodes by a graph-level global pooling function to obtain a semantic graph vector.

[0009] As a preferred scheme of the real-time warning method of the digital twin Internet of Things device, the composition of the extended state vector includes: normalizing the structure running state vector of the physical device in the current sampling period, the corresponding digital twin model output state vector, and the semantic graph vector output by the graph neural network, and combining them into an extended state vector according to a set splicing order. The physical device structure running state vector is a statistical feature expression including temperature, voltage, current and vibration; the digital twin model output state vector is a state vector generated by a digital twin simulation model corresponding to the target device based on historical running data, current environment parameters and model structure parameters after state evolution calculation in the current sampling period; the simulation state prediction vector includes structure parameters consistent with the state dimensions of the physical device, including simulation pressure value, predicted temperature, fatigue loss rate and load response coefficient.

[0010] As a preferred scheme of the digital twin Internet of Things device real-time warning method, wherein: the state estimation result includes: after inputting the extended state vector into the extended Kalman filter, the state estimation value of the last period is collected as the initial state of the current prediction, the time characteristics and device dynamic characteristics in the extended state vector are weighted and superimposed using a nonlinear combination method to generate the predicted state value of the current period; based on the error covariance matrix of the last period, the covariance estimation value of the current predicted state is calculated, and the preset process noise covariance is added; The feature dimension corresponding to the device observation information in the predicted state value is extracted as the observation mapping result, and each dimension is compared with the actual observation value of the physical device in the current sampling period to obtain the prediction residual of each dimension; according to the proportional relationship between the prediction residual of each dimension and the corresponding covariance estimation value, the weight coefficient for residual correction is calculated, the predicted state value is corrected using a weighted average method, and the state estimation value of the current period is generated, and the corresponding covariance matrix is updated synchronously.

[0011] As a preferred scheme of the digital twin Internet of Things device real-time warning method, wherein: the generation of a plurality of abnormal state samples includes: based on the state estimation value and the covariance matrix of the current period, a multi-dimensional Gaussian distribution model is constructed with the state estimation value as the mean value and the covariance matrix as the covariance; A set number of abnormal state samples are generated from the Gaussian distribution model by a Monte Carlo sampling method; the residual of each abnormal state sample and the corresponding state estimation value is calculated by dimension to obtain a plurality of residual vectors; the residual mean vector is obtained by averaging the residual vectors in each dimension; the residual mean vector is converted into a diagonal matrix form and added to the original covariance matrix by elements to obtain the corrected covariance matrix.

[0012] As a preferred scheme of the digital twin Internet of Things device real-time warning method, wherein: the generation of a plurality of abnormal state samples includes: based on the state estimation value and the covariance matrix of the current period, a multi-dimensional Gaussian distribution model is constructed with the state estimation value as the mean value and the covariance matrix as the covariance;

[0013] In a second aspect, the present application provides a digital-twin-based real-time warning system for an Internet of Things device, comprising: a multi-modal data acquisition module for acquiring time-series signal data, spatial environment parameters, and a digital-twin model output state; a graph structure construction module for extracting multi-modal features, constructing a set of modal feature nodes and edges, and generating a graph structure; a semantic graph generation module for inputting the graph structure into a graph neural network and generating a semantic graph vector; a state fusion module for constructing an extended state vector and inputting the extended state vector into an extended Kalman filter to output a state estimation result; an anomaly simulation module for performing Monte Carlo simulation based on the state estimation result, generating an abnormal state sample, and calculating a residual mean; a warning generation module for generating a warning label according to the residual mean and a risk threshold; a parameter optimization module for updating attention weights and covariance initialization parameters according to the residual mean and the warning label.

[0014] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the digital-twin-based real-time warning for an Internet of Things device according to the first aspect of the present application.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the digital-twin-based real-time warning for an Internet of Things device according to the first aspect of the present application.

[0016] The present application has the following advantages: by fusing time-series signals, spatial environment parameters, and digital-twin states, a graph structure based on modal similarity and mutual information weighting is constructed, improving the representation ability of multi-source features; graph neural networks and cross-modal attention mechanisms are introduced to realize semantic-level information aggregation, improving the discriminability of graph representation; an extended state vector containing physical states, simulation states, and semantic vectors is constructed, and a nonlinear state fusion mechanism is introduced, effectively improving the accuracy and stability of state estimation; Monte Carlo sampling and residual statistical correction covariance are combined to enhance the simulation capability for potential abnormalities of the device; differential response control is performed according to a risk threshold, realizing the linkage and closed loop of warning and control. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Figure 1 Flowchart of the real-time early warning method for the digital twin Internet of Things device. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0020] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0021] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0022] Embodiment 1, reference Figure 1 For one embodiment of the present application, the embodiment provides a real-time early warning method for digital twin Internet of Things device, comprising the following steps: S1: Collecting multi-modal original data of the Internet of Things device, constructing a graph structure, and generating a semantic graph vector.

[0023] The edge acquisition module collects multi-modal original data in the running process of the Internet of Things device with a period of 5ms, specifically including: Time sequence modal signal: Collecting device sensor data stream such as temperature, vibration, pressure, voltage, current, etc., to form a time sequence tensor , wherein is the modal dimension, is the sampling window length.

[0024] Spatial environment modal: Collecting device position, indoor environment parameters and other information to generate a spatial vector , including position coordinates, air temperature, humidity, etc.

[0025] Twin model state mode: call the digital twin model synchronized with the target device to generate the simulation state vector of the current cycle , including predicted pressure, temperature, structural fatigue indicators, etc.

[0026] All raw data are cut and time-aligned by sliding window to form sample packages .

[0027] Among them, represents the time series mode signal tensor collected at time , including continuous sampling values of multiple sensors (such as temperature, vibration, etc.); represents the number of modes, that is, how many different types of sensors there are; represents the sampling window length (in points, such as a 5-second window sampling 1000 times ); represents the spatial environment feature vector at time ; represents the environmental parameter dimension, such as 3D GPS coordinates + room temperature + humidity + air pressure, etc., a total of dimensions; represents the digital twin simulation state vector at time ; represents the number of state variables output by the twin model; represents the original sample set within a time window period, including three types of modal data.

[0028] In each time window, a heterogeneous modal graph is constructed, which specifically includes: Node set : Each modal feature is regarded as an independent node, a total of nodes are constructed.

[0029] For time series nodes: use STFT to extract frequency domain features to generate embedding vectors.

[0030] For spatial nodes: normalize the original environmental quantities into standard vector form.

[0031] For twin nodes: extract key simulation variables as embedding features to ensure alignment with the dimension of physical features.

[0032] Edge set : connection relationships are constructed according to the semantic or structural similarity between modes.

[0033] If the cosine similarity between the features of two nodes exceeds the set threshold, an edge is established.

[0034] The edge weight is jointly weighted by the modal mutual information, physical adjacency and functional correlation to form a sparse and heterogeneous structure graph.

[0035] Feature vector of all graph nodes The initial matrix is formed according to the STFT, normalization and embedding strategy at the time of construction:

[0036] For the first Layer graph convolution, the node representation is updated as follows:

[0037] Wherein, represents the initial feature vector of the th graph node; represents the total number of graph nodes N = the number of time nodes M + the number of spatial nodes K + the number of twin nodes L; represents the dimension of each node feature; represents the initial input feature matrix of the graph neural network, and each row is a node feature; represents the node feature matrix of the th layer; represents the learnable weight matrix of the th layer graph convolution; represents the adjacency matrix after adding the self-loop ( ); : The degree matrix of , the diagonal line is the degree of each node; represents the activation function (such as ReLU).

[0038] For heterogeneous modal nodes, an attention mechanism is added to dynamically adjust the information aggregation weight of different source nodes.

[0039] Construct query vector , key vector :

[0040] Wherein, represents the weight when the node aggregates information from neighbors , which is dynamically calculated according to the modal semantic correlation; and represent the node index; represents the query vector (Query) of the node ; represents the key vector (Key) of the node ; represents the neighbor node set of the node ; denotes the attention space dimension (used for scaling).

[0041] The final node update is:

[0042] The final embedding vectors of all nodes (as in the layer output) are denoted as , and the graph semantic vector is obtained by performing graph-level pooling:

[0043] where is the number of nodes, is the initial feature dimension of each node; denotes the feature vector of node in the th layer; is the graph convolution weight; denotes the final embedding vector of node i in the Lth layer; denotes the semantic graph vector of the entire graph, with a dimension of .

[0044] If the discrimination ability needs to be enhanced, learnable weighted pooling or attention weighted pooling can be used instead of averaging.

[0045] The semantic graph vector represents the global semantic expression of multi-modal features in the graph space within the current time window; it is concatenated with the following two state vectors to form the extended state vector .

[0046] The digital twin state vector comes from the output of the twin model, such as predicted temperature, pressure, and wear rate; The physical state vector is the current structural operating state collected by the device within the sampling period.

[0047] The final extended state vector is:

[0048] where denotes vector concatenation, with the same order as the EKF input structure to ensure dimension correspondence; denotes the extended state vector; denotes the state vector from the twin model, with a dimension of ; denotes the current state vector from the physical sampling data, with a dimension of .

[0049] Existing multi-modal data fusion methods mostly rely on simple feature concatenation or fixed rule mapping, which cannot fully exploit the potential semantic relationship between modalities and lack robust perception ability for complex device state changes. In contrast, by constructing a heterogeneous graph structure to represent the multi-modal raw data of Internet of Things devices, including time series signals, spatial environment parameters, and digital twin simulation states, the modal information is uniformly expressed in the form of nodes in the graph structure. The construction of edges is based on the joint definition of semantic similarity, physical adjacency, and functional association, making the graph structure more context-aware.

[0050] In terms of node feature representation, short-time Fourier transform, standard normalization, and simulation variable embedding methods are introduced respectively to process different modal data, ensuring that the node embedding vector can accurately represent the frequency domain characteristics and structural state of each type of modal information. The construction process automatically selects connections with actual relevance by combining mutual information, spatial proximity, and device structure coupling, generating a sparse but highly expressive topology. This strategy effectively avoids the expression degradation problems caused by dense graph structure or unbalanced edge weight distribution in traditional methods.

[0051] During the graph neural network processing, a multi-layer graph convolution structure is used for aggregation calculation of nodes, and a cross-modal attention mechanism is introduced to dynamically adjust the aggregation weight of neighbor nodes, making the semantic fusion process between different modalities more precise and flexible. Finally, a weighted graph-level pooling strategy is used to output a graph-level representation vector, which can capture the spatio-temporal relationship between cross-modal features and their potential reaction capability to device state, providing a structured and context-related high-dimensional representation for subsequent state estimation, effectively improving the quality of semantic modeling and information fusion accuracy in complex state scenarios.

[0052] S2: Obtain the digital twin model state corresponding to the target device, and compose an extended state vector consisting of the digital twin state, physical device state, and semantic graph vector. Input the extended state vector into an extended Kalman filter for state fusion to obtain the state estimation result.

[0053] Normalize each input vector independently for the current sampling period: Digital twin state vector

[0054] Physical device state vector

[0055] Semantic graph vector

[0056] Concatenate the normalized vectors in the order of "twin-physical-semantic":

[0057] Let the filter output of the previous period be Covariance matrix .

[0058] where, denotes the normalized digital twin state vector with dimension ; denotes the normalized physical device state vector with dimension ; denotes the normalized semantic graph vector with dimension ; denotes the final extended state vector with dimension .

[0059] A linearized state transition matrix is used to perform a linear prediction on the extended state vector:

[0060] According to and the covariance of the last period, the prediction covariance is calculated, and the process noise covariance is superimposed :

[0061] where, denotes the predicted state vector (a priori estimate) at the current time ; denotes the state transition matrix, which is used to linearly map the state of the last period to the current period; denotes the state estimation vector output in the last period; denotes the prediction state covariance matrix of the current period; denotes the state estimation covariance matrix of the last period; denotes the transpose of the state transition matrix; denotes the process noise covariance matrix, which is used to represent the uncertainty introduced by the model.

[0062] The first dimensions of the predicted state vector corresponding to the physical quantities are extracted as the predicted observation vector , and the actual observation vector is .

[0063] Calculate the observation residual:

[0064] Construct the measurement matrix as , and calculate the Kalman gain:

[0065] The predicted state is weighted and corrected using Kalman gain to generate a filtered state estimate:

[0066] And update the covariance simultaneously:

[0067] in, Represents the observation residual vector; This represents the observation vector collected in the current period, corresponding to the normalized physical state. ; The observation vector representing the current periodic prediction is given by... Center front Extraction from each dimension; This represents the Kalman gain matrix for the current period; The observation matrix is ​​defined as follows: ; This represents the transpose of the observation matrix; Represents the observation noise covariance matrix; This represents the matrix inversion operation; This represents the estimated state value after fusion in the current cycle; Represents the identity matrix, with dimensions AND same; This represents the covariance matrix of the state estimate after the current cycle fusion.

[0068] In the multi-source state fusion stage, min-max normalization and a preset splicing order are introduced. Digital twin states, physical observation states, and map semantic vectors are mapped to a unified numerical range before being combined, providing stable input for subsequent filters. Unlike traditional direct splicing or simple weighting, this normalization-splicing process ensures that various feature dimensions participate in prediction at the same scale, avoiding the situation where high-dimensional semantic vectors mask low-dimensional physical quantities or twin quantities, thereby enhancing fusion accuracy.

[0069] In linearized prediction, a state transition matrix is ​​used to map the extended state in one step. Then, a process noise matrix is ​​superimposed in the covariance prediction to provide local linear approximation support for the nonlinear system. Accurate observation residual calculation is achieved by rigorously extracting the dimensions corresponding to the physical states from the observation matrix and generating the predicted observation vector. Subsequently, a Kalman gain is constructed based on the ratio of the residual to the predicted covariance to complete the prediction-update closed loop, enabling the filter to maintain high robustness in the face of abrupt changes or non-Gaussian interference.

[0070] After importing the residual weight correction, the real-time state estimation value and covariance matrix are updated, forming a feedback online calibration mechanism. This mechanism not only digests new measurement errors in each cycle, but also automatically adjusts the confidence of each dimension through the gain matrix, overcoming the limitations of the fixed noise assumption. Compared with existing fixed parameter or static covariance schemes, the filter parameters adapt to dynamic changes in the environment, significantly shortening the response delay and improving the stability of state estimation.

[0071] S3: Perform Monte Carlo simulation according to state estimation results, generate multiple abnormal state samples, calculate the residual mean between abnormal samples and current estimation results, and correct the covariance parameters of the extended Kalman filter to generate a warning label.

[0072] The state estimation vector output by S2 as the mean; the covariance matrix output by S2 as the covariance, construct a multi-dimensional Gaussian distribution:

[0073] Set the number of abnormal samples (eg 500), sample using Gaussian distribution abnormal state samples:

[0074] where each ; represents the state estimation vector of the current cycle (output by S2), with a dimension of ; represents the state estimation covariance matrix of the current cycle (output by S2), with a dimension of ; represents a multi-dimensional Gaussian distribution, the former is the mean, and the latter is the covariance matrix; represents the abnormal state vector sampled from the above Gaussian distribution ; represents the set number of abnormal state samples (e.g. 500); represents a set of samples generated.

[0075] For each abnormal sample, calculate the difference with the current estimation vector dimension by dimension:

[0076] Get residual vectors .

[0077] Calculate the mean of all residual vectors in each dimension to obtain the residual mean vector:

[0078]

[0079] wherein, represents the residual error vector between the th abnormal sample and the current state estimation; represents the result of averaging all abnormal residual error vectors in each dimension, still with dimension, called the residual mean vector; represents the sum of all residual error vectors by dimension.

[0080] The residual mean vector is filled in the diagonal to generate a diagonal matrix .

[0081] Element-wise addition with the original covariance matrix results in the modified covariance matrix:

[0082] Set the medium risk threshold and the high risk threshold (either different in each dimension, or a unified threshold); Compare each element of in turn: if any component , the current period warning label is “high risk”; otherwise, if any component , the “medium risk”; if all components are less than , the “low risk”.

[0083] wherein, represents the conversion of the residual mean vector into a diagonal matrix, with the mean of each dimension located on the corresponding diagonal line; represents the generated diagonal matrix, representing the average residual error in each state dimension, with dimension; represents the modified covariance matrix; represents the original covariance matrix; represents the diagonal matrix composed of residual mean values from abnormal sampling.

[0084] Write the label into the log buffer and send it to the monitoring platform.

[0085] According to the risk level of the label , automatically select the preset device control strategy.

[0086] High risk: issue load shedding instructions, emergency shutdown, forced alarm.

[0087] Medium risk: prompt warning, suggest manual inspection.

[0088] Low risk: maintain normal operation of equipment, periodic record.

[0089] Optionally, the current cycle and As feedback input, dynamically fine-tune the attention allocation weights of the graph neural network, and the covariance initialization value of the extended Kalman filter in the subsequent cycle, to realize adaptive adjustment of parameters.

[0090] A multi-dimensional Gaussian modeling mechanism based on state estimation results is introduced. By taking the state estimation value of the last cycle as the mean value and the estimated covariance as the covariance matrix, a joint distribution is constructed, and Monte Carlo sampling is performed to generate multiple abnormal state samples. Compared with existing residual statistics or single-point anomaly detection methods, this structure fully utilizes the global information of the system-level state distribution, improves the coverage and diversity of abnormal sample generation, and provides a more stable and reliable data basis for subsequent residual structure analysis.

[0091] In the sample residual construction link, instead of directly using the point-by-point threshold strategy, a residual mean vector is introduced and a diagonal matrix is constructed to fuse with the original covariance matrix element by element, effectively enhancing the response ability of the filter to potential abnormal trends. This method involves the residual mean in covariance correction, enabling the filter to dynamically update the uncertainty at the state space level, making it more robust than traditional fixed covariance or error filters in dealing with non-Gaussian disturbances, especially suitable for potential deviation warning tasks in multi-dimensional complex systems.

[0092] The risk label generation process is based on a hierarchical threshold judgment mechanism based on residual mean values. After independent judgment of each dimension risk level, joint judgment is performed. This strategy differs from existing single modal or overall index aggregation judgment mode, and can trigger medium risk or high risk identification in advance when partial dimensional anomalies initially appear, improving the warning sensitivity and response timeliness. Combined with local log buffering and remote platform synchronization structure, label generation not only serves the current cycle control response, but also provides data closure for historical abnormal trend backtracking and parameter adaptive adjustment.

[0093] The embodiment also provides an Internet of Things device real-time warning system based on digital twinning, comprising: A multi-modal data acquisition module is used to acquire time series signal data, spatial environment parameters, and digital twinning model output states.

[0094] A graph structure construction module is used to extract multi-modal features, construct modal feature nodes and edge sets, and generate a graph structure.

[0095] The semantic graph generation module is configured to input the graph structure into a graph neural network to generate a semantic graph vector.

[0096] The anomaly simulation module is configured to perform Monte Carlo simulation based on the state estimation result to generate an abnormal state sample and calculate a residual mean.

[0097] The early warning generation module is configured to generate an early warning label according to the residual mean and a risk threshold.

[0098] The parameter optimization module is configured to update the attention weight and the covariance initialization parameter according to the residual mean and the early warning label.

[0099] The embodiment also provides a computer device suitable for the digital twin-based real-time early warning method of the Internet of Things device, which comprises a memory and a processor.

[0100] The computer device can be a terminal, and the computer device comprises a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. In addition, the input device can also be an external keyboard, touchpad or mouse, etc.

[0101] The embodiment also provides a storage medium on which a computer program is stored, the program being executed by a processor to implement the method for realizing real-time early warning of an IoT device based on a digital twin as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0102] To sum up, the application improves the safety of equipment operation and the accuracy of early warning by constructing a multi-modal graph structure to extract semantic features, combining physical and simulation states for state fusion, and implementing covariance correction and risk level identification based on abnormal sampling.

[0103] It should be noted that the above embodiments are only used to illustrate the technical solutions of the application rather than limit the application. Although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the application, and all of them should be covered in the scope of the claims of the application.

Claims

1. A digital-twin-based real-time warning method for an Internet of Things device, characterized in that, The method comprises the following steps: Collecting multi-modal raw data of Internet of Things devices, constructing a graph structure, and generating a semantic graph vector; Obtaining the state of the digital twin model corresponding to the target device, and combining the digital twin state, the physical device state and the semantic graph vector into an extended state vector, inputting the extended Kalman filter for state fusion to obtain a state estimation result; According to the state estimation result, a Monte Carlo simulation is performed to generate a plurality of abnormal state samples, the residual mean between the abnormal samples and the current estimation result is calculated, and the covariance parameter of the extended Kalman filter is corrected to generate a warning label.

2. The digital-twin-based real-time warning for IoT devices of claim 1, wherein: The multi-modal raw data includes time series signal data, spatial environment parameters and digital twin model output state; The graph structure is constructed by: extracting features from the multi-modal raw data to obtain time series modal features, spatial modal features and twin modal features; the above modal features are used as nodes of the graph structure to form a modal feature node set, wherein the initial feature vector of each node is an expression vector generated by frequency domain transformation of the corresponding modal raw data; when the cosine similarity between any two modal features exceeds a certain threshold, a connection relationship between the nodes is established to form an edge set, and the weight of each edge is set as the mutual information value between the corresponding modal features.

3. The digital-twin-based real-time warning for IoT devices of claim 2, wherein: The semantic graph vector is generated by: taking the initial feature vector of the modal feature node set as the input node feature, taking the edge set and its edge weight as the graph structure input, constructing the input graph of the graph neural network; performing multi-layer graph convolution operation on the input graph in turn, updating the embedding vector representation of the node layer by layer, and introducing a cross-modal attention mechanism in the node updating process to adjust the information aggregation weight according to the semantic correlation between different modalities; taking the embedding vector obtained by each node in the last layer of the graph convolution network as the final feature of the node, and aggregating the final features of all nodes through a graph-level global pooling function to obtain a semantic graph vector.

4. The digital-twin-based real-time warning for IoT devices of claim 3, wherein: The extended state vector is composed of the structural operating state vector of the physical device in the current sampling period, the corresponding digital twin model output state vector, and the semantic graph vector output by the graph neural network, which are normalized and combined into an extended state vector in a certain order. The physical device structural operating state vector is a statistical feature expression including temperature, voltage, current and vibration; The digital twin model output state vector is a state vector generated by the digital twin simulation model corresponding to the target device based on historical operating data, current environment parameters and model structure parameters after state evolution calculation in the current sampling period, and the simulation state prediction vector includes structural parameters consistent with the physical device state dimension, including, simulated pressure value, predicted temperature, fatigue loss rate and load response coefficient.

5. The digital-twin-based real-time warning for IoT devices of claim 4, wherein: The state estimation result includes: inputting the extended state vector into an extended Kalman filter, collecting the state estimation value of the last period as the initial state of the current prediction, using a nonlinear combination method to weight and superimpose the time characteristics and device dynamic characteristics in the extended state vector, and generating the predicted state value of the current period; based on the error covariance matrix of the last period, the covariance estimation value of the current predicted state is calculated, and the preset process noise covariance is added; The feature dimensions corresponding to the device observation information in the predicted state value are extracted as the observation mapping result, and the actual observation value of the physical device in the current sampling period is compared dimension by dimension to obtain the prediction residual of each dimension; according to the proportional relationship between the prediction residual of each dimension and the corresponding covariance estimation value, the weight coefficient for residual correction is calculated, the predicted state value is corrected by using the weighted average method, and the state estimation value of the current period is generated, and the corresponding covariance matrix is updated synchronously.

6. The digital-twin-based real-time warning for IoT devices of claim 5, wherein: The generation of multiple abnormal state samples includes: based on the state estimation value and the covariance matrix of the current period, a multi-dimensional Gaussian distribution model is constructed with the state estimation value as the mean value and the covariance matrix as the covariance; A set number of abnormal state samples are generated from the Gaussian distribution model by a Monte Carlo sampling method; the residual of each abnormal state sample and the corresponding state estimation value is calculated by dimension to obtain multiple residual vectors; the residual mean vector is obtained by averaging the residual vectors in each dimension; the residual mean vector is converted into a diagonal matrix form, and is added to the original covariance matrix element by element to obtain the corrected covariance matrix.

7. The digital-twin-based real-time warning for IoT devices of claim 6, wherein: The generation of the early warning label includes: setting a medium risk threshold and a high risk threshold, and comparing each dimension of the residual mean vector item by item; when at least one dimension element exceeds the high risk threshold, the early warning label of the current period is set to high risk; if all dimensions do not exceed the high risk threshold, but at least one dimension exceeds the medium risk threshold, the early warning label is set to medium risk; when all dimension residuals are lower than the medium risk threshold, the early warning label is set to low risk; the early warning label is written into the local log buffer and sent to the monitoring platform; according to the risk level corresponding to the early warning label, a predetermined response instruction associated with the device control interface is selected and executed.

8. A digital-twin-based real-time warning system for IoT devices, the digital-twin-based real-time warning system for IoT devices according to any one of claims 1 to 7, characterized in that, It includes: A multi-modal data acquisition module for acquiring time series signal data, spatial environment parameters and digital twin model output states; A graph structure construction module for extracting multi-modal features, constructing modal feature nodes and edge sets, and generating a graph structure; A semantic graph generation module for inputting the graph structure into a graph neural network to generate a semantic graph vector; A state fusion module for constructing an extended state vector and inputting it into an extended Kalman filter to output a state estimation result; An abnormal simulation module for performing Monte Carlo simulation based on the state estimation result to generate abnormal state samples and calculate residual mean values; An early warning generation module for generating an early warning label according to the residual mean value and the risk threshold; A parameter optimization module for updating attention weights and covariance initialization parameters according to the residual mean value and the early warning label.

Images