Internet of Things hardware equipment monitoring method based on artificial intelligence
By constructing spatiotemporally perceptive heterogeneous graphs and using graph neural network analysis, the challenges of cross-modal data alignment and implicit correlation in IoT monitoring were solved, enabling efficient prediction and early warning of equipment anomalies and faults.
Patent Information
- Application Number
- CN202511006022.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-22
- Publication Date
- 2025-10-31
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing IoT monitoring technologies struggle to effectively handle the complex relationships between cross-modal data, particularly the difficulty in aligning cross-modal data and the challenge in detecting implicit time-varying/causal relationships, resulting in insufficient ability to identify device anomalies and faults.
By acquiring the monitoring time-series data of each hardware device in the Internet of Things, a sliding window is established and a spatiotemporal sensing heterogeneous graph is constructed. The graph neural network is used to analyze the spatiotemporal sensing heterogeneous graph, mine the correlation feature data between multimodal data, and dynamically capture the implicit correlation between devices.
It achieves accurate alignment of cross-modal data and deep relationship mining, improves the adaptability and predictive ability of the monitoring system, and can provide early warning of potential faults.
Smart Images
Figure CN120875007A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network artificial intelligence technology, and in particular to an artificial intelligence-based method for monitoring Internet of Things (IoT) hardware devices. Background Technology
[0002] IoT monitoring technology has been widely applied in industries such as manufacturing, energy, and healthcare. It uses multimodal sensors (such as vibration, temperature, and vision sensors) to collect equipment operating data in real time, enabling the monitoring and early warning of hardware status. However, existing methods mainly rely on preset rules or simple statistical correlation analysis, which makes it difficult to effectively handle the complex relationships between cross-modal data. This difficulty is mainly reflected in two aspects.
[0003] On the one hand, cross-modal data presents alignment challenges. For example, in industrial scenarios, when the timestamp error between vibration sensors and infrared cameras exceeds 100 milliseconds, the correlation between abnormal equipment vibration and heating phenomena will fail, leading to the inaccurate identification of critical fault modes. This difficulty in aligning cross-modal data severely restricts the reliability of monitoring systems. On the other hand, the time-varying / causal relationships implicit in cross-modal data are difficult to discover through explicit rules or simple associations, requiring in-depth mining of deeper relationships. Taking chemical production as an example, the correlation between abnormal current in a reactor agitator motor and pressure fluctuations may dynamically change with material concentration, and is not an intuitive direct correlation. Traditional methods struggle to capture such nonlinear, dynamically evolving deep relationships.
[0004] Therefore, a new AI-based IoT monitoring method is needed to address the problem that existing IoT monitoring technologies struggle to uncover the implicit connections between hardware devices in the IoT. The aim is to improve the ability to predict device anomalies and failures by deeply mining potential correlation patterns in cross-modal data. Summary of the Invention
[0005] Therefore, this invention provides an artificial intelligence-based method for monitoring IoT hardware devices, which solves the problem that existing IoT monitoring technologies struggle to uncover implicit connections between hardware devices in the IoT.
[0006] This invention provides an artificial intelligence-based method for monitoring IoT hardware devices, comprising:
[0007] Acquire monitoring time-series data for each hardware device in the Internet of Things (IoT), and establish a sliding window within the monitoring time-series data;
[0008] Move the sliding window and build a spatiotemporal awareness heterogeneous graph for the monitoring time series data in the sliding window each time it moves. The nodes in the spatiotemporal awareness heterogeneous graph represent one type of monitoring time series data in the sliding window, and the edges between nodes represent a spatiotemporal proximity relationship between two types of monitoring time series data.
[0009] Based on the preset graph neural network analysis of the spatiotemporal sensing heterogeneous graph, the associated feature data corresponding to each spatiotemporal sensing heterogeneous graph is obtained. The associated feature data is used to represent the correlation between various monitoring time series data within the sliding window.
[0010] Based on the changing patterns of associated feature data, implicit relationships between hardware devices in the Internet of Things (IoT) can be obtained.
[0011] The present invention also provides a preferred embodiment: moving a sliding window and establishing a spatiotemporally aware heterogeneous graph for the monitoring time-series data within the sliding window during each movement, including:
[0012] Move the sliding window and use the monitoring time series data within the sliding window as the target monitoring time series data;
[0013] Feature extraction is performed on the time-series data of target monitoring to obtain the data features corresponding to each type of target monitoring time-series data;
[0014] A node is created for each type of target monitoring time series data. Each node has a corresponding node attribute vector, which includes the timestamp, spatial coordinates, modality type, and data features of the target monitoring time series data.
[0015] Based on the spatiotemporal proximity of the time series data for each target monitoring, edges are established between nodes, and each edge corresponds to a weight representing the spatiotemporal proximity.
[0016] Based on the nodes and edges, a spatiotemporal perception heterogeneous graph corresponding to the target monitoring time series data is obtained.
[0017] The present invention also provides a preferred embodiment: the edges in the spatiotemporally aware heterogeneous graph include temporal connection edges and spatial connection edges; based on the spatiotemporal proximity of the time series data of each target monitoring, edges are established between nodes, including:
[0018] Calculate the difference in timestamps and the distance between spatial coordinates corresponding to the time series data of two targets;
[0019] If the difference in timestamps is less than a preset time threshold, a time connection edge is established between the nodes corresponding to the two target monitoring time series data, and the weight corresponding to the edge is obtained according to the difference in timestamps.
[0020] If the distance between spatial coordinates is less than a preset spatial threshold, a spatial connection edge is established between the nodes corresponding to the time series data of the two targets, and the weight corresponding to the edge is obtained according to the distance between the spatial coordinates.
[0021] The present invention also provides a preferred embodiment: the preset graph neural network includes an input layer, a node fusion layer, an association analysis layer, and an output layer, wherein:
[0022] The input layer is used to: input spatiotemporally aware heterogeneous graphs;
[0023] The node fusion layer is used to fuse the information of each node and its neighboring nodes in the spatiotemporally aware heterogeneous graph to obtain the fused feature vector corresponding to each node.
[0024] The correlation analysis layer is used to obtain correlation feature data based on the fused feature vectors of all nodes. The correlation feature data is in matrix form, where each row and column corresponds to a type of monitoring time series data, and the elements in the matrix represent the degree of correlation between the two types of monitoring time series data corresponding to their respective rows and columns.
[0025] The output layer is used to output associated feature data.
[0026] This invention also provides a preferred embodiment: In the spatiotemporally aware heterogeneous graph, each node corresponds to a node attribute vector representing the attributes of the monitored time-series data, and each edge corresponds to a weight representing spatiotemporal proximity, including temporal connection edges and spatial connection edges; by fusing the information of each node and its neighboring nodes in the spatiotemporally aware heterogeneous graph, a fused feature vector corresponding to each node is obtained, including:
[0027] The concatenated vector is obtained by concatenating the node attribute vectors of the target node and its neighboring nodes, as well as the weights of the edges between them.
[0028] The concatenated vector is mapped to the correlation representation space to obtain the correlation feature data between the target node and its neighboring nodes;
[0029] Based on the distribution of the correlation feature data between the target node and all its neighboring nodes, the attention score between the target node and all its neighboring nodes is obtained;
[0030] Based on attention scores, different weight matrices are used to fuse the node attribute vectors of neighboring nodes into the attribute vectors of the target node according to the weights of temporal and spatial connection edges, respectively, to obtain the fused feature vector corresponding to the target node.
[0031] The present invention also provides a preferred embodiment: obtaining associated feature data based on the fused feature vectors of all nodes, including:
[0032] Calculate the cosine similarity between each pair of fused feature vectors;
[0033] A matrix is constructed based on the cosine similarity between every two fused feature vectors to serve as associated feature data.
[0034] This invention also provides a preferred embodiment: obtaining implicit associations between hardware devices in the Internet of Things based on the changing patterns of associated feature data, including:
[0035] The associated feature data are arranged based on the sliding order of the sliding window;
[0036] The arranged correlation feature data is input into a preset recurrent neural network model to obtain the data output by the preset recurrent neural network model. This data is used to represent the implicit correlation between hardware devices in the Internet of Things.
[0037] This invention also provides an artificial intelligence-based IoT hardware device monitoring system, comprising:
[0038] The data preparation module is used to acquire monitoring time-series data of each hardware device in the Internet of Things and to create a sliding window in the monitoring time-series data.
[0039] The association modeling module is used to move the sliding window and build a spatiotemporal awareness heterogeneous graph for the monitoring time series data in the sliding window each time it moves. The nodes in the spatiotemporal awareness heterogeneous graph represent one type of monitoring time series data in the sliding window, and the edges between nodes represent a spatiotemporal proximity relationship between two types of monitoring time series data.
[0040] The alignment analysis module is used to analyze the spatiotemporal sensing heterogeneous map according to the preset graph neural network to obtain the associated feature data corresponding to each spatiotemporal sensing heterogeneous map. The associated feature data is used to represent the correlation relationship between various monitoring time series data within the sliding window.
[0041] The correlation analysis module obtains the implicit correlations between hardware devices in the Internet of Things based on the changing patterns of correlation feature data.
[0042] The present invention also provides an electronic device, comprising:
[0043] Memory and processor;
[0044] The memory is used to store the program, and the processor is used to execute the steps in any of the above-described AI-based IoT hardware device monitoring methods when the program is executed.
[0045] The present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in any of the above-described methods for monitoring IoT hardware devices based on artificial intelligence.
[0046] The beneficial effects of adopting the above scheme are:
[0047] This invention provides an AI-based method for monitoring IoT hardware devices. First, it acquires monitoring time-series data for each hardware device in the IoT and establishes a sliding window within this data. Then, it moves the sliding window, creating a spatiotemporal-aware heterogeneous graph for each movement. Nodes in the graph represent one type of monitoring time-series data within the sliding window, and edges between nodes indicate a spatiotemporal proximity relationship between two types of monitoring time-series data. Next, it analyzes the spatiotemporal-aware heterogeneous graph using a pre-defined graph neural network to obtain associated feature data corresponding to each graph. This associated feature data represents the correlation between various monitoring time-series data within the sliding window. Finally, based on the changing patterns of the associated feature data, it obtains the implicit correlations between hardware devices in the IoT. This invention captures the local spatiotemporal features of monitoring data through a sliding window mechanism, constructs a spatiotemporally perceptive heterogeneous graph to simultaneously model the spatiotemporal proximity relationships between multimodal monitoring data, solves the problem of cross-modal data alignment, and preserves the spatiotemporal context information of the data. It uses graph neural networks to analyze the heterogeneous graph structure, which can automatically learn complex nonlinear correlation patterns between monitoring data. Finally, based on the analysis of the changing patterns of correlation feature data, it dynamically captures the time-varying / causal relationships between devices over time. This solves the problem that existing IoT monitoring technologies are unable to uncover the implicit correlations between hardware devices in the IoT, making the monitoring system more adaptable and predictive, and effectively preventing potential faults. Attached Figure Description
[0048] Figure 1 A flowchart illustrating the method for monitoring IoT hardware devices based on artificial intelligence, as provided by this invention.
[0049] Figure 2 for Figure 1 A detailed step diagram of step S102 is shown below;
[0050] Figure 3 The system architecture diagram of the AI-based IoT hardware device monitoring system provided by this invention. Detailed Implementation
[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0052] Please see Figure 1 A specific embodiment of the present invention discloses an artificial intelligence-based method for monitoring Internet of Things (IoT) hardware devices, comprising:
[0053] S101. Obtain the monitoring time series data of each hardware device in the Internet of Things, and establish a sliding window in the monitoring time series data;
[0054] S102. Move the sliding window and establish a spatiotemporal awareness heterogeneous graph for the monitoring time series data in the sliding window each time it is moved. The nodes in the spatiotemporal awareness heterogeneous graph represent one type of monitoring time series data in the sliding window, and the edges between the nodes represent a spatiotemporal proximity relationship between the two types of monitoring time series data.
[0055] S103. Analyze the spatiotemporal sensing heterogeneous graphs according to the preset graph neural network to obtain the associated feature data corresponding to each spatiotemporal sensing heterogeneous graph. The associated feature data is used to represent the correlation between various monitoring time series data within the sliding window.
[0056] S104. Based on the changing patterns of the associated feature data, the implicit relationships between hardware devices in the Internet of Things are obtained.
[0057] In the above process, the spatiotemporal proximity between the two types of monitoring time series data mainly refers to the similarity in the time of collection of the two types of data, and the similarity in the spatial location of the corresponding entities (which can be the hardware device itself or the monitored object). The specific range of "similarity" can be set manually according to the actual situation and experience, which will be explained in detail later.
[0058] In reality, hardware devices with implicit connections often have temporal or spatial relationships. Therefore, this embodiment constructs a spatiotemporal sensing heterogeneous graph to describe the spatiotemporal proximity relationships between various data, and uses a preset graph neural network to mine the connections between data, thereby first solving the problem of cross-modal risk alignment. For example, in an industrial production line, edges can be established between the nodes corresponding to vibration sensors and infrared cameras (the two collect data at different periods, and due to different triggering conditions, the timing of their data collection is irregular, making it difficult to align them using only timestamps, and their spatial locations overlap). Based on the spatiotemporal sensing heterogeneous graph, a graph neural network is used to mine the potential connections between the different modal data of the two (represented as associated feature data), linking "abnormal equipment vibration data + heat generation image" to achieve accurate alignment of cross-modal data.
[0059] The above process aligns data within a sliding window. Based on this, further global analysis of the changing patterns of all related feature data can be performed to deeply uncover underlying relationships. For example, in monitoring chemical reactors, analyzing related feature data reveals a strong correlation between the stirring motor current and material concentration, and a strong correlation between material concentration and reactor pressure. Furthermore, these two correlations exhibit a lag relationship, allowing the discovery of a hidden causal chain where "abnormal current leads to decreased stirring efficiency, resulting in uneven material mixing and pressure fluctuations," thus providing early warning of pressure anomalies.
[0060] Furthermore, in combination Figure 2 As shown, in a preferred embodiment, step S102, moving the sliding window and establishing a spatiotemporally aware heterogeneous graph for the monitoring time-series data within the sliding window during each movement, specifically includes:
[0061] S201. Move the sliding window and use the monitoring time series data within the sliding window as the target monitoring time series data;
[0062] S202. Extract features from the target monitoring time series data to obtain the data features corresponding to each type of target monitoring time series data;
[0063] S203. Establish a node for each type of target monitoring time series data. Each node corresponds to a node attribute vector. The node attribute vector includes the timestamp, spatial coordinates, modality type, and data characteristics of the target monitoring time series data.
[0064] S204. Based on the spatiotemporal proximity of the time series data of each target monitoring, establish edges between nodes, with each edge corresponding to a weight representing spatiotemporal proximity;
[0065] S205. Based on the nodes and edges, obtain the spatiotemporal sensing heterogeneous graph corresponding to the target monitoring time series data.
[0066] This embodiment extracts multi-dimensional features from the time-series monitoring data within the window to comprehensively characterize the spatiotemporal characteristics and physical attributes of the monitoring data. The node attribute vector integrates multi-source heterogeneous information (time, space, modality, and features), enabling the heterogeneous graph to accurately record the spatiotemporal reference of data acquisition while preserving the physical connotation of the original monitoring data. The weight of the edges is set by quantifying the spatiotemporal proximity to adjust the tightness of data association and achieve accurate modeling.
[0067] In the above process, the specific method for feature extraction from the target monitoring time series data can be implemented using any existing method according to actual needs. For example:
[0068] For sensor data, data features can be obtained by extracting time-series statistical features (mean, variance, maximum value);
[0069] For image data, data features can be obtained through lightweight CNNs (such as MobileNet) or by statistically analyzing the distribution of grayscale images;
[0070] For audio data, voiceprint features can be extracted by extracting the Mel spectrum to obtain the corresponding data features;
[0071] For text data, semantic features can be extracted using a pre-trained BERT model to obtain data features.
[0072] Furthermore, in a preferred embodiment, the edges in the spatiotemporally aware heterogeneous graph include temporal connection edges and spatial connection edges. Based on this, step S204 above, establishing edges between nodes according to the spatiotemporal proximity of the time-series data for each target monitoring, includes:
[0073] Calculate the difference in timestamps and the distance between spatial coordinates corresponding to the time series data of two targets;
[0074] If the difference in timestamps is less than a preset time threshold, a time connection edge is established between the nodes corresponding to the two target monitoring time series data, and the weight corresponding to the edge is obtained according to the difference in timestamps.
[0075] If the distance between spatial coordinates is less than a preset spatial threshold, a spatial connection edge is established between the nodes corresponding to the time series data of the two targets, and the weight corresponding to the edge is obtained according to the distance between the spatial coordinates.
[0076] This embodiment quantifies the timestamp difference and spatial coordinate distance, and sets independent time and spatial thresholds for dual judgment, enabling accurate identification of the proximity relationships of monitoring data in both time and space dimensions. This dual-dimensional connection mechanism not only achieves fine-grained modeling of complex spatiotemporal correlations between cross-modal data, but also enables the system to independently analyze the impact of temporal and spatial proximity on data correlations through a dynamic weight allocation mechanism (calculated based on actual time difference and spatial distance). This provides a richer and more accurate structured information foundation for subsequent graph neural network mining of deep correlations, significantly enhancing the accuracy and reliability of cross-modal data alignment.
[0077] Specifically, the process of calculating the weights of temporal and spatial connectivity edges can be expressed by the following two formulas:
[0078]
[0079] Where, ω tThe weight of the time-connected edge is represented by exp(), which represents the exponential function, Δt represents the difference in timestamps between the two nodes, and σ represents the weight of the time-connected edge. t The above formula represents the preset time smoothing parameter, which can be set based on experiments or experience. δ() represents the preset linear variation function, which can also be set based on experiments or experience. Δd represents the spatial coordinate distance between the two nodes. When calculating the weight of the time connection edge, the above formula introduces the spatial coordinate distance to increase the accuracy of the weight representation. It also uses an exponential function to enhance the influence of timestamp differences, highlighting the strong correlation of recent times to distinguish it from spatial connection edges.
[0080]
[0081] Where, ω d σ represents the weight of the spatial connection edge. d This represents the preset time smoothing parameter, which can be set based on experiments or experience. Similarly, when calculating the weights of spatial connection edges, the above formula introduces the difference in timestamps to increase the accuracy of the weight representation's influence. At the same time, it uses an exponential function to enhance the influence of spatial coordinate distance, highlighting strong correlations in close proximity to distinguish them from time connection edges.
[0082] Furthermore, in a preferred embodiment, the preset graph neural network in step S103 above includes an input layer, a node fusion layer, an association analysis layer, and an output layer, wherein:
[0083] The input layer is used to: input spatiotemporally aware heterogeneous graphs;
[0084] The node fusion layer is used to fuse the information of each node and its neighboring nodes in the spatiotemporally aware heterogeneous graph to obtain the fused feature vector corresponding to each node.
[0085] The correlation analysis layer is used to obtain correlation feature data based on the fused feature vectors of all nodes. The correlation feature data is in matrix form, where each row and column corresponds to a type of monitoring time series data, and the elements in the matrix represent the degree of correlation between the two types of monitoring time series data corresponding to their respective rows and columns.
[0086] The output layer is used to output associated feature data.
[0087] In this embodiment, the node fusion layer effectively integrates local spatiotemporal correlation features by aggregating information from each node and its neighboring nodes. This ensures that each node's feature vector not only contains its own monitoring data characteristics but also incorporates contextual information from surrounding data, significantly improving feature representation capabilities. In particular, this embodiment innovatively outputs correlation feature data in matrix form through a class-based correlation analysis layer. The matrix rows and columns correspond to different monitoring time-series data, and the matrix elements precisely quantify the degree of correlation between any two types of data. This structured representation clearly demonstrates the global correlation pattern and provides a mathematical foundation for subsequent implicit correlation mining.
[0088] Specifically, in a preferred embodiment, in the spatiotemporally aware heterogeneous graph, each node corresponds to a node attribute vector representing the attributes of the monitored time-series data, and each edge corresponds to a weight representing spatiotemporal proximity, including temporal connection edges and spatial connection edges. Based on this, the node fusion layer performs the following steps: fusing the information of each node and its neighboring nodes in the spatiotemporally aware heterogeneous graph to obtain a fused feature vector corresponding to each node, specifically including:
[0089] The concatenated vector is obtained by concatenating the node attribute vectors of the target node and its neighboring nodes, as well as the weights of the edges between them.
[0090] The concatenated vector is mapped to the correlation representation space to obtain the correlation feature data between the target node and its neighboring nodes;
[0091] Based on the distribution of the correlation feature data between the target node and all its neighboring nodes, the attention score between the target node and all its neighboring nodes is obtained;
[0092] Based on attention scores, different weight matrices are used to fuse the node attribute vectors of neighboring nodes into the attribute vectors of the target node according to the weights of temporal and spatial connection edges, respectively, to obtain the fused feature vector corresponding to the target node.
[0093] This embodiment is essentially an improvement upon existing graph neural networks incorporating attention mechanisms: Graph Attention Networks (GAT). This embodiment adds edge weights when concatenating the attribute vectors of the target node and its neighbors to fully integrate the spatiotemporal characteristics and topological relationships of the monitoring data. Mapping the concatenated vectors to a dedicated relevance space for modeling enables the system to accurately capture the spatiotemporal coupling patterns between different nodes, analyze the importance of each node (i.e., attention score), and allow the model to adaptively evaluate the importance of different neighboring nodes to the target node. The introduction of an attention mechanism in this embodiment allows the fusion process to dynamically adjust according to data characteristics, significantly improving the accuracy of feature representation. Furthermore, the differentiated processing mechanism for temporal and spatial connection edges further refines the spatiotemporal information fusion method, allowing the model to optimize the correlation modeling of the temporal and spatial dimensions separately.
[0094] The above fusion process can be expressed as the following formula:
[0095] a i,j =softmax(LeakyRelU(vW1[h i ||h j ||ω t ||ω d ]));
[0096] Where i represents the index of the target node, j represents the index of a neighboring node of the target node, and a i,j Let represent the attention score between target node i and its neighbor node j, softmax() represent the preset normalization function, LeakyRelU() represent the preset activation function, and is used to map the input vector to the relevance representation space to obtain relevance feature data. v is the vector used for learning and training in the node fusion layer, W1 is the weight matrix used for learning and training in the node fusion layer, and h i h represents the node attribute vector of target node i. j This represents the node attribute vector of the target node j, where || is the concatenation symbol, [h i ||h j ||ω t ||ω d ] indicates the concatenation of vectors.
[0097] h i '=f(W2h i +avg(a i,j (ω t W3h j +ω d W4h j )));
[0098] Among them, h i' represents the fusion feature vector of the target node, f() is another preset activation function (which can be flexibly set according to the actual situation), avg() represents the averaging function, and W2, W3 and W4 are also weight matrices for learning and training in the node fusion layer (the matrices with the above four different subscripts are different).
[0099] Understandably, after obtaining the fused feature vectors, any existing method can be used to analyze the correlation between these vectors. For example, the fused feature vectors can be input pairwise into a trained feedforward neural network to obtain the output correlation data, and then a correlation matrix can be constructed.
[0100] The present invention provides a more convenient method. In a preferred embodiment, the steps performed by the above-mentioned association analysis layer to obtain association feature data based on the fused feature vectors of all nodes specifically include:
[0101] Calculate the cosine similarity between each pair of fused feature vectors;
[0102] A matrix is constructed based on the cosine similarity between every two fused feature vectors to serve as associated feature data.
[0103] This embodiment employs cosine similarity, a simple and effective metric, to directly calculate the similarity between the fused feature vectors of each pair of nodes. This not only significantly simplifies the computational complexity of association modeling, enabling the system to efficiently process large-scale IoT monitoring data, but also ensures that the analysis results focus on the directional consistency between data points due to the insensitivity of cosine similarity to vector magnitude. This effectively eliminates the interference of differences in the magnitude of monitoring values on association judgment, achieving accurate capture of weak but critical association patterns between devices while ensuring real-time performance. This significantly improves the intelligence level and early warning accuracy of the IoT monitoring system.
[0104] Furthermore, in a preferred embodiment, step S104, obtaining the implicit associations between hardware devices in the Internet of Things based on the changing patterns of the associated feature data, specifically includes:
[0105] The associated feature data are arranged based on the sliding order of the sliding window;
[0106] The arranged correlation feature data is input into a preset recurrent neural network model to obtain the data output by the preset recurrent neural network model. This data is used to represent the implicit correlation between hardware devices in the Internet of Things.
[0107] This embodiment systematically arranges the associated feature data according to the temporal order of a sliding window, fully preserving the dynamic characteristics of the monitoring data's evolution over time. Combined with a pre-set recurrent neural network (RNN) to process the temporally arranged associated feature data, it fully leverages the unique advantages of RNNs in capturing long-term and short-term dependencies, effectively identifying implicit correlation patterns between devices that dynamically change over time. This design not only overcomes the limitation of traditional methods that can only capture static correlations and cannot model time-varying relationships, but also, through the recursive processing of temporal information by the recurrent neural network, can accurately discover causal chains with a temporal sequence, such as "device A malfunction leads to increased load on device B, which in turn leads to increased energy consumption on device C." It achieves the effect of automatically learning complex nonlinear time-varying laws in the data, avoiding the subjectivity and limitations of manual feature engineering, significantly improving the accuracy and generalization ability of implicit correlation detection, enabling the system to provide early warnings of potential fault chains, and providing strong technical support for predictive maintenance of industrial equipment.
[0108] Combination Figure 3 As shown, the present invention also provides an artificial intelligence-based IoT hardware device monitoring system, comprising:
[0109] The data preparation module 310 is used to acquire the monitoring time-series data of each hardware device in the Internet of Things and to establish a sliding window in the monitoring time-series data.
[0110] The association modeling module 320 is used to move the sliding window and build a spatiotemporal perception heterogeneous graph for the monitoring time series data in the sliding window each time it moves. The nodes in the spatiotemporal perception heterogeneous graph represent one type of monitoring time series data in the sliding window, and the edges between nodes represent a spatiotemporal proximity relationship between two types of monitoring time series data.
[0111] The alignment analysis module 330 is used to analyze the spatiotemporal sensing heterogeneous map according to the preset graph neural network to obtain the associated feature data corresponding to each spatiotemporal sensing heterogeneous map. The associated feature data is used to represent the correlation relationship between multiple monitoring time series data within the sliding window.
[0112] The correlation analysis module 340 obtains the implicit correlations between hardware devices in the Internet of Things based on the changing patterns of correlation feature data.
[0113] It should be noted that the corresponding systems provided in the above embodiments are computer program products that can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding content in the above method embodiments, and will not be repeated here.
[0114] The present invention also provides an electronic device, comprising:
[0115] Memory and processor;
[0116] The memory is used to store the program, and the processor is used to execute the steps in any of the above-described AI-based IoT hardware device monitoring methods when the program is executed.
[0117] The present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in any of the above-described methods for monitoring IoT hardware devices based on artificial intelligence.
[0118] This invention provides an AI-based method for monitoring IoT hardware devices. First, it acquires monitoring time-series data for each hardware device in the IoT and establishes a sliding window within this data. Then, it moves the sliding window, creating a spatiotemporal-aware heterogeneous graph for each movement. Nodes in the graph represent one type of monitoring time-series data within the sliding window, and edges between nodes indicate a spatiotemporal proximity relationship between two types of monitoring time-series data. Next, it analyzes the spatiotemporal-aware heterogeneous graph using a pre-defined graph neural network to obtain associated feature data corresponding to each graph. This associated feature data represents the correlation between various monitoring time-series data within the sliding window. Finally, based on the changing patterns of the associated feature data, it obtains the implicit correlations between hardware devices in the IoT. This invention captures the local spatiotemporal features of monitoring data through a sliding window mechanism, constructs a spatiotemporally perceptive heterogeneous graph to simultaneously model the spatiotemporal proximity relationships between multimodal monitoring data, solves the problem of cross-modal data alignment, and preserves the spatiotemporal context information of the data. It uses graph neural networks to analyze the heterogeneous graph structure, which can automatically learn complex nonlinear correlation patterns between monitoring data. Finally, based on the analysis of the changing patterns of correlation feature data, it dynamically captures the time-varying / causal relationships between devices over time. This solves the problem that existing IoT monitoring technologies are unable to uncover the implicit correlations between hardware devices in the IoT, making the monitoring system more adaptable and predictive, and effectively preventing potential faults.
[0119] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0120] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for monitoring IoT hardware devices based on artificial intelligence, characterized in that, include: Acquire monitoring time-series data for each hardware device in the Internet of Things (IoT), and establish a sliding window within the monitoring time-series data; Move the sliding window and build a spatiotemporal awareness heterogeneous graph for the monitoring time series data in the sliding window each time it moves. The nodes in the spatiotemporal awareness heterogeneous graph represent one type of monitoring time series data in the sliding window, and the edges between nodes represent a spatiotemporal proximity relationship between two types of monitoring time series data. Based on the preset graph neural network analysis of the spatiotemporal sensing heterogeneous graph, the associated feature data corresponding to each spatiotemporal sensing heterogeneous graph is obtained. The associated feature data is used to represent the correlation between various monitoring time series data within the sliding window. Based on the changing patterns of associated feature data, implicit relationships between hardware devices in the Internet of Things (IoT) can be obtained.
2. The method for monitoring IoT hardware devices based on artificial intelligence according to claim 1, characterized in that, The sliding window is moved, and a spatiotemporally aware heterogeneous graph is built for the monitoring time-series data within the sliding window at each movement, including: Move the sliding window and use the monitoring time series data within the sliding window as the target monitoring time series data; Feature extraction is performed on the time-series data of target monitoring to obtain the data features corresponding to each type of target monitoring time-series data; A node is created for each type of target monitoring time series data. Each node has a corresponding node attribute vector, which includes the timestamp, spatial coordinates, modality type, and data features of the target monitoring time series data. Based on the spatiotemporal proximity of the time series data for each target monitoring, edges are established between nodes, and each edge corresponds to a weight representing the spatiotemporal proximity. Based on the nodes and edges, a spatiotemporal perception heterogeneous graph corresponding to the target monitoring time series data is obtained.
3. The method for monitoring IoT hardware devices based on artificial intelligence according to claim 2, characterized in that, Edges in a spatiotemporally perceptive heterogeneous graph include temporal connection edges and spatial connection edges; Based on the spatiotemporal proximity of the time-series data for each target monitoring, edges are established between nodes, including: Calculate the difference in timestamps and the distance between spatial coordinates corresponding to the time series data of two targets; If the difference in timestamps is less than a preset time threshold, a time connection edge is established between the nodes corresponding to the two target monitoring time series data, and the weight corresponding to the edge is obtained according to the difference in timestamps. If the distance between spatial coordinates is less than a preset spatial threshold, a spatial connection edge is established between the nodes corresponding to the time series data of the two targets, and the weight corresponding to the edge is obtained according to the distance between the spatial coordinates.
4. The method for monitoring IoT hardware devices based on artificial intelligence according to claim 1, characterized in that, The pre-defined graph neural network includes an input layer, a node fusion layer, an association analysis layer, and an output layer, wherein: The input layer is used to: input spatiotemporally aware heterogeneous graphs; The node fusion layer is used to fuse the information of each node and its neighboring nodes in the spatiotemporally aware heterogeneous graph to obtain the fused feature vector corresponding to each node. The correlation analysis layer is used to obtain correlation feature data based on the fused feature vectors of all nodes. The correlation feature data is in matrix form, where each row and column corresponds to a type of monitoring time series data, and the elements in the matrix represent the degree of correlation between the two types of monitoring time series data corresponding to their respective rows and columns. The output layer is used to output associated feature data.
5. The method for monitoring IoT hardware devices based on artificial intelligence according to claim 4, characterized in that, In the spatiotemporal awareness heterogeneous graph, each node corresponds to a node attribute vector representing the attribute of the monitored time series data, and each edge corresponds to a weight representing the spatiotemporal proximity. The edges include temporal connection edges and spatial connection edges. By fusing information from each node and its neighboring nodes in the spatiotemporally aware heterogeneous graph, a fused feature vector is obtained for each node, including: The concatenated vector is obtained by concatenating the node attribute vectors of the target node and its neighboring nodes, as well as the weights of the edges between them. The concatenated vector is mapped to the correlation representation space to obtain the correlation feature data between the target node and its neighboring nodes; Based on the distribution of the correlation feature data between the target node and all its neighboring nodes, the attention score between the target node and all its neighboring nodes is obtained; Based on attention scores, different weight matrices are used to fuse the node attribute vectors of neighboring nodes into the attribute vectors of the target node according to the weights of temporal and spatial connection edges, respectively, to obtain the fused feature vector corresponding to the target node.
6. The method for monitoring IoT hardware devices based on artificial intelligence according to claim 4, characterized in that, Based on the fused feature vectors of all nodes, the associated feature data is obtained, including: Calculate the cosine similarity between each pair of fused feature vectors; A matrix is constructed based on the cosine similarity between every two fused feature vectors to serve as associated feature data.
7. The method for monitoring IoT hardware devices based on artificial intelligence according to claim 1, characterized in that, Based on the changing patterns of associated feature data, implicit associations between hardware devices in the Internet of Things (IoT) are obtained, including: The associated feature data are arranged based on the sliding order of the sliding window; The arranged correlation feature data is input into a preset recurrent neural network model to obtain the data output by the preset recurrent neural network model. This data is used to represent the implicit correlation between hardware devices in the Internet of Things.
8. An artificial intelligence-based IoT hardware device monitoring system, characterized in that, include: The data preparation module is used to acquire monitoring time-series data of each hardware device in the Internet of Things and to create a sliding window in the monitoring time-series data. The association modeling module is used to move the sliding window and build a spatiotemporal awareness heterogeneous graph for the monitoring time series data in the sliding window each time it moves. The nodes in the spatiotemporal awareness heterogeneous graph represent one type of monitoring time series data in the sliding window, and the edges between nodes represent a spatiotemporal proximity relationship between two types of monitoring time series data. The alignment analysis module is used to analyze the spatiotemporal sensing heterogeneous map according to the preset graph neural network to obtain the associated feature data corresponding to each spatiotemporal sensing heterogeneous map. The associated feature data is used to represent the correlation relationship between various monitoring time series data within the sliding window. The correlation analysis module obtains the implicit correlations between hardware devices in the Internet of Things based on the changing patterns of correlation feature data.
9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store the program, and the processor is used to execute the steps of any one of the artificial intelligence-based IoT hardware device monitoring methods in claims 1-7 when the program is executed.
10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, are capable of implementing the steps in any one of the artificial intelligence-based IoT hardware device monitoring methods according to claims 1-7.