Equipment-gateway cooperative Internet of Things data anomaly detection method and system

Abstract

The invention discloses a device-gateway cooperative Internet of Things data anomaly detection method and system, and the method comprises the steps: carrying out the localized preliminary anomaly detection of original data collected by Internet of Things devices in real time through a device-level anomaly detection algorithm based on a Gaussian Bayesian model; constructing a data anomaly detection model based on self-attention graph pooling and an E-GraphSAGE graph neural network, and carrying out secondary anomaly detection on normal data received from an Internet of Things equipment side at an edge gateway side; reporting a primary anomaly detection result detected by the equipment end and a secondary anomaly detection result detected by the gateway end to the cloud end based on a cooperative detection mechanism of the equipment end and the gateway end; abnormal event information reported by the equipment end and the gateway end is received through the cloud end, alarm prompt and data analysis are carried out, and the real-time performance and accuracy of multi-source heterogeneous data anomaly detection are guaranteed at the same time in a distributed environment where resources of the Internet of Things equipment are limited.

Description

Technical Field

[0001] This invention relates to the field of IoT data anomaly detection technology, and in particular to a device-gateway collaborative IoT data anomaly detection method and system. Background Technology

[0002] With the rapid development and widespread application of IoT technology, IoT devices are playing an increasingly important role in people's lives. Hundreds of millions of sensors, devices, and systems are generating massive amounts of data at an unprecedented scale, speed, and diversity. This data carries critical value such as device operating status, environmental information, and user behavior. It is against this backdrop of interconnectivity and data explosion that real-time, efficient anomaly detection has become more necessary and critical than ever before. Anomaly detection refers to identifying deviations from expected behavior, which may indicate device malfunction, external attacks, or other unexpected events. Implementing efficient data anomaly detection mechanisms has become a core technical requirement for ensuring reliable system operation, improving security situational awareness, and optimizing operational efficiency.

[0003] IoT devices and data are characterized by massive volume, multiple sources, and heterogeneity. A typical IoT system has a three-layer architecture: device-gateway-cloud center. Devices, gateways, and the cloud center have different computing capabilities and varying real-time requirements for anomaly detection. Traditional anomaly detection methods deploy a single anomaly detection model at a specific point in the system to detect anomalies across the entire system, failing to simultaneously meet multiple requirements such as real-time performance, reliability, and efficiency. IoT terminal devices are closer to sensitive data and users, demanding even higher real-time performance in anomaly detection. However, limited hardware and software computing resources prevent the deployment of more complex anomaly detection algorithms based on deep learning principles.

[0004] Traditional IoT data anomaly detection methods typically deploy algorithms solely in a cloud center or on a gateway. The cloud-center-based approach suffers from wasted bandwidth resources, high latency, and inability to meet real-time requirements. The single-edge-gateway-based approach faces challenges such as the high computational complexity of current mainstream algorithms, the large computational load and resource demands on the gateway due to unified calculation and analysis of all device data at the gateway, and the inability to guarantee real-time performance. In summary, the single-deployment model cannot balance real-time performance and accuracy, suffers from limited device resources, high cloud latency, and excessive pressure on the gateway to process all data.

[0005] Therefore, there is an urgent need for a device-gateway collaborative IoT data anomaly detection method that can simultaneously meet the requirements of real-time detection and accuracy in distributed environments where IoT device resources are limited. Summary of the Invention

[0006] The purpose of this invention is to provide a device-gateway collaborative IoT data anomaly detection method and system, which simultaneously deploys anomaly detection algorithms of different complexities on IoT terminal devices and edge gateways, ensuring real-time performance and accuracy of anomaly detection for multi-source heterogeneous data in a distributed environment where IoT device resources are limited.

[0007] This invention provides a device-gateway collaborative IoT data anomaly detection method, comprising: A device-level anomaly detection algorithm based on a Gaussian Bayes model performs localized preliminary anomaly detection on raw data collected in real time from IoT devices and generates an anomaly detection result. A data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network is constructed. Normal data received from IoT devices is subjected to secondary anomaly detection at the edge gateway. The secondary anomaly detection includes constructing the normal data into a network topology graph and encoding the anomaly detection problem into an edge classification task to generate secondary anomaly detection results. Based on the collaborative detection mechanism between the device and the gateway, the first anomaly detection result detected by the device and the second anomaly detection result detected by the gateway are reported to the cloud. The system receives abnormal event information reported by devices and gateways via the cloud, and provides alarm prompts and data analysis.

[0008] Preferably, the device-level anomaly detection algorithm based on the Gaussian Bayes model includes: Extract the number of data packets and bytes in IoT data traffic as data features, and construct a dataset data={x,y}, where X={x1,x2} represents the feature attributes, Y∈{0,1} represents the data label, 0 represents normal traffic, and 1 represents abnormal traffic; Using dataset D, calculate the prior probabilities for different categories, corresponding to the prior probabilities of normal and abnormal data flow, respectively; where, The prior probability of normal data traffic. This represents the prior probability of abnormal data traffic. Calculate the mean μ and variance σ of the Gaussian distribution for each feature under normal and abnormal data traffic conditions, respectively. 2 ; For the new sample data to be detected, the number of data packets and the number of bytes in the new sample data are extracted as data features. The conditional probability estimates of the data features under the normal class and the abnormal class are calculated according to the probability density function of the Gaussian distribution. , Among them, y c This represents the category label, where 'c' represents the normal / abnormal category and takes the value 0 or 1, and 'x' represents the category label. iThe characteristics of real-time data collected by IoT devices, namely the number of bytes and the number of packets; According to Bayes' theorem, combining prior probability and conditional probability, the posterior probability of the new sample data to be detected belonging to each category is calculated: , Among them, y i This indicates the category to which the current data sample belongs, x1 and x2 represent the data characteristics collected by IoT devices, and P(y i ) represents category y i The prior probability, P(x) j |y i ) represents category y i Time feature x j The conditional probability, P(x1,x2), represents the total probability of observing x1 and x2 across all categories; The calculated posterior probability of belonging to the normal class is compared with the posterior probability of belonging to the abnormal class. If the posterior probability of the abnormal class is greater than the posterior probability of the normal class, it is determined to be an abnormal data sample. For data samples identified as abnormal, the device will report information containing the event data of that sample to the cloud for alarm notification.

[0009] Preferably, the construction of the data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network includes: The raw data traffic received by the edge gateway from multiple terminal devices is cleaned, and non-numerical features are converted into numerical data through one-hot encoding. At the same time, the numerical features are subjected to Min-Max normalization. The preprocessed data flow is mapped into a graph structure represented by node and edge features; An attention-score-based edge feature graph pooling method scores node importance and compresses the graph structure to generate a pooled subgraph that retains key topological information. The subgraph is input into the E-GraphSAGE graph neural network for edge feature sampling and aggregation, and the node representation is updated through multi-layer neighborhood aggregation; The obtained edges are embedded into the input fully connected layer and the Softmax classifier to generate anomaly detection results for data traffic.

[0010] Preferably, the raw data includes identification data and feature data. The identification data includes fields for identifying the communication source and destination, namely the source / destination IP address and the source / destination port number. The feature data includes the traffic duration, number of bytes, and size of the transmitted data packet.

[0011] Preferably, the self-attention graph pooling includes: edge feature e uv Project the linear layer onto a space with the same dimension as the node feature X: , Where, d e Let d be the edge feature dimension. x W represents the node feature dimension. e and b e These are trainable parameters; Based on the projected edge features and adjacent node features X v The key edge features are adaptively selected using a gating mechanism as the gating value g for each edge. uv : Using the gate value g uv For the projected edge features We perform weighted analysis to obtain weighted edge features. ; The weighted edge features are combined with node features, the importance of edge features is dynamically adjusted through a gating mechanism, and the gating edge features of neighboring nodes are aggregated through graph convolution to generate a node importance score containing edge information. Key nodes and associated edge features are selected based on the node importance score Z and the preset pooling ratio to generate a pooled subgraph.

[0012] Preferably, the gating mechanism is implemented using the following formula: , in, For the Sigmoid function, W g and b g For trainable parameters, the gate value g uv ∈[0,1] Quantized edge feature e uv The importance of node v.

[0013] Preferably, the collaborative detection mechanism based on the device end and the gateway end includes: Obtain an anomaly detection result from an IoT device and determine whether the result is abnormal; If an anomaly detection result is abnormal, the current anomaly event information will be directly reported from the IoT device to the cloud. If the first anomaly detection result is normal, the current data traffic from the IoT device is sent to the IoT edge gateway for a second anomaly detection. If the secondary anomaly detection result is abnormal, the current anomaly event information will be reported to the cloud from the edge gateway.

[0014] This invention also provides a device-gateway collaborative IoT data anomaly detection system, comprising: The device-side detection module is used for device-level anomaly detection algorithms based on Gaussian Bayes models to perform localized preliminary anomaly detection on raw data collected in real time from IoT devices and generate an anomaly detection result. The gateway detection module is used to construct a data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network. It performs secondary anomaly detection on normal data received from IoT devices at the edge gateway. The secondary anomaly detection includes constructing the normal data into a network topology graph and encoding the anomaly detection problem into an edge classification task to generate secondary anomaly detection results. The collaborative reporting module is used to report the first anomaly detection result detected by the device and the second anomaly detection result detected by the gateway to the cloud based on the collaborative detection mechanism between the device and the gateway. The cloud platform processing module is used to receive abnormal event information reported by the device and gateway through the cloud, and to provide alarm prompts and data analysis.

[0015] This invention also provides an electronic device, comprising: The memory is used to store the processing program; A processor, when executing the processing program, implements the device-gateway collaborative IoT data anomaly detection method as described in the embodiments of the present invention.

[0016] This invention also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the device-gateway collaborative IoT data anomaly detection method described in this invention.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The device-gateway collaborative IoT data anomaly detection method and system provided by this invention deploys a lightweight anomaly detection algorithm based on a Gaussian Bayes model on IoT terminal devices with limited computing resources. This algorithm can promptly detect abnormal data at the data source and directly report it to the application platform for processing, ensuring real-time anomaly detection. Furthermore, on edge gateway devices with relatively strong computing resources, a gateway-level anomaly detection model based on an improved edge graph neural network algorithm is deployed to perform in-depth analysis of multi-source heterogeneous data from various terminal devices. Data anomaly detection is performed through graph neural networks, improving the accuracy of anomaly detection in IoT field systems. In scenarios with numerous IoT devices and heterogeneous data, the device-gateway collaborative IoT data anomaly detection method and system proposed in this invention offer better real-time performance, accuracy, and feasibility compared to single edge gateway-based or cloud center-based models, as detailed below: 1. Highly efficient collaboration and optimized resource allocation: This invention adopts a two-tier collaborative architecture of "initial screening at the device end + precise judgment at the gateway end." The device end uses a Gaussian Bayes model with minimal computational overhead, quickly filtering out most obviously abnormal traffic. This avoids uploading all data to the gateway for complex calculations, greatly reducing the processing pressure on the edge gateway. Only data deemed "normal" or suspicious by the device end is sent to the gateway for in-depth analysis, achieving precise allocation and optimized configuration of computing resources.

[0018] 2. High-precision detection, effectively uncovering complex attacks: This invention innovatively employs a graph neural network-based detection model at the gateway level. By constructing network traffic as a topology graph and transforming the anomaly detection problem into an edge classification task, it can effectively capture the complex relationships and communication patterns between network entities (devices). Combined with self-attention graph pooling technology, the model can adaptively focus on the most critical parts of the graph, suppressing noise interference, thereby more accurately identifying distributed, covert, and cooperative attacks (such as port scanning, botnet communication, etc.) that are difficult for traditional models to detect, significantly improving detection accuracy and recall.

[0019] 3. Low latency and high real-time performance: Most obvious anomalies can be identified and reported immediately on the device itself, without the need for long processing through a gateway and uploading to the cloud. For data requiring secondary detection, processing is also completed at the network edge, avoiding round trips to the cloud. This "near-source processing" mode greatly shortens the time from anomaly occurrence to system response, meeting the stringent real-time requirements of IoT scenarios.

[0020] 4. Reduced bandwidth costs and enhanced privacy protection: Because devices only report abnormal events and a small amount of suspicious data, instead of all raw traffic, the amount of data transmitted is greatly reduced, significantly lowering network bandwidth costs and communication expenses. At the same time, most sensitive raw data remains local or at the edge, reducing the risk of exposure to the cloud. In particular, through node hash anonymization, even if data is processed at the gateway, the leakage of direct identifying information such as device IP addresses is avoided, further enhancing the system's privacy protection capabilities.

[0021] 5. Strong System Robustness and Scalability: The collaborative mechanism of this invention does not rely on a single central node. Even if the cloud is temporarily unavailable, the local detection capabilities of the device and gateway can still function, providing basic security protection. As the scale of the IoT network expands, the system's processing capacity can be horizontally scaled simply by increasing the number of edge gateways, demonstrating excellent scalability. Attached Figure Description

[0022] Figure 1 This is a topology diagram of the device-gateway collaborative IoT data anomaly detection architecture in an embodiment of the present invention; Figure 2 This is a flowchart of the device-level anomaly detection method in an embodiment of the present invention; Figure 3 This is a flowchart of the gateway-level anomaly detection method in an embodiment of the present invention; Figure 4 This is a flowchart of the self-attention graph pooling method in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The term "comprising" and its variations as used herein are open-ended inclusion, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0025] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0026] It should be noted that the terms "a" and "a plurality of" used in this application disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0027] Example 1 This invention provides a device-gateway collaborative IoT data anomaly detection method. Under the cloud-pipe-edge-device architecture of an IoT system, this method achieves layered and efficient detection of data anomalies by collaboratively deploying anomaly detection algorithms of varying complexity at the device and edge gateway levels. Addressing the shortcomings of existing network intrusion detection methods, such as poor detection accuracy and weak generalization ability, this embodiment proposes a network intrusion detection method based on an improved E-GraphSAGE and CNN. First, traffic features are constructed into a graph structure. Then, a graph attention network is introduced on top of E-GraphSAGE to aggregate nodes based on the attention weights of neighboring nodes and edges, generating more discriminative node embeddings. Next, graph training is performed using deep graph information to fully learn the graph's topology and edge information. Finally, the extracted edge embedding features are input into a hierarchical attention residual convolutional network for detection and classification. Figure 1 As shown, the method includes the following steps: Step S1: A device-level anomaly detection algorithm based on a Gaussian Bayes model is used to perform localized preliminary anomaly detection on the raw data collected in real time by IoT devices, generating an anomaly detection result. On IoT devices with limited computing resources, the Gaussian Bayes model-based device-level anomaly detection algorithm is deployed to promptly detect abnormal data at the data source, improving detection real-time performance. This step promptly detects obvious abnormal data at the data source, and if an anomaly is detected, it is immediately reported, thereby maximizing the real-time performance of the detection. This step is executed on IoT terminal devices. Considering that these devices typically have limited computing power and memory, this invention selects the Gaussian Bayes model, which has extremely low computational overhead and high speed, as the first line of defense.

[0028] S2: Construct a data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network. Perform secondary anomaly detection on normal data received from IoT devices at the edge gateway. This secondary anomaly detection includes constructing a network topology graph from the normal data and encoding the anomaly detection problem as an edge classification task to generate secondary anomaly detection results. At the IoT edge gateway with sufficient computing resources, perform secondary anomaly detection on data received from IoT devices and deemed normal by the devices. The core of this secondary anomaly detection lies in constructing a network topology graph from the normal data and formalizing the anomaly detection problem as a task of classifying the edges in this network topology graph to generate secondary anomaly detection results. This step leverages the powerful relational reasoning capabilities of graph neural networks to uncover complex interactions between devices and identify more covert collaborative or distributed attacks that are difficult to detect with single-point detection at the device end.

[0029] S3: Based on the collaborative detection mechanism between the device and the gateway, the first anomaly detection result detected by the device and the second anomaly detection result detected by the gateway are reported to the cloud; S4: Receives abnormal event information reported by the device and gateway through the cloud, and provides alarm prompts and data analysis.

[0030] This embodiment addresses the common limitation of limited computing resources in IoT terminal devices, which hinders the deployment of complex anomaly detection algorithms. It proposes a device-level anomaly detection method based on a Gaussian Bayes model. This lightweight algorithm, grounded in statistical principles, is simple, fast, and requires minimal computing resources, making it suitable for deployment in IoT terminal devices. Anomaly detection is performed at the data source, ensuring real-time performance. Furthermore, with advancements in chip technology, edge gateways can now integrate lightweight AI chips, providing sufficient computing power and resources. Therefore, deploying a gateway-level anomaly detection algorithm based on self-attention graph pooling and E-GraphSAGE (a graph neural network algorithm) within the gateway simplifies the algorithm's complexity, meets the hardware resource requirements of edge gateways, and provides more in-depth and accurate anomaly detection capabilities.

[0031] To address the issue that existing data anomaly detection algorithms, deployed in a single mode (on a cloud computing server or edge gateway), cannot simultaneously meet the requirements of real-time performance and accuracy, this implementation first deploys the anomaly detection algorithm at the data site. This avoids the shortcomings of the cloud-centric mode, such as wasted transmission traffic, significant latency, and inability to guarantee real-time performance. Furthermore, the single edge gateway deployment mode suffers from limited computing power, making it unable to support overly complex computational deployments. This embodiment organically collaborates with the terminal device and the edge gateway to jointly construct a collaborative data anomaly detection method. A Gaussian Bayes-based data anomaly detection algorithm is deployed on the terminal device, while an E-GraphSAGE-based graph neural network algorithm is deployed on the edge gateway. Simultaneously, an attention graph pooling method is used to improve the E-GraphSAGE algorithm, reducing its complexity and enabling deployment on the edge gateway, thereby satisfying the overall requirements for real-time performance and accuracy of the data anomaly detection algorithm.

[0032] Reference Figure 2In this embodiment, step S1, a device-level anomaly detection algorithm based on the Gaussian Naïve Bayes (GNB) model, deployed on IoT terminal devices, operates on the following principle: Given that the number of data packets and bytes for each device follows a Gaussian distribution, the Gaussian parameters corresponding to each feature are estimated. For each feature value, the conditional probability is calculated using the Gaussian formula. The frequency of the category in the dataset is used as the prior probability, and the posterior probability of each feature value is applied as the feature value prediction distribution. This enables the terminal device to perform anomaly detection on data traffic and ensures real-time detection. Specifically, this includes: Step S11: Feature Extraction and Dataset Construction: Extract the number of data packets and bytes in the IoT data traffic as data features, and construct a dataset data={x,y}, where x represents the feature attributes contained in this dataset, including the two features of data traffic, namely the number of packets and the number of bytes, denoted as x={x1,x2}. y represents the label of this dataset, that is, normal traffic or abnormal traffic, using the number 0 to represent normal traffic and the number 1 to represent abnormal traffic, denoted as y={y0,y1}.

[0033] Step S12: Calculate the prior probability of each category using the known dataset: Calculate the prior probabilities of different categories using dataset D, corresponding to the prior probabilities of normal and abnormal data flow, respectively; where, The prior probability of normal data traffic. This represents the prior probability of abnormal data traffic. Step S13: Calculate Gaussian distribution parameters: Calculate the mean μ and variance σ of the Gaussian distribution for each feature when the data flow is normal and abnormal, respectively. 2 Calculate the expectation and variance of the Gaussian distributions corresponding to features x1 and x2 when the data flow is normal, and calculate the expectation and variance of the Gaussian distributions corresponding to features x1 and x2 when the data flow is abnormal.

[0034] Step S14: Feature extraction of the sample to be tested: For the new sample data to be tested, extract the number of data packets and the number of bytes in the new sample data to be tested as data features; Step S15: Calculate conditional probabilities: Calculate the conditional probability estimate for each feature attribute under each category based on the probability density function of the Gaussian distribution. , Where x represents the number of bytes and the number of packets, y c This represents the category label, where 'c' represents the normal / abnormal category and takes the value 0 or 1, and 'x' represents the category label. i The characteristics of real-time data collected by IoT devices, namely the number of bytes and the number of packets; Step S16: Calculate the posterior probability using Bayes' theorem: Based on Bayes' theorem, and combining prior and conditional probabilities, calculate the posterior probability that the new sample data to be detected belongs to each category: , if Then it belongs to the k-th class, where P(x1,x2) is the prior probability of the two features x1 and x2, and y i This indicates the category to which the current data sample belongs, x1 and x2 represent the data characteristics collected by IoT devices, and P(y i ) represents category y i The prior probability, P(x) j |y i P(x1,x2) represents the conditional probability of feature xⱼ when category yᵢ is defined, and P(x1,x2) represents the total probability of observing x1 and x2 across all categories. Furthermore, for newly added data samples of unknown types... Calculate the probability value of this sample based on the Gaussian Bayes model, and obtain , .

[0035] Step S17: Perform a judgment comparison: Compare the calculated posterior probability of belonging to the normal class with the posterior probability of belonging to the abnormal class. If the posterior probability of the abnormal class is greater than the posterior probability of the normal class, then it is determined to be an abnormal data sample; if... ≥ If so, then the sample is normal data. < If so, then the sample is abnormal data.

[0036] Step S18: Device-side decision reporting to the application platform: For data samples identified as abnormal, the device reports information containing the event data of that sample to the cloud for alarm notification. If an anomaly detection result is abnormal, the device immediately reports the current abnormal event information, such as device ID, timestamp, and abnormal characteristics, directly to the cloud and ends the detection process. If the result is normal, the data traffic is sent to the connected IoT edge gateway, proceeding to step S2.

[0037] This embodiment deploys a data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network in an edge gateway. First, the data is preprocessed and converted into a graph structure. Second, to improve model efficiency and feature extraction capabilities, self-attention graph pooling is used to reduce the graph data size while retaining key node information. Then, the E-GraphSAGE graph neural network algorithm aggregates multi-layer neighborhood features, effectively utilizing node and edge information to enhance the ability to capture the local structure of data traffic. Finally, a softmax classifier is used in the output module to detect and classify network traffic. The detection model balances computational efficiency and model expressive power through a collaborative design of edge feature fusion and dynamic pooling.

[0038] Graph Neural Networks (GNNs) are deep learning models for processing graph-structured data. They update node representations by aggregating information from nodes and their neighbors, thereby capturing complex relationships between nodes and the topological structure of the graph. GNNs offer significant advantages in processing non-Euclidean data, adapting to diverse graph structures, and effectively aggregating and propagating information. Traditional GNNs, such as the GraphSAGE model, have been successfully applied in various fields. However, these methods primarily focus on node features for node classification, neglecting edge features for edge classification. Anomaly detection systems aim to detect and identify anomalous data, corresponding to the edge classification problem in the graph representation of network data stream datasets, where key information is provided by edge features. To introduce edge features containing important information, the E-GraphSAGE (Edge-GraphSample and Aggregate) model was proposed, improving the GraphSAGE algorithm by incorporating edge information during node representation learning to enhance the graph's representation learning capabilities. Its core idea is to update the target node's representation by aggregating features from neighboring nodes. Meanwhile, by sampling a subset of neighboring nodes and aggregating features, information from different domains is accumulated to obtain a more comprehensive and richer node representation. This effectively detects complex data anomalies involving abnormal interactions between devices, which are difficult to detect using single-point methods. However, this model still suffers from problems such as feature decay, high computational resource consumption, and long training time. To enable the E-GraphSAGE model to be effectively deployed on edge gateway devices while improving detection capabilities, this invention proposes a self-attention graph pooling method. This method considers not only node features but also the features of connecting edges in the attention score calculation, thereby more accurately evaluating node importance. During the pooling process, key nodes are selectively retained based on the attention score and pooling rate, preserving key information of the graph structure while reducing the graph size and avoiding redundant computation.

[0039] In this embodiment, step S2 is executed on the IoT edge gateway. Edge gateways are typically equipped with more powerful CPUs and dedicated AI acceleration chips, capable of running more complex deep learning models. The purpose of this step is to perform more refined analysis on the "normal" data from the device, uncovering potential anomalies based on network topology. See also... Figure 3 As shown, it specifically includes: S21: Data Preprocessing and Graph Construction Symbolic data in data traffic often contains null values ​​or non-numeric fields. This paper uses one-hot encoding to convert such data into numerical data so that it can be used by the model in this paper. For category labels, normal traffic in the dataset is marked as 0, and abnormal traffic is marked as 1.

[0040] Data cleaning and standardization: The data received from the device is cleaned, and missing values ​​are handled. Non-numerical features (such as protocol type) are numerated using one-hot encoding. To address the dimensionality differences between different feature data and avoid imbalances in feature classification, this embodiment normalizes numerical features in the dataset, such as duration and byte count, using Min-Max Normalization. The Min-Max normalization method is used to linearly transform the original data, mapping it uniformly to the range [0,1].

[0041] Constructing a network topology graph: The source IP address and source port number in the data traffic are concatenated to generate a unique source node; the destination IP address and destination port number are concatenated to generate a unique destination node. Other characteristics of the data traffic, such as duration, number of bytes, and protocol type, are used as edge features connecting these two nodes. Data traffic can be divided into identifier data and feature data. Identifier data typically contains fields used to identify the communication source and destination, i.e., source / destination IP address and source / destination port number; feature data includes traffic duration, number of bytes, and size of transmitted data packets. To prevent source IP information from biasing the model, a random hash mapping can be performed on the source IP. To prevent source node information from affecting the labeling of malicious traffic, the source IP address is randomly mapped to improve the model's generalization ability. In this way, multiple data traffic records over a period of time are constructed into a network topology graph G=(V,E), where V is the set of nodes and E is the set of edges. Thus, the data anomaly detection problem is formalized as a task of classifying the edges in this network topology graph G (i.e., determining whether the traffic represented by each edge is normal or abnormal).

[0042] S22: Self-attention graph pooling: To reduce the computational complexity of graph neural networks and enable their efficient operation in edge gateways, this invention designs an edge feature graph pooling method based on an attention score selection mechanism. This method considers not only node features but also edge features, incorporating them into the attention mechanism to more accurately evaluate node importance and achieve graph sparsity compression. (Refer to...) Figure 4 The process is as follows: Edge feature projection: To effectively integrate edge features into the attention mechanism and overcome the shortcomings of traditional methods, edge features e are projected... uv Project the linear layer onto a space with the same dimension as the node feature X: , Where, d e Let d be the edge feature dimension. x W represents the node feature dimension. e and b e These are trainable parameters; Gating mechanism calculation: This step captures the core information of edge features through nonlinear mapping, while avoiding the dimensionality explosion problem caused by subsequent concatenation operations. Based on the projected edge features... and adjacent node features X v , The key edge features are adaptively selected using a gating mechanism, which serves as the gating value g for each edge. uv : , in, For the Sigmoid function, W g and b g For trainable parameters, the gate value g uv ∈[0,1] Quantized edge feature e uv The importance of node v.

[0043] Weighted edge features: using the gate value g uv For the projected edge features We perform weighted analysis to obtain weighted edge features. ; Generating node importance scores: Weighted edge features are combined with node features, and the importance of edge features is dynamically adjusted through a gating mechanism. Gated edge features from neighboring nodes are then aggregated using graph convolution to generate a node importance score Z containing edge information. The specific formula is as follows: Where A = A + I is the adjacency matrix with added self-loops, I is the identity matrix, and D is the corresponding degree matrix. This represents the aggregated gated edge feature matrix. These are the trainable parameters for the attention layer.

[0044] Pooling to generate a subgraph: Based on the node importance score Z and the preset pooling ratio, select [kN] key nodes and their associated edge features to generate a pooled subgraph. , subgraph after pooling Only the features of highly important nodes and their associated edges are retained.

[0045] Here, Z represents the vector of importance scores for all nodes after the self-attention mechanism. Each node has a score indicating its importance in the graph. `rank(Z,kN)` sorts the node importance score vector Z and selects the top kN nodes. Here, k is a preset pooling ratio (e.g., k=0.5 means retaining 50% of the nodes), and N is the total number of nodes in the original graph. Therefore, kN represents the number of nodes to be retained. `top-rank(Z,kN)` is an indexing operation that extracts the indices corresponding to the top kN nodes from the importance score vector Z. These indices point to the nodes with the highest importance that need to be retained. This step determines which nodes need to be retained during the pooling process. It selects the kN nodes with the highest importance, which is the core basis for achieving graph sparsity. out X is the node feature matrix of the subgraph after pooling, containing feature information of all selected and retained nodes. idx It is a subset of the feature matrices of all nodes in the original graph, containing the original features of the nodes pointed to by the idx index. ⊙ represents element-wise multiplication. Z mask It is a mask vector whose length is equal to the original number of nodes N. For the selected node in idx (i.e., the node with high importance), its corresponding Z mask The element value is 1; for unselected nodes, their Z-axis value is 1. mask The element's value is 0. A out It is the adjacency matrix of the pooled subgraph, which describes the preserved connections between nodes. A idx E is a submatrix of the original graph adjacency matrix A, containing the original connections between the nodes pointed to by the idx index. out E is the edge feature matrix of the pooled subgraph, containing feature information of the edges connecting all the selected nodes. idxIt is a submatrix of the original graph edge feature matrix E, containing the original features of the edges connecting the nodes pointed to by the idx indices. The core of the above formula lies in: calculating the importance score Z of each node through a self-attention mechanism, then selecting the most important kN nodes (idx) according to a preset pooling ratio k, and constructing a new, smaller subgraph G' based on this. This subgraph G' is composed of X... out (Node Features), A out (Adjacency relationship) and E out Composed of edge features, it retains the most important topological and feature information of the original graph, while significantly reducing the size of the graph and improving the efficiency of subsequent graph neural network computation.

[0046] This embodiment employs an edge feature graph pooling method based on an attention score selection mechanism. It considers not only node features but also the features of connecting edges in the attention score calculation, thereby more accurately assessing node importance. During the pooling process, the proposed method selectively retains key nodes based on the attention score and pooling rate, preserving crucial information about the graph structure while reducing the graph size and avoiding redundant computation.

[0047] S23: E-GraphSAGE Graph Neural Network Classification: The subgraph is input into the E-GraphSAGE graph neural network for edge feature sampling and aggregation. The node representation is updated through multi-layer neighborhood aggregation. The node features and edge features of the graph are input into the E-GraphSAGE graph neural network model for representation learning to learn its structural and contextual similarity or feature similarity.

[0048] Message Passing: E-GraphSAGE improves upon the standard GraphSAGE by aggregating not only the features of neighboring nodes but also the features of connecting edges in its message passing function. For node v, its neighbor is u, and its new representation is h. v Aggregate the representations h of all neighbors using an aggregation function (such as LSTM or mean function). u and the corresponding edge feature e uv This allows the node representation to encode information about its connections.

[0049] Edge classification: After iterative updates through multiple layers of E-GraphSAGE, the model learns high-quality node embeddings. For each edge (u,v), its final edge embedding can be obtained by concatenating the embedding representations of its two endpoints and the original features of the edge itself.

[0050] Output: The obtained edges are embedded into the input fully connected layer and the Softmax classifier to generate anomaly detection results for the data flow. That is, the embeddings of all edges are z... uvInput a fully connected layer, then pass it through a Softmax classifier to obtain the probability distribution of each edge belonging to normal (0) or abnormal (1). Based on the probability threshold, generate secondary anomaly detection results, that is, identify which data traffic is abnormal.

[0051] S24: Gateway decision: If an abnormal edge is found in the secondary anomaly detection result, the edge gateway will report the corresponding abnormal event information (including source / target device, time, anomaly type, etc.) to the cloud.

[0052] In this embodiment, the collaborative detection mechanism based on the device and gateway in step S3 includes: The system acquires the initial anomaly detection result from the IoT device and determines whether the result is abnormal. If the initial anomaly detection result is abnormal, the current anomaly event information is directly reported from the IoT device to the cloud. If the initial anomaly detection result is normal, the current data traffic from the IoT device is sent to the IoT edge gateway for secondary anomaly detection. If the secondary anomaly detection result is abnormal, the current anomaly event information is reported from the edge gateway to the cloud. This collaborative mechanism of "rapid interception at the device end and in-depth analysis at the gateway end" ensures a balance between real-time performance and accuracy, and minimizes network transmission and cloud load.

[0053] In this embodiment, step S4 involves receiving abnormal event information reported by the device and gateway via the cloud, and then providing alarm notifications and data analysis. The cloud platform, as the final management and presentation center, is responsible for receiving all abnormal event information from the device and gateway. Upon receiving an abnormal event, the cloud immediately sends an alarm to the network administrator through various channels (such as SMS, email, App push notifications, and management platform pop-ups). The alarm information includes key information such as the time of the anomaly, the device / gateway involved, and the anomaly type (device-level / gateway-level). The cloud stores all reported abnormal events in a database, forming a security event log. Administrators can query historical events and perform statistical analysis (such as calculating the frequency of anomalies within a certain time period and the most frequently attacked device types) through a visual interface. Furthermore, the powerful computing resources of the cloud can be used to conduct in-depth mining and correlation analysis of the long-term accumulated abnormal data to discover potential attack trends and new attack patterns. Based on this, the Gaussian Bayes model parameters of the device and the graph neural network model of the gateway are updated and distributed regularly, forming a continuously learning and evolving closed-loop security protection system.

[0054] Based on the same inventive concept, this embodiment also provides a device-gateway collaborative IoT data anomaly detection system, including: The device-side detection module is used for device-level anomaly detection algorithms based on Gaussian Bayes models to perform localized preliminary anomaly detection on raw data collected in real time from IoT devices and generate an anomaly detection result. The gateway detection module is used to construct a data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network. It performs secondary anomaly detection on normal data received from IoT devices at the edge gateway. The secondary anomaly detection includes constructing the normal data into a network topology graph and encoding the anomaly detection problem into an edge classification task to generate secondary anomaly detection results. The collaborative reporting module is used to report the first anomaly detection result detected by the device and the second anomaly detection result detected by the gateway to the cloud based on the collaborative detection mechanism between the device and the gateway. The cloud platform processing module receives abnormal event information reported by devices and gateways via the cloud, and performs alarm prompts and data analysis. This embodiment applies an anomaly detection system, which includes an IoT device layer, an edge gateway layer, a network layer, and an application layer. Device layer: Terminal devices collect data from the environment through sensors and actuators and perform simple on-device calculations. Edge gateway layer: Terminal devices connect to the edge gateway via interfaces / protocols, and the gateway performs calculations, processing, and analysis on the data from the devices. Network layer: Devices and gateways connect to the application platform based on wireless / wired network protocols for data exchange. Application layer: The application platform is deployed on a cloud center server, receiving data from devices and gateways, and performing data calculations, analysis, and display.

[0055] It should be noted that the division of the various modules in this device / system embodiment is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can also be implemented entirely in hardware; or some units can be implemented by processing element calls to software, while others can be implemented in hardware.

[0056] The implementation principle of the above modules has been described in the foregoing method embodiments, so it will not be repeated here.

[0057] Based on the same concept, some embodiments of this application also provide an electronic device. This electronic device includes a memory and a processor, wherein the memory stores a processing program, and the processor executes the processing program according to instructions. When the processor executes the processing program, the device-gateway collaborative IoT data anomaly detection method described in the foregoing embodiments is implemented.

[0058] In some embodiments of this application, a readable storage medium is also provided, which can be a non-volatile readable storage medium or a volatile readable storage medium. The readable storage medium stores instructions that, when executed on a computer, cause an electronic device containing such a readable storage medium to perform the aforementioned device-gateway cooperative IoT data anomaly detection method.

[0059] It is understood that, for the aforementioned device-gateway collaborative IoT data anomaly detection methods, if all are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0060] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0061] The program code for executing the technical solutions disclosed in this application can be written in any combination of one or more programming languages. These programming languages ​​include object-oriented programming languages—such as Python and C++—and conventional procedural programming languages—such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A device-gateway collaborative IoT data anomaly detection method, characterized in that, include: A device-level anomaly detection algorithm based on a Gaussian Bayes model performs localized preliminary anomaly detection on raw data collected in real time from IoT devices and generates an anomaly detection result. A data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network is constructed. Normal data received from IoT devices is subjected to secondary anomaly detection at the edge gateway. The secondary anomaly detection includes constructing the normal data into a network topology graph and encoding the anomaly detection problem into an edge classification task to generate secondary anomaly detection results. Based on the collaborative detection mechanism between the device and the gateway, the first anomaly detection result detected by the device and the second anomaly detection result detected by the gateway are reported to the cloud. The system receives abnormal event information reported by devices and gateways via the cloud, and provides alarm prompts and data analysis.

2. The device-gateway collaborative IoT data anomaly detection method according to claim 1, characterized in that, The device-level anomaly detection algorithm based on the Gaussian Bayes model includes: Extract the number of data packets and bytes in IoT data traffic as data features, and construct a dataset data={x,y}, where X={x1,x2} represents the feature attributes, Y∈{0,1} represents the data label, 0 represents normal traffic, and 1 represents abnormal traffic; Using dataset D, calculate the prior probabilities for different categories, corresponding to the prior probabilities of normal and abnormal data flow, respectively; where, The prior probability of normal data traffic. This represents the prior probability of abnormal data traffic. Calculate the mean μ and variance σ of the Gaussian distribution for each feature under normal and abnormal data traffic conditions, respectively. 2 ; For the new sample data to be detected, the number of data packets and the number of bytes in the new sample data are extracted as data features. The conditional probability estimates of the data features under the normal class and the abnormal class are calculated according to the probability density function of the Gaussian distribution. ， Among them, y c This represents the category label, where 'c' represents the normal / abnormal category and takes the value 0 or 1, and 'x' represents the category label. i This represents the characteristics of real-time data collected by IoT devices, namely the number of bytes and the number of packets. According to Bayes' theorem, combining prior probability and conditional probability, the posterior probability of the new sample data to be detected belonging to each category is calculated: ， Among them, y i This indicates the category to which the current data sample belongs, x1 and x2 represent the data characteristics collected by IoT devices, and P(y i ) represents category y i The prior probability, P(x) j |y i ) represents category y i Time feature x j The conditional probability, P(x1,x2), represents the total probability of observing x1 and x2 across all categories; The calculated posterior probability of belonging to the normal class is compared with the posterior probability of belonging to the abnormal class. If the posterior probability of the abnormal class is greater than the posterior probability of the normal class, it is determined to be an abnormal data sample. For data samples identified as abnormal, the device will report information containing the event data of that sample to the cloud for alarm notification.

3. The device-gateway collaborative IoT data anomaly detection method according to claim 1, characterized in that, The data anomaly detection model constructed based on self-attention graph pooling and E-GraphSAGE graph neural network includes: The raw data traffic received by the edge gateway from multiple terminal devices is cleaned, and non-numerical features are converted into numerical data through one-hot encoding. At the same time, the numerical features are subjected to Min-Max normalization. The preprocessed data flow is mapped into a graph structure represented by node and edge features; An attention-score-based edge feature graph pooling method scores node importance and compresses the graph structure to generate a pooled subgraph that retains key topological information. The subgraph is input into the E-GraphSAGE graph neural network for edge feature sampling and aggregation, and the node representation is updated through multi-layer neighborhood aggregation; The obtained edges are embedded into the input fully connected layer and the Softmax classifier to generate anomaly detection results for data traffic.

4. The device-gateway collaborative IoT data anomaly detection method according to claim 1, characterized in that, The raw data includes identification data and feature data. The identification data includes fields used to identify the communication source and destination, namely the source / destination IP address and the source / destination port number. The feature data includes the traffic duration, number of bytes, and size of the transmitted data packet.

5. The device-gateway collaborative IoT data anomaly detection method according to claim 1, characterized in that, The self-attention graph-based pooling includes: edge feature e uv Project the linear layer onto a space with the same dimension as the node feature X: ， Where, d e Let d be the edge feature dimension. x W represents the node feature dimension. e and b e These are trainable parameters; Based on the projected edge features and adjacent node features X v The key edge features are adaptively selected using a gating mechanism as the gating value g for each edge. uv : Using the gate value g uv For the projected edge features We perform weighted analysis to obtain weighted edge features. ; The weighted edge features are combined with node features, the importance of edge features is dynamically adjusted through a gating mechanism, and the gating edge features of neighboring nodes are aggregated through graph convolution to generate a node importance score containing edge information. Key nodes and associated edge features are selected based on the node importance score Z and the preset pooling ratio to generate a pooled subgraph.

6. The device-gateway collaborative IoT data anomaly detection method according to claim 5, characterized in that, The gating mechanism is implemented using the following formula: ， in, For the Sigmoid function, W g and b g For trainable parameters, the gate value g uv ∈[0,1] Quantized edge feature e uv The importance of node v.

7. The device-gateway collaborative IoT data anomaly detection method according to claim 1, characterized in that, The collaborative detection mechanism based on the device and gateway includes: Obtain an anomaly detection result from an IoT device and determine whether the result is abnormal; If an anomaly detection result is abnormal, the current anomaly event information will be directly reported from the IoT device to the cloud. If the first anomaly detection result is normal, the current data traffic from the IoT device is sent to the IoT edge gateway for a second anomaly detection. If the secondary anomaly detection result is abnormal, the current anomaly event information will be reported to the cloud from the edge gateway.

8. A device-gateway collaborative IoT data anomaly detection system, characterized in that, include: The device-side detection module is used for device-level anomaly detection algorithms based on Gaussian Bayes models to perform localized preliminary anomaly detection on raw data collected in real time from IoT devices and generate an anomaly detection result. The gateway detection module is used to construct a data anomaly detection model based on self-attention graph pooling and E-GraphSAGE graph neural network. It performs secondary anomaly detection on normal data received from IoT devices at the edge gateway. The secondary anomaly detection includes constructing the normal data into a network topology graph and encoding the anomaly detection problem into an edge classification task to generate secondary anomaly detection results. The collaborative reporting module is used to report the first anomaly detection result detected by the device and the second anomaly detection result detected by the gateway to the cloud based on the collaborative detection mechanism between the device and the gateway. The cloud platform processing module is used to receive abnormal event information reported by the device and gateway through the cloud, and to provide alarm prompts and data analysis.

9. An electronic device, characterized in that, include: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the device-gateway collaborative IoT data anomaly detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the device-gateway collaborative IoT data anomaly detection method according to any one of claims 1-7.

Images