A method and system for monitoring Internet of Things (IoT) network faults

Abstract

The application discloses a kind of Internet of Things network fault monitoring systems, including the following modules: hierarchical acquisition module, data preprocessing module, adaptive threshold module, abnormal analysis module, fault location module, hierarchical response module, closed-loop optimization module, the application also discloses a kind of Internet of Things network fault monitoring method, including the following steps: hierarchical data acquisition, data preprocessing and feature screening, adaptive threshold generation, abnormal clustering and fault preliminary judgment, fault mode matching and accurate positioning, hierarchical response and closed-loop optimization.In the application, the threshold is dynamically adjusted by combining historical data with real-time network load, and the threshold validity is verified with the device type weight correction, which effectively improves the threshold adaptability and fault determination accuracy, and then through K-means clustering, the abnormal classification, structured fault mode library and weighted matching algorithm are cooperatively determined, and the location identification information is accurately positioned, which greatly improves the accuracy of abnormal analysis and fault location.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) technology, and in particular to an IoT network fault monitoring method and system. Background Technology

[0002] With the widespread adoption of IoT technology, its applications have covered multiple fields such as smart homes, industrial control, smart parks, and intelligent transportation, forming a large-scale network system consisting of massive terminal devices, multi-level gateways, and complex transmission links.

[0003] Most existing detection methods use fixed thresholds for fault determination, without considering the dynamic changes in the load of IoT networks. For example, when the network load increases significantly during peak industrial production periods, indicators such as transmission delay and bandwidth utilization will fluctuate naturally, and fixed thresholds are prone to causing a large number of false alarms. During low-load periods, some minor abnormal data may not be detected because the threshold is too high, resulting in missed faults. Therefore, these methods cannot meet the monitoring needs of different operating scenarios.

[0004] Accordingly, this application proposes a method and system for monitoring Internet of Things (IoT) network faults. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method and system for monitoring Internet of Things (IoT) network faults.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] An Internet of Things (IoT) network fault monitoring system includes the following modules:

[0008] The hierarchical acquisition module consists of a terminal acquisition unit, a gateway acquisition unit, and a cloud acquisition unit. It collects device operation data and link status data respectively, and summarizes terminal data and transmission performance data. It supports dynamic configuration of acquisition frequency.

[0009] The data preprocessing module includes a standardization processing unit and a feature filtering unit, which respectively realize the unification of multi-dimensional data units, the filtering of core features, and the removal of redundant data.

[0010] The adaptive threshold module includes a historical data storage unit and a threshold dynamic adjustment unit, which respectively store nearly 30 days of normal operation data and calculate and dynamically adjust the threshold.

[0011] The anomaly analysis module includes a threshold comparison unit and a K-means clustering unit, which respectively mark abnormal data points, perform cluster analysis, and determine suspected faults.

[0012] The fault location module includes a fault mode library unit and a matching location unit, which respectively store fault modes and feature combination rules, matching features, and locate faults.

[0013] The graded response module includes an alert information generation unit, a push unit, and a self-healing trigger unit, which respectively generate alert information, push information, and execute a minor fault self-healing strategy.

[0014] The closed-loop optimization module includes a data recording unit and a parameter optimization unit, which respectively store fault handling data and update the mode library and periodically optimize monitoring parameters.

[0015] A method for detecting faults in an Internet of Things (IoT) network includes the following steps:

[0016] Layered data acquisition: Construct a three-level acquisition architecture of terminal, gateway, and cloud. The terminal layer collects CPU utilization, memory usage, power supply current, and operating temperature. The gateway layer collects signal strength, link error rate, and connection stability duration. The cloud layer summarizes transmission rate, latency, packet loss rate, and bandwidth usage. The acquisition frequency is dynamically adjusted according to the data type. During the acquisition process, the sliding window mean method is used to remove instantaneous fluctuation data.

[0017] Data preprocessing and feature selection: The collected data is standardized, and Pearson correlation coefficient is used to select core features that are strongly correlated with faults and remove redundant data;

[0018] Adaptive threshold generation: Initial thresholds for each core feature are determined based on historical normal operation data, and the thresholds are dynamically adjusted in combination with real-time network load to generate an adaptive threshold set;

[0019] Anomaly clustering and initial fault assessment: Real-time data is compared with an adaptive threshold set, anomaly data points are marked and classified using the K-means clustering algorithm, and suspected faults are determined by combining the anomaly consistency of continuous collection cycles.

[0020] Fault mode matching and precise location: Construct a fault mode library containing equipment faults, link faults, and transmission faults. Combine the core features of suspected faults with the mode library to confirm the fault type, and combine the location identification information to locate the fault node.

[0021] Tiered response and closed-loop optimization: Generate tiered warning information based on fault level, record fault handling data and update the fault mode library, and optimize monitoring parameters regularly.

[0022] Preferably, the hierarchical data acquisition sets the acquisition frequency according to the differences in data types, with the acquisition frequency of equipment operation, link status, and transmission performance data increasing sequentially; the sliding window mean method has a window containing multiple acquisition periods, and takes the arithmetic mean of the target data point and its adjacent data of the same type as the final valid data at that time point.

[0023] Preferably, the data preprocessing adopts Min-Max standardization to map the data to a fixed numerical range, and the Pearson correlation coefficient for feature selection is set to a preset threshold. The core features retained include CPU utilization, memory usage, link error rate, transmission delay, and packet loss rate.

[0024] Preferably, in the adaptive threshold generation, the initial threshold is determined using the quantile method based on historical normal operation data for a preset period of time; the network operating load is the ratio of the number of currently online devices to the maximum number of supported devices, and the threshold is adjusted according to the load range: the threshold is lowered in the low load range, the threshold remains unchanged in the medium load range, and the threshold is raised in the high load range.

[0025] Preferably, the number of clusters in the K-means clustering algorithm is set to three, corresponding to mild anomalies, moderate anomalies, and severe anomalies, respectively, and the cluster distance is Manhattan distance; at the same time, the initial cluster centers are determined by the elbow rule, and the iteration termination condition is reaching a preset number of times or the cluster center offset is less than a preset value; when moderate or above anomalies occur in multiple consecutive collection cycles, they are judged as suspected faults.

[0026] Preferably, the fault mode library is constructed by: combining historical log extraction, operation and maintenance record summarization, and simulated fault test to collect sufficient fault data, including core features, fault types, and processing results, and using decision tree algorithm to train feature combination rules to construct a structured mode library that supports dynamic updates;

[0027] Fault matching employs a weighted matching algorithm, assigning weights based on the correlation coefficient between features and faults. If the matching degree reaches a preset value, the fault type is confirmed. Accurate fault node location is achieved by combining terminal ID, gateway ID, and link segment number.

[0028] Preferably, the graded response strategy is as follows: minor faults trigger a self-healing process; if data recovers after multiple monitoring cycles, the alarm is turned off; if it does not recover, it is upgraded to a moderate fault; moderate faults are pushed to the operation and maintenance platform; severe faults are simultaneously pushed to the operation and maintenance platform and the mobile terminals of operation and maintenance personnel.

[0029] Preferably, the network load calculation also introduces device type weight correction: terminals are divided into key devices and ordinary devices, and the weighted online device count to the total weighted count is calculated according to different weights, which is used as the final load value.

[0030] Preferably, the feature selection and closed-loop optimization rules are as follows: if the Pearson correlation coefficient of a certain data dimension is lower than the preset lower limit for multiple consecutive optimization cycles, it is removed from the collection list;

[0031] If the correlation coefficient of a newly added dimension consistently meets the standard, it is included in the core feature set; closed-loop optimization incorporates threshold validity verification, statistically analyzes false positive and false negative rates, and if the upper limit is exceeded, the quantile value is adjusted and the initial threshold is recalculated.

[0032] The present invention has the following beneficial effects:

[0033] 1. This invention dynamically adjusts thresholds by combining historical data with real-time network load, and combines this with device type weight correction and threshold validity verification to effectively improve threshold adaptability and fault determination accuracy. Furthermore, it achieves anomaly classification through K-means clustering, collaborative judgment with a structured fault pattern library and weighted matching algorithm, and precise location based on location identification information, significantly improving the accuracy of anomaly analysis and fault location. On this basis, through a graded response mechanism, a self-healing strategy for minor faults, and a closed-loop optimization mechanism, it significantly enhances the timeliness of fault response and the long-term stability of the system. This reduces invalid alarms and lowers the maintenance burden, while enabling rapid self-healing of minor faults and preventing the escalation of small faults. It also adapts to changes in network scale, device status, and scenarios, ensuring long-term stable operation of the system.

[0034] 2. This invention constructs a three-level acquisition architecture of terminal, gateway, and cloud and dynamically adapts the acquisition frequency. Combined with the sliding window mean method to filter out interference data, it significantly enhances the relevance and effectiveness of data acquisition, avoids invalid data occupying transmission and storage resources, and provides a high-quality data foundation for subsequent fault analysis.

[0035] 3. This invention unifies the data dimensions through Min-Max standardization and uses Pearson correlation coefficient to accurately screen core features, greatly optimizing the efficiency of data processing and feature screening, avoiding redundant data interference, and providing a clear basis for fault diagnosis. Attached Figure Description

[0036] Figure 1 This is an overall flowchart of the steps in the IoT network fault monitoring method proposed in this invention;

[0037] Figure 2 This is an overall flowchart of an Internet of Things (IoT) network fault monitoring system proposed in this invention;

[0038] Figure 3 This is a flowchart of an IoT network fault monitoring method proposed in this invention. Detailed Implementation

[0039] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0040] An Internet of Things (IoT) network fault monitoring system includes the following modules:

[0041] The hierarchical acquisition module consists of a terminal acquisition unit, a gateway acquisition unit, and a cloud acquisition unit. It collects device operation data and link status data respectively, and summarizes terminal data and transmission performance data. It supports dynamic configuration of acquisition frequency.

[0042] The data preprocessing module includes a standardization processing unit and a feature filtering unit, which respectively realize the unification of multi-dimensional data units, the filtering of core features, and the removal of redundant data.

[0043] The adaptive threshold module includes a historical data storage unit and a threshold dynamic adjustment unit, which respectively store nearly 30 days of normal operation data and calculate and dynamically adjust the threshold.

[0044] The anomaly analysis module includes a threshold comparison unit and a K-means clustering unit, which respectively mark abnormal data points, perform cluster analysis, and determine suspected faults.

[0045] The fault location module includes a fault mode library unit and a matching location unit, which respectively store fault modes and feature combination rules, matching features, and locate faults.

[0046] The graded response module includes an alert information generation unit, a push unit, and a self-healing trigger unit, which respectively generate alert information, push information, and execute a minor fault self-healing strategy.

[0047] The closed-loop optimization module includes a data recording unit and a parameter optimization unit, which respectively store fault handling data and update the mode library and periodically optimize monitoring parameters.

[0048] A method for monitoring faults in an Internet of Things (IoT) network includes the following steps:

[0049] Layered data acquisition: A three-tiered acquisition architecture is constructed, consisting of terminals, gateways, and the cloud. The terminal layer collects CPU utilization, memory usage, power supply current, and operating temperature; the gateway layer collects signal strength, link error rate, and connection stability duration; and the cloud layer aggregates transmission rate, latency, packet loss rate, and bandwidth usage. In this layered data acquisition, the acquisition frequency is set differently according to data type, with the acquisition frequency of device operation data, link status data, and transmission performance data increasing sequentially. The sliding window mean method uses a window size of multiple acquisition periods. Specifically, for each data point, the arithmetic mean of the same type of data in its own acquisition period and adjacent acquisition periods is calculated as the final valid data for that acquisition time point.

[0050] Data preprocessing and feature selection: The collected data is standardized, and Pearson correlation coefficient is used to select core features strongly correlated with faults, eliminating redundant data. Min-Max standardization is used in data preprocessing to map the data to a fixed numerical range. The Pearson correlation coefficient for feature selection is set to a preset threshold. If the Pearson correlation coefficient of a certain data dimension is lower than the preset lower limit for multiple consecutive optimization cycles, it is removed from the collection list. If the Pearson correlation coefficient of a newly added data dimension reaches the core feature selection threshold for multiple consecutive optimization cycles, it is included in the core feature set. The newly added data dimension includes the remaining storage space of the device. The validity of the threshold is verified simultaneously during the closed-loop optimization process. The false alarm rate and false negative rate of the threshold are counted in each optimization cycle. If the false alarm rate or false negative rate exceeds the corresponding upper limit, the quantile value of the quantile method is adjusted and the initial threshold is recalculated. The core features retained include CPU utilization, memory usage, link bit error rate, transmission delay, and packet loss rate.

[0051] Adaptive threshold generation: Initial thresholds for each core feature are determined based on historical normal operation data, and the thresholds are dynamically adjusted in conjunction with real-time network load to generate an adaptive threshold set. In the adaptive threshold generation, the initial thresholds are determined using the quantile method based on historical normal operation data for a preset period. The network load is the ratio of the number of currently online devices to the maximum number of supported devices. The thresholds are adjusted according to the load range: the thresholds are lowered in the low load range, kept unchanged in the medium load range, and raised in the high load range.

[0052] The calculation of network load also includes device type weight adjustment: terminal devices are divided into key devices and ordinary devices according to their functions. Key devices include industrial control sensors and security cameras, while ordinary devices include environmental temperature and humidity sensors. Different weights are set for key devices and ordinary devices respectively. The network load is calculated as the ratio of the weighted number of online devices to the total number of weighted devices.

[0053] Anomaly Clustering and Initial Fault Assessment: Real-time data is compared with an adaptive threshold set to mark abnormal data points and classify them using the K-means clustering algorithm. The number of clusters in the K-means clustering algorithm is set to three, corresponding to mild, moderate, and severe anomalies, respectively. The Manhattan distance is used for clustering distance. At the same time, the initial cluster centers are determined by the elbow rule. The clustering iteration terminates when a preset number of iterations is reached or the cluster center offset is less than a preset value. Based on the above clustering results, when moderate or higher-level abnormal data points appear and the same type of anomaly exists in multiple consecutive collection cycles, it is judged as a suspected fault.

[0054] Fault Mode Matching and Precise Location: A fault mode library containing equipment faults, link faults, and transmission faults is constructed. The construction process of the fault mode library is as follows: Sufficient historical fault data is collected by combining extraction of historical operation logs of the IoT network, summarization of operation and maintenance records, and collection of data through simulated fault scenario tests. Each set of data includes core feature parameters, fault type, and processing results. Then, a decision tree algorithm is used to train the above historical fault data to extract feature combination rules. Specifically, CPU overload faults can be determined based on the standardized value of CPU utilization and the continuous collection cycle, and link signal interference faults can be determined based on the standardized value of link bit error rate and signal strength. A structured fault mode library is constructed based on the extracted feature combination rules, and this mode library supports dynamic updates. In the fault mode matching stage, a weighted matching algorithm is used to assign corresponding weights to each core feature according to the correlation coefficient between the core feature and the fault. When the matching degree reaches a preset value, the fault type is confirmed. At the same time, combined with location identification information including terminal device ID, gateway ID, and transmission link segment number, the fault node is accurately located.

[0055] Tiered Response and Closed-Loop Optimization: Tiered warning information is generated based on the fault level. For minor faults where the standardized value of the core feature exceeds the threshold but does not meet the criteria for moderate anomaly, a dedicated self-healing trigger process is executed, automatically triggering a preset self-healing strategy. This strategy specifically includes clearing terminal device memory, switching gateway links, and dynamically allocating transmission bandwidth. After self-healing, continuous monitoring is performed for multiple data collection cycles. If the data returns to normal, the alarm is turned off; otherwise, it is upgraded to a moderate fault. Moderate faults are pushed to the operation and maintenance platform, while severe faults are simultaneously pushed to the mobile terminals of operation and maintenance personnel. Fault handling data is recorded and the fault mode library is updated. Monitoring parameters are optimized regularly.

[0056] Example 1: Taking a smart park IoT network as a specific application scenario, the network covers 1,200 terminal devices (including 200 industrial control sensors, 300 security cameras, and 700 environmental temperature and humidity sensors), 25 edge gateways, and 1 cloud server. The core functions are monitoring the operating status of park equipment, security early warning, and environmental regulation.

[0057] A layered data acquisition system is implemented, constructing a three-tiered acquisition architecture of terminal, gateway, and cloud. The acquisition frequency is set according to the differences in data types, with device operation data acquisition frequency at 1 time / minute, link status data acquisition frequency at 5 times / minute, and transmission performance data acquisition frequency at 10 times / minute, ensuring that core data acquisition is timely and without redundancy.

[0058] Terminal Layer Data Acquisition: For terminal devices such as industrial control sensors, security cameras, and temperature and humidity sensors, four types of operational data are collected for each device: CPU utilization, memory usage, power supply current, and operating temperature. CPU utilization ranges from 0% to 100%, memory usage from 0% to 100%, power supply current from 0A to 5A, and operating temperature from -10℃ to 60℃. Each terminal device has a built-in acquisition module that periodically uploads data to its corresponding edge gateway. Gateway Layer Data Acquisition: Each edge gateway collects three types of link status data in real time: signal strength, link error rate, and connection stability duration. Signal strength ranges from -100dBm to 0dBm, link error rate from 0 to 1, and connection stability duration is in seconds. The gateway also receives and caches the device operational data uploaded from the terminal layer. Cloud Layer Aggregation: Every 10 minutes, the cloud server aggregates four types of transmission performance data from each edge gateway: transmission rate, transmission latency, packet loss rate (range 0 to 1), and bandwidth usage, forming a comprehensive network data set.

[0059] During the data collection process, a sliding window mean method was used to remove transient fluctuations. The window size was set to 3 collection periods. The formula for calculating the sliding window mean is as follows:

[0060]

[0061] in, For the first The final valid data at any given moment. For the first The collected values ​​at each moment, For the first The collected values ​​at each moment, For the first The collected values ​​at any given time.

[0062] The specific operation is as follows: Taking the CPU usage data of a security camera as an example, if the data collected by the device in the 1st, 2nd, and 3rd minutes are 45%, 75%, and 48% respectively, then take the arithmetic mean of these three values: 56% was selected as the final valid data for the second minute to filter out interference values ​​caused by momentary device lag. Data preprocessing and feature filtering, Min-Max normalization: The Min-Max normalization algorithm was applied to all collected valid data to map the data to... Interval, eliminating differences in data units, calculation formula: ;

[0063] in, For standardized data, This is the original collected data. This is the minimum value for this data dimension. This represents the maximum value for this data dimension.

[0064] For example, the power supply current of an industrial control sensor is 3A. The maximum current of this type of sensor is 5A and the minimum current is 0A. The standardized value is: ;

[0065] A gateway has a signal strength of -70dBm, a maximum signal strength of 0dBm, and a minimum signal strength of -100dBm. The standardized value is: .

[0066] Pearson correlation coefficient feature selection: The preset threshold for the Pearson correlation coefficient is set to 0.7. Core features are selected by calculating the correlation coefficients between each data dimension and the occurrence of network failures. The formula for calculating the Pearson correlation coefficient is:

[0067]

[0068] in, The correlation coefficient is... For the sample size, For the first The values ​​of each data dimension This represents the mean of this data dimension. For the first The fault label value of each sample (1 for fault, 0 for normal). This represents the average value of the fault label.

[0069] Calculation example: Selecting historical data from the past 6 months, the correlation coefficient between CPU utilization and equipment downtime is 0.82, the correlation coefficient between link error rate and data transmission failure is 0.85, and the correlation coefficient between transmission delay and network lag is 0.78, all of which are higher than the threshold of 0.7; while the correlation coefficient between power supply current and fault is 0.41, and the correlation coefficient between operating temperature and fault is 0.35, both of which are lower than the threshold of 0.7.

[0070] Calculation example: Selecting historical data from the past 6 months, sample size The correlation coefficients between CPU utilization and device downtime were 0.82, between link error rate and data transmission failure was 0.85, and between transmission delay and network lag was 0.78, all of which were higher than the threshold of 0.7. However, the correlation coefficients between power supply current and faults were 0.41 and between operating temperature and faults were 0.35, both of which were lower than the threshold of 0.7.

[0071] Screening results: Five core features are retained: CPU utilization, memory usage, link error rate, transmission delay, and packet loss rate. Redundant data dimensions such as power supply current, operating temperature, signal strength, connection stability duration, transmission rate, and bandwidth usage are removed.

[0072] Initial threshold determination: Based on nearly 30 days of historical network operation data, the P95 quantile method was used to determine the initial thresholds for each core feature. The formula for calculating the quantile value is as follows: in, Quantity value , The rounding up symbol, The quantile ratio is 0.95 in this embodiment. For the sample size, For the sorted number Each data value.

[0073] Calculation Example: A total of 43,200 normal operation data points regarding transmission latency over the past 30 days were collected. This data volume was calculated by multiplying 25 gateways by 10 times / minute, 60 minutes, and 24 hours. After sorting the data from smallest to largest: .

[0074] The 41040th data value was taken as the initial threshold, which was calculated to be 80ms. Therefore, the initial threshold for transmission delay was set to 80ms. The initial threshold for CPU utilization was set to 70%, and the initial threshold for link error rate was set to 0.05.

[0075] Load weight adjustment calculation: Terminal devices are divided into critical devices and general devices according to their functions. Industrial control sensors and security cameras are critical devices, with a weight of 1.2; environmental temperature and humidity sensors are general devices, with a weight of 1.0. The network operating load calculation formula is as follows: Network operating load calculation formula:

[0076] in, For the number of equipment types, For the first The number of online devices of this type For the first The weight of the device class For the first The total number of devices of this type.

[0077] Calculation Example: During a certain period, the number of critical devices online is 450, derived from 180 industrial control sensors and 270 security cameras. The number of ordinary devices online is 650. Therefore, the load is:

[0078] Dynamic threshold adjustment: The thresholds for each core feature are adjusted based on the load range. The threshold adjustment calculation formula is as follows: The K-means clustering algorithm performs anomaly classification on the preprocessed core feature data. The specific parameters and execution rules are as follows: ,in, The adjusted threshold As the initial threshold, The adjustment ratio is adjusted (negative values ​​for downward adjustments, positive values ​​for upward adjustments). Specific adjustment rules: When the load is ≤30%, the threshold is lowered by 10%. If the initial transmission delay threshold is 80ms, it will be adjusted as follows: When 30% < load ≤ 70%, keep the initial threshold unchanged. When the load is >70%, the threshold is increased by 15%. If the initial transmission delay threshold is 80ms, it will be increased to: .

[0079] Anomaly clustering and initial fault assessment: The K-means clustering algorithm is used to classify anomalies in the preprocessed core feature data. Specific parameters and execution rules are as follows: Clustering parameter settings: The number of clusters is set to 3, corresponding to mild, moderate, and severe anomalies, respectively. The Manhattan distance is used for clustering distance, and the formula for calculating the Manhattan distance is:

[0080] in, For the sample With sample distance, The number of feature dimensions. For the sample The 1 eigenvalue, For the sample The Each feature value.

[0081] Clustering execution rules: Initial cluster centers are determined using the elbow rule. The clustering iteration termination condition is set to reach 50 iterations or the cluster center offset is less than 0.001. The formula for calculating the cluster center offset is:

[0082] in, This represents the total offset of the cluster centers. The number of clusters, The number of feature dimensions. For the first The iteration of the ... Cluster centers, For the first The iteration of the ... Cluster centers.

[0083] Two core feature data of a gateway, transmission delay and link error rate, were selected for clustering. The initial cluster centers were determined to be (20ms, 0.01), (50ms, 0.03), and (90ms, 0.06) using the elbow rule. After 32 iterations, the cluster center offset decreased to 0.0008, which met the termination condition, and the iteration was stopped.

[0084] Suspected fault determination: If there are moderate or higher abnormal data points in the clustering results, and the same type of abnormality exists in two consecutive collection cycles, it is determined to be a suspected fault.

[0085] If the CPU usage of a security camera shows moderate anomalies in the clustering results at the 5th and 6th minutes, the device is considered to have a suspected fault. If the CPU usage is only moderately abnormal in the 5th minute and returns to normal in the 6th minute, it is considered to be a momentary fluctuation and is not listed as a suspected fault.

[0086] Fault mode matching and precise location, and fault mode library construction: More than 5000 sets of historical fault data were collected through a combination of extracting historical operation logs from IoT networks, summarizing maintenance records, and collecting data from simulated fault scenario tests. Each set of data includes core feature parameters, fault type, and processing result. A decision tree algorithm was used to train the historical data and extract feature combination rules. Some of these rules are as follows:

[0087] Rule 1: A standardized CPU utilization value > 0.85 for three consecutive data collection cycles indicates a CPU overload fault. Rule 2: A standardized link bit error rate > 0.05 and signal strength < -70dBm indicates a link signal interference fault. Rule 3: A transmission delay > 92ms and packet loss rate > 0.08 indicates a network bandwidth congestion fault. A structured fault mode library is built based on these rules and supports dynamic updates based on new fault data.

[0088] Fault Type Weighted Matching: A weighted matching algorithm is used, assigning weights based on the correlation coefficients between core features and faults. Specifically, the weights are: link error rate 0.25, CPU utilization 0.22, packet loss rate 0.20, transmission delay 0.18, and memory usage 0.15. The weighted matching degree is calculated using the following formula: ,in, The total match score. The number of core features For the first The matching score of each feature (1 for a complete match, a proportional value for a partial match, and 0 for no match). For the first The weights of each feature.

[0089] For example, the core feature matching scores of a suspected faulty device are: link error rate 0.92, CPU utilization 0.91, packet loss rate 0.88, transmission delay 0.89, and memory usage 0.80. The total matching degree is:

[0090]

[0091] Match the device CPU overload fault in the pattern library and confirm the fault type.

[0092] Accurate fault node location: By combining the terminal device ID, gateway ID, and transmission link segment number, the fault node can be located. The matching result is a device CPU overload fault. The corresponding terminal device ID is CAM-012, the gateway ID is GW-05, and the transmission link segment number is LINK-05-03. Therefore, the faulty device can be directly located as security camera No. 12 under gateway No. 5, and the faulty link is the third transmission link from gateway No. 5 to the cloud.

[0093] Tiered response and closed-loop optimization for handling minor faults: For minor faults where the standardized value of core characteristics exceeds the threshold but does not meet the criteria for moderate anomalies, a preset self-healing strategy is automatically triggered. This strategy specifically includes clearing terminal device memory, switching gateway links, and dynamically allocating transmission bandwidth. After self-healing, continuous monitoring is performed for three data collection cycles. If the data returns to normal, the alarm is turned off; if it does not recover, the fault is escalated to a moderate fault.

[0094] For example, a temperature and humidity sensor with device ID SEN-215 experiences a minor fault. The system automatically and remotely clears the device's cache memory, switches its data transmission link from the original link to a backup link, and allocates an additional 5Mbps bandwidth to this link. After three monitoring cycles, the device's CPU utilization drops from 68% to 42%, returning to normal, and the alarm is disabled. Moderate fault handling: Moderate faults are pushed to the smart park operation and maintenance platform, where maintenance personnel can view fault details and perform remote troubleshooting and repair. Severe fault handling: Severe faults are simultaneously pushed to the maintenance personnel's mobile terminals, triggering an emergency repair process. Maintenance personnel must arrive on-site within 30 minutes to handle the situation.

[0095] Closed-loop optimization and feature selection optimization: If the Pearson correlation coefficient of a certain data dimension is lower than 0.65 for three consecutive optimization cycles, it is removed from the collection list; if a new data dimension is added, such as the remaining storage space of a device, and its Pearson correlation coefficient is ≥0.7 for two consecutive optimization cycles, it is included in the core feature set.

[0096] Threshold validity verification: The false positive rate and false negative rate of the threshold are calculated for each optimization cycle. The formulas for calculating the false positive rate and false negative rate are as follows:

[0097] ;

[0098] If the false alarm rate is greater than 8% or the false negative rate is greater than 5%, adjust the quantile values ​​of the quantile method, such as from P95 to P94 or P96, and recalculate the initial threshold. Pattern library update: Input the processing data for each fault and the newly extracted feature rules into the fault pattern library, and regularly optimize monitoring parameters to ensure the system adapts to changes in network operating status over the long term.

[0099] Example 2: Used for traffic facility operation monitoring, traffic flow scheduling and fault early warning. Except for the hierarchical data acquisition, anomaly clustering and initial fault judgment, fault mode matching and precise location steps, the remaining steps (data preprocessing and feature screening, adaptive threshold generation, hierarchical response and closed-loop optimization) are the same as in Example 1.

[0100] Layered Data Acquisition: Adopting a three-tier architecture (terminal, gateway, cloud) consistent with Example 1, adapting to the distributed deployment requirements of terminals such as traffic lights, cameras, and road condition sensors in intelligent transportation scenarios; Differentiated Acquisition Frequency Settings: Device operation data 2 times / minute, link status data 6 times / minute, and transmission performance data 12 times / minute; Employing the Kalman filter algorithm, data is iteratively corrected through state prediction and observation updates, filtering out instantaneous deviations caused by vehicle electromagnetic interference to ensure data reliability. Anomaly Clustering and Initial Fault Judgment: The DBSCAN clustering algorithm is used to adapt to the irregular distribution of fault data in traffic scenarios, setting a neighborhood radius and MinPts=5 to distinguish between light, moderate, and severe anomalies; the judgment criterion is the occurrence of moderate or higher anomalies in three consecutive acquisition cycles, balancing false alarms and missed alarms to ensure timely fault identification. Fault mode matching and precise location: More than 4,000 sets of data were collected through historical fault mining, import of vendor cases, and on-site simulation tests, covering core fault types; the random forest algorithm was used to train feature rules to improve generalization ability; the cosine similarity matching algorithm of Example 1 was used, and the similarity ≥90% was set as the confirmation threshold to reduce fault confusion; the fault location was combined with the traffic scenario-specific equipment number and link identifier to provide accurate basis for operation and maintenance and improve handling efficiency.

[0101] Example 3 is used for monitoring industrial production equipment, ensuring data transmission, and rapidly handling faults. Except for the hierarchical data acquisition, anomaly clustering and initial fault diagnosis, and fault mode matching and precise location steps, the rest is consistent with Example 1. Hierarchical data acquisition: It adopts the same three-level acquisition architecture as Example 1, adapting to the distributed deployment needs of production equipment such as machine tools, sensors, and controllers in industrial workshops; Differentiated acquisition frequency settings: equipment operation data once every 30 seconds, link status data 8 times / minute, and transmission performance data 15 times / minute; A median filtering algorithm is used, taking the median value of 5 consecutive acquisition cycles as valid data to filter out instantaneous distortion caused by motor vibration and ensure data reliability. Anomaly clustering and initial fault diagnosis: A fuzzy C-means clustering algorithm is used to adapt to the fuzzy boundary characteristics of fault data in industrial scenarios, calculating the anomaly probability through a membership function; The suspected fault judgment standard is a membership degree ≥0.8 for medium or higher anomalies, meeting the standard for two consecutive acquisition cycles, balancing production continuity and timely anomaly identification to ensure stable equipment operation. Fault mode matching and precise location: By extracting historical fault logs, recording operation and maintenance experience, and simulating extreme working conditions, more than 6,000 sets of data are collected to comprehensively cover core industrial fault types; feature rules are trained using support vector machine algorithm; Manhattan distance matching algorithm is used, with a distance ≤0.15 set as the confirmation threshold to accurately distinguish similar faults; combined with industrial equipment number, gateway location, and link identifier, precise location is achieved, helping operation and maintenance to quickly handle the situation and reduce production interruption time.

[0102] It should be noted that in the comparative embodiments, Comparative Example 1 abandons the layered architecture and full-process data processing, adopting a simplified monitoring mode of direct terminal connection to the cloud plus periodic inspections. Core process: Terminal devices connect directly to the cloud; the cloud checks the online status of terminals every 30 minutes; terminals upload snapshots of two basic parameters—device voltage and communication signal strength—every hour; the cloud only compares whether the parameters are within a fixed range, and determines a fault and issues a unified alarm if there is no response or the parameters exceed limits. There is no data filtering, feature analysis, anomaly classification, or closed-loop optimization; faults require on-site inspection by maintenance personnel. Comparative Example 2 adopts a passive alarm triggering plus manual verification monitoring approach, without actively collecting and analyzing data. Core process: Terminals only send alarms to the cloud when there is a power outage or hardware failure; the cloud only receives and stores alarm signals and pushes them to maintenance personnel, without automatic analysis and location; fault type confirmation and node location rely entirely on on-site inspection by maintenance personnel, without automated self-healing or optimization mechanisms. Specifically, as shown in Table 1:

[0103] Table 1: Performance Comparison of Examples and Comparative Examples

[0104] Comparison Projects Example 1 Example 2 Example 3 Comparative Example 1 Comparative Example 2 Fault identification accuracy (%) 95 94 93 75 80 Missed report rate (%) 4 5 6 15 12 Adaptability to scenarios with a load ≥ 70% Stable (performance degradation ≤4%) Stable (performance degradation ≤6%) Stable (performance degradation ≤5%) Unstable (frequent lag / packet loss) Unstable (significantly delayed alarms)

[0105] Specifically, in terms of accuracy, the fault identification accuracy of Example 1 (95%), Example 2 (94%), and Example 3 (93%) are all in the high-level range of above 93%, with a difference of only 2%. This indicates that the core solution based on hierarchical data collection, anomaly clustering, and pattern matching can achieve accurate fault identification in different scenarios. However, the accuracy of Comparative Example 1 (75%) and Comparative Example 2 (80%) is significantly lower, 13%-20% lower than the Example 1 group. The core reason is that the comparative examples lack key steps such as data filtering and feature analysis, and only judge faults through simple parameter comparison or passive alarms, which is easily affected by interference and leads to identification deviation.

[0106] In terms of false negative rate, the false negative rates of the implementation sample groups were all controlled at a low level of 4%-6%, with Implementation Sample 1 showing the best performance at 4% and Implementation Sample 3 at 6%. The low false negative rate can effectively avoid missing critical faults and ensure network operation security. The false negative rates of the comparative sample groups were significantly higher, reaching 15% in Comparative Sample 1 and 12% in Comparative Sample 2. This was mainly because the comparative samples lacked anomaly classification and continuous monitoring mechanisms, making it impossible to accurately capture hidden and intermittent faults, which posed serious security risks.

[0107] Specifically, when the network load is ≥70%, Examples 1 to 3 all maintain stable operation, with performance degradation controlled within 6%, and all three can meet the stable monitoring requirements under high load scenarios. This is due to the hierarchical acquisition architecture, dynamic threshold adjustment, and optimized clustering algorithm adopted by the Example 1 group, which can effectively cope with the data transmission pressure and interference under high load. In contrast, the two comparative examples are unstable under high load scenarios. Comparative example 1 experiences frequent stuttering and packet loss, while comparative example 2 has significant alarm delay. The core issue is that their simplified architecture and single monitoring logic cannot withstand high load pressure and are only suitable for low load and simple network environments.

[0108] The above-mentioned IoT network fault monitoring embodiments are all based on a three-level hierarchical data acquisition architecture of terminal, gateway, and cloud. Through hierarchical data acquisition, anomaly clustering and initial fault judgment, fault mode matching and precise location of core links, combined with differentiated algorithm selection and multi-source fault data training, a set of technical solutions has been formed that can achieve accurate fault identification, rapid location and efficient handling.

[0109] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An Internet of Things (IoT) network fault monitoring system, characterized in that, Includes the following modules: The hierarchical acquisition module consists of a terminal acquisition unit, a gateway acquisition unit, and a cloud acquisition unit. It collects device operation data and link status data respectively, and summarizes terminal data and transmission performance data. It supports dynamic configuration of acquisition frequency. The data preprocessing module includes a standardization processing unit and a feature filtering unit, which respectively realize the unification of multi-dimensional data units, the filtering of core features, and the removal of redundant data. The adaptive threshold module includes a historical data storage unit and a threshold dynamic adjustment unit, which respectively store nearly 30 days of normal operation data and calculate and dynamically adjust the threshold. The anomaly analysis module includes a threshold comparison unit and a K-means clustering unit, which respectively mark abnormal data points, perform cluster analysis, and determine suspected faults. The fault location module includes a fault mode library unit and a matching location unit, which respectively store fault modes and feature combination rules, matching features, and locate faults. The graded response module includes an alert information generation unit, a push unit, and a self-healing trigger unit, which respectively generate alert information, push information, and execute a minor fault self-healing strategy. The closed-loop optimization module includes a data recording unit and a parameter optimization unit, which respectively store fault handling data and update the mode library and periodically optimize monitoring parameters.

2. The monitoring method of the Internet of Things network fault monitoring system as described in claim 1, characterized in that, Includes the following steps: Layered data acquisition: Construct a three-level acquisition architecture of terminal, gateway, and cloud. The terminal layer collects CPU utilization, memory usage, power supply current, and operating temperature. The gateway layer collects signal strength, link error rate, and connection stability duration. The cloud layer summarizes transmission rate, latency, packet loss rate, and bandwidth usage. The acquisition frequency is dynamically adjusted according to the data type. During the acquisition process, the sliding window mean method is used to remove instantaneous fluctuation data. Data preprocessing and feature selection: The collected data is standardized, and Pearson correlation coefficient is used to select core features that are strongly correlated with faults and remove redundant data; Adaptive threshold generation: Initial thresholds for each core feature are determined based on historical normal operation data, and the thresholds are dynamically adjusted in combination with real-time network load to generate an adaptive threshold set; Anomaly clustering and initial fault assessment: Real-time data is compared with an adaptive threshold set, anomaly data points are marked and classified using the K-means clustering algorithm, and suspected faults are determined by combining the anomaly consistency of continuous collection cycles. Fault mode matching and precise location: Construct a fault mode library containing equipment faults, link faults, and transmission faults. Combine the core features of suspected faults with the mode library to confirm the fault type, and combine the location identification information to locate the fault node. Tiered response and closed-loop optimization: Generate tiered warning information based on fault level, record fault handling data and update the fault mode library, and optimize monitoring parameters regularly.

3. The IoT network fault monitoring method according to claim 2, characterized in that, Layered data acquisition sets the acquisition frequency according to the differences in data types, with the acquisition frequency of equipment operation, link status, and transmission performance data increasing sequentially; the sliding window mean method has a window containing multiple acquisition periods, and takes the arithmetic mean of the target data point and its adjacent data of the same type as the final valid data at that time point.

4. The IoT network fault monitoring method according to claim 2, characterized in that, Min-Max standardization was used in the data preprocessing to map the data to a fixed numerical range. The Pearson correlation coefficient for feature selection was set to a preset threshold. The core features retained included CPU utilization, memory usage, link error rate, transmission delay, and packet loss rate.

5. The IoT network fault monitoring method according to claim 2, characterized in that, In adaptive threshold generation, the initial threshold is determined using the quantile method based on historical normal operation data for a preset period of time; the network operating load is the ratio of the number of currently online devices to the maximum number of supported devices. The threshold is adjusted according to the load range: the threshold is lowered in the low load range, the threshold remains unchanged in the medium load range, and the threshold is raised in the high load range.

6. The IoT network fault monitoring method according to claim 2, characterized in that, The K-means clustering algorithm is set to three clusters, corresponding to mild anomalies, moderate anomalies, and severe anomalies, respectively. The cluster distance is Manhattan distance. At the same time, the initial cluster centers are determined by the elbow rule, and the iteration termination condition is reaching a preset number of times or the cluster center offset is less than a preset value. When moderate or above anomalies occur in multiple consecutive collection cycles, they are judged as suspected faults.

7. The IoT network fault monitoring method according to claim 2, characterized in that, The fault mode library is constructed by combining historical log extraction, operation and maintenance record summarization, and simulated fault test to collect sufficient fault data, including core features, fault types, and processing results. The decision tree algorithm is used to train the feature combination rules to build a structured mode library that supports dynamic updates. Fault matching employs a weighted matching algorithm, assigning weights based on the correlation coefficient between features and faults. If the matching degree reaches a preset value, the fault type is confirmed. Accurate fault node location is achieved by combining terminal ID, gateway ID, and link segment number.

8. The IoT network fault monitoring method according to claim 2, characterized in that, The tiered response strategy is as follows: minor faults trigger a self-healing process; if data recovers after multiple monitoring cycles, the alarm is turned off; if data does not recover, the fault is escalated to a moderate fault; moderate faults are pushed to the operations and maintenance platform. Severe faults are simultaneously pushed to the operation and maintenance platform and the mobile terminals of operation and maintenance personnel.

9. The IoT network fault monitoring method according to claim 2, characterized in that, The network load calculation also introduces device type weight correction: terminals are divided into key devices and ordinary devices, and the weighted online device count is calculated as the ratio of the total number of weighted devices to the total number of weighted devices, which is used as the final load value.

10. The method for monitoring Internet of Things (IoT) network faults according to claim 2, characterized in that, The feature selection and closed-loop optimization rules are as follows: if the Pearson correlation coefficient of a certain data dimension is lower than the preset lower limit for multiple consecutive optimization cycles, it will be removed from the collection list; If the correlation coefficient of a newly added dimension consistently meets the standard, it is included in the core feature set; closed-loop optimization incorporates threshold validity verification, statistically analyzes false positive and false negative rates, and if the upper limit is exceeded, the quantile value is adjusted and the initial threshold is recalculated.

Images