A method and system for analyzing SIM card traffic data in the Internet of Things

Abstract

The present invention discloses a method and system for analyzing SIM card traffic data for the Internet of Things. The method obtains SIM card traffic data from the system, extracts characteristic vectors of time sensitivity, packet discreteness, and protocol diversity from the SIM card traffic data, and obtains characteristic traffic data. The method then groups the characteristic traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data. A hybrid prediction model is established, and the grouped traffic data is input into the hybrid prediction model for analysis to obtain a traffic assessment result. The DTW dynamic time warping algorithm is used to determine the traffic assessment result. If an abnormality is determined, the abnormal cause of the abnormal traffic is analyzed, and abnormal links and traffic usage are blocked. This method significantly improves the accuracy and reliability of traffic prediction, providing an accurate prediction basis for Internet of Things traffic management.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to an Internet of Things SIM card traffic data analysis method and system. Background Art

[0002] Traditional IoT SIM card traffic analysis methods often rely on single statistical models or simple thresholds, making them ineffective in processing complex and volatile traffic data. Some methods identify anomalies solely by monitoring total traffic volume and usage frequency, failing to deeply analyze key characteristics such as time sensitivity, packet dispersion, and protocol diversity. This results in low accuracy in identifying anomaly traffic and is prone to misjudgments and missed detections. A single model is unable to comprehensively and accurately predict IoT SIM card traffic trends. Furthermore, in the anomaly detection phase, traditional methods rely on fixed thresholds or simple pattern matching, making them unable to adapt to dynamic changes in traffic patterns. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and design a method for analyzing SIM card traffic data of the Internet of Things.

[0004] To achieve the above-mentioned purpose, the technical solution of the present invention is as follows: further, in the above-mentioned method for analyzing the flow data of an Internet of Things SIM card, the method for analyzing the flow data of an Internet of Things SIM card comprises the following steps:

[0005] Acquire SIM card traffic data in the system, extract feature vectors of time sensitivity, data packet dispersion, and protocol diversity from the SIM card traffic data, and obtain feature traffic data;

[0006] The characteristic flow data is grouped based on an improved CURE clustering algorithm to obtain grouped flow data;

[0007] Establishing an ARIMA-Prophet-XGBoost hybrid prediction model, inputting the packet traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtaining a traffic evaluation result;

[0008] The traffic evaluation result is judged by using the DTW dynamic time warping algorithm. If it is judged to be an abnormal situation, the abnormal cause of the abnormal traffic is analyzed, and the abnormal link and traffic use are blocked.

[0009] Furthermore, in the above-mentioned method for analyzing SIM card traffic data of the Internet of Things, the steps of acquiring SIM card traffic data in the system, extracting feature vectors of time sensitivity, packet discreteness, and protocol diversity from the SIM card traffic data, and obtaining feature traffic data include:

[0010] Obtain SIM card traffic data from the system, define different time windows, calculate the traffic mean, traffic variance, and peak period ratio for each time window, and obtain the time sensitivity feature vector;

[0011] Analyze the distribution of the SIM card traffic data, calculate the data packet size standard deviation, data packet size coefficient of variation, adjacent data packet interval time, interval time mean and interval time variance, and obtain a data packet dispersion feature vector;

[0012] Identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, count the proportion of each protocol in the traffic, and obtain a protocol diversity feature vector.

[0013] Furthermore, in the above-mentioned method for analyzing traffic data of an IoT SIM card, the improved CURE clustering algorithm is used to group the characteristic traffic data to obtain grouped traffic data, including:

[0014] Based on the CURE clustering algorithm, a density-based representative point selection method is used to select points in high-density areas in each cluster as representative points;

[0015] The weighted Euclidean distance is introduced to assign different weights according to the importance of each feature in the feature vector;

[0016] In the process of clustering and merging data, the stability judgment of the merged cluster data is added to obtain the improved CURE clustering algorithm.

[0017] Furthermore, in the above-mentioned method for analyzing traffic data of an IoT SIM card, the improved CURE clustering algorithm is used to group the characteristic traffic data to obtain grouped traffic data, including:

[0018] The characteristic flow data is standardized by Min-Max standardization, and each characteristic value is converted into the same dimension to obtain standard flow data;

[0019] Using an improved representative point selection method, initial representative points are selected from the standard flow data, the initial number of clusters is set, and initial cluster flow data is obtained;

[0020] Calculating the distance between each cluster in the initial cluster traffic data, selecting the two clusters with the smallest distance to merge, updating the representative point of the merged cluster during the merging process, and recalculating the distance to other clusters according to the improved distance measurement method to obtain the second cluster traffic data;

[0021] Clustering iteration is performed based on the second cluster traffic data, and clustering is terminated when the number of clusters reaches a preset target number to obtain group traffic data.

[0022] Furthermore, in the above-mentioned method for analyzing traffic data of an IoT SIM card, the ARIMA-Prophet-XGBoost hybrid prediction model is established, and the packet traffic data is input into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain a traffic evaluation result, including:

[0023] Obtain the time series data in the system, use the ARIMA model to train the time series data, and obtain the ARIMA prediction results.

[0024] The ARIMA prediction results are used as input features and input into the Prophet model together with the original time series data for training. The Prophet model is used to process seasonality and trend to obtain the Prophet model prediction results;

[0025] The ARIMA prediction results, Prophet model prediction results and time series data are input into the XGBoost model for training to obtain the ARIMA-Prophet-XGBoost hybrid prediction model.

[0026] Furthermore, in the above-mentioned method for analyzing traffic data of an IoT SIM card, the method further includes establishing an ARIMA-Prophet-XGBoost hybrid prediction model, inputting the packet traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtaining a traffic evaluation result.

[0027] The grouped traffic data is input into the ARIMA-Prophet-XGBoost hybrid prediction model to perform time series analysis and feature extraction on the data;

[0028] Through the processing of the three models of ARIMA, Prophet and XGBoost, the traffic assessment results are output, including at least the traffic prediction value and the prediction confidence interval.

[0029] Furthermore, in the aforementioned method for analyzing IoT SIM card traffic data, the traffic evaluation result is judged using the DTW dynamic time warping algorithm. If an abnormal situation is determined, the abnormal cause of the abnormal traffic is analyzed, and the abnormal connection and traffic usage are blocked, including:

[0030] Convert the current traffic assessment results into time series data and use the DTW algorithm to calculate the similarity distance with each pattern in the normal traffic pattern library;

[0031] Setting a similarity threshold, when the similarity distance is greater than the similarity threshold, it is determined to be an abnormal situation;

[0032] When an abnormal situation is determined, the abnormal traffic is analyzed based on time sensitivity, packet discreteness and protocol diversity characteristics, and the cause of the abnormality is determined by combining the working status of the IoT device and the network environment information.

[0033] Furthermore, in an Internet of Things SIM card traffic data analysis system, the Internet of Things SIM card traffic data analysis system includes the following modules:

[0034] A data acquisition module is used to obtain SIM card traffic data in the system, extract feature vectors of time sensitivity, data packet discreteness and protocol diversity from the SIM card traffic data, and obtain feature traffic data;

[0035] A data classification module, configured to group the characteristic traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data;

[0036] A traffic evaluation module is used to establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the packet traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtain a traffic evaluation result;

[0037] The analysis and judgment module is used to judge the traffic evaluation result using the DTW dynamic time warping algorithm. If it is judged to be an abnormal situation, the abnormal cause of the abnormal traffic is analyzed and the abnormal link and traffic use are blocked.

[0038] Furthermore, in a system for implementing the above-mentioned method for analyzing SIM card traffic data of the Internet of Things, the data acquisition module includes the following submodules:

[0039] The acquisition submodule is used to obtain SIM card traffic data in the system, define different time windows, calculate the traffic mean, traffic variance and peak period ratio for each time window, and obtain the time sensitivity feature vector;

[0040] An analysis submodule is used to analyze the distribution of the SIM card traffic data, calculate the standard deviation of the data packet size, the coefficient of variation of the data packet size, the interval time between adjacent data packets, the mean of the interval time and the variance of the interval time, and obtain a data packet dispersion feature vector;

[0041] The identification submodule is used to identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT and CoAP, and count the proportion of each protocol in the traffic to obtain a protocol diversity feature vector.

[0042] Furthermore, in a system implementing the above-mentioned method for analyzing SIM card traffic data of the Internet of Things, the data classification module includes the following submodules:

[0043] The selection submodule is used to select points in high-density areas as representative points in each cluster using the density-based representative point selection method based on the CURE clustering algorithm;

[0044] A submodule is introduced to introduce weighted Euclidean distance, which assigns different weights to each feature according to its importance in the feature vector;

[0045] The judgment submodule is used to add a judgment on the stability of the merged cluster data during the process of clustering and merging the data, so as to obtain an improved CURE clustering algorithm.

[0046] Its beneficial effects include obtaining SIM card traffic data from the system, extracting feature vectors of time sensitivity, packet discreteness, and protocol diversity from the SIM card traffic data, and obtaining characteristic traffic data; grouping the characteristic traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data; establishing an ARIMA-Prophet-XGBoost hybrid prediction model, inputting the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtaining traffic assessment results; and using the DTW dynamic time warping algorithm to determine the traffic assessment results. If abnormal, the abnormal traffic is analyzed for its cause and abnormal links and traffic usage are blocked. The improved clustering algorithm improves distance metric accuracy and avoids unnecessary merging, making the clustering results more consistent with actual traffic distribution. This improved clustering algorithm rationally groups characteristic traffic data, providing structured data for subsequent prediction models and enhancing the model's adaptability and analytical capabilities for different traffic patterns. It effectively compensates for the shortcomings of single models, significantly improving the accuracy and reliability of traffic predictions and providing an accurate prediction basis for IoT traffic management. 3. Comprehensively consider similarity distance and traffic change amplitude to formulate anomaly judgment criteria, making anomaly detection more realistic and reducing misjudgments and missed detections. When an anomaly occurs, the cause is deeply analyzed from a multi-feature perspective, and targeted blocking measures are implemented on the device, network, and server sides to effectively prevent the spread of abnormal traffic, ensure the safe and stable operation of IoT devices and networks, and reduce the security risks and economic losses caused by traffic anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0048] Figure 1 This is a schematic diagram of a first embodiment of a method for analyzing SIM card traffic data of an Internet of Things according to an embodiment of the present invention;

[0049] Figure 2 Schematic diagram of a second embodiment of a method for analyzing SIM card traffic data of an Internet of Things according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of a first embodiment of an Internet of Things SIM card traffic data analysis system in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0053] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, a method for analyzing traffic data of an Internet of Things SIM card comprises the following steps:

[0054] Step 101: Acquire SIM card traffic data in the system, extract feature vectors of time sensitivity, packet discreteness, and protocol diversity from the SIM card traffic data, and obtain feature traffic data;

[0055] Specifically, in this embodiment, the SIM card traffic data in the system is obtained, different time windows are defined, and the traffic mean, traffic variance, and peak period ratio are calculated for each time window to obtain a time sensitivity feature vector;

[0056] Analyze the distribution of SIM card traffic data, calculate the standard deviation of data packet size, coefficient of variation of data packet size, interval time between adjacent data packets, mean interval time and variance of interval time, and obtain the data packet dispersion feature vector;

[0057] Identify the communication protocols used in SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, count the proportion of each protocol in the traffic, and obtain the protocol diversity feature vector.

[0058] Step 102: grouping the characteristic traffic data based on the improved CURE clustering algorithm to obtain grouped traffic data;

[0059] Specifically, in this embodiment, based on the CURE clustering algorithm, a density-based representative point selection method is used to select points in high-density areas in each cluster as representative points;

[0060] The weighted Euclidean distance is introduced to assign different weights according to the importance of each feature in the feature vector;

[0061] In the process of clustering and merging data, the stability judgment of the merged cluster data is added to obtain the improved CURE clustering algorithm.

[0062] The characteristic flow data is standardized by Min-Max standardization, and each characteristic value is converted into the same dimension to obtain the standard flow data;

[0063] Using the improved representative point selection method, the initial representative points are selected from the standard flow data, the initial cluster number is set, and the initial cluster flow data is obtained;

[0064] Calculate the distance between each cluster in the initial cluster traffic data, select the two clusters with the smallest distance to merge, update the representative point of the merged cluster during the merging process, and recalculate the distance to other clusters according to the improved distance measurement method to obtain the second cluster traffic data;

[0065] Clustering iteration is performed based on the second cluster traffic data, and clustering is terminated when the number of clusters reaches a preset target number to obtain group traffic data.

[0066] Step 103: Establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtain a traffic evaluation result;

[0067] Specifically, in this embodiment, the time series data in the system is obtained, and the time series data is trained using the ARIMA model to obtain the ARIMA prediction results.

[0068] The ARIMA forecast results are used as input features and input into the Prophet model together with the original time series data for training. The Prophet model is used to process seasonality and trend to obtain the Prophet model forecast results.

[0069] The ARIMA prediction results, Prophet model prediction results and time series data are input into the XGBoost model for training to obtain the ARIMA-Prophet-XGBoost hybrid prediction model.

[0070] The grouped traffic data is input into the ARIMA-Prophet-XGBoost hybrid forecasting model to perform time series analysis and feature extraction on the data;

[0071] Through the processing of the three models of ARIMA, Prophet and XGBoost, the traffic assessment results are output, including at least the traffic prediction value and the prediction confidence interval.

[0072] Feature Engineering: In addition to raw traffic features and ARIMA and Prophet prediction results, time features (such as hour, day, day of the week, etc.), lagged features (such as traffic values from the previous 1-7 days), and rolling statistical features (such as moving average, moving standard deviation, etc.) are also constructed as input.

[0073] Parameter optimization: Use random search or Bayesian optimization methods to adjust hyperparameters such as learning rate, tree depth, and subsampling rate to minimize the loss function (such as mean squared error MSE) on the validation set.

[0074] Applicable scenarios: It can capture nonlinear relationships and complex interaction effects in data, has good adaptability to high-dimensional and heterogeneous data, and is suitable for processing traffic prediction problems involving multiple influencing factors.

[0075] (1) Hybrid model establishment

[0076] Data preprocessing

[0077] Time series transformation: Arrange the grouped traffic data in chronological order to ensure uniform time intervals. For unevenly spaced data, resample using linear interpolation or spline interpolation.

[0078] Missing value handling

[0079] Short-term missing values: Linear interpolation is used to perform linear estimation based on the values of adjacent time points. For example, if the flow data for a certain hour is missing, the average flow rate of the two hours before and after is used as the fill value.

[0080] Long-term missing values: Use time series forecasting methods to train ARIMA or Prophet models using historical data to predict and fill missing values.

[0081] Outlier detection and processing

[0082] Z-score method: Calculate the Z-score (the number of standard deviations from the mean) of each data point, and consider data points with an absolute value of Z-score greater than 3 as outliers.

[0083] IQR method: Calculate the interquartile range IQR = Q3-Q1, and consider data points smaller than Q1-1.5IQR or larger than Q3+1.5IQR as outliers.

[0084] Treatment method: For detected outliers, median replacement or model-based predicted value replacement is used.

[0085] Model training process

[0086] Phase 1: ARIMA model training

[0087] Data partitioning: Divide historical traffic data into a training set (such as the first 80%) and a validation set (the last 20%) in chronological order.

[0088] Parameter estimation: Estimate the parameters of the ARIMA model using maximum likelihood estimation on the training set.

[0089] Model evaluation: Calculate the prediction error (such as MSE, RMSE, MAE) on the validation set and adjust the parameters based on the evaluation results.

[0090] Forecast generation: Use the trained ARIMA model to predict traffic in future time periods and obtain an ARIMA forecast sequence.

[0091] Phase 2: Prophet model training

[0092] Data preparation: Merge the original traffic data and ARIMA forecast results to construct the input data frame of the Prophet model, which contains ds (time column) and y (target value column).

[0093] Model fitting: Fit the Prophet model on the merged data, setting appropriate trend, seasonality, and holiday parameters.

[0094] Forecast generation: Use the trained Prophet model to predict future time periods and obtain a Prophet forecast sequence.

[0095] Phase 3: XGBoost model training

[0096] Feature construction: Combine the original traffic features, ARIMA prediction results, Prophet prediction results, and other engineering features (such as time features, lag features, etc.) into a feature matrix.

[0097] Data partitioning: Divide the feature matrix into training set, validation set and test set in chronological order.

[0098] Model training: Use XGBoost for training on the training set, and perform early stopping and parameter tuning on the validation set.

[0099] (2) Analysis process

[0100] Predictive Execution

[0101] Batch prediction: For offline analysis scenarios, historical data and future time periods are input at once to generate a complete prediction sequence.

[0102] Online prediction: For real-time monitoring scenarios, a sliding window mechanism is used to obtain the latest traffic data each time to update the model and predict traffic in the short term.

[0103] Outcome evaluation

[0104] Evaluation metrics: Calculates multiple evaluation metrics, including mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), etc.

[0105] Confidence interval calculation: Through the Monte Carlo simulation method, multiple possible prediction paths are generated, and the confidence interval of the predicted value (such as 95% confidence interval) is calculated to quantify the prediction uncertainty.

[0106] Results visualization

[0107] Time series graph: Draw a time series graph of actual flow values, predicted values, and confidence intervals to intuitively display the prediction results.

[0108] Error analysis chart: Draw the distribution histogram and QQ chart of the prediction error to test whether the error obeys the normal distribution.

[0109] Feature Importance Graph: For the XGBoost model, a feature importance graph is plotted to show the contribution of each input feature to the prediction results, helping to analyze the key factors affecting traffic changes.

[0110] Step 104: Use the DTW dynamic time warping algorithm to judge the traffic assessment result. If it is judged to be an abnormal situation, analyze the cause of the abnormal traffic and block the abnormal link and traffic use.

[0111] Specifically, in this embodiment, the current traffic evaluation result is converted into time series data, and the similarity distance is calculated using the DTW algorithm with each pattern in the normal traffic pattern library;

[0112] Set a similarity threshold. When the similarity distance is greater than the similarity threshold, it is considered an abnormal situation.

[0113] When an abnormal situation is determined, the abnormal traffic is analyzed based on time sensitivity, packet discreteness and protocol diversity characteristics, and the cause of the abnormality is determined by combining the working status of the IoT device and the network environment information.

[0114] Its beneficial effects are: 1. Improving the accuracy of distance measurement; increasing cluster stability judgment to avoid unnecessary merging, so that the clustering results are more consistent with the actual traffic distribution. This improved clustering algorithm can reasonably group characteristic traffic data, provide structured data for subsequent prediction models, and enhance the model's adaptability and analysis capabilities for different traffic patterns. 2. Effectively make up for the shortcomings of a single model, significantly improve the accuracy and reliability of traffic prediction, and provide an accurate prediction basis for IoT traffic management. 3. Comprehensively considering similarity distance and traffic change amplitude to formulate anomaly judgment standards, so that anomaly detection is more in line with actual conditions and reduces misjudgments and missed judgments. When an anomaly occurs, the cause of the anomaly is deeply analyzed from a multi-feature perspective, and targeted blocking measures are taken on the device side, network side, and server side to effectively prevent the spread of abnormal traffic, ensure the safe and stable operation of IoT devices and networks, and reduce security risks and economic losses caused by traffic anomalies.

[0115] See also Figure 2 In an IoT SIM card traffic data analysis method, characteristic traffic data is grouped based on an improved CURE clustering algorithm to obtain grouped traffic data, including the following steps:

[0116] Step 201: Standardize the characteristic flow data by Min-Max standardization, convert each characteristic value into the same dimension, and obtain standard flow data;

[0117] Step 202: Using the improved representative point selection method, select initial representative points in the standard flow data, set the initial number of clusters, and obtain initial cluster flow data;

[0118] Step 203: Calculate the distance between each cluster in the initial cluster traffic data, select the two clusters with the smallest distance to merge, update the representative point of the merged cluster during the merging process, and recalculate the distance to other clusters according to the improved distance measurement method to obtain the second cluster traffic data;

[0119] Step 204: Perform clustering iteration based on the second cluster traffic data, and terminate clustering when the number of clusters reaches a preset target number to obtain group traffic data.

[0120] See also Figure 3 In an IoT SIM card traffic data analysis system, the IoT SIM card traffic data analysis system includes the following modules:

[0121] The data acquisition module is used to obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, data packet discreteness and protocol diversity in the SIM card traffic data, and obtain characteristic traffic data;

[0122] The data classification module is used to group the characteristic traffic data based on the improved CURE clustering algorithm to obtain grouped traffic data;

[0123] The traffic evaluation module is used to establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtain the traffic evaluation results;

[0124] The analysis and judgment module is used to judge the traffic evaluation results using the DTW dynamic time warping algorithm. If it is judged to be an abnormal situation, the abnormal cause of the abnormal traffic is analyzed and the abnormal link and traffic use are blocked.

[0125] Specifically, this embodiment can also be implemented in the following ways:

[0126] 1. Obtaining SIM card traffic data and extracting feature vectors

[0127] (1) Data Acquisition

[0128] Data sources include IoT devices (such as sensors and smart terminals), communication base stations (including 2G / 3G / 4G / 5G base stations), and servers (IoT platform servers, application servers, etc.). On the IoT device side, a lightweight data collection agent is embedded to collect SIM card traffic data sent and received by the device in real time. On the communication base station side, processed traffic statistics are obtained using the base station interface. On the server side, SIM card-related traffic information is extracted from log files and databases.

[0129] Data collection technology

[0130] Real-time data collection: Utilizes real-time data transmission technologies based on message queues, such as Kafka. At the data source end, collected SIM card traffic data is sent to a Kafka message queue in real time, ensuring real-time and reliable data. Message queues offer high throughput, scalability, and fault tolerance, making them suitable for IoT environments where a large number of devices generate data simultaneously.

[0131] Scheduled collection: Set a fixed interval (such as every minute or every hour) and use a scheduled task script (such as a Cron task in Linux) to batch collect SIM card traffic data from the data source. This method is suitable for scenarios where real-time performance is not a high requirement but regular data aggregation is required.

[0132] (2) Feature vector extraction

[0133] Time sensitivity feature vector

[0134] Define different time windows, including minute, hour, day, and week levels. For each time window, calculate the following statistics:

[0135] Traffic mean: The average value of SIM card traffic within the time window, reflecting the overall level of traffic during the time period.

[0136] Traffic variance: measures the degree of traffic fluctuation within the time window. The larger the variance, the more drastic the traffic change and the higher the time sensitivity.

[0137] Peak Hour Percentage: Counts the percentage of traffic that falls within preset peak hours (such as 9:00 AM to 11:00 AM and 3:00 PM to 5:00 PM on weekdays) within the time window, reflecting the concentration of traffic during peak hours.

[0138] The sliding window technology is used to dynamically adjust the time window to capture the traffic change characteristics at different time scales.

[0139] Packet dispersion feature vector

[0140] Analyze the size distribution of data packets and calculate the following indicators:

[0141] Packet size standard deviation: reflects the degree of dispersion of packet sizes. A larger standard deviation indicates a greater difference in packet sizes and a higher degree of dispersion.

[0142] Data packet size coefficient of variation: The ratio of the standard deviation to the mean. It eliminates the influence of the mean on the degree of dispersion and is more suitable for comparing the dispersion of data packet sizes under different means.

[0143] Interval between adjacent data packets: Calculates the time interval between two adjacent data packets and counts the following characteristics:

[0144] Mean interval time: The average interval time reflects the average frequency of data packet sending.

[0145] Interval time variance: measures the fluctuation of the interval time. The larger the variance, the more uneven the time intervals between data packets are sent and the higher the dispersion.

[0146] Protocol Diversity Feature Vector

[0147] Identify the communication protocols used in traffic, such as TCP, UDP, HTTP, MQTT, CoAP, etc.

[0148] The proportion of each protocol in the traffic is counted as a characteristic indicator of protocol diversity.

[0149] 2. Feature traffic data grouping based on improved CURE clustering algorithm

[0150] (1) Basic principles of the CURE algorithm

[0151] The CURE (Clustering Using Representatives) algorithm is a representative point-based clustering algorithm. It represents clusters by selecting multiple representative points within each cluster. It can handle non-spherical data and outliers and is insensitive to noise. The basic steps include: randomly selecting sample points, partitioning and sampling the sample points, initializing the clusters, and forming the final clusters by moving and merging representative points.

[0152] (2) Description of improvements

[0153] Optimized representative point selection: The traditional CURE algorithm randomly selects representative points, which may result in the representative points not being able to fully reflect the characteristics of the cluster. The improved CURE algorithm uses a density-based representative point selection method to select points in high-density areas within each cluster as representative points, ensuring that the representative points can more accurately represent the core characteristics of the cluster.

[0154] Improved distance metrics: Weighted Euclidean distance is introduced, assigning different weights to each feature in the feature vector based on its importance. For the three features of time sensitivity, packet dispersion, and protocol diversity, the analytic hierarchy process (AHP) is used to determine the weights of each feature based on actual application scenarios, improving the accuracy of the distance metric.

[0155] Iterative process optimization: During cluster merging, we now assess the stability of the merged clusters. After merging two clusters, we calculate the compactness within the clusters and the separation between them. If the merged cluster performance does not improve, we abandon the merge, avoiding unnecessary cluster merging and improving clustering effectiveness.

[0156] (3) Grouping process

[0157] Data preprocessing: The extracted characteristic flow data is standardized and each characteristic value is converted to the same dimension. Commonly used methods include Z-score standardization and Min-Max standardization.

[0158] Initialize clustering: Set the initial number of clusters based on experience or data characteristics. Use an improved representative point selection method to select initial representative points in the data set.

[0159] Cluster merging: Calculate the distance between clusters and select the two clusters with the smallest distance to merge. During the merging process, update the representative point of the merged cluster and recalculate the distance to other clusters using the improved distance metric.

[0160] Termination condition: When the number of clusters reaches the preset target number, or the stability condition is no longer met after the clusters are merged, the clustering process is terminated and the group traffic data is obtained.

[0161] 3. Establishment and Analysis of ARIMA-Prophet-XGBoost Hybrid Forecasting Model

[0162] (1) Model composition and characteristics

[0163] ARIMA (Autoregressive Integrated Moving Average): Suitable for processing time series data, it can capture trends and seasonal variations in the data. By determining the appropriate parameters p (autoregressive order), d (difference order), and q (moving average order), a model is established to forecast time series.

[0164] Prophet: A time series forecasting tool developed by Facebook, it excels at processing data with significant seasonality and trends, automatically detecting and modeling seasonal and holiday effects in the data. The Prophet model offers advantages such as high flexibility and easy parameter adjustment.

[0165] XGBoost (Extreme Gradient Boosting Trees): An efficient machine learning algorithm based on gradient boosting decision trees. It can handle nonlinear data and complex relationships, with high prediction accuracy and efficiency. XGBoost supports parallel computing and can quickly train models on large amounts of data.

[0166] (2) Hybrid model establishment

[0167] Data preprocessing: Perform time series conversion on packet traffic data, extracting timestamps and corresponding traffic feature values. Missing value processing is performed, using linear interpolation or time series prediction methods to fill missing data. Outlier detection is performed, using the Z-score method or IQR method to identify and process outliers.

[0168] Model training order

[0169] First, the ARIMA model is used to train the time series data to obtain the ARIMA prediction results.

[0170] The ARIMA forecast result is used as one of the input features and input into the Prophet model together with the original time series data for training. The Prophet model takes advantage of its ability to handle seasonality and trends to further improve the forecast accuracy.

[0171] The prediction results of the ARIMA and Prophet models and the original feature data are used as input to the XGBoost model for training. The XGBoost model can capture nonlinear relationships and complex patterns, and combines the advantages of the first two models to obtain the final hybrid prediction model.

[0172] Model fusion method: The prediction results of the three models are fused by weighted averaging. The weights are determined by cross-validation, so that the fused prediction results have the best performance on the training set and validation set.

[0173] (3) Analysis process

[0174] The grouped traffic data is input into the established hybrid prediction model. The model first performs time series analysis and feature extraction on the data, and then processes it layer by layer through the three models of ARIMA, Prophet and XGBoost, and finally outputs the traffic assessment results, including traffic prediction values, prediction confidence intervals, etc.

[0175] 4. Anomaly Judgment and Processing Based on DTW Dynamic Time Warping Algorithm

[0176] (1) Basic principles of the DTW algorithm

[0177] The DTW algorithm is used to measure the similarity between two time series. It uses dynamic programming to find the optimal alignment path between the two time series, minimizing the distance between them. In traffic analysis, the DTW algorithm can be used to compare actual traffic assessment results with normal traffic patterns to determine whether anomalies exist.

[0178] (2) Application to traffic assessment results

[0179] Establishing normal traffic patterns: Collect historical normal traffic data, and establish a time series pattern library of normal traffic through preprocessing and feature extraction.

[0180] Similarity calculation: Convert the current traffic assessment results into time series data and use the DTW algorithm to calculate the similarity distance with each pattern in the normal traffic pattern library. Set a similarity threshold. When the similarity distance exceeds the threshold, it is judged as an abnormal situation.

[0181] Abnormality judgment criteria: A reasonable abnormality judgment criteria is formulated by comprehensively considering factors such as similarity distance and traffic change amplitude. For example, when the similarity distance exceeds the threshold and the traffic change amplitude exceeds a preset percentage, it is determined to be an abnormal situation.

[0182] (III) Analysis and treatment of abnormal causes

[0183] Abnormal Cause Analysis: When an abnormal situation is identified, a detailed analysis of the abnormal traffic is performed. Starting with characteristics such as time sensitivity, packet dispersion, and protocol diversity, the abnormal traffic is analyzed based on these characteristics. Combined with information such as the operating status of IoT devices and the network environment, the cause of the abnormality is determined, such as device failure, network attack, or malware infection.

[0184] Block abnormal links and traffic usage

[0185] On the IoT device side, remote instructions are sent through the device management platform to close the network interface with abnormal connection or disable related applications to block the generation of abnormal traffic.

[0186] On the communication network side, we cooperate with operators to block the IP addresses or ports of abnormal links through network security devices such as firewalls and intrusion detection systems to prevent the transmission of abnormal traffic in the network.

[0187] On the server side, services related to abnormal traffic are monitored and restricted, such as limiting access frequency, closing abnormal service ports, etc., to ensure the safe and stable operation of the server.

[0188] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for analyzing SIM card traffic data of the Internet of Things, characterized in that: The method for analyzing the traffic data of the Internet of Things SIM card comprises the following steps: Acquire SIM card traffic data in the system, extract feature vectors of time sensitivity, data packet dispersion, and protocol diversity from the SIM card traffic data, and obtain feature traffic data; The characteristic flow data is grouped based on an improved CURE clustering algorithm to obtain grouped flow data; Obtain the time series data in the system, use the ARIMA model to train the time series data, and obtain the ARIMA prediction results. The ARIMA prediction results are used as input features and input into the Prophet model together with the original time series data for training. The Prophet model is used to process seasonality and trend to obtain the Prophet model prediction results; The ARIMA prediction results, Prophet model prediction results and time series data are input into the XGBoost model for training to obtain an ARIMA-Prophet-XGBoost hybrid prediction model; The grouped traffic data is input into the ARIMA-Prophet-XGBoost hybrid prediction model to perform time series analysis and feature extraction on the data; Through the processing of ARIMA, Prophet and XGBoost models, the flow assessment results are output, including at least the flow prediction value and prediction confidence interval; The traffic evaluation result is judged by using the DTW dynamic time warping algorithm. If it is judged to be an abnormal situation, the abnormal cause of the abnormal traffic is analyzed, and the abnormal link and traffic use are blocked.

2. The method for analyzing SIM card traffic data of the Internet of Things according to claim 1, wherein: The acquiring system acquires SIM card traffic data, extracts feature vectors of time sensitivity, data packet discreteness, and protocol diversity from the SIM card traffic data, and obtains feature traffic data, including: Obtain SIM card traffic data from the system, define different time windows, calculate the traffic mean, traffic variance, and peak period ratio for each time window, and obtain the time sensitivity feature vector; Analyze the distribution of the SIM card traffic data, calculate the data packet size standard deviation, data packet size coefficient of variation, adjacent data packet interval time, interval time mean and interval time variance, and obtain a data packet dispersion feature vector; Identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, count the proportion of each protocol in the traffic, and obtain a protocol diversity feature vector.

3. The method for analyzing SIM card traffic data of the Internet of Things according to claim 1, wherein: The improved CURE clustering algorithm is used to group the characteristic traffic data to obtain grouped traffic data, including: Based on the CURE clustering algorithm, a density-based representative point selection method is used to select points in high-density areas in each cluster as representative points; The weighted Euclidean distance is introduced to assign different weights according to the importance of each feature in the feature vector; In the process of clustering and merging data, the stability judgment of the merged cluster data is added to obtain the improved CURE clustering algorithm.

4. The method for analyzing SIM card traffic data of the Internet of Things according to claim 1, wherein: The improved CURE clustering algorithm is used to group the characteristic traffic data to obtain grouped traffic data, including: The characteristic flow data is standardized by Min-Max standardization, and each characteristic value is converted into the same dimension to obtain standard flow data; Using an improved representative point selection method, initial representative points are selected from the standard flow data, the initial number of clusters is set, and initial cluster flow data is obtained; Calculating the distance between each cluster in the initial cluster traffic data, selecting the two clusters with the smallest distance to merge, updating the representative point of the merged cluster during the merging process, and recalculating the distance to other clusters according to the improved distance measurement method to obtain the second cluster traffic data; Clustering iteration is performed based on the second cluster traffic data, and clustering is terminated when the number of clusters reaches a preset target number to obtain group traffic data.

5. The method for analyzing SIM card traffic data of the Internet of Things according to claim 1, wherein: The DTW dynamic time warping algorithm is used to judge the traffic assessment result. If it is judged to be an abnormal situation, the abnormal cause of the abnormal traffic is analyzed and the abnormal link and traffic use are blocked, including: Convert the current traffic assessment results into time series data and use the DTW algorithm to calculate the similarity distance with each pattern in the normal traffic pattern library; Setting a similarity threshold, when the similarity distance is greater than the similarity threshold, it is determined to be an abnormal situation; When an abnormal situation is determined, the abnormal traffic is analyzed based on time sensitivity, packet discreteness and protocol diversity characteristics, and the cause of the abnormality is determined by combining the working status of the IoT device and the network environment information.

6. An Internet of Things SIM card traffic data analysis system, characterized in that: The Internet of Things SIM card traffic data analysis system includes the following modules: A data acquisition module is used to obtain SIM card traffic data in the system, extract feature vectors of time sensitivity, data packet discreteness and protocol diversity from the SIM card traffic data, and obtain feature traffic data; A data classification module, configured to group the characteristic traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data; A traffic assessment module is used to obtain time series data in the system, train the time series data using an ARIMA model to obtain ARIMA prediction results, use the ARIMA prediction results as input features, input them together with the original time series data into a Prophet model for training, use the Prophet model to process seasonality and trend, and obtain Prophet model prediction results; input the ARIMA prediction results, Prophet model prediction results, and time series data into an XGBoost model for training to obtain an ARIMA-Prophet-XGBoost hybrid prediction model; input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model, perform time series analysis and feature extraction on the data; and output a traffic assessment result through processing by the ARIMA, Prophet, and XGBoost models, which includes at least a traffic prediction value and a prediction confidence interval; The analysis and judgment module is used to judge the traffic evaluation result using the DTW dynamic time warping algorithm. If it is judged to be an abnormal situation, the abnormal cause of the abnormal traffic is analyzed and the abnormal link and traffic use are blocked.

7. The Internet of Things SIM card traffic data analysis system according to claim 6, characterized in that: The data acquisition module includes the following submodules: The acquisition submodule is used to obtain SIM card traffic data in the system, define different time windows, calculate the traffic mean, traffic variance and peak period ratio for each time window, and obtain the time sensitivity feature vector; An analysis submodule is used to analyze the distribution of the SIM card traffic data, calculate the standard deviation of the data packet size, the coefficient of variation of the data packet size, the interval time between adjacent data packets, the mean of the interval time and the variance of the interval time, and obtain a data packet dispersion feature vector; The identification submodule is used to identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT and CoAP, and count the proportion of each protocol in the traffic to obtain a protocol diversity feature vector.

8. The Internet of Things SIM card traffic data analysis system according to claim 6, characterized in that: The data classification module includes the following submodules: The selection submodule is used to select points in high-density areas as representative points in each cluster using the density-based representative point selection method based on the CURE clustering algorithm; A submodule is introduced to introduce weighted Euclidean distance, which assigns different weights to each feature according to its importance in the feature vector; The judgment submodule is used to add a judgment on the stability of the merged cluster data during the process of clustering and merging the data, so as to obtain an improved CURE clustering algorithm.

Images