Internet of Things SIM card flow data analysis method and system

Abstract

The invention discloses an Internet of Things SIM card traffic data analysis method and system, and the method comprises the steps: obtaining SIM card traffic data in a system, extracting feature vectors of time sensitivity, data packet dispersion and protocol diversity in the SIM card traffic data, and obtaining feature traffic data; grouping the feature traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data; establishing a hybrid prediction model, and inputting the grouped traffic data into the hybrid prediction model for analysis to obtain a traffic evaluation result; and judging the flow evaluation result by utilizing a DTW (Dynamic Time Warping) algorithm, and if the flow evaluation result is judged to be abnormal, carrying out abnormal reason analysis on the abnormal flow, and blocking abnormal link and flow use. The accuracy and the reliability of traffic prediction are remarkably improved, and an accurate prediction basis is provided for traffic management of the Internet of Things.

Description

Technical Field

[0001] The present invention relates to the technical field of the Internet of Things, and in particular to a method and system for analyzing the traffic data of Internet of Things SIM cards. Background Art

[0002] Traditional methods for analyzing the traffic of Internet of Things SIM cards mostly use a single statistical model or simple threshold judgment, making it difficult to effectively process complex and changing traffic data. Some methods only judge anomalies by monitoring the total traffic volume and usage frequency, and cannot deeply analyze key features such as time sensitivity, packet dispersion, and protocol diversity, resulting in a low accuracy rate for identifying abnormal traffic and being prone to misjudgment and missed judgment. A single model is difficult to comprehensively and accurately predict the changing trend of Internet of Things SIM card traffic. In addition, in the anomaly detection link, traditional methods rely on fixed thresholds or simple pattern matching and cannot adapt to the dynamic changes of traffic patterns. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and design a method for analyzing the traffic data of Internet of Things SIM cards.

[0004] Furthermore, in the above method for analyzing the traffic data of Internet of Things SIM cards, the method for analyzing the traffic data of Internet of Things SIM cards includes the following steps: Obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data, and obtain the feature traffic data; Group the feature traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data; Establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtain a traffic evaluation result; Use the DTW dynamic time warping algorithm to judge the traffic evaluation result. If it is judged as an abnormal situation, analyze the cause of the abnormal traffic and block the abnormal link and traffic usage.

[0005] Furthermore, in the above method for analyzing the traffic data of Internet of Things SIM cards, the step of obtaining the SIM card traffic data in the system, extracting the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data, and obtaining the feature traffic data includes: Obtain the SIM card traffic data in the system, define different time windows, and calculate the traffic mean, traffic variance, and peak period ratio for each time window to obtain the time sensitivity feature vector; Analyze the distribution of the SIM card traffic data, calculate the standard deviation of the data packet size, the coefficient of variation of the data packet size, the time interval between adjacent data packets, the mean of the interval time, and the variance of the interval time, to obtain the data packet dispersion feature vector; Identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, and count the proportion of each protocol in the traffic to obtain the protocol diversity feature vector.

[0006] Furthermore, in the above method for analyzing IoT SIM card traffic data, the improved CURE clustering algorithm is used to group the feature traffic data to obtain grouped traffic data, including: Based on the CURE clustering algorithm, use the density-based representative point selection method to select points in the high-density area in each cluster as representative points; Introduce the weighted Euclidean distance and assign different weights according to the importance of each feature in the feature vector; During the process of clustering and merging the data, add the judgment of the stability of the clustered data after merging to obtain the improved CURE clustering algorithm.

[0007] Furthermore, in the above method for analyzing IoT SIM card traffic data, the improved CURE clustering algorithm is used to group the feature traffic data to obtain grouped traffic data, including: Perform standardization processing on the feature traffic data through Min-Max standardization, convert each feature value to the same dimension to obtain the standard traffic data; Use the improved representative point selection method to select initial representative points in the standard traffic data, set the initial number of clusters to obtain the initial clustered traffic data; Calculate the distance between each cluster in the initial clustered traffic data, select the two clusters with the smallest distance for merging, update the representative points of the merged clusters during the merging process, and recalculate the distance from other clusters according to the improved distance metric method to obtain the second clustered traffic data; Based on the second clustered traffic data, perform clustering iteration, and terminate the clustering when the number of clusters reaches the preset target number to obtain the grouped traffic data.

[0008] Furthermore, in the above method for analyzing IoT SIM card traffic data, establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain the traffic evaluation result, including: Obtain the time series data in the system, train the time series data using the ARIMA model, and obtain the ARIMA prediction result. Use the ARIMA prediction result as an input feature, and input it together with the original time series data into the Prophet model for training. Use the Prophet model to handle seasonality and trends, and obtain the Prophet model prediction result. Input the ARIMA prediction result, the Prophet model prediction result, and the time series data into the XGBoost model for training to obtain the ARIMA-Prophet-XGBoost hybrid prediction model.

[0009] Furthermore, in the above method for analyzing IoT SIM card traffic data, when establishing the ARIMA-Prophet-XGBoost hybrid prediction model and inputting the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain the traffic evaluation result, it further includes: Input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model to perform time series analysis and feature extraction on the data. Through the processing of the ARIMA, Prophet, and XGBoost models, output the traffic evaluation result, which at least includes the traffic prediction value and the prediction confidence interval.

[0010] Furthermore, in the above method for analyzing IoT SIM card traffic data, when using the DTW dynamic time warping algorithm to judge the traffic evaluation result, if it is judged as an abnormal situation, analyze the cause of the abnormal traffic and block the abnormal link and traffic usage, including: Convert the current traffic evaluation result into time series data, and calculate the similarity distance with each pattern in the normal traffic pattern library using the DTW algorithm. Set a similarity threshold. When the similarity distance is greater than the similarity threshold, it is judged as an abnormal situation. When it is judged as an abnormal situation, analyze the abnormal traffic from the characteristics of time sensitivity, packet dispersion, and protocol diversity, and determine the cause of the abnormality in combination with the working status of the IoT device and the network environment information.

[0011] Furthermore, in an IoT SIM card traffic data analysis system, the IoT SIM card traffic data analysis system includes the following modules: A data acquisition module, which is used to obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data, and obtain the feature traffic data. A data classification module, which is used to group the feature traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data; A traffic evaluation module, which is used to establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtain a traffic evaluation result; An analysis and judgment module, which is used to judge the traffic evaluation result by using the DTW dynamic time warping algorithm. If it is judged as an abnormal situation, it will analyze the cause of the abnormal traffic, and block the abnormal link and traffic usage.

[0012] Furthermore, in the system for implementing the above-mentioned method for analyzing IoT SIM card traffic data, the data acquisition module includes the following sub-modules: An acquisition sub-module, which is used to acquire the SIM card traffic data in the system, define different time windows, calculate the traffic mean value, traffic variance, and peak period occupancy ratio for each time window, and obtain a time sensitivity feature vector; An analysis sub-module, which is used to analyze the distribution of the SIM card traffic data, calculate the standard deviation of the packet size, the coefficient of variation of the packet size, the time interval between adjacent packets, the mean interval time, and the interval time variance, and obtain a packet dispersion feature vector; An identification sub-module, which is used to identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, and count the proportion of each protocol in the traffic to obtain a protocol diversity feature vector.

[0013] Furthermore, in the system for implementing the above-mentioned method for analyzing IoT SIM card traffic data, the data classification module includes the following sub-modules: A selection sub-module, which is used to select points in the high-density area as representative points in each cluster based on the CURE clustering algorithm and using a density-based representative point selection method; An introduction sub-module, which is used to introduce the weighted Euclidean distance and assign different weights according to the importance of each feature in the feature vector; A judgment sub-module, which is used to add a judgment on the stability of the clustered data after merging during the process of clustering and merging the data to obtain an improved CURE clustering algorithm.

[0014] Its beneficial effects are as follows. By obtaining the SIM card traffic data in the system, extracting the eigenvectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data, the characteristic traffic data is obtained; based on the improved CURE clustering algorithm, the characteristic traffic data is grouped to obtain grouped traffic data; an ARIMA-Prophet-XGBoost hybrid prediction model is established, and the grouped traffic data is input into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain a traffic evaluation result; the DTW dynamic time warping algorithm is used to judge the traffic evaluation result. If it is judged as an abnormal situation, the cause of the abnormal traffic is analyzed, and the abnormal link and traffic usage are blocked. 1. Improve the accuracy of distance measurement; increase the judgment of clustering stability to avoid unnecessary mergers, making the clustering result more in line with the actual traffic distribution. This improved clustering algorithm can reasonably group the characteristic traffic data, provide structured data for the subsequent prediction model, and enhance the adaptability and analysis ability of the model to different traffic patterns. 2. Effectively make up for the deficiencies of a single model, significantly improve the accuracy and reliability of traffic prediction, and provide an accurate prediction basis for Internet of Things traffic management. 3. Comprehensively consider the similarity distance and the amplitude of traffic change to formulate an abnormal judgment standard, making the abnormal detection more in line with the actual situation and reducing false positives and false negatives. When an abnormality occurs, the cause of the abnormality is deeply analyzed from multiple feature perspectives, and targeted blocking measures are taken at the device end, network side, and server side to effectively prevent the spread of abnormal traffic, ensure the safe and stable operation of Internet of Things devices and networks, and reduce the security risks and economic losses caused by traffic anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0016] Figure 1 Schematic diagram of the first embodiment of a method for analyzing Internet of Things SIM card traffic data in an embodiment of the present invention; Figure 2 Schematic diagram of the second embodiment of a method for analyzing Internet of Things SIM card traffic data in an embodiment of the present invention; Figure 3 Schematic diagram of the first embodiment of a system for analyzing Internet of Things SIM card traffic data in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0018] Those skilled in the art of the present technology can understand that unless specifically stated, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups.

[0019] The present invention will be specifically described below in conjunction with the accompanying drawings. As Figure 1 shown, a method for analyzing the traffic data of an Internet of Things (IoT) SIM card, the method for analyzing the traffic data of the IoT SIM card includes the following steps: Step 101: Obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, packet dispersion and protocol diversity in the SIM card traffic data, and obtain the feature traffic data; Specifically, in this embodiment, the SIM card traffic data in the system is obtained, different time windows are defined, and for each time window, the traffic mean, traffic variance and peak period ratio are calculated to obtain the time sensitivity feature vector; Analyze the distribution of the SIM card traffic data, calculate the packet size standard deviation, packet size coefficient of variation, adjacent packet interval time, interval time mean and interval time variance to obtain the packet dispersion feature vector; Identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT and CoAP, and count the proportion of each protocol in the traffic to obtain the protocol diversity feature vector.

[0020] Step 102: Group the feature traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data; Specifically, in this embodiment, based on the CURE clustering algorithm, a density-based representative point selection method is used to select points in the high-density area in each cluster as representative points; Introduce the weighted Euclidean distance, and assign different weights according to the importance of each feature in the feature vector; In the process of clustering and merging the data, add the judgment of the stability of the clustered data after merging to obtain the improved CURE clustering algorithm.

[0021] Normalize the feature traffic data through Min - Max normalization, convert each feature value to the same dimension, and obtain the standard traffic data; Use the improved representative point selection method to select initial representative points in the standard traffic data, set the initial number of clusters, and obtain the initial clustered traffic data; Calculate the distances between clusters in the initial clustered traffic data, select the two clusters with the smallest distance for merging, update the representative point of the merged cluster during the merging process, and recalculate the distances to other clusters according to the improved distance metric method to obtain the second clustered traffic data; Perform clustering iteration based on the second clustered traffic data, and terminate the clustering when the number of clusters reaches the preset target number to obtain the grouped traffic data.

[0022] Step 103: Establish an ARIMA - Prophet - XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA - Prophet - XGBoost hybrid prediction model for analysis, and obtain the traffic evaluation result; Specifically, in this embodiment, obtain the time - series data in the system, use the ARIMA model to train the time - series data to obtain the ARIMA prediction result, Take the ARIMA prediction result as an input feature, and input it together with the original time - series data into the Prophet model for training. Use the Prophet model to handle seasonality and trends to obtain the Prophet model prediction result; Input the ARIMA prediction result, the Prophet model prediction result, and the time - series data into the XGBoost model for training to obtain the ARIMA - Prophet - XGBoost hybrid prediction model.

[0023] Input the grouped traffic data into the ARIMA - Prophet - XGBoost hybrid prediction model to perform time - series analysis and feature extraction on the data; Through the processing of the ARIMA, Prophet, and XGBoost models, output the traffic evaluation result, which at least includes the traffic prediction value and the prediction confidence interval.

[0024] Feature engineering: In addition to the original traffic features and the prediction results of ARIMA and Prophet, also construct time features (such as hour, day, day of the week, etc.), lag features (such as traffic values in the previous 1 - 7 days), and rolling statistical features (such as moving average, moving standard deviation, etc.) as inputs.

[0025] Parameter optimization: Use random search or Bayesian optimization methods to adjust hyperparameters such as the learning rate, tree depth, and subsampling rate to minimize the loss function (such as mean squared error MSE) on the validation set.

[0026] Applicable scenarios: It can capture non - linear relationships and complex interaction effects in data, has good adaptability to high - dimensional and heterogeneous data, and is suitable for dealing with traffic prediction problems containing multiple influencing factors.

[0027] (I) Establishment of hybrid model Data pre - processing Time - series conversion: Arrange the grouped traffic data in chronological order to ensure uniform time intervals. For data with non - uniform intervals, linear interpolation or spline interpolation is used for resampling.

[0028] Missing value handling Short - term missing: Use the linear interpolation method to linearly estimate according to the values of adjacent time points. For example, if the traffic data for a certain hour is missing, the average value of the traffic in the two adjacent hours is taken as the filling value.

[0029] Long - term missing: Use time - series prediction methods, train ARIMA or Prophet models using historical data, and predict and fill in the missing values.

[0030] Outlier detection and handling Z - score method: Calculate the Z - score (the multiple of the standard deviation from the mean) for each data point, and consider the data points with an absolute Z - score greater than 3 as outliers.

[0031] IQR method: Calculate the inter - quartile range IQR = Q3 - Q1, and consider the data points less than Q1 - 1.5IQR or greater than Q3 + 1.5IQR as outliers.

[0032] Handling method: For the detected outliers, use median replacement or replacement with predicted values based on the model.

[0033] Model training process First stage: Training of ARIMA model Data division: Divide the historical traffic data into a training set (e.g., the first 80%) and a validation set (the last 20%) in chronological order.

[0034] Parameter estimation: Use the maximum likelihood estimation method to estimate the parameters of the ARIMA model on the training set.

[0035] Model evaluation: Calculate the prediction errors (such as MSE, RMSE, MAE) on the validation set, and adjust the parameters according to the evaluation results.

[0036] Prediction generation: Use the trained ARIMA model to predict the traffic for future time periods, and obtain the ARIMA prediction sequence.

[0037] Second stage: Training of Prophet model Data Preparation: Combine the original traffic data and the ARIMA prediction results to construct the input data frame for the Prophet model, which includes ds (time column) and y (target value column).

[0038] Model Fitting: Fit the Prophet model on the combined data, setting appropriate trend, seasonality, and holiday parameters.

[0039] Prediction Generation: Use the trained Prophet model to predict future time periods and obtain the Prophet prediction sequence.

[0040] Phase 3: XGBoost Model Training Feature Construction: Combine the original traffic features, ARIMA prediction results, Prophet prediction results, and other engineering features (such as time features, lag features, etc.) into a feature matrix.

[0041] Data Partitioning: Partition the feature matrix into training set, validation set, and test set in chronological order.

[0042] Model Training: Use XGBoost to train on the training set, and perform early stopping and parameter tuning through the validation set.

[0043] (2) Analysis Process Prediction Execution Batch Prediction: For the offline analysis scenario, input historical data and future time periods at once to generate a complete prediction sequence.

[0044] Online Prediction: For the real-time monitoring scenario, adopt a sliding window mechanism, obtain the latest traffic data each time to update the model, and predict the traffic in the near future.

[0045] Result Evaluation Evaluation Metrics: Calculate multiple evaluation metrics, including Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), etc.

[0046] Confidence Interval Calculation: Through the Monte Carlo simulation method, generate multiple possible prediction paths, calculate the confidence interval of the predicted values (such as 95% confidence interval), and quantify the prediction uncertainty.

[0047] Result Visualization Time Series Plot: Plot the time series plot of the actual traffic values, predicted values, and confidence intervals to visually display the prediction effect.

[0048] Error Analysis Plot: Plot the distribution histogram and QQ plot of the prediction errors to check whether the errors follow a normal distribution.

[0049] Feature Importance Plot: For the XGBoost model, a feature importance plot is drawn to show the contribution degree of each input feature to the prediction result, helping to analyze the key factors affecting traffic changes.

[0050] Step 104: Use the DTW (Dynamic Time Warping) algorithm to judge the traffic evaluation result. If it is judged as an abnormal situation, analyze the cause of the abnormal traffic, and block the abnormal link and traffic usage.

[0051] Specifically, in this embodiment, the current traffic evaluation result is converted into time series data, and the similarity distance is calculated with each pattern in the normal traffic pattern library using the DTW algorithm; Set a similarity threshold. When the similarity distance is greater than the similarity threshold, it is judged as an abnormal situation; When it is judged as an abnormal situation, analyze the abnormal traffic from the aspects of time sensitivity, packet dispersion degree, and protocol diversity characteristics, and determine the cause of the abnormality in combination with the working status of the IoT device and network environment information.

[0052] Its beneficial effects are as follows: 1. Improve the accuracy of distance measurement; increase the judgment of clustering stability to avoid unnecessary mergers, making the clustering result more in line with the actual traffic distribution. This improved clustering algorithm can reasonably group the feature traffic data, provide structured data for the subsequent prediction model, and enhance the adaptability and analysis ability of the model to different traffic patterns. 2. Effectively make up for the deficiencies of a single model, significantly improve the accuracy and reliability of traffic prediction, and provide an accurate prediction basis for IoT traffic management. 3. Comprehensively consider the similarity distance and the amplitude of traffic change to formulate an abnormal judgment criterion, making the abnormal detection more in line with the actual situation, reducing false positives and false negatives. When an abnormality occurs, deeply analyze the cause of the abnormality from multiple feature perspectives, and take targeted blocking measures at the device side, network side, and server side to effectively prevent the spread of abnormal traffic, ensure the safe and stable operation of IoT devices and networks, and reduce the security risks and economic losses caused by traffic anomalies.

[0053] Please refer to Figure 2 , in a method for analyzing IoT SIM card traffic data, the steps of grouping the feature traffic data based on an improved CURE clustering algorithm to obtain grouped traffic data are as follows: Step 201: Standardize the feature traffic data through Min - Max standardization, convert each feature value to the same dimension, and obtain the standard traffic data; Step 202: Use an improved representative point selection method to select initial representative points in the standard traffic data, set the initial number of clusters, and obtain the initial clustered traffic data; Step 203: Calculate the distances between clusters in the initial cluster traffic data, select the two clusters with the smallest distance for merging, update the representative points of the merged cluster during the merging process, and recalculate the distances from other clusters according to the improved distance metric method to obtain the second cluster traffic data; Step 204: Perform clustering iteration based on the second cluster traffic data, and terminate the clustering when the number of clusters reaches the preset target number to obtain the grouped traffic data.

[0054] Please refer to Figure 3 , in an Internet of Things SIM card traffic data analysis system, the Internet of Things SIM card traffic data analysis system includes the following modules: Data acquisition module, used to obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data to obtain the feature traffic data; Data classification module, used to group the feature traffic data based on the improved CURE clustering algorithm to obtain the grouped traffic data; Traffic evaluation module, used to establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain the traffic evaluation result; Analysis and judgment module, used to use the DTW dynamic time warping algorithm to judge the traffic evaluation result. If it is judged as an abnormal situation, analyze the cause of the abnormal traffic, and block the abnormal link and traffic usage.

[0055] Specifically, this embodiment can also be implemented in the following ways: I. Obtain SIM card traffic data and extract feature vectors (I) Data acquisition Data sources: Cover Internet of Things devices (such as sensors, smart terminals, etc.), communication base stations (including 2G / 3G / 4G / 5G base stations), and servers (Internet of Things platform servers, application servers, etc.). On the Internet of Things device side, a lightweight data acquisition proxy program is embedded to collect the SIM card traffic data sent and received by the device in real time; on the communication base station side, the processed traffic statistical data is obtained through the interface of the base station; on the server side, the traffic information related to the SIM card is extracted from the log files and databases.

[0056] Data acquisition technology Real-time collection: Use real-time data transmission technology based on message queues, such as Kafka. At the data source end, the collected SIM card traffic data is sent to the Kafka message queue in real time to ensure the real-time and reliability of the data. The message queue has the characteristics of high throughput, scalability, and fault tolerance, and can adapt to the scenario where a large number of devices generate data at the same time in the IoT environment.

[0057] Scheduled collection: Set a fixed time interval (such as every minute or every hour) and use a scheduled task script (such as Cron task in Linux) to batch obtain SIM card traffic data from the data source. This method is suitable for scenarios where real-time requirements are not high but data needs to be summarized regularly.

[0058] 2. Feature vector extraction Time Sensitivity Feature Vector Define different time windows, including minute, hour, day, and week levels. For each time window, calculate the following statistics: Traffic mean: The average value of SIM card traffic in the time window, reflecting the overall level of traffic in this time period.

[0059] Traffic variance: measures the degree of fluctuation of traffic within the time window. The larger the variance, the more drastic the traffic change and the higher the time sensitivity.

[0060] Peak period ratio: Statistics on the ratio of traffic in preset peak periods (such as 9:00-11:00 a.m. and 3:00-5:00 p.m. on weekdays) within the time window, reflecting the concentration of traffic in peak periods.

[0061] The sliding window technology is used to dynamically adjust the time window to capture the traffic change characteristics at different time scales.

[0062] Packet dispersion feature vector Analyze the size distribution of data packets and calculate the following indicators: Data packet size standard deviation: reflects the degree of dispersion of data packet size. The larger the standard deviation, the greater the difference in data packet size and the higher the dispersion.

[0063] Data packet size coefficient of variation: The ratio of the standard deviation to the mean, which eliminates the influence of the mean on the degree of dispersion, is more suitable for comparing the dispersion of data packet sizes under different means.

[0064] Interval between adjacent data packets: Calculates the time interval between two adjacent data packets and counts the following features: Mean interval time: average interval time, reflecting the average frequency of data packet sending.

[0065] Variance of the interval time: It measures the fluctuation of the interval time. The larger the variance, the more uneven the time intervals of data packet transmission and the higher the dispersion degree.

[0066] Protocol diversity feature vector Identify the communication protocols used in the traffic, such as TCP, UDP, HTTP, MQTT, CoAP, etc.

[0067] Statistically analyze the proportion of each protocol in the traffic as a characteristic index of protocol diversity.

[0068] II. Grouping of characteristic traffic data based on the improved CURE clustering algorithm (I) Basic principle of the CURE algorithm The CURE (Clustering Using Representatives) algorithm is a clustering algorithm based on representative points. It represents each cluster by selecting multiple representative points in each cluster, can handle non-spherical data and outliers, and is insensitive to noise. The basic steps include: randomly selecting sample points, partitioning and sampling the sample points, initializing the clusters, and forming the final clusters by moving and merging the representative points.

[0069] (II) Explanation of the improvement points Optimization of representative point selection: The traditional CURE algorithm randomly selects representative points, which may result in the representative points not being able to well reflect the characteristics of the clusters. After improvement, a density-based representative point selection method is adopted, and points in the high-density area in each cluster are selected as representative points to ensure that the representative points can more accurately represent the core characteristics of the clusters.

[0070] Improvement of distance metric: The weighted Euclidean distance is introduced, and different weights are assigned according to the importance of each feature in the feature vector. For the three features of time sensitivity, data packet dispersion, and protocol diversity, the weights of each feature are determined by the Analytic Hierarchy Process (AHP) according to the actual application scenario to improve the accuracy of distance metric.

[0071] Optimization of the iteration process: During the cluster merging process, the judgment of the stability of the merged clusters is added. When two clusters are merged, the compactness within the clusters and the separation between the clusters are calculated. If the performance of the merged clusters is not improved, the merge is abandoned to avoid unnecessary cluster merging and improve the clustering effect.

[0072] (III) Grouping process Data preprocessing: Standardize the extracted characteristic traffic data to convert each feature value to the same dimension. Common methods include Z-score standardization and Min-Max standardization.

[0073] Initial clustering: Set the initial number of clusters according to experience or data characteristics. Use an improved representative point selection method to select initial representative points in the dataset.

[0074] Cluster merging: Calculate the distances between clusters, and select the two clusters with the smallest distance for merging. During the merging process, update the representative points of the merged cluster, and recalculate the distances from other clusters according to the improved distance measurement method.

[0075] Termination condition: When the number of clusters reaches the preset target number, or when the stability condition is no longer satisfied after cluster merging, terminate the clustering process to obtain grouped traffic data.

[0076] III. Establishment and analysis of the ARIMA-Prophet-XGBoost hybrid prediction model (I) Model composition and characteristics ARIMA (Autoregressive Integrated Moving Average Model): Suitable for processing time series data, capable of capturing the trends and seasonal variations in the data. By determining appropriate p (autoregressive order), d (differencing order), and q (moving average order) parameters, establish a model to predict the time series.

[0077] Prophet: A time series prediction tool developed by Facebook, good at dealing with data with obvious seasonality and trends, capable of automatically detecting and modeling seasonal effects, holiday effects, etc. in the data. The Prophet model has the advantages of high flexibility and easy parameter adjustment.

[0078] XGBoost (Extreme Gradient Boosting): An efficient machine learning algorithm, based on gradient boosting decision trees, capable of handling non-linear data and complex relationships, with high prediction accuracy and efficiency. XGBoost supports parallel computing and can quickly train models on large-scale data.

[0079] (II) Establishment of the hybrid model Data preprocessing: Perform time series conversion on the grouped traffic data, extract timestamps and corresponding traffic feature values. Handle missing values, and use linear interpolation or time series prediction methods to fill in missing data; perform outlier detection, and use the Z-score method or IQR method to identify and handle outliers.

[0080] Model training order First, use the ARIMA model to train the time series data to obtain the ARIMA prediction results.

[0081] Take the ARIMA prediction result as one of the input features, and input it together with the original time series data into the Prophet model for training. Utilize the advantages of the Prophet model in dealing with seasonality and trends to further improve the prediction accuracy.

[0082] Take the prediction results of the ARIMA and Prophet models and the original feature data as input, and input them into the XGBoost model for training. The XGBoost model can capture non-linear relationships and complex patterns, and integrate the advantages of the previous two models to obtain the final hybrid prediction model.

[0083] Model fusion method: Adopt the weighted average method to fuse the prediction results of the three models. The weights are determined by the cross-validation method, so that the fused prediction results have the optimal performance on the training set and the validation set.

[0084] (III) Analysis process Input the grouped traffic data into the established hybrid prediction model. The model first conducts time series analysis and feature extraction on the data, and then through the successive processing of the ARIMA, Prophet, and XGBoost models, finally outputs the traffic evaluation results, including traffic prediction values, prediction confidence intervals, etc.

[0085] IV. Anomaly Judgment and Handling Based on the DTW Dynamic Time Warping Algorithm (I) Basic principle of the DTW algorithm The DTW algorithm is an algorithm for measuring the similarity between two time series. Through the method of dynamic programming, find the optimal alignment path between the two time series to minimize the distance between them. In traffic analysis, the DTW algorithm can be used to compare the similarity between the actual traffic evaluation results and the normal traffic patterns to determine whether there are abnormal situations.

[0086] (II) Application to traffic evaluation result judgment Establishment of normal traffic patterns: Collect historical normal traffic data, and establish a time series pattern library of normal traffic through preprocessing and feature extraction.

[0087] Similarity calculation: Convert the current traffic evaluation results into time series data, and calculate the similarity distance with each pattern in the normal traffic pattern library using the DTW algorithm. Set a similarity threshold. When the similarity distance is greater than the threshold, it is judged as an abnormal situation.

[0088] Anomaly judgment criteria: Comprehensively consider factors such as similarity distance and traffic change amplitude, and formulate reasonable anomaly judgment criteria. For example, when the similarity distance exceeds the threshold and the traffic change amplitude exceeds the preset percentage, it is determined as an abnormal situation.

[0089] (3) Analysis and Handling of Abnormal Causes Analysis of abnormal causes: When an abnormal situation is determined, a detailed analysis of the abnormal traffic is carried out. Starting from characteristics such as time sensitivity, packet dispersion, and protocol diversity, analyze the performance of the abnormal traffic in these characteristics, and combine information such as the working status of IoT devices and the network environment to determine the abnormal causes, such as device failures, network attacks, malware infections, etc.

[0090] Blocking Abnormal Links and Traffic Usage On the IoT device side, remotely send instructions through the device management platform to close the network interface of the abnormal connection or disable the relevant application program to block the generation of abnormal traffic.

[0091] On the communication network side, cooperate with the operator to block the IP address or port of the abnormal link through network security devices such as firewalls and intrusion detection systems to prevent the transmission of abnormal traffic in the network.

[0092] On the server side, monitor and restrict the services related to the abnormal traffic, such as restricting the access frequency and closing the abnormal service ports, to ensure the safe and stable operation of the server.

[0093] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. In an Internet of Things SIM card traffic data analysis method, it is characterized in that The method for analyzing the traffic data of the Internet of Things SIM card includes the following steps: Obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data, and obtain the feature traffic data; Group the feature traffic data based on the improved CURE clustering algorithm to obtain grouped traffic data; Establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis, and obtain the traffic evaluation result; Use the DTW dynamic time warping algorithm to judge the traffic evaluation result. If it is judged as an abnormal situation, analyze the cause of the abnormal traffic, and block the abnormal link and traffic usage.

2. The method for analyzing the traffic data of an IoT SIM card according to claim 1, characterized in that, The step of obtaining the SIM card traffic data in the system, extracting the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data, and obtaining the feature traffic data includes: Obtain the SIM card traffic data in the system, define different time windows, and calculate the traffic mean, traffic variance, and peak period ratio for each time window to obtain the time sensitivity feature vector; Analyze the distribution of the SIM card traffic data, calculate the packet size standard deviation, packet size coefficient of variation, adjacent packet interval time, interval time mean, and interval time variance to obtain the packet dispersion feature vector; Identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, and count the proportion of each protocol in the traffic to obtain the protocol diversity feature vector.

3. The method for analyzing the traffic data of an IoT SIM card according to claim 1, wherein, The step of grouping the feature traffic data based on the improved CURE clustering algorithm to obtain grouped traffic data includes: Based on the CURE clustering algorithm, use the density-based representative point selection method to select points in the high-density area as representative points in each cluster; Introduce the weighted Euclidean distance and assign different weights according to the importance of each feature in the feature vector; During the process of clustering and merging the data, add the judgment of the stability of the clustered data after merging to obtain the improved CURE clustering algorithm.

4. The method for analyzing the traffic data of an IoT SIM card according to claim 1, wherein The step of grouping the feature traffic data based on the improved CURE clustering algorithm to obtain grouped traffic data includes: Perform standardization processing on the feature traffic data through Min-Max standardization, convert each feature value to the same dimension, and obtain the standard traffic data; Use the improved representative point selection method to select the initial representative points in the standard traffic data, set the initial number of clusters, and obtain the initial clustered traffic data; Calculate the distance between each cluster in the initial clustered traffic data, select the two clusters with the smallest distance for merging, update the representative points of the merged cluster during the merging process, and recalculate the distance from other clusters according to the improved distance metric method to obtain the second clustered traffic data; Perform clustering iteration based on the second clustered traffic data, and terminate the clustering when the number of clusters reaches the preset target number to obtain the grouped traffic data.

5. The method for analyzing the traffic data of an IoT SIM card according to claim 1, wherein, The establishment of the ARIMA-Prophet-XGBoost hybrid prediction model, inputting the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain a traffic evaluation result, includes: Obtain the time series data in the system, and use the ARIMA model to train the time series data to obtain the ARIMA prediction result. Take the ARIMA prediction result as an input feature, and input it together with the original time series data into the Prophet model for training. Use the Prophet model to handle seasonality and trend to obtain the Prophet model prediction result. Input the ARIMA prediction result, the Prophet model prediction result, and the time series data into the XGBoost model for training to obtain the ARIMA-Prophet-XGBoost hybrid prediction model.

6. The method for analyzing the traffic data of an IoT SIM card according to claim 1, wherein The establishment of the ARIMA-Prophet-XGBoost hybrid prediction model, inputting the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain a traffic evaluation result, further includes: Input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model, and perform time series analysis and feature extraction on the data. Through the processing of the three models of ARIMA, Prophet, and XGBoost, output the traffic evaluation result, including at least the traffic prediction value and the prediction confidence interval.

7. The method for analyzing the traffic data of an IoT SIM card according to claim 1, wherein Using the DTW dynamic time warping algorithm to judge the traffic evaluation result. If it is judged as an abnormal situation, analyze the cause of the abnormal traffic, and block the abnormal link and traffic usage, including: Convert the current traffic evaluation result into time series data, and calculate the similarity distance with each pattern in the normal traffic pattern library using the DTW algorithm. Set a similarity threshold. When the similarity distance is greater than the similarity threshold, it is judged as an abnormal situation. When it is judged as an abnormal situation, analyze the abnormal traffic from the features of time sensitivity, packet dispersion, and protocol diversity, and determine the cause of the abnormality in combination with the working status of the IoT device and the network environment information.

8. An Internet of Things SIM card traffic data analysis system, characterized in that, The IoT SIM card traffic data analysis system includes the following modules: A data collection module, used to obtain the SIM card traffic data in the system, extract the feature vectors of time sensitivity, packet dispersion, and protocol diversity in the SIM card traffic data to obtain the feature traffic data. A data classification module, used to group the feature traffic data based on the improved CURE clustering algorithm to obtain the grouped traffic data. A traffic evaluation module, used to establish an ARIMA-Prophet-XGBoost hybrid prediction model, input the grouped traffic data into the ARIMA-Prophet-XGBoost hybrid prediction model for analysis to obtain a traffic evaluation result. An analysis and judgment module is used to judge the traffic evaluation result by using the DTW (Dynamic Time Warping) algorithm. If it is judged as an abnormal situation, it analyzes the cause of the abnormal traffic and blocks the abnormal link and traffic usage.

9. The Internet of Things SIM card traffic data analysis system according to claim 8, wherein, The data acquisition module includes the following sub-modules: An acquisition sub-module is used to obtain the SIM card traffic data in the system, define different time windows, calculate the traffic mean, traffic variance, and peak period proportion for each time window, and obtain the time sensitivity feature vector. An analysis sub-module is used to analyze the distribution of the SIM card traffic data, calculate the standard deviation of the packet size, the coefficient of variation of the packet size, the interval time between adjacent packets, the mean interval time, and the interval time variance, and obtain the packet dispersion feature vector. An identification sub-module is used to identify the communication protocols used in the SIM card traffic data, including at least TCP, UDP, HTTP, MQTT, and CoAP, and count the proportion of each protocol in the traffic to obtain the protocol diversity feature vector.

10. A system for analyzing the traffic data of an IoT SIM card according to claim 8, characterized in that, The data classification module includes the following sub-modules: A selection sub-module is used to select points in the high-density area as representative points in each cluster based on the CURE (Clustering Using Representatives) clustering algorithm and the density-based representative point selection method. An introduction sub-module is used to introduce the weighted Euclidean distance and assign different weights according to the importance of each feature in the feature vector. A judgment sub-module is used to add the judgment of the stability of the clustered data after merging during the process of clustering and merging the data to obtain an improved CURE clustering algorithm.

Images