Lightweight Internet of Things malicious traffic detection method and device based on large model driving

Abstract

The embodiment of the invention discloses a lightweight Internet of Things malicious traffic detection method and device based on large model driving. A specific embodiment of the method comprises the following steps: carrying out traffic acquisition operation in an internet environment to obtain an original traffic data set; performing data enhancement processing on the malicious traffic data set through a large language model to generate each piece of sample malicious traffic data; constructing each task traffic data set according to each piece of sample malicious traffic data and the original traffic data set; performing meta-feature selection processing on each task traffic data set in each task traffic data set to generate traffic meta-feature data; training each classifier model according to each task traffic data set and each corresponding target traffic feature data to obtain a target classifier model; and inputting the to-be-detected traffic data into the target classifier model to obtain a malicious traffic monitoring result. According to the embodiment, high-reliability malicious traffic detection can be effectively realized for the lightweight Internet.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more particularly to a lightweight Internet of Things malicious traffic detection method and device based on large model driving. Background Art

[0002] With the rapid development of IoT technology, the number of connected devices worldwide has exceeded tens of billions. The deep integration of scenarios such as smart cities and the Industrial Internet of Things has significantly improved social productivity. However, IoT devices suffer from limited computing resources, weak security protections, diverse devices, and data sparsity, making them a prime target for cyberattacks. In this context, building a lightweight and efficient malicious traffic detection mechanism for the IoT has become a core challenge in ensuring IoT security. Currently, lightweight malicious traffic detection on the Internet is commonly performed using detection methods based on rule matching, machine learning, or deep learning models.

[0003] IoT traffic features highly heterogeneous characteristics, making traditional feature selection methods ineffective in capturing hidden attack patterns. Furthermore, deep learning models (such as ResNet and LSTM models) have limited ability to represent temporal protocol features and rely on large amounts of annotated data, resulting in unstable detection performance when data is scarce. Furthermore, existing methods suffer from high computational overhead, making them unsuitable for deployment on resource-constrained IoT devices. They also suffer from high false positive rates and poor generalization. Summary of the Invention

[0004] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] Some embodiments of the present disclosure propose a lightweight IoT malicious traffic detection method and device based on large model driving to solve the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a lightweight IoT malicious traffic detection method driven by a large model, the method comprising: performing traffic collection operations in an Internet environment to obtain an original traffic data set, wherein the above original traffic data includes a malicious traffic data set and a normal traffic data set; performing data enhancement processing on the malicious traffic data set through a large language model to generate each sample malicious traffic data set; constructing each task traffic data set based on the above each sample malicious traffic data set and the original traffic data set; performing meta-feature selection processing on each task traffic data set in the above each task traffic data set to generate traffic meta-feature data; training each classifier model based on the above each task traffic data set and the corresponding each target traffic feature data to obtain a target classifier model; inputting the traffic data to be detected into the above target classifier model to obtain a malicious traffic monitoring result.

[0007] In a second aspect, some embodiments of the present disclosure provide a lightweight IoT malicious traffic detection device driven by a large model, the device comprising: a traffic collection unit, configured to perform traffic collection operations in an Internet environment to obtain an original traffic data set, wherein the above-mentioned original traffic data includes a malicious traffic data set and a normal traffic data set; a data enhancement unit, configured to perform data enhancement processing on the malicious traffic data set through a large language model to generate each sample malicious traffic data set; a construction unit, configured to construct each task traffic data set based on the above-mentioned each sample malicious traffic data set and the original traffic data set; a feature selection unit, configured to perform meta-feature selection processing on each task traffic data set in the above-mentioned each task traffic data set to generate traffic meta-feature data; a training unit, configured to train each classifier model based on the above-mentioned each task traffic data set and the corresponding each target traffic feature data to obtain a target classifier model; an input unit, configured to input the traffic data to be detected into the above-mentioned target classifier model to obtain a malicious traffic monitoring result.

[0008] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0009] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0010] The aforementioned embodiments of the present disclosure have the following beneficial effects: The large-model-driven lightweight IoT malicious traffic detection methods of some embodiments of the present disclosure can effectively achieve highly reliable malicious traffic detection for lightweight Internet applications. Specifically, the low reliability of malicious traffic detection in relevant lightweight Internet scenarios is due to the highly heterogeneous nature of IoT traffic features, making traditional feature selection methods difficult to effectively capture hidden attack patterns. Secondly, deep learning models (such as ResNet and LSTM models) have limited ability to represent timing protocol features and rely on large amounts of annotated data, resulting in unstable detection performance when data is scarce. Furthermore, existing methods have high computational overhead, making them unsuitable for deployment on resource-constrained IoT devices, and they also suffer from high false positive rates and poor generalization. Therefore, the large-model-driven lightweight IoT malicious traffic detection methods of some embodiments of the present disclosure provide a new approach to lightweight IoT malicious traffic detection. First, the large-model-based small-sample data augmentation algorithm can enhance malicious traffic data in small-sample scenarios, thereby improving detection effectiveness. Secondly, a robust traffic feature selection algorithm based on meta-learning can dynamically select robust features and quickly adapt to new attack traffic detection tasks. Finally, a multi-classifier optimization malicious traffic detection algorithm based on grid search is used to optimize the hyperparameters of multiple detection models. The collaborative mechanism of multiple detection models is then used to effectively achieve highly reliable malicious traffic detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0012] Figure 1 Schematic diagram of the MTDM-LMD architecture of a lightweight IoT malicious traffic detection method based on a large model drive according to some embodiments of the present disclosure;

[0013] Figure 2 is a flowchart of some embodiments of a lightweight IoT malicious traffic detection method driven by a large model according to the present disclosure;

[0014] Figure 3 1 is a schematic structural diagram of some embodiments of a lightweight IoT malicious traffic detection device driven by a large model according to the present disclosure;

[0015] Figure 4 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure;

[0016] Figure 5This is a schematic diagram comparing the characteristics of malicious traffic generated by the lightweight IoT malicious traffic detection method based on a large model drive according to the present disclosure and the real malicious traffic;

[0017] Figure 6 2. It is a performance comparison diagram of different traffic generation algorithms for the lightweight IoT malicious traffic detection method driven by a large model according to the present disclosure;

[0018] Figure 7 Schematic diagram of the distribution of selected traffic features between normal traffic and malicious traffic according to the lightweight IoT malicious traffic detection method driven by a large model disclosed herein;

[0019] Figure 8 This is a schematic diagram of the time consumption of different traffic feature selection algorithms according to the lightweight IoT malicious traffic detection method driven by a large model disclosed in the present invention;

[0020] Figure 9 3. This is a schematic diagram comparing the model accuracy before and after hyperparameter optimization of the lightweight IoT malicious traffic detection method based on a large model drive according to the present disclosure;

[0021] Figure 10 This is a performance comparison diagram of different classification algorithms for the lightweight IoT malicious traffic detection method driven by a large model according to the present disclosure. DETAILED DESCRIPTION

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0023] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0024] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0025] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0026] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0027] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0028] Figure 1 This is a schematic diagram of the MTDM-LMD architecture of a lightweight IoT malicious traffic detection method driven by a large model according to some embodiments of the present disclosure.

[0029] exist Figure 1 In the architectural diagram, the MTDM-LMD framework is mainly composed of three modules: large model data enhancement (LLM-Based Data Augmentation), machine meta-learning dynamic feature selection (Meta-Learning-Based Feature Selection) and grid search multi-classifier detection (Grid Search Multi-Classifier Detection). Figure 1 As shown, the computing device first maps captured real malicious traffic data into natural language using a large model. Then, using natural language and prior knowledge, it generates sample traffic that approximates real malicious traffic. Adversarial detection is used to continuously adjust the quality of the generated data. The enhanced generated traffic and the original traffic are processed through meta-learning dynamic feature selection. The machine can adjust detection parameters in real time based on weights to select robust traffic features. After the feature selection process, a grid-search-based multi-classifier detection is used, and a voting mechanism is employed to reduce the bias of detection in a single traffic classification model. Finally, the malicious traffic detection results are obtained.

[0030] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, or it can be implemented as a single software or software module. No specific limitation is made here. It should be understood that Figure 1 The number of computing devices in can be any number according to implementation requirements.

[0031] Continue to refer Figure 2 , shows a process 200 of some embodiments of the lightweight IoT malicious traffic detection method based on a large model drive according to the present disclosure. The lightweight IoT malicious traffic detection method based on a large model drive includes the following steps:

[0032] Step 201: perform traffic collection operations in an Internet environment to obtain an original traffic data set.

[0033] In some embodiments, the execution entity (e.g., a computing device) of the large-model-driven lightweight IoT malicious traffic detection method can perform traffic collection operations in an Internet environment to obtain a raw traffic data set. The raw traffic data includes a malicious traffic data set and a normal traffic data set. The raw traffic data in the raw traffic data set can be structured data that has been manually annotated (to mark normal traffic data and malicious traffic data). In practice, the execution entity can deploy a data capture tool (e.g., tcpdump, Wireshark, or Bro / Zeek) at the edge node to capture raw network data packets and extract key fields (e.g., source / destination IP, port, protocol type, packet size, timestamp, flags, etc.) to obtain structured raw traffic data as the raw traffic data set.

[0034] It should be noted that the method disclosed in this disclosure uses the BoTIoT-2018 dataset as the original traffic dataset.

[0035] In step 202, the malicious traffic data set is enhanced using a large language model to generate malicious traffic data of various samples.

[0036] In some embodiments, the execution entity may perform data enhancement processing on the malicious traffic data set using a large language model to generate individual sample malicious traffic data. The individual sample malicious traffic data may be artificially generated malicious traffic data.

[0037] In some optional implementations of some embodiments, the execution entity may perform data enhancement processing on the malicious traffic dataset using a large language model to generate each sample of malicious traffic data through the following steps:

[0038] In the first step, for each malicious traffic data in the above malicious traffic dataset, the following data augmentation steps are performed:

[0039] In the first sub-step, the malicious traffic data is semantically mapped through the above-mentioned large language model to obtain a description text of the malicious traffic data. Structured traffic data usually has a clear format and semantics. Through semantic mapping, the large model can convert malicious traffic data into a natural language description. For example, converting the input real traffic file data "pkSeqID:3000040, flgs: e, flgs_number:1, proto:udp, proto_number:..." into the natural language "the data packet sequence number is 3000040, the flag is e, the value of the flag is 1, the protocol type is udp, and the protocol number is 3" can help the large language model better learn the characteristics of the data. By adding rich contextual information and making it possible to more accurately understand the true meaning of the malicious traffic data in the subsequent generation of diversified sample malicious traffic data. In practice, assuming that the real structured malicious traffic data obtained from the Internet , then through the large language model The mapped natural language is used as the description text of malicious traffic data .

[0040] The second sub-step is to perform prompt engineering on the above-mentioned large language model through the malicious data description text to generate various malicious traffic data to be verified. The above-mentioned prompt engineering refers to dynamically guiding the large model through artificial design and construction of precise prompt words, so that the large model can output higher quality content. After semantically mapping the structured data to obtain the malicious traffic data description text, rich background and context information is added on the basis of the high interpretability of natural language. For example, when enhancing the malicious traffic data generated by the dataset, adding the following information can generate samples that are closer to the real traffic: "The generated malicious traffic description must include abnormal protocol behavior or load characteristics. Improve sample diversity through temperature coefficient adjustment and **feature constraint instructions** (such as "generate different IPs, ports and load modes"), while ensuring compliance with protocol specifications (such as MQTT payload length ≤ 256MB)." As a learning example of malicious traffic, input large language model middle, Will be based on Perform input semantic parsing, deep feature extraction, generative mapping, dynamic verification and feedback to analyze the data structure of malicious traffic And feature distribution , in order to generate malicious traffic data to be verified that is close to real malicious traffic For different types of attack traffic, through dynamic guidance and multiple rounds of dialogue, the large model continuously learns the characteristics of different types of attack traffic, thereby generating malicious traffic samples that are close to the real situation, increasing the diversity of attack sample generation.

[0041] The third sub-step is to perform post-verification on each malicious traffic data to be verified to obtain each sample malicious traffic data. First, conduct adversarial verification: the generated malicious traffic to be verified is input as a sample into the binary classifier. The classifier will calculate the probability that it is real traffic data based on the characteristics of the sample. If the accuracy is relatively low, it means that the classifier has difficulty distinguishing between real malicious traffic and generated malicious traffic, and the generated malicious traffic is closer to the real traffic. After the adversarial verification, the verification result accuracy rate is output. If the accuracy rate is lower than the threshold τ, the generated malicious traffic is considered qualified and can be directly used as sample malicious traffic data. If the rate is higher than the threshold τ, it enters the data optimization stage. , using prior knowledge , including protocol specifications, traffic characteristics, time series, statistical parameters, network behavior, attack identification, incremental rules, etc. to correct the contradictory fields in the data and generate Specifically, the prior knowledge in the protocol specifications, such as the HTTP protocol specifications, must strictly follow the request method (such as GET, POST), status code (such as 200, 404) and header information (such as User-Agent, Host), the prior knowledge in traffic characteristics (such as DDos attack is an attack characterized by a large number of small data packets), and the prior knowledge in attack identification (such as SQL attack will contain SQL statement fragments in the request parameters). Finally, the entire algorithm outputs As the final generated sample malicious traffic data.

[0042] Step 203: construct each task traffic data set based on each sample malicious traffic data and the original traffic data set.

[0043] In some embodiments, the execution subject may construct each task traffic data set based on each sample malicious traffic data and the original traffic data set. In practice, the execution subject first generates each sample malicious traffic sample. Compared with the original traffic dataset Merge into a task traffic dataset for traffic detection , different attacks generate different data sets, let the generated data set of all task traffic be . Each Divided into training traffic data With test traffic data .

[0044] Step 204 : performing meta-feature selection processing on each task traffic data set in each task traffic data set to generate traffic meta-feature data.

[0045] In some embodiments, the execution entity may perform meta-feature selection processing on each of the task traffic data sets to generate traffic meta-feature data.

[0046] In some optional implementations of some embodiments, the execution entity may perform meta-feature selection processing on each of the task traffic data sets in the above-mentioned task traffic data sets through the following steps to generate target traffic feature data:

[0047] In the first step, for each of the above task traffic data sets, perform the following steps:

[0048] The first sub-step is to extract the traffic meta-features from the task traffic dataset to obtain traffic meta-feature data. In practice, the execution entity can extract the traffic meta-features from the task traffic dataset. Extracting raw feature data , and the following is the traffic data set for each task Extracting meta-features ,These meta-features include statistical features such as the mean and variance of the ,traffic and data distribution.

[0049] The second sub-step is to train the meta-learner with the above traffic meta-feature data to obtain the trained meta-learner. In practice, different traffic detection tasks (i.e., task traffic datasets) will be used to extract a common model parameter during the meta-learner training phase. For each traffic detection task , using the meta-features extracted from and the initial model parameters Train the meta-learner and calculate the task loss Then sum up all the task losses and find the parameters that minimize the detection task loss. , the formula is as follows:

[0050] .

[0051] The meta-learning method used in practice is based on optimization. By optimizing the loss functions of multiple tasks, it finds universal model parameters, allowing the traffic detection model to quickly adapt to new tasks. Robust traffic feature selection based on meta-learning can quickly adapt to new tasks by learning from multiple related malicious traffic detection tasks. This dynamically helps the model select robust features and more quickly identify attack traffic.

[0052] The third sub-step is to perform dynamic robust flow feature selection on the above-mentioned flow meta-feature data through the above-mentioned post-training meta-learning to obtain each target flow feature data. and the weight increment for task performance adjustment Calculate the importance of the nth feature in the above original feature data ,According to different traffic detection tasks (i.e., task traffic datasets), Will be updated, combined with smoothing parameters , the calculation formula for dynamically adjusting feature weights is as follows:

[0053] .

[0054] Afterwards, the meta-learner model selects the top N features with the most important feature weights to generate the selected robust traffic feature subset (i.e., each target traffic feature), and obtain each target traffic feature data for subsequent classifier training.

[0055] Step 205 : training each classifier model according to each task traffic data set and each corresponding target traffic feature data to obtain a target classifier model.

[0056] In some embodiments, the execution subject can train each classifier model based on each task traffic data set and the corresponding target traffic feature data to obtain a target classifier model. The first classifier, the second classifier, and the third classifier included in the classifier model can be XGBoost, LightGBM, and CatBoost classifier models, respectively. Malicious traffic detection algorithm uses robust traffic feature subsets Classification is performed using three classification algorithms, XGBoost, LightGBM, and CatBoost, after grid search hyperparameter optimization. All three classification models / algorithms are based on the principle of gradient boosting decision trees, building a powerful ensemble model through iterative decision tree training. They all demonstrate high efficiency when processing large datasets. After the three classifiers each make traffic predictions, a voting mechanism is used to determine the final malicious traffic detection result.

[0057] In some optional implementations of some embodiments, the execution entity may train each classifier model according to each of the task traffic data sets and the corresponding target traffic feature data through the following steps to obtain a target classifier model:

[0058] In the first step, for each of the above task traffic data sets, perform the following steps:

[0059] In the first sub-step, the hyperparameters of the first classifier, the second classifier, and the third classifier are optimized using the target traffic feature data corresponding to the task traffic dataset to obtain the first optimal hyperparameter, the second optimal hyperparameter, and the third optimal hyperparameter. It consists of a traffic feature matrix X and a target vector Y. Part of X and Y will be used as a training set for grid search hyperparameter optimization. The parameters of the three classification algorithms, XGBoost, LightGBM, and CatBoost, include learning rate, regularization parameter, tree depth, maximum number of iterations, etc. These parameters together constitute the parameter space Ω. The hyperparameter optimization algorithm traverses the parameter space Ω and uses each parameter combination for cross-validation training and evaluation. Finally, the parameter combination with the highest validation set score is selected. (i.e., the first optimal hyperparameter, the second optimal hyperparameter, and the third optimal hyperparameter) are used to train the model. In practice, the execution entity can use a grid search hyperparameter optimization algorithm to optimize the hyperparameters of the first, second, and third classifiers to obtain the first optimal hyperparameter, the second optimal hyperparameter, and the third optimal hyperparameter.

[0060] In the second sub-step, based on the first optimal hyperparameter, the second optimal hyperparameter and the third optimal hyperparameter, the first classifier, the second classifier and the third classifier are trained through the target traffic feature data corresponding to the task traffic dataset to obtain the target classifier model. In practice, the execution subject can substitute the first optimal hyperparameter, the second optimal hyperparameter and the third optimal hyperparameter into the corresponding classifier model, and then use the task traffic dataset to train the target classifier model. Including training flow data and test traffic data The first, second and third classifiers are trained. After the three trained models are trained on the same training set, they make predictions respectively. 、 、 The above-mentioned execution entity will count the most frequently appearing options among multiple prediction results to determine the final prediction result. Voting prediction can effectively reduce the bias caused by a single classification model in prediction, and the trained first, second, and third classifiers are determined as the target classifier models.

[0061] Step 206: Input the traffic data to be detected into the target classifier model to obtain malicious traffic monitoring results.

[0062] In some embodiments, the execution entity may input the traffic data to be detected into the target classifier model to obtain malicious traffic monitoring results.

[0063] In some optional implementations of some embodiments, the execution entity may input the traffic data to be detected into the target classifier model to obtain malicious traffic detection results through the following steps:

[0064] The first step is to extract target traffic features from the traffic data to be tested, thereby obtaining the traffic feature data to be tested. In practice, the execution entity may extract target features corresponding to the task type (attack type) of the traffic data to be tested from the traffic data to obtain the traffic feature data to be tested.

[0065] The second step is to input the above traffic feature data to the target classifier model to obtain the malicious traffic detection results. In practice, the target classifier model includes three optimized classification models to make predictions respectively. 、 、 The above-mentioned execution entity will count the options that appear most frequently in the prediction results to determine the final prediction result. Through voting prediction, the bias caused by a single classification model in the prediction can be effectively reduced, thereby improving the detection robustness of the malicious traffic algorithm and obtaining the malicious traffic detection results.

[0066] Performance analysis experiments for this disclosed lightweight, large-model-driven IoT malicious traffic detection method were conducted under the Windows 11 operating system, using a 13th Gen Intel(R) Core(TM) i5-13500H 2.60 GHz CPU, 16GB of RAM, and Python version 3.11. The BoTIoT-2018 dataset was used for model learning. This dataset covers a wide range of attack traffic types and is generated from a real testbed. It outperforms other traffic datasets in terms of both diversity and authenticity. It is well-suited for meta-learning algorithms to learn from diverse attack traffic, thereby improving the efficiency of robust feature selection algorithms.

[0067] First, enhance performance testing for small sample malicious traffic data:

[0068] First, a set of malicious traffic was generated using the traditional synthetic minority oversampling algorithm. The synthetic minority oversampling algorithm used simple interpolation to generate malicious traffic until the predetermined oversampling ratio was reached, which served as a comparative experiment. At the same time, a set of malicious traffic was generated using the large model and optimized through multiple rounds of questioning. The t-SNE clustering results of the two malicious traffic generation results and the real malicious traffic are shown in Figure 2. Figure 5 shown.

[0069] from Figure 5As can be seen from the figure, the real malicious traffic forms multiple clustering areas in the clustering diagram. The malicious traffic generated by the synthetic minority oversampling algorithm is consistent with the real malicious traffic in some feature distributions, but there are still many differences. The optimized malicious traffic generated by the large model maintains a high distribution consistency with the real malicious traffic in feature distribution, indicating that it is closer to the real sample. At the same time, an adversarial test is performed on the optimized traffic generated based on the large model. After mixing the generated traffic with the real traffic, the classifier is asked to classify it. If the classifier has difficulty distinguishing, it means that the generated traffic is close to the real traffic. In actual experiments, three classifiers can be selected: complete feature selection, variance selection, and ridge regression. The detection accuracy is 0.536, which cannot clearly distinguish the two types of traffic. It also shows that the large model has a good enhancement effect on the generated data.

[0070] In addition, the experiment also uses the traffic generated by these two methods and mixed with real normal traffic samples to detect malicious traffic. Because the optimized malicious traffic generated by the large model maintains a high distribution consistency with the real malicious traffic in terms of feature distribution, the model has better classification performance through learning, and the accuracy will be higher than that of the simple random oversampling algorithm. Figure 6 shown.

[0071] Second, for the performance analysis of feature selection algorithm:

[0072] In the feature selection algorithm performance analysis, traffic meta-features of various attacks were extracted from the BoTIoT-2018 dataset. The traffic feature weights were dynamically calculated and selected based on the meta-features and initial parameters. Finally, the top six features with the largest weights were selected from the 42 traffic features. The six features and their weight scores are as follows: proto_number (network protocol number): 0.12567421, flgs (TCP flag bit combination): 0.1215754, flgs_number (TCP flag bit combination numerical value): 0.039347, proto (network protocol name): 0.032881, TnP_PerDst (number of source IP connections under each target IP address): 0.026371, saddr (source IP address): 0.025835. Figure 7 This figure shows the distribution of the selected traffic features between normal and malicious traffic. The horizontal axis represents the range of the traffic feature value, and the vertical axis represents the frequency of occurrence of the traffic feature. Each sub-figure uses a histogram and kernel density estimation (KDE) curve to illustrate the difference in the distribution of the traffic feature between normal and malicious traffic.

[0073] In this experiment, the feature selection time and accuracy based on different feature selection algorithms are as follows: Figure 8As shown. The feature selection algorithm based on model importance of meta-learning shows obvious advantages. Its time consumption is 342.561ms, which is much lower than other algorithms. When analyzing the same malicious traffic task, it also shows a higher accuracy of 99.50%. For this result, the analysis is as follows: Assume that the number of features is n, the number of traffic detection tasks is M, the number of samples for each task is N, the number of samples for the new task is K, and the complexity of the model update is T. Then the complexity of the meta-training stage is O (M × N + K × T), which can show obvious advantages when facing detection tasks with high similarity of traffic feature sets. In comparison, both the forward feature selection and full feature selection algorithms are O (n 2 ), the computational complexity is high. The computational complexity of the backward feature selection is O(2 n ), it is necessary to iteratively remove features from the feature subset containing all features and generate a new feature subset. Therefore, the time consumption is particularly increased when there are many features.

[0074] Experiments have shown that when faced with traffic detection tasks that require dynamic feature selection, the meta-learning algorithm can flexibly assign weights to features to quickly respond and provide good feature selection effects.

[0075] Third, for malicious traffic detection algorithm testing:

[0076] Before conducting malicious traffic detection, the model parameters are optimized using the grid search hyperparameter algorithm. In this experiment, the detection accuracy of three classification algorithms, XGBoost, LightGBM, and CatBoost, is recorded. The experimental results are as follows: Figure 9 As shown in the figure, the grid hyperparameter algorithm traverses the parameter space and searches for the optimal parameter combination for each model. As can be seen from the figure, the detection accuracy of the three models has been improved after grid hyperparameter optimization.

[0077] In order to evaluate the performance level of the multi-classifier detection algorithm, the detection results of XGBoost, LightGBM, CatBoost and the multi-classifier detection algorithm are compared under the condition of selecting the same detection task. The F1Score, Recall, Precision and Accuracy obtained in the experiment are as follows: Figure 10 shown.

[0078] like Figure 10As shown in , the multi-classifier detection algorithm outperforms the single classifier in F1 Score, Recall, Precision and Accuracy, especially in Precision, which reaches 100%. This shows that the model has a high proportion of positive samples among the samples predicted as positive by the model. The advantage of the voting mechanism of the multi-classifier is that it reduces the variance. Assuming that the error rates of the three classifiers are , then the error rate after integrating the three models is:

[0079] .

[0080] when When 0.3, In this way, through multi-model integration, the error rate can be reduced by 28% compared to the single model error rate of 0.3.

[0081] In summary, different models have different sensitive areas in the feature space, and multi-model voting can reduce variance, improve detection accuracy, and enable the malicious traffic detection model to have higher attack detection capabilities.

[0082] The aforementioned embodiments of the present disclosure have the following beneficial effects: The large-model-driven lightweight IoT malicious traffic detection methods of some embodiments of the present disclosure can effectively achieve highly reliable malicious traffic detection for lightweight Internet applications. Specifically, the low reliability of malicious traffic detection in relevant lightweight Internet scenarios is due to the highly heterogeneous nature of IoT traffic features, making traditional feature selection methods difficult to effectively capture hidden attack patterns. Secondly, deep learning models (such as ResNet and LSTM models) have limited ability to represent timing protocol features and rely on large amounts of annotated data, resulting in unstable detection performance when data is scarce. Furthermore, existing methods have high computational overhead, making them unsuitable for deployment on resource-constrained IoT devices, and they also suffer from high false positive rates and poor generalization. Therefore, the large-model-driven lightweight IoT malicious traffic detection methods of some embodiments of the present disclosure provide a new approach to lightweight IoT malicious traffic detection. First, the large-model-based small-sample data augmentation algorithm can enhance malicious traffic data in small-sample scenarios, thereby improving detection effectiveness. Secondly, a robust traffic feature selection algorithm based on meta-learning can dynamically select robust features and quickly adapt to new attack traffic detection tasks. Finally, a multi-classifier optimization malicious traffic detection algorithm based on grid search is used to optimize the hyperparameters of multiple detection models. The collaborative mechanism of multiple detection models is then used to effectively achieve highly reliable malicious traffic detection.

[0083] Further references Figure 3As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a lightweight IoT malicious traffic detection device driven by a large model. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the lightweight IoT malicious traffic detection device driven by a large model can be specifically applied to various electronic devices.

[0084] like Figure 3 As shown, a lightweight IoT malicious traffic detection device 300 driven by a large model in some embodiments includes: a traffic collection unit 301, a data enhancement unit 302, a construction unit 303, a feature selection unit 304, a training unit 305 and an input unit 306. Among them, the traffic collection unit 301 is configured to perform traffic collection operations in an Internet environment to obtain an original traffic data set, wherein the above-mentioned original traffic data includes a malicious traffic data set and a normal traffic data set; the data enhancement unit 302 is configured to perform data enhancement processing on the malicious traffic data set through a large language model to generate each sample malicious traffic data; the construction unit 303 is configured to construct each task traffic data set based on the above-mentioned each sample malicious traffic data and the original traffic data set; the feature selection unit 304 is configured to perform meta-feature selection processing on each task traffic data set in the above-mentioned each task traffic data set to generate traffic meta-feature data; the training unit 305 is configured to train each classifier model based on the above-mentioned each task traffic data set and the corresponding each target traffic feature data to obtain a target classifier model; the input unit 306 is configured to input the traffic data to be detected into the above-mentioned target classifier model to obtain a malicious traffic monitoring result.

[0085] It is understandable that the units recorded in the lightweight IoT malicious traffic detection device 300 driven by a large model are similar to those in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the lightweight IoT malicious traffic detection device 300 driven by a large model and the units contained therein, and will not be repeated here.

[0086] Reference below Figure 4 , which shows a structural schematic diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Figure 6As shown, the computer device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions, which, when executed, may enable the processor to execute any of the above methods. The processor is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, may enable the processor to execute any of the above methods. The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present disclosure, and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0087] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0088] In one embodiment, the processor is used to run a computer program stored in a memory to implement the following steps: performing traffic collection operations in an Internet environment to obtain an original traffic data set, wherein the original traffic data includes a malicious traffic data set and a normal traffic data set; performing data enhancement processing on the malicious traffic data set through a large language model to generate each sample malicious traffic data set; constructing each task traffic data set based on the above each sample malicious traffic data set and the original traffic data set; performing meta-feature selection processing on each task traffic data set in the above each task traffic data set to generate traffic meta-feature data; training each classifier model based on the above each task traffic data set and the corresponding each target traffic feature data to obtain a target classifier model; inputting the traffic data to be detected into the above target classifier model to obtain a malicious traffic monitoring result.

[0089] An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the method described above in the present disclosure.

[0090] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., provided on the computer device.

[0091] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0092] The above descriptions are merely some preferred embodiments of the present disclosure and illustrate the underlying technical principles. Those skilled in the art should understand that the scope of the invention encompassed by the embodiments of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A lightweight IoT malicious traffic detection method based on large model driving, characterized by: include: Performing a traffic collection operation in an Internet environment to obtain an original traffic data set, wherein the original traffic data includes a malicious traffic data set and a normal traffic data set; Performing data enhancement processing on the malicious traffic dataset using a large language model to generate malicious traffic data for each sample; Constructing each task traffic data set based on each sample malicious traffic data and the original traffic data set; Performing meta-feature selection processing on each task traffic data set in each task traffic data set to generate traffic meta-feature data; Training each classifier model according to each task traffic data set and each corresponding target traffic feature data to obtain a target classifier model; The traffic data to be detected is input into the target classifier model to obtain malicious traffic monitoring results.

2. The method according to claim 1, characterized in that The malicious traffic dataset is enhanced using a large language model to generate malicious traffic data for each sample, including: For each malicious traffic data in the malicious traffic dataset, the following data enhancement steps are performed: Using the large language model, semantic mapping is performed on the malicious traffic data to obtain a description text of the malicious traffic data; Using malicious data description text, the large language model is prompted to generate various malicious traffic data to be verified; Post-verification is performed on each malicious traffic data to be verified to obtain each sample malicious traffic data.

3. The method according to claim 2, characterized in that The performing meta-feature selection processing on each task traffic data set in each task traffic data set to generate target traffic feature data includes: For each of the task traffic data sets, perform the following steps: Extracting flow meta-features from the task flow data set to obtain flow meta-feature data; Training a meta-learner using the traffic meta-feature data to obtain a trained meta-learner; Through the post-training meta-learning, dynamic robust flow feature selection is performed on the flow meta-feature data to obtain various target flow feature data.

4. The method according to claim 3, characterized in that The method of training each classifier model according to each task traffic data set and each corresponding target traffic feature data to obtain a target classifier model includes: For each of the task traffic data sets, perform the following steps: Performing hyperparameter optimization on the first classifier, the second classifier, and the third classifier of each classifier using the target traffic feature data corresponding to the task traffic data set to obtain the first optimal hyperparameter, the second optimal hyperparameter, and the third optimal hyperparameter; Based on the first optimal hyperparameter, the second optimal hyperparameter and the third optimal hyperparameter, the first classifier, the second classifier and the third classifier are trained through the target traffic feature data corresponding to the task traffic data set to obtain a target classifier model.

5. The method according to claim 4, characterized in that The inputting of the traffic data to be detected into the target classifier model to obtain malicious traffic detection results includes: Extracting target flow characteristics from the flow data to be detected to obtain flow characteristic data to be detected; The traffic feature data to be detected is input into the target classifier model to obtain malicious traffic detection results.

6. A lightweight IoT malicious traffic detection device based on a large model drive, characterized in that: include: A traffic collection unit is configured to perform traffic collection operations in an Internet environment to obtain an original traffic data set, wherein the original traffic data includes a malicious traffic data set and a normal traffic data set; The data enhancement unit is configured to perform data enhancement processing on the malicious traffic dataset using a large language model to generate malicious traffic data for each sample; A construction unit is configured to construct each task traffic data set according to each sample malicious traffic data and the original traffic data set; a feature selection unit configured to perform meta-feature selection processing on each task traffic data set in each task traffic data set to generate traffic meta-feature data; The training unit is configured to train each classifier model according to each task traffic data set and each corresponding target traffic feature data to obtain a target classifier model; The input unit is configured to input the traffic data to be detected into the target classifier model to obtain malicious traffic monitoring results.

7. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable medium, characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images