Abnormal traffic detection method and device and electronic equipment

Abstract

The invention discloses an abnormal traffic detection method and device and electronic equipment. Relates to the field of artificial intelligence, and the method comprises the steps: obtaining target traffic data which is network traffic data collected and processed on a network edge device; extracting target feature data matched with the N features from the target traffic data; and on the basis of the target feature data, an abnormal traffic detection model is adopted to obtain an abnormal detection result of the target traffic data, the abnormal traffic detection model is obtained by carrying out classified learning on a lightweight width learning system model on the basis of M groups of feature data, and the M groups of feature data are feature data extracted from the M groups of traffic data and matched with N features. Through application of the method and the device, the problems of abnormal traffic detection delay and low response efficiency caused by the fact that the real-time operation capability of a deep learning model on a resource-constrained edge device is limited by high calculation complexity and large storage requirements in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to an abnormal traffic detection method, device and electronic equipment. Background Art

[0002] The solutions for abnormal traffic detection in edge computing in related technologies are mainly based on shallow machine learning methods and deep learning methods. The model structure of the shallow machine learning method is simple, but the detection rate and accuracy are low; while the deep learning method has a high detection rate, but the training process is complicated, with high storage and computing overhead, and high requirements for various resources and hardware, which is not suitable for edge computing devices with limited resources. In addition, the standard width learning system in related technologies is a new and efficient flat neural network with a simple network structure and fast training. It can greatly shorten the training time of the neural network while ensuring accuracy. However, when inputting data with a large number of samples or high-dimensional data, the use of pseudo-inverse algorithms will place high requirements on the memory of the computing device, which limits the implementation of the width learning system on devices with limited resources, and there is a problem that the detection accuracy and computing and storage overhead cannot be taken into account.

[0003] The high computational complexity and large storage requirements in related technologies limit the real-time operation capabilities of deep learning models on resource-constrained edge devices, resulting in delays in abnormal traffic detection and low response efficiency. No effective solution has been proposed so far. Summary of the invention

[0004] The main purpose of this application is to provide an abnormal traffic detection method, device and electronic device to solve the problem that high computational complexity and large storage requirements in related technologies limit the real-time operation capability of deep learning models on resource-constrained edge devices, resulting in abnormal traffic detection delays and low response efficiency.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, an abnormal traffic detection method is provided. The method includes: obtaining target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; extracting target feature data matching N features from the target traffic data, wherein the N features are obtained by feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; based on the target feature data, using an abnormal traffic detection model to obtain an abnormal detection result of the target traffic data, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data matching the N features extracted from the M groups of traffic data.

[0006] Optionally, before obtaining anomaly detection results of target traffic data by using an abnormal traffic detection model based on target feature data, the method further includes: determining importance scores corresponding to K features respectively by using a feature selection algorithm based on M groups of traffic data, wherein each group of traffic data includes feature values ​​corresponding to K features respectively, and K is an integer greater than N; based on the importance scores corresponding to the K features respectively, selecting N features from the K features.

[0007] Optionally, based on M groups of traffic data, a feature selection algorithm is used to determine the importance scores corresponding to the K features, including: performing feature selection on the M groups of traffic data to obtain M groups of initial data, wherein the feature selection is used to remove redundant features in the M groups of traffic data; based on the M groups of initial data, an initial extreme gradient boosting model is trained to obtain a target extreme gradient boosting model; based on the target extreme gradient boosting model, the importance scores corresponding to the K features are determined.

[0008] Optionally, based on the target extreme gradient boosting model, determining the importance scores corresponding to the K features respectively, including: determining the number of times the K features are used in the decision trees included in the target extreme gradient boosting model; determining the importance scores corresponding to the K features based on the number of times the K features are used in the decision trees included in the target extreme gradient boosting model.

[0009] Optionally, before obtaining anomaly detection results of target traffic data by using an abnormal traffic detection model based on target feature data, the method also includes: extracting feature data matching N features from M groups of traffic data to obtain M groups of feature data; based on the M groups of feature data, using shift quantization technology to perform classification learning on the lightweight width learning system model to obtain an abnormal traffic detection model.

[0010] Optionally, based on M groups of feature data, a shift quantization technique is used to perform classification learning on the lightweight width learning system model to obtain an abnormal traffic detection model, including: iteratively quantizing the weights of the lightweight width learning system model using a shift quantization technique until a predetermined termination condition is reached; based on the quantization weights obtained when the predetermined termination condition is reached, and the M groups of feature data, the lightweight width learning system model is parameter optimized to obtain an abnormal traffic detection model.

[0011] Optionally, the weights of the lightweight width learning system model are iteratively quantized using a shift quantization technique until a predetermined termination condition is reached, including: based on a preset quantization bit width value range, the random weights in the lightweight width learning system model are quantized using a shift quantization technique to obtain a quantized model; based on a preset splitting rate, the output weights of the quantization model are divided into quantized weights and unquantized weights; and the unquantized weights in the quantization model are iteratively updated using a back propagation algorithm until a predetermined termination condition is reached.

[0012] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an abnormal traffic detection device is provided. The device includes: an acquisition module, which is used to acquire target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; an extraction module, which is used to extract target feature data matching N features from the target traffic data, wherein the N features are obtained by feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; a detection module, which is used to obtain an abnormal detection result of the target traffic data based on the target feature data by using an abnormal traffic detection model, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data matching the N features extracted from the M groups of traffic data.

[0013] Optionally, the device also includes: a first determination submodule, used to determine the importance scores corresponding to K features respectively based on M groups of traffic data using a feature selection algorithm, wherein each group of traffic data includes feature values ​​corresponding to K features respectively, and K is an integer greater than N; a first screening submodule, used to screen out N features from the K features based on the importance scores corresponding to the K features respectively.

[0014] Optionally, the detection module also includes: a first selection submodule, used to perform feature selection on M groups of traffic data to obtain M groups of initial data, wherein the feature selection is used to remove redundant features in the M groups of traffic data; a first training submodule, used to train the initial extreme gradient boosting model based on the M groups of initial data to obtain a target extreme gradient boosting model; and a first determination submodule, used to determine the importance scores corresponding to the K features respectively based on the target extreme gradient boosting model.

[0015] Optionally, the first determination submodule also includes: a second determination submodule, used to determine the number of times the K features are used in the decision trees included in the target extreme gradient boosting model; and a third determination submodule, used to determine the importance scores corresponding to the K features based on the number of times the K features are used in the decision trees included in the target extreme gradient boosting model.

[0016] Optionally, the device also includes: a first extraction submodule, used to extract feature data matching N features from M groups of traffic data to obtain M groups of feature data; a first learning submodule, used to classify and learn the lightweight width learning system model based on the M groups of feature data using shift quantization technology to obtain an abnormal traffic detection model.

[0017] Optionally, the first learning submodule includes: a first quantization submodule, used to iteratively quantize the weights of the lightweight width learning system model using a shift quantization technique until a predetermined termination condition is reached; a first optimization submodule, used to optimize the parameters of the lightweight width learning system model based on the quantization weights obtained when the predetermined termination condition is reached, and M groups of feature data, to obtain an abnormal traffic detection model.

[0018] Optionally, the first quantization submodule includes: a second quantization submodule, which is used to quantize the random weights in the lightweight width learning system model based on a preset quantization bit width value range and adopt a shift quantization technique to obtain a quantized model; a first segmentation submodule, which is used to divide the output weights of the quantization model into quantized weights and unquantized weights based on a preset segmentation rate; and a first iteration submodule, which is used to iteratively update the unquantized weights in the quantization model using a back-propagation algorithm until a predetermined termination condition is reached.

[0019] In an embodiment of the present application, by acquiring target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; extracting target feature data matching N features from the target traffic data, wherein the N features are obtained through feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; based on the target feature data, an abnormal traffic detection model is adopted to obtain an abnormal detection result of the target traffic data, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data matching N features extracted from M groups of traffic data, thereby achieving efficient, real-time and resource-lightweight abnormal traffic detection in edge computing scenarios, in order to balance the purpose of detection performance and computing cost, thereby achieving the technical effect of improving the efficiency of abnormal traffic detection in edge computing scenarios, reducing the consumption of computing and storage resources, and improving detection accuracy, thereby solving the technical problems of detection delay and low response efficiency caused by high computing complexity and large storage requirements limiting the real-time operation capability of deep learning models on resource-constrained edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 A hardware structure block diagram of a computer terminal for implementing an abnormal traffic detection method is shown;

[0022] Figure 2 is a flow chart of an abnormal traffic detection method provided according to an embodiment of the present application;

[0023] Figure 3 is a flow chart of an optional abnormal traffic detection method according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an abnormal flow detection device provided according to an embodiment of the present application;

[0025] Figure 5 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0029] The Lightweight Broad Learning System (LBLS) is a machine learning model designed specifically to run in environments with limited computing resources, such as edge computing devices. Compared with traditional deep learning models, it improves learning capabilities by increasing the width (i.e., the number of nodes) of the model rather than its depth, thereby achieving higher prediction accuracy with less computing resources and shorter training time.

[0030] Shift Quantization (SQ) is a method used to reduce the precision of weights and activation values ​​in neural network models, with the aim of reducing the computational complexity and memory usage of the model, thereby improving the operating efficiency on resource-constrained devices. The core idea is to use a lower-bit-width quantization value to approximate high-precision floating-point numbers, while compensating for the precision loss in the quantization process through shift operations.

[0031] eXtreme Gradient Boosting (XGBoost) is an efficient machine learning algorithm, especially suitable for solving supervised learning problems on large-scale data sets. It is based on the GradientBoosting Framework, which builds a series of weak prediction models (usually decision trees) and combines them to form a strong prediction model. The "extreme" of XGBoost is reflected in its optimization of computational efficiency and model performance, and its ability to handle large-scale data sets. At the same time, it reduces overfitting and improves the generalization ability of the model through regularization and other techniques.

[0032] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0033] Example 1

[0034] According to an embodiment of the present application, a method embodiment of abnormal traffic detection is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an abnormal traffic detection method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0036] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the abnormal flow detection method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the abnormal flow detection method described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0039] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0040] In the above operating environment, the present application provides a method for detecting abnormal traffic. Figure 2 is a flow chart of an abnormal flow detection method according to Example 1 of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0041] Step S200, obtaining target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device;

[0042] Optionally, edge devices refer to devices in the network that are close to data sources or users, such as routers, gateways, or IoT devices. In edge computing scenarios, data processing and analysis are close to the data generation point, which can reduce the delay in data transmission to the central server, improve response speed, and achieve real-time detection of abnormal traffic.

[0043] Optionally, the target traffic data is collected and processed on the edge device, that is, the data has been preliminarily cleaned, normalized or feature engineered before entering the model, which can ensure the data quality of the input model and improve the prediction accuracy of the model. The processed data is more in line with the input requirements of the model, which helps to improve the training efficiency and detection performance of the model. Among them, the target data traffic includes but is not limited to the size, frequency, source address, destination address, protocol type, etc. of the data packet, which is used to monitor network activities and identify potential abnormal behaviors or network attacks. Since the characteristic of edge computing is to process computing tasks close to the data source or user terminal, the traffic data here is obtained directly on the edge device for real-time analysis and detection of abnormal traffic, thereby achieving rapid response and reducing the burden on the central server.

[0044] It should be noted that processing data on edge devices with limited resources avoids the high computing and storage overhead that may be caused by centralized processing. This is especially critical for application scenarios such as abnormal traffic detection that require frequent monitoring and analysis of network conditions. The lightweight breadth learning system (LBLS) model can ensure detection accuracy and reduce computational complexity and memory usage. Therefore, processing data directly on edge devices can make full use of the lightweight characteristics of the model and achieve efficient operation.

[0045] Step S202, extracting target feature data matching N features from the target flow data, wherein the N features are obtained through feature selection based on M groups of flow data, and M and N are integers greater than or equal to 2;

[0046] Optionally, feature selection is a fundamental step in machine learning that identifies and uses a subset of features that contribute most to model predictions. By analyzing M sets of traffic data, the N most relevant features can be determined. These features can include, but are not limited to, packet size, transmission time, source / destination IP address, port number, protocol type, etc., which can describe the characteristics of traffic and distinguish normal traffic from abnormal traffic. Selecting N features from M sets of traffic data, where N is less than the total number of original features, can significantly reduce the input dimension of the abnormal traffic detection model, thereby reducing the computational complexity of model training and prediction.

[0047] In an optional embodiment, before obtaining anomaly detection results of target traffic data by using an abnormal traffic detection model based on target feature data, the method further includes: based on M groups of traffic data, using a feature selection algorithm to determine importance scores corresponding to K features, wherein each group of traffic data includes feature values ​​corresponding to K features, and K is an integer greater than N; based on the importance scores corresponding to the K features, N features are screened out from the K features.

[0048] Optionally, by performing feature selection on M groups of traffic data, determining the importance scores of K features, and selecting the N most important features, this step can ensure that the model learns only based on the most relevant features, avoiding the negative impact of irrelevant or redundant features on model performance. It should be noted that in abnormal traffic detection, reducing the number of model input features can reduce the computational burden, thereby accelerating the model training and prediction process and improving the overall detection efficiency, especially on resource-constrained network edge devices.

[0049] In an optional embodiment, based on M groups of traffic data, a feature selection algorithm is used to determine the importance scores corresponding to K features, including: performing feature selection on the M groups of traffic data to obtain M groups of initial data, wherein the feature selection is used to remove redundant features in the M groups of traffic data; based on the M groups of initial data, an initial extreme gradient boosting model is trained to obtain a target extreme gradient boosting model; based on the target extreme gradient boosting model, the importance scores corresponding to the K features are determined.

[0050] Optionally, by performing feature selection on M groups of traffic data, redundant features and irrelevant features can be removed to reduce the complexity of model training and the time of data preprocessing, thereby improving the efficiency of the overall detection process. XGBoost can be used as a tool for feature importance scoring. The target extreme gradient boosting model obtained through training can reveal which features play a key role in abnormal traffic detection, which is helpful for the subsequent optimization and training of the LBLS model. Based on the XGBoost model, the importance scores corresponding to the K features can be calculated to ensure that the model focuses on key features for learning and prediction.

[0051] In an optional embodiment, based on the target extreme gradient boosting model, the importance scores corresponding to the K features are determined, including: determining the number of times the K features are used in the decision trees included in the target extreme gradient boosting model; based on the number of times the K features are used in the decision trees included in the target extreme gradient boosting model, determining the importance scores corresponding to the K features.

[0052] Optionally, by counting the number of times each feature is used in the decision tree of the XGBoost model, the contribution of each feature to the classification prediction can be quantified. Among them, the importance score of the feature can indicate which features are the core of abnormal traffic detection, thereby guiding the training and optimization of the subsequent lightweight breadth learning system (LBLS) model.

[0053] In an optional embodiment, before obtaining an abnormal detection result of the target flow data by using an abnormal flow detection model based on the target feature data, the method also includes: extracting feature data matching N features from M groups of flow data to obtain M groups of feature data; based on the M groups of feature data, using shift quantization technology to perform classification learning on the lightweight width learning system model to obtain an abnormal flow detection model.

[0054] Optionally, feature data matching N features are extracted from M groups of traffic data, aiming to provide high-quality input data for model training. This embodiment can ensure that the model can learn on the most relevant features, thereby improving the detection efficiency and accuracy of the model. By adopting the shift quantization technology to classify and learn the LBLS model, the weights and biases of the model are quantized to a lower bit width, which can significantly reduce the storage requirements and computational complexity of the model, making the model lighter and easier to deploy and run on edge devices. By using quantized weights and biases, the LBLS model can reduce memory usage and computing resources while maintaining detection performance. Through parameter optimization, it can be ensured that the model can still achieve efficient abnormal traffic detection in a resource-constrained environment. Through feature selection, features that are irrelevant or redundant to abnormal detection can also be removed, thereby improving the efficiency and accuracy of the model. Irrelevant or redundant features not only increase the computational burden, but also introduce noise, affecting the prediction performance of the model. Selecting N features that are highly relevant to abnormal traffic detection can help the model more accurately identify abnormal patterns, improve detection rates, and reduce false alarm rates.

[0055] As an optional embodiment, feature selection is performed by XGBoost. Assume that a feature set includes M sets of feature data, and any set of feature data v in the M sets of feature data i =v 1 ,v 2 ,...,v M , where i = 1, 2, …, M. Then, the feature importance score IS is calculated based on the number of times each feature is used to partition the training data across all decision trees in the extreme gradient boosting model i . The feature importance score is expressed as:

[0056] IS i ={x|x=w i v i}

[0057] Among them, w i is the weight of each feature, v i Represents the features included in the feature set. The features selected by XGBoost are denoted as V = [V 1 ,V 2 ,...,V n ], where Vi =[F 1 ,F 2 ,...,F N ], N is the number of selected features. Then it is input into LBLS for classification.

[0058] Combined with the XGBoost algorithm to determine the importance score of the feature, the specific implementation steps are as follows:

[0059] Step S2041, data preparation, specifically includes:

[0060] Dataset construction: Collect and organize M groups of traffic data, each group of data contains feature values ​​corresponding to K features. These features can be but are not limited to packet size, transmission time, source / destination IP address, port, protocol type, etc.

[0061] Data preprocessing: Preprocess M groups of traffic data, including cleaning, missing value processing, outlier detection and processing, and converting non-numerical features into numerical types.

[0062] Step S2042, feature engineering, specifically includes:

[0063] Feature selection: After data preprocessing, preliminary feature selection is performed on the preprocessed traffic data to remove obviously irrelevant or redundant features and obtain M groups of initial data to reduce the amount of calculation and avoid overfitting.

[0064] Dataset division: The M groups of initial data obtained after preprocessing are divided into training set and test set, usually with a ratio of 70% training and 30% testing, to avoid data leakage and ensure the evaluation of the model's generalization ability.

[0065] Step S2043, XGBoost model training, specifically includes:

[0066] Model initialization: Use the XGBoost library to initialize an XGBoost classifier and set necessary parameters such as tree depth (max_depth), number of trees (n_estimators), learning rate (learning_rate), etc.

[0067] Model training: Use the training set data to train the XGBoost model. The model will learn how to predict whether the traffic is abnormal based on the feature values.

[0068] Step S2044, determining the feature importance score, specifically includes:

[0069] Get feature importance: After training, the XGBoost model can output the importance score of each feature. The score is based on the number of times the feature is used in the decision tree.

[0070] Interpretation of the scoring: The feature importance scores returned by XGBoost are a dictionary, where the keys are the feature names and the values are the importance scores. A higher score indicates that the feature plays a greater role in the model prediction.

[0071] Step S2045, Feature selection and optimization, specifically including:

[0072] Scoring and sorting: Based on the obtained importance scores, sort the K features to determine which features contribute the most to the abnormal traffic detection.

[0073] Feature reduction: According to the sorted results, select the top N (N < K) features for subsequent LBLS model training. This step can be achieved by setting a threshold to retain features with importance scores higher than the threshold, or directly retaining the top N features with the highest scores.

[0074] Evaluation and optimization: After feature selection in the XGBoost model, the performance of the model can be evaluated using the test set. If the model performance is found to be poor, the value of N can be adjusted, or the parameters of the XGBoost model can be adjusted, retrained and evaluated until the optimal feature set is found.

[0075] Optionally, feature selection by XGBoost can reduce the computational complexity, reduce the memory occupancy, adjust the accuracy of the model and reduce overfitting.

[0076] Step S204, Based on the target feature data, adopt an abnormal traffic detection model to obtain the abnormal detection result of the target traffic data, where the abnormal traffic detection model is obtained by classifying and learning the lightweight broad learning system model based on M groups of feature data, and the M groups of feature data are the feature data extracted from M groups of traffic data and matching N features.

[0077] Optionally, use the abnormal traffic detection model to analyze the target feature data, convert the traffic data after feature selection into a form that the model can understand, and judge whether the traffic is abnormal through the classification function of the model. Among them, the abnormal traffic detection model is obtained by learning and training M groups of feature data, which contain various attributes and behavior patterns of the traffic. The LBLS model establishes rules for distinguishing normal and abnormal traffic through learning these data, so as to be able to make accurate classifications on new data.

[0078] In this embodiment, by adopting the LBLS model for classification learning, the computational and storage requirements of the model can be reduced, enabling it to be deployed and run on network edge devices with limited resources. Ensure that even on devices with weak computing capabilities, the abnormal traffic detection model can respond quickly and achieve the abnormal detection of real-time traffic data.

[0079] In an optional embodiment, based on M groups of feature data, a shift quantization technique is used to perform classification learning on a lightweight width learning system model to obtain an abnormal traffic detection model, including: iteratively quantizing the weights of the lightweight width learning system model using a shift quantization technique until a predetermined termination condition is reached; based on the quantization weights obtained when the predetermined termination condition is reached, and the M groups of feature data, the lightweight width learning system model is parameter optimized to obtain an abnormal traffic detection model.

[0080] Optionally, the application of shift quantization technology can make the weights in the LBLS model iteratively quantized, reduce the accuracy requirements of the weight representation, and thus reduce the storage requirements and computational complexity of the abnormal traffic detection model. It helps to achieve fast and low-power abnormal traffic detection on edge devices. It should be noted that edge devices often have limited memory and computing power. By iteratively quantizing the weights to a predetermined termination condition, the abnormal traffic detection model can more efficiently utilize the resources of the edge device without significantly sacrificing detection performance. In addition, during the weight quantization process, the abnormal traffic detection model will perform parameter optimization based on M groups of feature data to ensure that even when the weights are quantized, the abnormal traffic detection model can still maintain a high detection accuracy and efficiency.

[0081] In an optional embodiment, the weights of the lightweight width learning system model are iteratively quantized using a shift quantization technique until a predetermined termination condition is reached, including: based on a preset quantization bit width value range, the random weights in the lightweight width learning system model are quantized using a shift quantization technique to obtain a quantized model; based on a preset splitting rate, the output weights of the quantization model are divided into quantized weights and unquantized weights; and the unquantized weights in the quantization model are iteratively updated using a back propagation algorithm until a predetermined termination condition is reached.

[0082] Optionally, the application of shift quantization technology can quantize the weights in the LBLS model, reduce the bit representation width of the weights, thereby reducing the storage requirements and computational complexity of the model, making the model more suitable for deployment at the edge of the network. Among them, the output weights of the quantized model are all quantized. By quantizing the weights, the acquired abnormal traffic detection model can run on the edge device with fewer computing resources, which is especially important for low-power and small devices, and can improve resource utilization efficiency. The preset termination conditions and split ratios can ensure that while reducing the burden on the model, the detection performance of the model is still maintained, avoiding a significant drop in performance due to resource constraints.

[0083] As an optional embodiment, in the process of constructing an abnormal traffic detection model using a lightweight width learning system based on shift quantization technology, first, in order to find the optimal bit width for each weight, the bit width of the weight is automatically selected from the bit width set during training without manual intervention. Secondly, in order to reduce or avoid the precision loss caused by random weight quantization, the fast training capability of the Broad Learning System (BLS) is utilized to obtain an optimal model for random weight quantization through grid search. Afterwards, in order to minimize the precision loss caused by the quantization of the output weights, the output weights are grouped on this model so that a part of the output weights with large contributions are quantized, while the other part is retrained through back propagation until all output weights are quantized. Finally, a lightweight BLS is obtained.

[0084] Optionally, the specific steps for building an abnormal traffic detection model are as follows:

[0085] Step S2041, feature data preprocessing: based on the XGBoost algorithm, K features are selected from the M groups of traffic data, and these features are sorted according to their importance to determine the key feature set for abnormal traffic detection.

[0086] Step S2042, quantization parameter setting: setting the value range b of the quantization bit width and the division rate p. These parameters are used to guide the weight quantization process in the LBLS model to adapt to the resource limitations of the edge computing device.

[0087] Step S2043, random weight quantization: Use shift quantization technology to quantize the random weights in the LBLS model to ensure that the model can run efficiently in an environment with limited resources.

[0088] Step S2044, output weight grouping quantization: the output weights of the LBLS model are divided into two parts, one part is the quantifiable weights, and the other part is the unquantized weights. The proportion of the quantifiable weights is determined by the division rate p.

[0089] Step S2045, iterative weight update: the unquantized weights are iteratively updated using the back propagation algorithm until all weights are quantized to optimize model parameters, reduce resource consumption, and maintain detection accuracy.

[0090] Step S2046, LBLS model training: use the quantized weights and M groups of training data after feature selection to initialize and train the LBLS model, optimize the model parameters, and obtain an abnormal traffic detection model to improve the accuracy of abnormal traffic detection.

[0091] It should be noted that the abnormal traffic detection method provided in the embodiment of the present application obtains target traffic data, wherein the target traffic data is network traffic data collected and processed on the network edge device; extracts target feature data matching N features from the target traffic data, wherein the N features are obtained by feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; based on the target feature data, an abnormal traffic detection model is used to obtain an abnormal detection result of the target traffic data, wherein the abnormal traffic detection model is obtained by classifying and learning the lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data matching N features extracted from M groups of traffic data, which solves the problem that the high computational complexity and large storage requirements in the related technology limit the real-time operation capability of the deep learning model on the resource-constrained edge device, resulting in abnormal traffic detection delay and low response efficiency. Thus, the abnormal traffic detection efficiency in the edge computing scenario is improved, the consumption of computing and storage resources is reduced, and the detection accuracy is improved.

[0092] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation method of abnormal traffic detection. The abnormal traffic detection in this embodiment is mainly divided into two parts: XGBoost and LBLS based on shift quantization technology. Among them, XGBoost is mainly responsible for training the features of traffic data and simplifying features through feature importance scoring, while the LBLS part is responsible for learning, classifying and predicting the optimal features of the screened traffic data. Figure 3 is a flow chart of an optional abnormal traffic detection method according to the present application, such as Figure 3 As shown, the method includes:

[0093] Step S1, based on the initial data set, feature selection is performed through XGBoost to select N features from K features, wherein the initial data set includes M groups of traffic data, and each group of traffic data includes characteristic values ​​corresponding to K features respectively; feature selection can reduce computational complexity, reduce memory usage, adjust the accuracy of the model and reduce overfitting.

[0094] Step S2, constructing a lightweight width learning system model based on shift quantization technology and constructing an abnormal traffic monitoring model;

[0095] Specifically, first, in order to find the optimal bit width for each weight, the bit width of the weight is automatically selected from the bit width set during training without manual intervention. Secondly, in order to reduce or avoid the precision loss caused by random weight quantization, the fast training capability of BLS is used to obtain an optimal model for random weight quantization through grid search. Afterwards, in order to minimize the precision loss caused by output weight quantization, the output weights are grouped on this model so that a part of the output weights with large contributions are quantized, while the other part is retrained through back propagation until all output weights are quantized. Finally, a lightweight BLS is obtained.

[0096] The specific steps to build an abnormal traffic detection model are as follows:

[0097] Step S21, setting the feature mapping function, the number of feature nodes, the number of enhancement nodes, the activation function, and the selection range of the quantization parameters;

[0098] Step S22, input the training data set X, Y after feature selection;

[0099] Step S23, quantize the random weight according to the following formula:

[0100] The weights can be represented as a set P:

[0101] P = {±2 I},0≤I≤b

[0102] Where I represents the bit width that can be selected during training, and b is the maximum bit width that can be selected during training. I is determined by:

[0103]

[0104] Among them, err represents the error value, W i is the floating point weight of any feature among the N features, W i_quant is the quantized weight corresponding to any feature.

[0105] Step S24, using grid search to obtain the best model for random weight quantization:

[0106] Z z =φ z (XW z +β z ) z ,β z ∈P z=1,2,...,r

[0107] Among them, Z z is the feature node matrix, φ(*) is the mapping function, X is the input sample, W z is the quantized weight, βz is the quantized bias, and r is the number of feature nodes. These feature nodes form enhanced nodes through the following formula:

[0108] H h =ξ h (ZW h +β h ) h ,β h ∈P h=1,2,...,t

[0109] Among them, H h is the enhancement node matrix, ξ(*) is the activation function, W h is the quantized enhancement layer weight, β h is the quantized enhancement layer bias, and t is the number of enhancement node groups.

[0110] After that, the BLS method is used to solve the output weights:

[0111] W * =((A * ) T A * +γI) -1 (A * ) T Y

[0112] Among them, A * =[Z Z |H h ] is the expansion matrix composed of feature nodes and enhanced nodes, γ is a regularization parameter, W * is the output weight.

[0113] Step S25, grouping the output weights:

[0114] W q =ηW *

[0115] W f =(1-η)W *

[0116] Among them, W * is the sorted floating-point output weight, W q is the quantized weight, W f is the remaining floating point weight. η is the split ratio, which can be 0.1, 0.2, 0.4, 0.6, 0.8.

[0117] Step S26, iteratively update the floating point weights using back propagation until all weights are quantized:

[0118]

[0119] Among them, W l =[W q |W f ] is a matrix composed of quantized weights and unquantized weights, W q is the quantized weight, W f is the remaining floating point weight, E(W l ) is the loss of the model, R(W l ) is the regularization term, λ 1 is the coefficient of the regularization term.

[0120]

[0121] in, is the weight after iterative update, is a floating point weight, ε is the learning rate for gradient descent, and k is the number of iterations.

[0122] Step S27, obtain the final output weight W l , based on the final output weight W l Build an abnormal traffic detection model;

[0123] Finally, the trained abnormal traffic detection model is used to predict abnormal traffic.

[0124] It should be noted that the abnormal traffic detection method proposed in this embodiment can reduce the memory and computing requirements during model training and online application while ensuring accuracy, and has the advantages of low energy consumption and low computational complexity, so that it can run in real time on devices with limited resources, achieving a balance between the lightweight and security of the abnormal traffic detection model.

[0125] It should still be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0126] Example 2

[0127] The embodiment of the present application also provides an abnormal flow detection device. It should be noted that the abnormal flow detection device of the embodiment of the present application can be used to execute the abnormal flow detection method provided in the embodiment of the present application. The abnormal flow detection device provided in the embodiment of the present application is introduced below.

[0128] According to an embodiment of the present application, a device for implementing the above abnormal flow detection method is also provided, such as Figure 4As shown, the device includes: an acquisition module 300, used to acquire target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; an extraction module 302, connected to the acquisition module 300, used to extract target feature data matching N features from the target traffic data, wherein the N features are obtained through feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; a detection module 304, connected to the extraction module 302, used to obtain an abnormal detection result of the target traffic data based on the target feature data by using an abnormal traffic detection model, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data extracted from the M groups of traffic data and matching the N features.

[0129] The abnormal traffic detection device provided in the embodiment of the present application is configured with an acquisition module 300 for acquiring target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; an extraction module 302 for extracting target feature data matching N features from the target traffic data, wherein the N features are obtained by feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; a detection module 304 for obtaining an abnormal detection result of the target traffic data by adopting an abnormal traffic detection model based on the target feature data, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data matching N features extracted from M groups of traffic data, which solves the problem that the high computational complexity and large storage requirements in the related art limit the real-time operation capability of the deep learning model on the resource-constrained edge device, resulting in abnormal traffic detection delay and low response efficiency. Thus, the abnormal traffic detection efficiency in the edge computing scenario is improved, the consumption of computing and storage resources is reduced, and the detection accuracy is improved.

[0130] Optionally, in the abnormal flow detection device provided in the embodiment of the present application, the device also includes: a first determination submodule, used to determine the importance scores corresponding to K features respectively based on M groups of flow data, using a feature selection algorithm, wherein each group of flow data includes feature values ​​corresponding to K features respectively, and K is an integer greater than N; a first screening submodule, used to screen out N features from the K features based on the importance scores corresponding to the K features respectively.

[0131] Optionally, in the abnormal flow detection device provided in the embodiment of the present application, the detection module also includes: a first selection submodule, used to perform feature selection on M groups of flow data to obtain M groups of initial data, wherein the feature selection is used to remove redundant features in the M groups of flow data; a first training submodule, used to train the initial extreme gradient boosting model based on the M groups of initial data to obtain a target extreme gradient boosting model; a first determination submodule, used to determine the importance scores corresponding to the K features respectively based on the target extreme gradient boosting model.

[0132] Optionally, in the abnormal traffic detection device provided in the embodiment of the present application, the first determination submodule also includes: a second determination submodule, used to determine the number of times the K features are used in the decision trees included in the target extreme gradient boosting model; and a third determination submodule, used to determine the importance scores corresponding to the K features based on the number of times the K features are used in the decision trees included in the target extreme gradient boosting model.

[0133] Optionally, in the abnormal flow detection device provided in the embodiment of the present application, the device also includes: a first extraction submodule, used to extract feature data matching N features from M groups of flow data to obtain M groups of feature data; a first learning submodule, used to classify and learn the lightweight width learning system model based on the M groups of feature data using shift quantization technology to obtain an abnormal flow detection model.

[0134] Optionally, in the abnormal flow detection device provided in the embodiment of the present application, the first learning sub-module includes: a first quantization sub-module, which is used to iteratively quantize the weights of the lightweight width learning system model using a shift quantization technique until a predetermined termination condition is reached; a first optimization sub-module, which is used to optimize the parameters of the lightweight width learning system model based on the quantization weights obtained when the predetermined termination condition is reached, and M groups of feature data, to obtain an abnormal flow detection model.

[0135] Optionally, in the abnormal traffic detection device provided in the embodiment of the present application, the first quantization submodule includes: a second quantization submodule, which is used to quantize the random weights in the lightweight width learning system model based on a preset quantization bit width value range and adopt a shift quantization technique to obtain a quantized model; a first segmentation submodule, which is used to divide the output weights of the quantization model into quantized weights and unquantized weights based on a preset segmentation rate; and a first iteration submodule, which is used to iteratively update the unquantized weights in the quantization model using a back-propagation algorithm until a predetermined termination condition is reached.

[0136] It should be noted that the acquisition module 300, extraction module 302, and detection module 304 correspond to steps S200 to S204 in Example 1, and the examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above-mentioned modules can also be run in the computer terminal 10 provided in Example 1 as part of the device.

[0137] Example 3

[0138] An embodiment of the present application may provide an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (only one is shown) processor 1002, memory 1004, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0139] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0140] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target traffic data, wherein the target traffic data is network traffic data collected and processed on the network edge device; extract target feature data matching N features from the target traffic data, wherein the N features are based on M groups of traffic data and obtained through feature selection, and M and N are integers greater than or equal to 2; based on the target feature data, use an abnormal traffic detection model to obtain an abnormal detection result of the target traffic data, wherein the abnormal traffic detection model is based on the M groups of feature data, and a lightweight width learning system model is classified and learned, and the M groups of feature data are feature data extracted from the M groups of traffic data and matching the N features.

[0141] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: based on M groups of traffic data, use a feature selection algorithm to determine the importance scores corresponding to K features, wherein each group of traffic data includes feature values ​​corresponding to K features, and K is an integer greater than N; based on the importance scores corresponding to the K features, select N features from the K features.

[0142] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: perform feature selection on M groups of traffic data to obtain M groups of initial data, wherein the feature selection is used to remove redundant features in the M groups of traffic data; based on the M groups of initial data, train the initial extreme gradient boosting model to obtain the target extreme gradient boosting model; based on the target extreme gradient boosting model, determine the importance scores corresponding to the K features respectively.

[0143] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: determine the number of times the K features are used in the decision trees included in the target extreme gradient boosting model; based on the number of times the K features are used in the decision trees included in the target extreme gradient boosting model, determine the importance scores corresponding to the K features.

[0144] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: extract feature data matching N features from M groups of traffic data to obtain M groups of feature data; based on the M groups of feature data, use shift quantization technology to classify and learn the lightweight width learning system model to obtain an abnormal traffic detection model.

[0145] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: iteratively quantize the weights of the lightweight width learning system model using shift quantization technology until a predetermined termination condition is reached; based on the quantization weights obtained when the predetermined termination condition is reached, and M groups of feature data, optimize the parameters of the lightweight width learning system model to obtain an abnormal traffic detection model.

[0146] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: based on the preset quantization bit width value range, the random weights in the lightweight width learning system model are quantized using the shift quantization technology to obtain a quantized model; based on the preset segmentation rate, the output weights of the quantization model are divided into quantized weights and unquantized weights; the unquantized weights in the quantization model are iteratively updated using the back propagation algorithm until the predetermined termination condition is reached.

[0147] By adopting the embodiment of the present application, a solution for abnormal traffic detection is provided. By acquiring target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; extracting target feature data matching N features from the target traffic data, wherein the N features are obtained by feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; based on the target feature data, an abnormal traffic detection model is adopted to obtain an abnormal detection result of the target traffic data, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data matching N features extracted from M groups of traffic data, thereby achieving high efficiency, real-time and resource-lightweight abnormal traffic detection in edge computing scenarios, balancing the purpose of detection performance and computing cost, and further solving the technical problems of detection delay and low response efficiency caused by high computational complexity and large storage requirements limiting the real-time operation capability of deep learning models on resource-constrained edge devices.

[0148] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations are shown.

[0149] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0150] Example 4

[0151] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the abnormal traffic detection method provided in the first embodiment.

[0152] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0153] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the abnormal traffic detection method.

[0154] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0155] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0160] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting abnormal traffic, characterized in that: include: Acquire target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; Extracting target feature data matching N features from the target flow data, wherein the N features are obtained through feature selection based on M groups of flow data, and M and N are integers greater than or equal to 2; Based on the target feature data, an abnormal traffic detection model is used to obtain an abnormal detection result of the target traffic data, wherein the abnormal traffic detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data extracted from the M groups of traffic data and matching the N features.

2. The method according to claim 1, characterized in that Before obtaining an abnormality detection result of the target flow data by using an abnormal flow detection model based on the target feature data, the method further includes: Based on the M groups of traffic data, a feature selection algorithm is used to determine importance scores corresponding to K features, wherein each group of traffic data includes feature values ​​corresponding to the K features, and K is an integer greater than N; The N features are selected from the K features based on the importance scores corresponding to the K features respectively.

3. The method according to claim 2, characterized in that The step of using a feature selection algorithm based on the M groups of traffic data to determine the importance scores corresponding to the K features respectively includes: Performing feature selection on the M groups of flow data to obtain M groups of initial data, wherein the feature selection is used to remove redundant features in the M groups of flow data; Based on the M groups of initial data, an initial extreme gradient boosting model is trained to obtain a target extreme gradient boosting model; Based on the target extreme gradient boosting model, importance scores corresponding to the K features are determined.

4. The method according to claim 3, characterized in that Determining the importance scores corresponding to the K features respectively based on the target extreme gradient boosting model includes: Determine the number of times the K features are used in the decision trees included in the target extreme gradient boosting model; Based on the number of times the K features are respectively used in the decision trees included in the target extreme gradient boosting model, importance scores corresponding to the K features are determined.

5. The method according to claim 1, characterized in that Before obtaining an abnormality detection result of the target flow data by using an abnormal flow detection model based on the target feature data, the method further includes: Extracting feature data matching the N features from the M groups of traffic data to obtain the M groups of feature data; Based on the M groups of feature data, the shift quantization technology is used to classify and learn the lightweight width learning system model to obtain the abnormal traffic detection model.

6. The method according to claim 5, characterized in that Based on the M groups of feature data, the shift quantization technology is used to classify and learn the lightweight width learning system model to obtain the abnormal traffic detection model, including: Iteratively quantizing the weights of the lightweight width learning system model using a shift quantization technique until a predetermined termination condition is reached; Based on the quantized weights obtained when the predetermined termination condition is reached and the M groups of feature data, the parameters of the lightweight width learning system model are optimized to obtain the abnormal traffic detection model.

7. The method according to claim 6, characterized in that The adopting the shift quantization technology to iteratively quantize the weights of the lightweight width learning system model until a predetermined termination condition is reached includes: Based on a preset quantization bit width value range, the shift quantization technology is used to quantize the random weights in the lightweight width learning system model to obtain a quantized model; Based on a preset split ratio, the output weights of the quantization model are divided into quantized weights and unquantized weights; The unquantized weights in the quantization model are iteratively updated using a back-propagation algorithm until the predetermined termination condition is reached.

8. An abnormal flow detection device, characterized in that: include: An acquisition module, used to acquire target traffic data, wherein the target traffic data is network traffic data collected and processed on a network edge device; An extraction module is used to extract target feature data matching N features from the target traffic data, wherein the N features are obtained through feature selection based on M groups of traffic data, and M and N are integers greater than or equal to 2; The detection module is used to obtain anomaly detection results of target flow data based on target feature data and using an abnormal flow detection model, wherein the abnormal flow detection model is obtained by classifying and learning a lightweight width learning system model based on M groups of feature data, and the M groups of feature data are feature data extracted from the M groups of flow data and matched with N features.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the abnormal traffic detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the abnormal traffic detection method described in any one of claims 1 to 7 when running.

Images