Network traffic detection model distillation method, system, and server

Abstract

The application provides a network traffic detection model distillation method, system and server, and relates to the technical field of network security. In the method, distillation is performed in the process of guiding the second model to learn and train by the first model. Related calculation characteristic parameters are optimized for the target hardware. The second model can adapt to low-precision calculation by simulating abnormal traffic. Therefore, the distillation loss and quantization error are simultaneously optimized in the training process, the performance overhead is reduced, the model precision is ensured, and the method is more suitable for deployment in edge computing devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a network traffic detection model distillation method, system and server. BACKGROUND

[0002] In the network traffic detection process, relevant neural network models have been used to achieve it, but the existing network traffic detection models generally have high computational intensity and large memory occupation when deployed on hardware; in addition, the existing model distillation methods do not fully consider the characteristics of the target hardware, resulting in a loss of inference efficiency after deployment. In addition, the existing abnormal traffic samples are scarce and single in pattern, resulting in insufficient model generalization ability; in the actual scene, some fixed parameter models are difficult to maintain optimal performance in a dynamic network environment.

[0003] The above network traffic detection model has technical defects, resulting in poor model detection effect, high computational cost, and difficulty in deployment on edge computing devices. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a network traffic detection model distillation method, system and server, which distills in the process of guiding the second model (student model) to learn and train by the first model (teacher model), optimizes the relevant calculation characteristic parameters according to the target hardware, and can adapt to low-precision calculation by simulating abnormal traffic, so as to simultaneously optimize the distillation loss and quantization error in the training process, reduce the performance overhead while ensuring the model accuracy, and be more suitable for deployment on edge computing devices.

[0005] In a first aspect, the present application provides a network traffic detection model distillation method, which comprises:

[0006] A first model is constructed based on the attribute parameters corresponding to the network traffic detection scene, and a second model is constructed based on the structure parameters of the first model;

[0007] Quantization nodes and dequantization nodes between the first model and the second model are determined according to the structure parameters of the first model and the second model, and quantization parameters of the first model and the second model are determined based on the quantization nodes and the dequantization nodes;

[0008] The calculation characteristic parameters of the target hardware corresponding to the network traffic detection scene are obtained, and the initial quantization model corresponding to the first model and the second model is determined using the calculation characteristic parameters;

[0009] The distillation loss function corresponding to the initial quantization model is determined by the quantization parameters, the backpropagation strategy of the second model is determined based on the propagation parameters corresponding to the initial quantization model, and the training strategy corresponding to the initial quantization model is determined using the distillation loss function and the backpropagation strategy.

[0010] After the initial quantization model is trained by using the training strategy and according to the preset training round, the quantization model corresponding to the network traffic detection scene is obtained.

[0011] Optionally, the step of constructing the first model based on the attribute parameter corresponding to the network traffic detection scene and constructing the second model based on the structure parameter of the first model comprises:

[0012] The traffic parameter, the scene parameter, the network parameter and the detection parameter corresponding to the network traffic detection scene are acquired, and the attribute parameter corresponding to the network traffic detection scene is determined based on the traffic parameter, the scene parameter, the network parameter and the detection parameter.

[0013] The first model corresponding to the network traffic detection scene is constructed according to the single-precision parameter based on the attribute parameter.

[0014] The layer structure corresponding to the second model is determined according to the structure parameter of the first model, and the second model is constructed based on the layer structure.

[0015] Optionally, the quantization node and the dequantization node between the first model and the second model are determined according to the structure parameters of the first model and the second model, comprising:

[0016] The structure parameters of the first model and the second model are acquired, and the pseudo-quantization module corresponding to the first model and the second model is initialized according to the structure parameters.

[0017] The quantization position parameter corresponding to the pseudo-quantization module is determined, and the dequantization position parameter corresponding to the pseudo-quantization module is determined according to the convolution parameter of the first model and the second model.

[0018] The quantization node is determined based on the quantization position parameter, and the dequantization node is determined based on the dequantization position parameter.

[0019] Optionally, the quantization parameter of the first model and the second model is determined based on the quantization node and the dequantization node, comprising:

[0020] The quantization effect corresponding to the first model and the second model during training is determined based on the quantization node and the dequantization node.

[0021] The symmetry parameter and the reading parameter corresponding to the first model and the second model are determined according to the quantization effect.

[0022] The quantization parameter of the first model and the second model is determined by using the symmetry parameter and the reading parameter.

[0023] Optionally, the calculation characteristic parameter of the target hardware corresponding to the network traffic detection scene is acquired, and the initial quantization model corresponding to the first model and the second model is determined by using the calculation characteristic parameter, comprising:

[0024] acquire a target hardware corresponding to the network traffic detection scene, determine a calculation characteristic parameter according to a performance parameter of the target hardware;

[0025] set a first setting parameter corresponding to the first model and a second setting parameter corresponding to the second model based on the calculation characteristic parameter; under the first setting parameter, the first model is used to guide the second model; under the second setting parameter, the second model adopts an EfficientNet-Lite model structure;

[0026] construct an initial quantization model by using the first setting parameter and the second setting parameter.

[0027] Optionally, a distillation loss function corresponding to the initial quantization model is determined by using the quantization parameter, including:

[0028] determine a first last-layer output value of the first model and a second last-layer output value of the second model by using the quantization parameter, and construct an initial distillation loss function corresponding to the initial quantization model according to the first last-layer output value and the second last-layer output value;

[0029] determine a first feature output value of the first model and a second feature output value of the second model by using the quantization parameter, and construct a feature loss function corresponding to the initial quantization model according to the first feature output value and the second feature output value;

[0030] construct a distillation loss function corresponding to the initial quantization model based on the initial distillation loss function and the feature loss function.

[0031] Optionally, a back propagation strategy of the second model is determined based on a propagation parameter corresponding to the initial quantization model, and a training strategy corresponding to the initial quantization model is determined by using the distillation loss function and the back propagation strategy, including:

[0032] determine an inference strategy corresponding to the first model based on the propagation parameter corresponding to the initial quantization model, and determine a forward propagation strategy corresponding to the second model according to the inference strategy;

[0033] determine a loss function calculation strategy corresponding to the initial quantization model according to the forward propagation strategy and by using the distillation loss function;

[0034] determine a back propagation strategy corresponding to the second model based on the propagation parameter and by using the forward propagation strategy;

[0035] construct an abnormal traffic corresponding to the network traffic detection scene according to an adversarial parameter corresponding to the initial quantization model, and determine an adversarial strategy corresponding to the initial quantization model after performing feature distillation on the abnormal traffic;

[0036] determine a training strategy corresponding to the initial quantization model based on the forward propagation strategy, the loss function calculation strategy, the back propagation strategy, and the adversarial strategy.

[0037] Optionally, after the initial quantization model is trained by using the training strategy and according to the preset training round, the step of obtaining the quantization model corresponding to the network traffic detection scene includes:

[0038] The learning rate parameter, the temperature parameter, the scaling factor parameter and the zero point parameter corresponding to the initial quantization model are determined by using the training strategy.

[0039] The initial quantization model is trained based on the learning rate parameter, the temperature parameter, the scaling factor parameter and the zero point parameter, and the training times corresponding to the initial quantization model are obtained in real time.

[0040] If the training times are lower than the preset training round, the initial quantization model is controlled to continue training; if the training times are equal to the preset training round, the initial quantization model is controlled to stop training, and the current initial quantization model is determined as the quantization model of the network traffic detection scene.

[0041] In a second aspect, the present application provides a network traffic detection model distillation system, which comprises:

[0042] A model construction module is configured to construct a first model based on attribute parameters corresponding to a network traffic detection scene, and construct a second model based on structure parameters of the first model.

[0043] A quantization parameter determination module is configured to determine quantization nodes and dequantization nodes between the first model and the second model according to the structure parameters of the first model and the second model, and determine quantization parameters of the first model and the second model based on the quantization nodes and the dequantization nodes.

[0044] A quantization model initialization module is configured to obtain calculation characteristic parameters of a target hardware corresponding to the network traffic detection scene, and determine initial quantization models corresponding to the first model and the second model by using the calculation characteristic parameters.

[0045] A quantization model training distillation module is configured to determine an initial loss function corresponding to the initial quantization model by using the quantization parameters, determine a back propagation strategy of the second model based on propagation parameters corresponding to the initial quantization model, and determine a training strategy corresponding to the initial quantization model by using the initial loss function and the back propagation strategy.

[0046] A quantization model generation module is configured to obtain a quantization model corresponding to the network traffic detection scene after the initial quantization model is trained by using the training strategy and according to a preset training round.

[0047] In a third aspect, the present application further provides a server comprising a processor and a memory, wherein the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the steps of the network traffic detection model distillation method provided in the first aspect.

[0048] In a fourth aspect, an embodiment of the present application further provides a storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the steps of the network traffic detection model distillation method according to the first aspect.

[0049] The network traffic detection model distillation method, system and server provided by the embodiment of the present application, in the process of obtaining the network traffic detection model, first constructs a first model based on the attribute parameters corresponding to the network traffic detection scene, and constructs a second model based on the structure parameters of the first model; then determines the quantization nodes and the dequantization nodes between the first model and the second model according to the structure parameters of the first model and the second model, and determines the quantization parameters of the first model and the second model based on the quantization nodes and the dequantization nodes; subsequently, obtains the calculation characteristic parameters of the target hardware corresponding to the network traffic detection scene, determines the initial quantization model corresponding to the first model and the second model by using the calculation characteristic parameters; then determines the distillation loss function corresponding to the initial quantization model by using the quantization parameters, determines the back propagation strategy of the second model based on the propagation parameters corresponding to the initial quantization model, and determines the training strategy corresponding to the initial quantization model by using the distillation loss function and the back propagation strategy; finally, obtains the quantization model corresponding to the network traffic detection scene by training the initial quantization model according to the training strategy and the preset training round. In the method, the distillation is performed in the process of guiding the second model (student model) to learn and train by the first model (teacher model), the related calculation characteristic parameters are optimized for the target hardware, and the second model can be adapted to low-precision calculation by simulating abnormal traffic, so that the distillation loss and the quantization error are simultaneously optimized in the training process, the performance overhead is reduced, the model precision is ensured, and the method is more suitable for deployment in edge computing devices.

[0050] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by the structure particularly pointed out in the description, claims and drawings.

[0051] In order to make the above-mentioned objects, features and advantages of the present application more apparent, the following preferred embodiments are specifically described with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the specific embodiments of the present application or the prior art, the drawings needed in the description of the specific embodiments or the prior art will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0053] Figure 1 The flow chart of the network traffic detection model distillation method provided by the embodiment of the present application;

[0054] Figure 2 The flow chart of step S101 of the network traffic detection model distillation method provided by the embodiment of the present application;

[0055] Figure 3 In step S102 of the network traffic detection model distillation method provided by the embodiment of the present application, the flow chart of determining the quantization nodes and the dequantization nodes between the first model and the second model according to the structure parameters of the first model and the second model;

[0056] Figure 4 In step S102 of the network traffic detection model distillation method provided by the embodiment of the present application, the flow chart of determining the quantization parameters of the first model and the second model based on the quantization nodes and the dequantization nodes;

[0057] Figure 5 The flow chart of step S103 of the network traffic detection model distillation method provided by the embodiment of the present application;

[0058] Figure 6 In step S104 of the network traffic detection model distillation method provided by the embodiment of the present application, the flow chart of determining the distillation loss function corresponding to the initial quantization model through the quantization parameters;

[0059] Figure 7 In step S104 of the network traffic detection model distillation method provided by the embodiment of the present application, the flow chart of determining the back propagation strategy of the second model based on the propagation parameters corresponding to the initial quantization model, and determining the training strategy corresponding to the initial quantization model by using the distillation loss function and the back propagation strategy;

[0060] Figure 8 The flow chart of step S105 of the network traffic detection model distillation method provided by the embodiment of the present application;

[0061] Figure 9 The flow chart of another network traffic detection model distillation method provided by the embodiment of the present application;

[0062] Figure 10A structural schematic diagram of a network traffic detection model distillation system provided for an embodiment of the present application is shown in FIG. 1.

[0063] Figure 11 A structural schematic diagram of a server provided for an embodiment of the present application is shown in FIG. 4.

[0064] Icon:

[0065] 1010 - model construction module; 1020 - quantization parameter determination module; 1030 - quantization model initialization module; 1040 - quantization model training distillation module; 1050 - quantization model generation module.

[0066] 101 - processor; 102 - memory; 103 - bus; 104 - communication interface. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the present application will be described clearly and completely below in conjunction with embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0068] In the network traffic detection process, a related neural network model has been used to achieve it, and its original design is particularly suitable for the deployment needs of edge computing devices, which are often limited by limited computing power, memory resources and energy consumption constraints, such as Internet of Things terminals, edge gateways, industrial control edge nodes and other scenarios, which need to achieve efficient and accurate traffic anomaly identification in resource-constrained environments.

[0069] The existing technology still has many dimensional pain points in this field, which can be further divided into the following four aspects:

[0070] First, traditional anomaly detection models (such as convolutional neural networks CNN, recurrent neural networks RNN, etc.) face significant resource bottlenecks when deployed on hardware. Such models, in pursuit of detection accuracy, often contain a large number of hierarchical structures and parameters, resulting in extremely high computational intensity (e.g., frequent high-dimensional matrix operations are required) and large memory usage (both model parameter storage and intermediate feature mapping cache require a large amount of space). In the scenario of edge devices with limited computing power (mostly low-power CPUs or lightweight NPUs) and tight memory resources, not only will the inference delay increase dramatically, but the model may even fail to run normally due to insufficient resources, severely affecting real-time detection capabilities.

[0071] Secondly, the existing model distillation method is not suitable for the characteristics of the target hardware, resulting in that the inference efficiency after deployment does not meet the expectation. The core of model distillation is to compress the model size through the 'teacher-student' architecture, but most current methods only focus on preserving the model accuracy, ignoring the underlying architecture characteristics of the target hardware, such as the computing power distribution of the hardware (such as the number of CPU cores, cache levels), data interaction bandwidth, instruction set support (such as whether compatible with SIMD acceleration), etc. This makes the distilled model, although the parameter size is reduced, the computing scheduling efficiency on the actual hardware is low, and the hardware computing power cannot be fully utilized, ultimately resulting in limited improvement of inference speed and obvious efficiency loss.

[0072] Thirdly, the scarcity and single pattern of abnormal traffic samples seriously restrict the generalization ability of the model. In the actual network environment, normal traffic accounts for more than 99%, while abnormal traffic (such as new network attacks, malicious intrusion behavior) not only has a small number of samples, but also the attack pattern continuously evolves (such as variant viruses, zero-day exploit). The existing model relies on supervised learning, and when training on limited and single abnormal samples, it is easy to fall into overfitting - that is, the recognition accuracy of known abnormalities in the training set is high, but the detection ability of new and variant abnormalities is poor, making it difficult to adapt to complex and variable network threat environment.

[0073] Fourthly, under the dynamic network environment, it is difficult for a fixed parameter model to maintain the best detection performance. The network environment is always in dynamic change: on the one hand, the network topology (such as device access, routing adjustment), traffic characteristics (such as bandwidth fluctuation, protocol update) will continuously change; on the other hand, the means of attackers are also constantly upgrading (such as new attack strategies to evade detection). The parameters of the traditional model are fixed after training, and the model structure or parameters cannot be dynamically adjusted according to the real-time network state, resulting in poor adaptability of the model to dynamic changes - when the network environment changes significantly, the detection accuracy of the model will decrease sharply, the false positive rate and the false negative rate will increase, making it difficult to play a long-term stable detection role.

[0074] As can be seen, the above network traffic detection model has technical defects, resulting in poor model detection effect and high computing cost, making it difficult to deploy in edge computing devices. Based on this, the present application provides a network traffic detection model distillation method, system and server, which distills the process of learning and training of the second model (student model) guided by the first model (teacher model), optimizes the related computing characteristic parameters for the target hardware, and can adapt to low-precision computing by simulating abnormal traffic, thereby synchronously optimizing the distillation loss and quantization error in the training process, reducing the performance overhead while ensuring the model accuracy, and being more suitable for deployment in edge computing devices.

[0075] For the convenience of understanding the present embodiment, first, a network traffic detection model distillation method disclosed by the present embodiment is introduced in detail. The present method involves two parties, namely a client and a cloud. As shown in Figure 1 The method comprises the following steps.

[0076] Step S101: constructing a first model based on attribute parameters corresponding to a network traffic detection scene, and constructing a second model based on structure parameters of the first model.

[0077] This step is the basic construction process of the model. First, a first model (teacher model) is constructed according to the attribute parameters of the network traffic detection scene (such as the traffic protocol type to be detected, the feature dimension, the real-time requirement, etc.). The model needs to have strong abnormal recognition ability, and usually adopts a high complexity structure to ensure accuracy. Then, based on the core structure parameters of the teacher model, the level is simplified or the parameter size is compressed to construct a second model (student model), so that it retains the key feature extraction capability while being more suitable for lightweight deployment.

[0078] Step S102: determining the quantization node and the dequantization node between the first model and the second model according to the structure parameters of the first model and the second model, and determining the quantization parameters of the first model and the second model based on the quantization node and the dequantization node.

[0079] This step mainly realizes the quantization node and parameter setting process. According to the structure of the teacher model and the student model (such as the position of the calculation-intensive layer, the feature interaction node, etc.), the quantization node (mostly the calculation-intensive layer such as convolution layer and fully connected layer) and the dequantization node (mostly before feature fusion or output, used to reduce the accumulation of quantization error) are determined. Based on these nodes, the quantization parameters are further determined, including the quantization bit number (such as 8 bits, 16 bits), the scaling factor (mapping floating-point value to integer range), etc., which provides specific rules for model quantization.

[0080] Step S103: obtaining the calculation characteristic parameters of the target hardware corresponding to the network traffic detection scene, and determining the initial quantization model corresponding to the first model and the second model by using the calculation characteristic parameters.

[0081] This step mainly realizes the hardware characteristic adaptation and initial quantization model generation process. First, the calculation characteristic parameters of the target hardware (such as the processor type, memory capacity, and algorithmic upper limit of the edge device) are obtained, and then the quantization strategy is adjusted in combination with these parameters. For example, if the hardware supports INT8 instruction acceleration, 8-bit quantization is preferred; if the memory is limited, the data storage mode after quantization is optimized. Based on this, the initial quantization model adapted to the hardware is generated to ensure that the model structure matches the hardware capability.

[0082] Step S104: Determine the distillation loss function corresponding to the initial quantization model through the quantization parameter, determine the back propagation strategy of the second model based on the propagation parameter corresponding to the initial quantization model, and determine the training strategy corresponding to the initial quantization model by using the distillation loss function and the back propagation strategy.

[0083] According to the quantization parameter of step S102, the distillation loss function is designed (usually the "soft label" loss of the teacher model output is fused with the "hard label" loss of the real label, and the knowledge transfer and the basic accuracy are balanced). In view of the non-differentiability of the quantization operation, methods such as straight through estimator (STE) are used to determine the back propagation strategy of the student model, to ensure the stability of gradient calculation. Finally, the loss function and the back propagation strategy are combined to determine the training strategy (such as learning rate scheduling, iterative optimization method, etc.).

[0084] Step S105: After training the initial quantization model by using the training strategy and according to the preset training round, a quantization model corresponding to the network traffic detection scene is obtained.

[0085] According to the preset training round, the initial quantization model is trained according to the training strategy determined in step S104. Through multiple rounds of iterative optimization, the student model fully learns the knowledge of the teacher model, while adapting to the quantization rules and hardware characteristics. After training, a quantization model that can be directly deployed in the target scene is obtained, which can not only maintain high detection accuracy, but also meet the resource constraints of edge devices.

[0086] Optionally, the step S101 of constructing the first model based on the attribute parameter corresponding to the network traffic detection scene and constructing the second model based on the structure parameter of the first model, as shown in Figure 2 , includes:

[0087] Step S201: Obtain traffic parameters, scene parameters, network parameters and detection parameters corresponding to the network traffic detection scene; determine attribute parameters corresponding to the network traffic detection scene based on the traffic parameters, scene parameters, network parameters and detection parameters.

[0088] Firstly, four types of core parameters of the network traffic detection scene are collected: traffic parameters (such as protocol type, packet length, feature dimension, etc.); scene parameters (such as deployment device type, environmental stability, etc.); network parameters (such as bandwidth limit, delay requirement, etc.); detection parameters (such as target attack type, accuracy and false alarm rate requirement, etc.). By integrating these parameters, the attribute parameters reflecting the core requirements of the scene are extracted, providing clear basis for subsequent model design (for example, in a scene with high real-time requirement, the attribute parameter will highlight the "low delay" feature).

[0089] Step S202: Construct a first model corresponding to the network traffic detection scene according to the single precision parameter based on the attribute parameter.

[0090] Based on the attribute parameters determined in step S201, a first model is constructed using single-precision parameters (such as 32-bit floating-point numbers, FP32). Single-precision parameters can retain more feature details and ensure that the model has strong abnormal recognition capability; at the same time, the model structure will be adjusted according to the attribute parameters - for example, for time-series strong flow scenarios, LSTM or Transformer architecture can be used to accurately capture the dynamic pattern of flow.

[0091] Step S203: Determine the layer structure corresponding to the second model according to the structure parameters of the first model, and construct the second model based on the layer structure.

[0092] Taking the structure parameters (such as the number of layers, core layer types, feature mapping dimensions, etc.) of the first model as a benchmark, the model structure is simplified: for example, the number of convolution layers is reduced, the parameter size of the fully connected layer is compressed, or a lightweight operator is used to replace a complex calculation unit. By retaining the core feature extraction layer, it is ensured that the student model can effectively inherit the knowledge of the teacher model while being lightweight, and adapt to the resource constraints of edge devices. These three sub-steps are progressive, from scene characteristic analysis to model structure design, laying the foundation for the “teacher-student” distillation framework to adapt to scene requirements.

[0093] Optionally, the quantization nodes and the dequantization nodes between the first model and the second model are determined according to the structure parameters of the first model and the second model, as shown in Figure 3 , including:

[0094] Step S301: Obtain the structure parameters of the first model and the second model, and initialize the pseudo-quantization modules corresponding to the first model and the second model according to the structure parameters.

[0095] First, obtain the structure parameters of the first model (teacher model) and the second model (student model), including the types of each layer (such as convolution layer, fully connected layer, activation layer), parameter size (such as the number of convolution kernels, the number of neurons), calculation complexity (such as operations per second) and the like. Based on these structure parameters, pseudo-quantization modules are initialized in the two models. The role of such modules is to simulate the quantization process (such as converting floating-point parameters to low-precision integers) during model training, providing adaptation basis for subsequent real quantization deployment, and ensuring that quantization errors are effectively learned and compensated in training.

[0096] Step S302: Determine the quantization position parameters corresponding to the pseudo-quantization module, and determine the dequantization position parameters corresponding to the pseudo-quantization module according to the convolution parameters of the first model and the second model.

[0097] Based on the function positioning of the pseudo quantization module, the quantization position parameter is first determined: usually, the layers with high calculation intensity and high parameter proportion in the model (such as convolutional layers and fully connected layers) are selected as the quantization core position, which can significantly reduce the calculation amount and memory occupation after quantization, and is the area with the largest quantization benefit.

[0098] At the same time, the dequantization position parameter is determined in combination with the convolution parameters (such as the convolution kernel size, the output feature map dimension, and the convolution step) of the first model and the second model: the dequantization needs to be set at the key connection of the quantization operation (such as after the convolutional layer output and before the feature fusion layer), so as to avoid the accumulation of quantization errors when the errors are transmitted between layers, and to ensure that the subsequent feature processing (such as residual connection and multi-scale feature fusion) can be based on relatively complete information.

[0099] Step S303: determining the quantization node based on the quantization position parameter, and determining the dequantization node based on the dequantization position parameter.

[0100] Based on the quantization position parameter obtained in step S302, the corresponding layer is determined as the quantization node. These nodes will perform real quantization operations (such as FP32 to INT8) in actual deployment; based on the dequantization position parameter, the corresponding connection is determined as the dequantization node. These nodes are responsible for restoring the low-precision data after quantization to floating-point format to ensure the high-precision calculation requirements of the subsequent layers of the model.

[0101] Through this node division, model compression can be realized through the quantization node, and error accumulation can be controlled through the dequantization node, balancing the model efficiency and detection accuracy.

[0102] Optionally, the quantization parameters of the first model and the second model are determined based on the quantization node and the dequantization node, such as Figure 4 As shown in the figure, it comprises:

[0103] Step S401: determining the quantization effect of the first model and the second model during training based on the quantization node and the dequantization node.

[0104] Based on the determined quantization node (such as convolutional layer and fully connected layer) and dequantization node (such as the connection before feature fusion), the quantization and dequantization operations are simulated in the model training process to evaluate the corresponding quantization effect. The quantization effect mainly includes: the precision loss degree of the quantized model (such as the error of the output and the original floating-point model), the distribution change of the intermediate features (such as whether the key abnormal patterns are retained after quantization), the improvement amplitude of the calculation efficiency (such as the memory occupation reduction ratio and the inference speed improvement multiple), etc. By tracking these indicators, the actual influence of the quantization operation on the model performance is determined to provide a basis for subsequent parameter adjustment.

[0105] Step S402: determining the symmetry parameters and reading parameters corresponding to the first model and the second model according to the quantization effect.

[0106] According to the quantization effect evaluated in step S401, further determine two types of core parameters:

[0107] Symmetry parameters: mainly used to define the symmetry of the quantization strategy (such as whether to use symmetric quantization, mapping the data range with zero point as the center; or asymmetric quantization, determining the mapping range according to the actual distribution of data). For example, if the quantization effect shows that the feature distribution is symmetric (such as the mean value close to 0), the symmetric parameter is preferred to simplify the calculation; if the feature distribution is obviously offset (such as most values are positive), the asymmetric parameter is adjusted to reduce the precision loss.

[0108] Reading parameters: related to the storage and reading rules of quantized data, such as the bit width of quantized data (8 bits / 16 bits), byte alignment mode (to adapt to the efficiency of hardware memory access), etc. For example, if the quantization effect shows that 8-bit quantization can meet the precision requirement, and the target hardware supports 8-bit reading acceleration, the reading parameter is set to 8-bit bit width.

[0109] Step S403: determining the quantization parameters of the first model and the second model using the symmetry parameters and reading parameters.

[0110] Combining the symmetry parameters and reading parameters obtained in step S402, calculate and determine the complete quantization parameters of the first model (teacher model) and the second model (student model). These quantization parameters include: scaling factor (proportion of mapping floating point value to quantized integer range), zero point offset (compensation value to eliminate quantization deviation), which is determined by the symmetry parameter (zero point offset is usually 0 in symmetric quantization, and needs to be calculated according to the data range in asymmetric quantization); quantization bit number, data storage format is clear from reading parameter, to ensure the adaptation with hardware reading efficiency. The final quantization parameters will serve as the specific execution rules for the actual quantization operation of the model, which not only ensures that the quantized data can be efficiently processed by hardware, but also maximally reduces the impact on detection precision.

[0111] Optionally, obtain the computing characteristic parameters of the target hardware corresponding to the network traffic detection scene, and determine the initial quantization model corresponding to the first model and the second model using the computing characteristic parameters, as shown in step S103: Figure 5 , which includes:

[0112] Step S501: obtaining the target hardware corresponding to the network traffic detection scene, and determining the computing characteristic parameters according to the performance parameters of the target hardware.

[0113] Firstly, the target hardware corresponding to the network traffic detection scene is determined (such as edge gateway, Internet of Things terminal, etc.), and the core performance parameters of the hardware are collected, such as processor type, computing power index, memory configuration, instruction set support and energy consumption limit, etc. Based on these performance parameters, the computing characteristic parameters reflecting the computing power and constraints of the hardware are extracted, which provide the hardware adaptation basis for subsequent model setting.

[0114] Step S502: setting first model corresponding first setting parameter and second model corresponding second setting parameter based on computing characteristic parameter; wherein, under first setting parameter, first model is used to guide second model; under second setting parameter, second model adopts EfficientNet-Lite model structure.

[0115] Based on the computing characteristic parameters obtained in step S501, the first model (teacher model) and the second model (student model) are set with specific parameters. The first setting parameter is for the teacher model, which needs to ensure that it can effectively output high-quality "knowledge" to guide the student model, while adapting to the basic computing power of the hardware (if the hardware computing power is limited, the batch processing size of the teacher model can be limited to avoid overload). The second setting parameter is for the student model, which specifies that it adopts the EfficientNet-Lite architecture. This architecture significantly reduces the amount of calculation while maintaining accuracy through designs such as depth separable convolution and dynamic resolution adjustment, naturally adapting to the resource constraints of edge hardware; the setting parameter will also be further optimized in combination with the hardware characteristics.

[0116] Step S503: constructing an initial quantization model using the first setting parameter and the second setting parameter.

[0117] The first setting parameter and the second setting parameter determined in step S502 are applied to the first model and the second model respectively, and combined with the quantization nodes and the basic quantization rules determined in the early stage, an initial quantization model is constructed. This model needs to meet the following conditions: the teacher model can stably output guidance information under the first setting parameter, the student model has a lightweight basis under the second setting parameter (EfficientNet-Lite structure + hardware adaptation optimization), and the quantization logic of the two models is deeply matched with the computing characteristic parameters of the target hardware (such as quantization bit width, data access mode matching hardware instruction set), providing a hardware-friendly initial structure for subsequent training.

[0118] Optionally, the distillation loss function corresponding to the initial quantization model is determined by the quantization parameters, such as Figure 6 as shown, including:

[0119] Step S601: determining the first end layer output value of the first model and the second end layer output value of the second model using the quantization parameters, and constructing an initial distillation loss function corresponding to the initial quantization model according to the first end layer output value and the second end layer output value.

[0120] First, based on the determined quantization parameters (such as scaling factors, quantization bit numbers, etc.), the first end layer output value (i.e., the final prediction result of the model, such as the classification probability distribution of abnormal traffic) of the first model (teacher model) and the second end layer output value of the second model (student model) are calculated respectively. Since the quantization operation may affect the output accuracy, the output value here needs to be processed by quantization-dequantization to fit the actual deployment scenario. Subsequently, an initial distillation loss function is constructed according to the two end layer output values, which mainly measures the difference between the final prediction results of the student model and the teacher model, and the commonly used form is KL divergence (measuring the similarity of two probability distributions), to ensure that the student model can learn the decision logic of the teacher model (such as high confidence prediction for specific abnormal traffic).

[0121] Step S602: Determine the first feature output value of the first model and the second feature output value of the second model using the quantization parameters, and construct a feature loss function corresponding to the initial quantization model according to the first feature output value and the second feature output value.

[0122] Using the quantization parameters, the intermediate layer outputs (i.e., feature output values, such as traffic feature maps generated by convolution layers, weight distribution of attention layers, etc.) of the first model and the second model are further extracted, which are respectively denoted as the first feature output value and the second feature output value. These intermediate features reflect the step-by-step processing of the model on the traffic data, and are the key details of "knowledge transfer".

[0123] Based on these feature output values, a feature loss function is constructed, which is commonly used in the form of L2 distance (measuring the difference between feature vectors) or cosine similarity (measuring the consistency of feature directions). The function is to let the student model not only imitate the final conclusion of the teacher model, but also learn its intermediate logic of feature extraction, avoiding the loss of deep patterns by only learning the surface prediction results.

[0124] Step S603: Construct a distillation loss function corresponding to the initial quantization model based on the initial distillation loss function and the feature loss function.

[0125] The initial distillation loss function obtained in step S601 (focusing on the difference between the end layer outputs) and the feature loss function obtained in step S602 (focusing on the difference between the intermediate features) are combined by weighting, to construct a final distillation loss function corresponding to the initial quantization model. The weight distribution needs to be adjusted in combination with the quantization scenario: for example, if the loss of end layer output accuracy after quantization is large, the weight of the initial distillation loss function can be increased; if the intermediate features have a more critical impact on detection accuracy (such as traffic time series features), the weight of the feature loss function can be increased. The final loss function can comprehensively guide the student model to learn "decision results + feature processing logic" from the teacher model, maximizing the knowledge transfer effect under the quantization constraint.

[0126] Optionally, the back propagation strategy of the second model is determined based on the propagation parameter corresponding to the initial quantization model, and the training strategy corresponding to the initial quantization model is determined by using the distillation loss function and the back propagation strategy, as shown in Figure 7 as shown, comprising:

[0127] Step S701: determining the inference strategy corresponding to the first model based on the propagation parameter corresponding to the initial quantization model, and determining the forward propagation strategy corresponding to the second model according to the inference strategy.

[0128] This step realizes the determination process of the inference and forward propagation strategy. Based on the propagation parameter (such as the data flow rule of each layer, the calculation logic of the quantization / dequantization node, etc.) of the initial quantization model, the inference strategy of the first model (teacher model) is first determined, that is, the flow of the teacher model processing the input traffic data (such as the feature extraction sequence, the intermediate result transmission mode), to ensure that it can stably output high-precision decision information. Then, according to the inference strategy of the teacher model, the forward propagation strategy of the second model (student model) is determined, so that the input processing and feature transmission path of the student model correspond to the teacher model (such as the same feature dimension mapping, matching layer calculation sequence), to provide a consistent "data flow framework" for the knowledge comparison and migration of the two, and avoid the knowledge transfer deviation caused by the process difference.

[0129] Step S702: determining the loss function calculation strategy corresponding to the initial quantization model according to the forward propagation strategy and by using the distillation loss function.

[0130] In combination with the forward propagation strategy (determining the data flow path of the student model) determined in step S701 and the distillation loss function (including the last layer output loss and the intermediate feature loss) constructed in the early stage, the loss function calculation strategy is formulated. This strategy needs to determine the nodes of loss calculation (such as calculating the loss at the last layer and the intermediate feature layer of the student model respectively), the calculation time (such as calculating in real time after each iteration, or calculating after accumulating multiple batches), the weight dynamic adjustment rule (such as focusing on the intermediate feature loss in the early stage of training to stabilize the feature learning, and increasing the last layer output loss in the later stage to optimize the final accuracy), to ensure that the loss value can accurately reflect the "knowledge gap" between the student model and the teacher model.

[0131] Step S703: determining the back propagation strategy corresponding to the second model based on the propagation parameter and by using the forward propagation strategy.

[0132] This step is the determination process of the back propagation strategy, which is based on the propagation parameters of the initial quantization model (especially the gradient processing rules of the quantization nodes) and combined with the forward propagation strategy (clear data flow path) to determine the back propagation strategy of the student model. Due to the non-differentiability of the quantization operation (such as integer mapping), the gradient approximation method (such as the straight-through estimator STE) needs to be included in the strategy, that is, the forward propagation performs quantization, and when back propagation, the derivative of the quantization operation is ignored, and the gradient is directly calculated using the dequantized data to avoid gradient explosion or disappearance. At the same time, the gradient transmission path (corresponding to the reverse of the forward propagation path) and the gradient clipping threshold (to prevent amplification of quantization errors) are determined to ensure stable parameter updating.

[0133] Step S704: According to the adversarial parameters corresponding to the initial quantization model, the abnormal traffic corresponding to the network traffic detection scene is constructed, and the initial quantization model is determined after the feature distillation of the abnormal traffic.

[0134] This step is mainly the determination process of the adversarial strategy, which generates abnormal traffic samples (such as variant traffic simulating DDoS attacks, data packets containing hidden malicious codes) in the network traffic detection scene according to the adversarial parameters of the initial quantization model (such as the disturbance intensity of common attacks, the feature distribution of abnormal traffic). The core distinguishing features (such as abnormal packet interval, abnormal protocol field value) are extracted by performing "feature distillation" on these abnormal traffics, and the adversarial strategy is determined based on this: for example, mix abnormal traffic samples in a certain proportion during training to force the model to learn anti-interference features; or dynamically adjust the disturbance intensity of abnormal samples to enhance the generalization ability of the model to new attacks.

[0135] Step S705: Determine the training strategy corresponding to the initial quantization model based on the forward propagation strategy, the loss function calculation strategy, the back propagation strategy and the adversarial strategy.

[0136] The forward propagation strategy of step S701, the loss function calculation strategy of S702, the back propagation strategy of S703, and the adversarial strategy of S704 are integrated to form the complete training strategy of the initial quantization model. The strategy needs to be clear: the total number of training iterations, the dynamic scheduling of learning rate (such as high initial learning rate for fast convergence, and low learning rate for fine-tuning in later period), the mixing frequency of adversarial samples, and the updating rule of model parameters (such as updating the core feature layer of the student model first), etc. Finally, the collaborative optimization of "efficient knowledge transfer + quantization error control + adversarial robustness improvement" is realized to ensure that the trained model can adapt to hardware and has strong detection capability.

[0137] Optionally, after training the initial quantization model using the training strategy and according to the preset training rounds, the step S105 of obtaining the quantization model corresponding to the network traffic detection scene is shown as follows: Figure 8 ​

[0138] Step S801: Determine the learning rate parameter, temperature parameter, scaling factor parameter and zero point parameter corresponding to the initial quantization model using the training strategy.

[0139] According to the training strategy determined in advance, the core parameters required for training the initial quantization model are extracted and clarified. Specifically, the learning rate parameter: controls the amplitude of model parameter update (such as setting the initial learning rate to 0.001, which is attenuated later with training iterations), affects the convergence speed and stability; the temperature parameter: adjusts the smoothness of the “soft label” output by the teacher model in the distillation loss calculation (such as a higher temperature value, the label distribution is smoother, which is beneficial for the student model to learn more generalized patterns); the scaling factor parameter and the zero point parameter: the core mapping parameters in the quantization process (the scaling factor is used to map the floating-point features to the integer range, and the zero point parameter compensates for the quantization bias), which directly affect the quantization accuracy. These parameters together constitute the “adjustment knobs” of training, ensuring efficient convergence of the model under quantization constraints.

[0140] Step S802: Train the initial quantization model based on the learning rate parameter, temperature parameter, scaling factor parameter and zero point parameter, and obtain the training times corresponding to the initial quantization model in real time.

[0141] Based on the learning rate, temperature, scaling factor and zero point parameters determined in step S801, the training process of the initial quantization model is started: the learning rate controls the parameter update rhythm, the temperature parameter optimizes the soft label knowledge transfer, and the scaling factor and zero point parameter ensures the stability of the quantization process. The training times are recorded and monitored in real time (such as completing one full data iteration is counted as 1 training), ensuring that the training progress can be tracked.

[0142] Step S803: If the training times are less than the preset training rounds, control the initial quantization model to continue training; if the training times are equal to the preset training rounds, control the initial quantization model to stop training, and determine the current initial quantization model as the quantization model for the network traffic detection scenario.

[0143] Compare the real-time training times with the preset training rounds (such as 100 rounds, set according to the complexity of the scene to ensure that the model learns sufficiently): if the training times do not reach the preset rounds, continue training with the current parameters until the target rounds are approached, at which point the learning rate can be fine-tuned (such as reduced to 1 / 10 of the initial value) for fine-tuning; if the training times reach the preset rounds, stop training, at which point the model has adapted to the quantization rules and hardware characteristics through multiple iterations, and the detection accuracy (such as abnormal recognition accuracy, F1 score) reaches the preset standard, and it is determined as the final quantization model for the network traffic detection scenario, which can be directly deployed on the target edge device.

[0144] For example, Figure 9The flowchart of another network traffic detection model distillation method is shown, in which the teacher model is a pre-trained high-precision floating-point model, which can be trained using full precision (FP32).

[0145] The student model adopts a lightweight structure design. You can choose EfficientNet-Lite to design a lightweight structure and remove large-range activation functions and other operations that are not conducive to quantization.

[0146] The pseudo-quantization module inserts quantization-dequantization nodes (QuantStub and DeQuantStub) during training to simulate the quantization effect during inference. Quantization configurations allow selection of symmetric / asymmetric, layer-by-layer / channel-by-channel quantization schemes, and quantization parameters (such as 8-bit integers) can be set according to the target hardware.

[0147] The pseudo-quantization module can be inserted in PyTorch using `torch.quantization.prepare_qat` to insert a pseudo-quantization node, as follows:

[0148] class QuantizableStudentModel(nn.Module):

[0149] def __init__(self):

[0150] super().__init__()

[0151] self.quant = torch.quantization.QuantStub()

[0152] self.conv1 = nn.Conv2d(3, 64, kernel_size=3)

[0153] self.dequant = torch.quantization.DeQuantStub()

[0154] def forward(self, x):

[0155] x = self.quant(x) # Quantize the input

[0156] x = self.conv1(x)

[0157] x = self.dequant(x) # Dequantize output

[0158] return x

[0159] The loss function combines the distillation loss (soft label, feature matching) and the quantization regularization term. Specifically, the distillation loss is:

[0160] def distill_loss(teacher_logits, student_logits, temperature=5.0):

[0161] soft_teacher = F.softmax(teacher_logits / temperature, dim=1)

[0162] soft_student = F.log_softmax(student_logits / temperature, dim=1)

[0163] return F.kl_div(soft_student, soft_teacher, reduction="batchmean") *(temperature ** 2)

[0164] The intermediate layer MSE feature matching loss is adopted:

[0165] def feature_loss(teacher_features, student_features):

[0166] loss = 0

[0167] for t_feat, s_feat in zip(teacher_features, student_features):

[0168] loss += F.mse_loss(t_feat, s_feat)

[0169] return loss

[0170] The total loss is set as: total_loss = α * distill_loss + β * feature_loss.

[0171] The training process first sets the optimization indicators for the quantization hardware, analyzes the computing characteristics of the target hardware, and establishes an operator performance benchmark library; reconstructs the deep separable convolution module and sets the 4-bit channel-level quantization strategy; and realizes the instruction mapping of the quantization operator and the training core through the hardware acceleration library (CANN).

[0172] Second, initialize the quantization-aware training as follows:

[0173] student_model.qconfig = torch.quantization.get_default_qat_qconfig("fbgemm")

[0174] torch.quantization.prepare_qat(student_model, inplace=True)

[0175] Then, design the training loop process as follows:

[0176] for epoch in range(num_epochs):

[0177] student_model.train()

[0178] for inputs, labels in dataloader:

[0179] # Teacher model inference

[0180] with torch.no_grad():

[0181] teacher_outputs = teacher_model(inputs)

[0182] # Student model forward (including pseudo quantization)

[0183] student_outputs = student_model(inputs)

[0184] # Calculate loss

[0185] loss_distill = distill_loss(teacher_outputs, student_outputs)

[0186] loss_total = loss_distill + 0.5 * feature_loss(...)# Weighted sum

[0187] # Backpropagation and optimization

[0188] optimizer.zero_grad()

[0189] loss_total.backward()

[0190] optimizer.step()

[0191] Then, the abnormal traffic data is constructed for the dynamic distillation process, and a generative adversarial network (ST-GAN) method is used to construct abnormal traffic, and on this basis, three-level feature distillation is performed, that is, the feature map level is aligned by attention feature loss; the timing level is based on the dynamic time warping distance (DTW) timing mode matching; and the decision level is based on the KL divergence (Kullback-Leibler Divergence) optimization classification boundary.

[0192] Finally, the final quantization model is converted, as follows:

[0193] student_model.eval()

[0194] torch.quantization.convert(student_model, inplace=True).

[0195] During the loop, relevant optimizations can be performed, such as learning rate adjustment: using a lower initial learning rate (such as 1e-4) to avoid quantization noise interference convergence; temperature parameter (Temperature): gradually adjusting (from 5.0 to 1.0) to balance soft / hard label learning; pseudo-quantization parameter update: updating the scale factor (scale) and zero point (zero_point) through moving average (EMA); gradient processing: using the straight-through estimator (STE) method to bypass the non-differentiability of the quantization operation.

[0196] In actual scenarios, the above method reduces the model memory usage by 60%-75% (compared to the FP32 benchmark); the inference speed is improved by 3-5 times (measured on edge computing devices); the abnormal detection F1-score is improved by more than 12%; and dynamic reconstruction of the model structure is supported (parameters can be adjusted within 50ms).

[0197] As can be seen from the network traffic detection model distillation method in the above embodiment, in this method, distillation is performed in the process of guiding the second model (student model) to learn and train by the first model (teacher model), the relevant calculation characteristic parameters are optimized for the target hardware, and the second model can adapt to low-precision calculation by simulating abnormal traffic, thereby synchronously optimizing the distillation loss and quantization error in the training process, reducing the performance overhead while ensuring the model accuracy, and being more suitable for deployment in edge computing devices.

[0198] Corresponding to the network traffic detection model distillation method embodiment described above, the embodiment of the application also provides a network traffic detection model distillation system, as shown in the figure, the system comprises: Figure 10

[0199] The model construction module 1010 is configured to construct a first model based on attribute parameters corresponding to a network traffic detection scenario, and construct a second model based on structure parameters of the first model.

[0200] The quantization parameter determination module 1020 is configured to determine quantization nodes and dequantization nodes between the first model and the second model according to the structure parameters of the first model and the second model, and determine quantization parameters of the first model and the second model based on the quantization nodes and the dequantization nodes.

[0201] The quantization model initialization module 1030 is configured to obtain calculation characteristic parameters of a target hardware corresponding to the network traffic detection scenario, and determine initial quantization models corresponding to the first model and the second model by using the calculation characteristic parameters.

[0202] The quantization model training distillation module 1040 is configured to determine an initial loss function corresponding to the initial quantization models by using the quantization parameters, determine a back propagation strategy of the second model based on propagation parameters corresponding to the initial quantization models, and determine a training strategy corresponding to the initial quantization models by using the initial loss function and the back propagation strategy.

[0203] The quantization model generation module 1050 is configured to obtain a quantization model corresponding to the network traffic detection scenario by training the initial quantization models according to a preset training round number by using the training strategy.

[0204] Through the network traffic detection model distillation system described above, it can be known that in the system, distillation is performed in the process of guiding the second model (student model) to learn and train by the first model (teacher model), the related calculation characteristic parameters are optimized for the target hardware, and the second model can be adapted to low-precision calculation by simulating abnormal traffic, so that the distillation loss and the quantization error are simultaneously optimized in the training process, the performance overhead is reduced, and the model precision is ensured, which is more suitable for deployment in edge computing devices.

[0205] The network traffic detection model distillation system provided by the embodiment of the application has the same implementation principle and technical effects as the network traffic detection model distillation method embodiment described above. For brevity of description, the part of the system embodiment not mentioned can be referred to the corresponding content in the network traffic detection model distillation method embodiment described above.

[0206] The embodiment also provides a server, and a structure diagram of the server is as shown in Figure 11 ​As shown, the device includes a processor 101 and a memory 102; wherein the memory 102 is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the steps of the network traffic detection model distillation method described above.

[0207] Figure 11 The server as shown also includes a bus 103 and a communication interface 104, and the processor 101, the communication interface 104 and the memory 102 are connected through the bus 103.

[0208] Wherein, the memory 102 can contain a high-speed random access memory (RAM, Random Access Memory), and can also include a non-volatile memory, for example at least one disk memory. The bus 103 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 11 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0209] The communication interface 104 is configured to connect with at least one user terminal and other network elements through a network interface, and send the encapsulated IPv4 packet or IPv4 packet to the user terminal through the network interface.

[0210] The processor 101 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 101 or the instruction in the form of software. The processor 101 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102 and combines the hardware to complete the steps of the method of the above embodiments.

[0211] The embodiment of the present application further provides a storage medium, and the storage medium stores a computer program. When the computer program is run by a processor, the steps of the network traffic detection model distillation method in the above embodiment are executed.

[0212] In several embodiments provided in the present application, it should be understood that the disclosed system, device, equipment and method can be implemented by other ways. The system embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, equipment or unit, and can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0214] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0215] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various program code storage media.

[0216] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, used to illustrate the technical solutions of the present application, and not to limit them, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: any skilled person in the art within the technical range disclosed by the present application, they can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and all should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A distillation method for a network traffic detection model, characterized in that, The method includes: A first model is constructed based on the attribute parameters corresponding to the network traffic detection scenario, and a second model is constructed based on the structural parameters of the first model. The quantization node and dequantization node between the first model and the second model are determined based on the structural parameters of the first model and the second model, and the quantization parameters of the first model and the second model are determined based on the quantization node and the dequantization node. Obtain the computational characteristic parameters of the target hardware corresponding to the network traffic detection scenario, and use the computational characteristic parameters to determine the initial quantization model corresponding to the first model and the second model; The distillation loss function corresponding to the initial quantization model is determined by the quantization parameters, the backpropagation strategy of the second model is determined based on the propagation parameters corresponding to the initial quantization model, and the training strategy corresponding to the initial quantization model is determined by the distillation loss function and the backpropagation strategy. After training the initial quantization model using the training strategy and according to the preset training rounds, the quantization model corresponding to the network traffic detection scenario is obtained. Obtaining the computational characteristic parameters of the target hardware corresponding to the network traffic detection scenario, and using the computational characteristic parameters to determine the initial quantization models corresponding to the first model and the second model, includes: Obtain the target hardware corresponding to the network traffic detection scenario, and determine the computing characteristic parameters based on the performance parameters of the target hardware; Based on the computational characteristic parameters, the first setting parameters corresponding to the first model and the second setting parameters corresponding to the second model are set; wherein, under the first setting parameters, the first model is used to guide the second model; under the second setting parameters, the second model adopts the EfficientNet-Lite model structure; The initial quantization model is constructed using the first setting parameters and the second setting parameters.

2. The network traffic detection model distillation method according to claim 1, characterized in that, The steps of constructing a first model based on attribute parameters corresponding to the network traffic detection scenario, and constructing a second model based on the structural parameters of the first model, include: Obtain the traffic parameters, scenario parameters, network parameters, and detection parameters corresponding to the network traffic detection scenario; determine the attribute parameters corresponding to the network traffic detection scenario based on the traffic parameters, scenario parameters, network parameters, and detection parameters; Based on the attribute parameters, the first model corresponding to the network traffic detection scenario is constructed according to single-precision parameters; The layer structure corresponding to the second model is determined based on the structural parameters of the first model, and the second model is constructed based on the layer structure.

3. The network traffic detection model distillation method according to claim 1, characterized in that, Determining the quantization and dequantization nodes between the first and second models based on the structural parameters of the first and second models includes: Obtain the structural parameters of the first model and the second model, and initialize the pseudo-quantization modules corresponding to the first model and the second model according to the structural parameters; Determine the quantization position parameters corresponding to the pseudo-quantization module, and determine the dequantization position parameters corresponding to the pseudo-quantization module based on the convolution parameters of the first model and the second model; The quantization node is determined based on the quantization position parameters, and the dequantization node is determined based on the dequantization position parameters.

4. The network traffic detection model distillation method according to claim 1, characterized in that, Determining the quantization parameters of the first model and the second model based on the quantization node and the inverse quantization node includes: The quantization effect of the first model and the second model during training is determined based on the quantization node and the dequantization node. Based on the quantization effect, determine the symmetry parameters and readout parameters corresponding to the first model and the second model; The quantization parameters of the first model and the second model are determined using the symmetry parameters and the reading parameters.

5. The network traffic detection model distillation method according to claim 1, characterized in that, Determining the distillation loss function corresponding to the initial quantization model using the quantization parameters includes: The first final layer output value of the first model and the second final layer output value of the second model are determined using the quantization parameters, and the initial distillation loss function corresponding to the initial quantization model is constructed based on the first final layer output value and the second final layer output value. The first feature output value of the first model and the second feature output value of the second model are determined using the quantization parameters, and the feature loss function corresponding to the initial quantization model is constructed based on the first feature output value and the second feature output value. The distillation loss function corresponding to the initial quantization model is constructed based on the initial distillation loss function and the feature loss function.

6. The network traffic detection model distillation method according to claim 1, characterized in that, Based on the propagation parameters corresponding to the initial quantization model, the backpropagation strategy of the second model is determined, and the training strategy corresponding to the initial quantization model is determined using the distillation loss function and the backpropagation strategy, including: Based on the propagation parameters corresponding to the initial quantization model, the inference strategy corresponding to the first model is determined, and the forward propagation strategy corresponding to the second model is determined according to the inference strategy. Based on the forward propagation strategy and using the distillation loss function, the loss function calculation strategy corresponding to the initial quantization model is determined. Based on the propagation parameters and using the forward propagation strategy, determine the back propagation strategy corresponding to the second model; Based on the adversarial parameters corresponding to the initial quantization model, construct the abnormal traffic corresponding to the network traffic detection scenario, and determine the adversarial strategy corresponding to the initial quantization model after performing feature distillation on the abnormal traffic. The training strategy corresponding to the initial quantization model is determined based on the forward propagation strategy, the loss function calculation strategy, the backpropagation strategy, and the adversarial strategy.

7. The network traffic detection model distillation method according to claim 1, characterized in that, The step of obtaining the quantization model corresponding to the network traffic detection scenario after training the initial quantization model using the training strategy and according to a preset training round includes: The training strategy is used to determine the learning rate parameter, temperature parameter, scaling factor parameter, and zero-point parameter corresponding to the initial quantization model; The initial quantization model is trained based on the learning rate parameter, the temperature parameter, the scaling factor parameter, and the zero point parameter, and the number of training iterations corresponding to the initial quantization model is obtained in real time. If the number of training iterations is less than the preset number of training rounds, the initial quantization model is controlled to continue training; if the number of training iterations is equal to the preset number of training rounds, the initial quantization model is controlled to stop training, and the current initial quantization model is determined as the quantization model for the network traffic detection scenario.

8. A network traffic detection model distillation system, characterized in that, The system includes: The model building module is used to build a first model based on the attribute parameters corresponding to the network traffic detection scenario, and to build a second model based on the structural parameters of the first model. The quantization parameter determination module is used to determine the quantization node and the dequantization node between the first model and the second model based on the structural parameters of the first model and the second model, and to determine the quantization parameters of the first model and the second model based on the quantization node and the dequantization node. The quantization model initialization module is used to obtain the computational characteristic parameters of the target hardware corresponding to the network traffic detection scenario, and use the computational characteristic parameters to determine the initial quantization model corresponding to the first model and the second model. The quantization model training distillation module is used to determine the initial loss function corresponding to the initial quantization model through the quantization parameters, determine the backpropagation strategy of the second model based on the propagation parameters corresponding to the initial quantization model, and determine the training strategy corresponding to the initial quantization model using the initial loss function and the backpropagation strategy. The quantization model generation module is used to train the initial quantization model using the training strategy and according to a preset training round to obtain the quantization model corresponding to the network traffic detection scenario. The quantization model initialization module is further configured to: acquire the target hardware corresponding to the network traffic detection scenario; determine the computational characteristic parameters based on the performance parameters of the target hardware; set a first setting parameter corresponding to the first model and a second setting parameter corresponding to the second model based on the computational characteristic parameters; wherein, under the first setting parameter, the first model is used to guide the second model; under the second setting parameter, the second model adopts the EfficientNet-Lite model structure; and construct the initial quantization model using the first setting parameter and the second setting parameter.

9. A server, characterized in that, The device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the steps of the network traffic detection model distillation method according to any one of claims 1 to 7.

Images