Network anomaly rapid detection method based on knowledge distillation framework

Abstract

The invention belongs to the technical field of network anomaly detection, and aims to provide a network anomaly rapid detection method based on a knowledge distillation framework. According to the invention, the detection speed can be improved, the problem of single knowledge source in knowledge distillation is solved, and the detection accuracy is high. Specifically, knowledge in a teacher model with a large parameter quantity and a large number of layers is distilled into a student model with a small parameter quantity and a small number of layers through a knowledge distillation method, and the trained student model is used for network anomaly detection. According to the method, the problem that a deep neural network with a large parameter quantity is used for network anomaly detection is avoided, and the detection speed is higher; besides, a multi-student model structure is introduced on the basis of response-based knowledge distillation, so that the dependence of the student model on a teacher model is relieved, the problem of single knowledge source of the student model in knowledge distillation is solved, and the network anomaly detection accuracy of the trained student model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network anomaly detection, and specifically relates to a network anomaly rapid detection method based on a knowledge distillation framework. Background Art

[0002] Network anomaly detection plays a vital role in modern network security. As network attacks become increasingly sophisticated, timely detection and response to potential abnormal behaviors are essential to ensure the security of information systems. Anomaly detection can help identify problems including malicious attacks, data leaks, system failures, and abnormal user behavior, and prevent data loss, service interruptions, and other serious security incidents. Through effective network anomaly detection, organizations can detect potential threats at an early stage, take necessary preventive measures, reduce security risks, and improve system stability and reliability.

[0003] In 2014, Sakurada et al. proposed a machine learning-based method for network anomaly detection in the paper "Anomaly detection using autoencoders with nonlinear dimensionality reduction" (2014, Proceedings of the MLSDA 2014 2nd workshop on machine learning for sensory data analysis, pp. 4-11; Sakurada M, Yairi T. Anomaly detection using autoencoders with nonlinear dimensionality reduction [C]. Proceedings of the MLSDA 2014 2nd workshop on machine learning for sensory data analysis. 2014: 4-11.), which models the complex correlation of multivariate time series in a simple linear way. With the booming development of deep learning and its excellent nonlinear fitting ability, in 2019, Su et al. proposed anomaly detection of multivariate time series based on recurrent neural network in the paper "Robust anomaly detection for multivariate time series through stochastic recurrent neural network". However, the above scheme has not yet explicitly modeled the complex spatial dependencies between different network devices.

[0004] In 2019, the authors Ngo et al. found in the paper "Fence GAN: Towards better anomaly detection" (2019, IEEE 31St International Conference on tools with artificial intelligence, pp. 141-148; Ngo PC, Winarto AA, Kou CKL, et al. Fence GAN: Towards better anomaly detection [C]. 2019 IEEE 31St International Conference on tools with artificial intelligence (ICTAI). IEEE, 2019: 141-148.) that the traditional GAN ​​(Generative Adversarial Networks) loss is not directly consistent with the anomaly detection goal, and made a simple modification to the GAN loss. However, the training process of GAN is relatively complex, and many hyperparameters need to be carefully adjusted, such as learning rate, number of network layers, etc. In addition, GAN tends to be unstable, and mode collapse or mode collapse may occur in the early stages of training. At the same time, the GAN network needs to run the generator in full and possible additional calculations, which cannot meet the faster detection speed.

[0005] In 2021, Henry Clausen et al. proposed CBAM, a contextual model for network anomaly detection based on a deep bidirectional LSTM network, in the paper "CBAM: A Contextual Model for Network Anomaly Detection" (Computers, Vol. 10, p. 79; Clausen H, Grov G, Aspinall D. Cbam: A contextual model for network anomaly detection [J]. Computers, 2021, 10(6): 79.). CBAM learns short-term sequence patterns in network traffic, maps traffic events to a discrete state set based on protocol, port, direction, and size, and trains in a self-supervised manner without labeling attack traffic to reduce the false positive rate and maintain robustness to concept drift. However, the spatial attention mechanism used in CBAM mainly captures local spatial information through two-dimensional convolution, and the feature extraction of the spatial dimension is limited. In addition, the attention mechanism needs to introduce additional attention parameters, which increases the computational overhead.

[0006] In 2022, Chen Aiguo et al. proposed an efficient network anomaly detection algorithm based on deep belief network (DBN) and long short-term memory network (LSTM) in "An efficient network behavior anomaly detection using a hybrid DBN-LSTM network" (Computers&Security, vol. 114, p. 102600). The algorithm automatically extracts features and reduces data dimensions through DBN, and then uses LSTM network to capture time series information to achieve classification of network behavior data. However, in this method, the author uses DBN to reduce the dimensionality of the data, and some data information will be lost during the dimensionality reduction process, which ultimately leads to poor performance of the model in the actual application environment. In addition, DBN is sensitive to the distribution and scale of the input data. If the distribution of the input data changes or lacks diversity, it may affect the performance of DBN and cause the learned features to fail.

[0007] In 2023, Gabriele Gemmi et al. proposed four machine learning methods based on different principles for network anomaly detection in "Anomaly detection for fault detection in wireless community networks using machine learning" (Computer Communications, Volume 202, Pages 191-203; Cerdà-Alabern L, Iuhasz G, Gemmi G. Anomaly detection for fault detection in wireless community networks using machine learning [J]. Computer Communications, 2023, 202: 191-203.): principal component analysis (PCA), isolation forest (Isolation Forest), clustering-based local anomaly factor (CBLOF) and variational autoencoder (VAE). Among them, VAE performs best, and for specific networks, the use of specific feature selection methods can improve the detection accuracy. However, the computational cost of VAE is high, and manually taking different feature selection methods for different networks will greatly increase the overhead of network anomaly detection, and it cannot be widely used and deployed.

[0008] In 2024, the authors Zhou Xiuzhuang et al. proposed a self-supervised network anomaly detection network RealNet in the paper "RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection" (2024, Proceedings of the IEEE / CVF conference on computer vision and pattern recognition CVPR, 16699--16708 pages; Zhang X, Xu M, Zhou X. Realnet: A feature selection network with realistic synthetic anomaly for anomaly detection [C]. Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2024: 16699-16708.), which improves the network anomaly detection performance by combining three component innovations: SDAS synthetic reality anomaly, AFS feature selection, and RRS residual selection. However, RealNet is a self-supervised model, and the effect of self-supervised learning is highly dependent on the designed pre-tasks, and RealNet's AFS feature selection faces the challenge of excessive computational overhead of high-dimensional data.

[0009] In recent years, deep neural networks have achieved remarkable success in the field of network anomaly detection, which is mainly attributed to their ability to process large-scale data sets and deal with unknown network anomalies. However, deploying these large-scale deep neural networks on resource-constrained devices (such as embedded systems) faces significant challenges, mainly due to the computational overhead of deep neural networks.

[0010] In order to solve the above problems, a variety of model compression and acceleration technologies have been developed. Among these technologies, knowledge distillation has attracted much attention due to its effectiveness in training smaller student models based on larger teacher models. It helps to solve the problems of high computational overhead and slow detection speed caused by the current network anomaly detection based on deep neural networks.

[0011] In knowledge distillation, the type of knowledge transferred plays a crucial role in the performance of the student model. Generally, this knowledge is divided into three types: 1) response-based knowledge; 2) feature-based knowledge; 3) relationship-based knowledge. Among them, response-based knowledge distillation focuses on the final output layer of the teacher model, aiming to train the student model to replicate the prediction results of the teacher model. Compared with feature-based knowledge distillation, response-based knowledge distillation has a stronger ability to capture global structural information, and compared with relationship-based knowledge distillation, the calculation process of response-based knowledge distillation is simpler. However, response-based knowledge distillation has the problem of a single source of knowledge and is highly dependent on the teacher model.

[0012] Therefore, it is necessary to propose a solution for rapid network anomaly detection based on the knowledge distillation framework to address the defects of traditional anomaly detection solutions such as high computational overhead and slow detection speed, while solving the problem of single knowledge source in existing knowledge distillation. Summary of the invention

[0013] The present invention aims to solve the above technical problems at least to a certain extent, and provides a method for rapid detection of network anomalies based on a knowledge distillation framework.

[0014] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for rapid detection of network anomalies based on a knowledge distillation framework, comprising: Get a teacher model and a student model set including multiple student models; Acquire a network anomaly detection data set; wherein the network anomaly detection data set includes multiple training samples; Randomly extract training samples from the network anomaly detection data set, and input the training samples into the teacher model and each student model of the student model set, so as to obtain prediction values ​​of the teacher model and each student model of the student model set for the training samples; According to the prediction values ​​of the teacher model and each student model in the student model set for the training sample, the loss function value of each student model in the student model set is calculated; The stochastic gradient value of each student model is calculated according to the loss function value of each student model, and each student model in the student model set is back-propagated and updated based on the stochastic gradient value of each student model to obtain an updated student model set including multiple updated student models; The student model set is updated to an updated student model set, and training samples are randomly extracted from the network anomaly detection data set again until the obtained loss function value is less than a preset threshold or reaches a preset maximum number of iterations, and the final updated student model set is output; The final updated student model set is used as the trained student model set; A network anomaly detection process is performed on the target network based on the trained student model set.

[0015] In one possible design, obtaining a teacher model includes: Acquire network data generated by multiple network devices, and adopt a dynamic graph construction method combined with a deep graph convolutional network model to learn the spatial dependency relationship between the network devices from the network data, and then construct a spatial modeling module matching the multiple network devices; wherein the network data includes time series data generated by each network device; Based on the time series data generated by each network device, the time dependency within a single time series data is extracted to generate the time feature representation of each network device, and then a time modeling module matching multiple network devices is constructed; Based on the spatial modeling module and the temporal modeling module, a comprehensive spatiotemporal feature representation is generated, and then a spatiotemporal fusion module matching multiple network devices is constructed; The spatial modeling module and the temporal modeling module are collaboratively trained through the spatiotemporal fusion module to obtain a teacher model.

[0016] In a possible design, a dynamic graph construction method is used in combination with a deep graph convolutional network model to learn the spatial dependency relationship between network devices from the network data, and then a spatial modeling module matching multiple network devices is constructed, including: Inputting the network data into a dynamic graph construction module so that the dynamic graph construction module updates the graph structure layer by layer based on a deep graph convolutional network model, thereby obtaining a graph structure between network devices and node features of the graph structure; Wherein, the graph structure is: ; In the formula, K represents the total number of layers of the deep graph convolutional network model, K is a natural number greater than 1, X K-1 Represents the first K -1 node features generated by the deep graph convolutional network layer, DGC ( ) represents a function in the dynamic graph construction module for determining whether there is a connection relationship between nodes,R n×n express n OK n A matrix of columns, n is the total number of network devices; The node characteristics of the graph structure are: X K ;in, X K Represents the first K Node features generated by deep graph convolutional network layers; Obtain node features of each deep graph convolutional network layer of the deep graph convolutional network model, and combine them into a node feature set; Obtaining a projection vector, and obtaining an implicit high-order dependency relationship between network devices based on the node feature set and the projection vector; According to the high-order dependency and the node feature set, a spatial correlation feature of deep coupling between network devices is obtained; A spatial modeling module matching multiple network devices is constructed based on the graph structure, the node features of the graph structure and the spatial correlation features.

[0017] In one possible design, the deep graph convolutional network model k The expression of the node features generated by the deep graph convolutional network layer is: ; In the formula, k ∈{2,3,……, K+ 1}, represents the normalized symmetric graph structure, , A k-1 Indicates the first k- 1 graph structure, D Indicates k- 1 Graph Structure A k-1 The degree matrix of I represents the identity matrix; The expression of the node feature set is: ; in, X 0 represents the preset initial node features of the deep graph convolutional network model, d Indicates the preset node feature dimension; The high-order dependencies are: ; In the formula, represents the high-order dependency, Reshape ( ) represents a function used to adjust the dimension of a matrix. represents the importance of each deep graph convolutional network layer, , c represents the projection vector, c ∈ R d×1 ; The spatial correlation characteristics are: ; In the formula, represents the spatial correlation feature, Represents the matrix multiplication symbol.

[0018] In a possible design, based on the time series data generated by each network device, the time dependency within a single time series data is extracted to generate the time feature representation of each network device, and then a time modeling module matching multiple network devices is constructed, including: Divide the time series data generated by each network device into multiple patches, and project the multiple patches of each network device into a high-dimensional space to obtain the high-dimensional time series data of each network device; According to the multiple patches of each network device, multiple new patches of each network device are obtained based on the self-attention module, and the new patches of each network device are combined and processed to obtain new network data of each network device; Perform batch normalization on the sum of the new network data of each network device and the high-dimensional time series data to obtain the initial time feature representation of each network device; A feedforward neural network is used to enhance the initial time feature representation of each network device to obtain an enhanced time feature representation of each network device; Reprojecting the enhanced temporal feature representation of each network device to the initial dimension of the network data to obtain the temporal feature representation of each network device; According to the time characteristic representation of each network device, a time modeling module matching multiple network devices is constructed.

[0019] In a possible design, the high-dimensional time series data of any network device is represented as: ; In the formula, Z ∈ R N×d , Z= ( z 1, z 2,…, z i ,…,z N ), z i Indicates the first of the plurality of patches of any network device i Patches, z i ∈ R d , N Indicates the total number of patches, d Represents the preset node feature dimension, Z ' represents multiple patches of any network device, Z '∈ R N×P , P represents the length of the patch, W Z represents the preset weight matrix, W Z ∈ R P×d ; According to multiple patches of any network device, a new patch of any network device is obtained based on a self-attention module, and the new patch of any network device is combined and processed to obtain new network data of any network device, including: Initialize relative position encoding parameters w pos = { w 1, w 2,…, w j-i ,…, w N-1};in, w j-i Indicates a patch for any network device i and patches j The index of w j-i ∈ R d ; The attention weights and normalized attention weights between the patches of any network device are calculated according to the relative position encoding parameters; wherein the patches of any network device i and patches j The attention weight between is: ; In the formula, Patches for any of the network devices i and patches j The attention weights between z jIndicates the first of the plurality of patches of any network device j Patches, z j ∈ R d , β Q and β K represents the preset weight matrix parameters, β Q ∈ R d×d , β K ∈ R d×d , a ij Indicates a patch for any network device i and patches j The degree of embedded attention, a ij =w j-i ; Any patch of the network device i and patches j The normalized attention weight between is: ; In the formula, Patches for any of the network devices i and patches j The normalized attention weights between μ ie Indicates a patch for any network device i and patches e The attention weight between Based on the multiple patches of any network device and the normalized attention weights between the patches, multiple new patches of any network device are calculated; wherein the first one of the multiple patches of any network device is the same as the first one. i The new patches corresponding to the patches are: ; In the formula, The first of the plurality of patches for any one of the network devices i The new patch corresponding to the patch, β V represents the preset weight matrix parameters, β V ∈ R d×d ; Combining and processing the multiple new patches of any network device to obtain new network data of any network device; wherein the new network data of any network device is represented as: Z attn =( z '1, z '2,…, z ' i ,…, z ' N ); In the formula, Z attn ∈ R N×d ; The initial time characteristic of any network device is expressed as: Y = BatchNorm ( Z + Z attn ); In the formula, Y is the initial time characteristic representation of any of the network devices, BatchNorm ( ) is the batch normalization function.

[0020] In a possible design, the enhanced time characteristic of any network device is expressed as: ; In the formula, is the enhanced temporal feature representation of any network device, BatchNorm ( ) is the batch normalization function, ; Y An initial time characteristic representation representing any one of the network devices; , Enhanced temporal representation of any network device, Relu ( ) represents the activation function, W 1 and W 2 represents the preset weight matrix, b 1 and b 2 represents the weight matrix W 1 and W 2 corresponding to the bias; The time characteristic of any network device is expressed as: ; In the formula, is a time characteristic representation of any of the network devices, Y ' is a feature representation obtained by flattening the enhanced temporal feature representation of any of the network devices, Y '∈R 1×(N×d) , W Linear represents the preset linear layer weight matrix, W Linear ∈ R (N×d)×d ; .

[0021] In a possible design, the spatiotemporal feature is expressed as: ; In the formula, is the spatiotemporal feature representation, , softmax ( ) represents the activation function, Y out represents the temporal modeling module, H out represents the spatial correlation feature in the spatial modeling module, W tp and W sp represents the preset weight parameter, W tp R d×d , W sp R d×d , T represents the transpose symbol, d Indicates the preset node feature dimension.

[0022] In one possible design, the loss function value of any student model is calculated using the cross entropy loss function and the KL divergence.

[0023] In one possible design, the first i The loss function value of the student model is: ; In the formula, is the first i The loss function value of the student model, a , b and c ' represents the preset training weight, K ' represents the total number of student models in the student model set, Indicates i The hard cross entropy loss of the student model, Indicates iThe soft cross entropy loss of the student model, Indicates i The predicted value of the student model, Represents the first k The predicted value of the student model, Indicates i The KL divergence of the student model.

[0024] In a second aspect, the present invention provides a network anomaly rapid detection system based on a knowledge distillation framework, comprising: An initial model acquisition module, used to acquire a teacher model and a student model set including multiple student models; A model training module, which is in communication with the initial model acquisition module and is used to acquire a network anomaly detection data set; wherein the network anomaly detection data set includes a plurality of training samples; The model training module is further used to randomly extract training samples from the network anomaly detection data set, and input the training samples into the teacher model and each student model of the student model set, so as to obtain the prediction values ​​of the teacher model and each student model of the student model set for the training samples; The model training module is further used to calculate the loss function value of each student model in the student model set according to the prediction value of the teacher model and each student model in the student model set for the training sample; The model training module is further used to calculate the stochastic gradient value of each student model according to the loss function value of each student model, and perform back-propagation update on each student model in the student model set based on the stochastic gradient value of each student model to obtain an updated student model set including multiple updated student models; The model training module is further used to update the student model set to an updated student model set, and randomly extract training samples from the network anomaly detection data set again until the obtained loss function value is less than a preset threshold or reaches a preset maximum number of iterations, and output the final updated student model set; The model training module is further used to use the final updated student model set as the trained student model set; The anomaly detection module is in communication with the model training module and is used to perform network anomaly detection processing on the target network based on the trained student model set.

[0025] In a third aspect, the present invention provides an electronic device, comprising: a memory for storing computer program instructions; and, A processor is used to execute the computer program instructions to complete the operation of a method for rapid detection of network anomalies based on a knowledge distillation framework as described in any one of the above.

[0026] In a fourth aspect, the present invention provides a computer program product, including a computer program or instructions, which, when executed by a computer, implements a method for rapid detection of network anomalies based on a knowledge distillation framework as described in any one of the above.

[0027] The beneficial effects of the present invention are: The present invention discloses a method for rapid detection of network anomalies based on a knowledge distillation framework, which can improve the detection speed, solve the problem of single knowledge source in knowledge distillation, and has high detection accuracy. Specifically, during the implementation of the present invention, the knowledge in the teacher model with large parameters and many layers is distilled into the student model with small parameters and few layers through the knowledge distillation method, and the trained student model is used for network anomaly detection. Compared with the traditional anomaly detection scheme, the present invention avoids the problem of using a deep neural network with a large number of parameters for network anomaly detection, and the detection speed is faster; in addition, the present invention introduces a multi-student model structure based on response-based knowledge distillation, which is conducive to alleviating the dependence of the student model on the teacher model, solves the problem of single knowledge source of the student model in knowledge distillation, and improves the accuracy of the trained student model in network anomaly detection.

[0028] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method for rapid network anomaly detection based on the knowledge distillation framework in Example 1; Figure 2 is a schematic diagram of the process of updating the graph structure in Example 1; Figure 3 is a schematic diagram of a process of training a student model set including two student models as exemplified in Embodiment 1; Figure 4 This is a module block diagram of the network anomaly rapid detection model based on the knowledge distillation framework in Example 2. DETAILED DESCRIPTION

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0031] Embodiment 1: This embodiment discloses a method for rapid detection of network anomalies based on a knowledge distillation framework, which can be executed by, but is not limited to, a computer device or a virtual machine with certain computing resources, such as a personal computer, a smart phone, a personal digital assistant, or a wearable device, or by a virtual machine.

[0032] like Figure 1 As shown, a method for rapid detection of network anomalies based on a knowledge distillation framework may include but is not limited to the following steps: S1. Obtain a teacher model and a student model set including multiple student models; it should be understood that, in this embodiment, the teacher model and the student model are both used for network anomaly detection, and the two can also be respectively referred to as a network anomaly detection teacher model and a network anomaly detection student model. To simplify the description, this article will refer to them as teacher models and student models.

[0033] It should be noted that, in this embodiment, the teacher model is a model with a large number of parameters and many layers, and the student model in the student model set is a model with a small number of parameters and few layers, such as a shallow convolutional neural network (CNN) and a lightweight Transformer model; in addition, in this embodiment, the structures of multiple student models can be the same or different, depending on the design goals and requirements. For example, when used to improve the robustness of the student model or to improve accuracy through integrated prediction, multiple student models with the same structure can be used. If the goal is to take into account diversity (different student models are good at processing different types of inputs or specific features), then multiple student models can be designed into different architectures. In this embodiment, multiple student models use student models of different types or with different degrees of pre-training.

[0034] In this embodiment, before training the student model set, multiple student models in the student model set are initialized to different initial conditions in advance, even if the multiple student models use different initial parameters, so as to avoid multiple student models falling into the same local optimal solution. At the same time, it can help increase the prediction diversity of multiple student models and the prediction performance in the application stage.

[0035] In step S1, the teacher model adopts a dynamic deep graph convolution and enhanced Transformer model (Dynamic Deep Graph Convolution with Enhanced Transformer Networks, DDGCT). Correspondingly, the teacher model is obtained, including: S101. Obtain network data generated by multiple network devices, and adopt a dynamic graph construction method, combined with a deep graph convolutional network model, to learn the spatial dependency relationship between each network device from the network data, and then construct a spatial modeling module that matches the multiple network devices; wherein the network data includes time series data generated by each network device. It should be understood that network devices include servers, routers, switches, and personal computers, etc., which are not limited here.

[0036] It should be noted that there are complex and nonlinear topological connections between network devices in the network, which lack effective representation. For this reason, this embodiment proposes a new dynamic graph construction method to learn the dependency relationship between network devices through a data-driven method. Specifically, in step S101, a dynamic graph construction method is used, combined with a deep graph convolutional network model, to learn the spatial dependency relationship between each network device from the network data, and then construct a spatial modeling module that matches multiple network devices, including: S1011. Input the network data into a dynamic graph construction (DGC) module, so that the dynamic graph construction module updates the graph structure layer by layer based on a deep graph convolutional network model, and then obtains the graph structure between network devices and the node features of the graph structure.

[0037] Specifically, in this embodiment, the network data is represented as ,in l and n Respectively represent the length of the input time series and the number of network devices. After obtaining the network data, the network data is standardized in advance through the data preprocessing step, and the length is A sliding window is used to generate fixed-length network data X '= .

[0038] Figure 2 It is a flowchart for updating the graph structure. Figure 2The Dynamic Graph Construction (DGC) box in represents the dynamic graph construction module, which includes an input layer (Input layer) and multiple processing layers (DGC Layer 1, ..., DGC Layer K-1 and DGC Layer K); Figure 2 The Deep Graph Convolution Network (DGCN) box in Figure 1 represents a deep graph convolution network model, which includes K deep graph convolution network layers (DGCN Layer 1, ..., DGCN Layer K).

[0039] like Figure 2 As shown, in this embodiment, the dynamic graph construction module updates the graph structure layer by layer based on the deep graph convolutional network model, including: the dynamic graph construction module updates the graph structure layer by layer based on the network data X 0 Get the first graph structure between network devices A 1 , and the first graph structure A 1 Input the first deep graph convolutional network layer (i.e. Figure 2 DGCN Layer 1 in ), so that the first deep graph convolutional network layer generates the first node feature X 1 ; The dynamic graph construction module is based on the first node feature X 1 Get the second graph structure between network devices A 2 , and the second graph structure A 2 Inputting a second deep graph convolutional network layer of the deep graph convolutional network model so that the second deep graph convolutional network layer generates a second node feature X 2 ; ...; The dynamic graph construction module is based on the K-2 node feature X K-2 Get the K-1th graph structure between network devices A K-1 , and the K-1th graph structure A K-1 Input the K-1th deep graph convolutional network layer of the deep graph convolutional network model so that the K-1th deep graph convolutional network layer generates the K-1th node feature X K-1 ; The dynamic graph construction module is based on the K-1 node feature X K-1 Get the Kth graph structure between network devices AK , and the Kth graph structure A K Input the Kth deep graph convolutional network layer (i.e. Figure 2 DGCN Layer K in the Kth deep graph convolutional network layer generates the Kth node feature X K It should be understood that the structure of the Kth figure A K That is, the final graph structure between network devices, the Kth node feature X K That is, the node features of the final graph structure.

[0040] Wherein, the graph structure is: ; In the formula, K represents the total number of layers of the deep graph convolutional network model, K is a natural number greater than 1, X K-1 Represents the first K -1 node features generated by the deep graph convolutional network layer, DGC ( ) represents a function in the dynamic graph construction module for determining whether there is a connection relationship between nodes, R n×n express n OK n A matrix of columns, n The total number of network devices.

[0041] Specifically, in this embodiment, the function It is implemented using a multilayer perceptron (MLP), that is, a multilayer perceptron is used to calculate the neighborhood weights between nodes in the graph structure, and a threshold function is further used to determine whether there is a connection relationship between the nodes, thereby obtaining the final graph structure.

[0042] As an example, k -1 ( k ∈{2,3,……, K+ 1}) Node features generated by deep graph convolutional network layers X K-1 As the query matrix and key matrix, the function DGC ( ) can be calculated to get the k Nodes in the graph structure and (Network equipment and ), the adjacency weight between them is calculated as follows: ; In the formula, Sigmoid ( ) represents the Sigmoid activation function, Relu ( ) represents the Relu activation function, and Respectively represent the first k -1 nodes generated by the deep graph convolutional network layer and The node characteristics of and Indicates k The weight matrix of the deep graph convolutional network layer is the default value. d Represents the preset node feature dimension, x represents the dimension of the hidden layer in the MLP, T Represents the transpose symbol.

[0043] According to k The adjacency weights between nodes in the graph structure can be obtained by k Graph Structure A k , which is an adjacency matrix, such as Figure 2 Specifically, in this embodiment, according to the obtained adjacency weights , and further use a differentiable threshold function to determine whether there is a connection relationship between network devices. Specifically, the adjacency weight is normalized by a factor that is almost equal to itself but has no gradient, as shown below: ; In the formula, Indicates that during the training phase, the adjacency weights The gradient of separation, The default minimum value is set to a very small value to prevent overflow.

[0044] In this embodiment, the adjacency matrix A k Elements in The calculation method is as follows: ; In the formula, μ is a preset threshold value, which is set to 0.5 in this embodiment.

[0045] In addition, in this embodiment, a Hard Concrete distribution is introduced. The norm retains the key connection relationship between nodes and prunes the model by forcing the weight to zero during training to minimize the redundant dependencies between network devices in the graph structure, while preventing the introduction of unnecessary information and overfitting of the model. The expression for the norm is as follows: ; The implementation of this differentiable threshold function can regularize the gradient to a certain extent, thus enabling the dynamic graph building module to fine-tune the connection relationship between network devices.

[0046] Therefore, in this embodiment, the dynamic graph construction module can adaptively construct a graph structure between network devices so as to perform a feature update step between network devices based on the graph structure.

[0047] In this embodiment, the node features of the graph structure are: X K ;in, X K Represents the first K Node features generated by deep graph convolutional network layers.

[0048] S1012. Obtain node features of each deep graph convolutional network layer of the deep graph convolutional network model, and combine them into a node feature set.

[0049] The traditional graph convolutional network model realizes information aggregation between nodes by iteratively performing propagation and transformation. k The expression of the node features generated by the graph convolutional network layer is: ; In the formula, k ∈{2,3,……, K+ 1}, represents the normalized symmetric graph structure, , D' Represents the adjacency matrix A' k-1 The degree matrix of represents the identity matrix, Represents the first k- 1 Node features generated by the graph convolutional network layer, Represents the traditional graph convolutional network model k- 1. Trainable weight matrices of graph convolutional network layers.

[0050] For the deep graph convolutional network model proposed in this embodiment, the deep graph convolutional network model is kThe expression of the node features generated by the deep graph convolutional network layer is: ; In the formula, k ∈{2,3,……, K+ 1}, represents the normalized symmetric graph structure, , A k-1 Indicates the first k- 1 graph structure, D Indicates k- 1 Graph Structure A k-1 The degree matrix of I represents the identity matrix; Due to the entanglement of graph convolution transformation and propagation, the performance of graph convolutional network models is affected. k Compared with the expression of node features generated by the graph convolutional network layer, the deep graph convolutional network model in this embodiment has no learnable weight matrix .

[0051] Correspondingly, after After layer iteration, the node features of each deep graph convolutional network layer of the deep graph convolutional network model can be obtained, and a node feature set can be obtained based on this. The expression of the node feature set is: ; in, X 0 represents the preset initial node features of the deep graph convolutional network model, d represents the preset node feature dimension; it is obvious that the node feature set Contains node information from lower layers and deeper layers.

[0052] S1013. Obtain a projection vector, and obtain an implicit high-order dependency relationship between network devices based on the node feature set and the projection vector.

[0053] In this embodiment, a trainable projection vector is used c ∈ R d×1 Adaptively determine which layers of the deep graph convolutional network model can better capture the information of node features. In this way, the implicit high-order important dependencies between network devices can be more effectively captured. Specifically, in step S1013 of this embodiment, the high-order dependencies are: ; In the formula, represents the high-order dependency, Reshape ( ) represents a function used to adjust the matrix dimension. Reshape from a three-dimensional tensor to a four-dimensional tensor for the next step of calculation; Indicates the importance of each deep graph convolutional network layer, which is a value between 0 and 1. , c represents the projection vector, c ∈ R d×1 , The weighted sum of the features of each node at different depth graph convolutional network layers reflects the contribution of each layer to the node feature representation.

[0054] S1014. Obtain spatial correlation features of deep coupling between network devices based on the high-order dependency and the node feature set.

[0055] In step S1014, the spatial correlation feature is: ; In the formula, represents the spatial correlation feature, Represents the matrix multiplication symbol.

[0056] It should be noted that in this embodiment, after the network device graph structure is constructed, information propagation between network devices is performed by using a deep graph convolutional network model. By adopting a deep graph convolutional network model, the over-smoothing problem can be effectively alleviated, and the parameters of the traditional graph convolutional network model can be reduced to prevent overfitting.

[0057] S1015. Construct a spatial modeling module matching multiple network devices according to the graph structure, the node features of the graph structure and the spatial correlation features.

[0058] S102. Based on the time series data generated by each network device, the time dependency within a single time series data is extracted, and a time feature representation of each network device is generated, thereby constructing a time modeling module that matches multiple network devices.

[0059] In step S102, based on the time series data generated by each network device, the time dependency within the single time series data is extracted to generate the time feature representation of each network device, and then a time modeling module matching multiple network devices is constructed, including: S1021. Divide the time series data generated by each network device into multiple patches, and project the multiple patches of each network device into a high-dimensional space to obtain the high-dimensional time series data of each network device.

[0060] In step S1021, the time series data generated by any network device is After patch division, the obtained data can be expressed as: ;in, N Indicates the total number of patches, P Represents the length of the patch, which is calculated as follows: ; In the formula, Indicates the floor symbol; If the length of the time series data sequence is not divisible by the length of the patch, the last value at the end of the sequence will be padded before the patch division. S times to maintain consistent length, S represents the step length, S is a natural number greater than 0. This approach ensures that the length of the patch is consistent when it is divided and prevents information loss. S Equal to the length of the patch P , then the divided patches are non-overlapping. If the step size S and the length of the patch P If they are not equal, then the step length S Represents the non-overlapping area between two patches. By using patches, the ability of transformer to capture comprehensive semantic information in the sequence can be significantly improved, thereby effectively modeling the long-term dependencies of time series data.

[0061] Furthermore, in step S1021, the high-dimensional time series data of any network device is expressed as: ; In the formula, Z ∈ R N×d , Z= ( z 1, z 2,…, z i ,…, z N ), z i Indicates the first of the plurality of patches of any network device i Patches, z i ∈ R d , N Indicates the total number of patches, d Represents the preset node feature dimension, Z ' represents multiple patches of any network device, Z '∈ R N×P ,P represents the length of the patch, W Z represents the preset trainable weight matrix, W Z ∈ R P×d .

[0062] S1022. According to the multiple patches of each network device, multiple new patches of each network device are respectively obtained based on the self-attention module, and the new patches of each network device are respectively combined and processed to obtain new network data of each network device.

[0063] Specifically, in step S1022, according to multiple patches of any network device, a new patch of any network device is obtained based on the self-attention module, and the new patch of any network device is combined and processed to obtain new network data of any network device, including: S10221. Initialize relative position encoding parameters w pos = { w 1, w 2,…, w j-i ,…, w N-1};in, w j-i Indicates a patch for any network device i and patches j The index of w j-i ∈ R d .

[0064] S10222. Calculate the attention weights and normalized attention weights between the patches of any network device according to the relative position encoding parameters; wherein the patches of any network device and patches The attention weight between is: ; In the formula, Patches for any of the network devices i and patches j The attention weights between z j Indicates the first of the plurality of patches of any network device j Patches, z j ∈ R d , β Qand β K Represents the preset trainable weight matrix parameters, β Q ∈ R d×d , β K ∈ R d×d , a ij Indicates a patch for any network device i and patches j The degree of embedded attention, a ij =w j-i ;in, and They represent the traditional attention for the interaction between query and key in the preset self-attention mechanism, and They represent the relative position metrics of query and key in the preset self-attention mechanism respectively.

[0065] Any patch of the network device and patches The normalized attention weight between is: ; In the formula, Patches for any of the network devices i and patches j The normalized attention weights between μ ie Indicates a patch for any network device i and patches e The attention weight between S10223. Based on the multiple patches of any network device and the normalized attention weights between the patches, multiple new patches of any network device are calculated; specifically, in this embodiment, the normalized attention weights are applied to weight and sum each patch to calculate multiple new patches of any network device; wherein, the first one of the multiple patches of any network device is the same as the first one of the multiple patches of any network device. i The new patches corresponding to the patches are: ; In the formula, The first of the plurality of patches for any one of the network devices i The new patch corresponding to the patch, β V Represents the preset trainable weight matrix parameters, β V ∈ Rd×d .

[0066] S10224. Combine and process the multiple new patches of any network device to obtain new network data of any network device; wherein the new network data of any network device is represented as: Z attn =( z '1, z '2,…, z ' i ,…, z ' N ); In the formula, Z attn ∈ R N×d .

[0067] Based on step S1022 of this embodiment, a new self-attention mechanism is developed, which combines relative position encoding and increases the interaction between the relative position encodings of each query, key, and value in the self-attention mechanism, which can more accurately simulate the temporal order relationship of time series data.

[0068] S1023. Perform batch normalization processing on the sum of the new network data of each network device and the high-dimensional time series data to obtain the initial time feature representation of each network device.

[0069] Specifically, in step S1023, the initial time characteristic of any network device is expressed as: Y = BatchNorm ( Z + Z attn ); In the formula, Y is the initial time characteristic representation of any of the network devices, BatchNorm ( ) is the batch normalization function.

[0070] It should be noted that in step S1023 of this embodiment, batch normalization is used instead of layer normalization to reduce the impact of outliers. At the same time, residual connections (high-dimensional time series) are introduced. Z attn ), which prevents gradient vanishing and thus learns deeper dependencies among network devices.

[0071] S1024. Use a feedforward neural network to enhance the initial time feature representation of each network device to obtain an enhanced time feature representation of each network device.

[0072] Specifically, in step S1024, the enhanced time characteristic of any network device is expressed as: ; In the formula, is the enhanced temporal feature representation of any network device, BatchNorm ( ) is the batch normalization function, ; Y An initial time characteristic representation representing any one of the network devices; , Enhanced temporal representation of any network device, Relu ( ) represents the activation function, W 1 and W 2 represents the preset weight matrix, b 1 and b 2 represents the weight matrix W 1 and W 2 corresponds to the offset.

[0073] S1025. Reproject the enhanced temporal feature representation of each network device to the initial dimension of the network data to obtain the temporal feature representation of each network device. Specifically, in this embodiment, a linear layer is used to reproject the enhanced temporal feature representation of each network device to the same dimension as the input to obtain a univariate output.

[0074] Specifically, in step S1025, the time characteristic of any network device is expressed as: ; In the formula, is a time characteristic representation of any of the network devices, Y ' is a feature representation obtained by flattening the enhanced temporal feature representation of any of the network devices, Y '∈ R 1×(N×d) , W Linear represents the preset linear layer weight matrix, which is used to map the enhanced temporal feature representation to the target dimension. W Linear ∈ R (N×d)×d ; .

[0075] S1026. Based on the time feature representation of each network device, a time modeling module matching multiple network devices is constructed.

[0076] In this embodiment, the time dependency relationship of each network device can be captured through the above steps S1021-S1026, and a multivariate output can be obtained by connecting n single variables together. Yout ∈ R N×d , used to characterize a temporal modeling module that matches multiple network devices.

[0077] S103. Based on the spatial modeling module and the temporal modeling module, a comprehensive spatiotemporal feature representation is generated, and then a spatiotemporal fusion module matching multiple network devices is constructed; it should be noted that the spatiotemporal feature representation is used to connect the dependencies captured by the spatial modeling module and the temporal modeling module, and the spatiotemporal fusion module can serve as a bridge connecting the spatial modeling module and the temporal modeling module.

[0078] Specifically, in step S103, the spatiotemporal feature is expressed as: ; In the formula, is the spatiotemporal feature representation, , which represents the “spatial-temporal attention”, a mechanism for measuring the impact of different time steps on each time series; softmax ( ) represents the activation function, Y out represents the temporal modeling module, H out represents the spatial correlation feature in the spatial modeling module, W tp and W sp represents the preset weight parameter, W tp R d×d , W sp R d×d , T represents the transpose symbol, d Indicates the preset node feature dimension.

[0079] S104. The spatial modeling module and the temporal modeling module are collaboratively trained through the spatiotemporal fusion module to obtain a teacher model.

[0080] In step S104 of this embodiment, the spatiotemporal feature representation It is used as the input of a hybrid spatiotemporal neural network model that integrates the spatial modeling module and the temporal modeling module. The model consists of fully connected layers and finally produces the prediction results. During the collaborative training process, the loss function is expressed as: ; That is, this embodiment uses the predicted value and the true value g t In timestamp t The root mean square deviation and norm on l 0 is used as the loss function. Based on the expression of the loss function, the parameters of the hybrid spatiotemporal neural network model can be further optimized using the stochastic gradient descent and back propagation algorithms, and its parameters are updated through the Adam optimizer to obtain the final teacher model.

[0081] S2. Obtain a network anomaly detection data set; wherein the network anomaly detection data set includes multiple training samples, and each training sample includes a target network and a label matching the target network.

[0082] S3. randomly extract training samples from the network anomaly detection data set, and input the training samples into the teacher model and each student model of the student model set respectively, so as to obtain the prediction values ​​of the teacher model and each student model of the student model set for the training samples respectively.

[0083] S4. According to the prediction values ​​of the teacher model and each student model in the student model set for the training samples, the loss function value of each student model in the student model set is calculated.

[0084] In step S4 of this embodiment, the loss function value of any student model is calculated using a cross entropy loss function and a KL divergence (Kullback-Leibler Divergence).

[0085] Specifically, in step S4, the first i The loss function value of the student model is: ; In the formula, is the first i The loss function value of the student model, a , b and c ' represents the preset training weight, K ' represents the total number of student models in the student model set, where, is a preset training coefficient to ensure that the training is mainly guided by supervised learning of true labels and soft predictions of the teacher model; Indicates i The hard cross entropy loss of the student model, Indicates i The soft cross entropy loss of the student model, Indicates i The predicted value of the student model, Represents the first k The predicted value of the student model, Indicates i The KL divergence of the student model.

[0086] In this embodiment, KL divergence is introduced and used as a part of the loss function, which is helpful to reduce the deviation between student models.

[0087] S5. Calculate the stochastic gradient value of each student model according to the loss function value of each student model, and perform back propagation update on each student model in the student model set based on the stochastic gradient value of each student model. For example, the parameters in the student model , γ is the preset learning rate, which is used to control the update step size. i ∈{0,1,…, N}, and obtain an updated student model set including multiple updated student models.

[0088] S6. Update the student model set to an updated student model set, and randomly extract training samples from the network anomaly detection dataset again, that is, return to step S3 for iterative training, until the obtained loss function value is less than a preset threshold or reaches a preset maximum number of iterations, and output the final updated student model set, which is also the updated student model set obtained in the last iteration.

[0089] S7. Use the final updated student model set as the trained student model set.

[0090] S8. Perform network anomaly detection processing on the target network based on the trained student model set. Specifically, in this embodiment, during the anomaly detection process, the predicted values ​​of multiple trained student models in the trained student model set can be combined to obtain a comprehensive detection result through average fusion, weighted fusion or voting mechanism, which is not limited here.

[0091] As an example, Figure 3 As shown, when the student model set includes two student models, assuming that the training sample contains Sub-objects , and the training samples may belong to M categories, then the corresponding N The label corresponding to the sub-object is recorded as ,in At this time, firstly, the training samples Use a tokenizer tool such as Huggingface (a machine learning platform that provides datasets, pre-trained models, and courses) to split the word to obtain its corresponding token , that is Figure 3 These tokens are then fed into the teacher model α , student model β 1 and β 2. Accordingly, the predicted values ​​of the outputs of these three models can be obtained Z : ; in It is a model α , β 1 and β 2 corresponds to the predicted value of the output (combined Figure 3 ), F t , F 1 and F 2 represents the calculation function of the three models. Then, the training sample X Belongs to category m The probability can be calculated as: ; ; In the formula, , T 'Indicates the preset distillation temperature; Represents the teacher model α For training samples X Classify and get the category it belongs to m probability; Represents the student model β i For training samples X Classify and get the category it belongs to m In this way, we can get the predicted value of the student model ,in .

[0092] To calculate the two student model predictions and The difference between the soft prediction values ​​is also calculated. and The KL divergence between them.

[0093] At this point, for the first student model β 1. Its loss function contains three loss terms: , and ,in, represents the hard cross entropy loss of the first student model, ; represents the soft cross entropy loss of the first student model, ; is the KL divergence, ; Correspondingly, the loss function value of the first student model is: .

[0094] Figure 3 In, with the student model β 1 and β 2 The corresponding loss functions are respectively used Loss 1 and Loss 2 means that after multiple iterations of training, the trained student model set can be finally obtained, that is, the trained student model β 1 and β 2 is the set of trained student models.

[0095] This embodiment can improve the detection speed, solve the problem of single knowledge source in knowledge distillation, and has high detection accuracy. Specifically, during the implementation of this embodiment, the knowledge in the teacher model with large parameters and many layers is distilled into the student model with small parameters and few layers through the knowledge distillation method, and the trained student model is used for network anomaly detection. Compared with the traditional anomaly detection scheme, this embodiment avoids the problem of using deep neural networks with large parameters for network anomaly detection, and the detection speed is faster; in addition, this embodiment introduces a multi-student model structure based on response-based knowledge distillation, which is conducive to alleviating the dependence of the student model on the teacher model, solves the problem of single knowledge source of the student model in knowledge distillation, and improves the accuracy of network anomaly detection by the trained student model.

[0096] Embodiment 2: This embodiment discloses a network anomaly rapid detection system based on a knowledge distillation framework for implementing the network anomaly rapid detection method based on a knowledge distillation framework in Embodiment 1; Figure 4 As shown, the network anomaly rapid detection system based on the knowledge distillation framework includes: An initial model acquisition module, used to acquire a teacher model and a student model set including multiple student models; A model training module, which is in communication with the initial model acquisition module and is used to acquire a network anomaly detection data set; wherein the network anomaly detection data set includes a plurality of training samples; The model training module is further used to randomly extract training samples from the network anomaly detection data set, and input the training samples into the teacher model and each student model of the student model set, so as to obtain the prediction values ​​of the teacher model and each student model of the student model set for the training samples; The model training module is further used to calculate the loss function value of each student model in the student model set according to the prediction value of the teacher model and each student model in the student model set for the training sample; The model training module is further used to calculate the stochastic gradient value of each student model according to the loss function value of each student model, and perform back-propagation update on each student model in the student model set based on the stochastic gradient value of each student model to obtain an updated student model set including multiple updated student models; The model training module is further used to update the student model set to an updated student model set, and randomly extract training samples from the network anomaly detection data set again until the obtained loss function value is less than a preset threshold or reaches a preset maximum number of iterations, and output the final updated student model set; The model training module is further used to use the final updated student model set as the trained student model set; The anomaly detection module is in communication with the model training module and is used to perform network anomaly detection processing on the target network based on the trained student model set.

[0097] It should be noted that the working process, working details and technical effects of the network anomaly rapid detection system based on the knowledge distillation framework provided in this embodiment 2 can be found in embodiment 1 and will not be repeated here.

[0098] Embodiment 3: Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smart phone, a tablet computer, a laptop computer, or a desktop computer, etc. The electronic device may be called a user terminal, a portable terminal, a desktop terminal, etc. The electronic device includes: a memory for storing computer program instructions; and, A processor is used to execute the computer program instructions to complete the operation of a network anomaly rapid detection method based on a knowledge distillation framework as described in any one of Example 1.

[0099] Embodiment 4: Based on any one of Embodiments 1 to 3, this embodiment discloses a computer program product, including a computer program or an instruction, which, when executed by a computer, implements a network anomaly rapid detection method based on a knowledge distillation framework as described in any one of Embodiments 1. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0100] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapid network anomaly detection based on a knowledge distillation framework, characterized in that: include: Get a teacher model and a student model set including multiple student models; Acquire a network anomaly detection data set; wherein the network anomaly detection data set includes multiple training samples; Randomly extract training samples from the network anomaly detection data set, and input the training samples into the teacher model and each student model of the student model set, so as to obtain prediction values ​​of the teacher model and each student model of the student model set for the training samples; According to the prediction values ​​of the teacher model and each student model in the student model set for the training sample, the loss function value of each student model in the student model set is calculated; The stochastic gradient value of each student model is calculated according to the loss function value of each student model, and each student model in the student model set is back-propagated and updated based on the stochastic gradient value of each student model to obtain an updated student model set including multiple updated student models; The student model set is updated to an updated student model set, and training samples are randomly extracted from the network anomaly detection data set again until the obtained loss function value is less than a preset threshold or reaches a preset maximum number of iterations, and the final updated student model set is output; The final updated student model set is used as the trained student model set; A network anomaly detection process is performed on the target network based on the trained student model set.

2. According to claim 1, a method for rapid network anomaly detection based on a knowledge distillation framework is characterized in that: Get the teacher model, including: Acquire network data generated by multiple network devices, and adopt a dynamic graph construction method combined with a deep graph convolutional network model to learn the spatial dependency relationship between the network devices from the network data, and then construct a spatial modeling module matching the multiple network devices; wherein the network data includes time series data generated by each network device; Based on the time series data generated by each network device, the time dependency within a single time series data is extracted to generate the time feature representation of each network device, and then a time modeling module matching multiple network devices is constructed; Based on the spatial modeling module and the temporal modeling module, a comprehensive spatiotemporal feature representation is generated, and then a spatiotemporal fusion module matching multiple network devices is constructed; The spatial modeling module and the temporal modeling module are collaboratively trained through the spatiotemporal fusion module to obtain a teacher model.

3. According to claim 2, a method for rapid detection of network anomalies based on a knowledge distillation framework is characterized in that: A dynamic graph construction method is used in combination with a deep graph convolutional network model to learn the spatial dependency relationship between network devices from the network data, and then a spatial modeling module matching multiple network devices is constructed, including: Inputting the network data into a dynamic graph construction module so that the dynamic graph construction module updates the graph structure layer by layer based on a deep graph convolutional network model, thereby obtaining a graph structure between network devices and node features of the graph structure; Wherein, the graph structure is: ； In the formula, K represents the total number of layers of the deep graph convolutional network model, K is a natural number greater than 1, X K-1 Represents the first K -1 node features generated by the deep graph convolutional network layer, DGC ( ) represents a function in the dynamic graph construction module for determining whether there is a connection relationship between nodes, R n×n express n OK n A matrix of columns, n is the total number of network devices; The node characteristics of the graph structure are: X K ;in, X K Represents the first K Node features generated by deep graph convolutional network layers; Obtain node features of each deep graph convolutional network layer of the deep graph convolutional network model, and combine them into a node feature set; Obtaining a projection vector, and obtaining an implicit high-order dependency relationship between network devices based on the node feature set and the projection vector; According to the high-order dependency and the node feature set, a spatial correlation feature of deep coupling between network devices is obtained; A spatial modeling module matching multiple network devices is constructed based on the graph structure, the node features of the graph structure and the spatial correlation features.

4. According to claim 3, a method for rapid detection of network anomalies based on a knowledge distillation framework is characterized in that: The deep graph convolutional network model k The expression of the node features generated by the deep graph convolutional network layer is: ； In the formula, k ∈{2,3,……, K+ 1}, represents the normalized symmetric graph structure, , A k-1 Indicates the first k- 1 graph structure, D Indicates k- 1 Graph Structure A k-1 The degree matrix of I represents the identity matrix; The expression of the node feature set is: ； in, X 0 represents the preset initial node features of the deep graph convolutional network model, d Indicates the preset node feature dimension; The high-order dependencies are: ； In the formula, represents the high-order dependency, Reshape ( ) represents a function used to adjust the dimension of a matrix. represents the importance of each deep graph convolutional network layer, , c represents the projection vector, c ∈ R d×1 ; The spatial correlation characteristics are: ； In the formula, represents the spatial correlation feature, Represents the matrix multiplication symbol.

5. According to claim 2, a method for rapid detection of network anomalies based on a knowledge distillation framework is characterized in that: Based on the time series data generated by each network device, the time dependency within a single time series data is extracted to generate the time feature representation of each network device, and then a time modeling module matching multiple network devices is constructed, including: Divide the time series data generated by each network device into multiple patches, and project the multiple patches of each network device into a high-dimensional space to obtain the high-dimensional time series data of each network device; According to the multiple patches of each network device, multiple new patches of each network device are obtained based on the self-attention module, and the new patches of each network device are combined and processed to obtain new network data of each network device; Perform batch normalization on the sum of the new network data of each network device and the high-dimensional time series data to obtain the initial time feature representation of each network device; A feedforward neural network is used to enhance the initial time feature representation of each network device to obtain an enhanced time feature representation of each network device; Reprojecting the enhanced temporal feature representation of each network device to the initial dimension of the network data to obtain the temporal feature representation of each network device; According to the time characteristic representation of each network device, a time modeling module matching multiple network devices is constructed.

6. According to claim 5, a method for rapid detection of network anomalies based on a knowledge distillation framework is characterized in that: The high-dimensional time series data of any network device is expressed as: ； In the formula, Z ∈ R N×d , Z= ( z 1, z 2,…, z i ,…, z N ), z i Indicates the first of the plurality of patches of any network device i Patches, z i ∈ R d , N Indicates the total number of patches, d Represents the preset node feature dimension, Z ' represents multiple patches of any network device, Z '∈ R N×P , P represents the length of the patch, W Z represents the preset weight matrix, W Z ∈ R P×d ; According to multiple patches of any network device, a new patch of any network device is obtained based on a self-attention module, and the new patch of any network device is combined and processed to obtain new network data of any network device, including: Initialize relative position encoding parameters w pos = { w 1, w 2,…, w j-i ,…, w N-1 };in, w j-i Indicates a patch for any network device i and patches j The index of w j-i ∈ R d ; The attention weights and normalized attention weights between the patches of any network device are calculated according to the relative position encoding parameters; wherein the patches of any network device i and patches j The attention weight between is: ； In the formula, Patches for any of the network devices i and patches j The attention weights between z j Indicates the first of the plurality of patches of any network device j Patches, z j ∈ R d , β Q and β K represents the preset weight matrix parameters, β Q ∈ R d×d , β K ∈ R d ×d , a ij Indicates a patch for any network device i and patches j The degree of embedded attention, a ij =w j-i ; Any patch of the network device i and patches j The normalized attention weight between is: ； In the formula, Patches for any of the network devices i and patches j The normalized attention weights between μ ie Indicates a patch for any network device i and patches e The attention weight between Based on the multiple patches of any network device and the normalized attention weights between the patches, multiple new patches of any network device are calculated; wherein the first one of the multiple patches of any network device is the same as the first one. i The new patches corresponding to the patches are: ； In the formula, The first of the plurality of patches for any one of the network devices i The new patch corresponding to the patch, β V represents the preset weight matrix parameters, β V ∈ R d×d ; Combining and processing the multiple new patches of any network device to obtain new network data of any network device; wherein the new network data of any network device is represented as: Z attn =( z '1, z '2,…, z ' i ,…, z ' N )； In the formula, Z attn ∈ R N×d ; The initial time characteristic of any network device is expressed as: Y = BatchNorm ( Z + Z attn )； In the formula, Y is the initial time characteristic representation of any of the network devices, BatchNorm ( ) is the batch normalization function.

7. According to claim 5, a method for rapid detection of network anomalies based on a knowledge distillation framework is characterized in that: The enhanced time characteristics of any network device are expressed as: ； In the formula, is the enhanced temporal feature representation of any network device, BatchNorm ( ) is the batch normalization function, ; Y An initial time characteristic representation representing any one of the network devices; , Enhanced temporal representation of any network device, Relu ( ) represents the activation function, W 1 and W 2 represents the preset weight matrix, b 1 and b 2 represents the weight matrix W 1 and W 2 corresponding to the bias; The time characteristic of any network device is expressed as: ； In the formula, is a time characteristic representation of any of the network devices, Y ' is a feature representation obtained by flattening the enhanced temporal feature representation of any of the network devices, Y '∈ R 1×(N×d) , W Linear represents the preset linear layer weight matrix, W Linear ∈ R (N×d)×d ; .

8. According to claim 2, a method for rapid detection of network anomalies based on a knowledge distillation framework is characterized in that: The spatiotemporal features are expressed as: ； In the formula, is the spatiotemporal feature representation, , softmax ( ) represents the activation function, Y out represents the temporal modeling module, H out represents the spatial correlation feature in the spatial modeling module, W tp and W sp Represents the preset weight parameter, W tp R d×d , W sp R d×d , T represents the transpose symbol, d Indicates the preset node feature dimension.

9. The method for rapid network anomaly detection based on a knowledge distillation framework according to claim 1 is characterized in that: The loss function value of any student model is calculated using the cross entropy loss function and KL divergence.

10. The method for rapid network anomaly detection based on the knowledge distillation framework according to claim 9 is characterized in that: The student model set i The loss function value of the student model is: ； In the formula, is the first i The loss function value of the student model, a , b and c ' represents the preset training weight, K ' represents the total number of student models in the student model set, Indicates i The hard cross entropy loss of the student model, Indicates i The soft cross entropy loss of the student model, Indicates i The predicted value of the student model, Represents the first k The predicted value of the student model, Indicates i The KL divergence of the student model.

Images