Lightweight meta-learning-based rapid and accurate network traffic space-time tensor completion method and system

Abstract

The invention discloses a lightweight rapid and accurate network traffic space-time tensor completion method and system based on meta-learning. Comprising the following steps: acquiring a data tensor, and dividing the data tensor into sub-tensors with the same size as input; initializing the current-stage embedding by utilizing the last-stage final embedding, and inputting the current-stage sub-tensor into a fixed parameter migration module to obtain the current-stage intermediate embedding; the embedding of the current stage is frozen, the sub-tensor of the current stage is input into the migration module again, and migration module parameters are updated by using a loss function; fusing the middle embedding of the current stage with the final embedding of the previous stage, and inputting the fused embedding into the migration module with the updated parameters to obtain the final embedding of the current stage; the sub-tensor of the current stage is finally embedded and recovered by using the current stage; and repeating the training until the complete data set tensor is recovered. According to the method, rapid convergence of newly arrived data is realized through meta-learning, historical information is saved, and the training speed and precision are improved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning of network traffic, and more specifically, relates to a lightweight, fast and accurate network traffic spatiotemporal tensor completion method and system based on meta-learning. Background Art

[0002] In today's information society, accurate prediction and analysis of network traffic are crucial to network management and security. As a monitoring tool, the traffic matrix records traffic data between nodes. In order to capture the time evolution of traffic, the study serializes it into a network traffic measurement tensor. There are two main methods to measure the traffic matrix: indirect inference by network tomography, but the performance is limited when there are few links and many nodes, and reasonable assumptions are required; direct measurement such as OpenTM has high accuracy, but faces challenges such as equipment support, high cost and transmission loss, resulting in incomplete data. In applications such as anomaly detection and fault recovery, a complete traffic matrix is ​​indispensable. The present invention focuses on the problem of traffic data recovery, and explores the inference of network-wide traffic data through measurements of partial nodes to improve network security and stability, reduce costs, and optimize resource allocation.

[0003] Traditional tensor completion methods are less effective in utilizing space and time. First, there is a large amount of topological information between nodes in the network. This spatial information can better utilize the relationship between nodes to aggregate the information of neighboring nodes. Secondly, in network traffic, data changes over time and may have similar patterns in the same time period on different dates. This shows that traffic data has temporal stability and periodicity, and the temporal correlation can be used to better restore the embedded information of other times. However, current tensor completion methods usually directly embed different patterns in the same way, ignoring the important spatial and temporal correlations.

[0004] Network traffic models need to process a large amount of new data every day, so how to deal with newly arrived data is an important criterion for model efficiency. The current methods to adapt the model to newly arrived data are: 1) Retrain all data. It can make full use of new data and historical data to restore missing entries and improve overall performance, but considering the increasing data, it may lead to long training time and high resource consumption. 2) Retrain on newly arrived data. Training only on newly arrived data is a solution with faster training speed and less resource consumption, but in the case of high missingness, it will greatly reduce the recovery ability of the model. 3) Retrain through fine-tuning. Fine-tuning is to reuse the previous parameters to the current model as initialization, and the newly arrived data can be used to update the parameters. However, the fine-tuning method only uses the most recent data for update, does not take into account the historical data well, and the historical knowledge is not retained, which may lead to forgetting problems. The accuracy of data recovery may be affected.

[0005] The invention patent with the prior art publication number CN112988723A proposes a traffic data repair method based on spatial self-attention graph convolutional recurrent neural network, which uses the fully connected layer as the input layer to map the input to a high-dimensional space to improve the model's expressiveness; the bidirectional graph convolution gated recurrent unit is obtained by replacing the fully connected layer in the gated recurrent unit with a graph convolution, which can simultaneously model local spatial correlation and temporal correlation; the multi-head spatial self-attention module is used to capture the implicit spatial correlation of the road network, while also aggregating the information of each node from a global perspective; the convolutional layer is used as the output layer to attenuate the feature dimension. This solution is to complete the entire tensor data, resulting in long training time and high resource consumption. Summary of the invention

[0006] In order to overcome the defects of the data completion methods in the prior art, such as long training time, high resource consumption, and failure to make full use of new data and historical data to restore missing entries, the present invention provides a lightweight, fast and accurate network traffic spatiotemporal tensor completion method and system based on meta-learning.

[0007] The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows:

[0008] The first aspect of the present invention provides a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning, comprising the following steps:

[0009] Get the public dataset tensor, divide it into sub-tensors of equal size and use them as input for each training stage in sequence;

[0010] Start training, obtain the final embedding matrix of the previous stage according to the preset method, use it as the initialization of the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module with fixed parameters, and obtain the intermediate embedding matrix of the current stage;

[0011] Freeze the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module again, and use the loss function to update the migration module parameters;

[0012] The intermediate embedding matrix of the current stage is fused with the final embedding matrix of the previous stage, and the fused embedding matrix is ​​input into the migration module after the parameter update to obtain the final embedding matrix of the current stage;

[0013] Restore the sub-tensor of the current stage using the final embedding of the current stage;

[0014] Repeat the above training steps until all sub-tensors are restored to obtain the restored tensor of the entire dataset.

[0015] Furthermore, the preset method for obtaining the final embedding matrix of the previous stage is: if there is training in the previous stage, the final embedding matrix of the previous stage is directly used; if there is no training in the previous stage, the embedding matrix after random initialization is used as the final embedding matrix of the previous stage.

[0016] Furthermore, the random initialization is standard normal initialization.

[0017] Furthermore, the migration module sets three embedding matrices of source nodes, target nodes, and time slices for migration training according to the characteristics of network data using the CP decomposition method.

[0018] Furthermore, there are two different embedded migration network forms in the migration module. For source nodes and target nodes, a modified GCN network is used for migration; for time slices, a modified time series network is used for migration.

[0019] Furthermore, the improved GCN network is to add a KAN network to the GCN network, and replace the traditional weight parameters commonly used in the GCN network through the core learning mechanism of KAN, and the embedding matrix migration method of the source node and the target node is:

[0020] Take the final embedding of the previous stage and the embedding of the current stage that does not contain historical information for feature fusion;

[0021] The fused embedding matrix and the network topology adjacency matrix obtained from the dataset are input into the improved GCN network to calculate the final embedding.

[0022] Furthermore, the improved time series network is a masked multi-head probabilistic self-attention time network.

[0023] Furthermore, the method for restoring the sub-tensor at the current stage specifically includes:

[0024] Find out the indices of the unobserved entries of the subtensor at the current stage;

[0025] extracting the corresponding feature vector from the final embedding of the current stage using the entry index;

[0026] The extracted feature vector is input into the dot product-based interaction function to recover the sub-tensor of the current stage.

[0027] The second aspect of the present invention provides a lightweight, fast and accurate network traffic spatiotemporal tensor completion system based on meta-learning, the system comprising: a memory, a processor, the memory comprising a lightweight, fast and accurate network traffic spatiotemporal tensor completion method program based on meta-learning, and the lightweight, fast and accurate network traffic spatiotemporal tensor completion method program based on meta-learning is executed by the processor to implement the following steps:

[0028] Get the public dataset tensor, divide it into sub-tensors of equal size and use them as input for each training stage in sequence;

[0029] Start training, obtain the final embedding matrix of the previous stage as the initialization of the embedding matrix of the current stage according to the preset method, input the sub-tensor of the current stage into the migration module with fixed parameters, capture the features of the current stage, and obtain the intermediate embedding matrix of the current stage;

[0030] Freeze the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module again, and use the loss function to update the migration module parameters;

[0031] The intermediate embedding matrix of the current stage is fused with the final embedding matrix of the previous stage, and the fused embedding matrix is ​​input into the migration module after the parameter update to obtain the final embedding matrix of the current stage;

[0032] Restore the sub-tensor of the current stage using the final embedding of the current stage;

[0033] Repeat the above training steps until all sub-tensors are restored to obtain the restored tensor of the entire dataset.

[0034] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0035] The present invention introduces meta-learning into the tensor completion model. By splitting the continuous traffic data into individual sub-tensors, the tensor completion model can quickly adapt to the newly arrived sub-tensor data using the embedded information learned in the previous sub-tensor. Compared with the long training time of retraining all data and the problem of forgetting historical information in fine-tuning retraining, the present invention achieves rapid convergence of newly arrived data through meta-learning, while preserving rich historical information, thereby improving the speed and accuracy of training.

[0036] For different nodes, the present invention designs two different embedding migration network forms for the migration module. For source nodes and target nodes with spatial information, a modified GCN network is designed for the migration of spatial embedding. Compared with ordinary graph convolutional neural networks, after adding the KAN network, the graph convolutional neural network can converge faster. Through the core learning mechanism of KAN, the spline replaces the traditional weight parameters commonly used in neural networks, so that the module can adaptively model the complex relationship of spatial information in network traffic data by adjusting the spline shape, thereby minimizing the approximation error and enhancing the network's ability to learn subtle patterns from high-dimensional data sets; for time slices with rich time information, a masked multi-head probabilistic self-attention time network is designed for the migration of time slot embedding. Compared with ordinary time series models, this scheme uses the form of masking to enable the model to directly output all target time embeddings through a forward process, and uses the probabilistic sparse attention mechanism to enhance the model performance, so that the model can screen out important parameters and reduce the computational complexity. At the same time, by refining the time embedding in the network traffic data, more accurate knowledge migration can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the purpose and technical solution of the present invention clearer, the present invention provides the following drawings and descriptions:

[0038] Figure 1 A system flow chart provided for an embodiment of the present invention;

[0039] Figure 2 A structural diagram of GCN-KAN provided in an embodiment of the present invention;

[0040] Figure 3 A comparison diagram of normalized mean square error provided by an embodiment of the present invention;

[0041] Figure 4 A comparison chart of normalized mean absolute error provided by an embodiment of the present invention;

[0042] Figure 5 A comparison chart of processing time provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0045] The present invention provides a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning, such as Figure 1 The figure shows a lightweight meta-learning-based fast and accurate network flow spatiotemporal tensor completion method system flow chart. In order to obtain complete network flow tensor data, the main idea of ​​the present invention is: first, the network flow data is modeled as a three-dimensional tensor The rows represent source nodes, the columns represent target nodes, and the depth represents the time slice. Then each item x ijk represents the network traffic data from the starting node i to the destination node j in the kth time slot. Since some node pairs are not measured or information is lost in a certain time slot, the tensor It is generally a sparse, incomplete tensor. Then all given observed entries Ω can be expressed as:

[0046] Ω={(i,j,k)|x ijk ≠0}

[0047] In order to solve the network traffic data recovery problem, the given observed entries Ω are used to recover the missing entries. In a specific embodiment, it is assumed that the matrix of node pairs in the first time slice is:

[0048]

[0049] Then the zeros in the matrix are missing, and the non-zero values ​​are observable, so the observable item indices are (1,1,2) and (1,3,3), which means that the first row and second column and the third row and third column in the first time slice are observable values, and the observable values ​​are needed to complete the zero values ​​in the matrix.

[0050] Then the ultimate goal of data recovery is to satisfy the following formula:

[0051]

[0052] Among them, A, B, and C are the source node, target node, and time slice factor matrix respectively. represents the raw tensor data that has values ​​only in the observation set Ω, Represents the tensor reconstructed by factor matrices A, B, and C, and calculates the error on the Ω set. It is the Frobenius norm, that is, the sum of the squares of the elements of the matrix, which represents the measure of error. The goal of the formula is to minimize the error between the reconstructed tensor and the observed partial data by optimizing A, B, and C.

[0053] The specific steps of the present invention are as follows:

[0054] S1: First, obtain the public dataset tensor. In a specific embodiment, the Abilene and GEANT datasets are used. In order to simulate the continuous arrival of new data, the data is divided into sub-tensors of the same size in a non-overlapping manner, which is expressed as:

[0055]

[0056] Each sub-tensor can be represented as new data arriving at different stages t, so retraining the new data arriving at stage t can be expressed as:

[0057]

[0058] in, To save resources and quickly adapt to new data, the model is incrementally updated in a sequential manner for the restored tensor, expressed as:

[0059]

[0060] For newly arrived data, the previous methods of retraining only on new data and retraining through fine-tuning cannot solve the problems of fast model update and underutilization of historical information. Therefore, the present invention introduces meta-learning into parameter update, regards each sub-tensor as an independent task, and incorporates the knowledge in the old tasks into the historical information through meta-learning. When facing new tasks, the parameters can be adjusted quickly to promote the adaptation of the model, and the learning efficiency can be improved through knowledge transfer.

[0061] S2: Start training, obtain the final embedding matrix of the previous stage according to the preset method, use it as the initialization of the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module with fixed parameters, capture the features of the current stage, and obtain the intermediate embedding matrix of the current stage;

[0062] It should be noted that the preset method for obtaining the final embedding matrix of the previous stage is: if there is training in the previous stage, the final embedding matrix of the previous stage is directly used; if there is no training in the previous stage, the embedding matrix initialized with the standard normal is used as the final embedding matrix of the previous stage.

[0063] The migration module sets three embedding matrices of source node A, target node B, and time slice C according to the characteristics of network data according to the CP decomposition method for migration training. Let the embedding [A, B, C] of the current stage be W(t) , embed the final of the previous stage into W (t -1) As the initialization of the current embed Then fix the parameter θ of the migration module, use the migration module to capture the feature information of the current stage, and only update the intermediate embedding matrix of the current stage The loss function of tensor completion is expressed as:

[0064]

[0065] Among them, L is the objective function, which can be the mean square loss function or the mean absolute loss function, λ1 is a hyperparameter, and fθ represents the migration module.

[0066] For source nodes and target nodes with spatial information, the GCN-KAN network is used for migration. The graph convolutional neural network GCN is a powerful representation that is widely used to learn graph structured data. It updates the representation of each node by explicitly considering the connection relationship between nodes and aggregating the information of neighboring nodes. On this basis, the KAN network is used to replace the traditional activation function. Compared with traditional activation functions, KAN uses a learnable activation function at the edge of the network. This design allows each weight parameter in KAN to be replaced by a single variable function. These functions are usually parameterized in the form of spline functions, which provides extremely high flexibility and can simulate complex functions with fewer parameters, enhancing the interpretability of the model. The GCN-KAN structure is as follows: Figure 2 shown.

[0067] Taking the source node as an example, take the final embedding A(t-1) of the previous stage and the embedding that does not contain historical information in the current stage For feature fusion, the simplest fusion method is used, and the formula is:

[0068]

[0069] Then the fused embedding matrix And the network topology adjacency matrix adj obtained from the data set is input into the GCN-KAN network to calculate the final embedding A(t), the formula is:

[0070]

[0071] Among them, D is the degree matrix and I is the identity matrix, which are automatically generated when inputting GCN. Through feature fusion and knowledge transfer, while obtaining historical information, the spatial information of the network topology is used to obtain the final embedding A(t) of the source node. In the same way, the final embedding B(t) of the target node can be obtained.

[0072] For time slices with rich temporal information, a masked multi-head probabilistic attention time network is used to obtain the temporal embedding C(t). Because a single attention mechanism can only mine limited information, multiple attentions are used to calculate the information represented by the subspace at different positions. At the same time, different heads can focus on the representation of different features in the sequence, thereby better extracting the input feature information. For a time series, the output should only depend on the output before that moment, but not on the output after that moment. The role of the mask is to mask certain values ​​so that they have no effect when the parameters are updated. Using a masked multi-head probabilistic self-attention network, all target time embeddings are directly output through a forward process, and the probabilistic sparse attention mechanism is used to enhance the model performance, so that the model can screen out important parameters and reduce the computational complexity. At the same time, more accurate knowledge transfer is achieved by refining the time embedding in network traffic data.

[0073] S3: Freeze the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module again, use the back propagation algorithm to calculate the gradient of the loss function with respect to the model parameters, so that the migration model obtains the historical knowledge before stage t, and updates the parameter θ through the optimization algorithm Adam. The loss function is expressed as:

[0074]

[0075] Among them, λ2 is a hyperparameter, fθ represents the migration module, is the embedding of the current stage, and W(t-1) is the final embedding of the previous stage.

[0076] S4: Fuse the intermediate embedding matrix of the current stage with the final embedding matrix of the previous stage, and use the fused embedding matrix to input into the migration module after parameter update to obtain the final embedding matrix of the current stage;

[0077] S5: Use the final embedding of the current stage to restore the sub-tensor of the current stage. The specific process is:

[0078] Find out the indices of the unobserved entries of the subtensor at the current stage;

[0079] extracting the corresponding feature vector from the final embedding of the current stage using the entry index;

[0080] Input the extracted feature vector into the dot product-based interaction function to recover the sub-tensor of the current stage The dot product-based interaction function is as follows:

[0081]

[0082] Among them, (i, j, k) is the index of the entry that needs to be restored, R is the rank of CP decomposition, that is, the rank of tensor decomposition, which controls the complexity of decomposition.

[0083] σ(x)=1 / (1+exp(-x))

[0084] It is a nonlinear activation function. The interaction function enables the model to capture nonlinear interaction patterns and is conducive to migration.

[0085] S6: Repeat the above training steps until all sub-tensors are restored to obtain the restored tensor of the entire data set.

[0086] In order to evaluate the performance of the LSTTC of the present invention, the dimension of the embedded feature is designed to be 20, the window size of the time module is set to 48, the number of time slices of each sub-tensor is set to 288, and the existing algorithms in Table 1 below are used for comparative experiments.

[0087] Table 1

[0088]

[0089]

[0090] The results are as follows Figure 3 ,4,5, from Figure 3 and Figure 4 It can be seen that the LSTTC of the present invention has lower normalized mean square error and normalized mean absolute error under the same sampling rate, which significantly improves the quality and accuracy of data recovery. Figure 5 It can be seen that when processing the same data set, the recovery time required by the LSTTC of the present invention is greatly shortened, thereby achieving higher execution efficiency and faster response speed.

[0091] The second aspect of the present invention provides a lightweight, fast and accurate network traffic spatiotemporal tensor completion system based on meta-learning, characterized in that the system comprises: a memory, a processor, the memory comprises a lightweight, fast and accurate network traffic spatiotemporal tensor completion method program based on meta-learning, and the lightweight, fast and accurate network traffic spatiotemporal tensor completion method program based on meta-learning is executed by the processor to implement the following steps:

[0092] Get the public dataset tensor, divide it into sub-tensors of equal size and use them as input for each training stage in sequence;

[0093] Start training, obtain the final embedding matrix of the previous stage according to the preset method, use it as the initialization of the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module with fixed parameters, capture the features of the current stage, and obtain the intermediate embedding matrix of the current stage;

[0094] Freeze the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module again, and use the loss function to update the migration module parameters;

[0095] The intermediate embedding matrix of the current stage is fused with the final embedding matrix of the previous stage, and the fused embedding matrix is ​​input into the migration module after the parameter update to obtain the final embedding matrix of the current stage;

[0096] Restore the sub-tensor of the current stage using the final embedding of the current stage;

[0097] Repeat the above training steps until all sub-tensors are restored to obtain the restored tensor of the entire dataset.

[0098] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning, characterized by: The steps include: Get the public dataset tensor, divide it into sub-tensors of equal size and use them as input for each training stage in sequence; Start training, obtain the final embedding matrix of the previous stage according to the preset method, use it as the initialization of the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module with fixed parameters, and obtain the intermediate embedding matrix of the current stage; Freeze the embedding matrix of the current stage, input the sub-tensor of the current stage into the migration module again, and use the loss function to update the migration module parameters; The intermediate embedding matrix of the current stage is fused with the final embedding matrix of the previous stage, and the fused embedding matrix is ​​input into the migration module after the parameter update to obtain the final embedding matrix of the current stage; Restore the sub-tensor of the current stage using the final embedding matrix of the current stage; Repeat the above training steps until all sub-tensors are restored to obtain the restored tensor of the entire dataset.

2. According to claim 1, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The preset method for obtaining the final embedding matrix of the previous stage is: if there is training in the previous stage, the final embedding matrix of the previous stage is directly used; if there is no training in the previous stage, the embedding matrix after random initialization is used as the final embedding matrix of the previous stage.

3. According to claim 2, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The random initialization is a standard normal initialization.

4. According to claim 1, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The migration module sets up three embedding matrices of source nodes, target nodes, and time slices according to the characteristics of network data using the CP decomposition method for migration training.

5. According to claim 4, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: There are two different embedding migration network forms in the migration module. For source nodes and target nodes, an improved GCN network is used to migrate the embedding matrix; for time slices, an improved time series network is used to migrate the embedding matrix.

6. According to claim 5, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The improved GCN network is a GCN network that adds a KAN network, and replaces the traditional weight parameters commonly used in the GCN network through the core learning mechanism of KAN, the spline.

7. According to claim 5, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The embedding matrix migration method of the source node and the target node is: Take the final embedding of the previous stage and the embedding of the current stage that does not contain historical information for feature fusion; The fused embedding matrix and the network topology adjacency matrix obtained from the dataset are input into the improved GCN network to calculate the final embedding.

8. According to claim 5, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The improved time series network is a masked multi-head probabilistic self-attention time network.

9. According to claim 1, a lightweight, fast and accurate network traffic spatiotemporal tensor completion method based on meta-learning is characterized in that: The method for restoring the sub-tensor at the current stage specifically includes: Find out the indices of the unobserved entries of the subtensor at the current stage; Extracting the corresponding feature vector from the final embedding matrix of the current stage using the entry index; The extracted feature vector is input into the dot product-based interaction function to recover the sub-tensor of the current stage.

10. A lightweight, fast and accurate network traffic spatiotemporal tensor completion system based on meta-learning, characterized by: The system includes: a memory and a processor, wherein the memory includes a lightweight meta-learning-based fast and accurate network traffic spatiotemporal tensor completion method program, and when the lightweight meta-learning-based fast and accurate network traffic spatiotemporal tensor completion method program is executed by the processor, the steps of a lightweight meta-learning-based fast and accurate network traffic spatiotemporal tensor completion method as described in any one of claims 1 to 9 are implemented.

Images