A network traffic matrix estimation model, method, and system

Abstract

The present application relates to the technical field of network engineering and artificial intelligence, and particularly relates to a network traffic matrix estimation model, method and system. In the training process, an examiner module is introduced as a self-supervised guide to guide a flow generation network module to learn the spatiotemporal correlation inside the traffic matrix, to provide a reasonable evaluation signal for unobserved source pair flow for the estimation model, to effectively make up for the problem of missing supervision caused by sparse training data, and to enable the model obtained through training to achieve high-precision traffic matrix estimation based on link load and loss information assumption.

Description

Technical Field

[0001] This invention relates to the fields of network engineering and artificial intelligence technology, and in particular to a network traffic matrix estimation model, method and system. Background Technology

[0002] The Traffic Matrix (TM) is a core metric for network observability, used to characterize the traffic distribution between all origin-destination (OD) pairs in a network. It is valuable in tasks such as traffic engineering, anomaly detection, and capacity planning. However, directly measuring the traffic of the entire network's OD pairs is costly and difficult to scale, especially in large-scale networks where it consumes significant amounts of router ternary content-addressable memory (TCAM) resources. Therefore, researchers commonly employ Network Tomography (NT) techniques to infer the complete TM from link loads obtained at low cost.

[0003] Traditional NT methods typically assume that the routing matrix is ​​known and solve the underdetermined linear equation Y=AX by introducing statistical priors (such as Poisson distribution, gravity model) or deep learning models. However, modern networks widely adopt adaptive routing strategies, resulting in dynamic changes to the routing matrix, or even the inability to know it, rendering the above methods ineffective.

[0004] To overcome the reliance on routing matrices, existing technologies have proposed TM estimation methods based on reversible flow generation models (such as the FlowTM method). This method achieves high-precision estimation without the need for routing matrices by constructing a bidirectional reversible mapping between TM and link load, and proves that lost information can be decoupled from the observable part.

[0005] However, the inventors discovered a significant technical flaw in existing generative flow-based models (including CN118101497B, Network Traffic Matrix Estimation Model, Method, and System Based on Generative Flow Model): their training process heavily relies on a large amount of complete historical TM data. In real-world network environments, obtaining large-scale, high-quality complete TM data is extremely difficult: if a complete TM is available, estimation is unnecessary; if no complete TM is available, the model struggles to converge due to a lack of effective supervision signals. Especially when the proportion of observable OD flows is extremely low (e.g., below 5% or even lower), existing models cannot learn reasonable patterns in the unobserved portion, leading to a sharp decline in estimation performance.

[0006] While existing research has attempted to alleviate the data sparsity problem through matrix completion or self-supervised learning, these methods often implicitly rely on routing structures or fail to effectively model the spatiotemporal context consistency of TMs. Therefore, there is an urgent need for a new mechanism that is independent of routing matrices and can maintain high-accuracy estimation capabilities with very limited observation data to address the cold start or data scarcity challenges in real-world network deployments. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies that rely on routing matrices or complete observation data for traffic matrix estimation, this invention proposes a training method for a network traffic matrix estimation model that can adapt to situations with limited observation data and no routing matrix. The trained model can achieve high-precision traffic matrix estimation based on assumptions about link load and loss information.

[0008] This invention proposes a training method for a network traffic matrix estimation model, which first constructs a basic model and a training dataset {X}. n ,Y n};X n and Y n These are the traffic matrix and link load at time step n, respectively;

[0009] The basic model includes: a flow generation network module and a reviewer module; the flow generation network module generates link load and latent variables Z following a specified distribution based on the flow matrix; the concatenated vector of link load Y and latent variables Z has the same dimension as the flow matrix X in column vector format; the reviewer module generates a probability matrix to label the authenticity of flow pairs based on the flow matrix;

[0010] After training the base model to convergence on the learning dataset, the flow generation network module is extracted as the estimation model, and the inverse operation of the estimation model is performed to obtain the flow matrix estimate based on the link load.

[0011] Preferably, the training method consists of two stages:

[0012] The first stage of the basic model applies to the training samples (X). n ,Y n The processing method for ) is as follows:

[0013] Random noise Z is obtained by randomly sampling from a specified distribution. n The stream generation network module is for X in column vector format. n [Y] with the same dimensions n Z n Perform the inverse operation to obtain the composite flow matrix. Using a mask matrix M n Obstruction of traffic matrix X n Afterwards, from Fill in the X with collected elementsn Occluded elements yield a mixed flow matrix. ;

[0014] The reviewer module is based on a hybrid flow matrix. Generation probability matrix P n To label the mixed flow matrix Each element in the matrix X is derived from actual observed flow. n The probability of;

[0015] The second stage of the basic model applies to the training samples (X). n ,Y n The processing method for ) is as follows:

[0016] The flow generation network module for the flow matrix X n The estimated link load is obtained through processing. And the lost information z that conforms to the specified distribution n ; and z n The concatenated vector and column vector format of X n Same dimensions; Stream generation network module Perform the inverse operation to obtain the reconstructed flow matrix. ;

[0017] Random noise Z is obtained by randomly sampling from a specified distribution. n The stream generation network module for [Y] n Z n Perform the inverse operation to obtain the composite flow matrix. The reviewer module is based on a synthetic flow matrix. Generate probability matrix;

[0018] The first stage constructs a loss function based on the processing results of the reviewer module to update the base model. When the first stage training converges, it proceeds to the second stage. The second stage constructs a loss function based on the processing of the stream generation network module and the reviewer module to update the base model. When the second stage training converges, the stream generation network module is extracted as the estimated model.

[0019] Preferably, the loss function of the second stage includes at least one of generation loss, reversibility loss, link load loss, independence loss, and estimation loss;

[0020] Reversibility loss is used to measure the reconstructed traffic matrix obtained by the flow generation network module through link load estimation and inverse operation. With the observed flow matrix X n The distance;

[0021] Link load loss is used to measure the flow generation network module based on X. nEstimated link load and the actual link load Y n The distance;

[0022] Independence loss is used to measure the probability distribution generated by the reviewer module and p. Y (y n The difference between products of the specified distributions; p Y (y n Y represents the actual link load. n The probability distribution;

[0023] The estimated loss is the traffic matrix X after masking. n and The distance;

[0024] The generation loss is used to measure the distance between the probability matrix output by the reviewer module and the all-1 vector.

[0025] Preferably, the distance between vectors is represented by the square of the 2-norm.

[0026] Preferably, the loss of independence is measured using the maximum average difference.

[0027] The preferred formula for estimating the loss is:

[0028] ;

[0029] To The mask matrix used for masking processing, where d is the set of training samples for a single round in the second stage.

[0030] Preferably, the loss function used in the first stage of training is the sum of the distance between the probability matrix output by the reviewer module and the mask matrix, plus the gradient loss of the reviewer module;

[0031] Gradient loss of the reviewer module for:

[0032] ;

[0033] in, For the first phase, targeting the training samples (X) n ,Y n The generated mixed flow matrix, for The probability matrix obtained after processing the basic model for exist The gradient on; 1 is an all-1 vector.

[0034] The present invention proposes a network traffic matrix estimation device, which includes a memory and a processor. The memory stores a computer program, and the processor is connected to the memory. The processor is used to execute the computer program to implement the training method of the network traffic matrix estimation model.

[0035] The present invention proposes a network traffic matrix estimation method. First, the link load Y0 is obtained, and a random number vector Z0 is generated. The concatenated vector [Y0,Z0] of Y0 and Z0 has the same dimension as the traffic matrix in column vector format. Then, the inverse operation of the estimation model is performed on [Y0,Z0] to obtain the matrix estimation result X0.

[0036] The present invention proposes a network traffic matrix estimation system, which includes a memory and a processor. The memory stores a computer program, and the processor is connected to the memory. The processor is used to execute the computer program to implement the network traffic matrix estimation method.

[0037] The advantages of this invention are:

[0038] (1) The training method of the network traffic matrix estimation model proposed in this invention introduces a reviewer module as a self-supervised guide during the training process, which guides the flow generation network module to learn the spatiotemporal correlation within the traffic matrix, providing the estimation model with a reasonable evaluation signal for unobserved source point pairs (OD) flows, effectively making up for the lack of supervision caused by the sparsity of training data, so that the trained model can achieve high-precision traffic matrix estimation based on the assumptions of link load and lost information.

[0039] (2) The present invention proposes a two-stage alternating training mechanism. The first stage of training focuses on the reviewer module to establish reliable discrimination ability; the second stage focuses on guiding the flow generation network module (FlowTM) to generate a complete flow matrix (TM) that conforms to the real network behavior pattern, avoiding collusion and cheating of the model and ensuring the generation quality.

[0040] (3) The present invention designs a hybrid input strategy and a novel generation loss Lgen+: during evaluation, the true value of the observed position is retained, and only the generated value is used to fill the unobserved position. The goal is to make the probability matrix output by the reviewer module approach all one, so that the model can make full use of the structural information hidden in the link load to infer the missing traffic, which further improves the accuracy of flow generation.

[0041] (4) The present invention significantly improves the practicality in scenarios with scarce data: Experiments show that, under the condition of using only 2% to 30% of the training data and completely unknown routing matrix, the estimation accuracy of the present invention is comparable to the performance of FlowTM under complete data, which greatly expands the deployment feasibility of generative TM estimation method in real network. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall architecture of the network traffic matrix estimation model provided in this embodiment of the invention;

[0043] Figure 2 This is a flowchart illustrating the network traffic matrix estimation method provided in this embodiment of the invention.

[0044] Figure 3(a): Comparison of cumulative error distribution for elephant flow estimation in the Abilene dataset at low observation rates;

[0045] Figure 3(b): Comparison of cumulative distribution of large-flow estimation error in the Abilene dataset under high observation ratio;

[0046] Figure 3(c): Comparison of cumulative error distribution for Mice Flows estimation on the Abilene dataset at low observation rates;

[0047] Figure 3(d): Comparison of cumulative distribution of small stream estimation error in the Abilene dataset under high observation ratio;

[0048] Figure 3(e): Comparison of cumulative distribution of large-flow estimation error in the GÉANT dataset at low observation ratios;

[0049] Figure 3(f): Comparison of cumulative distribution of flow estimation error in the GÉANT dataset under high observation ratio;

[0050] Figure 3(g): Comparison of cumulative distribution of small stream estimation errors in the GÉANT dataset at low observation ratios;

[0051] Figure 3(h): Comparison of cumulative distribution of small stream estimation errors in the GÉANT dataset under high observation ratio. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, this embodiment proposes a network traffic matrix estimation model to establish a bijective transformation between the traffic matrix X and the link load Y.

[0054] Specifically, the estimation models include embedded networks and fluid-based networks.

[0055] An embedding network is used to encode and decode the traffic matrix X to obtain the embedding vector X'. Specifically, the embedding network consists of an encoder and a decoder. The encoder maps the high-dimensional and skewed original traffic matrix X to the latent space to obtain the embedding vector X'; the decoder restores the traffic matrix X from the embedding vector X'. To accommodate the skewed distribution of the traffic data, the encoder includes a fully connected layer and a Tanh activation function to compress the embedding vector X' to the interval [-1, 1].

[0056] A flow-based network performs an invertible transformation on the embedding vector X' to obtain the link load Y and the latent variable Z (representing lost information) following a specified simple distribution G (such as a Gaussian distribution or a standard normal distribution; referred to as the specified distribution). Specifically, the flow-based network contains a series of invertible coupling layers. The concatenated vector of the link load Y and the latent variable Z has the same dimensions as the flow matrix X in column vector format.

[0057] The mathematical expression for a fluid-based network is: and its inverse transformation F represents a fluid-based network, F -1 This represents the inverse of a flow-based network.

[0058] The mathematical expression for the estimation model is: , , This represents the estimation model, also denoted as the stream generation network module. for The inverse operation.

[0059] Reference Figure 2 The training method for the estimation model includes the following steps:

[0060] S1. Construct the learning dataset; specifically including the following steps S11-S12.

[0061] S11. Obtain the original dataset {Y,X}, where Y is the network link load data set and X is the historical traffic matrix data set; Y={Y1,Y2,…,Y} T}, X={X1,X2,…,X T}; where Y1, Y2 and Y T These represent the link loads at time step 1, time step 2, and time step T, respectively, X1, X2, and X... T These are the flow matrices at time step 1, time step 2, and time step T, respectively.

[0062] S12. Construct the learning dataset {X} n ,Y n}, X n and Y nThese represent the traffic matrix and link load at time step n, respectively.

[0063] S2. Construct a basic model that includes a flow generation network module and a reviewer module. The flow generation network module has the same structure as the estimation model. The reviewer module is used to extract spatiotemporal correlation features from the flow matrix and generate a probability map (i.e., the probability matrix of the authenticity of each source point in the flow) to guide the learning of the flow generation network module.

[0064] Specifically, the reviewer module includes a sequentially connected downsampling path, an upsampling path, and an output layer.

[0065] The downsampling path consists of multiple 3D convolutional layers (the kernel size can be set to...). Step size is This is used to progressively extract deep features of the traffic matrix of the input reviewer module in the time and space (Origin-Destination) dimensions, and compress the spatial resolution of the feature map.

[0066] The upsampling path consists of multiple 3D transposed convolutional layers, and the upsampling path and the downsampling path form a symmetrical convolutional structure, which is used to gradually recover the temporal and spatial dimensions of the deep features output by the downsampling path to obtain the feature map.

[0067] A sigmoid activation layer is set at the end of the output layer. The output layer converts the feature map output by the upsampled path into a probability map P with the same dimension as the traffic matrix of the input reviewer module. Each element in the probability matrix has a value between [0, 1], which represents the confidence level that the traffic data at the corresponding position is a "true observation".

[0068] S3, from the learning dataset {X n ,Y n Train the base model until convergence;

[0069] This step specifically includes the following sub-steps:

[0070] S31. Extract a batch of training samples (X) from the learning dataset. n ,Y n The link load Y is generated by the flow generation network module. n and random noise Z n The process is performed to obtain the composite flow matrix. Random noise Z n The link load Y is randomly sampled from a specified simple distribution G. n and random noise Z n The concatenated vector and column vector format of X n Same dimensions.

[0071] S32, X n and The mixture yields a mixed flow matrix. , obtain samples ( M n ,Y n ).

[0072] In this step, a mask matrix M is specifically used. n Obstruction of traffic matrix X n Obtain the missing flow matrix Then from the synthetic flow matrix Supplementing the missing traffic matrix The occlusion elements are used to obtain the mixed flow matrix. The formula is expressed as:

[0073] ;

[0074] Indicates a mask.

[0075] S33, Mixed flow matrix Input the reviewer module, and the reviewer module generates the probability matrix P. n Used to label mixed flow matrices Each element in the matrix X is derived from actual observed flow. n The probability of is denoted as:

[0076] ;

[0077] S34, in sample (X) n M n ,Y n The loss function is calculated on}), and then passed through the loss function L. ins Backpropagation updates the stream generation network module and the reviewer module;

[0078] ;

[0079] ;

[0080] in, d 1 represents the sample set corresponding to the training samples extracted in step S31. The gradient loss for the reviewer module; for The probability matrix obtained after processing the basic model. for exist gradient on; M n For X n Masking processes all mask matrices.

[0081] S35. Repeat steps S31-S34 above until the basic model converges.

[0082] S4, in learning dataset {X n ,Y n The loss function L is used in} F After updating the base model to convergence, the flow generation network module is extracted as the network traffic matrix estimation model required by this invention.

[0083] In step S4, the base model applies the sample (X) n ,Y n The processing procedure is as follows:

[0084] S41, The flow generation network module processes the flow matrix X n Processing to obtain , The estimated link load can be denoted as ;z n The estimated lost information conforms to a simple distribution G (or simply the specified distribution); and z n The concatenated vector and column vector format of X n Same dimensions; then... Perform the inverse operation to obtain the reconstructed flow matrix. ;

[0085] S42. Randomly sample vector Z from a simple distribution G. n , vector Z n With Y n Concatenation forms X in column vector format n Same-dimensional vector [Y n Z n X in column vector format. n The dimension is m.

[0086] S43, regarding vector [Y] n Z n Perform the inverse operation of the flow generation network module to obtain the composite flow matrix. The reviewer module is based on a synthetic flow matrix. Generation probability matrix ;

[0087] In step S4, we first start with the learning dataset {X} n ,Y n A batch of training samples is extracted from the dataset (denoted as d for a single batch of training samples). The training samples are processed using steps S41-S43 above, and the loss function L is calculated. F Through the loss function L FBackpropagation updates the stream generation network module and the reviewer module; when the loss function L F If convergence is achieved, the fixed-flow generation network module is used as the estimation model.

[0088] ;

[0089] Among them, L inv L link L indep L est+ h and L gen+ These are reversibility loss, link load loss, independence loss, estimation loss, and generation loss, respectively. , , , and All of these are weighted.

[0090] Specifically:

[0091] (1)

[0092] (2)

[0093] (3)

[0094] (4)

[0095] (5)

[0096] In formula (3) ， for The probability distribution of p Y (y n ) is X n The corresponding real link load Y n The probability distribution of p z (z n ) for z n The distribution is (in this application, it is a simple distribution G). MMD represents the maximum mean difference. Specifically, the stream generation network module is based on X. n Generate a vector of the same dimension [Y] n Z n probability distribution of ] The vector corresponding to the highest probability is taken as the final output. .

[0097] When the loss function L F If convergence is achieved, the fixed-flow generation network module is used as the estimation model.

[0098] Specifically, the convergence condition for the first stage can be set as the loss function converging, or the number of model updates reaching a set value. The convergence condition for the second stage is also set as the loss function converging. Loss function convergence means that the range of the loss function in the most recent multiple iterations is less than a set floating range.

[0099] In practical applications, the steps for estimating the flow matrix are as follows:

[0100] Step 1: Obtain the link load Y0 and generate a random number vector Z0. The concatenated vector [Y0, Z0] of Y0 and Z0 has the same dimension as the traffic matrix in column vector format. Perform the inverse operation of the estimation model on [Y0, Z0] to obtain the matrix estimation result X0.

[0101] .

[0102] To verify the effectiveness of the proposed method in scenarios with limited training data, this embodiment conducted comparative experiments on two real backbone network datasets (Abilene and GÉANT). The experiments compared the proposed method (FlowTM+) with three representative learning-based benchmark methods (MNETME, GAN, and VAE).

[0103] The experiment set four different observation ratios for historical data in the Transaction Matrix (TM): 2%, 30%, 70%, and 100%. 2% indicates that only 2% of the historical TM data was visible during the training phase, with the remaining data considered lost. Similarly, 30% indicates that only 30% of the historical TM data was visible during the training phase, with the remaining data considered lost; 70% indicates that only 70% of the historical TM data was visible during the training phase, with the remaining data considered lost; and 100% indicates zero loss.

[0104] Table 1 presents the experimental results of estimation accuracy (NMAE, NRMSE) and distribution similarity (MMD) for each method under different observation ratios. Specifically, in this embodiment, the dataset with a fixed observation ratio is divided into a training set and a test set. The model is trained on the training set, and the model performance is tested on the test set after training. Table 1 shows the test results of the trained model on the test set.

[0105] The three existing methods selected in this embodiment are as follows:

[0106] MNETME: This model learns the correlation between link load and traffic matrix through a backpropagation neural network containing the routing matrix, and uses the expectation-maximization algorithm to adjust the output, thereby achieving traffic matrix estimation based on link load.

[0107] GAN: This model consists of a generator and a discriminator. It learns the latent distribution of real traffic matrix data to generate a traffic matrix estimate that satisfies the mapping relationship between link load and traffic matrix.

[0108] VAE: This model uses an encoder to map the input data into a probability distribution in a latent space, uses a decoder to generate new data samples from the probability distribution in the latent space, and samples the estimated traffic matrix from the latent distribution that best fits the given link load and routing matrix.

[0109] Table 1 shows the performance of different methods on the two datasets:

[0110]

[0111] In Table 1 above, FlowTM+ represents the method of the present invention.

[0112] Table 1 summarizes the NMAE, NRMSE, and MMD metrics for each method under different observation ratios. As shown in the table, FlowTM+ consistently outperforms the baseline methods in all scenarios. Although the accuracy of all methods decreases to varying degrees as the observation ratio decreases, FlowTM+ maintains relatively stable performance, while the performance of other baseline methods declines significantly. This indicates that incomplete TM data leads to insufficient supervision, thereby significantly reducing estimation accuracy.

[0113] Notably, on the Abilene dataset, FlowTM+ achieved the estimation accuracy of the best benchmark method trained with 100% TM data using only 2% of the observations. Similarly, on the GÉANT dataset, FlowTM+ achieved performance comparable to the best benchmark method trained with all observations, requiring only 30% of the TM data. Furthermore, FlowTM+ also achieved a substantial improvement over the best benchmark in the MMD metric (measuring distribution similarity). Even with only 2% of the TM data, FlowTM+ reduced the MMD of the fully trained benchmark by 11.9% on the Abilene dataset and by 48.2% on the GÉANT dataset, demonstrating its superior ability to capture and learn the underlying distribution of TM.

[0114] These results demonstrate that FlowTM+ can effectively extract unobserved TM information from link load measurements and the spatiotemporal distribution of incomplete TMs. This capability significantly improves estimation accuracy in practical applications while substantially reducing the sampling cost of learning-based methods.

[0115] This embodiment further demonstrates the invention through ablation experiments, comparing FlowTM+ with the FlowTM model that lacks an auditor module during the training process (see CN118101497B Network Traffic Matrix Estimation Model, Method and System Based on Generative Flow Model).

[0116] Suppose there are T time-time samples. Calculate the mean flow rate for OD at each time-time, and sort the mean values ​​from largest to smallest. The first 20% of time-time samples with the largest mean flow rate are considered large flows, and the remaining 80% of time-time samples are considered small flows. Figures 3(a) to 3(h) This is a comparative diagram of the normalized mean absolute error (NMAE) cumulative distribution function (CDF) of embodiments of the present invention under different network datasets (Abilene and GÉANT), for different flow types (large flows and small flows) and different data observation ratios (low observation ratio and high observation ratio). This set of figures illustrates the performance differences between the method of the present invention (FlowTM+) and the basic method without the inclusion of a reviewer module (FlowTM, see CN118101497B Network Flow Matrix Estimation Model, Method and System Based on Generative Flow Model) and other benchmark methods in fine-grained flow estimation. The results show that the method of the present invention achieves higher estimation accuracy in various scenarios (the curve is closer to the upper left corner), especially in the large flow estimation task with a low observation ratio, where the present invention shows a significant performance advantage over the benchmark method.

[0117] Of course, those skilled in the art will recognize that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0118] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0119] The technologies, shapes, and structures not described in detail in this invention are all known technologies.

Claims

1. A training method for a network traffic matrix estimation model, characterized in that, First, we build the basic model and the training dataset {X}. n ,Y n };X n and Y n These are the traffic matrix and link load at time step n, respectively; The basic model includes: a flow generation network module and a reviewer module; the flow generation network module generates link load and latent variables Z following a specified distribution based on the flow matrix; the concatenated vector of link load Y and latent variables Z has the same dimension as the flow matrix X in column vector format; the reviewer module generates a probability matrix to label the authenticity of flow pairs based on the flow matrix; After training the base model to convergence on the learning dataset, the flow generation network module is extracted as the estimation model, and the inverse operation of the estimation model is performed to obtain the flow matrix estimate based on the link load. The training method consists of two phases: The first stage of the basic model applies to the training samples (X). n ,Y n The processing method for ) is as follows: Random noise Z is obtained by randomly sampling from a specified distribution. n The stream generation network module is for X in column vector format. n [Y] with the same dimensions n Z n Perform the inverse operation to obtain the composite flow matrix. Using a mask matrix M n Obstruction of traffic matrix X n Afterwards, from Collect elements to fill X n Occluded elements yield a mixed flow matrix. ; The reviewer module is based on a hybrid flow matrix. Generation probability matrix P n To label the mixed flow matrix Each element in the matrix is ​​derived from the actual observed flow matrix X. n The probability of; The second stage of the basic model applies to the training samples (X). n ,Y n The processing method for ) is as follows: The flow generation network module for the flow matrix X n The estimated link load is obtained through processing. And the lost information z that conforms to the specified distribution n ; and z n The concatenated vector and column vector format of X n Same dimensions; Stream generation network module Perform the inverse operation to obtain the reconstructed flow matrix. ; Random noise Z is obtained by randomly sampling from a specified distribution. n The stream generation network module for [Y] n Z n Perform the inverse operation to obtain the composite flow matrix. The reviewer module is based on a synthetic flow matrix. Generate probability matrix; The first stage constructs a loss function based on the processing results of the reviewer module to update the base model. When the first stage training converges, it proceeds to the second stage. The second stage constructs a loss function based on the processing of the stream generation network module and the reviewer module to update the base model. When the second stage training converges, the stream generation network module is extracted as the estimated model.

2. The training method for the network traffic matrix estimation model as described in claim 1, characterized in that, The loss function for the second stage includes at least one of the following: generation loss, reversibility loss, link load loss, independence loss, and estimated loss. Reversibility loss is used to measure the reconstructed traffic matrix obtained by the flow generation network module through link load estimation and inverse operation. With the observed flow matrix X n The distance; Link load loss is used to measure the flow generation network module based on X. n Estimated link load and the actual link load Y n The distance; Independence loss is used to measure the probability distribution generated by the reviewer module and p. Y (y n The difference between products of the specified distributions; p Y (y n Y represents the actual link load. n The probability distribution; The estimated loss is the traffic matrix X after masking. n and The distance; The generation loss is used to measure the distance between the probability matrix output by the reviewer module and the all-1 vector.

3. The training method for the network traffic matrix estimation model as described in claim 2, characterized in that, The distance between vectors is represented by the square of the 2-norm.

4. The training method for the network traffic matrix estimation model as described in claim 2, characterized in that, The loss of independence is measured using the maximum average difference.

5. The training method for the network traffic matrix estimation model as described in claim 2, characterized in that, The formula for estimating the loss is: To The mask matrix used for masking processing, where d is the set of training samples for a single round in the second stage.

6. The training method for the network traffic matrix estimation model as described in claim 1, characterized in that, The loss function used in the first stage of training is the sum of the distance between the probability matrix output by the reviewer module and the mask matrix, plus the gradient loss of the reviewer module. Gradient loss of the reviewer module for: in, For the first phase, targeting the training samples (X) n ,Y n The generated mixed flow matrix, for The probability matrix obtained after processing the basic model. for exist The gradient on; 1 is an all-1 vector.

7. A network traffic matrix estimation device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, the processor is connected to the memory, and the processor is used to execute the computer program to implement the training method of the network traffic matrix estimation model as described in any one of claims 1-6.

8. A network traffic matrix estimation method employing the training method of the network traffic matrix estimation model as described in any one of claims 1-6, characterized in that, First, obtain the link load Y0 and generate a random number vector Z0. The concatenated vector [Y0,Z0] of Y0 and Z0 has the same dimension as the traffic matrix in column vector format. Then, perform the inverse operation of the estimation model on [Y0,Z0] to obtain the matrix estimation result X0.

9. A network traffic matrix estimation system, characterized in that, It includes a memory and a processor. The memory stores a computer program, and the processor is connected to the memory. The processor is used to execute the computer program to implement the network traffic matrix estimation method as described in claim 8.

Images