Method for completing missing traffic data, terminal device and storage medium

Abstract

The application discloses a kind of missing flow data completion method, terminal equipment and storage medium, by carrying out the embedding of source node, destination node and time slot Two two outer product operation, feature map is constructed.This method more effectively simulates and excavates the flow relationship between source node, destination node and time.Further, advanced feature mining is carried out using convolutional neural network and time series convolutional neural network, so that the model has stronger expression ability.In addition, the application considers the order and numerical distribution of network traffic between different source nodes, destination nodes and time slots, and extracts potential patterns and dependencies from the structure and arrangement of flow data through comparative loss calculation, and learns the size and distribution difference of flow values.Therefore, the model can more accurately complete the data completion task in the case of missing flow data.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and missing traffic data completion, and in particular to a method, terminal device and storage medium for missing traffic data completion. Background Technology

[0002] A traffic matrix is ​​a two-dimensional matrix that represents the traffic between all possible sources and destinations within a given IP domain. It is a crucial basis for network planning, helping network designers understand network traffic demands, distribution, and trends, thus enabling appropriate capacity planning, link selection, routing strategies, and traffic engineering. Traffic matrices can also be used for network monitoring and fault diagnosis, reflecting network performance, reliability, efficiency, and potential issues such as congestion, anomalies, or attacks.

[0003] However, the traffic matrix may contain missing elements for various reasons. For example, (1) some network devices in the network do not support flow-level traffic measurement. In this case, traffic data cannot be obtained from these devices, resulting in the missing elements in the traffic matrix. (2) Data acquisition system failure or unreliable transmission protocol leads to the loss of measurement data. In this case, due to system or network problems, traffic data may be lost or delayed during acquisition or transmission, resulting in the missing elements in the traffic matrix. (3) some network devices in the network only run measurement functions in certain time slots to reduce measurement costs. In this case, due to the device's strategy or limitations, the acquisition frequency or time interval of traffic data may be insufficient or inconsistent, resulting in the missing elements in the traffic matrix.

[0004] The existence of missing traffic data will impact downstream tasks such as network planning and management, network monitoring, and fault diagnosis. Therefore, recovering missing traffic data is crucial and of great significance.

[0005] For traffic matrices, the current mainstream recovery method relies on data modeling based on third-order tensors. The source, target, and time slot correspond to the three dimensions of the third-order tensor, and missing data is imputed using this third-order tensor. Tensor imputation methods are mainly implemented in three categories: The first category is models based on deep embedding networks. These methods use deep learning models, typically including embedding layers and convolutional neural networks, to extract high-order embedding features and fit the missing traffic data using these features. The second category is models based on temporal embedding networks. These methods use embedding layers and recurrent neural networks such as long short-term memory networks to handle the temporal relationships of traffic data. They can capture the evolution and dependencies of data over different time intervals. Through temporal embedding, the model can better understand the dynamic characteristics of traffic data and thus fit the missing data more accurately. The third category is models based on generative adversarial networks (GANs). GANs are powerful generative models that include a generator and a discriminator. In this case, the generator is used to generate missing traffic data, while the discriminator is used to evaluate the realism of the generated data. Through adversarial training, the model can generate high-quality missing data and improve the accuracy of missing data imputation.

[0006] However, the existing third-order tensor completion techniques mentioned above all have some significant technical problems when used for missing traffic completion. Regarding the first type of method, due to its simple design, it does not simulate the cross-domain traffic relationships between originating nodes, destination nodes, and time slots. For example, some traffic may primarily occur at specific originating or destination nodes on a particular workday, or a specific pair of originating and destination nodes may communicate frequently. Ignoring these patterns leads to inaccurate recovery results. Furthermore, it fails to adequately consider the size and distribution of traffic data. This is crucial for effective network management and congestion control. Large traffic volumes, such as video transmissions, consume more bandwidth and resources, potentially leading to network congestion and latency. In contrast, small traffic volumes, such as web page traffic, have less impact on bandwidth and resources and network load, but require faster response times. Ignoring the size and distribution of traffic data can affect the accuracy of recovery and impact downstream tasks. Regarding the second type of method, it considers the temporal characteristics of traffic, but recurrent neural networks typically have too many parameters, making them prone to overfitting, especially with limited data. This may result in the model performing well on training data but exhibiting poor generalization performance on unseen data. Furthermore, similar to the first type of method, it also fails to adequately simulate the cross-domain traffic relationships between originating nodes, destination nodes, and time slots, and does not fully consider the size and distribution of traffic data. Regarding the third type of method, its computational complexity is too high, and it requires careful design of the generator and discriminator based on the size of the IP domain. This may limit its feasibility in practical applications. Additionally, it shares the same problems as the first two methods: it fails to adequately simulate the cross-domain traffic relationships between originating nodes, destination nodes, and time slots, and it does not fully consider the size and distribution of traffic data.

[0007] Currently, some studies enhance the expressive power of models through cross-domain augmentation. However, in the field of traffic data completion, effectively combining network communication pattern analysis with domain modeling and fusion to support a more accurate understanding and completion of traffic data is a highly complex task. Furthermore, one method utilizes contrastive learning for point cloud missing data completion. This method uses training missing point cloud data from the same original point cloud dataset as positive samples and training missing point cloud data from other original point cloud datasets as negative samples, performing contrastive learning to train the network model to recover the missing point cloud data. However, in the field of traffic data completion, traffic data may originate from different networks and devices, and its distribution may vary significantly. Such variability makes it difficult to establish an effective similarity metric in contrastive learning. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method, terminal device and storage medium for completing missing traffic data, which addresses the shortcomings of the prior art. This solves the problems of the prior art not fully simulating the cross-domain traffic relationship between the origin node, destination node and time slot, being prone to overfitting and having poor accuracy, and the prior art not fully considering the size relationship and distribution of traffic data.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for completing missing traffic data, comprising the following steps:

[0010] S1. Construct a tensor X∈R from the network traffic. I×J×K Where I, J, and K represent the number of source nodes, target nodes, and time slots, respectively, and each entity x in the tensor... ijk Let X represent the flow from source node i to destination node j in time slot k. In tensor X, the set of observed entities is Ω.

[0011] S2. Identify the missing data entities in tensor X, that is, the data entities that are not in the entity set Ω, which are the missing entities, and mark the missing entities in the form of triples;

[0012] S3. Input the missing entities marked in the form of triples into the trained missing traffic data completion model to complete the missing traffic data;

[0013] The missing traffic data completion model includes:

[0014] Embedded mapping module, used to map tensors X∈R I×J×K Each source node, destination node, and time slot in the vector space is mapped to an independent vector space, and an embedding of dimension d is obtained, resulting in the embedding of source node i as a. i The embedding of the target node j is b. j The embedding of time slot k is c k ;

[0015] Cross-domain embedding module, used to obtain feature maps The feature map is then expanded by stacking the feature maps of the first t time series to obtain a time series feature map. Extract feature maps High-level vector representation u ij Extracting temporal feature maps The output vector characterizes u jk and u ik Serial splicing u ij u jk and u ik The splicing feature v is obtained. ijk ;in, Indicates the outer product operation;

[0016] The comparison module is used to compare v ijk Define the corresponding tensor entity, anchor nodes, positive sample sets, and negative sample sets, calculate the loss function, and optimize v. ijk Positive sample set B + From the current training batch, away from v ijk The corresponding tensor entity consists of the m nearest entities; negative sample set B - In the current training batch, excluding anchor node u and the positive sample set B + External entity composition;

[0017] The interaction module is used to process the optimized v ijk Input the data into the multilayer sensor and obtain the output flow rate value.

[0018] This invention proposes a cross-domain embedding module that details the cross-domain traffic relationships between source nodes, destination nodes, and time slots in a tensor. This enhances the model's expressive power and addresses the problems of existing technologies failing to adequately simulate the cross-domain traffic relationships between origin nodes, destination nodes, and time slots, leading to overfitting and poor accuracy. It also solves the problem of existing technologies not fully considering the magnitude and distribution of traffic data. Furthermore, this invention, through a comparison module, considers the order and numerical distribution of network traffic among different source nodes, destination nodes, and time slot triples. By calculating the comparison loss, it extracts potential patterns and dependencies from the structure and arrangement of traffic data, learning the magnitude and distribution differences of traffic values, thereby improving the accuracy of missing traffic data completion.

[0019] The training process for the missing traffic data completion model includes:

[0020] The observed traffic data is randomly divided into a training set and a validation set;

[0021] The training set is used as input to the missing traffic data completion model to train the missing traffic data completion model, resulting in a trained missing traffic data completion model.

[0022] Extract feature maps High-level vector representation u ij The specific implementation process includes: converting feature maps Inputting a 2D convolution module yields a high-level vector representation u. ij ;

[0023] Extracting temporal feature maps The output vector characterizes u jk and u ik The specific implementation process includes: converting the time series feature map Input a two-dimensional temporal convolution module, flatten the output of the two-dimensional temporal convolution module, and output a high-level vector representation u. ik and u jk .

[0024] By using a 2D convolution module, the spatial structure of non-temporal feature maps can be utilized. By using a 2D temporal convolution module, the temporal information in the temporal feature maps can be better utilized, thereby improving the performance of missing traffic data completion tasks.

[0025] The expression for the loss function L of the missing traffic data completion model is:

[0026] L = L 补全损失 +λL 对比损失 ;

[0027]

[0028]

[0029] in, To complete the model output for missing traffic data, u is the anchor node, i.e., x ijk vector representation v ijk B + B is the set of positive samples corresponding to the anchor node. - Let s be the set of negative samples corresponding to the anchor node. u，w Let τ be the cosine similarity between the vector representations of anchor node u and negative sample w, where w represents the negative sample, e is the natural base, and τ is the temperature coefficient.

[0030] By minimizing the loss function, we can more accurately complete missing traffic data; on the other hand, we can bring entities with similar values ​​in the embedding space as close as possible, while keeping dissimilar entities as far apart as possible. This process can effectively capture the similarity and dissimilarity between network traffic entities, more accurately extract potential patterns and dependencies from the structure and arrangement of traffic data, and learn the magnitude and distribution differences of traffic values, thereby achieving more accurate and reliable traffic data completion. In this invention, τ is set to 0.001.

[0031] In this invention, the multilayer perceptron includes three cascaded fully connected layers, with the last fully connected layer connected to the tanh activation function.

[0032] In step S3, the trained missing traffic data completion model includes an embedding mapping module, a cross-domain embedding module, and an interaction module connected in sequence.

[0033] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the above method.

[0034] As an inventive concept, the present invention also provides a computer storage medium storing a program for performing the steps of the method described above.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] 1. This invention proposes a cross-domain enhancement technique. A feature map is constructed by performing pairwise outer product operations on the embeddings of source nodes, destination nodes, and time slots. This method more effectively simulates and mines the flow relationships between source nodes, destination nodes, and time. Furthermore, advanced feature mining is performed using convolutional neural networks and temporal convolutional neural networks, giving the model stronger expressive power. Therefore, this model can more accurately complete data completion tasks even when flow data is missing.

[0037] 2. This invention proposes a contrastive learning method for optimizing traffic embedding. This method defines samples with traffic values ​​close to those of the anchor node as positive samples and samples with more distant values ​​as negative samples, and then performs contrastive learning optimization. This processing ensures that in the feature space, traffic samples with similar traffic values ​​are as close as possible, while samples with more distant values ​​are correspondingly further apart. Thus, the model can learn the magnitude and distribution differences of traffic values, thereby achieving more accurate and reliable traffic data completion.

[0038] 3. This invention proposes a missing traffic data completion model based on cross-domain enhancement and contrastive learning. Compared to traditional tensor-based traffic completion models, this model more effectively mines the communication relationships between nodes and can more accurately understand the traffic volume and distribution differences between nodes. Therefore, this model has higher accuracy and reliability when performing missing traffic data completion tasks. Attached Figure Description

[0039] Figure 1 This is a flowchart of the missing traffic data completion method based on cross-domain enhancement and contrastive learning in an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the model structure during the online execution phase of the traffic data completion model in an embodiment of the present invention;

[0041] Figure 3 It is the two-dimensional convolution module in the completion model of this invention embodiment;

[0042] Figure 4This is the two-dimensional temporal convolution module in the completion model of this invention embodiment;

[0043] Figure 5 This is a schematic diagram of the model structure during the training phase of the traffic data completion model in an embodiment of the present invention;

[0044] Figure 6 This is a performance comparison of the traffic data completion model of this invention with other completion algorithms. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] This invention proposes a method for missing traffic data completion based on cross-domain enhancement and contrastive learning. The method employs a pairwise joint embedding approach for cross-domain enhancement, thereby capturing more traffic patterns. Simultaneously, the method proposes dividing samples into positive and negative based on proximity information and utilizing contrastive learning to train the embedding, ensuring that traffic data with similar values ​​are located closer together in the embedding space, thus preserving the order and distribution of the traffic data.

[0048] like Figure 1 As shown, Embodiment 1 of the present invention provides a method for missing traffic data completion based on cross-domain enhancement and contrastive learning, including the following steps:

[0049] Step (1) Construct a tensor X∈R from the network traffic. I×J×K Each entity x in the tensor ijk Let X represent the flow from source node i to destination node j in time slot k. In tensor X, the set of observed entities is Ω.

[0050] Step (2) identifies missing data entities in the tensor X obtained in step (1). These missing data entities can be labeled by comparing them with the observed entity set Ω. Any data entity not in Ω is considered missing and labeled as a triple (i, j, k), indicating that there is a missing flow from node i to destination node j in time slot k.

[0051] Step (3) inputs the triplet (i, j, k) marked in step (2) into the trained missing traffic data completion model to complete the missing traffic data. The trained missing traffic data completion model consists of three modules: an embedding mapping module, a cross-domain embedding module, and an interaction module. When it is necessary to complete the traffic data corresponding to (i, j, k), the following steps are performed:

[0052] ① Embedded mapping. For the missing traffic entity x ijk First, through the embedding mapping module, the input triples (i, j, k) are mapped to three independent vector spaces, resulting in three independent d-dimensional embeddings. For example... Figure 2 As shown in (a), the embedding of the source node i is obtained as a. i The embedding of the target node j is b. j The embedding of time slot k is c k This step aims to obtain vector representations of the source node i, the destination node j, and the time slot k.

[0053] ② Pairwise joint embedding. The embedding a of the source node i obtained in the previous step... i The embedding of the target node j in b j Embedding of time slot k c k The input is fed into the cross-domain embedding module. The cross-domain embedding module receives a i b j and c k As input, such as Figure 2 As shown in (b), calculate the outer product pairwise to obtain... As a feature map, where This refers to the outer product operation. Pairwise outer products can simulate cross-domain relationships between traffic, because some source and destination nodes may communicate frequently, some destination nodes may receive more frequent communication within a certain time period, and some source nodes may have specific communication patterns at certain times. To fully extract time-series information, for feature maps containing time... and Additional stacking of the feature maps of the corresponding first t time series, i.e. After the above steps, for embedding a i b j and c k Feature maps can be obtained. and time series feature map Next, feature map and time series feature map They will be processed separately. For feature maps... Input as follows Figure 3The illustrated 2D convolutional module performs advanced feature extraction. This module contains three convolutional layers with channels [32, 16, 8], a kernel size of 2, and a stride of 1. Finally, the output of the 2D convolutional module is flattened to output a high-level vector representation u. ij For time series feature maps Input as follows Figure 4 The two-dimensional temporal convolution module shown is used for processing. This module contains two temporal convolutional layers and one attention layer. The kernel size of the temporal convolutional layers is 1*2, the stride is 1, and the offset factor is d=2. i-1 Where i represents the i-th layer. Finally, the output of the two-dimensional temporal convolution module is flattened to output the high-level vector representation u. ik and u jk Three high-level vector representations of u ij u jk and u ik The concatenation will be spliced ​​into v ijk As x ijk The cross-domain embedding module can simulate cross-domain relationships between traffic through pairwise outer products. For example, some traffic may mainly occur at specific origin or destination nodes on a specific workday, or a specific pair of origin and destination nodes may communicate frequently. Two-dimensional convolutional modules and two-dimensional temporal convolutional modules can further capture these traffic relationships, obtaining better vector representations to improve the accuracy of data completion.

[0054] ③ Interactive function mapping. For example... Figure 2 As shown in (c), x ijk vector representation v ijk The missing flow value is output through a multilayer perceptron containing three fully connected layers and a tanh activation function.

[0055] Specifically, the missing traffic data completion model in step (3) is trained through the following steps:

[0056] Step (3-1) involves tensor modeling of the observed traffic data, which is then divided into a training set and a validation set at a ratio of 9:1. In this embodiment, a total of 12 source nodes and 12 destination nodes were randomly collected, and 10% of the traffic data from 3000 time slots was used as the dataset, which was then divided into a training set and a validation set at a ratio of 9:1.

[0057] Step (3-1) inputs the training set obtained after processing in step (3-1) into the missing traffic data completion model.

[0058] Step (3-3) uses the backpropagation algorithm to update and optimize the model parameters to complete the missing traffic data, so as to obtain the updated model.

[0059] Specifically, the model parameters are initialized using the Xavier method. To accelerate the descent of the loss value, the backpropagation algorithm in this step uses Adam as the optimizer, due to its advantages of momentum and adaptive learning rate.

[0060] Step (3-4) iteratively trains the device identification model updated in step (3-3) until the loss function of the model reaches its minimum.

[0061] The loss function for the missing traffic data completion model is L, which is composed of the completion loss L. 补全损失 And contrast loss L 对比损失 It consists of two parts.

[0062] ①Complete the loss L 补全损失 :

[0063]

[0064] Where, x ijk This represents the traffic from source node i to destination node j in time slot k. To complete the model's output, Ω represents the set of observed tensor entities, i.e., the observed flow.

[0065] ② Comparison of loss L 对比损失 :

[0066]

[0067] Where u is the anchor node, i.e., x ijk vector representation v ijk B + Let be the set of positive samples corresponding to anchor node u. This set is determined by the distance from x in the current training batch. ijk It consists of the m most recent entities. In this embodiment, m is 2, which means B + Includes x ijk The two most recent training samples. Positive samples are represented by v. B - For anchor node x ijk The corresponding set of negative samples. This is derived from the set of negative samples in the current training batch, excluding x. ijk It consists of entities outside the positive sample set. Negative samples are represented by w. s u，v S represents the cosine similarity between the anchor node u and the positive sample v vector. u，wLet be the cosine similarity between the vector representations of anchor node u and negative sample w. Let e ​​be the natural base. τ is the temperature coefficient, set to 0.001 in this embodiment. By using contrastive loss, samples with similar flow rates can be made as close as possible in the feature space, while samples with dissimilar flow rates can be made as far apart as possible. This allows the model to learn the order and distribution relationship of flow rates.

[0068] Finally, the loss function L for the missing traffic data completion model:

[0069] L = L 补全损失 +λL 对比损失

[0070] Here, λ is a learnable weighting factor used to balance traffic completion and the relationship between the size of the learned traffic and the weights distributed during the training process.

[0071] Step (3-5) uses the validation set in the dataset obtained in step (3-4) to iteratively validate the missing traffic completion model after iterative training until the completion accuracy reaches the optimal level, thereby obtaining the trained completion model, which is then used to recover the missing traffic data.

[0072] For the model of this invention, its training phase includes four modules (such as...). Figure 5 (As shown): Embedded mapping module, cross-domain embedding module, comparison module, interaction module.

[0073] Module 1 is the embedding mapping module. This module will embed tensors X∈R. I×J×K Each source node, destination node, and time slot in the vector space is mapped to an independent vector space, yielding an embedding of dimension d. For example, the embedding of source node i is a. i The embedding of the target node j is b. j The embedding of time slot k is c k This module will output an embedded a i b j and c k .

[0074] Module 2 is the cross-domain embedding module. This module first performs an outer product on the embeddings obtained from Module 1 to obtain the feature map. Through the pairwise outer product operations described above, cross-domain relationships between traffic can be simulated to enhance the model's expressive power. For example, some traffic may primarily occur at specific origin or destination nodes on a particular workday, or a specific pair of origin and destination nodes may communicate frequently. Next, this module processes the feature map containing temporal information... Perform temporal extension by stacking the feature maps of the first t time steps to obtain the temporal feature map. Then, the feature map is processed. For the feature map... The output will be a high-level vector representation u, passed through three convolutional layers with channel numbers of [32, 16, 8] respectively, a kernel size of 2, and a stride of 1. ij For time series feature maps This will be achieved through a two-dimensional temporal convolution submodule, which contains two TCN layers. The convolution kernels in each TCN layer are 1x2 in size, with a stride of 1 and an offset factor of d = 2. i-1 Here, i represents the i-th layer. Above the two TCN layers, there is an attention layer used to weight the influence of different time sequences, and the output vector represents u. jk and u ik Three high-level vector representations of u ij u jk and u ik The concatenation will be spliced ​​into v ijk It is then sent to the next module.

[0075] Module 3 is the comparison module. This module first compares v. ijk Corresponding x ijk Define anchor nodes, positive sample sets, and negative sample sets. Anchor node u is x. ijk Positive sample set B + From the current training batch, distance x ijk The set consists of the m most recent entities. The negative sample set B. - In the current training batch, excluding anchor node u and the positive sample set B + The external entity composition. This module calculates the loss function:

[0076]

[0077] Optimize the vector representation v of the anchor node ijk This ensures that its vector representation is as close as possible to the vector representation of the positive sample set in the feature space, and as far away as possible from the vector representation of the negative sample set in the feature space, thereby learning the magnitude relationship and distribution of the flow. Here, u is the anchor node, i.e., x... ijk vector representation v ijk B + Let be the set of positive samples corresponding to anchor node u. This set is determined by the distance from x in the current training batch. ijk It consists of the m most recent entities. Positive samples are represented by v. B - For anchor node x ijk The corresponding set of negative samples. This is derived from the set of negative samples in the current training batch, excluding x. ijk It consists of entities outside the positive sample set. Negative samples are represented by w. s u，v Let s be the cosine similarity between the anchor node u and the positive sample v vector. u，w Let be the cosine similarity between the vector representations of anchor node u and negative sample w. Let e ​​be the natural base. τ be the temperature coefficient.

[0078] Module four is the interactive module. ijk vector representation v ijk The flow rate is output through a multilayer perceptron containing three fully connected layers and a tanh activation function.

[0079] During the online execution phase, the embedding mapping module, cross-domain embedding module, and interaction module will be retained, while the comparison module will be removed because the online execution traffic data completion phase does not require backpropagation to update the model or to calculate the comparison loss.

[0080] Test Results

[0081] In the testing section, NMAE and NRMSE were used to evaluate the difference between the imputed and true values. Additionally, KL divergence was used to measure the difference between the imputed and true traffic distributions, and NDCG@200 was used to measure the ranking accuracy of the top 200 largest flows. Smaller NMAE and NRMSE values ​​indicate higher imputed accuracy. Smaller KL divergence values ​​indicate a closer approximation between the imputed and true traffic distributions. Larger NDCG@200 values ​​indicate higher accuracy in ranking the size of the imputed traffic flows.

[0082]

[0083]

[0084]

[0085] Where Ω represents the set of observed flows, x ijk Represents actual traffic data. This represents the estimated missing flow.

[0086]

[0087] Among them, y i This represents the i-th largest actual traffic. This represents the estimated i-th largest actual flow after completion.

[0088] Figure 6 This paper demonstrates the performance of the missing traffic data completion algorithm proposed in this invention compared to other completion algorithms on three open-source datasets. The three datasets are described below:

[0089] (1) Abilene. Records communication between 12 nodes of the US backbone network, collecting traffic every 5 minutes for 24 weeks. The first 3000 time slots are selected in the performance test section.

[0090] (2) GEANT. The communication between 23 nodes of the European GEANT network was recorded, with traffic collected every 15 minutes for 15 months. The first 3000 time slots were selected for the performance test.

[0091] (3) WS-DREAM. It records the traffic communication between 142 user nodes and 4500 web service nodes. It is collected once every 15 minutes, for a total of 64 times, that is, there are a total of 64 time slots.

[0092] Figure 6 The figure illustrates the model performance under three datasets with sampling rates of [2%, 4%, 6%, 8%, 10%], specifically the model's completion performance for the remaining unobserved traffic when the observed proportions are [2%, 4%, 6%, 8%, 10%]. As shown in the figure, the missing traffic data completion model based on cross-domain enhancement and contrastive learning proposed in this embodiment of the invention has higher data completion accuracy, the completed traffic data distribution is closer to the real traffic data distribution, and the ranking of traffic size is more accurate.

[0093] Table 1 shows the model performance of the embodiments of the present invention on three datasets with a missing proportion of 90%. It can be seen that the embodiments of the present invention achieve low traffic data completion error, low KL divergence, and high NDCG@200. Specifically, Abilene, with a 90% missing data ratio, has a completion error NMAE of only 0.1692, an NRMSE of only 0.2062, a KL divergence as low as 0.0471, and an NDCG@200 as high as 0.9918. This fully demonstrates that the missing traffic data completion model based on cross-domain enhancement and contrastive learning proposed in the embodiments of the present invention can achieve high completion accuracy and can learn the order of traffic size and traffic distribution, thus possessing a certain degree of reliability.

[0094] Table 1. Model performance of the present invention embodiment with a dataset missing rate of 90%.

[0095] Abilene 0.1692 0.2062 0.0471 0.9918 GEANT 0.2014 0.2128 0.1280 0.9972 WS-DREAM 0.2515 0.3425 0.1029 0.9206

[0096] Example 2

[0097] Embodiment 2 of the present invention provides a terminal device corresponding to Embodiment 1 above. The terminal device can be a processing device for a client, such as a mobile phone, a laptop, a tablet computer, a desktop computer, etc., to execute the method of the above embodiments.

[0098] The terminal device in this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in Embodiment 1 described above.

[0099] In some implementations, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0100] In other implementations, the processor can be any type of general-purpose processor, such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation here.

[0101] Example 3

[0102] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to Embodiment 1 above, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, they implement the steps of the method of Embodiment 1 above.

[0103] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0104] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0108] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for completing missing traffic data, characterized in that, Includes the following steps: S1. Construct a tensor from the network traffic. Where I, J, and K represent the number of source nodes, target nodes, and time slots, respectively, and each entity in the tensor... This represents the flow from source node i to destination node j in time slot k, expressed in a tensor. In the middle, the set of observed entities is ; S2. Identify the missing data entities in tensor X, i.e., those not in the entity set. The data entities in the data are missing entities, and the missing entities are marked in the form of triples; S3. Input the missing entities marked in the form of triples into the trained missing traffic data completion model to complete the missing traffic data; The missing traffic data completion model includes: Embedded mapping module, used to map tensors Each source node, destination node, and time slot in the vector space is mapped to an independent vector space, and an embedding of dimension d is obtained. The embedding of source node i is then obtained as follows: The embedding of the target node j is The embedding of time slot k is ; Cross-domain embedding module, used to obtain feature maps , , The feature map is then expanded by stacking the feature maps of the first t time series to obtain a time series feature map. Extract feature maps High-level vector representation Extracting temporal feature maps The output vector characterization and Serial splicing , and To obtain splicing features ;in, Indicates the outer product operation; The comparison module is used to compare based on Define the corresponding tensor entity, anchor nodes, positive sample sets, and negative sample sets, calculate the loss function, and optimize. Positive sample set From the current training batch, away The corresponding tensor entity consists of the m nearest entities; negative sample set In the current training batch, excluding anchor node u and the positive sample set External entity composition; The interaction module is used to provide the optimized... Input the data into the multilayer sensor to obtain the output flow rate value; The expression for the loss function L of the missing traffic data completion model is: ； ； ； in, To complete the model output for missing traffic data, u is the anchor node. Vector representation of splicing features , This is the set of positive samples corresponding to the anchor node. The set of negative samples corresponding to the anchor node. Let e ​​be the cosine similarity between the vector representations of anchor node u and negative sample w, where w represents the negative sample and e is the natural base. This is the temperature coefficient.

2. The method for completing missing traffic data according to claim 1, characterized in that, The training process for the missing traffic data completion model includes: The observed traffic data is randomly divided into a training set and a validation set; The training set is used as input to the missing traffic data completion model to train the missing traffic data completion model, resulting in a trained missing traffic data completion model.

3. The method for completing missing traffic data according to claim 1, characterized in that, Extract feature maps High-level vector representation The specific implementation process includes: converting feature maps Inputting a 2D convolution module yields a high-level vector representation. ; Extracting temporal feature maps The output vector characterization and The specific implementation process includes: converting the time series feature map The input is a two-dimensional temporal convolutional module, i.e., a 2D-TCN module. The output of the 2D-TCN module is flattened to output a high-level vector representation. and .

4. The method for completing missing traffic data according to claim 1, characterized in that, Set to 0.

001.

5. The method for completing missing traffic data according to claim 1, characterized in that, The multilayer perceptron includes three cascaded fully connected layers, with the last fully connected layer connected to a tanh activation function.

6. The method for completing missing traffic data according to any one of claims 1 to 5, characterized in that, In step S3, the trained missing traffic data completion model includes an embedding mapping module, a cross-domain embedding module, and an interaction module connected in sequence.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory; characterized in that, The processor executes a computer program in memory to implement the steps of the method described in any one of claims 1 to 6.

8. A computer storage medium, characterized in that, It stores a program for performing the steps of the method as described in any one of claims 1 to 6.

Images