A tunnel traffic association method and device based on a noise reduction model

Abstract

The present invention provides a method and apparatus for tunnel traffic association based on a noise reduction model. The method includes: obtaining a plurality of ingress node flows and a plurality of egress node flows of a pre-established network tunnel, where the network tunnel is used for a client to access a corresponding network; determining at least one candidate ingress node flow corresponding to each egress node flow, and inputting each egress node flow into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the egress node flow to obtain a mapped ingress node flow; respectively calculating the statistical distance between the mapped ingress node flow and at least one candidate ingress node flow, screening the candidate ingress node flows according to the statistical distance, and taking the candidate ingress node flow corresponding to the minimum statistical distance as the target ingress node flow associated with the egress node flow; by having different encoding layers responsible for removing different types of noise, the accuracy of the association result can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network traffic measurement and analysis, and particularly to a correlation method and device for tunnel traffic based on a noise reduction model. Background Art

[0002] The tunneling technology is one of the commonly used means to protect user privacy. Especially for multi-hop tunnels, it can well hide user behaviors and protect user privacy. However, due to its privacy protection characteristics, tunnel services are often misused for activities such as network attacks and information theft. Therefore, determining the hidden communication relationships in tunnel services and tracing the origin of abnormal network behaviors are of great significance for maintaining network security.

[0003] As more and more tunnel nodes are built on public network services such as CDN and public clouds, abnormal network behaviors are mixed with normal network traffic, bringing huge challenges to the tracing work. Since most tunnel services adopt strong encryption measures, it is not realistic to analyze the data source based on content; the method of tracing traffic hop by hop requires the correlator to master a large number of node resources, which is also difficult to achieve in real scenarios. Therefore, at present, only the traffic correlation technology can achieve the goal of confirming communication relationships under limited conditions. Traffic correlation is the process of corresponding the traffic of the tunnel entrance node to the traffic of the exit node related to it (belonging to the same communication session), using content-independent features and only completing at both ends of the tunnel.

[0004] The current traffic correlation methods are mainly based on statistical distance methods, regarding the statistical distance as an index to measure the correlation of two flows. Common indicators include cosine distance, Pearson coefficient, mutual information, etc. Each data flow is abstracted into a vector describing its morphological characteristics (such as packet direction sequence, packet size sequence, packet interval sequence, etc.), and then the similarity is calculated using the calculation formula of the corresponding index. The method based on statistical distance only captures shallow external features, and slight network noise will significantly change the vector representation of the data flow, affecting the correlation accuracy. Summary of the Invention

[0005] The present invention provides a correlation method and device for tunnel traffic based on a noise reduction model to solve the defect of low correlation accuracy of tunnel traffic in the prior art.

[0006] The present invention provides a correlation method for tunnel traffic based on a noise reduction model, including:

[0007] Obtaining a plurality of entrance node flows and a plurality of exit node flows of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network;

[0008] Determine at least one candidate inlet node flow corresponding to each outlet node flow, and input each of the outlet node flows into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the outlet node flows, obtaining the mapped inlet node flows;

[0009] Calculate the statistical distances between the mapped inlet node flows and at least one candidate inlet node flow respectively, screen the candidate inlet node flows according to the statistical distances, and take the candidate inlet node flow corresponding to the minimum statistical distance as the target inlet node flow associated with the outlet node flow;

[0010] Wherein, the noise reduction model is used to learn the mapping relationship between the associated sample inlet node flows and sample outlet node flows; the noise reduction model includes: a first encoding layer and a second encoding layer, the first encoding layer is used to remove network noise, and the second encoding layer is used to remove confusion noise to restore the input outlet node flow to the corresponding mapped inlet node flow.

[0011] According to an association method for tunnel traffic provided by the present invention, obtaining a plurality of inlet node flows and a plurality of outlet node flows of the network tunnel includes:

[0012] At each inlet node, statistically calculate the packet size sequence and the corresponding packet time sequence of the inlet node data stream; define a first time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each first time window to obtain the cumulative transmission volume of the inlet data stream within each first time window, and take the sequence of the cumulative transmission volumes of the inlet data stream within a plurality of first time windows as the inlet node flow;

[0013] At each outlet node, statistically calculate the packet size sequence and the corresponding packet time sequence of the outlet node data stream; define a second time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each second time window to obtain the cumulative transmission volume of the outlet data stream within each second time window, and take the sequence of the cumulative transmission volumes of the inlet data stream within a plurality of second time windows as the outlet node flow.

[0014] According to an association method for tunnel traffic provided by the present invention, the associated sample inlet node flows and sample outlet node flows include: a first sample inlet node flow and a first sample outlet node flow containing network noise, and a second sample inlet node flow and a second sample outlet node flow containing network noise and confusion noise;

[0015] The training method of the noise reduction model includes:

[0016] Input the first sample inlet node stream and the first sample outlet node stream containing network noise into the first encoding layer to be trained, so as to obtain the trained first encoding layer;

[0017] Input the second sample inlet node stream and the second sample outlet node stream containing network noise and confusion noise into the trained first encoding layer and the second encoding layer to be trained in sequence, so as to obtain the trained second encoding layer.

[0018] According to a correlation method for tunnel traffic provided by the present invention, inputting the first sample inlet node stream and the first sample outlet node stream containing network noise into the first encoding layer to be trained to obtain the trained first encoding layer includes:

[0019] Use the first sample outlet node stream containing network noise as the input, and use the first sample inlet node stream as the optimization target to be fitted;

[0020] Input the first sample inlet node stream and the first sample outlet node stream into the first encoding layer to be trained, and determine the optimal parameter set of the first encoding layer by minimizing the squared difference loss between the output of the first encoding layer and the first sample inlet node stream, so as to obtain the trained first encoding layer.

[0021] According to a correlation method for tunnel traffic provided by the present invention, inputting the second sample inlet node stream and the second sample outlet node stream containing network noise and confusion noise into the trained first encoding layer and the second encoding layer to be trained in sequence to obtain the trained second encoding layer includes:

[0022] Input the second sample outlet node stream containing network noise and confusion noise into the trained first encoding layer to obtain the second sample outlet node stream with network noise removed;

[0023] Use the second sample outlet node stream with network noise removed as the input, use the second sample inlet node stream as the optimization target to be fitted, and input the second sample inlet node stream and the second sample outlet node stream with network noise removed into the second encoding layer to be trained;

[0024] Determine the optimal parameter set of the second encoding layer by minimizing the squared difference loss between the output of the second encoding layer and the second sample inlet node stream, so as to obtain the trained second encoding layer.

[0025] According to a correlation method for tunnel traffic provided by the present invention, input each of the outlet node streams into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the outlet node streams to obtain the mapped inlet node streams, including:

[0026] Input each of the said exit node flows into the first encoding layer to remove network noise, obtaining corresponding noise-reduced exit node flows;

[0027] Input each of the noise-reduced exit node flows into the second encoding layer to remove confusion noise, obtaining the mapped entry node flows.

[0028] The present invention also provides a tunnel traffic association device based on a noise reduction model, including:

[0029] A node flow acquisition module, configured to acquire a plurality of entry node flows and a plurality of exit node flows of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network;

[0030] A noise reduction processing module, configured to determine at least one candidate entry node flow corresponding to each exit node flow, input each of the exit node flows into a pre-trained noise reduction model to sequentially perform network noise and confusion noise removal processing on the exit node flows, obtaining mapped entry node flows;

[0031] A node flow screening module, configured to respectively calculate the statistical distance between the mapped entry node flows and at least one candidate entry node flow, screen the candidate entry node flows according to the statistical distance, and use the candidate entry node flow corresponding to the minimum statistical distance as the target entry node flow associated with the exit node flow;

[0032] Wherein, the noise reduction model is used to learn the mapping relationship between associated sample entry node flows and sample exit node flows; the noise reduction model includes: a first encoding layer and a second encoding layer, the first encoding layer is used to remove network noise, and the second encoding layer is used to remove confusion noise to restore the input exit node flows to corresponding mapped entry node flows.

[0033] According to a tunnel traffic association device provided by the present invention, the noise reduction processing module is specifically configured to: input each of the exit node flows into the first encoding layer to remove network noise, obtaining corresponding noise-reduced exit node flows; input each of the noise-reduced exit node flows into the second encoding layer to remove confusion noise, obtaining the mapped entry node flows.

[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps of any one of the above-mentioned tunnel traffic association methods based on a noise reduction model are implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the tunnel traffic association method based on a noise reduction model as described in any one of the above are implemented.

[0036] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the tunnel traffic association method based on a noise reduction model as described in any one of the above are implemented.

[0037] For the tunnel traffic association method and device based on a noise reduction model provided by the present invention, when the inlet node flow and the outlet node flow are obtained, at least one candidate inlet node flow corresponding to each outlet node flow is determined. Then, the outlet node flow is processed to remove network noise and confusion noise to obtain a mapped inlet node flow. Then, a target inlet node flow associated with the outlet node flow is determined from the at least one candidate inlet node flow. Since confusion noise is generated in the traffic initiation stage and network noise is generated in the traffic transmission stage, different coding layers are responsible for removing different types of noise. Compared with the existing methods, the accuracy of the association result can be improved.

[0038] Secondly, in the embodiments of the present invention, by defining the first time window and the second time window as units to respectively count the cumulative transmission amount of the inlet data stream and the cumulative transmission amount of the outlet data stream, this granularity is independent of the number of data packets included in the data stream and the arrival order of these data packets. Therefore, in the case of packet loss, recombination, retransmission, etc., compared with the representation in terms of data packets, the description of the cumulative transmission amount of the data stream is more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0040] Figure 1 is one of the flow diagrams of the tunnel traffic association method based on a noise reduction model provided by the embodiments of the present invention;

[0041] Figure 2 is a schematic diagram of the tunnel traffic provided by the embodiments of the present invention;

[0042] Figure 3 is the second flow diagram of the tunnel traffic association method based on a noise reduction model provided by the embodiments of the present invention;

[0043] Figure 4 is one of the flow diagrams of the training method of the noise reduction model provided by the embodiments of the present invention;

[0044] Figure 5 It is the second flow schematic diagram of the training method of the noise reduction model provided by the embodiment of the present invention;

[0045] Figure 6 It is the third flow schematic diagram of the training method of the noise reduction model provided by the embodiment of the present invention;

[0046] Figure 7 It is the structural schematic diagram of the tunnel traffic association device based on the noise reduction model provided by the embodiment of the present invention;

[0047] Figure 8 It is the structural schematic diagram of the electronic device provided by the embodiment of the present invention. Detailed implementation manners

[0048] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0049] The following combines Figures 1-7 to describe the tunnel traffic association method and device based on the noise reduction model in the embodiments of the present invention.

[0050] The embodiments of the present invention disclose a tunnel traffic association method based on a noise reduction model. Refer to Figure 1 , including:

[0051] Step 101, obtain multiple inlet node flows and multiple outlet node flows of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network.

[0052] It should be explained that a network tunnel is a way to transfer data between networks through an Internet infrastructure. Through the network tunnel, a client can access network-side services. In this embodiment, the network tunnel can be simply understood as a path from an inlet node to an outlet node. Correspondingly, there is an inlet node flow at the inlet node and an outlet node flow at the outlet node.

[0053] In this embodiment, an automated script is used to generate and capture the network tunnel inlet node flow for accessing a specific website at the client, and at the same time, capture the tunnel outlet node flow at the post-positioned outlet of the network tunnel.

[0054] In a network tunnel, there are multiple ingress node flows and egress node flows. It is possible that one ingress node flow is associated with multiple egress node flows, or multiple ingress node flows are associated with one egress node flow. The purpose of the embodiments of the present invention is to find the most relevant one-to-one relationship in a many-to-one relationship.

[0055] Specifically, taking the case where one egress node flow is associated with multiple ingress node flows as an example in the embodiments of the present invention, for each egress node flow with multiple candidate ingress node flows having an association relationship, the ingress node flow with the most relevant relationship is determined as the target ingress node flow, so as to determine that this ingress node flow and the egress node flow belong to the same communication session. Refer to Figure 2 , Figure 2 In, a network tunnel is formed between host A, host B, host C and host D, host E, host F. Among them, for host F as the egress node, the ingress nodes associated with it include host A, host B, and host C, that is, (A, F), (B, F), and (C, F). Through the method of this embodiment, the ingress node (host A) with the most relevant relationship to the egress node (host F) can be determined.

[0056] Of course, the method of this embodiment is also applicable to the situation where one ingress node flow is associated with multiple egress node flows. Since the processing steps are basically the same, this embodiment will not repeat the method of selecting the target egress node flow with the most relevance to the ingress node flow from multiple egress node flows.

[0057] Step 102: Determine at least one candidate ingress node flow corresponding to each egress node flow, and input each of the egress node flows into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the egress node flows, and obtain mapped ingress node flows.

[0058] Among them, the noise reduction model is used to learn the mapping relationship between associated sample ingress node flows and sample egress node flows; the noise reduction model includes: a first encoding layer and a second encoding layer, the first encoding layer is used to remove network noise, and the second encoding layer is used to remove confusion noise to restore the input egress node flow to the corresponding mapped ingress node flow.

[0059] Specifically, the sample ingress node flows and sample egress node flows in this embodiment include the first sample ingress node flow and the first sample egress node flow containing network noise as a non-confusion node flow sequence pair, and the second sample ingress node flow and the second sample egress node flow containing network noise and confusion noise as a confusion node flow sequence pair, and the first encoding layer and the second encoding layer are trained with the non-confusion node flow sequence and the confusion node flow sequence respectively.

[0060] When constructing the training data, a sequence pair composed of two associated inlet node flows and outlet node flows is used as a positive sample sequence pair; in addition, a part of the inlet node flows are separately selected, and 99 unassociated outlet node flows are randomly assigned to each inlet node flow to form a negative sample sequence pair, simulating the situation where the traffic generated by the communication relationship to be traced accounts for 1% of the total traffic captured at the gateway.

[0061] During training, the loss function adopted corresponds to the statistical distance method for subsequent association operations: if the Euclidean distance is used to determine the similarity of two node flows, the mean squared error function is set as the loss function of the preprocessing noise reduction model. Similarly, if the cosine distance is used to determine the similarity of two node flows, the cosine similarity function is set as the loss function of the preprocessing noise reduction model; if the information entropy is used to determine the similarity of two node flows, the mutual information function is set as the loss function of the preprocessing noise reduction model.

[0062] Specifically, step 102 includes: inputting each of the outlet node flows into the first encoding layer to remove network noise, obtaining the corresponding noise-reduced outlet node flows; inputting each of the noise-reduced outlet node flows into the second encoding layer to remove confounding noise, obtaining the mapped inlet node flows.

[0063] In this embodiment, the confounding noise is generated in the traffic initiation stage, and the network noise is generated in the traffic transmission stage. Therefore, in this embodiment, the first encoding layer and the second encoding layer of the noise reduction model are used to perform noise reduction on the outlet node flows in stages, and the encoding layers of different layers are responsible for fitting different types of noise. Compared with the existing method of directly training an end-to-end neural network model, the staged noise reduction has higher association accuracy for the confounding traffic, shorter flow observation time and faster association speed for the non-confounding traffic.

[0064] Step 103: Calculate the statistical distance between the mapped inlet node flows and at least one candidate inlet node flow respectively, screen the candidate inlet node flows according to the statistical distance, and use the candidate inlet node flow corresponding to the minimum statistical distance as the target inlet node flow associated with the outlet node flow.

[0065] In this embodiment, the statistical distance can be realized by calculating the Euclidean distance, cosine similarity or mutual information. The statistical distance can describe the degree of association between two node flows.

[0066] In this embodiment, converting the outlet node flows into mapped inlet node flows and then calculating the statistical distance between the mapped inlet node flows and at least one candidate inlet node flow can enhance the similarity of the relevant node flows and ensure the association effect in the scenario where the network noise is relatively complex.

[0067] The tunnel traffic association method and device based on a noise reduction model provided by an embodiment of the present invention, when obtaining the inlet node flow and the outlet node flow, determine at least one candidate inlet node flow corresponding to each outlet node flow, and then perform noise removal processing on the outlet node flow for network noise and confusion noise to obtain a mapped inlet node flow, and then determine a target inlet node flow associated with the outlet node flow from at least one candidate inlet node flow. Since confusion noise is generated in the traffic initiation stage and network noise is generated in the traffic transmission stage, different coding layers are responsible for removing different types of noise. Compared with the existing method, the accuracy of the association result can be improved.

[0068] Before step 101, the method further includes: filtering the data packets of the data stream of the inlet node and the data packets of the data stream of the outlet node, filtering out data packets without actual payloads such as TCP acknowledgment packets, and filtering out retransmission packets such as TCP Retransmission and Dup ACK. The filtered data packets are aggregated into a unidirectional data stream in the upstream or downstream direction according to the five-tuple information to obtain a data stream set.

[0069] Wherein, the five-tuple information refers to the source IP address, the destination IP address, the source port number, the destination port number, and the transport layer protocol.

[0070] Filter out data streams with too short lengths in the obtained data stream set, that is, obtain the inlet node data stream and the outlet node data stream.

[0071] Further, referring to Figure 3 , step 101 includes:

[0072] 301. At each inlet node, count the data packet size sequence of the inlet node data stream and the corresponding data packet time sequence; define a first time window, and perform cumulative calculation according to the data packet size sequence corresponding to the data packet time sequence within each first time window to obtain the cumulative transmission volume of the inlet data stream within each first time window, and use the cumulative transmission volume sequence of the inlet data stream within multiple first time windows as the inlet node flow.

[0073] Specifically, assume that the packet size sequence of a certain inlet node data stream is pkt_size and the data packet time sequence is pkt_interval. Cumulative calculation is performed on the two sequences respectively, that is, each element value is added to its previous element value in sequence to obtain new sequences pkt_size′ and pkt_interval′. There is such a correlation between the two new sequences: at the moment of pkt_interval′[i], this data stream has transmitted pkt_size′[i] bytes.

[0074] Then, define a fixed first time window win_size. At the end of each first time window, count the total number of bytes that have been transmitted in this data stream. Assume that the data stream has transmitted f(x*win_size) bytes at time x*win_size, then f(x*win_size) = pkt_size′[i], pkt_interval′[i] ≤ x*win_size < pkt_interval′[i + 1], and so on until the cumulative transmission volume sequence reaches a predetermined length.

[0075] For example, taking the number of bytes per 5 ms as 10, the obtained ingress node stream can be [10, 20, 30, 40, 50…, 200].

[0076] Furthermore, after obtaining the cumulative transmission volume sequence of the ingress data stream, perform a normalization operation on the cumulative transmission volume sequence of the ingress data stream. Replace each sequence value in the sequence with the percentage of the data transmission volume at a certain moment in the total volume, which can better fit the data volume burst during the transmission process. Generally speaking, the former pays more attention to the absolute growth trend of the data volume change, while the latter focuses on the relative growth trend, that is, the change of the data growth rate.

[0077] 302. At each egress node, count the packet size sequence of the egress node data stream and the corresponding packet time sequence; define a second time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each second time window to obtain the cumulative transmission volume of the egress data stream within each second time window. Use the cumulative transmission volume sequences within multiple second time windows as the egress node stream.

[0078] The specific method for obtaining the egress node stream is basically the same as the specific method for obtaining the ingress node stream, and will not be elaborated in this embodiment.

[0079] Through steps 301 to 302, the embodiments of the present invention respectively count the cumulative transmission volume of the ingress data stream and the cumulative transmission volume of the egress data stream by defining the first time window and the second time window as units. This granularity has nothing to do with how many data packets are included in the data stream and the arrival order of these data packets. Therefore, in the case of packet loss, recombination, retransmission, etc., compared with the characterization in terms of data packets, its characterization of the cumulative transmission volume of the data stream is more stable.

[0080] Furthermore, referring to Figure 4 , the training method of the noise reduction model in this embodiment includes:

[0081] 401. Input the first sample ingress node stream and the first sample egress node stream containing network noise into the first encoding layer to be trained to obtain a trained first encoding layer.

[0082] Specifically, referring to Figure 5 , step 401 includes:

[0083] 501. Use the first sample exit node stream containing network noise as the input, and use the first sample entry node stream as the optimization target to be fitted.

[0084] 502. Input the first sample entry node stream and the first sample exit node stream into the first encoding layer to be trained. By minimizing the squared difference loss between the output of the first encoding layer and the first sample entry node stream, determine the optimal parameter set of the first encoding layer to obtain the trained first encoding layer.

[0085] Regarding the problem that network noise affects the correlation effect, this method regards the first sample entry node stream as a traffic sample without noise, and regards the first sample exit node stream as a sample entry node stream mixed with network noise. In the first encoding layer, with the first sample exit node stream as the input and the related first sample entry node stream as the output, the first encoding layer obtains the mapping relationship from the first sample exit node stream to the first sample entry node stream on the training data. After the first encoding layer is trained, the process of transforming from the input to the output according to the learned mapping relationship is actually a process of denoising the exit node stream to make it closer to the form of the entry node stream.

[0086] Specifically, the denoising process of the autoencoder network is shown in formula (1).

[0087] ω a = argmin θ ∑ (α,β)∈NO (α - De(En(β,θ),θ T )) 2 (1)

[0088] Among them, θ represents the set of parameters of the model that makes the entire formula take the minimum value. That is, when the distance value represented by the formula is optimized to the minimum, θ at this moment is used as the optimal parameter set ω a ; θ T represents the transpose of the parameter matrix θ. Here, it means that the parameter of the function De is the transpose of the parameter of the function En, representing that the encoding and decoding processes are symmetric.

[0089] ω a represents the set of parameters of the first encoding layer Encoder A , NO represents the set of non-confused data flow sequence pairs, and the functions De / En represent the decoding / encoding processes of the first encoding layer Encoder A .

[0090] After obtaining the associated first sample inlet node stream and the first sample outlet node stream, a data stream sequence pair in the form of (α, β) is formed. Use it to train the first encoding layer Encoder A . During training, the first sample outlet node stream β is regarded as the input with noise, and the first sample inlet node stream α is used as the optimization target to be fitted. By minimizing the square difference loss between the output of the first encoding layer Encoder A and the optimization target, the optimal parameter set of the first encoding layer is determined to obtain the trained first encoding layer.

[0091] 402. Input the second sample inlet node stream and the second sample outlet node stream containing network noise and confusion noise into the trained first encoding layer and the second encoding layer to be trained in sequence to obtain the trained second encoding layer.

[0092] When training the second encoding layer, the first encoding layer Encoder A can be used to remove the network noise from the second sample node stream pair first, and then the second sample node stream pair after removing the network noise is used to train the second encoding layer Encoder B . In this way, the second encoding layer Encoder B only needs to fit the confusion noise.

[0093] Specifically, referring to Figure 6 , step 402 includes:

[0094] 601. Input the second sample outlet node stream containing network noise and confusion noise into the trained first encoding layer to obtain the second sample outlet node stream after removing the network noise.

[0095] Specifically, the second sample outlet node stream after removing the network noise is obtained by referring to the following formula (2):

[0096] δ′ = De(En(δ, ω a ), ω a T ) (2)

[0097] where δ is the second sample outlet node stream of the confusion noise;

[0098] δ′ is the second sample outlet node stream after removing the network noise;

[0099] ω a represents the parameter set of the first encoding layer Encoder A ;

[0100] After δ passes through the encoding En and decoding De of the first encoding layer, the second sample outlet node stream δ′ after removing the network noise is generated.

[0101] 602. Take the second sample exit node stream after removing network noise as the input, take the second sample entry node stream as the optimization target to be fitted, and input the second sample entry node stream and the second sample exit node stream after removing network noise into the second encoding layer to be trained.

[0102] 603. Determine the optimal parameter set of the second encoding layer by minimizing the squared difference loss between the output of the second encoding layer and the second sample entry node stream, so as to obtain the trained second encoding layer.

[0103] Specifically, the second encoding layer refers to the following formula (3):

[0104] ω b = argmin θ ∑ (γ,δ)∈O (γ - De(En(δ′, θ), θ T )) 2 (3)

[0105] where ω b represents the parameter set of the second encoding layer Encoder B ; O represents the set of node streams for training, γ represents the second sample entry node stream; δ′ represents the second sample exit node stream after removing network noise; θ represents the parameter set of the model that makes the entire formula take the minimum value, that is, when the distance value represented by the formula is optimized to the minimum, θ at this moment is used as the optimal parameter set ω b ; θ T represents the transpose of the parameter matrix θ. Here, it means that the parameter of the function De is obtained after the transpose of the parameter of the function En, representing that the encoding and decoding processes are symmetric.

[0106] After training, it is also necessary to test the output effects of the first encoding layer and the second encoding layer. In the test stage, the non-confused node stream and the confused node stream can be distinguished. For the non-confused node stream sequence pair to be tested, it only needs to be processed by Encoder A to generate the confused noise; for the confused node stream sequence pair to be tested, it needs to go through the removal processes of the confused noise and the network noise of Encoder A and Encoder B in sequence.

[0107] The processed confused node stream sequence pair can use the traditional statistical distance method to calculate the similarity between the two node streams in the sequence pair to determine whether they are associated, and further determine the training effects of the first encoding layer and the second encoding layer.

[0108] Through the method of this embodiment, experiments have proved that for non-confused node flows, both this method and the existing traffic association method based on convolutional neural networks can achieve a true positive rate of more than 90%, while the false positive rate is controlled within 10 -2 or less; when achieving the same effect, the length of the time-sharing byte cumulative sequence used in this method is 200, and combined with a time window size of 0.05 s, the required flow observation length can be obtained as 10 seconds; while the existing method needs to observe 300 data packets in each node flow, and according to the dataset situation, observing 300 data packets takes from 11 to 64 seconds, so this method can use more short data flows for faster traffic association. For confused node flows, the true positive rate of this method is better than that of the existing convolutional neural network method.

[0109] Next, the tunnel traffic association device based on the noise reduction model provided by the embodiment of the present invention will be described. The tunnel traffic association device based on the noise reduction model described below can be correspondingly referred to the tunnel traffic association method based on the noise reduction model described above.

[0110] The embodiment of the present invention provides a tunnel traffic association device based on a noise reduction model. Refer to Figure 7 , including:

[0111] A node flow acquisition module 701, configured to acquire a plurality of inlet node flows and a plurality of outlet node flows of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network;

[0112] A noise reduction processing module 702, configured to determine at least one candidate inlet node flow corresponding to each outlet node flow, and input each outlet node flow into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the outlet node flow to obtain a mapped inlet node flow;

[0113] A node flow screening module 703, configured to calculate the statistical distance between the mapped inlet node flow and at least one candidate inlet node flow respectively, screen the candidate inlet node flows according to the statistical distance, and use the candidate inlet node flow corresponding to the smallest statistical distance as the target inlet node flow associated with the outlet node flow;

[0114] Wherein, the noise reduction model is used to learn the mapping relationship between the associated sample inlet node flow and the sample outlet node flow; the noise reduction model includes: a first coding layer and a second coding layer, the first coding layer is used to remove network noise, and the second coding layer is used to remove confusion noise to restore the input outlet node flow to the corresponding mapped inlet node flow.

[0115] Optionally, the node flow acquisition module 701 is specifically configured to:

[0116] At each ingress node, count the packet size sequence of the ingress node data stream and the corresponding packet time sequence; define a first time window, and perform cumulative calculation based on the packet size sequence corresponding to the packet time sequence within each first time window to obtain the cumulative transmission volume of the ingress data stream within each first time window, and use the cumulative transmission volume sequence of the ingress data stream within multiple first time windows as the ingress node stream;

[0117] At each egress node, count the packet size sequence of the egress node data stream and the corresponding packet time sequence; define a second time window, and perform cumulative calculation based on the packet size sequence corresponding to the packet time sequence within each second time window to obtain the cumulative transmission volume of the egress data stream within each second time window, and use the cumulative transmission volume sequence of the ingress data stream within multiple second time windows as the egress node stream.

[0118] Optionally, the apparatus further includes: a model training module, configured to input a first sample ingress node stream and a first sample egress node stream including network noise into a first encoding layer to be trained, so as to obtain a trained first encoding layer;

[0119] Input a second sample ingress node stream and a second sample egress node stream including network noise and confusion noise into the trained first encoding layer and a second encoding layer to be trained in sequence, so as to obtain a trained second encoding layer.

[0120] Optionally, the model training module is specifically configured to: use the first sample egress node stream including network noise as an input, and use the first sample ingress node stream as an optimization target to be fitted;

[0121] Input the first sample ingress node stream and the first sample egress node stream into the first encoding layer to be trained, and determine an optimal parameter set of the first encoding layer by minimizing the mean squared error loss between the output of the first encoding layer and the first sample ingress node stream, so as to obtain a trained first encoding layer.

[0122] Optionally, the model training module is specifically configured to: input the second sample egress node stream including network noise and confusion noise into the trained first encoding layer to obtain a second sample egress node stream with network noise removed;

[0123] Use the second sample egress node stream with network noise removed as an input, use the second sample ingress node stream as an optimization target to be fitted, and input the second sample ingress node stream and the second sample egress node stream with network noise removed into the second encoding layer to be trained;

[0124] Determine the optimal parameter set of the second encoding layer by minimizing the squared difference loss between the output of the second encoding layer and the second sample entry node stream, so as to obtain the trained second encoding layer.

[0125] Optionally, the noise reduction processing module 702 is specifically configured to: input each of the exit node streams into the first encoding layer to remove network noise, so as to obtain corresponding denoised exit node streams; input each of the denoised exit node streams into the second encoding layer to remove confusion noise, so as to obtain the mapped entry node streams.

[0126] The tunnel traffic association device based on the noise reduction model provided by the embodiment of the present invention, when obtaining the entry node stream and the exit node stream, determines at least one candidate entry node stream corresponding to each exit node stream, and then performs network noise and confusion noise removal processing on the exit node stream to obtain the mapped entry node stream, and then determines the target entry node stream associated with the exit node stream from at least one candidate entry node stream. Since the confusion noise is generated in the traffic initiation stage and the network noise is generated in the traffic transmission stage, different encoding layers are responsible for removing different types of noise. Compared with the existing method, the accuracy of the association result can be improved.

[0127] Figure 8 An example of the physical structure diagram of an electronic device is shown in Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 complete mutual communication through the communication bus 840. The processor 810 may call the logical instructions in the memory 830 to execute the tunnel traffic association method based on the noise reduction model, including: obtaining a plurality of entry node streams and a plurality of exit node streams of a pre-established network tunnel; where the network tunnel is used for a client to access a corresponding network; determining at least one candidate entry node stream corresponding to each exit node stream, inputting each of the exit node streams into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the exit node stream to obtain a mapped entry node stream; respectively calculating the statistical distance between the mapped entry node stream and at least one candidate entry node stream, screening the candidate entry node streams according to the statistical distance, and using the candidate entry node stream corresponding to the smallest statistical distance as the target entry node stream associated with the exit node stream.

[0128] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0129] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tunnel traffic association method based on a noise reduction model provided by the above-mentioned various methods, including: obtaining multiple ingress node flows and multiple egress node flows of a pre-established network tunnel; wherein the network tunnel is used for a client to access a corresponding network; determining at least one candidate ingress node flow corresponding to each egress node flow, inputting each egress node flow into a pre-trained noise reduction model to sequentially remove network noise and confounding noise from the egress node flow to obtain a mapped ingress node flow; respectively calculating the statistical distance between the mapped ingress node flow and at least one candidate ingress node flow, screening the candidate ingress node flows according to the statistical distance, and using the candidate ingress node flow corresponding to the minimum statistical distance as the target ingress node flow associated with the egress node flow.

[0130] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for performing the tunnel traffic association method based on the noise reduction model provided by the above-mentioned various methods, including: obtaining a plurality of ingress node flows and a plurality of egress node flows of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network; determining at least one candidate ingress node flow corresponding to each egress node flow, and inputting each egress node flow into a pre-trained noise reduction model to sequentially perform network noise and confusion noise removal processing on the egress node flow to obtain a mapped ingress node flow; respectively calculating the statistical distance between the mapped ingress node flow and at least one candidate ingress node flow, screening the candidate ingress node flow according to the statistical distance, and using the candidate ingress node flow corresponding to the smallest statistical distance as the target ingress node flow associated with the egress node flow.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A tunnel traffic correlation method based on a noise reduction model, characterized in that, Including: Obtain a plurality of ingress node flows and a plurality of egress node flows of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network; Determine at least one candidate ingress node flow corresponding to each egress node flow, and input each of the egress node flows into a pre-trained noise reduction model to sequentially remove network noise and obfuscation noise from the egress node flows, so as to obtain a mapped ingress node flow; Calculate the statistical distances between the mapped ingress node flow and at least one candidate ingress node flow respectively, screen the candidate ingress node flows according to the statistical distances, and use the candidate ingress node flow corresponding to the smallest statistical distance as the target ingress node flow associated with the egress node flow; Wherein, the noise reduction model includes: a first encoding layer and a second encoding layer, the first encoding layer is used to remove network noise, and the second encoding layer is used to remove obfuscation noise, so as to restore the input egress node flow to the corresponding mapped ingress node flow; Wherein, obtaining a plurality of ingress node flows and a plurality of egress node flows of the network tunnel includes: At each ingress node, count the packet size sequence of the ingress node data stream and the corresponding packet time sequence; define a first time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each first time window to obtain the cumulative transmission volume of the ingress data stream within each first time window, and use the cumulative transmission volume sequence of the ingress data stream within a plurality of first time windows as the ingress node flow; At each egress node, count the packet size sequence of the egress node data stream and the corresponding packet time sequence; define a second time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each second time window to obtain the cumulative transmission volume of the egress data stream within each second time window, and use the cumulative transmission volume sequence of the egress data stream within a plurality of second time windows as the egress node flow.

2. The tunnel traffic association method according to claim 1, wherein The sample ingress node flow and the sample egress node flow for training the noise reduction model include: a first sample ingress node flow and a first sample egress node flow containing network noise, and a second sample ingress node flow and a second sample egress node flow containing network noise and obfuscation noise; The training method of the noise reduction model includes: Input the first sample ingress node flow and the first sample egress node flow containing network noise as training data into the to-be-trained first encoding layer to obtain a trained first encoding layer; Input the second sample ingress node flow and the second sample egress node flow containing network noise and obfuscation noise as training data into the trained first encoding layer and the to-be-trained second encoding layer in sequence to obtain a trained second encoding layer.

3. The tunnel traffic correlation method according to claim 2, wherein Inputting the first sample ingress node flow and the first sample egress node flow containing network noise as training data into the to-be-trained first encoding layer to obtain a trained first encoding layer includes: Using the first sample outlet node stream containing network noise as the input and the first sample inlet node stream as the optimization target to be fitted; Inputting the first sample inlet node stream and the first sample outlet node stream into the first encoding layer to be trained, and determining the optimal parameter set of the first encoding layer by minimizing the mean squared error loss between the output of the first encoding layer and the first sample inlet node stream, so as to obtain the trained first encoding layer.

4. The tunnel traffic association method according to claim 2, wherein Using the second sample inlet node stream and the second sample outlet node stream containing network noise and confusion noise as training data and inputting them into the trained first encoding layer and the second encoding layer to be trained in sequence, so as to obtain the trained second encoding layer, including: Inputting the second sample outlet node stream containing network noise and confusion noise into the trained first encoding layer to obtain the second sample outlet node stream with network noise removed; Using the second sample outlet node stream with network noise removed as the input, the second sample inlet node stream as the optimization target to be fitted, and inputting the second sample inlet node stream and the second sample outlet node stream with network noise removed into the second encoding layer to be trained; Determining the optimal parameter set of the second encoding layer by minimizing the mean squared error loss between the output of the second encoding layer and the second sample inlet node stream, so as to obtain the trained second encoding layer.

5. The tunnel traffic correlation method according to claim 1, wherein Inputting each of the outlet node streams into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the outlet node streams to obtain the mapped inlet node streams, including: Inputting each of the outlet node streams into the first encoding layer to remove network noise and obtaining the corresponding noise-reduced outlet node streams; Inputting each of the noise-reduced outlet node streams into the second encoding layer to remove confusion noise and obtaining the mapped inlet node streams.

6. A tunnel traffic association device based on a noise reduction model, characterized in that, Including: A node stream acquisition module for acquiring multiple inlet node streams and multiple outlet node streams of a pre-established network tunnel; wherein, the network tunnel is used for a client to access a corresponding network; A noise reduction processing module for determining at least one candidate inlet node stream corresponding to each outlet node stream, inputting each of the outlet node streams into a pre-trained noise reduction model to sequentially remove network noise and confusion noise from the outlet node streams to obtain the mapped inlet node streams; A node stream screening module for respectively calculating the statistical distances between the mapped inlet node streams and at least one candidate inlet node stream, screening the candidate inlet node streams according to the statistical distances, and using the candidate inlet node stream corresponding to the minimum statistical distance as the target inlet node stream associated with the outlet node stream; Wherein, the noise reduction model includes: a first encoding layer and a second encoding layer, the first encoding layer is used to remove network noise, and the second encoding layer is used to remove confusion noise to restore the input outlet node stream to the corresponding mapped inlet node stream; The node stream acquisition module is specifically used for: At each ingress node, count the packet size sequence of the ingress node data stream and the corresponding packet time sequence; define a first time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each first time window to obtain the cumulative transmission volume of the ingress data stream within each first time window, and use the cumulative transmission volume sequence of the ingress data stream within multiple first time windows as the ingress node flow; At each egress node, count the packet size sequence of the egress node data stream and the corresponding packet time sequence; define a second time window, and perform cumulative calculation according to the packet size sequence corresponding to the packet time sequence within each second time window to obtain the cumulative transmission volume of the egress data stream within each second time window, and use the cumulative transmission volume sequence of the egress data stream within multiple second time windows as the egress node flow.

7. The tunnel traffic correlation device according to claim 6, wherein The noise reduction processing module is specifically configured to: Input each of the egress node flows into the first encoding layer to remove network noise, and obtain the corresponding noise-reduced egress node flows; Input each of the noise-reduced egress node flows into the second encoding layer to remove confusion noise, and obtain the mapped ingress node flows.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps of the tunnel traffic association method based on the noise reduction model according to any one of claims 1 to 5 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the tunnel traffic association method based on the noise reduction model according to any one of claims 1 to 5 are implemented.

Images