Information bottleneck-based interpretable street-level IP address positioning method

Abstract

The invention belongs to the field of neural networks in machine learning, and discloses an interpretable street-level IP address positioning method based on information bottleneck, which mainly utilizes a graph neural network and an information bottleneck theory to carry out geographic positioning on a target IP and discriminate important road sign nodes influencing a positioning result. Screening out a landmark IP matched with the target IP by using a composition mode taking the target IP as the center; secondly, an attention weight matrix between nodes is obtained by calculating the attribute similarity of a target IP and a landmark IP; thirdly, an information bottleneck theory is applied to the attention weight matrix, and important road sign nodes influencing target IP geographic positioning are gradually obtained; and finally, performing linear affine change by using a multi-layer perceptron, and decoding in the hidden space to obtain latitude and longitude coordinates of the target IP. According to the method, the problem of sparse network nodes in rural and remote areas in IP positioning is relieved, and the interpretability of the prediction model is provided.

Description

Technical Field

[0001] The present invention belongs to the field of neural networks in machine learning and is a method based on deep learning. It mainly utilizes graph neural networks (GNN) and information bottleneck theory (Information Bottleneck theory) to geolocate the target IP while identifying important landmark nodes that affect the positioning results. Background Art

[0002] IP geolocation is a technology that determines the geographic location of a target device through its IP protocol address. It provides an estimate of the device's location. Its main purpose is to estimate the geographic location through the user's IP address in scenarios that are independent of the client. Such methods usually do not rely on users to actively share their location data, but use alternative methods to determine the location without direct customer input. In practical applications, IP geolocation plays an important role in advertising recommendations, network scenario division, etc. In addition, it can also be applied to public safety aspects such as crime tracking and social governance.

[0003] With the popularity of Social Network Service (SNS), many location-aware applications, such as spectrum allocation, smart cities, Geo web, etc., require the accurate geographic location of Internet hosts. Widely used client-dependent geolocation technologies obtain location data from sources such as the Global Positioning System (GPS), cell phone towers, and Wi-Fi. These technologies rely on users to voluntarily enable location services and share their location data. However, not all users are willing to do so due to privacy concerns or other reasons. In addition, client-dependent systems rely on devices with specific hardware capabilities to determine location. Devices without these features, such as basic IoT devices or some embedded systems, cannot participate in client-dependent geolocation. In addition, client-dependent geolocation systems may have difficulty accurately locating devices indoors, because GPS signals may not be accessible indoors, and Wi-Fi or cellular signals may be less reliable due to multipath effects.

[0004] Traditional IP geolocation methods can be divided into two types. One method uses different information collected from the Internet to locate IPs. This information is usually obtained from public databases, including domain names, autonomous systems, IP addresses, and Internet registration information. The other method relies on network delay measurements and uses multiple techniques with predefined delay-distance conversion functions to accurately locate IP addresses.

[0005] With the development of machine learning, neural networks (NNs) are used to reduce the empirical errors of the assumption rules designed by experts. The present invention uses graph neural networks to establish the connection between the target host and the surrounding coordinates, reconstructing the router-centric mapping method in the past research into a target node-centric mapping method, which significantly reduces the problem of too sparse IP maps in rural or remote areas. Summary of the invention

[0006] The purpose of the present invention is to reduce the sparsity of the IP graph by relying on a target node-centered graph composition method while introducing the information bottleneck theory to explain the model prediction results, that is, to find out the key landmark nodes that affect the results.

[0007] The idea of ​​the present invention is to use graph neural networks to model the network topology relationship and aggregate neighbor node information, and then update the node representation by calculating the attention weight matrix. On this basis, the variational information bottleneck strategy in the information bottleneck theory is introduced to limit the amount of information contained in the potential representation to achieve the interpretability of the model. Finally, the compressed potential representation is decoded by a multi-layer perceptron (MLP) to obtain the prediction result.

[0008] Based on the above invention ideas, the present invention proposes an interpretable street-level IP address positioning method based on information bottleneck, including the following steps: first, apply a composition method centered on the target IP to screen out landmark IPs that match the target IP; second, obtain the attention weight matrix between nodes by calculating the attribute similarity between the target IP and the landmark IP; third, apply the information bottleneck theory in the attention weight matrix, and gradually obtain the important landmark nodes that affect the geographic positioning of the target IP; finally, use a multi-layer perceptron to perform linear affine transformation, and decode the longitude and latitude coordinates of the target IP from the latent space. The present invention alleviates the problem of sparse network nodes in rural and remote areas in IP positioning, and provides the interpretability of the prediction model.

[0009] First, we define the present invention as a regression task, which is defined as follows:

[0010] Following the previous setup of IP location machine learning models, we define the geolocation task with the aid of a set of reference hosts (i.e., landmarks) with known locations. We classify some landmarks as active (unknown locations) with the ability to initiate network measurements to other landmarks, and others as passive (known locations) that can be detected by active landmarks. Then, we give N landmarks Its knowledge (extracted from the Internet), network measurements and the location marked by longitude and latitude The data-driven model then determines the T and network measurement m T The destination IP of the value:

[0011]

[0012] in, represents the predicted geographic location of the target IP, and Θ represents all learnable parameters.

[0013] So far, we have defined the IP location task. Now let’s introduce its model process in detail, which includes the following steps:

[0014] S1, first, we obtained attribute knowledge from an open source database (WHIOS website), and then obtained network measurement data by collecting trace route data from four probe hosts across different regions. The attribute knowledge and network measurement data together constitute the dataset.

[0015] S2, data preprocessing stage, different encoding methods are used for the original data according to different data properties and feature categories. Category encoding is used for IP addresses, autonomous system numbers, Internet service providers, etc. In addition, in order to solve the problem of inconsistent feature latitudes of some data sets, one-hot encoding is also used for features such as autonomous system names and addresses;

[0016] S3, obtaining the index. The present invention uses a serial number to identify each node, and obtains the index of the training and test data sets according to the random seed, the ratio of the training set to the test set, and the ratio of the target point to the landmark point.

[0017] S4, obtain neighbor nodes, and use the target node-centric mapping method to model the network topology relationship. Different from the previous method of limiting the mapping range by aggregating IP nodes at the router level, the present invention designates the target host as the central node and integrates multiple last-hop routers to establish a target-centric IP graph. At the same time, in order to solve the problem of computational complexity, the target-centric mapping method is only used for the target node with less than 10 adjacent nodes, while other target nodes rely on the router-centric mapping method;

[0018] S5, after the composition is completed, according to the attributes of the IP nodes, including Internet data and network measurement data, the attention mechanism is used to calculate the attention score between the target node and the landmark node in the IP graph, which is defined as:

[0019] A T,l =exp(v T σ(W1{x T ,mT}+W2{x l ,m l}+b))(2)

[0020] in and are trainable matrices and vectors. So far, we have obtained the attention weight matrix of the IP graph through formula (2).

[0021] S6, then we introduced the information bottleneck theory. In previous studies, the information bottleneck at the graph level is defined as:

[0022]

[0023] According to formula (3), minimize This is equivalent to maximizing the first term Minimize the second term maximize That is, maximizing the mutual information between the graph representation and the label, thereby achieving high prediction accuracy and minimizing This means minimizing the information between the graph representation and the original attributes of the graph, that is, achieving information compression. Through this formula, we can achieve information compression while ensuring the accuracy of the model. For graph neural networks, a graph consists of So we can define G IB Defined as:

[0024] Where X -. is a set of features related to the task, A -. The essence of the method for achieving interpretability in the present invention is to learn a sparse adjacency matrix to find the key neighbor nodes that affect the prediction results, namely A -. , this matrix effectively eliminates unnecessary topological neighbors and retains only the most relevant landmarks. Therefore, we can define the formula to find the optimal adjacency matrix:

[0025]

[0026] So far, we have obtained the interpretability of this model through the information bottleneck theory, but directly estimating this mutual information poses a practical challenge to the computational complexity of the model, so we use the variational information bottleneck method to achieve this goal.

[0027] S7, for formula (4), we mainly focus on the second part I (A IB ; A), we can get through the proof:

[0028]

[0029] We use the training dataset to approximate Then, by building the model To estimate Then minimize I(A IB ; A) can be optimized by analyzing the KL divergence:

[0030]

[0031] We set is a priori Bernoulli distribution with probability parameter p = 0.5, indicating that the connection between nodes is equally likely to be retained or predicted. As the model training deepens, the optimized target mask matrix is ​​finally obtained.

[0032] S8, after passing the information bottleneck we get The mask matrix is ​​obtained by performing the Hadamard product with the attention weight matrix obtained by formula (2) to obtain the attention weight matrix after the information bottleneck. This matrix retains the key nodes that play an important role in the prediction results and filters out the redundant non-critical nodes. So far, we have achieved the interpretability of the model output results through the information bottleneck theory.

[0033] S9, after achieving interpretability through the variational information bottleneck method, we use the obtained attention weight matrix to propagate information in the graph neural network and update the target features.

[0034] S10, after the target feature is updated, we send it to the Multilayer Perceptron for output. MLP consists of three layers, each layer contains multiple neurons (or nodes), and each neuron is connected to all neurons in the previous layer. Each layer of MLP includes two main components: linear transformation layer and activation function. The input data is passed from the first layer of the network to the last layer layer by layer, and finally the output of the model is generated.

[0035] At this point, we have implemented an explainable street-level IP address location method based on information bottleneck.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. The present invention effectively alleviates the problem of reduced accuracy caused by sparse network nodes in rural and border areas. The present invention designates the target host as the central node and integrates multiple last-hop routers to establish a target-centric IP map, reducing the proportion of isolated nodes (no neighbor nodes under one router) from 9.73% to 4.46%, and reducing the sparse rate (the number of neighbor nodes under one router is less than 10) from 14.24% to 8.64%, thereby improving the accuracy of geolocation.

[0038] 2. The present invention is based on a variational graph information bottleneck method, which utilizes the existing training data set to approximate the distribution of the mask matrix, optimizes the mask matrix through KL divergence, and finally obtains a mask matrix that can be used to explain the output results, thereby achieving the interpretability of the model.

[0039] 3. While achieving interpretability, the present invention can flexibly retain different numbers of nodes according to the prior Bernoulli distribution with different probability parameters. In the experiment of the present invention, we set it to p=0.5. In actual application, we can set it to a relatively small value to retain a small number of the most critical nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 (a) Construct the target node IP graph for the router-centric and target node-centric graph construction methods respectively.

[0041] Figure 1 (b) The process of the model achieving interpretability through the information bottleneck.

[0042] Figure 1 (c) The process of model training.

[0043] Explanation of terms

[0044] GNN is the abbreviation of Graph Neural Networks, which means "graph neural network". It is a neural network framework specially designed for processing graph structure data. It can effectively capture the complex relationships between nodes. The design of this network structure is inspired by the fields of graph theory and neural networks, and is widely used in tasks such as node classification, graph classification, link prediction and graph generation.

[0045] The Information Bottleneck theory is a technique used in the field of deep learning and machine learning to improve the interpretability of models. Its core idea is to compress input data during the learning process, remove irrelevant information, and retain information that is useful for the current task.

[0046] The Attention Mechanism is a mechanism used to strengthen the attention of deep learning models to input data, and is used to process sequence and structured data. This mechanism was originally inspired by the way the human brain processes information. By giving different weights to different parts of the input data, the model can dynamically focus on different parts of the input data, thereby improving the performance of the model. DETAILED DESCRIPTION

[0047] The present invention is further described below in conjunction with the accompanying drawings.

[0048] Example

[0049] As attached Figure 1 As shown in (a), after dividing the data set, we designate the target host as the central node, and then find the last hop address with the minimum delay between the target node and the landmark node, and match the two. If they are the same, they can be regarded as IP nodes under the same router. For target nodes with more than ten landmark nodes, there is no need to add nodes for them; for sparse nodes with less than or equal to ten landmark nodes, we design a new construction method, that is, integrating multiple last-hop routers to establish an IP graph centered on the target, which can effectively reduce the sparsity of the graph.

[0050] As attached Figure 1 As shown in (b), after the composition is completed, the IP graph is sent to the GNN network to calculate the attribute similarity, and then the similarity matrix is ​​filtered. The nodes greater than the threshold are retained and the nodes less than the threshold are discarded, thus obtaining a sparse adjacency matrix. The dot product is then performed on it and the original attention matrix to obtain a sparse attention matrix.

[0051] As attached Figure 1 As shown in (c), the training process of the model is to compress the IP graph through the information bottleneck and then send it to the GNN network for feature propagation and update, and finally send it to the multi-layer perceptron to obtain the prediction result after passing through the linear transformation layer.

[0052] Application Examples

[0053] The interpretable street-level IP address positioning method based on information bottleneck provided by the embodiment is used to evaluate our model on three different IP geolocation datasets (New York, Los Angeles, and Shanghai). For data processing, we randomly select 80% of the IPs as the training set and the remaining 20% ​​as the test set. During training, we use 70% of the IPs in the training set as landmark nodes and 30% as target nodes. During testing, we use the training dataset as landmark nodes and the other datasets as target nodes. The detailed statistics of the data and landmark features are shown in Table 1. We compare ExGeo with 9 popular baselines, including 3 latency measurement methods, 3 attribute learning methods, and 3 graph learning methods. We configure the learning rate of the New York and Los Angeles datasets to 0.002 and 0.001. For the Shanghai dataset, β is kept constant at 0.01 on all three datasets. The hidden size of the GNN layer (excluding the last layer) is set to 32. In order to comprehensively evaluate the performance of our model, we use three standard evaluation indicators: root mean square error (RMSE), mean absolute error (MAE), and median error. To reduce computational complexity, we specifically apply the target-centric IP graph to targets with less than 10 neighbors, while the others are still based on the router-centric IP graph. The results are shown in bold in Table 1.

[0054] Table 1: Results of IP geolocation on three datasets

[0055]

[0056] The remaining methods in the table are described as follows:

[0057] GeoPing is a network latency-based method that determines the geographic location of a target by associating its IP address with the nearest landmark server based on latency. NCRGeo calculates the distance between the router and the landmark server and then uses multi-lateration to estimate the geographic location. XLBoost-Geo extracts information from web pages and associates the location of the target with the coordinates of the nearest landmark located near a public router. Its implementation process can be found in the literature [Yucheng Wang, Hongsong Zhu, Jinfa Wang, JieLiu, Yong Wang, and Limin Sun, "Xlboost-geo: An ip geolocation system based on extreme landmark boosting," arXiv preprint arXiv:2010.13396, 2020]

[0058] LightGBM is an advanced gradient boosting framework that uses a tree-based learning algorithm for geolocation. MLP-Geo detects the router ID between the host and the target as additional information. Tab-Net is an automatic model for learning feature interactions in tabular data. It uses self-attention neural networks for feature engineering and geolocation. The implementation process can be found in the reference [Sercan Arik and Tomas Pfister, "Tabnet: Attentiveinterpretable tabular learning," in AAAI, 2021, pp.66796687]

[0059] GNN-Geo is a framework that establishes connections between landmarks to use surrounding information to identify the location of the target. GraphGeo establishes neighborhood relationships between IP hosts from both topological and semantic perspectives. TrustGeo incorporates sources of uncertainty into the graph learning process, aiming to improve the accuracy and credibility of IP geolocation. For its implementation process, please refer to the literature [WenxinTai, Bin Chen, Fan Zhou, Ting Zhong, GoceTrajcevski, Yong Wang, and Kai Chen, "Trustgeo: Uncertainty-aware dynamic graph learning for trustworthy ipgeolocation," in SIGKDD, 2023]

[0060] From the experimental results in Table 1, it can be seen that the interpretable street-level IP address positioning method based on information bottleneck proposed by us has higher positioning accuracy than some existing methods while achieving model interpretability.

[0061] It can be seen that the present invention uses the information bottleneck theory to achieve the interpretability of the model, and uses a method of constructing a map using multiple last-hop routers to reduce the sparsity of the IP map, improve the positioning accuracy, and achieve IP geolocation that is superior to existing methods.

[0062] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An interpretable street-level IP address location method based on information bottleneck, characterized in that: The following steps are involved: First, a target IP-centered mapping method is used to screen out landmark IPs that match the target IP; Secondly, the attention weight matrix between nodes is obtained by calculating the attribute similarity between the target IP and the landmark IP; Thirdly, by applying the information bottleneck theory in the attention weight matrix, we gradually obtained the important landmark nodes that affect the geographic location of the target IP. Finally, a multi-layer perceptron is used to perform linear affine transformation and decode the longitude and latitude coordinates of the target IP from the latent space.

2. The interpretable street-level IP address location method based on information bottleneck according to claim 1 is characterized in that: The specific steps include: S1, obtained attribute knowledge from the open source database WHIOS website, and obtained network measurement data by collecting trace route data through four detection hosts across different regions. The attribute knowledge and network measurement data together constitute the dataset; S2, data preprocessing stage, different encoding methods are used for the original data according to different data properties and feature categories. Category Encoding is used for IP addresses, autonomous system numbers, Internet service providers, etc. In addition, One-Hot Encoding is used for features such as autonomous system names and addresses to address the problem of inconsistent feature latitudes in some data sets; S3, obtain the index, use the serial number to identify each node, and obtain the index of the training and test data sets according to the random seed, the ratio of the training set to the test set, and the ratio of the target point to the landmark point; S4, obtain neighbor nodes, use the target node-centric composition method to model the network topology relationship, designate the target host as the central node, and integrate multiple last-hop routers to build a target-centric IP graph. At the same time, only the target node with less than 10 neighbor nodes uses the target-centric composition method, while other target nodes rely on the router-centric composition method; S5, after the composition is completed, according to the attributes of the IP nodes, including Internet data and network measurement data, the attention mechanism is used to calculate the attention score between the target node and the landmark node in the IP graph, which is defined as: From T,l =exp(v T σ(W1{x T ,m T }+W2{x l ,m l }+b)) (2) in, and are trainable matrices and vectors; so far, the attention weight matrix of the IP graph is obtained through formula (2); S6, then, the information bottleneck theory is introduced; information bottleneck at the graph level is defined as: According to formula (3), minimize This is equivalent to maximizing the first term Minimize the second term maximize That is, maximizing the mutual information between the graph representation and the label, thereby achieving high prediction accuracy and minimizing This means minimizing the information between the graph representation and the original attributes of the graph, that is, achieving information compression. Through this formula, information compression can be achieved while ensuring the accuracy of the model. For graph neural networks, a graph consists of Definition, so we can put G IB Defined as: Where X IB is a set of features related to the task, A IB The essence of the method to achieve interpretability is to learn a sparse adjacency matrix to find the key neighbor nodes that affect the prediction results, namely A IB , this matrix effectively eliminates unnecessary topological neighbors and retains only the most relevant landmarks; therefore, the following formula can be defined to find the optimal adjacency matrix: So far, the interpretability of this model has been obtained through the information bottleneck theory; the variational information bottleneck method is used to achieve this goal; S7, for formula (4), we mainly focus on the second part, (A IB ; A), through proof we can get: Use the training dataset to approximate Then, by building the model To estimate Then minimize, (A IB ; A) can be optimized by analyzing the KL divergence: set up is a priori Bernoulli distribution with probability parameter p = 0.5, indicating that the connection between nodes is equally likely to be retained or predicted. As the model training deepens, the optimized target mask matrix is ​​finally obtained. S8, after passing the information bottleneck, The mask matrix is ​​obtained by performing the Hadamard product with the attention weight matrix obtained by formula (2) to obtain the attention weight matrix after the information bottleneck. This matrix retains the key nodes that play an important role in the prediction results and filters out the redundant non-critical nodes. At this point, the interpretability of the model output results is achieved through the information bottleneck theory. S9, after achieving interpretability through the variational information bottleneck method, the obtained attention weight matrix is ​​used to propagate information in the graph neural network and update the target features; S10, after the target feature is updated, it is sent to the Multilayer Perceptron for output; MLP consists of three layers, each layer contains multiple neurons or nodes, and each neuron is connected to all neurons in the previous layer; each layer of MLP includes two main components: linear transformation layer and activation function, and the input data is passed from the first layer of the network to the last layer layer by layer, and finally the output of the model is generated.

Images