A decision optimization-based internet topology reinforcement probing method

Abstract

The application discloses an internet topology reinforcement detection method based on decision optimization, and belongs to the field of internet topology detection. The method firstly constructs a topology detection knowledge graph to represent prior knowledge related to topology detection, then selects an optimal topology detection decision based on the knowledge graph by constructing heuristic rules and a strategy network, then trains and optimizes the strategy network by adaptive reinforcement learning to update the topology detection strategy by calculating the gain reward of the topology detection, and finally triggers a detection strategy for a multi-path by identifying and judging path stability. Through heuristic screening, reinforcement strategy decision and iterative update of detection feedback, the detection process can automatically explore high-value areas, reduce repeated detection, improve topology coverage and effectiveness, and guarantee the authenticity, integrity and dynamicity of internet topology detection in a large-scale internet environment with strong dynamics and strong heterogeneity.

Description

Technical Field

[0001] This invention belongs to the field of Internet topology detection, and in particular relates to an enhanced Internet topology detection method based on decision optimization. Background Technology

[0002] The most common method for Internet topology probing is traceroute. This method reveals the path structure hop-by-hop by sending a series of time-to-live (TTL) limited probe packets from a source probe point (VP) to a target IP address. The TTL of each probe packet is set sequentially to 1, 2, 3…N. The TTL is decremented by 1 as the packet passes through each intermediate router. When the TTL reaches 0, the intermediate router discards the probe packet and returns an Internet Control Message Protocol (ICMP) timeout response message. By parsing the source address and time information in these timeout messages, the IP address and round-trip time (RTT) of each router interface in the path can be obtained, thus constructing the IP interface-level Internet topology. Based on this, combined with alias resolution, Point of Presence (PoP) aggregation, and Autonomous System (AS) mapping techniques, the Internet topology at the router level, PoP level, and even AS level can be further inferred. Internet topology is an important foundation for understanding the structural characteristics of the Internet and is widely used in network operation management, performance measurement, fault location and security analysis.

[0003] However, the internet itself possesses inherent characteristics such as its massive scale, high autonomy, rapid updates, and lack of unified regulation. Furthermore, operators and related network institutions typically do not publicly disclose their internal structure and routing data due to commercial competition and security considerations. Therefore, even if some global probing projects provide publicly available measurement results, the authenticity, completeness, and dynamism of their data are difficult to guarantee. Simultaneously, the internet topology continuously changes dynamically due to factors such as routing policy adjustments, load balancing, link failures, and recovery. Traditional traceroute probing strategies struggle to adapt to this high dynamism, especially as the widespread multipath forwarding mechanism leads to unstable path observation results, making the topology reconstruction process more complex. Moreover, the internet's immense scale, diverse links, and frequent path forks make it extremely difficult to obtain a complete snapshot of the topology using only static traceroute probing strategies. Therefore, achieving highly accurate, dynamic coverage, and highly complete topology probing in the current internet environment remains a significant challenge.

[0004] Traditional traceroute, in network environments with per-flow load balancing, can generate spurious links due to different probe packets being assigned to different paths. To address this issue, Paris traceroute precisely controls header fields such as the 5-tuple in probe packets, ensuring that probe packets on the same path maintain the same flow identifier. This effectively avoids spurious links caused by load balancing and improves the authenticity of path probing. Building upon Paris traceroute, the Multipath Detection Algorithm (MDA) systematically explores all possible paths between the source and destination caused by load balancing by progressively increasing the number of probe packets over hops. However, it incurs significant overhead due to the large number of probe packets sent over certain hops. Furthermore, the MDA-Lite algorithm summarizes common multipath topology patterns in the Internet and simplifies the path completeness test of MDA based on these prior patterns. While maintaining high accuracy, it significantly reduces the number of probe packets and lowers network load. The latest Neural Network Gradient-Guided Method (NGM) constructs a training set using initial probes of a small number of random target IP addresses. It trains the neural network to learn the topological distributions corresponding to different target IP addresses, and then uses the gradient information of the neural network to generate more valuable candidate target IP addresses. By probing these candidate targets, the training set is continuously expanded, and the neural network model is iteratively updated. This method can discover more IP addresses and links under the same probe rate, improving topology collection efficiency.

[0005] While the aforementioned technologies have improved the accuracy and coverage of Internet topology probing to some extent, the following major problems remain: 1) They cannot simultaneously ensure the authenticity, dynamism, completeness, and cost control of the probes. Although these technologies can improve multipath coverage, they still require a large number of redundant probe packets in large-scale networks, resulting in high probe costs; 2) They are insufficient in handling dynamic changes in topology. Existing technologies mainly assume that the path structure is relatively stable within the probe period, making it difficult to effectively cope with dynamic path changes caused by routing adjustments, temporary congestion, load balancing, etc., in real-world networks; 3) Policy design relies on empirical rules and lacks intelligent decision-making capabilities. It cannot flexibly adjust the probe strategy based on real-time network feedback, thus making it difficult to discover the most topology structures with the fewest probe packets; 4) They lack decision optimization mechanisms for large-scale Internets. Current methods often cannot efficiently select from the vast target space (VP × IP prefix × target IP address × starting TTL), resulting in large measurement redundancy and low efficiency. Summary of the Invention

[0006] Due to the vast scale, highly heterogeneous structure, and dynamic nature of the Internet topology, traditional Internet topology measurement methods often rely on fixed strategies (such as random selection of destination IP addresses, periodic probing, and static VP-IP prefix allocation) for topology path probing. However, such static probing methods cannot adjust the probing direction in real time based on topology gains, nor can they efficiently find new IP nodes and links within a limited probing budget. This results in high probing redundancy, rapid gain decay, and insufficient coverage of dynamic areas, making it difficult to guarantee the authenticity, completeness, and dynamism of topology probing. To address this, this invention overcomes the limitations of static probing strategies and proposes a decision optimization-based Internet topology reinforcement probing method. This method first constructs a topology probing knowledge graph to represent prior knowledge related to topology probing. Then, based on the knowledge graph, it constructs a heuristic rule and policy network to select the optimal topology probing decision. Next, it uses adaptive reinforcement learning to train and optimize the policy network to update the topology probing policy by calculating the gain reward of topology probing, and finally, it triggers probing policies for multiple paths by identifying multi-paths and judging path stability. This invention enables the detection process to automatically explore high-value areas, reduce redundant detection, and improve topology coverage and effectiveness in a large-scale Internet environment characterized by strong dynamism and heterogeneity through heuristic screening, enhanced strategy decision-making, and iterative updates of detection feedback, thereby ensuring the authenticity, integrity, and dynamism of Internet topology detection.

[0007] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0008] A decision optimization-based method for reinforcing Internet topology detection includes the following steps:

[0009] Step 1: Topology Probe Knowledge Graph Construction: Historical probe data, IP prefix attributes, and path statistical features from the source probe point VP to the prefix are uniformly constructed into a knowledge graph; then, the IP prefix of the probe target is embedded and encoded using the knowledge graph.

[0010] Step 2: Topology Probe Decision Selection: Construct a policy network for the probe decision space consisting of VP×IP prefix×target IP address×starting time-to-live (TTL). Quickly filter candidate decisions using heuristic rules to compress the search space. Further select the optimal probe decision based on a centralized policy network that integrates knowledge features.

[0011] Step 3: Topology Detection Decision Update: Construct a reward function based on the new IP address, link, autonomous system (AS), city, multi-path branch, and boundary routing information obtained from the actual detection. At the same time, use detection redundancy as a penalty term to detect the target and update the knowledge graph. Calculate the detection reward, build a reinforcement learning state, and perform reinforcement learning update training.

[0012] Step 4: Topology multipath detection: In the enhanced detection process, a multipath identification and stability judgment mechanism is introduced to track IP nodes that have forked and have not yet converged.

[0013] Furthermore, the knowledge graph is constructed in the following ways:

[0014] Step 1.1: Construct the schema layer of the topology exploration knowledge graph; nodes include network entity nodes, geographic entity nodes, and node attributes; relationships include connection relationships between network entities, affiliation relationships between network entity nodes, and geographic location relationships;

[0015] Step 1.2: Based on the pattern layer of the topology detection knowledge graph, collect and construct the data layer of the Internet topology knowledge graph. The data includes Traceroute data, Border Gateway Protocol (BGP) routing data, network operator data, Internet Exchange Center (IXP) data, and geographic location data.

[0016] Step 1.3: Perform knowledge graph embedding encoding on the IP prefix of the probe target to generate a vectorized representation of the IP prefix.

[0017] Furthermore, the policy network adopts a multi-head neural network structure, including an IP prefix feature network, a detection action scoring network, and a detection value evaluation network. The three networks are modeled independently and are all multilayer perceptron structures.

[0018] The IP prefix feature network input layer receives IP prefix semantic features and outputs the relative detection value of the corresponding IP prefix in the current detection phase. The scores of all candidate actions together constitute the probability distribution of the actions. vector;

[0019] The input layer of the detection action scoring network receives a detection triplet feature vector consisting of VP, target IP address, starting TTL parameter, and IP prefix semantic features, and outputs the expected reward score of the detection action.

[0020] The input layer of the detection value assessment network receives the overall operating status composed of global detection statistics, and the output layer estimates the expected cumulative revenue under the current detection status.

[0021] The semantic features of the IP prefix include probe coverage, importance, recent probe increment, and knowledge graph embedding encoding of the IP prefix. The probe coverage represents the proportion of IP addresses that have been probed in the IP prefix, and the importance represents the significance of the IP prefix.

[0022] The overall operational status includes normalized detection coverage and path diversity.

[0023] Furthermore, the topology detection decision selection is specifically as follows:

[0024] Based on statistical measures and policy constraints in knowledge graphs, a rapid screening mechanism using three heuristic rules is introduced to compress the candidate set for IP prefix detection decisions. The final score for each detection decision is determined by this mechanism. for:

[0025]

[0026] in , and These are the detection coverage rates. ,importance and recent detection increments The weight, Before screening The IP prefix with the highest score is selected as the candidate set for detection decisions;

[0027] After quickly filtering the candidate set of IP prefix detection decisions, a complete detection decision is treated as a discrete detection action. In each round of detection, the policy network selects an action from the candidate detection decision set to execute. The IP prefix semantic features of the detection action are input into the IP prefix feature network of the policy network for scoring and ranking to obtain the top actions with the highest relative detection value scores. 1 target IP prefix; then randomly select from each target IP prefix. If there are not enough unprobeed IP addresses, select 20 IP addresses for the first probe. When probing, retrieve all remaining unprobeced IP addresses; then select the corresponding VP and starting TTL for each target IP address; during the first probe, select all VPs for each IP address and set the starting TTL to 1; when it is not the first probe, for paths probing different VPs with the target IP prefix, if there are nodes with the same prefix, group these VPs together, select the VP with the closest hop distance from each group, and if there are multiple closest VPs, randomly select one; then, based on the selected VP and IP prefix, proceed from the multipath state. If there are no multiple paths, the starting TTL is set to the minimum hop count from the VP to the IP prefix minus 1. If there is a multipath convergence interval in the multipath state, the largest TTL in the convergence interval is selected and incremented by 1. If there are multiple paths but they have not yet converged, the smallest starting TTL is selected.

[0028] Furthermore, the topology detection decision update is as follows:

[0029] Step 3.1: First, based on the prefix obtained through the IP prefix feature network... Each target IP prefix is ​​probed using paristraceroute. The knowledge graph is then updated based on the probe results, including updating probe coverage, border route IP prefixes, newly added IP addresses and links, newly probed cities, IP addresses repeatedly probed, newly added ASs, and multipath status. Path diversity ;

[0030] Step 3.2: Then, calculate the reward for this probe based on the updated knowledge graph. The reward function includes two parts: topology gain reward and redundancy penalty. The topology gain reward includes the number of newly added IP addresses, links, ASs, cities, multipaths, and border routing nodes in this probe, while the redundancy penalty includes the sum of the number of duplicate IP addresses and links detected.

[0031] Next, reinforcement learning is used to update the training by taking the overall running status, rewards, and IP prefix semantic features of the probe actions as input: First, the IP prefix semantic features of the probe actions in this round are fed into the probe action scoring network to obtain the probability distribution of the actions. ,use The function obtains the action probability vector Convert the probability vector into logarithmic form Then, the reward for this round of detection is evenly distributed to the detection actions, and the loss functions of the detection value evaluation network and the detection action scoring network are calculated respectively.

[0032] The detection value assessment network employs a reward-based value regression objective, and its parameters are updated by minimizing the error between predicted value and actual reward. The loss function of the detection action scoring network... The calculation is as follows:

[0033]

[0034] in This means that policy updates should not be propagated back to the parameters of the value assessment network;

[0035] Finally, determine whether all target IP prefix detections have been completed. If not, proceed to step 2 for topology detection decision selection; otherwise, proceed to the next step.

[0036] Furthermore, the topological multipath detection is specifically as follows:

[0037] First, construct a policy network for topology multipath probing using the same method as in step 2;

[0038] When performing topology multipath probing, the actual candidate probing actions are obtained in the following way: from the multipath state Identify all IP prefixes that still have multiple path branches and have not yet converged, along with their corresponding VPs and unstable converged path branch points. Score these candidate IP prefixes based on the number of non-converged hops, the number of VPs that can detect the branch, and their importance. After sorting by score, select the top... The most important IP prefixes are identified, and each IP prefix is ​​assigned a VP, probe IP address, and TTL. The semantic features of these IP prefixes for actual candidate probe actions are constructed to form a probe decision task.

[0039] Based on the detection decision task, the detection is assigned, the knowledge graph is updated in the same way as in step 2, the overall running status and reward are calculated, and then reinforcement learning is performed to update and train the system using the overall running status, reward and the semantic features of the IP prefix of the action as input.

[0040] When the number of times the newly added reward is lower than the set threshold is lower than the detection threshold, the detection stops and the result is output; otherwise, the topology multipath detection continues.

[0041] This invention proposes an enhanced Internet topology detection method based on decision optimization. First, a topology detection knowledge graph is constructed based on historical detection data, IP prefix attributes, and detection statistical characteristics. Then, in each round of detection, a detection decision space consisting of (VP × IP prefix × target IP address × initial TTL) is built, and candidate detection actions are evaluated and selected based on a policy network. Next, a reward is constructed based on the newly discovered topology gain after the detection is performed, and the policy network is iteratively updated. Finally, a multi-path identification and stability judgment mechanism is introduced to further detect nodes with path forks that have not yet converged, thereby achieving continuous and intelligent enhanced detection of the Internet topology.

[0042] The method of this invention effectively overcomes the limitations of existing technologies, such as low detection efficiency, high resource consumption, and difficulty in adapting to dynamic topology changes. Compared with traditional methods, this invention introduces a decision-making mechanism combining knowledge graphs and reinforcement learning, enabling the detection strategy to fully utilize historical topology information and statistical features, and autonomously learn the topology gain patterns brought about by different detection actions. This allows for the discovery of more effective IP addresses, links, and multi-path structures under the same detection budget. Furthermore, by identifying and judging the stability of multi-path fork nodes, this invention avoids invalid repeated detection of converged paths, while improving coverage of dynamic topology characteristics such as load balancing and routing adjustments, ensuring the authenticity, completeness, and dynamism of Internet topology detection. Attached Figure Description

[0043] Figure 1 A schematic diagram of the decision optimization method for enhancing Internet topology detection.

[0044] Figure 2This is a schematic diagram of the schema layer of the Internet topology knowledge graph. Detailed Implementation

[0045] To better understand the purpose, structure, and function of this invention, the following detailed description of an Internet topology enhancement detection method based on decision optimization is provided in conjunction with the accompanying drawings.

[0046] The proposed Internet topology enhancement detection method based on decision optimization is as follows: Figure 1 As shown, it is mainly divided into four parts:

[0047] The topology detection knowledge graph is constructed by unifying historical detection data, IP prefix attributes, and path statistical features from VP to prefix into a knowledge graph;

[0048] For topology probing decision selection, a probing decision space consisting of (VP×IP prefix×target IP address×starting TTL) is constructed. Candidate decisions are quickly screened using heuristic rules to compress the search space. The optimal probing decision is further selected based on a centralized policy network that integrates knowledge features.

[0049] The topology detection decision update constructs a reward function based on the new IP addresses, links, ASs, cities, multi-path branches, and boundary routing information obtained from actual detection, while using detection redundancy as a penalty term, so that the policy can automatically learn the decision rules that maximize topology gain.

[0050] Topology multipath detection introduces multipath identification and stability judgment mechanisms during the enhanced detection process, tracks IP nodes that have forked and have not yet converged, and triggers targeted repeated detection strategies when necessary.

[0051] The specific implementation method is as follows:

[0052] Step 1: Topology Detection Knowledge Graph Construction: The construction of the topology detection knowledge graph consists of three parts: schema layer construction, data layer construction, and IP prefix knowledge graph embedding and encoding.

[0053] Step 1.1: First construct as follows Figure 2 The pattern layer of the topology detection knowledge graph shown.

[0054] The nodes in the diagram include network entity nodes, geographical entity nodes, and node attributes, specifically: IP interface (IP address), VP (IP address), AS (AS number, AS level), network operator (name), IXP (name), IP prefix (prefix and length), subprefix (prefix and length), city (name), province (name), and country (name), where IXP represents Internet Exchange Point.

[0055] The relationships in the diagram include connectivity between network entities, affiliation between network entity nodes, and geographical location relationships. Specific relationships include: (IP interface, traceroute, IP interface), (IP interface, belongs to prefix, subprefix), (subprefix, is subprefix, IP prefix), (VP, probe prefix, subprefix), (VP, belongs to prefix, IP prefix), (VP, located in city, city), (VP, belongs to, AS), (IP prefix, belongs to operator, network operator), (IP prefix, belongs to AS, AS), (IP prefix, located in city, city), (city, located in province, province), (province, located in country, country), (AS, belongs to operator, network operator), (network operator, operates IXP, IXP), (AS, exchanges traffic, IXP), (AS, advertises route, AS), (AS, contains prefix, IP prefix), (IXP, contains prefix, IP prefix). The attribute of these relationships is: advertised route (AS business relationship type).

[0056] Step 1.2: Then, based on the pattern layer of the topology detection knowledge graph, collect and construct the data layer of the Internet topology knowledge graph, whose data includes:

[0057] 1) Traceroute data: This data is obtained from traceoute measurements. It includes information about IP interfaces and their traceoute connections, IP interface attributes, and VP probe prefixes.

[0058] 2) Border Gateway Protocol (BGP) Routing Data: BGP routing data comes from BGP route path information collected by Route-Views and RIPE RIS route collectors. From this data, IP prefixes and their associated ASs and AS topology can be extracted. The target IP prefixes are divided into / 24-length sub-prefixes. Based on the obtained AS topology, the AS business relationship type is inferred using the AS-Rank method. Specifically, the Bron-Kerbosch algorithm is first used to find all clusters, and the largest cluster with the highest transmission rate (i.e., the number of unique neighbors appearing on both sides of the AS link) and a size not exceeding 10 ASs is selected. Subsequently, ASs that meet the following conditions are further selected to form a set. These AS at most are An AS is not connected and will not be simultaneously connected to another AS on the same path. Two or more ASs co-occur. Finally, based again on the sum of transmission degrees, [the following is considered]... The Bron-Kerbosch algorithm is applied to obtain the final largest clique AS. Furthermore, the AS is processed hierarchically, and the largest clique AS obtained by the AS-Rank method is used as... Hierarchical, the nodes with a degree greater than 100 in the customer AS of the largest AS are considered as... The hierarchy will treat ASs without customers as... Hierarchy, and the remaining AS as Hierarchy. This data includes IP prefixes, subprefixes, ASs, advertised routing relationships between ASs, AS business relationship types, and AS hierarchy information.

[0059] 3) Network Operator Data: This data comes from the Center for Applied Internet Data Analysis (CAIDA), which aggregates routing registration information from five routing registries: ARIN for North America, LACNIC for South America, RIPE NCC for Europe, AFRINIC for Africa, and APNIC for Asia / Pacific, including Australia. This data includes the ASs operated by network operators.

[0060] 4) IXP Data: This data comes from CAIDA's aggregated PeeringDB database, Hurricane Electric, Packet Clearing House, Wikipedia, BGP Looking Glass, and GeoNames database. This data includes the IXP's network operator, city, IP prefixes, and member AS information.

[0061] 5) Geographic Location Data: This data is publicly available and downloadable, sourced from internet geographic location databases. It includes the IP prefix and the city, province, and country where the VPN is located.

[0062] Step 1.3: Based on the construction of the pattern layer and data layer, knowledge graph embedding encoding is performed on the target IP prefixes to generate a vectorized representation of the IP prefixes that can be used for subsequent reasoning and policy decision-making. This module adopts the RESCAL knowledge graph embedding algorithm based on tensor decomposition and has been adapted for the Internet topology detection scenario. First, the knowledge graph is represented as triples, and high-order interaction information between entities related to IP prefixes in the knowledge graph is learned by sharing entity matrices and relation matrices. Through the trained entity matrix, a low-dimensional dense vector encoding of each IP prefix can be obtained, which is used to represent its network structure location, operational attributes, and neighbor context information. This invention uses this encoding as one of the input features of the reinforcement learning policy network, thereby realizing intelligent detection decision-making based on structural semantics.

[0063] Step 2: Topology Probe Decision Selection: In the topology probe decision selection stage, a policy network is constructed with a probe decision space consisting of (VP×IP prefix×target IP address×starting TTL). Candidate decisions are quickly screened through heuristic rules to compress the search space. The optimal probe decision is further selected based on a centralized policy network that integrates knowledge features.

[0064] Step 2.1: First, construct the strategy network for topology reinforcement detection. This network adopts a multi-head neural network structure, and models the IP prefix importance assessment, detection action scoring and detection value assessment independently by different network substructures.

[0065] Specifically, the IP prefix feature network adopts a multilayer perceptron structure, with the input layer receiving the feature-encoded semantic features of the IP prefix. The hidden layer extracts high-level features through linear transformation and ReLU activation function, and introduces layer normalization operation to enhance training stability and feature distribution consistency. The output layer generates a scalar score value to represent the relative detection value of the corresponding IP prefix at the current detection stage. The scores of all candidate actions together constitute the probability distribution of the action. Vectors are used to provide a basis for subsequent detection decisions.

[0066] The detection action scoring network also adopts a multilayer perceptron structure. The input layer receives the VP, target IP address and starting TTL parameters, and IP prefix semantic features. The constructed probe triplet feature vector describes the contextual information of a specific probe action. The hidden layer extracts the potential reward features related to the action through linear mapping and ReLU activation function, and improves the model's generalization ability in diverse probe scenarios through layer normalization. The output layer generates a scalar value to represent the expected reward score of the probe action, which is then used to construct the action selection probability distribution in the policy decision stage.

[0067] Detection Value Assessment Network It also adopts a multilayer perceptron structure, with the input layer receiving the overall operating status composed of global detection statistics. The hidden layer abstracts the state features through linear mapping and ReLU nonlinear activation function, and the output layer generates a scalar value to estimate the expected cumulative reward under the current probe state, thereby providing a baseline reference for policy updates.

[0068] By leveraging the collaborative efforts of multiple heads, the policy network's ability to express complex network topology dynamics can be significantly enhanced, interference between different types of features can be reduced, and the convergence efficiency and stability of reinforcement learning policies can be improved.

[0069] The semantic features of the IP prefix It mainly consists of four concatenated features: detection coverage ,importance Detection increment Knowledge graph embedding encoding of IP prefixes.

[0070] Detection coverage This indicates the proportion of IP addresses that have been probed within this IP prefix. For IP prefixes that have already been probed, [the percentage will be displayed here]. Set it to infinity.

[0071] importance This indicates the importance of the IP prefix, which is determined by five indicators: the AS level of the AS to which the IP prefix belongs, the node degree in the AS topology, whether it is in an IXP, the number of cities it is located in, and whether it is a border route.

[0072]

[0073] in , , , and These are the weights of five indicators: the AS level to which the IP prefix belongs, the node degree of the AS in the AS topology, whether it is in an IXP, the number of cities it is located in, and whether it is a border route. Its value depends on the specific detection situation.

[0074] , , , and These are the scores corresponding to these five indicators. The scores are based on the AS level to which the IP prefix belongs. , , and Four levels, and their scores Set them to 1, 0.66, 0.33, and 0 respectively. This sets the node degree of the AS to which the IP prefix belongs in the AS topology. Normalization is performed to obtain its score. Nodes with a degree greater than 50 are all set to 1. When the IP prefix is ​​IXP, its score is... Set to 1 otherwise set to 0. This sets the number of cities containing the IP prefix. Normalization is performed to obtain its score. For those with a quantity greater than 10, set them to 1. When the IP prefix is ​​a border route, its score is... Set to 1, otherwise set to 0.

[0075] By the number of newly added IP addresses and number of links The sum is nonlinearly compressed to obtain the recent detection increment. :

[0076]

[0077] in Get the number of all newly added IP addresses. and number of links The 90th percentile of the sum.

[0078] Describe the overall operational status of the current Internet topology probe. Represented as:

[0079]

[0080] in This indicates that the average value has been taken. This state represents the result of calculating the detection coverage. and path diversity These two key statistical indicators are aggregated and normalized to form a low-dimensional continuous state vector.

[0081] For path diversity For each VP-IP prefix pair, a finite-length historical path signature sequence is maintained, and this history is updated after a new probe path is obtained. By statistically analyzing the proportion of different path signatures within the historical window, a path diversity index is calculated, with a value ranging from 0 to 1.

[0082] Step 2.2: Next, based on the statistics and policy constraints in the knowledge graph, a rapid screening mechanism is introduced using three heuristic rules (including priority for low detection coverage, priority for high importance, and priority for recent detection increments) to compress the candidate set of IP prefix detection decisions. Finally, the score of each detection decision is calculated. for:

[0083]

[0084] in , and These are the weights of detection coverage, importance, and recent detection increments, respectively. Its value depends on the specific detection conditions. Before screening The IP prefix with the highest score is selected as the candidate set for detection decisions.

[0085] Step 2.3: After quickly filtering the candidate set of IP prefix detection decisions, a complete detection decision is treated as a discrete detection action. In each round of detection, the policy network selects an action from the candidate detection decision set to execute. The IP prefix semantic features of the detection action are input into the IP prefix feature network of the policy network for scoring and ranking to obtain the top actions with the highest relative detection value scores. 1 target IP prefix. Next, randomly select from each target IP prefix. If there are not enough unprobeed IP addresses, select 20 IP addresses for the first probe. When probing, retrieve all remaining unprobeced IP addresses. Then, select a corresponding VPN and starting TTL for each target IP address. During the first probe, select all VPNs for each IP address and set the starting TTL to 1. If it's not the first probe, for paths probing different VPNs with the target IP prefix, if there are nodes with the same prefix, group these VPNs together. Select the VPN with the shortest hop distance from each group; if multiple shortest VPNs exist, select one randomly. Then, based on the selected VPN and IP prefix, proceed from the multipath state. Select the starting TTL. If there are never too many paths, the starting TTL is set to the minimum hop count from the VP to the IP prefix minus 1; if there are... If there is a multi-path convergence interval, select the largest TTL in the convergence interval and add 1; if there are multiple paths but no convergence, select the smallest starting TTL.

[0086] Step 3: Topology Detection Decision Update: In the topology detection decision update stage, the target is detected and the knowledge graph is updated, the detection reward is calculated, the reinforcement learning state is constructed, and reinforcement learning update training is carried out.

[0087] Step 3.1: First, based on the prefix obtained through the IP prefix feature network... Each target IP prefix is ​​probed using paristraceroute. The knowledge graph is then updated based on the probe results, including updating probe coverage, border route IP prefixes, newly added IP addresses and links, newly probed cities, IP addresses repeatedly probed, newly added ASs, and multipath status. Path diversity .

[0088] For multipath states If the IP prefix of a hop IP address in the path from VP to IP prefix changes, it is recorded as a new multipath hop. Then, it is checked whether the multipath hops in the path are stably converged. The condition for stable convergence is that all multipath fork points have been detected more than once and no new fork points have appeared.

[0089] Step 3.2: Then calculate the reward for this probe based on the updated knowledge graph. :

[0090]

[0091] The reward function consists of two parts: topology gain reward and redundancy penalty. The topology gain reward includes the number of newly added IP addresses during this probe. Number of links AS quantity Number of cities Number of multipaths and the number of border routing nodes The redundancy penalty, on the other hand, includes the sum of the number of duplicate IP addresses and links detected. The corresponding weighting coefficients , , , , , and The value depends on the specific detection situation.

[0092] Next, let's look at the overall operating status. ,award and the semantic features of the IP prefix of the probe action Using this as input, reinforcement learning is performed to update the training. First, the IP prefix semantic features of this round of probing actions are used. The probability distribution of actions is obtained by feeding the data into the action scoring network. ,use The function obtains the action probability vector Convert the probability vector into logarithmic form To adapt to the policy gradient loss function, a minimal constant 1e-8 is added to avoid numerical instability caused by log(0), making policy updates more stable and reliable. Then, the reward for this round of exploration is... The detection actions are evenly distributed, and the loss functions of the detection value assessment network and the detection action scoring network are calculated separately.

[0093] The detection value assessment network adopts a return-based value regression objective, and updates the network parameters by minimizing the error between the predicted value and the actual return.

[0094]

[0095] in , This indicates squaring each tensor element. The output of the value assessment network is shown here. As a benchmark term, it forms an advantage function with the actual rewards in this round. This reduces the variance of gradient estimation while maintaining the correct optimization direction, thereby improving learning efficiency and accelerating convergence. The loss function of the action scoring network is... :

[0096]

[0097] in This indicates that policy updates should not be propagated back to the parameters of the value evaluation network, used to block... Gradient propagation is performed on the probe action scoring network. The loss functions of the value evaluation network and the probe action scoring network are summed, and then the standard PyTorch optimization process is used: backpropagation and... The function updates the parameters, clears the previously accumulated gradients, calculates the gradients of all parameters based on the loss, and uses the gradients to update the parameters of the value evaluation network and the probe action scoring network.

[0098] Finally, determine whether all target IP prefix detections have been completed. If not, proceed to step 2 for topology detection decision selection; otherwise, proceed to the next step.

[0099] Step 4: Topology Multipath Probe: A dedicated identification and stability judgment mechanism is proposed for the multipath nature of Internet paths (especially the forks caused by load balancing, policy routing and temporary route convergence process) to improve multipath coverage and avoid resource waste caused by infinite repeated probes.

[0100] First, construct a policy network for topology multipath probing using the same method as in step 2.

[0101] When performing topology multipath probing, the actual candidate probing actions are obtained in the following way:

[0102] From multipath status Identify all IP prefixes that still have multiple path branches and have not yet converged, along with their corresponding VPs and unstable path branching points, and score these candidate IP prefixes:

[0103]

[0104] in This represents the number of non-converged jumps. This represents the number of VPs that can detect the fork. , and They represent , and The weight, Its value depends on the specific detection situation. After ranking according to the scores, the top score is selected. The most important IP prefixes are identified, and each IP prefix is ​​assigned the most suitable VP (the one with the most non-converged hops and the smallest average hop count), probe IP address (the branch point IP address with the most previous probes), and TTL (the hop count of the earliest non-converged branch point). This concentrates probe resources on tracking and converging multi-path branch points. The semantic features of these actual candidate probe actions for IP prefixes are constructed to form the probe decision task.

[0105] Based on the detection decision task, detection is assigned, and the knowledge graph is updated using the same method as in step 2. The state is then calculated. and rewards Then in state ,award Semantic features of IP prefixes for actions Using this as input, reinforcement learning is performed to update the training.

[0106] When the new reward is lower than the set threshold The number of times was lower than the detection threshold If the condition is met, stop probing and output the results; otherwise, continue topology multipath probing. These two thresholds depend on the specific probing task.

[0107] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for reinforcing Internet topology detection based on decision optimization, characterized in that, Includes the following steps: Step 1: Topology Probe Knowledge Graph Construction: Historical probe data, IP prefix attributes, and path statistical features from the source probe point VP to the prefix are uniformly constructed into a knowledge graph; then, the IP prefix of the probe target is embedded and encoded using the knowledge graph. Step 2: Topology Probe Decision Selection: Construct a policy network for the probe decision space consisting of VP×IP prefix×target IP address×starting time-to-live (TTL). Quickly filter candidate decisions using heuristic rules to compress the search space. Further select the optimal probe decision based on a centralized policy network that integrates knowledge features. Step 3: Topology Detection Decision Update: Construct a reward function based on the new IP address, link, autonomous system (AS), city, multi-path branch, and boundary routing information obtained from the actual detection. At the same time, use detection redundancy as a penalty term to detect the target and update the knowledge graph. Calculate the detection reward, build a reinforcement learning state, and perform reinforcement learning update training. Step 4: Topology multipath detection: In the enhanced detection process, a multipath identification and stability judgment mechanism is introduced to track IP nodes that have forked and have not yet converged.

2. The Internet topology reinforcement detection method based on decision optimization according to claim 1, characterized in that, The knowledge graph is constructed in the following ways: Step 1.1: Construct the schema layer of the topology exploration knowledge graph; nodes include network entity nodes, geographic entity nodes, and node attributes; relationships include connection relationships between network entities, affiliation relationships between network entity nodes, and geographic location relationships; Step 1.2: Based on the pattern layer of the topology detection knowledge graph, collect and construct the data layer of the Internet topology knowledge graph. The data includes Traceroute data, Border Gateway Protocol (BGP) routing data, network operator data, Internet Exchange Center (IXP) data, and geographic location data. Step 1.3: Perform knowledge graph embedding encoding on the IP prefix of the probe target to generate a vectorized representation of the IP prefix.

3. The Internet topology reinforcement detection method based on decision optimization according to claim 2, characterized in that, The policy network adopts a multi-head neural network structure, including an IP prefix feature network, a detection action scoring network, and a detection value assessment network. The three networks are modeled independently and are all multilayer perceptron structures. The IP prefix feature network input layer receives IP prefix semantic features and outputs the relative detection value of the corresponding IP prefix in the current detection phase. The scores of all candidate actions together constitute the probability distribution of the actions. vector; The input layer of the detection action scoring network receives a detection triplet feature vector consisting of VP, target IP address, starting TTL parameter, and IP prefix semantic features, and outputs the expected reward score of the detection action. The input layer of the detection value assessment network receives the overall operating status composed of global detection statistics, and the output layer estimates the expected cumulative revenue under the current detection status. The semantic features of the IP prefix include probe coverage, importance, recent probe increment, and knowledge graph embedding encoding of the IP prefix. The probe coverage represents the proportion of IP addresses that have been probed in the IP prefix, and the importance represents the significance of the IP prefix. The overall operational status includes normalized detection coverage and path diversity.

4. The Internet topology reinforcement detection method based on decision optimization according to claim 3, characterized in that, The specific topology detection decision selection is as follows: Based on statistical measures and policy constraints in knowledge graphs, a rapid screening mechanism using three heuristic rules is introduced to compress the candidate set for IP prefix detection decisions. The final score for each detection decision is determined by this mechanism. for: ； in , and These are the detection coverage rates. ,importance and recent detection increments The weight, Before screening The IP prefix with the highest score is selected as the candidate set for detection decisions; After quickly filtering the candidate set of IP prefix detection decisions, a complete detection decision is treated as a discrete detection action. In each round of detection, the policy network selects an action from the candidate detection decision set to execute. The IP prefix semantic features of the detection action are input into the IP prefix feature network of the policy network for scoring and ranking to obtain the top actions with the highest relative detection value scores. One target IP prefix; Next, random selection will be made from each target IP prefix. If there are not enough unprobeed IP addresses, select 20 IP addresses for the first probe. When probing, retrieve all remaining unprobeced IP addresses; then select the corresponding VP and starting TTL for each target IP address; during the first probe, select all VPs for each IP address and set the starting TTL to 1; when it is not the first probe, for paths probing different VPs with the target IP prefix, if there are nodes with the same prefix, group these VPs together, select the VP with the closest hop distance from each group, and if there are multiple closest VPs, randomly select one; then, based on the selected VP and IP prefix, proceed from the multipath state. If there are no multiple paths, the starting TTL is set to the minimum hop count from the VP to the IP prefix minus 1. If there is a multipath convergence interval in the multipath state, the largest TTL in the convergence interval is selected and incremented by 1. If there are multiple paths but they have not yet converged, the smallest starting TTL is selected.

5. The Internet topology reinforcement detection method based on decision optimization according to claim 4, characterized in that, The topology detection decision update is as follows: Step 3.1: First, based on the prefix obtained through the IP prefix feature network... Each target IP prefix is ​​probed using paristraceroute. The knowledge graph is then updated based on the probe results, including updating probe coverage, border route IP prefixes, newly added IP addresses and links, newly probed cities, IP addresses repeatedly probed, newly added ASs, and multipath status. Path diversity ; Step 3.2: Then, calculate the reward for this probe based on the updated knowledge graph. The reward function includes two parts: topology gain reward and redundancy penalty. The topology gain reward includes the number of newly added IP addresses, links, ASs, cities, multipaths, and border routing nodes in this probe, while the redundancy penalty includes the sum of the number of duplicate IP addresses and links detected. Next, reinforcement learning is used to update the training by taking the overall running status, rewards, and IP prefix semantic features of the probe actions as input: First, the IP prefix semantic features of the probe actions in this round are fed into the probe action scoring network to obtain the probability distribution of the actions. ,use The function obtains the action probability vector Convert the probability vector into logarithmic form Then, the reward for this round of detection is evenly distributed to the detection actions, and the loss functions of the detection value evaluation network and the detection action scoring network are calculated respectively. The detection value assessment network employs a reward-based value regression objective, and its parameters are updated by minimizing the error between predicted value and actual reward. The loss function of the detection action scoring network... The calculation is as follows: ； in This means that policy updates should not be propagated back to the parameters of the value assessment network; Finally, determine whether all target IP prefix detections have been completed. If not, proceed to step 2 for topology detection decision selection; otherwise, proceed to the next step.

6. The Internet topology reinforcement detection method based on decision optimization according to claim 5, characterized in that, The topology multipath detection is described in detail below: First, construct a policy network for topology multipath probing using the same method as in step 2; When performing topology multipath probing, the actual candidate probing actions are obtained in the following way: from the multipath state Identify all IP prefixes that still have multiple path branches and have not yet converged, along with their corresponding VPs and unstable converged path branch points. Score these candidate IP prefixes based on the number of non-converged hops, the number of VPs that can detect the branch, and their importance. After sorting by score, select the top... The most important IP prefixes are identified, and each IP prefix is ​​assigned a VP, probe IP address, and TTL. The semantic features of these IP prefixes for actual candidate probe actions are constructed to form a probe decision task. Based on the detection decision task, the detection is assigned, the knowledge graph is updated in the same way as in step 2, the overall running status and reward are calculated, and then reinforcement learning is performed to update and train the system using the overall running status, reward and the semantic features of the IP prefix of the action as input. When the number of times the newly added reward is lower than the set threshold is lower than the detection threshold, the detection stops and the result is output; otherwise, the topology multipath detection continues.

7. The Internet topology reinforcement detection method based on decision optimization according to claim 6, characterized in that, The importance calculation formula is as follows: ； in, Indicates importance, , , , and These are the weights of five indicators: the AS level to which the IP prefix belongs, the node degree of the AS in the AS topology, whether it is in an IXP, the number of cities it is located in, and whether it is a border route. ; , , , and The scores for the above five indicators are determined based on the AS level to which the IP prefix belongs. , , and Four levels, and their scores Set them to 1, 0.66, 0.33, and 0 respectively; this sets the node degree of the AS to which the IP prefix belongs in the AS topology. Normalization is performed to obtain its score. Nodes with a degree greater than 50 are all set to 1; when the IP prefix is ​​IXP, its score is set accordingly. Set to 1 otherwise set to 0; This will determine the number of cities containing the IP prefix. Normalization is performed to obtain its score. All values ​​greater than 10 are set to 1; when the IP prefix is ​​a border route, its score is set to 1. Set to 1, otherwise set to 0.

8. The Internet topology reinforcement detection method based on decision optimization according to claim 7, characterized in that, The recent detection increment is based on the number of newly added IP addresses. and number of links The sum is obtained by nonlinear compression: ； in, This indicates the recent increase in detection volume. Get the number of all newly added IP addresses. and number of links The 90th percentile of the sum.

9. The Internet topology reinforcement detection method based on decision optimization according to claim 8, characterized in that, The overall operating status is represented as follows: ； in This indicates taking the average value; for path diversity For each VP-IP prefix pair, a finite-length historical path signature sequence is maintained, and this history is updated after a new probe path is obtained. The path diversity index is calculated by statistically analyzing the proportion of different path signatures within the historical window.

Images